diff --git a/.ai/instructions.md b/.ai/instructions.md new file mode 100644 index 0000000000..5f314a0dc9 --- /dev/null +++ b/.ai/instructions.md @@ -0,0 +1,370 @@ +# ESPHome AI Collaboration Guide + +This document provides essential context for AI models interacting with this project. Adhering to these guidelines will ensure consistency and maintain code quality. + +## 1. Project Overview & Purpose + +* **Primary Goal:** ESPHome is a system to configure microcontrollers (like ESP32, ESP8266, RP2040, and LibreTiny-based chips) using simple yet powerful YAML configuration files. It generates C++ firmware that can be compiled and flashed to these devices, allowing users to control them remotely through home automation systems. +* **Business Domain:** Internet of Things (IoT), Home Automation. + +## 2. Core Technologies & Stack + +* **Languages:** Python (>=3.11), C++ (gnu++20) +* **Frameworks & Runtimes:** PlatformIO, Arduino, ESP-IDF. +* **Build Systems:** PlatformIO is the primary build system. CMake is used as an alternative. +* **Configuration:** YAML. +* **Key Libraries/Dependencies:** + * **Python:** `voluptuous` (for configuration validation), `PyYAML` (for parsing configuration files), `paho-mqtt` (for MQTT communication), `tornado` (for the web server), `aioesphomeapi` (for the native API). + * **C++:** `ArduinoJson` (for JSON serialization/deserialization), `AsyncMqttClient-esphome` (for MQTT), `ESPAsyncWebServer` (for the web server). +* **Package Manager(s):** `pip` (for Python dependencies), `platformio` (for C++/PlatformIO dependencies). +* **Communication Protocols:** Protobuf (for native API), MQTT, HTTP. + +## 3. Architectural Patterns + +* **Overall Architecture:** The project follows a code-generation architecture. The Python code parses user-defined YAML configuration files and generates C++ source code. This C++ code is then compiled and flashed to the target microcontroller using PlatformIO. + +* **Directory Structure Philosophy:** + * `/esphome`: Contains the core Python source code for the ESPHome application. + * `/esphome/components`: Contains the individual components that can be used in ESPHome configurations. Each component is a self-contained unit with its own C++ and Python code. + * `/tests`: Contains all unit and integration tests for the Python code. + * `/docker`: Contains Docker-related files for building and running ESPHome in a container. + * `/script`: Contains helper scripts for development and maintenance. + +* **Core Architectural Components:** + 1. **Configuration System** (`esphome/config*.py`): Handles YAML parsing and validation using Voluptuous, schema definitions, and multi-platform configurations. + 2. **Code Generation** (`esphome/codegen.py`, `esphome/cpp_generator.py`): Manages Python to C++ code generation, template processing, and build flag management. + 3. **Component System** (`esphome/components/`): Contains modular hardware and software components with platform-specific implementations and dependency management. + 4. **Core Framework** (`esphome/core/`): Manages the application lifecycle, hardware abstraction, and component registration. + 5. **Dashboard** (`esphome/dashboard/`): A web-based interface for device configuration, management, and OTA updates. + +* **Platform Support:** + 1. **ESP32** (`components/esp32/`): Espressif ESP32 family. Supports multiple variants (Original, C2, C3, C5, C6, H2, P4, S2, S3) with ESP-IDF framework. Arduino framework supports only a subset of the variants (Original, C3, S2, S3). + 2. **ESP8266** (`components/esp8266/`): Espressif ESP8266. Arduino framework only, with memory constraints. + 3. **RP2040** (`components/rp2040/`): Raspberry Pi Pico/RP2040. Arduino framework with PIO (Programmable I/O) support. + 4. **LibreTiny** (`components/libretiny/`): Realtek and Beken chips. Supports multiple chip families and auto-generated components. + +## 4. Coding Conventions & Style Guide + +* **Formatting:** + * **Python:** Uses `ruff` and `flake8` for linting and formatting. Configuration is in `pyproject.toml`. + * **C++:** Uses `clang-format` for formatting. Configuration is in `.clang-format`. + +* **Naming Conventions:** + * **Python:** Follows PEP 8. Use clear, descriptive names following snake_case. + * **C++:** Follows the Google C++ Style Guide. + +* **Component Structure:** + * **Standard Files:** + ``` + components/[component_name]/ + ├── __init__.py # Component configuration schema and code generation + ├── [component].h # C++ header file (if needed) + ├── [component].cpp # C++ implementation (if needed) + └── [platform]/ # Platform-specific implementations + ├── __init__.py # Platform-specific configuration + ├── [platform].h # Platform C++ header + └── [platform].cpp # Platform C++ implementation + ``` + + * **Component Metadata:** + - `DEPENDENCIES`: List of required components + - `AUTO_LOAD`: Components to automatically load + - `CONFLICTS_WITH`: Incompatible components + - `CODEOWNERS`: GitHub usernames responsible for maintenance + - `MULTI_CONF`: Whether multiple instances are allowed + +* **Code Generation & Common Patterns:** + * **Configuration Schema Pattern:** + ```python + import esphome.codegen as cg + import esphome.config_validation as cv + from esphome.const import CONF_KEY, CONF_ID + + CONF_PARAM = "param" # A constant that does not yet exist in esphome/const.py + + my_component_ns = cg.esphome_ns.namespace("my_component") + MyComponent = my_component_ns.class_("MyComponent", cg.Component) + + CONFIG_SCHEMA = cv.Schema({ + cv.GenerateID(): cv.declare_id(MyComponent), + cv.Required(CONF_KEY): cv.string, + cv.Optional(CONF_PARAM, default=42): cv.int_, + }).extend(cv.COMPONENT_SCHEMA) + + async def to_code(config): + var = cg.new_Pvariable(config[CONF_ID]) + await cg.register_component(var, config) + cg.add(var.set_key(config[CONF_KEY])) + cg.add(var.set_param(config[CONF_PARAM])) + ``` + + * **C++ Class Pattern:** + ```cpp + namespace esphome { + namespace my_component { + + class MyComponent : public Component { + public: + void setup() override; + void loop() override; + void dump_config() override; + + void set_key(const std::string &key) { this->key_ = key; } + void set_param(int param) { this->param_ = param; } + + protected: + std::string key_; + int param_{0}; + }; + + } // namespace my_component + } // namespace esphome + ``` + + * **Common Component Examples:** + - **Sensor:** + ```python + from esphome.components import sensor + CONFIG_SCHEMA = sensor.sensor_schema(MySensor).extend(cv.polling_component_schema("60s")) + async def to_code(config): + var = await sensor.new_sensor(config) + await cg.register_component(var, config) + ``` + + - **Binary Sensor:** + ```python + from esphome.components import binary_sensor + CONFIG_SCHEMA = binary_sensor.binary_sensor_schema().extend({ ... }) + async def to_code(config): + var = await binary_sensor.new_binary_sensor(config) + ``` + + - **Switch:** + ```python + from esphome.components import switch + CONFIG_SCHEMA = switch.switch_schema().extend({ ... }) + async def to_code(config): + var = await switch.new_switch(config) + ``` + +* **Configuration Validation:** + * **Common Validators:** `cv.int_`, `cv.float_`, `cv.string`, `cv.boolean`, `cv.int_range(min=0, max=100)`, `cv.positive_int`, `cv.percentage`. + * **Complex Validation:** `cv.All(cv.string, cv.Length(min=1, max=50))`, `cv.Any(cv.int_, cv.string)`. + * **Platform-Specific:** `cv.only_on(["esp32", "esp8266"])`, `esp32.only_on_variant(...)`, `cv.only_on_esp32`, `cv.only_on_esp8266`, `cv.only_on_rp2040`. + * **Framework-Specific:** `cv.only_with_framework(...)`, `cv.only_with_arduino`, `cv.only_with_esp_idf`. + * **Schema Extensions:** + ```python + CONFIG_SCHEMA = cv.Schema({ ... }) + .extend(cv.COMPONENT_SCHEMA) + .extend(uart.UART_DEVICE_SCHEMA) + .extend(i2c.i2c_device_schema(0x48)) + .extend(spi.spi_device_schema(cs_pin_required=True)) + ``` + +## 5. Key Files & Entrypoints + +* **Main Entrypoint(s):** `esphome/__main__.py` is the main entrypoint for the ESPHome command-line interface. +* **Configuration:** + * `pyproject.toml`: Defines the Python project metadata and dependencies. + * `platformio.ini`: Configures the PlatformIO build environments for different microcontrollers. + * `.pre-commit-config.yaml`: Configures the pre-commit hooks for linting and formatting. +* **CI/CD Pipeline:** Defined in `.github/workflows`. +* **Static Analysis & Development:** + * `esphome/core/defines.h`: A comprehensive header file containing all `#define` directives that can be added by components using `cg.add_define()` in Python. This file is used exclusively for development, static analysis tools, and CI testing - it is not used during runtime compilation. When developing components that add new defines, they must be added to this file to ensure proper IDE support and static analysis coverage. The file includes feature flags, build configurations, and platform-specific defines that help static analyzers understand the complete codebase without needing to compile for specific platforms. + +## 6. Development & Testing Workflow + +* **Local Development Environment:** Use the provided Docker container or create a Python virtual environment and install dependencies from `requirements_dev.txt`. +* **Running Commands:** Use the `script/run-in-env.py` script to execute commands within the project's virtual environment. For example, to run the linter: `python3 script/run-in-env.py pre-commit run`. +* **Testing:** + * **Python:** Run unit tests with `pytest`. + * **C++:** Use `clang-tidy` for static analysis. + * **Component Tests:** YAML-based compilation tests are located in `tests/`. The structure is as follows: + ``` + tests/ + ├── test_build_components/ # Base test configurations + └── components/[component]/ # Component-specific tests + ``` + Run them using `script/test_build_components`. Use `-c ` to test specific components and `-t ` for specific platforms. + * **Testing All Components Together:** To verify that all components can be tested together without ID conflicts or configuration issues, use: + ```bash + ./script/test_component_grouping.py -e config --all + ``` + This tests all components in a single build to catch conflicts that might not appear when testing components individually. Use `-e config` for fast configuration validation, or `-e compile` for full compilation testing. +* **Debugging and Troubleshooting:** + * **Debug Tools:** + - `esphome config .yaml` to validate configuration. + - `esphome compile .yaml` to compile without uploading. + - Check the Dashboard for real-time logs. + - Use component-specific debug logging. + * **Common Issues:** + - **Import Errors**: Check component dependencies and `PYTHONPATH`. + - **Validation Errors**: Review configuration schema definitions. + - **Build Errors**: Check platform compatibility and library versions. + - **Runtime Errors**: Review generated C++ code and component logic. + +## 7. Specific Instructions for AI Collaboration + +* **Contribution Workflow (Pull Request Process):** + 1. **Fork & Branch:** Create a new branch in your fork. + 2. **Make Changes:** Adhere to all coding conventions and patterns. + 3. **Test:** Create component tests for all supported platforms and run the full test suite locally. + 4. **Lint:** Run `pre-commit` to ensure code is compliant. + 5. **Commit:** Commit your changes. There is no strict format for commit messages. + 6. **Pull Request:** Submit a PR against the `dev` branch. The Pull Request title should have a prefix of the component being worked on (e.g., `[display] Fix bug`, `[abc123] Add new component`). Update documentation, examples, and add `CODEOWNERS` entries as needed. Pull requests should always be made with the PULL_REQUEST_TEMPLATE.md template filled out correctly. + +* **Documentation Contributions:** + * Documentation is hosted in the separate `esphome/esphome-docs` repository. + * The contribution workflow is the same as for the codebase. + +* **Best Practices:** + * **Component Development:** Keep dependencies minimal, provide clear error messages, and write comprehensive docstrings and tests. + * **Code Generation:** Generate minimal and efficient C++ code. Validate all user inputs thoroughly. Support multiple platform variations. + * **Configuration Design:** Aim for simplicity with sensible defaults, while allowing for advanced customization. + * **Embedded Systems Optimization:** ESPHome targets resource-constrained microcontrollers. Be mindful of flash size and RAM usage. + + **STL Container Guidelines:** + + ESPHome runs on embedded systems with limited resources. Choose containers carefully: + + 1. **Compile-time-known sizes:** Use `std::array` instead of `std::vector` when size is known at compile time. + ```cpp + // Bad - generates STL realloc code + std::vector values; + + // Good - no dynamic allocation + std::array values; + ``` + Use `cg.add_define("MAX_VALUES", count)` to set the size from Python configuration. + + **For byte buffers:** Avoid `std::vector` unless the buffer needs to grow. Use `std::unique_ptr` instead. + + > **Note:** `std::unique_ptr` does **not** provide bounds checking or iterator support like `std::vector`. Use it only when you do not need these features and want minimal overhead. + + ```cpp + // Bad - STL overhead for simple byte buffer + std::vector buffer; + buffer.resize(256); + + // Good - minimal overhead, single allocation + std::unique_ptr buffer = std::make_unique(256); + // Or if size is constant: + std::array buffer; + ``` + + 2. **Compile-time-known fixed sizes with vector-like API:** Use `StaticVector` from `esphome/core/helpers.h` for fixed-size stack allocation with `push_back()` interface. + ```cpp + // Bad - generates STL realloc code (_M_realloc_insert) + std::vector services; + services.reserve(5); // Still includes reallocation machinery + + // Good - compile-time fixed size, stack allocated, no reallocation machinery + StaticVector services; // Allocates all MAX_SERVICES on stack + services.push_back(record1); // Tracks count but all slots allocated + ``` + Use `cg.add_define("MAX_SERVICES", count)` to set the size from Python configuration. + Like `std::array` but with vector-like API (`push_back()`, `size()`) and no STL reallocation code. + + 3. **Runtime-known sizes:** Use `FixedVector` from `esphome/core/helpers.h` when the size is only known at runtime initialization. + ```cpp + // Bad - generates STL realloc code (_M_realloc_insert) + std::vector txt_records; + txt_records.reserve(5); // Still includes reallocation machinery + + // Good - runtime size, single allocation, no reallocation machinery + FixedVector txt_records; + txt_records.init(record_count); // Initialize with exact size at runtime + ``` + **Benefits:** + - Eliminates `_M_realloc_insert`, `_M_default_append` template instantiations (saves 200-500 bytes per instance) + - Single allocation, no upper bound needed + - No reallocation overhead + - Compatible with protobuf code generation when using `[(fixed_vector) = true]` option + + 4. **Small datasets (1-16 elements):** Use `std::vector` or `std::array` with simple structs instead of `std::map`/`std::set`/`std::unordered_map`. + ```cpp + // Bad - 2KB+ overhead for red-black tree/hash table + std::map small_lookup; + std::unordered_map tiny_map; + + // Good - simple struct with linear search (std::vector is fine) + struct LookupEntry { + const char *key; + int value; + }; + std::vector small_lookup = { + {"key1", 10}, + {"key2", 20}, + {"key3", 30}, + }; + // Or std::array if size is compile-time constant: + // std::array small_lookup = {{ ... }}; + ``` + Linear search on small datasets (1-16 elements) is often faster than hashing/tree overhead, but this depends on lookup frequency and access patterns. For frequent lookups in hot code paths, the O(1) vs O(n) complexity difference may still matter even for small datasets. `std::vector` with simple structs is usually fine—it's the heavy containers (`map`, `set`, `unordered_map`) that should be avoided for small datasets unless profiling shows otherwise. + + 5. **Detection:** Look for these patterns in compiler output: + - Large code sections with STL symbols (vector, map, set) + - `alloc`, `realloc`, `dealloc` in symbol names + - `_M_realloc_insert`, `_M_default_append` (vector reallocation) + - Red-black tree code (`rb_tree`, `_Rb_tree`) + - Hash table infrastructure (`unordered_map`, `hash`) + + **When to optimize:** + - Core components (API, network, logger) + - Widely-used components (mdns, wifi, ble) + - Components causing flash size complaints + + **When not to optimize:** + - Single-use niche components + - Code where readability matters more than bytes + - Already using appropriate containers + + * **State Management:** Use `CORE.data` for component state that needs to persist during configuration generation. Avoid module-level mutable globals. + + **Bad Pattern (Module-Level Globals):** + ```python + # Don't do this - state persists between compilation runs + _component_state = [] + _use_feature = None + + def enable_feature(): + global _use_feature + _use_feature = True + ``` + + **Good Pattern (CORE.data with Helpers):** + ```python + from esphome.core import CORE + + # Keys for CORE.data storage + COMPONENT_STATE_KEY = "my_component_state" + USE_FEATURE_KEY = "my_component_use_feature" + + def _get_component_state() -> list: + """Get component state from CORE.data.""" + return CORE.data.setdefault(COMPONENT_STATE_KEY, []) + + def _get_use_feature() -> bool | None: + """Get feature flag from CORE.data.""" + return CORE.data.get(USE_FEATURE_KEY) + + def _set_use_feature(value: bool) -> None: + """Set feature flag in CORE.data.""" + CORE.data[USE_FEATURE_KEY] = value + + def enable_feature(): + _set_use_feature(True) + ``` + + **Why this matters:** + - Module-level globals persist between compilation runs if the dashboard doesn't fork/exec + - `CORE.data` automatically clears between runs + - Typed helper functions provide better IDE support and maintainability + - Encapsulation makes state management explicit and testable + +* **Security:** Be mindful of security when making changes to the API, web server, or any other network-related code. Do not hardcode secrets or keys. + +* **Dependencies & Build System Integration:** + * **Python:** When adding a new Python dependency, add it to the appropriate `requirements*.txt` file and `pyproject.toml`. + * **C++ / PlatformIO:** When adding a new C++ dependency, add it to `platformio.ini` and use `cg.add_library`. + * **Build Flags:** Use `cg.add_build_flag(...)` to add compiler flags. diff --git a/.clang-tidy.hash b/.clang-tidy.hash index 18be8d78a9..2cd4319325 100644 --- a/.clang-tidy.hash +++ b/.clang-tidy.hash @@ -1 +1 @@ -07f621354fe1350ba51953c80273cd44a04aa44f15cc30bd7b8fe2a641427b7a +d7693a1e996cacd4a3d1c9a16336799c2a8cc3db02e4e74084151ce964581248 diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 5703d39be1..28437e6302 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -26,6 +26,7 @@ - [ ] RP2040 - [ ] BK72xx - [ ] RTL87xx +- [ ] nRF52840 ## Example entry for `config.yaml`: diff --git a/.github/actions/build-image/action.yaml b/.github/actions/build-image/action.yaml index 403b9d8c2a..9c7f051e05 100644 --- a/.github/actions/build-image/action.yaml +++ b/.github/actions/build-image/action.yaml @@ -47,7 +47,7 @@ runs: - name: Build and push to ghcr by digest id: build-ghcr - uses: docker/build-push-action@v6.18.0 + uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0 env: DOCKER_BUILD_SUMMARY: false DOCKER_BUILD_RECORD_UPLOAD: false @@ -73,7 +73,7 @@ runs: - name: Build and push to dockerhub by digest id: build-dockerhub - uses: docker/build-push-action@v6.18.0 + uses: docker/build-push-action@263435318d21b8e681c14492fe198d362a7d2c83 # v6.18.0 env: DOCKER_BUILD_SUMMARY: false DOCKER_BUILD_RECORD_UPLOAD: false diff --git a/.github/actions/restore-python/action.yml b/.github/actions/restore-python/action.yml index 3a7b301b60..f314e79ad9 100644 --- a/.github/actions/restore-python/action.yml +++ b/.github/actions/restore-python/action.yml @@ -17,12 +17,12 @@ runs: steps: - name: Set up Python ${{ inputs.python-version }} id: python - uses: actions/setup-python@v5.6.0 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: ${{ inputs.python-version }} - name: Restore Python virtual environment id: cache-venv - uses: actions/cache/restore@v4.2.3 + uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0 with: path: venv # yamllint disable-line rule:line-length diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md new file mode 120000 index 0000000000..a4b2fa310c --- /dev/null +++ b/.github/copilot-instructions.md @@ -0,0 +1 @@ +../.ai/instructions.md \ No newline at end of file diff --git a/.github/dependabot.yml b/.github/dependabot.yml index cf507bbaa6..528e69c478 100644 --- a/.github/dependabot.yml +++ b/.github/dependabot.yml @@ -9,6 +9,9 @@ updates: # Hypotehsis is only used for testing and is updated quite often - dependency-name: hypothesis - package-ecosystem: github-actions + labels: + - "dependencies" + - "github-actions" directory: "/" schedule: interval: daily @@ -20,11 +23,17 @@ updates: - "docker/login-action" - "docker/setup-buildx-action" - package-ecosystem: github-actions + labels: + - "dependencies" + - "github-actions" directory: "/.github/actions/build-image" schedule: interval: daily open-pull-requests-limit: 10 - package-ecosystem: github-actions + labels: + - "dependencies" + - "github-actions" directory: "/.github/actions/restore-python" schedule: interval: daily diff --git a/.github/workflows/auto-label-pr.yml b/.github/workflows/auto-label-pr.yml new file mode 100644 index 0000000000..1670bd1821 --- /dev/null +++ b/.github/workflows/auto-label-pr.yml @@ -0,0 +1,662 @@ +name: Auto Label PR + +on: + # Runs only on pull_request_target due to having access to a App token. + # This means PRs from forks will not be able to alter this workflow to get the tokens + pull_request_target: + types: [labeled, opened, reopened, synchronize, edited] + +permissions: + pull-requests: write + contents: read + +env: + SMALL_PR_THRESHOLD: 30 + MAX_LABELS: 15 + TOO_BIG_THRESHOLD: 1000 + COMPONENT_LABEL_THRESHOLD: 10 + +jobs: + label: + runs-on: ubuntu-latest + if: github.event.action != 'labeled' || github.event.sender.type != 'Bot' + steps: + - name: Checkout + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 + + - name: Generate a token + id: generate-token + uses: actions/create-github-app-token@67018539274d69449ef7c02e8e71183d1719ab42 # v2 + with: + app-id: ${{ secrets.ESPHOME_GITHUB_APP_ID }} + private-key: ${{ secrets.ESPHOME_GITHUB_APP_PRIVATE_KEY }} + + - name: Auto Label PR + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 + with: + github-token: ${{ steps.generate-token.outputs.token }} + script: | + const fs = require('fs'); + + // Constants + const SMALL_PR_THRESHOLD = parseInt('${{ env.SMALL_PR_THRESHOLD }}'); + const MAX_LABELS = parseInt('${{ env.MAX_LABELS }}'); + const TOO_BIG_THRESHOLD = parseInt('${{ env.TOO_BIG_THRESHOLD }}'); + const COMPONENT_LABEL_THRESHOLD = parseInt('${{ env.COMPONENT_LABEL_THRESHOLD }}'); + const BOT_COMMENT_MARKER = ''; + const CODEOWNERS_MARKER = ''; + const TOO_BIG_MARKER = ''; + + const MANAGED_LABELS = [ + 'new-component', + 'new-platform', + 'new-target-platform', + 'merging-to-release', + 'merging-to-beta', + 'core', + 'small-pr', + 'dashboard', + 'github-actions', + 'by-code-owner', + 'has-tests', + 'needs-tests', + 'needs-docs', + 'needs-codeowners', + 'too-big', + 'labeller-recheck', + 'bugfix', + 'new-feature', + 'breaking-change', + 'code-quality' + ]; + + const DOCS_PR_PATTERNS = [ + /https:\/\/github\.com\/esphome\/esphome-docs\/pull\/\d+/, + /esphome\/esphome-docs#\d+/ + ]; + + // Global state + const { owner, repo } = context.repo; + const pr_number = context.issue.number; + + // Get current labels and PR data + const { data: currentLabelsData } = await github.rest.issues.listLabelsOnIssue({ + owner, + repo, + issue_number: pr_number + }); + const currentLabels = currentLabelsData.map(label => label.name); + const managedLabels = currentLabels.filter(label => + label.startsWith('component: ') || MANAGED_LABELS.includes(label) + ); + + // Check for mega-PR early - if present, skip most automatic labeling + const isMegaPR = currentLabels.includes('mega-pr'); + + // Get all PR files with automatic pagination + const prFiles = await github.paginate( + github.rest.pulls.listFiles, + { + owner, + repo, + pull_number: pr_number + } + ); + + // Calculate data from PR files + const changedFiles = prFiles.map(file => file.filename); + const totalAdditions = prFiles.reduce((sum, file) => sum + (file.additions || 0), 0); + const totalDeletions = prFiles.reduce((sum, file) => sum + (file.deletions || 0), 0); + const totalChanges = totalAdditions + totalDeletions; + + console.log('Current labels:', currentLabels.join(', ')); + console.log('Changed files:', changedFiles.length); + console.log('Total changes:', totalChanges); + if (isMegaPR) { + console.log('Mega-PR detected - applying limited labeling logic'); + } + + // Fetch API data + async function fetchApiData() { + try { + const response = await fetch('https://data.esphome.io/components.json'); + const componentsData = await response.json(); + return { + targetPlatforms: componentsData.target_platforms || [], + platformComponents: componentsData.platform_components || [] + }; + } catch (error) { + console.log('Failed to fetch components data from API:', error.message); + return { targetPlatforms: [], platformComponents: [] }; + } + } + + // Strategy: Merge branch detection + async function detectMergeBranch() { + const labels = new Set(); + const baseRef = context.payload.pull_request.base.ref; + + if (baseRef === 'release') { + labels.add('merging-to-release'); + } else if (baseRef === 'beta') { + labels.add('merging-to-beta'); + } + + return labels; + } + + // Strategy: Component and platform labeling + async function detectComponentPlatforms(apiData) { + const labels = new Set(); + const componentRegex = /^esphome\/components\/([^\/]+)\//; + const targetPlatformRegex = new RegExp(`^esphome\/components\/(${apiData.targetPlatforms.join('|')})/`); + + for (const file of changedFiles) { + const componentMatch = file.match(componentRegex); + if (componentMatch) { + labels.add(`component: ${componentMatch[1]}`); + } + + const platformMatch = file.match(targetPlatformRegex); + if (platformMatch) { + labels.add(`platform: ${platformMatch[1]}`); + } + } + + return labels; + } + + // Strategy: New component detection + async function detectNewComponents() { + const labels = new Set(); + const addedFiles = prFiles.filter(file => file.status === 'added').map(file => file.filename); + + for (const file of addedFiles) { + const componentMatch = file.match(/^esphome\/components\/([^\/]+)\/__init__\.py$/); + if (componentMatch) { + try { + const content = fs.readFileSync(file, 'utf8'); + if (content.includes('IS_TARGET_PLATFORM = True')) { + labels.add('new-target-platform'); + } + } catch (error) { + console.log(`Failed to read content of ${file}:`, error.message); + } + labels.add('new-component'); + } + } + + return labels; + } + + // Strategy: New platform detection + async function detectNewPlatforms(apiData) { + const labels = new Set(); + const addedFiles = prFiles.filter(file => file.status === 'added').map(file => file.filename); + + for (const file of addedFiles) { + const platformFileMatch = file.match(/^esphome\/components\/([^\/]+)\/([^\/]+)\.py$/); + if (platformFileMatch) { + const [, component, platform] = platformFileMatch; + if (apiData.platformComponents.includes(platform)) { + labels.add('new-platform'); + } + } + + const platformDirMatch = file.match(/^esphome\/components\/([^\/]+)\/([^\/]+)\/__init__\.py$/); + if (platformDirMatch) { + const [, component, platform] = platformDirMatch; + if (apiData.platformComponents.includes(platform)) { + labels.add('new-platform'); + } + } + } + + return labels; + } + + // Strategy: Core files detection + async function detectCoreChanges() { + const labels = new Set(); + const coreFiles = changedFiles.filter(file => + file.startsWith('esphome/core/') || + (file.startsWith('esphome/') && file.split('/').length === 2) + ); + + if (coreFiles.length > 0) { + labels.add('core'); + } + + return labels; + } + + // Strategy: PR size detection + async function detectPRSize() { + const labels = new Set(); + + if (totalChanges <= SMALL_PR_THRESHOLD) { + labels.add('small-pr'); + return labels; + } + + const testAdditions = prFiles + .filter(file => file.filename.startsWith('tests/')) + .reduce((sum, file) => sum + (file.additions || 0), 0); + const testDeletions = prFiles + .filter(file => file.filename.startsWith('tests/')) + .reduce((sum, file) => sum + (file.deletions || 0), 0); + + const nonTestChanges = (totalAdditions - testAdditions) - (totalDeletions - testDeletions); + + // Don't add too-big if mega-pr label is already present + if (nonTestChanges > TOO_BIG_THRESHOLD && !isMegaPR) { + labels.add('too-big'); + } + + return labels; + } + + // Strategy: Dashboard changes + async function detectDashboardChanges() { + const labels = new Set(); + const dashboardFiles = changedFiles.filter(file => + file.startsWith('esphome/dashboard/') || + file.startsWith('esphome/components/dashboard_import/') + ); + + if (dashboardFiles.length > 0) { + labels.add('dashboard'); + } + + return labels; + } + + // Strategy: GitHub Actions changes + async function detectGitHubActionsChanges() { + const labels = new Set(); + const githubActionsFiles = changedFiles.filter(file => + file.startsWith('.github/workflows/') + ); + + if (githubActionsFiles.length > 0) { + labels.add('github-actions'); + } + + return labels; + } + + // Strategy: Code owner detection + async function detectCodeOwner() { + const labels = new Set(); + + try { + const { data: codeownersFile } = await github.rest.repos.getContent({ + owner, + repo, + path: 'CODEOWNERS', + }); + + const codeownersContent = Buffer.from(codeownersFile.content, 'base64').toString('utf8'); + const prAuthor = context.payload.pull_request.user.login; + + const codeownersLines = codeownersContent.split('\n') + .map(line => line.trim()) + .filter(line => line && !line.startsWith('#')); + + const codeownersRegexes = codeownersLines.map(line => { + const parts = line.split(/\s+/); + const pattern = parts[0]; + const owners = parts.slice(1); + + let regex; + if (pattern.endsWith('*')) { + const dir = pattern.slice(0, -1); + regex = new RegExp(`^${dir.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}`); + } else if (pattern.includes('*')) { + // First escape all regex special chars except *, then replace * with .* + const regexPattern = pattern + .replace(/[.+?^${}()|[\]\\]/g, '\\$&') + .replace(/\*/g, '.*'); + regex = new RegExp(`^${regexPattern}$`); + } else { + regex = new RegExp(`^${pattern.replace(/[.*+?^${}()|[\]\\]/g, '\\$&')}$`); + } + + return { regex, owners }; + }); + + for (const file of changedFiles) { + for (const { regex, owners } of codeownersRegexes) { + if (regex.test(file) && owners.some(owner => owner === `@${prAuthor}`)) { + labels.add('by-code-owner'); + return labels; + } + } + } + } catch (error) { + console.log('Failed to read or parse CODEOWNERS file:', error.message); + } + + return labels; + } + + // Strategy: Test detection + async function detectTests() { + const labels = new Set(); + const testFiles = changedFiles.filter(file => file.startsWith('tests/')); + + if (testFiles.length > 0) { + labels.add('has-tests'); + } + + return labels; + } + + // Strategy: PR Template Checkbox detection + async function detectPRTemplateCheckboxes() { + const labels = new Set(); + const prBody = context.payload.pull_request.body || ''; + + console.log('Checking PR template checkboxes...'); + + // Check for checked checkboxes in the "Types of changes" section + const checkboxPatterns = [ + { pattern: /- \[x\] Bugfix \(non-breaking change which fixes an issue\)/i, label: 'bugfix' }, + { pattern: /- \[x\] New feature \(non-breaking change which adds functionality\)/i, label: 'new-feature' }, + { pattern: /- \[x\] Breaking change \(fix or feature that would cause existing functionality to not work as expected\)/i, label: 'breaking-change' }, + { pattern: /- \[x\] Code quality improvements to existing code or addition of tests/i, label: 'code-quality' } + ]; + + for (const { pattern, label } of checkboxPatterns) { + if (pattern.test(prBody)) { + console.log(`Found checked checkbox for: ${label}`); + labels.add(label); + } + } + + return labels; + } + + // Strategy: Requirements detection + async function detectRequirements(allLabels) { + const labels = new Set(); + + // Check for missing tests + if ((allLabels.has('new-component') || allLabels.has('new-platform') || allLabels.has('new-feature')) && !allLabels.has('has-tests')) { + labels.add('needs-tests'); + } + + // Check for missing docs + if (allLabels.has('new-component') || allLabels.has('new-platform') || allLabels.has('new-feature')) { + const prBody = context.payload.pull_request.body || ''; + const hasDocsLink = DOCS_PR_PATTERNS.some(pattern => pattern.test(prBody)); + + if (!hasDocsLink) { + labels.add('needs-docs'); + } + } + + // Check for missing CODEOWNERS + if (allLabels.has('new-component')) { + const codeownersModified = prFiles.some(file => + file.filename === 'CODEOWNERS' && + (file.status === 'modified' || file.status === 'added') && + (file.additions || 0) > 0 + ); + + if (!codeownersModified) { + labels.add('needs-codeowners'); + } + } + + return labels; + } + + // Generate review messages + function generateReviewMessages(finalLabels) { + const messages = []; + const prAuthor = context.payload.pull_request.user.login; + + // Too big message + if (finalLabels.includes('too-big')) { + const testAdditions = prFiles + .filter(file => file.filename.startsWith('tests/')) + .reduce((sum, file) => sum + (file.additions || 0), 0); + const testDeletions = prFiles + .filter(file => file.filename.startsWith('tests/')) + .reduce((sum, file) => sum + (file.deletions || 0), 0); + const nonTestChanges = (totalAdditions - testAdditions) - (totalDeletions - testDeletions); + + const tooManyLabels = finalLabels.length > MAX_LABELS; + const tooManyChanges = nonTestChanges > TOO_BIG_THRESHOLD; + + let message = `${TOO_BIG_MARKER}\n### 📦 Pull Request Size\n\n`; + + if (tooManyLabels && tooManyChanges) { + message += `This PR is too large with ${nonTestChanges} line changes (excluding tests) and affects ${finalLabels.length} different components/areas.`; + } else if (tooManyLabels) { + message += `This PR affects ${finalLabels.length} different components/areas.`; + } else { + message += `This PR is too large with ${nonTestChanges} line changes (excluding tests).`; + } + + message += ` Please consider breaking it down into smaller, focused PRs to make review easier and reduce the risk of conflicts.\n\n`; + message += `For guidance on breaking down large PRs, see: https://developers.esphome.io/contributing/submitting-your-work/#how-to-approach-large-submissions`; + + messages.push(message); + } + + // CODEOWNERS message + if (finalLabels.includes('needs-codeowners')) { + const message = `${CODEOWNERS_MARKER}\n### 👥 Code Ownership\n\n` + + `Hey there @${prAuthor},\n` + + `Thanks for submitting this pull request! Can you add yourself as a codeowner for this integration? ` + + `This way we can notify you if a bug report for this integration is reported.\n\n` + + `In \`__init__.py\` of the integration, please add:\n\n` + + `\`\`\`python\nCODEOWNERS = ["@${prAuthor}"]\n\`\`\`\n\n` + + `And run \`script/build_codeowners.py\``; + + messages.push(message); + } + + return messages; + } + + // Handle reviews + async function handleReviews(finalLabels) { + const reviewMessages = generateReviewMessages(finalLabels); + const hasReviewableLabels = finalLabels.some(label => + ['too-big', 'needs-codeowners'].includes(label) + ); + + const { data: reviews } = await github.rest.pulls.listReviews({ + owner, + repo, + pull_number: pr_number + }); + + const botReviews = reviews.filter(review => + review.user.type === 'Bot' && + review.state === 'CHANGES_REQUESTED' && + review.body && review.body.includes(BOT_COMMENT_MARKER) + ); + + if (hasReviewableLabels) { + const reviewBody = `${BOT_COMMENT_MARKER}\n\n${reviewMessages.join('\n\n---\n\n')}`; + + if (botReviews.length > 0) { + // Update existing review + await github.rest.pulls.updateReview({ + owner, + repo, + pull_number: pr_number, + review_id: botReviews[0].id, + body: reviewBody + }); + console.log('Updated existing bot review'); + } else { + // Create new review + await github.rest.pulls.createReview({ + owner, + repo, + pull_number: pr_number, + body: reviewBody, + event: 'REQUEST_CHANGES' + }); + console.log('Created new bot review'); + } + } else if (botReviews.length > 0) { + // Dismiss existing reviews + for (const review of botReviews) { + try { + await github.rest.pulls.dismissReview({ + owner, + repo, + pull_number: pr_number, + review_id: review.id, + message: 'Review dismissed: All requirements have been met' + }); + console.log(`Dismissed bot review ${review.id}`); + } catch (error) { + console.log(`Failed to dismiss review ${review.id}:`, error.message); + } + } + } + } + + // Main execution + const apiData = await fetchApiData(); + const baseRef = context.payload.pull_request.base.ref; + + // Early exit for non-dev branches + if (baseRef !== 'dev') { + const branchLabels = await detectMergeBranch(); + const finalLabels = Array.from(branchLabels); + + console.log('Computed labels (merge branch only):', finalLabels.join(', ')); + + // Apply labels + if (finalLabels.length > 0) { + await github.rest.issues.addLabels({ + owner, + repo, + issue_number: pr_number, + labels: finalLabels + }); + } + + // Remove old managed labels + const labelsToRemove = managedLabels.filter(label => !finalLabels.includes(label)); + for (const label of labelsToRemove) { + try { + await github.rest.issues.removeLabel({ + owner, + repo, + issue_number: pr_number, + name: label + }); + } catch (error) { + console.log(`Failed to remove label ${label}:`, error.message); + } + } + + return; + } + + // Run all strategies + const [ + branchLabels, + componentLabels, + newComponentLabels, + newPlatformLabels, + coreLabels, + sizeLabels, + dashboardLabels, + actionsLabels, + codeOwnerLabels, + testLabels, + checkboxLabels + ] = await Promise.all([ + detectMergeBranch(), + detectComponentPlatforms(apiData), + detectNewComponents(), + detectNewPlatforms(apiData), + detectCoreChanges(), + detectPRSize(), + detectDashboardChanges(), + detectGitHubActionsChanges(), + detectCodeOwner(), + detectTests(), + detectPRTemplateCheckboxes() + ]); + + // Combine all labels + const allLabels = new Set([ + ...branchLabels, + ...componentLabels, + ...newComponentLabels, + ...newPlatformLabels, + ...coreLabels, + ...sizeLabels, + ...dashboardLabels, + ...actionsLabels, + ...codeOwnerLabels, + ...testLabels, + ...checkboxLabels + ]); + + // Detect requirements based on all other labels + const requirementLabels = await detectRequirements(allLabels); + for (const label of requirementLabels) { + allLabels.add(label); + } + + let finalLabels = Array.from(allLabels); + + // For mega-PRs, exclude component labels if there are too many + if (isMegaPR) { + const componentLabels = finalLabels.filter(label => label.startsWith('component: ')); + if (componentLabels.length > COMPONENT_LABEL_THRESHOLD) { + finalLabels = finalLabels.filter(label => !label.startsWith('component: ')); + console.log(`Mega-PR detected - excluding ${componentLabels.length} component labels (threshold: ${COMPONENT_LABEL_THRESHOLD})`); + } + } + + // Handle too many labels (only for non-mega PRs) + const tooManyLabels = finalLabels.length > MAX_LABELS; + + if (tooManyLabels && !isMegaPR && !finalLabels.includes('too-big')) { + finalLabels = ['too-big']; + } + + console.log('Computed labels:', finalLabels.join(', ')); + + // Handle reviews + await handleReviews(finalLabels); + + // Apply labels + if (finalLabels.length > 0) { + console.log(`Adding labels: ${finalLabels.join(', ')}`); + await github.rest.issues.addLabels({ + owner, + repo, + issue_number: pr_number, + labels: finalLabels + }); + } + + // Remove old managed labels + const labelsToRemove = managedLabels.filter(label => !finalLabels.includes(label)); + for (const label of labelsToRemove) { + console.log(`Removing label: ${label}`); + try { + await github.rest.issues.removeLabel({ + owner, + repo, + issue_number: pr_number, + name: label + }); + } catch (error) { + console.log(`Failed to remove label ${label}:`, error.message); + } + } diff --git a/.github/workflows/ci-api-proto.yml b/.github/workflows/ci-api-proto.yml index f51bd84186..c122859442 100644 --- a/.github/workflows/ci-api-proto.yml +++ b/.github/workflows/ci-api-proto.yml @@ -21,9 +21,9 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Set up Python - uses: actions/setup-python@v5.6.0 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: "3.11" @@ -47,7 +47,7 @@ jobs: fi - if: failure() name: Review PR - uses: actions/github-script@v7.0.1 + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 with: script: | await github.rest.pulls.createReview({ @@ -62,7 +62,7 @@ jobs: run: git diff - if: failure() name: Archive artifacts - uses: actions/upload-artifact@v4.6.2 + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: generated-proto-files path: | @@ -70,7 +70,7 @@ jobs: esphome/components/api/api_pb2_service.* - if: success() name: Dismiss review - uses: actions/github-script@v7.0.1 + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 with: script: | let reviews = await github.rest.pulls.listReviews({ diff --git a/.github/workflows/ci-clang-tidy-hash.yml b/.github/workflows/ci-clang-tidy-hash.yml index 1c7a62e40b..78d1c2b87f 100644 --- a/.github/workflows/ci-clang-tidy-hash.yml +++ b/.github/workflows/ci-clang-tidy-hash.yml @@ -6,6 +6,7 @@ on: - ".clang-tidy" - "platformio.ini" - "requirements_dev.txt" + - "sdkconfig.defaults" - ".clang-tidy.hash" - "script/clang_tidy_hash.py" - ".github/workflows/ci-clang-tidy-hash.yml" @@ -20,10 +21,10 @@ jobs: runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Set up Python - uses: actions/setup-python@v5.6.0 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: "3.11" @@ -41,7 +42,7 @@ jobs: - if: failure() name: Request changes - uses: actions/github-script@v7.0.1 + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 with: script: | await github.rest.pulls.createReview({ @@ -54,7 +55,7 @@ jobs: - if: success() name: Dismiss review - uses: actions/github-script@v7.0.1 + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 with: script: | let reviews = await github.rest.pulls.listReviews({ diff --git a/.github/workflows/ci-docker.yml b/.github/workflows/ci-docker.yml index f76ebba8e9..7111c61dda 100644 --- a/.github/workflows/ci-docker.yml +++ b/.github/workflows/ci-docker.yml @@ -43,13 +43,13 @@ jobs: - "docker" # - "lint" steps: - - uses: actions/checkout@v4.2.2 + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Set up Python - uses: actions/setup-python@v5.6.0 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: - python-version: "3.10" + python-version: "3.11" - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v3.11.1 + uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1 - name: Set TAG run: | diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2f63a16844..87e182fe4d 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -20,8 +20,8 @@ permissions: contents: read env: - DEFAULT_PYTHON: "3.10" - PYUPGRADE_TARGET: "--py310-plus" + DEFAULT_PYTHON: "3.11" + PYUPGRADE_TARGET: "--py311-plus" concurrency: # yamllint disable-line rule:line-length @@ -36,18 +36,18 @@ jobs: cache-key: ${{ steps.cache-key.outputs.key }} steps: - name: Check out code from GitHub - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Generate cache-key id: cache-key run: echo key="${{ hashFiles('requirements.txt', 'requirements_test.txt', '.pre-commit-config.yaml') }}" >> $GITHUB_OUTPUT - name: Set up Python ${{ env.DEFAULT_PYTHON }} id: python - uses: actions/setup-python@v5.6.0 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: ${{ env.DEFAULT_PYTHON }} - name: Restore Python virtual environment id: cache-venv - uses: actions/cache@v4.2.3 + uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0 with: path: venv # yamllint disable-line rule:line-length @@ -70,7 +70,7 @@ jobs: if: needs.determine-jobs.outputs.python-linters == 'true' steps: - name: Check out code from GitHub - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Restore Python uses: ./.github/actions/restore-python with: @@ -91,7 +91,7 @@ jobs: - common steps: - name: Check out code from GitHub - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Restore Python uses: ./.github/actions/restore-python with: @@ -105,6 +105,7 @@ jobs: script/ci-custom.py script/build_codeowners.py --check script/build_language_schema.py --check + script/generate-esp32-boards.py --check pytest: name: Run pytest @@ -112,10 +113,8 @@ jobs: fail-fast: false matrix: python-version: - - "3.10" - "3.11" - - "3.12" - - "3.13" + - "3.14" os: - ubuntu-latest - macOS-latest @@ -124,24 +123,16 @@ jobs: # Minimize CI resource usage # by only running the Python version # version used for docker images on Windows and macOS - - python-version: "3.13" + - python-version: "3.14" os: windows-latest - - python-version: "3.12" - os: windows-latest - - python-version: "3.10" - os: windows-latest - - python-version: "3.13" - os: macOS-latest - - python-version: "3.12" - os: macOS-latest - - python-version: "3.10" + - python-version: "3.14" os: macOS-latest runs-on: ${{ matrix.os }} needs: - common steps: - name: Check out code from GitHub - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Restore Python id: restore-python uses: ./.github/actions/restore-python @@ -161,12 +152,12 @@ jobs: . venv/bin/activate pytest -vv --cov-report=xml --tb=native -n auto tests --ignore=tests/integration/ - name: Upload coverage to Codecov - uses: codecov/codecov-action@v5.4.3 + uses: codecov/codecov-action@5a1091511ad55cbe89839c7260b706298ca349f7 # v5.5.1 with: token: ${{ secrets.CODECOV_TOKEN }} - name: Save Python virtual environment cache if: github.ref == 'refs/heads/dev' - uses: actions/cache/save@v4.2.3 + uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0 with: path: venv key: ${{ runner.os }}-${{ steps.restore-python.outputs.python-version }}-venv-${{ needs.common.outputs.cache-key }} @@ -181,10 +172,12 @@ jobs: clang-tidy: ${{ steps.determine.outputs.clang-tidy }} python-linters: ${{ steps.determine.outputs.python-linters }} changed-components: ${{ steps.determine.outputs.changed-components }} + changed-components-with-tests: ${{ steps.determine.outputs.changed-components-with-tests }} + directly-changed-components-with-tests: ${{ steps.determine.outputs.directly-changed-components-with-tests }} component-test-count: ${{ steps.determine.outputs.component-test-count }} steps: - name: Check out code from GitHub - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: # Fetch enough history to find the merge base fetch-depth: 2 @@ -208,6 +201,8 @@ jobs: echo "clang-tidy=$(echo "$output" | jq -r '.clang_tidy')" >> $GITHUB_OUTPUT echo "python-linters=$(echo "$output" | jq -r '.python_linters')" >> $GITHUB_OUTPUT echo "changed-components=$(echo "$output" | jq -c '.changed_components')" >> $GITHUB_OUTPUT + echo "changed-components-with-tests=$(echo "$output" | jq -c '.changed_components_with_tests')" >> $GITHUB_OUTPUT + echo "directly-changed-components-with-tests=$(echo "$output" | jq -c '.directly_changed_components_with_tests')" >> $GITHUB_OUTPUT echo "component-test-count=$(echo "$output" | jq -r '.component_test_count')" >> $GITHUB_OUTPUT integration-tests: @@ -219,15 +214,15 @@ jobs: if: needs.determine-jobs.outputs.integration-tests == 'true' steps: - name: Check out code from GitHub - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Set up Python 3.13 id: python - uses: actions/setup-python@v5.6.0 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: "3.13" - name: Restore Python virtual environment id: cache-venv - uses: actions/cache@v4.2.3 + uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0 with: path: venv key: ${{ runner.os }}-${{ steps.python.outputs.python-version }}-venv-${{ needs.common.outputs.cache-key }} @@ -286,13 +281,13 @@ jobs: pio_cache_key: tidyesp32-idf - id: clang-tidy name: Run script/clang-tidy for ZEPHYR - options: --environment nrf52-tidy --grep USE_ZEPHYR + options: --environment nrf52-tidy --grep USE_ZEPHYR --grep USE_NRF52 pio_cache_key: tidy-zephyr ignore_errors: false steps: - name: Check out code from GitHub - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: # Need history for HEAD~1 to work for checking changed files fetch-depth: 2 @@ -305,14 +300,14 @@ jobs: - name: Cache platformio if: github.ref == 'refs/heads/dev' - uses: actions/cache@v4.2.3 + uses: actions/cache@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0 with: path: ~/.platformio key: platformio-${{ matrix.pio_cache_key }}-${{ hashFiles('platformio.ini') }} - name: Cache platformio if: github.ref != 'refs/heads/dev' - uses: actions/cache/restore@v4.2.3 + uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4.3.0 with: path: ~/.platformio key: platformio-${{ matrix.pio_cache_key }}-${{ hashFiles('platformio.ini') }} @@ -360,120 +355,170 @@ jobs: # yamllint disable-line rule:line-length if: always() - test-build-components: - name: Component test ${{ matrix.file }} + test-build-components-splitter: + name: Split components for intelligent grouping (40 weighted per batch) runs-on: ubuntu-24.04 needs: - common - determine-jobs - if: github.event_name == 'pull_request' && fromJSON(needs.determine-jobs.outputs.component-test-count) > 0 && fromJSON(needs.determine-jobs.outputs.component-test-count) < 100 - strategy: - fail-fast: false - max-parallel: 2 - matrix: - file: ${{ fromJson(needs.determine-jobs.outputs.changed-components) }} + if: github.event_name == 'pull_request' && fromJSON(needs.determine-jobs.outputs.component-test-count) > 0 + outputs: + matrix: ${{ steps.split.outputs.components }} steps: - - name: Install dependencies - run: | - sudo apt-get update - sudo apt-get install libsdl2-dev - - name: Check out code from GitHub - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Restore Python uses: ./.github/actions/restore-python with: python-version: ${{ env.DEFAULT_PYTHON }} cache-key: ${{ needs.common.outputs.cache-key }} - - name: test_build_components -e config -c ${{ matrix.file }} - run: | - . venv/bin/activate - ./script/test_build_components -e config -c ${{ matrix.file }} - - name: test_build_components -e compile -c ${{ matrix.file }} - run: | - . venv/bin/activate - ./script/test_build_components -e compile -c ${{ matrix.file }} - - test-build-components-splitter: - name: Split components for testing into 20 groups maximum - runs-on: ubuntu-24.04 - needs: - - common - - determine-jobs - if: github.event_name == 'pull_request' && fromJSON(needs.determine-jobs.outputs.component-test-count) >= 100 - outputs: - matrix: ${{ steps.split.outputs.components }} - steps: - - name: Check out code from GitHub - uses: actions/checkout@v4.2.2 - - name: Split components into 20 groups + - name: Split components intelligently based on bus configurations id: split run: | - components=$(echo '${{ needs.determine-jobs.outputs.changed-components }}' | jq -c '.[]' | shuf | jq -s -c '[_nwise(20) | join(" ")]') - echo "components=$components" >> $GITHUB_OUTPUT + . venv/bin/activate + + # Use intelligent splitter that groups components with same bus configs + components='${{ needs.determine-jobs.outputs.changed-components-with-tests }}' + + # Only isolate directly changed components when targeting dev branch + # For beta/release branches, group everything for faster CI + if [[ "${{ github.base_ref }}" == beta* ]] || [[ "${{ github.base_ref }}" == release* ]]; then + directly_changed='[]' + echo "Target branch: ${{ github.base_ref }} - grouping all components" + else + directly_changed='${{ needs.determine-jobs.outputs.directly-changed-components-with-tests }}' + echo "Target branch: ${{ github.base_ref }} - isolating directly changed components" + fi + + echo "Splitting components intelligently..." + output=$(python3 script/split_components_for_ci.py --components "$components" --directly-changed "$directly_changed" --batch-size 40 --output github) + + echo "$output" >> $GITHUB_OUTPUT test-build-components-split: - name: Test split components + name: Test components batch (${{ matrix.components }}) runs-on: ubuntu-24.04 needs: - common - determine-jobs - test-build-components-splitter - if: github.event_name == 'pull_request' && fromJSON(needs.determine-jobs.outputs.component-test-count) >= 100 + if: github.event_name == 'pull_request' && fromJSON(needs.determine-jobs.outputs.component-test-count) > 0 strategy: fail-fast: false - max-parallel: 4 + max-parallel: ${{ (startsWith(github.base_ref, 'beta') || startsWith(github.base_ref, 'release')) && 8 || 4 }} matrix: components: ${{ fromJson(needs.test-build-components-splitter.outputs.matrix) }} steps: + - name: Show disk space + run: | + echo "Available disk space:" + df -h + - name: List components run: echo ${{ matrix.components }} - - name: Install dependencies - run: | - sudo apt-get update - sudo apt-get install libsdl2-dev + - name: Cache apt packages + uses: awalsh128/cache-apt-pkgs-action@acb598e5ddbc6f68a970c5da0688d2f3a9f04d05 # v1.5.3 + with: + packages: libsdl2-dev + version: 1.0 - name: Check out code from GitHub - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Restore Python uses: ./.github/actions/restore-python with: python-version: ${{ env.DEFAULT_PYTHON }} cache-key: ${{ needs.common.outputs.cache-key }} - - name: Validate config + - name: Validate and compile components with intelligent grouping run: | . venv/bin/activate - for component in ${{ matrix.components }}; do - ./script/test_build_components -e config -c $component - done - - name: Compile config - run: | - . venv/bin/activate - mkdir build_cache - export PLATFORMIO_BUILD_CACHE_DIR=$PWD/build_cache - for component in ${{ matrix.components }}; do - ./script/test_build_components -e compile -c $component - done + + # Check if /mnt has more free space than / before bind mounting + # Extract available space in KB for comparison + root_avail=$(df -k / | awk 'NR==2 {print $4}') + mnt_avail=$(df -k /mnt 2>/dev/null | awk 'NR==2 {print $4}') + + echo "Available space: / has ${root_avail}KB, /mnt has ${mnt_avail}KB" + + # Only use /mnt if it has more space than / + if [ -n "$mnt_avail" ] && [ "$mnt_avail" -gt "$root_avail" ]; then + echo "Using /mnt for build files (more space available)" + # Bind mount PlatformIO directory to /mnt (tools, packages, build cache all go there) + sudo mkdir -p /mnt/platformio + sudo chown $USER:$USER /mnt/platformio + mkdir -p ~/.platformio + sudo mount --bind /mnt/platformio ~/.platformio + + # Bind mount test build directory to /mnt + sudo mkdir -p /mnt/test_build_components_build + sudo chown $USER:$USER /mnt/test_build_components_build + mkdir -p tests/test_build_components/build + sudo mount --bind /mnt/test_build_components_build tests/test_build_components/build + else + echo "Using / for build files (more space available than /mnt or /mnt unavailable)" + fi + + # Convert space-separated components to comma-separated for Python script + components_csv=$(echo "${{ matrix.components }}" | tr ' ' ',') + + # Only isolate directly changed components when targeting dev branch + # For beta/release branches, group everything for faster CI + # + # WHY ISOLATE DIRECTLY CHANGED COMPONENTS? + # - Isolated tests run WITHOUT --testing-mode, enabling full validation + # - This catches pin conflicts and other issues in directly changed code + # - Grouped tests use --testing-mode to allow config merging (disables some checks) + # - Dependencies are safe to group since they weren't modified in this PR + if [[ "${{ github.base_ref }}" == beta* ]] || [[ "${{ github.base_ref }}" == release* ]]; then + directly_changed_csv="" + echo "Testing components: $components_csv" + echo "Target branch: ${{ github.base_ref }} - grouping all components" + else + directly_changed_csv=$(echo '${{ needs.determine-jobs.outputs.directly-changed-components-with-tests }}' | jq -r 'join(",")') + echo "Testing components: $components_csv" + echo "Target branch: ${{ github.base_ref }} - isolating directly changed components: $directly_changed_csv" + fi + echo "" + + # Show disk space before validation (after bind mounts setup) + echo "Disk space before config validation:" + df -h + echo "" + + # Run config validation with grouping and isolation + python3 script/test_build_components.py -e config -c "$components_csv" -f --isolate "$directly_changed_csv" + + echo "" + echo "Config validation passed! Starting compilation..." + echo "" + + # Show disk space before compilation + echo "Disk space before compilation:" + df -h + echo "" + + # Run compilation with grouping and isolation + python3 script/test_build_components.py -e compile -c "$components_csv" -f --isolate "$directly_changed_csv" pre-commit-ci-lite: name: pre-commit.ci lite runs-on: ubuntu-latest needs: - common - if: github.event_name == 'pull_request' && github.base_ref != 'beta' && github.base_ref != 'release' + if: github.event_name == 'pull_request' && !startsWith(github.base_ref, 'beta') && !startsWith(github.base_ref, 'release') steps: - name: Check out code from GitHub - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Restore Python uses: ./.github/actions/restore-python with: python-version: ${{ env.DEFAULT_PYTHON }} cache-key: ${{ needs.common.outputs.cache-key }} - - uses: pre-commit/action@v3.0.1 + - uses: esphome/action@43cd1109c09c544d97196f7730ee5b2e0cc6d81e # v3.0.1 fork with pinned actions/cache env: SKIP: pylint,clang-tidy-hash - - uses: pre-commit-ci/lite-action@v1.1.0 + - uses: pre-commit-ci/lite-action@5d6cc0eb514c891a40562a58a8e71576c5c7fb43 # v1.1.0 if: always() ci-status: @@ -487,7 +532,6 @@ jobs: - integration-tests - clang-tidy - determine-jobs - - test-build-components - test-build-components-splitter - test-build-components-split - pre-commit-ci-lite diff --git a/.github/workflows/codeowner-review-request.yml b/.github/workflows/codeowner-review-request.yml new file mode 100644 index 0000000000..563d55f42b --- /dev/null +++ b/.github/workflows/codeowner-review-request.yml @@ -0,0 +1,324 @@ +# This workflow automatically requests reviews from codeowners when: +# 1. A PR is opened, reopened, or synchronized (updated) +# 2. A PR is marked as ready for review +# +# It reads the CODEOWNERS file and matches all changed files in the PR against +# the codeowner patterns, then requests reviews from the appropriate owners +# while avoiding duplicate requests for users who have already been requested +# or have already reviewed the PR. + +name: Request Codeowner Reviews + +on: + # Needs to be pull_request_target to get write permissions + pull_request_target: + types: [opened, reopened, synchronize, ready_for_review] + +permissions: + pull-requests: write + contents: read + +jobs: + request-codeowner-reviews: + name: Run + if: ${{ !github.event.pull_request.draft }} + runs-on: ubuntu-latest + steps: + - name: Request reviews from component codeowners + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 + with: + script: | + const owner = context.repo.owner; + const repo = context.repo.repo; + const pr_number = context.payload.pull_request.number; + + console.log(`Processing PR #${pr_number} for codeowner review requests`); + + // Hidden marker to identify bot comments from this workflow + const BOT_COMMENT_MARKER = ''; + + try { + // Get the list of changed files in this PR + const { data: files } = await github.rest.pulls.listFiles({ + owner, + repo, + pull_number: pr_number + }); + + const changedFiles = files.map(file => file.filename); + console.log(`Found ${changedFiles.length} changed files`); + + if (changedFiles.length === 0) { + console.log('No changed files found, skipping codeowner review requests'); + return; + } + + // Fetch CODEOWNERS file from root + const { data: codeownersFile } = await github.rest.repos.getContent({ + owner, + repo, + path: 'CODEOWNERS', + ref: context.payload.pull_request.base.sha + }); + const codeownersContent = Buffer.from(codeownersFile.content, 'base64').toString('utf8'); + + // Parse CODEOWNERS file to extract all patterns and their owners + const codeownersLines = codeownersContent.split('\n') + .map(line => line.trim()) + .filter(line => line && !line.startsWith('#')); + + const codeownersPatterns = []; + + // Convert CODEOWNERS pattern to regex (robust glob handling) + function globToRegex(pattern) { + // Escape regex special characters except for glob wildcards + let regexStr = pattern + .replace(/([.+^=!:${}()|[\]\\])/g, '\\$1') // escape regex chars + .replace(/\*\*/g, '.*') // globstar + .replace(/\*/g, '[^/]*') // single star + .replace(/\?/g, '.'); // question mark + return new RegExp('^' + regexStr + '$'); + } + + // Helper function to create comment body + function createCommentBody(reviewersList, teamsList, matchedFileCount, isSuccessful = true) { + const reviewerMentions = reviewersList.map(r => `@${r}`); + const teamMentions = teamsList.map(t => `@${owner}/${t}`); + const allMentions = [...reviewerMentions, ...teamMentions].join(', '); + + if (isSuccessful) { + return `${BOT_COMMENT_MARKER}\n👋 Hi there! I've automatically requested reviews from codeowners based on the files changed in this PR.\n\n${allMentions} - You've been requested to review this PR as codeowner(s) of ${matchedFileCount} file(s) that were modified. Thanks for your time! 🙏`; + } else { + return `${BOT_COMMENT_MARKER}\n👋 Hi there! This PR modifies ${matchedFileCount} file(s) with codeowners.\n\n${allMentions} - As codeowner(s) of the affected files, your review would be appreciated! 🙏\n\n_Note: Automatic review request may have failed, but you're still welcome to review._`; + } + } + + for (const line of codeownersLines) { + const parts = line.split(/\s+/); + if (parts.length < 2) continue; + + const pattern = parts[0]; + const owners = parts.slice(1); + + // Use robust glob-to-regex conversion + const regex = globToRegex(pattern); + codeownersPatterns.push({ pattern, regex, owners }); + } + + console.log(`Parsed ${codeownersPatterns.length} codeowner patterns`); + + // Match changed files against CODEOWNERS patterns + const matchedOwners = new Set(); + const matchedTeams = new Set(); + const fileMatches = new Map(); // Track which files matched which patterns + + for (const file of changedFiles) { + for (const { pattern, regex, owners } of codeownersPatterns) { + if (regex.test(file)) { + console.log(`File '${file}' matches pattern '${pattern}' with owners: ${owners.join(', ')}`); + + if (!fileMatches.has(file)) { + fileMatches.set(file, []); + } + fileMatches.get(file).push({ pattern, owners }); + + // Add owners to the appropriate set (remove @ prefix) + for (const owner of owners) { + const cleanOwner = owner.startsWith('@') ? owner.slice(1) : owner; + if (cleanOwner.includes('/')) { + // Team mention (org/team-name) + const teamName = cleanOwner.split('/')[1]; + matchedTeams.add(teamName); + } else { + // Individual user + matchedOwners.add(cleanOwner); + } + } + } + } + } + + if (matchedOwners.size === 0 && matchedTeams.size === 0) { + console.log('No codeowners found for any changed files'); + return; + } + + // Remove the PR author from reviewers + const prAuthor = context.payload.pull_request.user.login; + matchedOwners.delete(prAuthor); + + // Get current reviewers to avoid duplicate requests (but still mention them) + const { data: prData } = await github.rest.pulls.get({ + owner, + repo, + pull_number: pr_number + }); + + const currentReviewers = new Set(); + const currentTeams = new Set(); + + if (prData.requested_reviewers) { + prData.requested_reviewers.forEach(reviewer => { + currentReviewers.add(reviewer.login); + }); + } + + if (prData.requested_teams) { + prData.requested_teams.forEach(team => { + currentTeams.add(team.slug); + }); + } + + // Check for completed reviews to avoid re-requesting users who have already reviewed + const { data: reviews } = await github.rest.pulls.listReviews({ + owner, + repo, + pull_number: pr_number + }); + + const reviewedUsers = new Set(); + reviews.forEach(review => { + reviewedUsers.add(review.user.login); + }); + + // Check for previous comments from this workflow to avoid duplicate pings + const comments = await github.paginate( + github.rest.issues.listComments, + { + owner, + repo, + issue_number: pr_number + } + ); + + const previouslyPingedUsers = new Set(); + const previouslyPingedTeams = new Set(); + + // Look for comments from github-actions bot that contain our bot marker + const workflowComments = comments.filter(comment => + comment.user.type === 'Bot' && + comment.body.includes(BOT_COMMENT_MARKER) + ); + + // Extract previously mentioned users and teams from workflow comments + for (const comment of workflowComments) { + // Match @username patterns (not team mentions) + const userMentions = comment.body.match(/@([a-zA-Z0-9_.-]+)(?![/])/g) || []; + userMentions.forEach(mention => { + const username = mention.slice(1); // remove @ + previouslyPingedUsers.add(username); + }); + + // Match @org/team patterns + const teamMentions = comment.body.match(/@[a-zA-Z0-9_.-]+\/([a-zA-Z0-9_.-]+)/g) || []; + teamMentions.forEach(mention => { + const teamName = mention.split('/')[1]; + previouslyPingedTeams.add(teamName); + }); + } + + console.log(`Found ${previouslyPingedUsers.size} previously pinged users and ${previouslyPingedTeams.size} previously pinged teams`); + + // Remove users who have already been pinged in previous workflow comments + previouslyPingedUsers.forEach(user => { + matchedOwners.delete(user); + }); + + previouslyPingedTeams.forEach(team => { + matchedTeams.delete(team); + }); + + // Remove only users who have already submitted reviews (not just requested reviewers) + reviewedUsers.forEach(reviewer => { + matchedOwners.delete(reviewer); + }); + + // For teams, we'll still remove already requested teams to avoid API errors + currentTeams.forEach(team => { + matchedTeams.delete(team); + }); + + const reviewersList = Array.from(matchedOwners); + const teamsList = Array.from(matchedTeams); + + if (reviewersList.length === 0 && teamsList.length === 0) { + console.log('No eligible reviewers found (all may already be requested, reviewed, or previously pinged)'); + return; + } + + const totalReviewers = reviewersList.length + teamsList.length; + console.log(`Requesting reviews from ${reviewersList.length} users and ${teamsList.length} teams for ${fileMatches.size} matched files`); + + // Request reviews + try { + const requestParams = { + owner, + repo, + pull_number: pr_number + }; + + // Filter out users who are already requested reviewers for the API call + const newReviewers = reviewersList.filter(reviewer => !currentReviewers.has(reviewer)); + const newTeams = teamsList.filter(team => !currentTeams.has(team)); + + if (newReviewers.length > 0) { + requestParams.reviewers = newReviewers; + } + + if (newTeams.length > 0) { + requestParams.team_reviewers = newTeams; + } + + // Only make the API call if there are new reviewers to request + if (newReviewers.length > 0 || newTeams.length > 0) { + await github.rest.pulls.requestReviewers(requestParams); + console.log(`Successfully requested reviews from ${newReviewers.length} new users and ${newTeams.length} new teams`); + } else { + console.log('All codeowners are already requested reviewers or have reviewed'); + } + + // Only add a comment if there are new codeowners to mention (not previously pinged) + if (reviewersList.length > 0 || teamsList.length > 0) { + const commentBody = createCommentBody(reviewersList, teamsList, fileMatches.size, true); + + await github.rest.issues.createComment({ + owner, + repo, + issue_number: pr_number, + body: commentBody + }); + console.log(`Added comment mentioning ${reviewersList.length} users and ${teamsList.length} teams`); + } else { + console.log('No new codeowners to mention in comment (all previously pinged)'); + } + } catch (error) { + if (error.status === 422) { + console.log('Some reviewers may already be requested or unavailable:', error.message); + + // Only try to add a comment if there are new codeowners to mention + if (reviewersList.length > 0 || teamsList.length > 0) { + const commentBody = createCommentBody(reviewersList, teamsList, fileMatches.size, false); + + try { + await github.rest.issues.createComment({ + owner, + repo, + issue_number: pr_number, + body: commentBody + }); + console.log(`Added fallback comment mentioning ${reviewersList.length} users and ${teamsList.length} teams`); + } catch (commentError) { + console.log('Failed to add comment:', commentError.message); + } + } else { + console.log('No new codeowners to mention in fallback comment'); + } + } else { + throw error; + } + } + + } catch (error) { + console.log('Failed to process codeowner review requests:', error.message); + console.error(error); + } diff --git a/.github/workflows/codeql.yml b/.github/workflows/codeql.yml index ddeb0a99d2..c24900d378 100644 --- a/.github/workflows/codeql.yml +++ b/.github/workflows/codeql.yml @@ -54,11 +54,11 @@ jobs: # your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages steps: - name: Checkout repository - uses: actions/checkout@v4 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 # Initializes the CodeQL tools for scanning. - name: Initialize CodeQL - uses: github/codeql-action/init@v3 + uses: github/codeql-action/init@16140ae1a102900babc80a33c44059580f687047 # v4.30.9 with: languages: ${{ matrix.language }} build-mode: ${{ matrix.build-mode }} @@ -86,6 +86,6 @@ jobs: exit 1 - name: Perform CodeQL Analysis - uses: github/codeql-action/analyze@v3 + uses: github/codeql-action/analyze@16140ae1a102900babc80a33c44059580f687047 # v4.30.9 with: category: "/language:${{matrix.language}}" diff --git a/.github/workflows/external-component-bot.yml b/.github/workflows/external-component-bot.yml new file mode 100644 index 0000000000..4fa020f63d --- /dev/null +++ b/.github/workflows/external-component-bot.yml @@ -0,0 +1,157 @@ +name: Add External Component Comment + +on: + pull_request_target: + types: [opened, synchronize] + +permissions: + contents: read # Needed to fetch PR details + issues: write # Needed to create and update comments (PR comments are managed via the issues REST API) + pull-requests: write # also needed? + +jobs: + external-comment: + name: External component comment + runs-on: ubuntu-latest + steps: + - name: Add external component comment + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 + with: + github-token: ${{ secrets.GITHUB_TOKEN }} + script: | + // Generate external component usage instructions + function generateExternalComponentInstructions(prNumber, componentNames, owner, repo) { + let source; + if (owner === 'esphome' && repo === 'esphome') + source = `github://pr#${prNumber}`; + else + source = `github://${owner}/${repo}@pull/${prNumber}/head`; + return `To use the changes from this PR as an external component, add the following to your ESPHome configuration YAML file: + + \`\`\`yaml + external_components: + - source: ${source} + components: [${componentNames.join(', ')}] + refresh: 1h + \`\`\``; + } + + // Generate repo clone instructions + function generateRepoInstructions(prNumber, owner, repo, branch) { + return `To use the changes in this PR: + + \`\`\`bash + # Clone the repository: + git clone https://github.com/${owner}/${repo} + cd ${repo} + + # Checkout the PR branch: + git fetch origin pull/${prNumber}/head:${branch} + git checkout ${branch} + + # Install the development version: + script/setup + + # Activate the development version: + source venv/bin/activate + \`\`\` + + Now you can run \`esphome\` as usual to test the changes in this PR. + `; + } + + async function createComment(octokit, owner, repo, prNumber, esphomeChanges, componentChanges) { + const commentMarker = ""; + const legacyCommentMarker = ""; + let commentBody; + if (esphomeChanges.length === 1) { + commentBody = generateExternalComponentInstructions(prNumber, componentChanges, owner, repo); + } else { + commentBody = generateRepoInstructions(prNumber, owner, repo, context.payload.pull_request.head.ref); + } + commentBody += `\n\n---\n(Added by the PR bot)\n\n${commentMarker}`; + + // Check for existing bot comment + const comments = await github.paginate( + github.rest.issues.listComments, + { + owner: owner, + repo: repo, + issue_number: prNumber, + per_page: 100, + } + ); + + const sorted = comments.sort((a, b) => new Date(b.updated_at) - new Date(a.updated_at)); + + const botComment = sorted.find(comment => + ( + comment.body.includes(commentMarker) || + comment.body.includes(legacyCommentMarker) + ) && comment.user.type === "Bot" + ); + + if (botComment && botComment.body === commentBody) { + // No changes in the comment, do nothing + return; + } + + if (botComment) { + // Update existing comment + await github.rest.issues.updateComment({ + owner: owner, + repo: repo, + comment_id: botComment.id, + body: commentBody, + }); + } else { + // Create new comment + await github.rest.issues.createComment({ + owner: owner, + repo: repo, + issue_number: prNumber, + body: commentBody, + }); + } + } + + async function getEsphomeAndComponentChanges(github, owner, repo, prNumber) { + const changedFiles = await github.rest.pulls.listFiles({ + owner: owner, + repo: repo, + pull_number: prNumber, + }); + + const esphomeChanges = changedFiles.data + .filter(file => file.filename !== "esphome/core/defines.h" && file.filename.startsWith('esphome/')) + .map(file => { + const match = file.filename.match(/esphome\/([^/]+)/); + return match ? match[1] : null; + }) + .filter(it => it !== null); + + if (esphomeChanges.length === 0) { + return {esphomeChanges: [], componentChanges: []}; + } + + const uniqueEsphomeChanges = [...new Set(esphomeChanges)]; + const componentChanges = changedFiles.data + .filter(file => file.filename.startsWith('esphome/components/')) + .map(file => { + const match = file.filename.match(/esphome\/components\/([^/]+)\//); + return match ? match[1] : null; + }) + .filter(it => it !== null); + + return {esphomeChanges: uniqueEsphomeChanges, componentChanges: [...new Set(componentChanges)]}; + } + + // Start of main code. + + const prNumber = context.payload.pull_request.number; + const {owner, repo} = context.repo; + + const {esphomeChanges, componentChanges} = await getEsphomeAndComponentChanges(github, owner, repo, prNumber); + if (componentChanges.length !== 0) { + await createComment(github, owner, repo, prNumber, esphomeChanges, componentChanges); + } diff --git a/.github/workflows/issue-codeowner-notify.yml b/.github/workflows/issue-codeowner-notify.yml new file mode 100644 index 0000000000..6faf956c87 --- /dev/null +++ b/.github/workflows/issue-codeowner-notify.yml @@ -0,0 +1,163 @@ +# This workflow automatically notifies codeowners when an issue is labeled with component labels. +# It reads the CODEOWNERS file to find the maintainers for the labeled components +# and posts a comment mentioning them to ensure they're aware of the issue. + +name: Notify Issue Codeowners + +on: + issues: + types: [labeled] + +permissions: + issues: write + contents: read + +jobs: + notify-codeowners: + name: Run + if: ${{ startsWith(github.event.label.name, format('component{0} ', ':')) }} + runs-on: ubuntu-latest + steps: + - name: Notify codeowners for component issues + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 + with: + script: | + const owner = context.repo.owner; + const repo = context.repo.repo; + const issue_number = context.payload.issue.number; + const labelName = context.payload.label.name; + + console.log(`Processing issue #${issue_number} with label: ${labelName}`); + + // Hidden marker to identify bot comments from this workflow + const BOT_COMMENT_MARKER = ''; + + // Extract component name from label + const componentName = labelName.replace('component: ', ''); + console.log(`Component: ${componentName}`); + + try { + // Fetch CODEOWNERS file from root + const { data: codeownersFile } = await github.rest.repos.getContent({ + owner, + repo, + path: 'CODEOWNERS' + }); + const codeownersContent = Buffer.from(codeownersFile.content, 'base64').toString('utf8'); + + // Parse CODEOWNERS file to extract component mappings + const codeownersLines = codeownersContent.split('\n') + .map(line => line.trim()) + .filter(line => line && !line.startsWith('#')); + + let componentOwners = null; + + for (const line of codeownersLines) { + const parts = line.split(/\s+/); + if (parts.length < 2) continue; + + const pattern = parts[0]; + const owners = parts.slice(1); + + // Look for component patterns: esphome/components/{component}/* + const componentMatch = pattern.match(/^esphome\/components\/([^\/]+)\/\*$/); + if (componentMatch && componentMatch[1] === componentName) { + componentOwners = owners; + break; + } + } + + if (!componentOwners) { + console.log(`No codeowners found for component: ${componentName}`); + return; + } + + console.log(`Found codeowners for '${componentName}': ${componentOwners.join(', ')}`); + + // Separate users and teams + const userOwners = []; + const teamOwners = []; + + for (const owner of componentOwners) { + const cleanOwner = owner.startsWith('@') ? owner.slice(1) : owner; + if (cleanOwner.includes('/')) { + // Team mention (org/team-name) + teamOwners.push(`@${cleanOwner}`); + } else { + // Individual user + userOwners.push(`@${cleanOwner}`); + } + } + + // Remove issue author from mentions to avoid self-notification + const issueAuthor = context.payload.issue.user.login; + const filteredUserOwners = userOwners.filter(mention => + mention !== `@${issueAuthor}` + ); + + // Check for previous comments from this workflow to avoid duplicate pings + const comments = await github.paginate( + github.rest.issues.listComments, + { + owner, + repo, + issue_number: issue_number + } + ); + + const previouslyPingedUsers = new Set(); + const previouslyPingedTeams = new Set(); + + // Look for comments from github-actions bot that contain codeowner pings for this component + const workflowComments = comments.filter(comment => + comment.user.type === 'Bot' && + comment.body.includes(BOT_COMMENT_MARKER) && + comment.body.includes(`component: ${componentName}`) + ); + + // Extract previously mentioned users and teams from workflow comments + for (const comment of workflowComments) { + // Match @username patterns (not team mentions) + const userMentions = comment.body.match(/@([a-zA-Z0-9_.-]+)(?![/])/g) || []; + userMentions.forEach(mention => { + previouslyPingedUsers.add(mention); // Keep @ prefix for easy comparison + }); + + // Match @org/team patterns + const teamMentions = comment.body.match(/@[a-zA-Z0-9_.-]+\/[a-zA-Z0-9_.-]+/g) || []; + teamMentions.forEach(mention => { + previouslyPingedTeams.add(mention); + }); + } + + console.log(`Found ${previouslyPingedUsers.size} previously pinged users and ${previouslyPingedTeams.size} previously pinged teams for component ${componentName}`); + + // Remove previously pinged users and teams + const newUserOwners = filteredUserOwners.filter(mention => !previouslyPingedUsers.has(mention)); + const newTeamOwners = teamOwners.filter(mention => !previouslyPingedTeams.has(mention)); + + const allMentions = [...newUserOwners, ...newTeamOwners]; + + if (allMentions.length === 0) { + console.log('No new codeowners to notify (all previously pinged or issue author is the only codeowner)'); + return; + } + + // Create comment body + const mentionString = allMentions.join(', '); + const commentBody = `${BOT_COMMENT_MARKER}\n👋 Hey ${mentionString}!\n\nThis issue has been labeled with \`component: ${componentName}\` and you've been identified as a codeowner of this component. Please take a look when you have a chance!\n\nThanks for maintaining this component! 🙏`; + + // Post comment + await github.rest.issues.createComment({ + owner, + repo, + issue_number: issue_number, + body: commentBody + }); + + console.log(`Successfully notified new codeowners: ${mentionString}`); + + } catch (error) { + console.log('Failed to process codeowner notifications:', error.message); + console.error(error); + } diff --git a/.github/workflows/needs-docs.yml b/.github/workflows/needs-docs.yml deleted file mode 100644 index 628b5cc5e3..0000000000 --- a/.github/workflows/needs-docs.yml +++ /dev/null @@ -1,24 +0,0 @@ -name: Needs Docs - -on: - pull_request: - types: [labeled, unlabeled] - -jobs: - check: - name: Check - runs-on: ubuntu-latest - steps: - - name: Check for needs-docs label - uses: actions/github-script@v7.0.1 - with: - script: | - const { data: labels } = await github.rest.issues.listLabelsOnIssue({ - owner: context.repo.owner, - repo: context.repo.repo, - issue_number: context.issue.number - }); - const needsDocs = labels.find(label => label.name === 'needs-docs'); - if (needsDocs) { - core.setFailed('Pull request needs docs'); - } diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index b4518b27b5..2b3b3bdc1b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -20,7 +20,7 @@ jobs: branch_build: ${{ steps.tag.outputs.branch_build }} deploy_env: ${{ steps.tag.outputs.deploy_env }} steps: - - uses: actions/checkout@v4.2.2 + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Get tag id: tag # yamllint disable rule:line-length @@ -60,9 +60,9 @@ jobs: contents: read id-token: write steps: - - uses: actions/checkout@v4.2.2 + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Set up Python - uses: actions/setup-python@v5.6.0 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: "3.x" - name: Build @@ -70,7 +70,7 @@ jobs: pip3 install build python3 -m build - name: Publish - uses: pypa/gh-action-pypi-publish@v1.12.4 + uses: pypa/gh-action-pypi-publish@ed0c53931b1dc9bd32cbe73a98c7f6766f8a527e # v1.13.0 with: skip-existing: true @@ -92,22 +92,22 @@ jobs: os: "ubuntu-24.04-arm" steps: - - uses: actions/checkout@v4.2.2 + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Set up Python - uses: actions/setup-python@v5.6.0 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: - python-version: "3.10" + python-version: "3.11" - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v3.11.1 + uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1 - name: Log in to docker hub - uses: docker/login-action@v3.4.0 + uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 with: username: ${{ secrets.DOCKER_USER }} password: ${{ secrets.DOCKER_PASSWORD }} - name: Log in to the GitHub container registry - uses: docker/login-action@v3.4.0 + uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 with: registry: ghcr.io username: ${{ github.actor }} @@ -138,7 +138,7 @@ jobs: # version: ${{ needs.init.outputs.tag }} - name: Upload digests - uses: actions/upload-artifact@v4.6.2 + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2 with: name: digests-${{ matrix.platform.arch }} path: /tmp/digests @@ -168,27 +168,27 @@ jobs: - ghcr - dockerhub steps: - - uses: actions/checkout@v4.2.2 + - uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Download digests - uses: actions/download-artifact@v4.3.0 + uses: actions/download-artifact@634f93cb2916e3fdff6788551b99b062d0335ce0 # v5.0.0 with: pattern: digests-* path: /tmp/digests merge-multiple: true - name: Set up Docker Buildx - uses: docker/setup-buildx-action@v3.11.1 + uses: docker/setup-buildx-action@e468171a9de216ec08956ac3ada2f0791b6bd435 # v3.11.1 - name: Log in to docker hub if: matrix.registry == 'dockerhub' - uses: docker/login-action@v3.4.0 + uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 with: username: ${{ secrets.DOCKER_USER }} password: ${{ secrets.DOCKER_PASSWORD }} - name: Log in to the GitHub container registry if: matrix.registry == 'ghcr' - uses: docker/login-action@v3.4.0 + uses: docker/login-action@5e57cd118135c172c3672efd75eb46360885c0ef # v3.6.0 with: registry: ghcr.io username: ${{ github.actor }} @@ -220,7 +220,7 @@ jobs: - deploy-manifest steps: - name: Trigger Workflow - uses: actions/github-script@v7.0.1 + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 with: github-token: ${{ secrets.DEPLOY_HA_ADDON_REPO_TOKEN }} script: | @@ -246,7 +246,7 @@ jobs: environment: ${{ needs.init.outputs.deploy_env }} steps: - name: Trigger Workflow - uses: actions/github-script@v7.0.1 + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 with: github-token: ${{ secrets.DEPLOY_ESPHOME_SCHEMA_REPO_TOKEN }} script: | diff --git a/.github/workflows/stale.yml b/.github/workflows/stale.yml index b79939fc8e..5843b3a5e0 100644 --- a/.github/workflows/stale.yml +++ b/.github/workflows/stale.yml @@ -15,36 +15,52 @@ concurrency: jobs: stale: + if: github.repository_owner == 'esphome' runs-on: ubuntu-latest steps: - - uses: actions/stale@v9.1.0 + - name: Stale + uses: actions/stale@5f858e3efba33a5ca4407a664cc011ad407f2008 # v10.1.0 with: + debug-only: ${{ github.ref != 'refs/heads/dev' }} # Dry-run when not run on dev branch + remove-stale-when-updated: true + operations-per-run: 400 + + # The 90 day stale policy for PRs + # - PRs + # - No PRs marked as "not-stale" + # - No Issues (see below) days-before-pr-stale: 90 days-before-pr-close: 7 - days-before-issue-stale: -1 - days-before-issue-close: -1 - remove-stale-when-updated: true stale-pr-label: "stale" exempt-pr-labels: "not-stale" stale-pr-message: > There hasn't been any activity on this pull request recently. This pull request has been automatically marked as stale because of that and will be closed if no further activity occurs within 7 days. - Thank you for your contributions. - # Use stale to automatically close issues with a - # reference to the issue tracker - close-issues: - runs-on: ubuntu-latest - steps: - - uses: actions/stale@v9.1.0 - with: - days-before-pr-stale: -1 - days-before-pr-close: -1 - days-before-issue-stale: 1 - days-before-issue-close: 1 - remove-stale-when-updated: true + If you are the author of this PR, please leave a comment if you want + to keep it open. Also, please rebase your PR onto the latest dev + branch to ensure that it's up to date with the latest changes. + + Thank you for your contribution! + + # The 90 day stale policy for Issues + # - Issues + # - No Issues marked as "not-stale" + # - No PRs (see above) + days-before-issue-stale: 90 + days-before-issue-close: 7 stale-issue-label: "stale" exempt-issue-labels: "not-stale" stale-issue-message: > - https://github.com/esphome/esphome/issues/430 + There hasn't been any activity on this issue recently. Due to the + high number of incoming GitHub notifications, we have to clean some + of the old issues, as many of them have already been resolved with + the latest updates. + + Please make sure to update to the latest ESPHome version and + check if that solves the issue. Let us know if that works for you by + adding a comment 👍 + + This issue has now been marked as stale and will be closed if no + further activity occurs. Thank you for your contributions. diff --git a/.github/workflows/status-check-labels.yml b/.github/workflows/status-check-labels.yml new file mode 100644 index 0000000000..e44fd18132 --- /dev/null +++ b/.github/workflows/status-check-labels.yml @@ -0,0 +1,30 @@ +name: Status check labels + +on: + pull_request: + types: [labeled, unlabeled] + +jobs: + check: + name: Check ${{ matrix.label }} + runs-on: ubuntu-latest + strategy: + fail-fast: false + matrix: + label: + - needs-docs + - merge-after-release + steps: + - name: Check for ${{ matrix.label }} label + uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0 + with: + script: | + const { data: labels } = await github.rest.issues.listLabelsOnIssue({ + owner: context.repo.owner, + repo: context.repo.repo, + issue_number: context.issue.number + }); + const hasLabel = labels.find(label => label.name === '${{ matrix.label }}'); + if (hasLabel) { + core.setFailed('Pull request cannot be merged, it is labeled as ${{ matrix.label }}'); + } diff --git a/.github/workflows/sync-device-classes.yml b/.github/workflows/sync-device-classes.yml index a38825fc45..9479645ccc 100644 --- a/.github/workflows/sync-device-classes.yml +++ b/.github/workflows/sync-device-classes.yml @@ -13,16 +13,16 @@ jobs: if: github.repository == 'esphome/esphome' steps: - name: Checkout - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 - name: Checkout Home Assistant - uses: actions/checkout@v4.2.2 + uses: actions/checkout@08c6903cd8c0fde910a37f88322edcfb5dd907a8 # v5.0.0 with: repository: home-assistant/core path: lib/home-assistant - name: Setup Python - uses: actions/setup-python@v5.6.0 + uses: actions/setup-python@e797f83bcb11b83ae66e0230d6156d7c80228e7c # v6.0.0 with: python-version: 3.13 @@ -30,13 +30,18 @@ jobs: run: | python -m pip install --upgrade pip pip install -e lib/home-assistant + pip install -r requirements_test.txt pre-commit - name: Sync run: | python ./script/sync-device_class.py + - name: Run pre-commit hooks + run: | + python script/run-in-env.py pre-commit run --all-files + - name: Commit changes - uses: peter-evans/create-pull-request@v7.0.8 + uses: peter-evans/create-pull-request@271a8d0340265f705b14b6d32b9829c1cb33d45e # v7.0.8 with: commit-message: "Synchronise Device Classes from Home Assistant" committer: esphomebot diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 118253861d..9e0e71d388 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -11,7 +11,7 @@ ci: repos: - repo: https://github.com/astral-sh/ruff-pre-commit # Ruff version. - rev: v0.12.3 + rev: v0.14.1 hooks: # Run the linter. - id: ruff @@ -40,7 +40,7 @@ repos: rev: v3.20.0 hooks: - id: pyupgrade - args: [--py310-plus] + args: [--py311-plus] - repo: https://github.com/adrienverge/yamllint.git rev: v1.37.1 hooks: diff --git a/CLAUDE.md b/CLAUDE.md new file mode 120000 index 0000000000..49e811ff05 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1 @@ +.ai/instructions.md \ No newline at end of file diff --git a/CODEOWNERS b/CODEOWNERS index 2975080ba9..09bd15137a 100644 --- a/CODEOWNERS +++ b/CODEOWNERS @@ -9,6 +9,7 @@ pyproject.toml @esphome/core esphome/*.py @esphome/core esphome/core/* @esphome/core +.github/** @esphome/core # Integrations esphome/components/a01nyub/* @MrSuicideParrot @@ -39,11 +40,11 @@ esphome/components/analog_threshold/* @ianchi esphome/components/animation/* @syndlex esphome/components/anova/* @buxtronix esphome/components/apds9306/* @aodrenah -esphome/components/api/* @OttoWinter +esphome/components/api/* @esphome/core esphome/components/as5600/* @ammmze esphome/components/as5600/sensor/* @ammmze esphome/components/as7341/* @mrgnr -esphome/components/async_tcp/* @OttoWinter +esphome/components/async_tcp/* @esphome/core esphome/components/at581x/* @X-Ryl669 esphome/components/atc_mithermometer/* @ahpohl esphome/components/atm90e26/* @danieltwagner @@ -61,14 +62,16 @@ esphome/components/bedjet/fan/* @jhansche esphome/components/bedjet/sensor/* @javawizard @jhansche esphome/components/beken_spi_led_strip/* @Mat931 esphome/components/bh1750/* @OttoWinter +esphome/components/bh1900nux/* @B48D81EFCC esphome/components/binary_sensor/* @esphome/core esphome/components/bk72xx/* @kuba2k2 esphome/components/bl0906/* @athom-tech @jesserockz @tarontop esphome/components/bl0939/* @ziceva -esphome/components/bl0940/* @tobias- +esphome/components/bl0940/* @dan-s-github @tobias- esphome/components/bl0942/* @dbuezas @dwmw2 esphome/components/ble_client/* @buxtronix @clydebarrow -esphome/components/bluetooth_proxy/* @jesserockz +esphome/components/ble_nus/* @tomaszduda23 +esphome/components/bluetooth_proxy/* @bdraco @jesserockz esphome/components/bme280_base/* @esphome/core esphome/components/bme280_spi/* @apbodrov esphome/components/bme680_bsec/* @trvrnrth @@ -87,10 +90,11 @@ esphome/components/bp1658cj/* @Cossid esphome/components/bp5758d/* @Cossid esphome/components/button/* @esphome/core esphome/components/bytebuffer/* @clydebarrow -esphome/components/camera/* @DT-art1 @bdraco +esphome/components/camera/* @bdraco @DT-art1 +esphome/components/camera_encoder/* @DT-art1 esphome/components/canbus/* @danielschramm @mvturnho esphome/components/cap1188/* @mreditor97 -esphome/components/captive_portal/* @OttoWinter +esphome/components/captive_portal/* @esphome/core esphome/components/ccs811/* @habbie esphome/components/cd74hc4067/* @asoehlke esphome/components/ch422g/* @clydebarrow @jesterret @@ -117,7 +121,7 @@ esphome/components/dallas_temp/* @ssieb esphome/components/daly_bms/* @s1lvi0 esphome/components/dashboard_import/* @esphome/core esphome/components/datetime/* @jesserockz @rfdarter -esphome/components/debug/* @OttoWinter +esphome/components/debug/* @esphome/core esphome/components/delonghi/* @grob6000 esphome/components/dfplayer/* @glmnet esphome/components/dfrobot_sen0395/* @niklasweber @@ -137,15 +141,17 @@ esphome/components/ens160_base/* @latonita @vincentscode esphome/components/ens160_i2c/* @latonita esphome/components/ens160_spi/* @latonita esphome/components/ens210/* @itn3rd77 +esphome/components/epaper_spi/* @esphome/core esphome/components/es7210/* @kahrendt esphome/components/es7243e/* @kbx81 esphome/components/es8156/* @kbx81 esphome/components/es8311/* @kahrendt @kroimon esphome/components/es8388/* @P4uLT esphome/components/esp32/* @esphome/core -esphome/components/esp32_ble/* @Rapsssito @jesserockz -esphome/components/esp32_ble_client/* @jesserockz -esphome/components/esp32_ble_server/* @Rapsssito @clydebarrow @jesserockz +esphome/components/esp32_ble/* @bdraco @jesserockz @Rapsssito +esphome/components/esp32_ble_client/* @bdraco @jesserockz +esphome/components/esp32_ble_server/* @clydebarrow @jesserockz @Rapsssito +esphome/components/esp32_ble_tracker/* @bdraco esphome/components/esp32_camera_web_server/* @ayufan esphome/components/esp32_can/* @Sympatron esphome/components/esp32_hosted/* @swoboda1337 @@ -154,16 +160,16 @@ esphome/components/esp32_rmt/* @jesserockz esphome/components/esp32_rmt_led_strip/* @jesserockz esphome/components/esp8266/* @esphome/core esphome/components/esp_ldo/* @clydebarrow +esphome/components/espnow/* @jesserockz esphome/components/ethernet_info/* @gtjadsonsantos esphome/components/event/* @nohat -esphome/components/event_emitter/* @Rapsssito esphome/components/exposure_notifications/* @OttoWinter esphome/components/ezo/* @ssieb esphome/components/ezo_pmp/* @carlos-sarmiento esphome/components/factory_reset/* @anatoly-savchenkov esphome/components/fastled_base/* @OttoWinter esphome/components/feedback/* @ianchi -esphome/components/fingerprint_grow/* @OnFreund @alexborro @loongyh +esphome/components/fingerprint_grow/* @alexborro @loongyh @OnFreund esphome/components/font/* @clydebarrow @esphome/core esphome/components/fs3000/* @kahrendt esphome/components/ft5x06/* @clydebarrow @@ -199,7 +205,7 @@ esphome/components/heatpumpir/* @rob-deutsch esphome/components/hitachi_ac424/* @sourabhjaiswal esphome/components/hm3301/* @freekode esphome/components/hmac_md5/* @dwmw2 -esphome/components/homeassistant/* @OttoWinter @esphome/core +esphome/components/homeassistant/* @esphome/core @OttoWinter esphome/components/homeassistant/number/* @landonr esphome/components/homeassistant/switch/* @Links2004 esphome/components/honeywell_hih_i2c/* @Benichou34 @@ -224,18 +230,18 @@ esphome/components/iaqcore/* @yozik04 esphome/components/ili9xxx/* @clydebarrow @nielsnl68 esphome/components/improv_base/* @esphome/core esphome/components/improv_serial/* @esphome/core -esphome/components/ina226/* @Sergio303 @latonita +esphome/components/ina226/* @latonita @Sergio303 esphome/components/ina260/* @mreditor97 esphome/components/ina2xx_base/* @latonita esphome/components/ina2xx_i2c/* @latonita esphome/components/ina2xx_spi/* @latonita esphome/components/inkbird_ibsth1_mini/* @fkirill -esphome/components/inkplate6/* @jesserockz +esphome/components/inkplate/* @jesserockz @JosipKuci esphome/components/integration/* @OttoWinter esphome/components/internal_temperature/* @Mat931 esphome/components/interval/* @esphome/core esphome/components/jsn_sr04t/* @Mafus1 -esphome/components/json/* @OttoWinter +esphome/components/json/* @esphome/core esphome/components/kamstrup_kmp/* @cfeenstra1024 esphome/components/key_collector/* @ssieb esphome/components/key_provider/* @ssieb @@ -243,14 +249,17 @@ esphome/components/kuntze/* @ssieb esphome/components/lc709203f/* @ilikecake esphome/components/lcd_menu/* @numo68 esphome/components/ld2410/* @regevbr @sebcaps +esphome/components/ld2412/* @Rihan9 esphome/components/ld2420/* @descipher esphome/components/ld2450/* @hareeshmu +esphome/components/ld24xx/* @kbx81 esphome/components/ledc/* @OttoWinter esphome/components/libretiny/* @kuba2k2 esphome/components/libretiny_pwm/* @kuba2k2 esphome/components/light/* @esphome/core esphome/components/lightwaverf/* @max246 esphome/components/lilygo_t5_47/touchscreen/* @jesserockz +esphome/components/lm75b/* @beormund esphome/components/ln882x/* @lamauny esphome/components/lock/* @esphome/core esphome/components/logger/* @esphome/core @@ -271,8 +280,8 @@ esphome/components/max7219digit/* @rspaargaren esphome/components/max9611/* @mckaymatthew esphome/components/mcp23008/* @jesserockz esphome/components/mcp23017/* @jesserockz -esphome/components/mcp23s08/* @SenexCrenshaw @jesserockz -esphome/components/mcp23s17/* @SenexCrenshaw @jesserockz +esphome/components/mcp23s08/* @jesserockz @SenexCrenshaw +esphome/components/mcp23s17/* @jesserockz @SenexCrenshaw esphome/components/mcp23x08_base/* @jesserockz esphome/components/mcp23x17_base/* @jesserockz esphome/components/mcp23xxx_base/* @jesserockz @@ -292,6 +301,8 @@ esphome/components/microphone/* @jesserockz @kahrendt esphome/components/mics_4514/* @jesserockz esphome/components/midea/* @dudanov esphome/components/midea_ir/* @dudanov +esphome/components/mipi_dsi/* @clydebarrow +esphome/components/mipi_rgb/* @clydebarrow esphome/components/mipi_spi/* @clydebarrow esphome/components/mitsubishi/* @RubyBailey esphome/components/mixer/speaker/* @kahrendt @@ -324,6 +335,7 @@ esphome/components/nextion/text_sensor/* @senexcrenshaw esphome/components/nfc/* @jesserockz @kbx81 esphome/components/noblex/* @AGalfra esphome/components/npi19/* @bakerkj +esphome/components/nrf52/* @tomaszduda23 esphome/components/number/* @esphome/core esphome/components/one_wire/* @ssieb esphome/components/online_image/* @clydebarrow @guillempages @@ -334,7 +346,7 @@ esphome/components/ota/* @esphome/core esphome/components/output/* @esphome/core esphome/components/packet_transport/* @clydebarrow esphome/components/pca6416a/* @Mat931 -esphome/components/pca9554/* @clydebarrow @hwstar +esphome/components/pca9554/* @bdraco @clydebarrow @hwstar esphome/components/pcf85063/* @brogon esphome/components/pcf8563/* @KoenBreeman esphome/components/pi4ioe5v6408/* @jesserockz @@ -345,9 +357,9 @@ esphome/components/pm2005/* @andrewjswan esphome/components/pmsa003i/* @sjtrny esphome/components/pmsx003/* @ximex esphome/components/pmwcs3/* @SeByDocKy -esphome/components/pn532/* @OttoWinter @jesserockz -esphome/components/pn532_i2c/* @OttoWinter @jesserockz -esphome/components/pn532_spi/* @OttoWinter @jesserockz +esphome/components/pn532/* @jesserockz @OttoWinter +esphome/components/pn532_i2c/* @jesserockz @OttoWinter +esphome/components/pn532_spi/* @jesserockz @OttoWinter esphome/components/pn7150/* @jesserockz @kbx81 esphome/components/pn7150_i2c/* @jesserockz @kbx81 esphome/components/pn7160/* @jesserockz @kbx81 @@ -356,7 +368,7 @@ esphome/components/pn7160_spi/* @jesserockz @kbx81 esphome/components/power_supply/* @esphome/core esphome/components/preferences/* @esphome/core esphome/components/psram/* @esphome/core -esphome/components/pulse_meter/* @TrentHouliston @cstaahl @stevebaxter +esphome/components/pulse_meter/* @cstaahl @stevebaxter @TrentHouliston esphome/components/pvvx_mithermometer/* @pasiz esphome/components/pylontech/* @functionpointer esphome/components/qmp6988/* @andrewpc @@ -378,6 +390,7 @@ esphome/components/rp2040_pwm/* @jesserockz esphome/components/rpi_dpi_rgb/* @clydebarrow esphome/components/rtl87xx/* @kuba2k2 esphome/components/rtttl/* @glmnet +esphome/components/runtime_stats/* @bdraco esphome/components/safe_mode/* @jsuanet @kbx81 @paulmonigatti esphome/components/scd4x/* @martgras @sjtrny esphome/components/script/* @esphome/core @@ -396,7 +409,8 @@ esphome/components/sensirion_common/* @martgras esphome/components/sensor/* @esphome/core esphome/components/sfa30/* @ghsensdev esphome/components/sgp40/* @SenexCrenshaw -esphome/components/sgp4x/* @SenexCrenshaw @martgras +esphome/components/sgp4x/* @martgras @SenexCrenshaw +esphome/components/sha256/* @esphome/core esphome/components/shelly_dimmer/* @edge90 @rnauber esphome/components/sht3xd/* @mrtoy-me esphome/components/sht4x/* @sjtrny @@ -418,6 +432,7 @@ esphome/components/speaker/media_player/* @kahrendt @synesthesiam esphome/components/spi/* @clydebarrow @esphome/core esphome/components/spi_device/* @clydebarrow esphome/components/spi_led_strip/* @clydebarrow +esphome/components/split_buffer/* @jesserockz esphome/components/sprinkler/* @kbx81 esphome/components/sps30/* @martgras esphome/components/ssd1322_base/* @kbx81 @@ -461,13 +476,13 @@ esphome/components/template/event/* @nohat esphome/components/template/fan/* @ssieb esphome/components/text/* @mauritskorse esphome/components/thermostat/* @kbx81 -esphome/components/time/* @OttoWinter +esphome/components/time/* @esphome/core esphome/components/tlc5947/* @rnauber esphome/components/tlc5971/* @IJIJI esphome/components/tm1621/* @Philippe12 esphome/components/tm1637/* @glmnet esphome/components/tm1638/* @skykingjwc -esphome/components/tm1651/* @freekode +esphome/components/tm1651/* @mrtoy-me esphome/components/tmp102/* @timsavage esphome/components/tmp1075/* @sybrenstuvel esphome/components/tmp117/* @Azimath @@ -505,7 +520,7 @@ esphome/components/wake_on_lan/* @clydebarrow @willwill2will54 esphome/components/watchdog/* @oarcher esphome/components/waveshare_epaper/* @clydebarrow esphome/components/web_server/ota/* @esphome/core -esphome/components/web_server_base/* @OttoWinter +esphome/components/web_server_base/* @esphome/core esphome/components/web_server_idf/* @dentra esphome/components/weikai/* @DrCoolZic esphome/components/weikai_i2c/* @DrCoolZic @@ -523,6 +538,7 @@ esphome/components/wk2204_spi/* @DrCoolZic esphome/components/wk2212_i2c/* @DrCoolZic esphome/components/wk2212_spi/* @DrCoolZic esphome/components/wl_134/* @hobbypunk90 +esphome/components/wts01/* @alepee esphome/components/x9c/* @EtienneMD esphome/components/xgzp68xx/* @gcormier esphome/components/xiaomi_hhccjcy10/* @fariouche @@ -535,5 +551,7 @@ esphome/components/xiaomi_xmwsdj04mmc/* @medusalix esphome/components/xl9535/* @mreditor97 esphome/components/xpt2046/touchscreen/* @nielsnl68 @numo68 esphome/components/xxtea/* @clydebarrow +esphome/components/zephyr/* @tomaszduda23 esphome/components/zhlt01/* @cfeenstra1024 esphome/components/zio_ultrasonic/* @kahrendt +esphome/components/zwave_proxy/* @kbx81 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 7be7bdac2c..303b548310 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -7,7 +7,7 @@ project and be sure to join us on [Discord](https://discord.gg/KhAMKrd). **See also:** -[Documentation](https://esphome.io) -- [Issues](https://github.com/esphome/issues/issues) -- [Feature requests](https://github.com/esphome/feature-requests/issues) +[Documentation](https://esphome.io) -- [Issues](https://github.com/esphome/esphome/issues) -- [Feature requests](https://github.com/orgs/esphome/discussions) --- diff --git a/Doxyfile b/Doxyfile index 1f5ac5aa1b..034fa3fa37 100644 --- a/Doxyfile +++ b/Doxyfile @@ -48,7 +48,7 @@ PROJECT_NAME = ESPHome # could be handy for archiving the generated documentation or if some version # control system is used. -PROJECT_NUMBER = 2025.8.0-dev +PROJECT_NUMBER = 2025.11.0-dev # Using the PROJECT_BRIEF tag one can provide an optional one line description # for a project that appears at the top of each page and should give viewer a diff --git a/GEMINI.md b/GEMINI.md new file mode 120000 index 0000000000..49e811ff05 --- /dev/null +++ b/GEMINI.md @@ -0,0 +1 @@ +.ai/instructions.md \ No newline at end of file diff --git a/README.md b/README.md index 4f527870b8..0439b1bc06 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ --- -[Documentation](https://esphome.io) -- [Issues](https://github.com/esphome/issues/issues) -- [Feature requests](https://github.com/esphome/feature-requests/issues) +[Documentation](https://esphome.io) -- [Issues](https://github.com/esphome/esphome/issues) -- [Feature requests](https://github.com/orgs/esphome/discussions) --- diff --git a/docker/build.py b/docker/build.py index 921adac7ab..4d093cf88d 100755 --- a/docker/build.py +++ b/docker/build.py @@ -90,7 +90,7 @@ def main(): def run_command(*cmd, ignore_error: bool = False): print(f"$ {shlex.join(list(cmd))}") if not args.dry_run: - rc = subprocess.call(list(cmd)) + rc = subprocess.call(list(cmd), close_fds=False) if rc != 0 and not ignore_error: print("Command failed") sys.exit(1) diff --git a/esphome/__main__.py b/esphome/__main__.py index d8a79c018a..d9bdfb175b 100644 --- a/esphome/__main__.py +++ b/esphome/__main__.py @@ -2,20 +2,27 @@ import argparse from datetime import datetime import functools +import getpass import importlib import logging import os +from pathlib import Path import re import sys import time +from typing import Protocol import argcomplete +# Note: Do not import modules from esphome.components here, as this would +# cause them to be loaded before external components are processed, resulting +# in the built-in version being used instead of the external component one. from esphome import const, writer, yaml_util import esphome.codegen as cg from esphome.config import iter_component_configs, read_config, strip_default_ids from esphome.const import ( ALLOWED_NAME_CHARS, + CONF_API, CONF_BAUD_RATE, CONF_BROKER, CONF_DEASSERT_RTS_DTR, @@ -34,14 +41,17 @@ from esphome.const import ( CONF_PORT, CONF_SUBSTITUTIONS, CONF_TOPIC, + ENV_NOGITIGNORE, PLATFORM_ESP32, PLATFORM_ESP8266, PLATFORM_RP2040, SECRETS_FILES, ) from esphome.core import CORE, EsphomeError, coroutine +from esphome.enum import StrEnum from esphome.helpers import get_bool_env, indent, is_ip_address from esphome.log import AnsiFore, color, setup_log +from esphome.types import ConfigType from esphome.util import ( get_serial_ports, list_yaml_files, @@ -53,6 +63,23 @@ from esphome.util import ( _LOGGER = logging.getLogger(__name__) +class ArgsProtocol(Protocol): + device: list[str] | None + reset: bool + username: str | None + password: str | None + client_id: str | None + topic: str | None + file: str | None + no_logs: bool + only_generate: bool + show_secrets: bool + dashboard: bool + configuration: str + name: str + upload_speed: str | None + + def choose_prompt(options, purpose: str = None): if not options: raise EsphomeError( @@ -85,53 +112,249 @@ def choose_prompt(options, purpose: str = None): return options[opt - 1][1] +class Purpose(StrEnum): + UPLOADING = "uploading" + LOGGING = "logging" + + +class PortType(StrEnum): + SERIAL = "SERIAL" + NETWORK = "NETWORK" + MQTT = "MQTT" + MQTTIP = "MQTTIP" + + +# Magic MQTT port types that require special handling +_MQTT_PORT_TYPES = frozenset({PortType.MQTT, PortType.MQTTIP}) + + +def _resolve_with_cache(address: str, purpose: Purpose) -> list[str]: + """Resolve an address using cache if available, otherwise return the address itself.""" + if CORE.address_cache and (cached := CORE.address_cache.get_addresses(address)): + _LOGGER.debug("Using cached addresses for %s: %s", purpose.value, cached) + return cached + return [address] + + def choose_upload_log_host( - default, check_default, show_ota, show_mqtt, show_api, purpose: str = None -): - options = [] - for port in get_serial_ports(): - options.append((f"{port.path} ({port.description})", port.path)) - if default == "SERIAL": - return choose_prompt(options, purpose=purpose) - if (show_ota and "ota" in CORE.config) or (show_api and "api" in CORE.config): - options.append((f"Over The Air ({CORE.address})", CORE.address)) - if default == "OTA": - return CORE.address - if ( - show_mqtt - and (mqtt_config := CORE.config.get(CONF_MQTT)) - and mqtt_logging_enabled(mqtt_config) - ): - options.append((f"MQTT ({mqtt_config[CONF_BROKER]})", "MQTT")) - if default == "OTA": - return "MQTT" - if default is not None: - return default + default: list[str] | str | None, + check_default: str | None, + purpose: Purpose, +) -> list[str]: + # Convert to list for uniform handling + defaults = [default] if isinstance(default, str) else default or [] + + # If devices specified, resolve them + if defaults: + resolved: list[str] = [] + for device in defaults: + if device == "SERIAL": + serial_ports = get_serial_ports() + if not serial_ports: + _LOGGER.warning("No serial ports found, skipping SERIAL device") + continue + options = [ + (f"{port.path} ({port.description})", port.path) + for port in serial_ports + ] + resolved.append(choose_prompt(options, purpose=purpose)) + elif device == "OTA": + # ensure IP adresses are used first + if is_ip_address(CORE.address) and ( + (purpose == Purpose.LOGGING and has_api()) + or (purpose == Purpose.UPLOADING and has_ota()) + ): + resolved.extend(_resolve_with_cache(CORE.address, purpose)) + + if purpose == Purpose.LOGGING: + if has_api() and has_mqtt_ip_lookup(): + resolved.append("MQTTIP") + + if has_mqtt_logging(): + resolved.append("MQTT") + + if has_api() and has_non_ip_address(): + resolved.extend(_resolve_with_cache(CORE.address, purpose)) + + elif purpose == Purpose.UPLOADING: + if has_ota() and has_mqtt_ip_lookup(): + resolved.append("MQTTIP") + + if has_ota() and has_non_ip_address(): + resolved.extend(_resolve_with_cache(CORE.address, purpose)) + else: + resolved.append(device) + if not resolved: + _LOGGER.error("All specified devices: %s could not be resolved.", defaults) + return resolved + + # No devices specified, show interactive chooser + options = [ + (f"{port.path} ({port.description})", port.path) for port in get_serial_ports() + ] + + if purpose == Purpose.LOGGING: + if has_mqtt_logging(): + mqtt_config = CORE.config[CONF_MQTT] + options.append((f"MQTT ({mqtt_config[CONF_BROKER]})", "MQTT")) + + if has_api(): + if has_resolvable_address(): + options.append((f"Over The Air ({CORE.address})", CORE.address)) + if has_mqtt_ip_lookup(): + options.append(("Over The Air (MQTT IP lookup)", "MQTTIP")) + + elif purpose == Purpose.UPLOADING and has_ota(): + if has_resolvable_address(): + options.append((f"Over The Air ({CORE.address})", CORE.address)) + if has_mqtt_ip_lookup(): + options.append(("Over The Air (MQTT IP lookup)", "MQTTIP")) + if check_default is not None and check_default in [opt[1] for opt in options]: - return check_default - return choose_prompt(options, purpose=purpose) + return [check_default] + return [choose_prompt(options, purpose=purpose)] -def mqtt_logging_enabled(mqtt_config): +def has_mqtt_logging() -> bool: + """Check if MQTT logging is available.""" + if CONF_MQTT not in CORE.config: + return False + + mqtt_config = CORE.config[CONF_MQTT] + + # enabled by default + if CONF_LOG_TOPIC not in mqtt_config: + return True + log_topic = mqtt_config[CONF_LOG_TOPIC] if log_topic is None: return False + if CONF_TOPIC not in log_topic: return False - if log_topic.get(CONF_LEVEL, None) == "NONE": + + return log_topic.get(CONF_LEVEL, None) != "NONE" + + +def has_mqtt() -> bool: + """Check if MQTT is available.""" + return CONF_MQTT in CORE.config + + +def has_api() -> bool: + """Check if API is available.""" + return CONF_API in CORE.config + + +def has_ota() -> bool: + """Check if OTA is available.""" + return CONF_OTA in CORE.config + + +def has_mqtt_ip_lookup() -> bool: + """Check if MQTT is available and IP lookup is supported.""" + from esphome.components.mqtt import CONF_DISCOVER_IP + + if CONF_MQTT not in CORE.config: return False - return True + # Default Enabled + if CONF_DISCOVER_IP not in CORE.config[CONF_MQTT]: + return True + return CORE.config[CONF_MQTT][CONF_DISCOVER_IP] -def get_port_type(port): +def has_mdns() -> bool: + """Check if MDNS is available.""" + return CONF_MDNS not in CORE.config or not CORE.config[CONF_MDNS][CONF_DISABLED] + + +def has_non_ip_address() -> bool: + """Check if CORE.address is set and is not an IP address.""" + return CORE.address is not None and not is_ip_address(CORE.address) + + +def has_ip_address() -> bool: + """Check if CORE.address is a valid IP address.""" + return CORE.address is not None and is_ip_address(CORE.address) + + +def has_resolvable_address() -> bool: + """Check if CORE.address is resolvable (via mDNS, DNS, or is an IP address).""" + # Any address (IP, mDNS hostname, or regular DNS hostname) is resolvable + # The resolve_ip_address() function in helpers.py handles all types via AsyncResolver + return CORE.address is not None + + +def mqtt_get_ip(config: ConfigType, username: str, password: str, client_id: str): + from esphome import mqtt + + return mqtt.get_esphome_device_ip(config, username, password, client_id) + + +def _resolve_network_devices( + devices: list[str], config: ConfigType, args: ArgsProtocol +) -> list[str]: + """Resolve device list, converting MQTT magic strings to actual IP addresses. + + This function filters the devices list to: + - Replace MQTT/MQTTIP magic strings with actual IP addresses via MQTT lookup + - Deduplicate addresses while preserving order + - Only resolve MQTT once even if multiple MQTT strings are present + - If MQTT resolution fails, log a warning and continue with other devices + + Args: + devices: List of device identifiers (IPs, hostnames, or magic strings) + config: ESPHome configuration + args: Command-line arguments containing MQTT credentials + + Returns: + List of network addresses suitable for connection attempts + """ + network_devices: list[str] = [] + mqtt_resolved: bool = False + + for device in devices: + port_type = get_port_type(device) + if port_type in _MQTT_PORT_TYPES: + # Only resolve MQTT once, even if multiple MQTT entries + if not mqtt_resolved: + try: + mqtt_ips = mqtt_get_ip( + config, args.username, args.password, args.client_id + ) + network_devices.extend(mqtt_ips) + except EsphomeError as err: + _LOGGER.warning( + "MQTT IP discovery failed (%s), will try other devices if available", + err, + ) + mqtt_resolved = True + elif device not in network_devices: + # Regular network address or IP - add if not already present + network_devices.append(device) + + return network_devices + + +def get_port_type(port: str) -> PortType: + """Determine the type of port/device identifier. + + Returns: + PortType.SERIAL for serial ports (/dev/ttyUSB0, COM1, etc.) + PortType.MQTT for MQTT logging + PortType.MQTTIP for MQTT IP lookup + PortType.NETWORK for IP addresses, hostnames, or mDNS names + """ if port.startswith("/") or port.startswith("COM"): - return "SERIAL" + return PortType.SERIAL if port == "MQTT": - return "MQTT" - return "NETWORK" + return PortType.MQTT + if port == "MQTTIP": + return PortType.MQTTIP + return PortType.NETWORK -def run_miniterm(config, port, args): +def run_miniterm(config: ConfigType, port: str, args) -> int: from aioesphomeapi import LogParser import serial @@ -173,7 +396,9 @@ def run_miniterm(config, port, args): .replace(b"\n", b"") .decode("utf8", "backslashreplace") ) - time_str = datetime.now().time().strftime("[%H:%M:%S]") + time_ = datetime.now() + nanoseconds = time_.microsecond // 1000 + time_str = f"[{time_.hour:02}:{time_.minute:02}:{time_.second:02}.{nanoseconds:03}]" safe_print(parser.parse_line(line, time_str)) backtrace_state = platformio_api.process_stacktrace( @@ -208,12 +433,15 @@ def wrap_to_code(name, comp): return wrapped -def write_cpp(config): +def write_cpp(config: ConfigType) -> int: + if not get_bool_env(ENV_NOGITIGNORE): + writer.write_gitignore() + generate_cpp_contents(config) return write_cpp_file() -def generate_cpp_contents(config): +def generate_cpp_contents(config: ConfigType) -> None: _LOGGER.info("Generating C++ source...") for name, component, conf in iter_component_configs(CORE.config): @@ -224,15 +452,18 @@ def generate_cpp_contents(config): CORE.flush_tasks() -def write_cpp_file(): - writer.write_platformio_project() - +def write_cpp_file() -> int: code_s = indent(CORE.cpp_main_section) writer.write_cpp(code_s) + + from esphome.build_gen import platformio + + platformio.write_project() + return 0 -def compile_program(args, config): +def compile_program(args: ArgsProtocol, config: ConfigType) -> int: from esphome import platformio_api _LOGGER.info("Compiling app...") @@ -243,7 +474,9 @@ def compile_program(args, config): return 0 if idedata is not None else 1 -def upload_using_esptool(config, port, file, speed): +def upload_using_esptool( + config: ConfigType, port: str, file: str, speed: int +) -> str | int: from esphome import platformio_api first_baudrate = speed or config[CONF_ESPHOME][CONF_PLATFORMIO_OPTIONS].get( @@ -271,24 +504,24 @@ def upload_using_esptool(config, port, file, speed): def run_esptool(baud_rate): cmd = [ - "esptool.py", + "esptool", "--before", - "default_reset", + "default-reset", "--after", - "hard_reset", + "hard-reset", "--baud", str(baud_rate), "--port", port, "--chip", mcu, - "write_flash", + "write-flash", "-z", - "--flash_size", + "--flash-size", "detect", ] for img in flash_images: - cmd += [img.offset, img.path] + cmd += [img.offset, str(img.path)] if os.environ.get("ESPHOME_USE_SUBPROCESS") is None: import esptool @@ -308,7 +541,7 @@ def upload_using_esptool(config, port, file, speed): return run_esptool(115200) -def upload_using_platformio(config, port): +def upload_using_platformio(config: ConfigType, port: str): from esphome import platformio_api upload_args = ["-t", "upload", "-t", "nobuild"] @@ -317,8 +550,8 @@ def upload_using_platformio(config, port): return platformio_api.run_platformio_cli_run(config, CORE.verbose, *upload_args) -def check_permissions(port): - if os.name == "posix" and get_port_type(port) == "SERIAL": +def check_permissions(port: str): + if os.name == "posix" and get_port_type(port) == PortType.SERIAL: # Check if we can open selected serial port if not os.access(port, os.F_OK): raise EsphomeError( @@ -330,32 +563,34 @@ def check_permissions(port): raise EsphomeError( "You do not have read or write permission on the selected serial port. " "To resolve this issue, you can add your user to the dialout group " - f"by running the following command: sudo usermod -a -G dialout {os.getlogin()}. " + f"by running the following command: sudo usermod -a -G dialout {getpass.getuser()}. " "You will need to log out & back in or reboot to activate the new group access." ) -def upload_program(config, args, host): +def upload_program( + config: ConfigType, args: ArgsProtocol, devices: list[str] +) -> tuple[int, str | None]: + host = devices[0] try: module = importlib.import_module("esphome.components." + CORE.target_platform) if getattr(module, "upload_program")(config, args, host): - return 0 + return 0, host except AttributeError: pass - if get_port_type(host) == "SERIAL": + if get_port_type(host) == PortType.SERIAL: check_permissions(host) + + exit_code = 1 if CORE.target_platform in (PLATFORM_ESP32, PLATFORM_ESP8266): file = getattr(args, "file", None) - return upload_using_esptool(config, host, file, args.upload_speed) + exit_code = upload_using_esptool(config, host, file, args.upload_speed) + elif CORE.target_platform == PLATFORM_RP2040 or CORE.is_libretiny: + exit_code = upload_using_platformio(config, host) + # else: Unknown target platform, exit_code remains 1 - if CORE.target_platform in (PLATFORM_RP2040): - return upload_using_platformio(config, args.device) - - if CORE.is_libretiny: - return upload_using_platformio(config, host) - - return 1 # Unknown target platform + return exit_code, host if exit_code == 0 else None ota_conf = {} for ota_item in config.get(CONF_OTA, []): @@ -371,46 +606,46 @@ def upload_program(config, args, host): from esphome import espota2 remote_port = int(ota_conf[CONF_PORT]) - password = ota_conf.get(CONF_PASSWORD, "") - - if ( - CONF_MQTT in config # pylint: disable=too-many-boolean-expressions - and (not args.device or args.device in ("MQTT", "OTA")) - and ( - ((config[CONF_MDNS][CONF_DISABLED]) and not is_ip_address(CORE.address)) - or get_port_type(host) == "MQTT" - ) - ): - from esphome import mqtt - - host = mqtt.get_esphome_device_ip( - config, args.username, args.password, args.client_id - ) - + password = ota_conf.get(CONF_PASSWORD) if getattr(args, "file", None) is not None: - return espota2.run_ota(host, remote_port, password, args.file) + binary = Path(args.file) + else: + binary = CORE.firmware_bin - return espota2.run_ota(host, remote_port, password, CORE.firmware_bin) + # Resolve MQTT magic strings to actual IP addresses + network_devices = _resolve_network_devices(devices, config, args) + + return espota2.run_ota(network_devices, remote_port, password, binary) -def show_logs(config, args, port): +def show_logs(config: ConfigType, args: ArgsProtocol, devices: list[str]) -> int | None: + try: + module = importlib.import_module("esphome.components." + CORE.target_platform) + if getattr(module, "show_logs")(config, args, devices): + return 0 + except AttributeError: + pass + if "logger" not in config: raise EsphomeError("Logger is not configured!") - if get_port_type(port) == "SERIAL": + + port = devices[0] + port_type = get_port_type(port) + + if port_type == PortType.SERIAL: check_permissions(port) return run_miniterm(config, port, args) - if get_port_type(port) == "NETWORK" and "api" in config: - if config[CONF_MDNS][CONF_DISABLED] and CONF_MQTT in config: - from esphome import mqtt - - port = mqtt.get_esphome_device_ip( - config, args.username, args.password, args.client_id - )[0] + # Check if we should use API for logging + # Resolve MQTT magic strings to actual IP addresses + if has_api() and ( + network_devices := _resolve_network_devices(devices, config, args) + ): from esphome.components.api.client import run_logs - return run_logs(config, port) - if get_port_type(port) == "MQTT" and "mqtt" in config: + return run_logs(config, network_devices) + + if port_type in (PortType.NETWORK, PortType.MQTT) and has_mqtt_logging(): from esphome import mqtt return mqtt.show_logs( @@ -420,7 +655,7 @@ def show_logs(config, args, port): raise EsphomeError("No remote or local logging method configured (api/mqtt/logger)") -def clean_mqtt(config, args): +def clean_mqtt(config: ConfigType, args: ArgsProtocol) -> int | None: from esphome import mqtt return mqtt.clear_topic( @@ -428,13 +663,13 @@ def clean_mqtt(config, args): ) -def command_wizard(args): +def command_wizard(args: ArgsProtocol) -> int | None: from esphome import wizard - return wizard.wizard(args.configuration) + return wizard.wizard(Path(args.configuration)) -def command_config(args, config): +def command_config(args: ArgsProtocol, config: ConfigType) -> int | None: if not CORE.verbose: config = strip_default_ids(config) output = yaml_util.dump(config, args.show_secrets) @@ -449,7 +684,7 @@ def command_config(args, config): return 0 -def command_vscode(args): +def command_vscode(args: ArgsProtocol) -> int | None: from esphome import vscode logging.disable(logging.INFO) @@ -457,7 +692,7 @@ def command_vscode(args): vscode.read_config(args) -def command_compile(args, config): +def command_compile(args: ArgsProtocol, config: ConfigType) -> int | None: exit_code = write_cpp(config) if exit_code != 0: return exit_code @@ -471,23 +706,23 @@ def command_compile(args, config): return 0 -def command_upload(args, config): - port = choose_upload_log_host( +def command_upload(args: ArgsProtocol, config: ConfigType) -> int | None: + # Get devices, resolving special identifiers like OTA + devices = choose_upload_log_host( default=args.device, check_default=None, - show_ota=True, - show_mqtt=False, - show_api=False, - purpose="uploading", + purpose=Purpose.UPLOADING, ) - exit_code = upload_program(config, args, port) - if exit_code != 0: - return exit_code - _LOGGER.info("Successfully uploaded program.") - return 0 + + exit_code, _ = upload_program(config, args, devices) + if exit_code == 0: + _LOGGER.info("Successfully uploaded program.") + else: + _LOGGER.warning("Failed to upload to %s", devices) + return exit_code -def command_discover(args, config): +def command_discover(args: ArgsProtocol, config: ConfigType) -> int | None: if "mqtt" in config: from esphome import mqtt @@ -496,19 +731,17 @@ def command_discover(args, config): raise EsphomeError("No discover method configured (mqtt)") -def command_logs(args, config): - port = choose_upload_log_host( +def command_logs(args: ArgsProtocol, config: ConfigType) -> int | None: + # Get devices, resolving special identifiers like OTA + devices = choose_upload_log_host( default=args.device, check_default=None, - show_ota=False, - show_mqtt=True, - show_api=True, - purpose="logging", + purpose=Purpose.LOGGING, ) - return show_logs(config, args, port) + return show_logs(config, args, devices) -def command_run(args, config): +def command_run(args: ArgsProtocol, config: ConfigType) -> int | None: exit_code = write_cpp(config) if exit_code != 0: return exit_code @@ -525,47 +758,58 @@ def command_run(args, config): program_path = idedata.raw["prog_path"] return run_external_process(program_path) - port = choose_upload_log_host( + # Get devices, resolving special identifiers like OTA + devices = choose_upload_log_host( default=args.device, check_default=None, - show_ota=True, - show_mqtt=False, - show_api=True, - purpose="uploading", + purpose=Purpose.UPLOADING, ) - exit_code = upload_program(config, args, port) - if exit_code != 0: + + exit_code, successful_device = upload_program(config, args, devices) + if exit_code == 0: + _LOGGER.info("Successfully uploaded program.") + else: + _LOGGER.warning("Failed to upload to %s", devices) return exit_code - _LOGGER.info("Successfully uploaded program.") + if args.no_logs: return 0 - port = choose_upload_log_host( - default=args.device, - check_default=port, - show_ota=False, - show_mqtt=True, - show_api=True, - purpose="logging", + + # For logs, prefer the device we successfully uploaded to + devices = choose_upload_log_host( + default=successful_device, + check_default=successful_device, + purpose=Purpose.LOGGING, ) - return show_logs(config, args, port) + return show_logs(config, args, devices) -def command_clean_mqtt(args, config): +def command_clean_mqtt(args: ArgsProtocol, config: ConfigType) -> int | None: return clean_mqtt(config, args) -def command_mqtt_fingerprint(args, config): +def command_clean_all(args: ArgsProtocol) -> int | None: + try: + writer.clean_all(args.configuration) + except OSError as err: + _LOGGER.error("Error cleaning all files: %s", err) + return 1 + _LOGGER.info("Done!") + return 0 + + +def command_mqtt_fingerprint(args: ArgsProtocol, config: ConfigType) -> int | None: from esphome import mqtt return mqtt.get_fingerprint(config) -def command_version(args): +def command_version(args: ArgsProtocol) -> int | None: safe_print(f"Version: {const.__version__}") return 0 -def command_clean(args, config): +def command_clean(args: ArgsProtocol, config: ConfigType) -> int | None: try: writer.clean_build() except OSError as err: @@ -575,13 +819,13 @@ def command_clean(args, config): return 0 -def command_dashboard(args): +def command_dashboard(args: ArgsProtocol) -> int | None: from esphome.dashboard import dashboard return dashboard.start_dashboard(args) -def command_update_all(args): +def command_update_all(args: ArgsProtocol) -> int | None: import click success = {} @@ -595,7 +839,7 @@ def command_update_all(args): safe_print(f"{half_line}{middle_text}{half_line}") for f in files: - safe_print(f"Updating {color(AnsiFore.CYAN, f)}") + safe_print(f"Updating {color(AnsiFore.CYAN, str(f))}") safe_print("-" * twidth) safe_print() if CORE.dashboard: @@ -607,10 +851,10 @@ def command_update_all(args): "esphome", "run", f, "--no-logs", "--device", "OTA" ) if rc == 0: - print_bar(f"[{color(AnsiFore.BOLD_GREEN, 'SUCCESS')}] {f}") + print_bar(f"[{color(AnsiFore.BOLD_GREEN, 'SUCCESS')}] {str(f)}") success[f] = True else: - print_bar(f"[{color(AnsiFore.BOLD_RED, 'ERROR')}] {f}") + print_bar(f"[{color(AnsiFore.BOLD_RED, 'ERROR')}] {str(f)}") success[f] = False safe_print() @@ -621,14 +865,14 @@ def command_update_all(args): failed = 0 for f in files: if success[f]: - safe_print(f" - {f}: {color(AnsiFore.GREEN, 'SUCCESS')}") + safe_print(f" - {str(f)}: {color(AnsiFore.GREEN, 'SUCCESS')}") else: - safe_print(f" - {f}: {color(AnsiFore.BOLD_RED, 'FAILED')}") + safe_print(f" - {str(f)}: {color(AnsiFore.BOLD_RED, 'FAILED')}") failed += 1 return failed -def command_idedata(args, config): +def command_idedata(args: ArgsProtocol, config: ConfigType) -> int: import json from esphome import platformio_api @@ -644,8 +888,9 @@ def command_idedata(args, config): return 0 -def command_rename(args, config): - for c in args.name: +def command_rename(args: ArgsProtocol, config: ConfigType) -> int | None: + new_name = args.name + for c in new_name: if c not in ALLOWED_NAME_CHARS: print( color( @@ -656,8 +901,7 @@ def command_rename(args, config): ) return 1 # Load existing yaml file - with open(CORE.config_path, mode="r+", encoding="utf-8") as raw_file: - raw_contents = raw_file.read() + raw_contents = CORE.config_path.read_text(encoding="utf-8") yaml = yaml_util.load_yaml(CORE.config_path) if CONF_ESPHOME not in yaml or CONF_NAME not in yaml[CONF_ESPHOME]: @@ -672,7 +916,7 @@ def command_rename(args, config): if match is None: new_raw = re.sub( rf"name:\s+[\"']?{old_name}[\"']?", - f'name: "{args.name}"', + f'name: "{new_name}"', raw_contents, ) else: @@ -692,29 +936,28 @@ def command_rename(args, config): new_raw = re.sub( rf"^(\s+{match.group(1)}):\s+[\"']?{old_name}[\"']?", - f'\\1: "{args.name}"', + f'\\1: "{new_name}"', raw_contents, flags=re.MULTILINE, ) - new_path = os.path.join(CORE.config_dir, args.name + ".yaml") + new_path: Path = CORE.config_dir / (new_name + ".yaml") print( - f"Updating {color(AnsiFore.CYAN, CORE.config_path)} to {color(AnsiFore.CYAN, new_path)}" + f"Updating {color(AnsiFore.CYAN, str(CORE.config_path))} to {color(AnsiFore.CYAN, str(new_path))}" ) print() - with open(new_path, mode="w", encoding="utf-8") as new_file: - new_file.write(new_raw) + new_path.write_text(new_raw, encoding="utf-8") - rc = run_external_process("esphome", "config", new_path) + rc = run_external_process("esphome", "config", str(new_path)) if rc != 0: print(color(AnsiFore.BOLD_RED, "Rename failed. Reverting changes.")) - os.remove(new_path) + new_path.unlink() return 1 cli_args = [ "run", - new_path, + str(new_path), "--no-logs", "--device", CORE.address, @@ -728,11 +971,11 @@ def command_rename(args, config): except KeyboardInterrupt: rc = 1 if rc != 0: - os.remove(new_path) + new_path.unlink() return 1 if CORE.config_path != new_path: - os.remove(CORE.config_path) + CORE.config_path.unlink() print(color(AnsiFore.BOLD_GREEN, "SUCCESS")) print() @@ -745,6 +988,7 @@ PRE_CONFIG_ACTIONS = { "dashboard": command_dashboard, "vscode": command_vscode, "update-all": command_update_all, + "clean-all": command_clean_all, } POST_CONFIG_ACTIONS = { @@ -753,14 +997,20 @@ POST_CONFIG_ACTIONS = { "upload": command_upload, "logs": command_logs, "run": command_run, + "clean": command_clean, "clean-mqtt": command_clean_mqtt, "mqtt-fingerprint": command_mqtt_fingerprint, - "clean": command_clean, "idedata": command_idedata, "rename": command_rename, "discover": command_discover, } +SIMPLE_CONFIG_ACTIONS = [ + "clean", + "clean-mqtt", + "config", +] + def parse_args(argv): options_parser = argparse.ArgumentParser(add_help=False) @@ -793,6 +1043,24 @@ def parse_args(argv): help="Add a substitution", metavar=("key", "value"), ) + options_parser.add_argument( + "--mdns-address-cache", + help="mDNS address cache mapping in format 'hostname=ip1,ip2'", + action="append", + default=[], + ) + options_parser.add_argument( + "--dns-address-cache", + help="DNS address cache mapping in format 'hostname=ip1,ip2'", + action="append", + default=[], + ) + options_parser.add_argument( + "--testing-mode", + help="Enable testing mode (disables validation checks for grouped component testing)", + action="store_true", + default=False, + ) parser = argparse.ArgumentParser( description=f"ESPHome {const.__version__}", parents=[options_parser] @@ -848,7 +1116,8 @@ def parse_args(argv): ) parser_upload.add_argument( "--device", - help="Manually specify the serial port/address to use, for example /dev/ttyUSB0.", + action="append", + help="Manually specify the serial port/address to use, for example /dev/ttyUSB0. Can be specified multiple times for fallback addresses.", ) parser_upload.add_argument( "--upload_speed", @@ -870,7 +1139,8 @@ def parse_args(argv): ) parser_logs.add_argument( "--device", - help="Manually specify the serial port/address to use, for example /dev/ttyUSB0.", + action="append", + help="Manually specify the serial port/address to use, for example /dev/ttyUSB0. Can be specified multiple times for fallback addresses.", ) parser_logs.add_argument( "--reset", @@ -899,7 +1169,8 @@ def parse_args(argv): ) parser_run.add_argument( "--device", - help="Manually specify the serial port/address to use, for example /dev/ttyUSB0.", + action="append", + help="Manually specify the serial port/address to use, for example /dev/ttyUSB0. Can be specified multiple times for fallback addresses.", ) parser_run.add_argument( "--upload_speed", @@ -947,6 +1218,13 @@ def parse_args(argv): "configuration", help="Your YAML configuration file(s).", nargs="+" ) + parser_clean_all = subparsers.add_parser( + "clean-all", help="Clean all build and platform files." + ) + parser_clean_all.add_argument( + "configuration", help="Your YAML configuration directory.", nargs="*" + ) + parser_dashboard = subparsers.add_parser( "dashboard", help="Create a simple web server for a dashboard." ) @@ -993,7 +1271,7 @@ def parse_args(argv): parser_update = subparsers.add_parser("update-all") parser_update.add_argument( - "configuration", help="Your YAML configuration file directories.", nargs="+" + "configuration", help="Your YAML configuration file or directory.", nargs="+" ) parser_idedata = subparsers.add_parser("idedata") @@ -1026,13 +1304,27 @@ def parse_args(argv): arguments = argv[1:] argcomplete.autocomplete(parser) + + if len(arguments) > 0 and arguments[0] in SIMPLE_CONFIG_ACTIONS: + args, unknown_args = parser.parse_known_args(arguments) + if unknown_args: + _LOGGER.warning("Ignored unrecognized arguments: %s", unknown_args) + return args + return parser.parse_args(arguments) def run_esphome(argv): + from esphome.address_cache import AddressCache + args = parse_args(argv) CORE.dashboard = args.dashboard + CORE.testing_mode = args.testing_mode + # Create address cache from command-line arguments + CORE.address_cache = AddressCache.from_cli_args( + args.mdns_address_cache, args.dns_address_cache + ) # Override log level if verbose is set if args.verbose: args.log_level = "DEBUG" @@ -1055,14 +1347,20 @@ def run_esphome(argv): _LOGGER.info("ESPHome %s", const.__version__) for conf_path in args.configuration: - if any(os.path.basename(conf_path) == x for x in SECRETS_FILES): + conf_path = Path(conf_path) + if any(conf_path.name == x for x in SECRETS_FILES): _LOGGER.warning("Skipping secrets file %s", conf_path) continue CORE.config_path = conf_path CORE.dashboard = args.dashboard - config = read_config(dict(args.substitution) if args.substitution else {}) + # For logs command, skip updating external components + skip_external = args.command == "logs" + config = read_config( + dict(args.substitution) if args.substitution else {}, + skip_external_update=skip_external, + ) if config is None: return 2 CORE.config = config diff --git a/esphome/address_cache.py b/esphome/address_cache.py new file mode 100644 index 0000000000..7c20be90f0 --- /dev/null +++ b/esphome/address_cache.py @@ -0,0 +1,142 @@ +"""Address cache for DNS and mDNS lookups.""" + +from __future__ import annotations + +import logging +from typing import TYPE_CHECKING + +if TYPE_CHECKING: + from collections.abc import Iterable + +_LOGGER = logging.getLogger(__name__) + + +def normalize_hostname(hostname: str) -> str: + """Normalize hostname for cache lookups. + + Removes trailing dots and converts to lowercase. + """ + return hostname.rstrip(".").lower() + + +class AddressCache: + """Cache for DNS and mDNS address lookups. + + This cache stores pre-resolved addresses from command-line arguments + to avoid slow DNS/mDNS lookups during builds. + """ + + def __init__( + self, + mdns_cache: dict[str, list[str]] | None = None, + dns_cache: dict[str, list[str]] | None = None, + ) -> None: + """Initialize the address cache. + + Args: + mdns_cache: Pre-populated mDNS addresses (hostname -> IPs) + dns_cache: Pre-populated DNS addresses (hostname -> IPs) + """ + self.mdns_cache = mdns_cache or {} + self.dns_cache = dns_cache or {} + + def _get_cached_addresses( + self, hostname: str, cache: dict[str, list[str]], cache_type: str + ) -> list[str] | None: + """Get cached addresses from a specific cache. + + Args: + hostname: The hostname to look up + cache: The cache dictionary to check + cache_type: Type of cache for logging ("mDNS" or "DNS") + + Returns: + List of IP addresses if found in cache, None otherwise + """ + normalized = normalize_hostname(hostname) + if addresses := cache.get(normalized): + _LOGGER.debug("Using %s cache for %s: %s", cache_type, hostname, addresses) + return addresses + return None + + def get_mdns_addresses(self, hostname: str) -> list[str] | None: + """Get cached mDNS addresses for a hostname. + + Args: + hostname: The hostname to look up (should end with .local) + + Returns: + List of IP addresses if found in cache, None otherwise + """ + return self._get_cached_addresses(hostname, self.mdns_cache, "mDNS") + + def get_dns_addresses(self, hostname: str) -> list[str] | None: + """Get cached DNS addresses for a hostname. + + Args: + hostname: The hostname to look up + + Returns: + List of IP addresses if found in cache, None otherwise + """ + return self._get_cached_addresses(hostname, self.dns_cache, "DNS") + + def get_addresses(self, hostname: str) -> list[str] | None: + """Get cached addresses for a hostname. + + Checks mDNS cache for .local domains, DNS cache otherwise. + + Args: + hostname: The hostname to look up + + Returns: + List of IP addresses if found in cache, None otherwise + """ + normalized = normalize_hostname(hostname) + if normalized.endswith(".local"): + return self.get_mdns_addresses(hostname) + return self.get_dns_addresses(hostname) + + def has_cache(self) -> bool: + """Check if any cache entries exist.""" + return bool(self.mdns_cache or self.dns_cache) + + @classmethod + def from_cli_args( + cls, mdns_args: Iterable[str], dns_args: Iterable[str] + ) -> AddressCache: + """Create cache from command-line arguments. + + Args: + mdns_args: List of mDNS cache entries like ['host=ip1,ip2'] + dns_args: List of DNS cache entries like ['host=ip1,ip2'] + + Returns: + Configured AddressCache instance + """ + mdns_cache = cls._parse_cache_args(mdns_args) + dns_cache = cls._parse_cache_args(dns_args) + return cls(mdns_cache=mdns_cache, dns_cache=dns_cache) + + @staticmethod + def _parse_cache_args(cache_args: Iterable[str]) -> dict[str, list[str]]: + """Parse cache arguments into a dictionary. + + Args: + cache_args: List of cache mappings like ['host1=ip1,ip2', 'host2=ip3'] + + Returns: + Dictionary mapping normalized hostnames to list of IP addresses + """ + cache: dict[str, list[str]] = {} + for arg in cache_args: + if "=" not in arg: + _LOGGER.warning( + "Invalid cache format: %s (expected 'hostname=ip1,ip2')", arg + ) + continue + hostname, ips = arg.split("=", 1) + # Normalize hostname for consistent lookups + normalized = normalize_hostname(hostname) + cache[normalized] = [ip.strip() for ip in ips.split(",")] + return cache diff --git a/esphome/automation.py b/esphome/automation.py index 34159561c2..99def9f273 100644 --- a/esphome/automation.py +++ b/esphome/automation.py @@ -15,7 +15,10 @@ from esphome.const import ( CONF_TYPE_ID, CONF_UPDATE_INTERVAL, ) +from esphome.core import ID +from esphome.cpp_generator import MockObj, MockObjClass, TemplateArgsType from esphome.schema_extractors import SCHEMA_EXTRACT, schema_extractor +from esphome.types import ConfigType from esphome.util import Registry @@ -49,11 +52,11 @@ def maybe_conf(conf, *validators): return validate -def register_action(name, action_type, schema): +def register_action(name: str, action_type: MockObjClass, schema: cv.Schema): return ACTION_REGISTRY.register(name, action_type, schema) -def register_condition(name, condition_type, schema): +def register_condition(name: str, condition_type: MockObjClass, schema: cv.Schema): return CONDITION_REGISTRY.register(name, condition_type, schema) @@ -164,43 +167,78 @@ XorCondition = cg.esphome_ns.class_("XorCondition", Condition) @register_condition("and", AndCondition, validate_condition_list) -async def and_condition_to_code(config, condition_id, template_arg, args): +async def and_condition_to_code( + config: ConfigType, + condition_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: conditions = await build_condition_list(config, template_arg, args) return cg.new_Pvariable(condition_id, template_arg, conditions) @register_condition("or", OrCondition, validate_condition_list) -async def or_condition_to_code(config, condition_id, template_arg, args): +async def or_condition_to_code( + config: ConfigType, + condition_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: conditions = await build_condition_list(config, template_arg, args) return cg.new_Pvariable(condition_id, template_arg, conditions) @register_condition("all", AndCondition, validate_condition_list) -async def all_condition_to_code(config, condition_id, template_arg, args): +async def all_condition_to_code( + config: ConfigType, + condition_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: conditions = await build_condition_list(config, template_arg, args) return cg.new_Pvariable(condition_id, template_arg, conditions) @register_condition("any", OrCondition, validate_condition_list) -async def any_condition_to_code(config, condition_id, template_arg, args): +async def any_condition_to_code( + config: ConfigType, + condition_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: conditions = await build_condition_list(config, template_arg, args) return cg.new_Pvariable(condition_id, template_arg, conditions) @register_condition("not", NotCondition, validate_potentially_and_condition) -async def not_condition_to_code(config, condition_id, template_arg, args): +async def not_condition_to_code( + config: ConfigType, + condition_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: condition = await build_condition(config, template_arg, args) return cg.new_Pvariable(condition_id, template_arg, condition) @register_condition("xor", XorCondition, validate_condition_list) -async def xor_condition_to_code(config, condition_id, template_arg, args): +async def xor_condition_to_code( + config: ConfigType, + condition_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: conditions = await build_condition_list(config, template_arg, args) return cg.new_Pvariable(condition_id, template_arg, conditions) @register_condition("lambda", LambdaCondition, cv.returning_lambda) -async def lambda_condition_to_code(config, condition_id, template_arg, args): +async def lambda_condition_to_code( + config: ConfigType, + condition_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: lambda_ = await cg.process_lambda(config, args, return_type=bool) return cg.new_Pvariable(condition_id, template_arg, lambda_) @@ -217,7 +255,12 @@ async def lambda_condition_to_code(config, condition_id, template_arg, args): } ).extend(cv.COMPONENT_SCHEMA), ) -async def for_condition_to_code(config, condition_id, template_arg, args): +async def for_condition_to_code( + config: ConfigType, + condition_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: condition = await build_condition( config[CONF_CONDITION], cg.TemplateArguments(), [] ) @@ -231,7 +274,12 @@ async def for_condition_to_code(config, condition_id, template_arg, args): @register_action( "delay", DelayAction, cv.templatable(cv.positive_time_period_milliseconds) ) -async def delay_action_to_code(config, action_id, template_arg, args): +async def delay_action_to_code( + config: ConfigType, + action_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: var = cg.new_Pvariable(action_id, template_arg) await cg.register_component(var, {}) template_ = await cg.templatable(config, args, cg.uint32) @@ -256,10 +304,15 @@ async def delay_action_to_code(config, action_id, template_arg, args): cv.has_at_least_one_key(CONF_CONDITION, CONF_ANY, CONF_ALL), ), ) -async def if_action_to_code(config, action_id, template_arg, args): +async def if_action_to_code( + config: ConfigType, + action_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: cond_conf = next(el for el in config if el in (CONF_ANY, CONF_ALL, CONF_CONDITION)) - conditions = await build_condition(config[cond_conf], template_arg, args) - var = cg.new_Pvariable(action_id, template_arg, conditions) + condition = await build_condition(config[cond_conf], template_arg, args) + var = cg.new_Pvariable(action_id, template_arg, condition) if CONF_THEN in config: actions = await build_action_list(config[CONF_THEN], template_arg, args) cg.add(var.add_then(actions)) @@ -279,9 +332,14 @@ async def if_action_to_code(config, action_id, template_arg, args): } ), ) -async def while_action_to_code(config, action_id, template_arg, args): - conditions = await build_condition(config[CONF_CONDITION], template_arg, args) - var = cg.new_Pvariable(action_id, template_arg, conditions) +async def while_action_to_code( + config: ConfigType, + action_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: + condition = await build_condition(config[CONF_CONDITION], template_arg, args) + var = cg.new_Pvariable(action_id, template_arg, condition) actions = await build_action_list(config[CONF_THEN], template_arg, args) cg.add(var.add_then(actions)) return var @@ -297,7 +355,12 @@ async def while_action_to_code(config, action_id, template_arg, args): } ), ) -async def repeat_action_to_code(config, action_id, template_arg, args): +async def repeat_action_to_code( + config: ConfigType, + action_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: var = cg.new_Pvariable(action_id, template_arg) count_template = await cg.templatable(config[CONF_COUNT], args, cg.uint32) cg.add(var.set_count(count_template)) @@ -320,9 +383,14 @@ _validate_wait_until = cv.maybe_simple_value( @register_action("wait_until", WaitUntilAction, _validate_wait_until) -async def wait_until_action_to_code(config, action_id, template_arg, args): - conditions = await build_condition(config[CONF_CONDITION], template_arg, args) - var = cg.new_Pvariable(action_id, template_arg, conditions) +async def wait_until_action_to_code( + config: ConfigType, + action_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: + condition = await build_condition(config[CONF_CONDITION], template_arg, args) + var = cg.new_Pvariable(action_id, template_arg, condition) if CONF_TIMEOUT in config: template_ = await cg.templatable(config[CONF_TIMEOUT], args, cg.uint32) cg.add(var.set_timeout_value(template_)) @@ -331,7 +399,12 @@ async def wait_until_action_to_code(config, action_id, template_arg, args): @register_action("lambda", LambdaAction, cv.lambda_) -async def lambda_action_to_code(config, action_id, template_arg, args): +async def lambda_action_to_code( + config: ConfigType, + action_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: lambda_ = await cg.process_lambda(config, args, return_type=cg.void) return cg.new_Pvariable(action_id, template_arg, lambda_) @@ -345,7 +418,12 @@ async def lambda_action_to_code(config, action_id, template_arg, args): } ), ) -async def component_update_action_to_code(config, action_id, template_arg, args): +async def component_update_action_to_code( + config: ConfigType, + action_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: comp = await cg.get_variable(config[CONF_ID]) return cg.new_Pvariable(action_id, template_arg, comp) @@ -359,7 +437,12 @@ async def component_update_action_to_code(config, action_id, template_arg, args) } ), ) -async def component_suspend_action_to_code(config, action_id, template_arg, args): +async def component_suspend_action_to_code( + config: ConfigType, + action_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: comp = await cg.get_variable(config[CONF_ID]) return cg.new_Pvariable(action_id, template_arg, comp) @@ -376,7 +459,12 @@ async def component_suspend_action_to_code(config, action_id, template_arg, args } ), ) -async def component_resume_action_to_code(config, action_id, template_arg, args): +async def component_resume_action_to_code( + config: ConfigType, + action_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +) -> MockObj: comp = await cg.get_variable(config[CONF_ID]) var = cg.new_Pvariable(action_id, template_arg, comp) if CONF_UPDATE_INTERVAL in config: @@ -385,43 +473,51 @@ async def component_resume_action_to_code(config, action_id, template_arg, args) return var -async def build_action(full_config, template_arg, args): +async def build_action( + full_config: ConfigType, template_arg: cg.TemplateArguments, args: TemplateArgsType +) -> MockObj: registry_entry, config = cg.extract_registry_entry_config( ACTION_REGISTRY, full_config ) action_id = full_config[CONF_TYPE_ID] builder = registry_entry.coroutine_fun - ret = await builder(config, action_id, template_arg, args) - return ret + return await builder(config, action_id, template_arg, args) -async def build_action_list(config, templ, arg_type): - actions = [] +async def build_action_list( + config: list[ConfigType], templ: cg.TemplateArguments, arg_type: TemplateArgsType +) -> list[MockObj]: + actions: list[MockObj] = [] for conf in config: action = await build_action(conf, templ, arg_type) actions.append(action) return actions -async def build_condition(full_config, template_arg, args): +async def build_condition( + full_config: ConfigType, template_arg: cg.TemplateArguments, args: TemplateArgsType +) -> MockObj: registry_entry, config = cg.extract_registry_entry_config( CONDITION_REGISTRY, full_config ) action_id = full_config[CONF_TYPE_ID] builder = registry_entry.coroutine_fun - ret = await builder(config, action_id, template_arg, args) - return ret + return await builder(config, action_id, template_arg, args) -async def build_condition_list(config, templ, args): - conditions = [] +async def build_condition_list( + config: ConfigType, templ: cg.TemplateArguments, args: TemplateArgsType +) -> list[MockObj]: + conditions: list[MockObj] = [] for conf in config: condition = await build_condition(conf, templ, args) conditions.append(condition) return conditions -async def build_automation(trigger, args, config): +async def build_automation( + trigger: MockObj, args: TemplateArgsType, config: ConfigType +) -> MockObj: arg_types = [arg[0] for arg in args] templ = cg.TemplateArguments(*arg_types) obj = cg.new_Pvariable(config[CONF_AUTOMATION_ID], templ, trigger) diff --git a/esphome/build_gen/__init__.py b/esphome/build_gen/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/esphome/build_gen/platformio.py b/esphome/build_gen/platformio.py new file mode 100644 index 0000000000..30dbb69d86 --- /dev/null +++ b/esphome/build_gen/platformio.py @@ -0,0 +1,100 @@ +from esphome.const import __version__ +from esphome.core import CORE +from esphome.helpers import mkdir_p, read_file, write_file_if_changed +from esphome.writer import find_begin_end, update_storage_json + +INI_AUTO_GENERATE_BEGIN = "; ========== AUTO GENERATED CODE BEGIN ===========" +INI_AUTO_GENERATE_END = "; =========== AUTO GENERATED CODE END ============" + +INI_BASE_FORMAT = ( + """; Auto generated code by esphome + +[common] +lib_deps = +build_flags = +upload_flags = + +""", + """ + +""", +) + + +def format_ini(data: dict[str, str | list[str]]) -> str: + content = "" + for key, value in sorted(data.items()): + if isinstance(value, list): + content += f"{key} =\n" + for x in value: + content += f" {x}\n" + else: + content += f"{key} = {value}\n" + return content + + +def get_ini_content(): + CORE.add_platformio_option( + "lib_deps", + [x.as_lib_dep for x in CORE.platformio_libraries.values()] + + ["${common.lib_deps}"], + ) + # Sort to avoid changing build flags order + CORE.add_platformio_option("build_flags", sorted(CORE.build_flags)) + + # Sort to avoid changing build unflags order + CORE.add_platformio_option("build_unflags", sorted(CORE.build_unflags)) + + # Add extra script for C++ flags + CORE.add_platformio_option("extra_scripts", [f"pre:{CXX_FLAGS_FILE_NAME}"]) + + content = "[platformio]\n" + content += f"description = ESPHome {__version__}\n" + + content += f"[env:{CORE.name}]\n" + content += format_ini(CORE.platformio_options) + + return content + + +def write_ini(content): + update_storage_json() + path = CORE.relative_build_path("platformio.ini") + + if path.is_file(): + text = read_file(path) + content_format = find_begin_end( + text, INI_AUTO_GENERATE_BEGIN, INI_AUTO_GENERATE_END + ) + else: + content_format = INI_BASE_FORMAT + full_file = f"{content_format[0] + INI_AUTO_GENERATE_BEGIN}\n{content}" + full_file += INI_AUTO_GENERATE_END + content_format[1] + write_file_if_changed(path, full_file) + + +def write_project(): + mkdir_p(CORE.build_path) + + content = get_ini_content() + write_ini(content) + + # Write extra script for C++ specific flags + write_cxx_flags_script() + + +CXX_FLAGS_FILE_NAME = "cxx_flags.py" +CXX_FLAGS_FILE_CONTENTS = """# Auto-generated ESPHome script for C++ specific compiler flags +Import("env") + +# Add C++ specific flags +""" + + +def write_cxx_flags_script() -> None: + path = CORE.relative_build_path(CXX_FLAGS_FILE_NAME) + contents = CXX_FLAGS_FILE_CONTENTS + if not CORE.is_host: + contents += 'env.Append(CXXFLAGS=["-Wno-volatile"])' + contents += "\n" + write_file_if_changed(path, contents) diff --git a/esphome/codegen.py b/esphome/codegen.py index 8e02ec1164..6decd77c62 100644 --- a/esphome/codegen.py +++ b/esphome/codegen.py @@ -12,6 +12,7 @@ from esphome.cpp_generator import ( # noqa: F401 ArrayInitializer, Expression, LineComment, + LogStringLiteral, MockObj, MockObjClass, Pvariable, diff --git a/esphome/components/a4988/a4988.cpp b/esphome/components/a4988/a4988.cpp index 72b3835cfd..b9efb4ea44 100644 --- a/esphome/components/a4988/a4988.cpp +++ b/esphome/components/a4988/a4988.cpp @@ -7,7 +7,6 @@ namespace a4988 { static const char *const TAG = "a4988.stepper"; void A4988::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (this->sleep_pin_ != nullptr) { this->sleep_pin_->setup(); this->sleep_pin_->digital_write(false); diff --git a/esphome/components/absolute_humidity/absolute_humidity.cpp b/esphome/components/absolute_humidity/absolute_humidity.cpp index c3cb159aed..2c5603ee3d 100644 --- a/esphome/components/absolute_humidity/absolute_humidity.cpp +++ b/esphome/components/absolute_humidity/absolute_humidity.cpp @@ -7,8 +7,6 @@ namespace absolute_humidity { static const char *const TAG = "absolute_humidity.sensor"; void AbsoluteHumidityComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->get_name().c_str()); - ESP_LOGD(TAG, " Added callback for temperature '%s'", this->temperature_sensor_->get_name().c_str()); this->temperature_sensor_->add_on_state_callback([this](float state) { this->temperature_callback_(state); }); if (this->temperature_sensor_->has_state()) { @@ -63,11 +61,10 @@ void AbsoluteHumidityComponent::loop() { ESP_LOGW(TAG, "No valid state from temperature sensor!"); } if (no_humidity) { - ESP_LOGW(TAG, "No valid state from temperature sensor!"); + ESP_LOGW(TAG, "No valid state from humidity sensor!"); } - ESP_LOGW(TAG, "Unable to calculate absolute humidity."); this->publish_state(NAN); - this->status_set_warning(); + this->status_set_warning(LOG_STR("Unable to calculate absolute humidity.")); return; } @@ -89,9 +86,8 @@ void AbsoluteHumidityComponent::loop() { es = es_wobus(temperature_c); break; default: - ESP_LOGE(TAG, "Invalid saturation vapor pressure equation selection!"); this->publish_state(NAN); - this->status_set_error(); + this->status_set_error("Invalid saturation vapor pressure equation selection!"); return; } ESP_LOGD(TAG, "Saturation vapor pressure %f kPa", es); diff --git a/esphome/components/absolute_humidity/sensor.py b/esphome/components/absolute_humidity/sensor.py index 62a2c8ab7c..caaa546e25 100644 --- a/esphome/components/absolute_humidity/sensor.py +++ b/esphome/components/absolute_humidity/sensor.py @@ -5,7 +5,7 @@ from esphome.const import ( CONF_EQUATION, CONF_HUMIDITY, CONF_TEMPERATURE, - ICON_WATER, + DEVICE_CLASS_ABSOLUTE_HUMIDITY, STATE_CLASS_MEASUREMENT, UNIT_GRAMS_PER_CUBIC_METER, ) @@ -27,8 +27,8 @@ EQUATION = { CONFIG_SCHEMA = ( sensor.sensor_schema( unit_of_measurement=UNIT_GRAMS_PER_CUBIC_METER, - icon=ICON_WATER, accuracy_decimals=2, + device_class=DEVICE_CLASS_ABSOLUTE_HUMIDITY, state_class=STATE_CLASS_MEASUREMENT, ) .extend( diff --git a/esphome/components/adc/__init__.py b/esphome/components/adc/__init__.py index 10b7df8638..15dc447b6c 100644 --- a/esphome/components/adc/__init__.py +++ b/esphome/components/adc/__init__.py @@ -1,24 +1,18 @@ from esphome import pins import esphome.codegen as cg -from esphome.components.esp32 import get_esp32_variant +from esphome.components.esp32 import VARIANT_ESP32P4, get_esp32_variant from esphome.components.esp32.const import ( VARIANT_ESP32, VARIANT_ESP32C2, VARIANT_ESP32C3, + VARIANT_ESP32C5, VARIANT_ESP32C6, VARIANT_ESP32H2, VARIANT_ESP32S2, VARIANT_ESP32S3, ) -from esphome.config_helpers import filter_source_files_from_platform import esphome.config_validation as cv -from esphome.const import ( - CONF_ANALOG, - CONF_INPUT, - CONF_NUMBER, - PLATFORM_ESP8266, - PlatformFramework, -) +from esphome.const import CONF_ANALOG, CONF_INPUT, CONF_NUMBER, PLATFORM_ESP8266 from esphome.core import CORE CODEOWNERS = ["@esphome/core"] @@ -51,82 +45,103 @@ SAMPLING_MODES = { "max": sampling_mode.MAX, } -adc1_channel_t = cg.global_ns.enum("adc1_channel_t") -adc2_channel_t = cg.global_ns.enum("adc2_channel_t") +adc_unit_t = cg.global_ns.enum("adc_unit_t", is_class=True) + +adc_channel_t = cg.global_ns.enum("adc_channel_t", is_class=True) # pin to adc1 channel mapping # https://github.com/espressif/esp-idf/blob/v4.4.8/components/driver/include/driver/adc.h ESP32_VARIANT_ADC1_PIN_TO_CHANNEL = { # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32/include/soc/adc_channel.h VARIANT_ESP32: { - 36: adc1_channel_t.ADC1_CHANNEL_0, - 37: adc1_channel_t.ADC1_CHANNEL_1, - 38: adc1_channel_t.ADC1_CHANNEL_2, - 39: adc1_channel_t.ADC1_CHANNEL_3, - 32: adc1_channel_t.ADC1_CHANNEL_4, - 33: adc1_channel_t.ADC1_CHANNEL_5, - 34: adc1_channel_t.ADC1_CHANNEL_6, - 35: adc1_channel_t.ADC1_CHANNEL_7, + 36: adc_channel_t.ADC_CHANNEL_0, + 37: adc_channel_t.ADC_CHANNEL_1, + 38: adc_channel_t.ADC_CHANNEL_2, + 39: adc_channel_t.ADC_CHANNEL_3, + 32: adc_channel_t.ADC_CHANNEL_4, + 33: adc_channel_t.ADC_CHANNEL_5, + 34: adc_channel_t.ADC_CHANNEL_6, + 35: adc_channel_t.ADC_CHANNEL_7, }, # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32c2/include/soc/adc_channel.h VARIANT_ESP32C2: { - 0: adc1_channel_t.ADC1_CHANNEL_0, - 1: adc1_channel_t.ADC1_CHANNEL_1, - 2: adc1_channel_t.ADC1_CHANNEL_2, - 3: adc1_channel_t.ADC1_CHANNEL_3, - 4: adc1_channel_t.ADC1_CHANNEL_4, + 0: adc_channel_t.ADC_CHANNEL_0, + 1: adc_channel_t.ADC_CHANNEL_1, + 2: adc_channel_t.ADC_CHANNEL_2, + 3: adc_channel_t.ADC_CHANNEL_3, + 4: adc_channel_t.ADC_CHANNEL_4, }, # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32c3/include/soc/adc_channel.h VARIANT_ESP32C3: { - 0: adc1_channel_t.ADC1_CHANNEL_0, - 1: adc1_channel_t.ADC1_CHANNEL_1, - 2: adc1_channel_t.ADC1_CHANNEL_2, - 3: adc1_channel_t.ADC1_CHANNEL_3, - 4: adc1_channel_t.ADC1_CHANNEL_4, + 0: adc_channel_t.ADC_CHANNEL_0, + 1: adc_channel_t.ADC_CHANNEL_1, + 2: adc_channel_t.ADC_CHANNEL_2, + 3: adc_channel_t.ADC_CHANNEL_3, + 4: adc_channel_t.ADC_CHANNEL_4, + }, + # ESP32-C5 ADC1 pin mapping - based on official ESP-IDF documentation + # https://docs.espressif.com/projects/esp-idf/en/latest/esp32c5/api-reference/peripherals/gpio.html + VARIANT_ESP32C5: { + 1: adc_channel_t.ADC_CHANNEL_0, + 2: adc_channel_t.ADC_CHANNEL_1, + 3: adc_channel_t.ADC_CHANNEL_2, + 4: adc_channel_t.ADC_CHANNEL_3, + 5: adc_channel_t.ADC_CHANNEL_4, + 6: adc_channel_t.ADC_CHANNEL_5, }, # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32c6/include/soc/adc_channel.h VARIANT_ESP32C6: { - 0: adc1_channel_t.ADC1_CHANNEL_0, - 1: adc1_channel_t.ADC1_CHANNEL_1, - 2: adc1_channel_t.ADC1_CHANNEL_2, - 3: adc1_channel_t.ADC1_CHANNEL_3, - 4: adc1_channel_t.ADC1_CHANNEL_4, - 5: adc1_channel_t.ADC1_CHANNEL_5, - 6: adc1_channel_t.ADC1_CHANNEL_6, + 0: adc_channel_t.ADC_CHANNEL_0, + 1: adc_channel_t.ADC_CHANNEL_1, + 2: adc_channel_t.ADC_CHANNEL_2, + 3: adc_channel_t.ADC_CHANNEL_3, + 4: adc_channel_t.ADC_CHANNEL_4, + 5: adc_channel_t.ADC_CHANNEL_5, + 6: adc_channel_t.ADC_CHANNEL_6, }, # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32h2/include/soc/adc_channel.h VARIANT_ESP32H2: { - 1: adc1_channel_t.ADC1_CHANNEL_0, - 2: adc1_channel_t.ADC1_CHANNEL_1, - 3: adc1_channel_t.ADC1_CHANNEL_2, - 4: adc1_channel_t.ADC1_CHANNEL_3, - 5: adc1_channel_t.ADC1_CHANNEL_4, + 1: adc_channel_t.ADC_CHANNEL_0, + 2: adc_channel_t.ADC_CHANNEL_1, + 3: adc_channel_t.ADC_CHANNEL_2, + 4: adc_channel_t.ADC_CHANNEL_3, + 5: adc_channel_t.ADC_CHANNEL_4, }, # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32s2/include/soc/adc_channel.h VARIANT_ESP32S2: { - 1: adc1_channel_t.ADC1_CHANNEL_0, - 2: adc1_channel_t.ADC1_CHANNEL_1, - 3: adc1_channel_t.ADC1_CHANNEL_2, - 4: adc1_channel_t.ADC1_CHANNEL_3, - 5: adc1_channel_t.ADC1_CHANNEL_4, - 6: adc1_channel_t.ADC1_CHANNEL_5, - 7: adc1_channel_t.ADC1_CHANNEL_6, - 8: adc1_channel_t.ADC1_CHANNEL_7, - 9: adc1_channel_t.ADC1_CHANNEL_8, - 10: adc1_channel_t.ADC1_CHANNEL_9, + 1: adc_channel_t.ADC_CHANNEL_0, + 2: adc_channel_t.ADC_CHANNEL_1, + 3: adc_channel_t.ADC_CHANNEL_2, + 4: adc_channel_t.ADC_CHANNEL_3, + 5: adc_channel_t.ADC_CHANNEL_4, + 6: adc_channel_t.ADC_CHANNEL_5, + 7: adc_channel_t.ADC_CHANNEL_6, + 8: adc_channel_t.ADC_CHANNEL_7, + 9: adc_channel_t.ADC_CHANNEL_8, + 10: adc_channel_t.ADC_CHANNEL_9, }, # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32s3/include/soc/adc_channel.h VARIANT_ESP32S3: { - 1: adc1_channel_t.ADC1_CHANNEL_0, - 2: adc1_channel_t.ADC1_CHANNEL_1, - 3: adc1_channel_t.ADC1_CHANNEL_2, - 4: adc1_channel_t.ADC1_CHANNEL_3, - 5: adc1_channel_t.ADC1_CHANNEL_4, - 6: adc1_channel_t.ADC1_CHANNEL_5, - 7: adc1_channel_t.ADC1_CHANNEL_6, - 8: adc1_channel_t.ADC1_CHANNEL_7, - 9: adc1_channel_t.ADC1_CHANNEL_8, - 10: adc1_channel_t.ADC1_CHANNEL_9, + 1: adc_channel_t.ADC_CHANNEL_0, + 2: adc_channel_t.ADC_CHANNEL_1, + 3: adc_channel_t.ADC_CHANNEL_2, + 4: adc_channel_t.ADC_CHANNEL_3, + 5: adc_channel_t.ADC_CHANNEL_4, + 6: adc_channel_t.ADC_CHANNEL_5, + 7: adc_channel_t.ADC_CHANNEL_6, + 8: adc_channel_t.ADC_CHANNEL_7, + 9: adc_channel_t.ADC_CHANNEL_8, + 10: adc_channel_t.ADC_CHANNEL_9, + }, + VARIANT_ESP32P4: { + 16: adc_channel_t.ADC_CHANNEL_0, + 17: adc_channel_t.ADC_CHANNEL_1, + 18: adc_channel_t.ADC_CHANNEL_2, + 19: adc_channel_t.ADC_CHANNEL_3, + 20: adc_channel_t.ADC_CHANNEL_4, + 21: adc_channel_t.ADC_CHANNEL_5, + 22: adc_channel_t.ADC_CHANNEL_6, + 23: adc_channel_t.ADC_CHANNEL_7, }, } @@ -135,54 +150,64 @@ ESP32_VARIANT_ADC1_PIN_TO_CHANNEL = { ESP32_VARIANT_ADC2_PIN_TO_CHANNEL = { # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32/include/soc/adc_channel.h VARIANT_ESP32: { - 4: adc2_channel_t.ADC2_CHANNEL_0, - 0: adc2_channel_t.ADC2_CHANNEL_1, - 2: adc2_channel_t.ADC2_CHANNEL_2, - 15: adc2_channel_t.ADC2_CHANNEL_3, - 13: adc2_channel_t.ADC2_CHANNEL_4, - 12: adc2_channel_t.ADC2_CHANNEL_5, - 14: adc2_channel_t.ADC2_CHANNEL_6, - 27: adc2_channel_t.ADC2_CHANNEL_7, - 25: adc2_channel_t.ADC2_CHANNEL_8, - 26: adc2_channel_t.ADC2_CHANNEL_9, + 4: adc_channel_t.ADC_CHANNEL_0, + 0: adc_channel_t.ADC_CHANNEL_1, + 2: adc_channel_t.ADC_CHANNEL_2, + 15: adc_channel_t.ADC_CHANNEL_3, + 13: adc_channel_t.ADC_CHANNEL_4, + 12: adc_channel_t.ADC_CHANNEL_5, + 14: adc_channel_t.ADC_CHANNEL_6, + 27: adc_channel_t.ADC_CHANNEL_7, + 25: adc_channel_t.ADC_CHANNEL_8, + 26: adc_channel_t.ADC_CHANNEL_9, }, # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32c2/include/soc/adc_channel.h VARIANT_ESP32C2: { - 5: adc2_channel_t.ADC2_CHANNEL_0, + 5: adc_channel_t.ADC_CHANNEL_0, }, # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32c3/include/soc/adc_channel.h VARIANT_ESP32C3: { - 5: adc2_channel_t.ADC2_CHANNEL_0, + 5: adc_channel_t.ADC_CHANNEL_0, }, + # ESP32-C5 has no ADC2 channels + VARIANT_ESP32C5: {}, # no ADC2 # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32c6/include/soc/adc_channel.h VARIANT_ESP32C6: {}, # no ADC2 # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32h2/include/soc/adc_channel.h VARIANT_ESP32H2: {}, # no ADC2 # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32s2/include/soc/adc_channel.h VARIANT_ESP32S2: { - 11: adc2_channel_t.ADC2_CHANNEL_0, - 12: adc2_channel_t.ADC2_CHANNEL_1, - 13: adc2_channel_t.ADC2_CHANNEL_2, - 14: adc2_channel_t.ADC2_CHANNEL_3, - 15: adc2_channel_t.ADC2_CHANNEL_4, - 16: adc2_channel_t.ADC2_CHANNEL_5, - 17: adc2_channel_t.ADC2_CHANNEL_6, - 18: adc2_channel_t.ADC2_CHANNEL_7, - 19: adc2_channel_t.ADC2_CHANNEL_8, - 20: adc2_channel_t.ADC2_CHANNEL_9, + 11: adc_channel_t.ADC_CHANNEL_0, + 12: adc_channel_t.ADC_CHANNEL_1, + 13: adc_channel_t.ADC_CHANNEL_2, + 14: adc_channel_t.ADC_CHANNEL_3, + 15: adc_channel_t.ADC_CHANNEL_4, + 16: adc_channel_t.ADC_CHANNEL_5, + 17: adc_channel_t.ADC_CHANNEL_6, + 18: adc_channel_t.ADC_CHANNEL_7, + 19: adc_channel_t.ADC_CHANNEL_8, + 20: adc_channel_t.ADC_CHANNEL_9, }, # https://github.com/espressif/esp-idf/blob/master/components/soc/esp32s3/include/soc/adc_channel.h VARIANT_ESP32S3: { - 11: adc2_channel_t.ADC2_CHANNEL_0, - 12: adc2_channel_t.ADC2_CHANNEL_1, - 13: adc2_channel_t.ADC2_CHANNEL_2, - 14: adc2_channel_t.ADC2_CHANNEL_3, - 15: adc2_channel_t.ADC2_CHANNEL_4, - 16: adc2_channel_t.ADC2_CHANNEL_5, - 17: adc2_channel_t.ADC2_CHANNEL_6, - 18: adc2_channel_t.ADC2_CHANNEL_7, - 19: adc2_channel_t.ADC2_CHANNEL_8, - 20: adc2_channel_t.ADC2_CHANNEL_9, + 11: adc_channel_t.ADC_CHANNEL_0, + 12: adc_channel_t.ADC_CHANNEL_1, + 13: adc_channel_t.ADC_CHANNEL_2, + 14: adc_channel_t.ADC_CHANNEL_3, + 15: adc_channel_t.ADC_CHANNEL_4, + 16: adc_channel_t.ADC_CHANNEL_5, + 17: adc_channel_t.ADC_CHANNEL_6, + 18: adc_channel_t.ADC_CHANNEL_7, + 19: adc_channel_t.ADC_CHANNEL_8, + 20: adc_channel_t.ADC_CHANNEL_9, + }, + VARIANT_ESP32P4: { + 49: adc_channel_t.ADC_CHANNEL_0, + 50: adc_channel_t.ADC_CHANNEL_1, + 51: adc_channel_t.ADC_CHANNEL_2, + 52: adc_channel_t.ADC_CHANNEL_3, + 53: adc_channel_t.ADC_CHANNEL_4, + 54: adc_channel_t.ADC_CHANNEL_5, }, } @@ -235,21 +260,9 @@ def validate_adc_pin(value): {CONF_ANALOG: True, CONF_INPUT: True}, internal=True )(value) + if CORE.is_nrf52: + return pins.gpio_pin_schema( + {CONF_ANALOG: True, CONF_INPUT: True}, internal=True + )(value) + raise NotImplementedError - - -FILTER_SOURCE_FILES = filter_source_files_from_platform( - { - "adc_sensor_esp32.cpp": { - PlatformFramework.ESP32_ARDUINO, - PlatformFramework.ESP32_IDF, - }, - "adc_sensor_esp8266.cpp": {PlatformFramework.ESP8266_ARDUINO}, - "adc_sensor_rp2040.cpp": {PlatformFramework.RP2040_ARDUINO}, - "adc_sensor_libretiny.cpp": { - PlatformFramework.BK72XX_ARDUINO, - PlatformFramework.RTL87XX_ARDUINO, - PlatformFramework.LN882X_ARDUINO, - }, - } -) diff --git a/esphome/components/adc/adc_sensor.h b/esphome/components/adc/adc_sensor.h index 28dfd2262c..526dd57fd5 100644 --- a/esphome/components/adc/adc_sensor.h +++ b/esphome/components/adc/adc_sensor.h @@ -3,12 +3,19 @@ #include "esphome/components/sensor/sensor.h" #include "esphome/components/voltage_sampler/voltage_sampler.h" #include "esphome/core/component.h" +#include "esphome/core/defines.h" #include "esphome/core/hal.h" #ifdef USE_ESP32 -#include -#include "driver/adc.h" -#endif // USE_ESP32 +#include "esp_adc/adc_cali.h" +#include "esp_adc/adc_cali_scheme.h" +#include "esp_adc/adc_oneshot.h" +#include "hal/adc_types.h" // This defines ADC_CHANNEL_MAX +#endif // USE_ESP32 + +#ifdef USE_ZEPHYR +#include +#endif namespace esphome { namespace adc { @@ -35,50 +42,91 @@ enum class SamplingMode : uint8_t { const LogString *sampling_mode_to_str(SamplingMode mode); -class Aggregator { +template class Aggregator { public: Aggregator(SamplingMode mode); - void add_sample(uint32_t value); - uint32_t aggregate(); + void add_sample(T value); + T aggregate(); protected: - uint32_t aggr_{0}; - uint32_t samples_{0}; + T aggr_{0}; + uint8_t samples_{0}; SamplingMode mode_{SamplingMode::AVG}; }; class ADCSensor : public sensor::Sensor, public PollingComponent, public voltage_sampler::VoltageSampler { public: -#ifdef USE_ESP32 - /// Set the attenuation for this pin. Only available on the ESP32. - void set_attenuation(adc_atten_t attenuation) { this->attenuation_ = attenuation; } - void set_channel1(adc1_channel_t channel) { - this->channel1_ = channel; - this->channel2_ = ADC2_CHANNEL_MAX; - } - void set_channel2(adc2_channel_t channel) { - this->channel2_ = channel; - this->channel1_ = ADC1_CHANNEL_MAX; - } - void set_autorange(bool autorange) { this->autorange_ = autorange; } -#endif // USE_ESP32 - - /// Update ADC values + /// Update the sensor's state by reading the current ADC value. + /// This method is called periodically based on the update interval. void update() override; - /// Setup ADC + + /// Set up the ADC sensor by initializing hardware and calibration parameters. + /// This method is called once during device initialization. void setup() override; + + /// Output the configuration details of the ADC sensor for debugging purposes. + /// This method is called during the ESPHome setup process to log the configuration. void dump_config() override; - /// `HARDWARE_LATE` setup priority + + /// Return the setup priority for this component. + /// Components with higher priority are initialized earlier during setup. + /// @return A float representing the setup priority. float get_setup_priority() const override; + +#ifdef USE_ZEPHYR + /// Set the ADC channel to be used by the ADC sensor. + /// @param channel Pointer to an adc_dt_spec structure representing the ADC channel. + void set_adc_channel(const adc_dt_spec *channel) { this->channel_ = channel; } +#endif + /// Set the GPIO pin to be used by the ADC sensor. + /// @param pin Pointer to an InternalGPIOPin representing the ADC input pin. void set_pin(InternalGPIOPin *pin) { this->pin_ = pin; } + + /// Enable or disable the output of raw ADC values (unprocessed data). + /// @param output_raw Boolean indicating whether to output raw ADC values (true) or processed values (false). void set_output_raw(bool output_raw) { this->output_raw_ = output_raw; } + + /// Set the number of samples to be taken for ADC readings to improve accuracy. + /// A higher sample count reduces noise but increases the reading time. + /// @param sample_count The number of samples (e.g., 1, 4, 8). void set_sample_count(uint8_t sample_count); + + /// Set the sampling mode for how multiple ADC samples are combined into a single measurement. + /// + /// When multiple samples are taken (controlled by set_sample_count), they can be combined + /// in one of three ways: + /// - SamplingMode::AVG: Compute the average (default) + /// - SamplingMode::MIN: Use the lowest sample value + /// - SamplingMode::MAX: Use the highest sample value + /// @param sampling_mode The desired sampling mode to use for aggregating ADC samples. void set_sampling_mode(SamplingMode sampling_mode); + + /// Perform a single ADC sampling operation and return the measured value. + /// This function handles raw readings, calibration, and averaging as needed. + /// @return The sampled value as a float. float sample() override; -#ifdef USE_ESP8266 - std::string unique_id() override; -#endif // USE_ESP8266 +#ifdef USE_ESP32 + /// Set the ADC attenuation level to adjust the input voltage range. + /// This determines how the ADC interprets input voltages, allowing for greater precision + /// or the ability to measure higher voltages depending on the chosen attenuation level. + /// @param attenuation The desired ADC attenuation level (e.g., ADC_ATTEN_DB_0, ADC_ATTEN_DB_11). + void set_attenuation(adc_atten_t attenuation) { this->attenuation_ = attenuation; } + + /// Configure the ADC to use a specific channel on a specific ADC unit. + /// This sets the channel for single-shot or continuous ADC measurements. + /// @param unit The ADC unit to use (ADC_UNIT_1 or ADC_UNIT_2). + /// @param channel The ADC channel to configure, such as ADC_CHANNEL_0, ADC_CHANNEL_3, etc. + void set_channel(adc_unit_t unit, adc_channel_t channel) { + this->adc_unit_ = unit; + this->channel_ = channel; + } + + /// Set whether autoranging should be enabled for the ADC. + /// Autoranging automatically adjusts the attenuation level to handle a wide range of input voltages. + /// @param autorange Boolean indicating whether to enable autoranging. + void set_autorange(bool autorange) { this->autorange_ = autorange; } +#endif // USE_ESP32 #ifdef USE_RP2040 void set_is_temperature() { this->is_temperature_ = true; } @@ -90,17 +138,32 @@ class ADCSensor : public sensor::Sensor, public PollingComponent, public voltage InternalGPIOPin *pin_; SamplingMode sampling_mode_{SamplingMode::AVG}; +#ifdef USE_ESP32 + float sample_autorange_(); + float sample_fixed_attenuation_(); + bool autorange_{false}; + adc_oneshot_unit_handle_t adc_handle_{nullptr}; + adc_cali_handle_t calibration_handle_{nullptr}; + adc_atten_t attenuation_{ADC_ATTEN_DB_0}; + adc_channel_t channel_{}; + adc_unit_t adc_unit_{}; + struct SetupFlags { + uint8_t init_complete : 1; + uint8_t config_complete : 1; + uint8_t handle_init_complete : 1; + uint8_t calibration_complete : 1; + uint8_t reserved : 4; + } setup_flags_{}; + static adc_oneshot_unit_handle_t shared_adc_handles[2]; +#endif // USE_ESP32 + #ifdef USE_RP2040 bool is_temperature_{false}; #endif // USE_RP2040 -#ifdef USE_ESP32 - adc_atten_t attenuation_{ADC_ATTEN_DB_0}; - adc1_channel_t channel1_{ADC1_CHANNEL_MAX}; - adc2_channel_t channel2_{ADC2_CHANNEL_MAX}; - bool autorange_{false}; - esp_adc_cal_characteristics_t cal_characteristics_[SOC_ADC_ATTEN_NUM] = {}; -#endif // USE_ESP32 +#ifdef USE_ZEPHYR + const struct adc_dt_spec *channel_ = nullptr; +#endif }; } // namespace adc diff --git a/esphome/components/adc/adc_sensor_common.cpp b/esphome/components/adc/adc_sensor_common.cpp index 797ab75045..748c8634b7 100644 --- a/esphome/components/adc/adc_sensor_common.cpp +++ b/esphome/components/adc/adc_sensor_common.cpp @@ -18,15 +18,15 @@ const LogString *sampling_mode_to_str(SamplingMode mode) { return LOG_STR("unknown"); } -Aggregator::Aggregator(SamplingMode mode) { +template Aggregator::Aggregator(SamplingMode mode) { this->mode_ = mode; // set to max uint if mode is "min" if (mode == SamplingMode::MIN) { - this->aggr_ = UINT32_MAX; + this->aggr_ = std::numeric_limits::max(); } } -void Aggregator::add_sample(uint32_t value) { +template void Aggregator::add_sample(T value) { this->samples_ += 1; switch (this->mode_) { @@ -47,7 +47,7 @@ void Aggregator::add_sample(uint32_t value) { } } -uint32_t Aggregator::aggregate() { +template T Aggregator::aggregate() { if (this->mode_ == SamplingMode::AVG) { if (this->samples_ == 0) { return this->aggr_; @@ -59,6 +59,12 @@ uint32_t Aggregator::aggregate() { return this->aggr_; } +#ifdef USE_ZEPHYR +template class Aggregator; +#else +template class Aggregator; +#endif + void ADCSensor::update() { float value_v = this->sample(); ESP_LOGV(TAG, "'%s': Voltage=%.4fV", this->get_name().c_str(), value_v); diff --git a/esphome/components/adc/adc_sensor_esp32.cpp b/esphome/components/adc/adc_sensor_esp32.cpp index ed1f3329ab..ab6a89fce0 100644 --- a/esphome/components/adc/adc_sensor_esp32.cpp +++ b/esphome/components/adc/adc_sensor_esp32.cpp @@ -8,145 +8,322 @@ namespace adc { static const char *const TAG = "adc.esp32"; -static const adc_bits_width_t ADC_WIDTH_MAX_SOC_BITS = static_cast(ADC_WIDTH_MAX - 1); +adc_oneshot_unit_handle_t ADCSensor::shared_adc_handles[2] = {nullptr, nullptr}; -#ifndef SOC_ADC_RTC_MAX_BITWIDTH -#if USE_ESP32_VARIANT_ESP32S2 -static const int32_t SOC_ADC_RTC_MAX_BITWIDTH = 13; -#else -static const int32_t SOC_ADC_RTC_MAX_BITWIDTH = 12; -#endif // USE_ESP32_VARIANT_ESP32S2 -#endif // SOC_ADC_RTC_MAX_BITWIDTH - -static const int ADC_MAX = (1 << SOC_ADC_RTC_MAX_BITWIDTH) - 1; -static const int ADC_HALF = (1 << SOC_ADC_RTC_MAX_BITWIDTH) >> 1; - -void ADCSensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->get_name().c_str()); - - if (this->channel1_ != ADC1_CHANNEL_MAX) { - adc1_config_width(ADC_WIDTH_MAX_SOC_BITS); - if (!this->autorange_) { - adc1_config_channel_atten(this->channel1_, this->attenuation_); - } - } else if (this->channel2_ != ADC2_CHANNEL_MAX) { - if (!this->autorange_) { - adc2_config_channel_atten(this->channel2_, this->attenuation_); - } - } - - for (int32_t i = 0; i <= ADC_ATTEN_DB_12_COMPAT; i++) { - auto adc_unit = this->channel1_ != ADC1_CHANNEL_MAX ? ADC_UNIT_1 : ADC_UNIT_2; - auto cal_value = esp_adc_cal_characterize(adc_unit, (adc_atten_t) i, ADC_WIDTH_MAX_SOC_BITS, - 1100, // default vref - &this->cal_characteristics_[i]); - switch (cal_value) { - case ESP_ADC_CAL_VAL_EFUSE_VREF: - ESP_LOGV(TAG, "Using eFuse Vref for calibration"); - break; - case ESP_ADC_CAL_VAL_EFUSE_TP: - ESP_LOGV(TAG, "Using two-point eFuse Vref for calibration"); - break; - case ESP_ADC_CAL_VAL_DEFAULT_VREF: - default: - break; - } +const LogString *attenuation_to_str(adc_atten_t attenuation) { + switch (attenuation) { + case ADC_ATTEN_DB_0: + return LOG_STR("0 dB"); + case ADC_ATTEN_DB_2_5: + return LOG_STR("2.5 dB"); + case ADC_ATTEN_DB_6: + return LOG_STR("6 dB"); + case ADC_ATTEN_DB_12_COMPAT: + return LOG_STR("12 dB"); + default: + return LOG_STR("Unknown Attenuation"); } } -void ADCSensor::dump_config() { - static const char *const ATTEN_AUTO_STR = "auto"; - static const char *const ATTEN_0DB_STR = "0 db"; - static const char *const ATTEN_2_5DB_STR = "2.5 db"; - static const char *const ATTEN_6DB_STR = "6 db"; - static const char *const ATTEN_12DB_STR = "12 db"; - const char *atten_str = ATTEN_AUTO_STR; +const LogString *adc_unit_to_str(adc_unit_t unit) { + switch (unit) { + case ADC_UNIT_1: + return LOG_STR("ADC1"); + case ADC_UNIT_2: + return LOG_STR("ADC2"); + default: + return LOG_STR("Unknown ADC Unit"); + } +} - LOG_SENSOR("", "ADC Sensor", this); - LOG_PIN(" Pin: ", this->pin_); - - if (!this->autorange_) { - switch (this->attenuation_) { - case ADC_ATTEN_DB_0: - atten_str = ATTEN_0DB_STR; - break; - case ADC_ATTEN_DB_2_5: - atten_str = ATTEN_2_5DB_STR; - break; - case ADC_ATTEN_DB_6: - atten_str = ATTEN_6DB_STR; - break; - case ADC_ATTEN_DB_12_COMPAT: - atten_str = ATTEN_12DB_STR; - break; - default: // This is to satisfy the unused ADC_ATTEN_MAX - break; +void ADCSensor::setup() { + // Check if another sensor already initialized this ADC unit + if (ADCSensor::shared_adc_handles[this->adc_unit_] == nullptr) { + adc_oneshot_unit_init_cfg_t init_config = {}; // Zero initialize + init_config.unit_id = this->adc_unit_; + init_config.ulp_mode = ADC_ULP_MODE_DISABLE; +#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || USE_ESP32_VARIANT_ESP32H2 + init_config.clk_src = ADC_DIGI_CLK_SRC_DEFAULT; +#endif // USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || + // USE_ESP32_VARIANT_ESP32H2 + esp_err_t err = adc_oneshot_new_unit(&init_config, &ADCSensor::shared_adc_handles[this->adc_unit_]); + if (err != ESP_OK) { + ESP_LOGE(TAG, "Error initializing %s: %d", LOG_STR_ARG(adc_unit_to_str(this->adc_unit_)), err); + this->mark_failed(); + return; } } + this->adc_handle_ = ADCSensor::shared_adc_handles[this->adc_unit_]; + this->setup_flags_.handle_init_complete = true; + + adc_oneshot_chan_cfg_t config = { + .atten = this->attenuation_, + .bitwidth = ADC_BITWIDTH_DEFAULT, + }; + esp_err_t err = adc_oneshot_config_channel(this->adc_handle_, this->channel_, &config); + if (err != ESP_OK) { + ESP_LOGE(TAG, "Error configuring channel: %d", err); + this->mark_failed(); + return; + } + this->setup_flags_.config_complete = true; + + // Initialize ADC calibration + if (this->calibration_handle_ == nullptr) { + adc_cali_handle_t handle = nullptr; + +#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || \ + USE_ESP32_VARIANT_ESP32S3 || USE_ESP32_VARIANT_ESP32H2 || USE_ESP32_VARIANT_ESP32P4 + // RISC-V variants and S3 use curve fitting calibration + adc_cali_curve_fitting_config_t cali_config = {}; // Zero initialize first +#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 3, 0) + cali_config.chan = this->channel_; +#endif // ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 3, 0) + cali_config.unit_id = this->adc_unit_; + cali_config.atten = this->attenuation_; + cali_config.bitwidth = ADC_BITWIDTH_DEFAULT; + + err = adc_cali_create_scheme_curve_fitting(&cali_config, &handle); + if (err == ESP_OK) { + this->calibration_handle_ = handle; + this->setup_flags_.calibration_complete = true; + ESP_LOGV(TAG, "Using curve fitting calibration"); + } else { + ESP_LOGW(TAG, "Curve fitting calibration failed with error %d, will use uncalibrated readings", err); + this->setup_flags_.calibration_complete = false; + } +#else // Other ESP32 variants use line fitting calibration + adc_cali_line_fitting_config_t cali_config = { + .unit_id = this->adc_unit_, + .atten = this->attenuation_, + .bitwidth = ADC_BITWIDTH_DEFAULT, +#if !defined(USE_ESP32_VARIANT_ESP32S2) + .default_vref = 1100, // Default reference voltage in mV +#endif // !defined(USE_ESP32_VARIANT_ESP32S2) + }; + err = adc_cali_create_scheme_line_fitting(&cali_config, &handle); + if (err == ESP_OK) { + this->calibration_handle_ = handle; + this->setup_flags_.calibration_complete = true; + ESP_LOGV(TAG, "Using line fitting calibration"); + } else { + ESP_LOGW(TAG, "Line fitting calibration failed with error %d, will use uncalibrated readings", err); + this->setup_flags_.calibration_complete = false; + } +#endif // USE_ESP32_VARIANT_ESP32C3 || ESP32C5 || ESP32C6 || ESP32S3 || ESP32H2 + } + + this->setup_flags_.init_complete = true; +} + +void ADCSensor::dump_config() { + LOG_SENSOR("", "ADC Sensor", this); + LOG_PIN(" Pin: ", this->pin_); ESP_LOGCONFIG(TAG, - " Attenuation: %s\n" - " Samples: %i\n" + " Channel: %d\n" + " Unit: %s\n" + " Attenuation: %s\n" + " Samples: %i\n" " Sampling mode: %s", - atten_str, this->sample_count_, LOG_STR_ARG(sampling_mode_to_str(this->sampling_mode_))); + this->channel_, LOG_STR_ARG(adc_unit_to_str(this->adc_unit_)), + this->autorange_ ? "Auto" : LOG_STR_ARG(attenuation_to_str(this->attenuation_)), this->sample_count_, + LOG_STR_ARG(sampling_mode_to_str(this->sampling_mode_))); + + ESP_LOGCONFIG( + TAG, + " Setup Status:\n" + " Handle Init: %s\n" + " Config: %s\n" + " Calibration: %s\n" + " Overall Init: %s", + this->setup_flags_.handle_init_complete ? "OK" : "FAILED", this->setup_flags_.config_complete ? "OK" : "FAILED", + this->setup_flags_.calibration_complete ? "OK" : "FAILED", this->setup_flags_.init_complete ? "OK" : "FAILED"); + LOG_UPDATE_INTERVAL(this); } float ADCSensor::sample() { - if (!this->autorange_) { - auto aggr = Aggregator(this->sampling_mode_); + if (this->autorange_) { + return this->sample_autorange_(); + } else { + return this->sample_fixed_attenuation_(); + } +} - for (uint8_t sample = 0; sample < this->sample_count_; sample++) { - int raw = -1; - if (this->channel1_ != ADC1_CHANNEL_MAX) { - raw = adc1_get_raw(this->channel1_); - } else if (this->channel2_ != ADC2_CHANNEL_MAX) { - adc2_get_raw(this->channel2_, ADC_WIDTH_MAX_SOC_BITS, &raw); - } - if (raw == -1) { - return NAN; - } +float ADCSensor::sample_fixed_attenuation_() { + auto aggr = Aggregator(this->sampling_mode_); - aggr.add_sample(raw); + for (uint8_t sample = 0; sample < this->sample_count_; sample++) { + int raw; + esp_err_t err = adc_oneshot_read(this->adc_handle_, this->channel_, &raw); + + if (err != ESP_OK) { + ESP_LOGW(TAG, "ADC read failed with error %d", err); + continue; } - if (this->output_raw_) { - return aggr.aggregate(); + + if (raw == -1) { + ESP_LOGW(TAG, "Invalid ADC reading"); + continue; } - uint32_t mv = - esp_adc_cal_raw_to_voltage(aggr.aggregate(), &this->cal_characteristics_[(int32_t) this->attenuation_]); - return mv / 1000.0f; + + aggr.add_sample(raw); } - int raw12 = ADC_MAX, raw6 = ADC_MAX, raw2 = ADC_MAX, raw0 = ADC_MAX; + uint32_t final_value = aggr.aggregate(); - if (this->channel1_ != ADC1_CHANNEL_MAX) { - adc1_config_channel_atten(this->channel1_, ADC_ATTEN_DB_12_COMPAT); - raw12 = adc1_get_raw(this->channel1_); - if (raw12 < ADC_MAX) { - adc1_config_channel_atten(this->channel1_, ADC_ATTEN_DB_6); - raw6 = adc1_get_raw(this->channel1_); - if (raw6 < ADC_MAX) { - adc1_config_channel_atten(this->channel1_, ADC_ATTEN_DB_2_5); - raw2 = adc1_get_raw(this->channel1_); - if (raw2 < ADC_MAX) { - adc1_config_channel_atten(this->channel1_, ADC_ATTEN_DB_0); - raw0 = adc1_get_raw(this->channel1_); - } + if (this->output_raw_) { + return final_value; + } + + if (this->calibration_handle_ != nullptr) { + int voltage_mv; + esp_err_t err = adc_cali_raw_to_voltage(this->calibration_handle_, final_value, &voltage_mv); + if (err == ESP_OK) { + return voltage_mv / 1000.0f; + } else { + ESP_LOGW(TAG, "ADC calibration conversion failed with error %d, disabling calibration", err); + if (this->calibration_handle_ != nullptr) { +#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || \ + USE_ESP32_VARIANT_ESP32S3 || USE_ESP32_VARIANT_ESP32H2 || USE_ESP32_VARIANT_ESP32P4 + adc_cali_delete_scheme_curve_fitting(this->calibration_handle_); +#else // Other ESP32 variants use line fitting calibration + adc_cali_delete_scheme_line_fitting(this->calibration_handle_); +#endif // USE_ESP32_VARIANT_ESP32C3 || ESP32C5 || ESP32C6 || ESP32S3 || ESP32H2 + this->calibration_handle_ = nullptr; } } - } else if (this->channel2_ != ADC2_CHANNEL_MAX) { - adc2_config_channel_atten(this->channel2_, ADC_ATTEN_DB_12_COMPAT); - adc2_get_raw(this->channel2_, ADC_WIDTH_MAX_SOC_BITS, &raw12); - if (raw12 < ADC_MAX) { - adc2_config_channel_atten(this->channel2_, ADC_ATTEN_DB_6); - adc2_get_raw(this->channel2_, ADC_WIDTH_MAX_SOC_BITS, &raw6); - if (raw6 < ADC_MAX) { - adc2_config_channel_atten(this->channel2_, ADC_ATTEN_DB_2_5); - adc2_get_raw(this->channel2_, ADC_WIDTH_MAX_SOC_BITS, &raw2); - if (raw2 < ADC_MAX) { - adc2_config_channel_atten(this->channel2_, ADC_ATTEN_DB_0); - adc2_get_raw(this->channel2_, ADC_WIDTH_MAX_SOC_BITS, &raw0); - } + } + + return final_value * 3.3f / 4095.0f; +} + +float ADCSensor::sample_autorange_() { + // Auto-range mode + auto read_atten = [this](adc_atten_t atten) -> std::pair { + // First reconfigure the attenuation for this reading + adc_oneshot_chan_cfg_t config = { + .atten = atten, + .bitwidth = ADC_BITWIDTH_DEFAULT, + }; + + esp_err_t err = adc_oneshot_config_channel(this->adc_handle_, this->channel_, &config); + + if (err != ESP_OK) { + ESP_LOGW(TAG, "Error configuring ADC channel for autorange: %d", err); + return {-1, 0.0f}; + } + + // Need to recalibrate for the new attenuation + if (this->calibration_handle_ != nullptr) { + // Delete old calibration handle +#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || \ + USE_ESP32_VARIANT_ESP32S3 || USE_ESP32_VARIANT_ESP32H2 || USE_ESP32_VARIANT_ESP32P4 + adc_cali_delete_scheme_curve_fitting(this->calibration_handle_); +#else + adc_cali_delete_scheme_line_fitting(this->calibration_handle_); +#endif + this->calibration_handle_ = nullptr; + } + + // Create new calibration handle for this attenuation + adc_cali_handle_t handle = nullptr; + +#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || \ + USE_ESP32_VARIANT_ESP32S3 || USE_ESP32_VARIANT_ESP32H2 || USE_ESP32_VARIANT_ESP32P4 + adc_cali_curve_fitting_config_t cali_config = {}; +#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 3, 0) + cali_config.chan = this->channel_; +#endif + cali_config.unit_id = this->adc_unit_; + cali_config.atten = atten; + cali_config.bitwidth = ADC_BITWIDTH_DEFAULT; + + err = adc_cali_create_scheme_curve_fitting(&cali_config, &handle); + ESP_LOGVV(TAG, "Autorange atten=%d: Calibration handle creation %s (err=%d)", atten, + (err == ESP_OK) ? "SUCCESS" : "FAILED", err); +#else + adc_cali_line_fitting_config_t cali_config = { + .unit_id = this->adc_unit_, + .atten = atten, + .bitwidth = ADC_BITWIDTH_DEFAULT, +#if !defined(USE_ESP32_VARIANT_ESP32S2) + .default_vref = 1100, +#endif + }; + err = adc_cali_create_scheme_line_fitting(&cali_config, &handle); + ESP_LOGVV(TAG, "Autorange atten=%d: Calibration handle creation %s (err=%d)", atten, + (err == ESP_OK) ? "SUCCESS" : "FAILED", err); +#endif + + int raw; + err = adc_oneshot_read(this->adc_handle_, this->channel_, &raw); + ESP_LOGVV(TAG, "Autorange atten=%d: Raw ADC read %s, value=%d (err=%d)", atten, + (err == ESP_OK) ? "SUCCESS" : "FAILED", raw, err); + + if (err != ESP_OK) { + ESP_LOGW(TAG, "ADC read failed in autorange with error %d", err); + if (handle != nullptr) { +#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || \ + USE_ESP32_VARIANT_ESP32S3 || USE_ESP32_VARIANT_ESP32H2 || USE_ESP32_VARIANT_ESP32P4 + adc_cali_delete_scheme_curve_fitting(handle); +#else + adc_cali_delete_scheme_line_fitting(handle); +#endif + } + return {-1, 0.0f}; + } + + float voltage = 0.0f; + if (handle != nullptr) { + int voltage_mv; + err = adc_cali_raw_to_voltage(handle, raw, &voltage_mv); + if (err == ESP_OK) { + voltage = voltage_mv / 1000.0f; + ESP_LOGVV(TAG, "Autorange atten=%d: CALIBRATED - raw=%d -> %dmV -> %.6fV", atten, raw, voltage_mv, voltage); + } else { + voltage = raw * 3.3f / 4095.0f; + ESP_LOGVV(TAG, "Autorange atten=%d: UNCALIBRATED FALLBACK - raw=%d -> %.6fV (3.3V ref)", atten, raw, voltage); + } + // Clean up calibration handle +#if USE_ESP32_VARIANT_ESP32C3 || USE_ESP32_VARIANT_ESP32C5 || USE_ESP32_VARIANT_ESP32C6 || \ + USE_ESP32_VARIANT_ESP32S3 || USE_ESP32_VARIANT_ESP32H2 || USE_ESP32_VARIANT_ESP32P4 + adc_cali_delete_scheme_curve_fitting(handle); +#else + adc_cali_delete_scheme_line_fitting(handle); +#endif + } else { + voltage = raw * 3.3f / 4095.0f; + ESP_LOGVV(TAG, "Autorange atten=%d: NO CALIBRATION - raw=%d -> %.6fV (3.3V ref)", atten, raw, voltage); + } + + return {raw, voltage}; + }; + + auto [raw12, mv12] = read_atten(ADC_ATTEN_DB_12); + if (raw12 == -1) { + ESP_LOGE(TAG, "Failed to read ADC in autorange mode"); + return NAN; + } + + int raw6 = 4095, raw2 = 4095, raw0 = 4095; + float mv6 = 0, mv2 = 0, mv0 = 0; + + if (raw12 < 4095) { + auto [raw6_val, mv6_val] = read_atten(ADC_ATTEN_DB_6); + raw6 = raw6_val; + mv6 = mv6_val; + + if (raw6 < 4095 && raw6 != -1) { + auto [raw2_val, mv2_val] = read_atten(ADC_ATTEN_DB_2_5); + raw2 = raw2_val; + mv2 = mv2_val; + + if (raw2 < 4095 && raw2 != -1) { + auto [raw0_val, mv0_val] = read_atten(ADC_ATTEN_DB_0); + raw0 = raw0_val; + mv0 = mv0_val; } } } @@ -155,19 +332,33 @@ float ADCSensor::sample() { return NAN; } - uint32_t mv12 = esp_adc_cal_raw_to_voltage(raw12, &this->cal_characteristics_[(int32_t) ADC_ATTEN_DB_12_COMPAT]); - uint32_t mv6 = esp_adc_cal_raw_to_voltage(raw6, &this->cal_characteristics_[(int32_t) ADC_ATTEN_DB_6]); - uint32_t mv2 = esp_adc_cal_raw_to_voltage(raw2, &this->cal_characteristics_[(int32_t) ADC_ATTEN_DB_2_5]); - uint32_t mv0 = esp_adc_cal_raw_to_voltage(raw0, &this->cal_characteristics_[(int32_t) ADC_ATTEN_DB_0]); + const int adc_half = 2048; + const uint32_t c12 = std::min(raw12, adc_half); - uint32_t c12 = std::min(raw12, ADC_HALF); - uint32_t c6 = ADC_HALF - std::abs(raw6 - ADC_HALF); - uint32_t c2 = ADC_HALF - std::abs(raw2 - ADC_HALF); - uint32_t c0 = std::min(ADC_MAX - raw0, ADC_HALF); - uint32_t csum = c12 + c6 + c2 + c0; + const int32_t c6_signed = adc_half - std::abs(raw6 - adc_half); + const uint32_t c6 = (c6_signed > 0) ? c6_signed : 0; // Clamp to prevent underflow - uint32_t mv_scaled = (mv12 * c12) + (mv6 * c6) + (mv2 * c2) + (mv0 * c0); - return mv_scaled / (float) (csum * 1000U); + const int32_t c2_signed = adc_half - std::abs(raw2 - adc_half); + const uint32_t c2 = (c2_signed > 0) ? c2_signed : 0; // Clamp to prevent underflow + + const uint32_t c0 = std::min(4095 - raw0, adc_half); + const uint32_t csum = c12 + c6 + c2 + c0; + + ESP_LOGVV(TAG, "Autorange summary:"); + ESP_LOGVV(TAG, " Raw readings: 12db=%d, 6db=%d, 2.5db=%d, 0db=%d", raw12, raw6, raw2, raw0); + ESP_LOGVV(TAG, " Voltages: 12db=%.6f, 6db=%.6f, 2.5db=%.6f, 0db=%.6f", mv12, mv6, mv2, mv0); + ESP_LOGVV(TAG, " Coefficients: c12=%u, c6=%u, c2=%u, c0=%u, sum=%u", c12, c6, c2, c0, csum); + + if (csum == 0) { + ESP_LOGE(TAG, "Invalid weight sum in autorange calculation"); + return NAN; + } + + const float final_result = (mv12 * c12 + mv6 * c6 + mv2 * c2 + mv0 * c0) / csum; + ESP_LOGV(TAG, "Autorange final: (%.6f*%u + %.6f*%u + %.6f*%u + %.6f*%u)/%u = %.6fV", mv12, c12, mv6, c6, mv2, c2, mv0, + c0, csum, final_result); + + return final_result; } } // namespace adc diff --git a/esphome/components/adc/adc_sensor_esp8266.cpp b/esphome/components/adc/adc_sensor_esp8266.cpp index 67128fb1f3..be14b252d4 100644 --- a/esphome/components/adc/adc_sensor_esp8266.cpp +++ b/esphome/components/adc/adc_sensor_esp8266.cpp @@ -17,7 +17,6 @@ namespace adc { static const char *const TAG = "adc.esp8266"; void ADCSensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->get_name().c_str()); #ifndef USE_ADC_SENSOR_VCC this->pin_->setup(); #endif @@ -38,7 +37,7 @@ void ADCSensor::dump_config() { } float ADCSensor::sample() { - auto aggr = Aggregator(this->sampling_mode_); + auto aggr = Aggregator(this->sampling_mode_); for (uint8_t sample = 0; sample < this->sample_count_; sample++) { uint32_t raw = 0; @@ -56,8 +55,6 @@ float ADCSensor::sample() { return aggr.aggregate() / 1024.0f; } -std::string ADCSensor::unique_id() { return get_mac_address() + "-adc"; } - } // namespace adc } // namespace esphome diff --git a/esphome/components/adc/adc_sensor_libretiny.cpp b/esphome/components/adc/adc_sensor_libretiny.cpp index f7c7e669ec..0b1393c2e7 100644 --- a/esphome/components/adc/adc_sensor_libretiny.cpp +++ b/esphome/components/adc/adc_sensor_libretiny.cpp @@ -9,7 +9,6 @@ namespace adc { static const char *const TAG = "adc.libretiny"; void ADCSensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->get_name().c_str()); #ifndef USE_ADC_SENSOR_VCC this->pin_->setup(); #endif // !USE_ADC_SENSOR_VCC @@ -31,7 +30,7 @@ void ADCSensor::dump_config() { float ADCSensor::sample() { uint32_t raw = 0; - auto aggr = Aggregator(this->sampling_mode_); + auto aggr = Aggregator(this->sampling_mode_); if (this->output_raw_) { for (uint8_t sample = 0; sample < this->sample_count_; sample++) { diff --git a/esphome/components/adc/adc_sensor_rp2040.cpp b/esphome/components/adc/adc_sensor_rp2040.cpp index 91d331270b..8496e0f41e 100644 --- a/esphome/components/adc/adc_sensor_rp2040.cpp +++ b/esphome/components/adc/adc_sensor_rp2040.cpp @@ -14,7 +14,6 @@ namespace adc { static const char *const TAG = "adc.rp2040"; void ADCSensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->get_name().c_str()); static bool initialized = false; if (!initialized) { adc_init(); @@ -42,7 +41,7 @@ void ADCSensor::dump_config() { float ADCSensor::sample() { uint32_t raw = 0; - auto aggr = Aggregator(this->sampling_mode_); + auto aggr = Aggregator(this->sampling_mode_); if (this->is_temperature_) { adc_set_temp_sensor_enabled(true); diff --git a/esphome/components/adc/adc_sensor_zephyr.cpp b/esphome/components/adc/adc_sensor_zephyr.cpp new file mode 100644 index 0000000000..2fb9d4b0e5 --- /dev/null +++ b/esphome/components/adc/adc_sensor_zephyr.cpp @@ -0,0 +1,207 @@ + +#include "adc_sensor.h" +#ifdef USE_ZEPHYR +#include "esphome/core/log.h" + +#include "hal/nrf_saadc.h" + +namespace esphome { +namespace adc { + +static const char *const TAG = "adc.zephyr"; + +void ADCSensor::setup() { + if (!adc_is_ready_dt(this->channel_)) { + ESP_LOGE(TAG, "ADC controller device %s not ready", this->channel_->dev->name); + return; + } + + auto err = adc_channel_setup_dt(this->channel_); + if (err < 0) { + ESP_LOGE(TAG, "Could not setup channel %s (%d)", this->channel_->dev->name, err); + return; + } +} + +#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE +static const LogString *gain_to_str(enum adc_gain gain) { + switch (gain) { + case ADC_GAIN_1_6: + return LOG_STR("1/6"); + case ADC_GAIN_1_5: + return LOG_STR("1/5"); + case ADC_GAIN_1_4: + return LOG_STR("1/4"); + case ADC_GAIN_1_3: + return LOG_STR("1/3"); + case ADC_GAIN_2_5: + return LOG_STR("2/5"); + case ADC_GAIN_1_2: + return LOG_STR("1/2"); + case ADC_GAIN_2_3: + return LOG_STR("2/3"); + case ADC_GAIN_4_5: + return LOG_STR("4/5"); + case ADC_GAIN_1: + return LOG_STR("1"); + case ADC_GAIN_2: + return LOG_STR("2"); + case ADC_GAIN_3: + return LOG_STR("3"); + case ADC_GAIN_4: + return LOG_STR("4"); + case ADC_GAIN_6: + return LOG_STR("6"); + case ADC_GAIN_8: + return LOG_STR("8"); + case ADC_GAIN_12: + return LOG_STR("12"); + case ADC_GAIN_16: + return LOG_STR("16"); + case ADC_GAIN_24: + return LOG_STR("24"); + case ADC_GAIN_32: + return LOG_STR("32"); + case ADC_GAIN_64: + return LOG_STR("64"); + case ADC_GAIN_128: + return LOG_STR("128"); + } + return LOG_STR("undefined gain"); +} + +static const LogString *reference_to_str(enum adc_reference reference) { + switch (reference) { + case ADC_REF_VDD_1: + return LOG_STR("VDD"); + case ADC_REF_VDD_1_2: + return LOG_STR("VDD/2"); + case ADC_REF_VDD_1_3: + return LOG_STR("VDD/3"); + case ADC_REF_VDD_1_4: + return LOG_STR("VDD/4"); + case ADC_REF_INTERNAL: + return LOG_STR("INTERNAL"); + case ADC_REF_EXTERNAL0: + return LOG_STR("External, input 0"); + case ADC_REF_EXTERNAL1: + return LOG_STR("External, input 1"); + } + return LOG_STR("undefined reference"); +} + +static const LogString *input_to_str(uint8_t input) { + switch (input) { + case NRF_SAADC_INPUT_AIN0: + return LOG_STR("AIN0"); + case NRF_SAADC_INPUT_AIN1: + return LOG_STR("AIN1"); + case NRF_SAADC_INPUT_AIN2: + return LOG_STR("AIN2"); + case NRF_SAADC_INPUT_AIN3: + return LOG_STR("AIN3"); + case NRF_SAADC_INPUT_AIN4: + return LOG_STR("AIN4"); + case NRF_SAADC_INPUT_AIN5: + return LOG_STR("AIN5"); + case NRF_SAADC_INPUT_AIN6: + return LOG_STR("AIN6"); + case NRF_SAADC_INPUT_AIN7: + return LOG_STR("AIN7"); + case NRF_SAADC_INPUT_VDD: + return LOG_STR("VDD"); + case NRF_SAADC_INPUT_VDDHDIV5: + return LOG_STR("VDDHDIV5"); + } + return LOG_STR("undefined input"); +} +#endif // ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE + +void ADCSensor::dump_config() { + LOG_SENSOR("", "ADC Sensor", this); + LOG_PIN(" Pin: ", this->pin_); +#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE + ESP_LOGV(TAG, + " Name: %s\n" + " Channel: %d\n" + " vref_mv: %d\n" + " Resolution %d\n" + " Oversampling %d", + this->channel_->dev->name, this->channel_->channel_id, this->channel_->vref_mv, this->channel_->resolution, + this->channel_->oversampling); + + ESP_LOGV(TAG, + " Gain: %s\n" + " reference: %s\n" + " acquisition_time: %d\n" + " differential %s", + LOG_STR_ARG(gain_to_str(this->channel_->channel_cfg.gain)), + LOG_STR_ARG(reference_to_str(this->channel_->channel_cfg.reference)), + this->channel_->channel_cfg.acquisition_time, YESNO(this->channel_->channel_cfg.differential)); + if (this->channel_->channel_cfg.differential) { + ESP_LOGV(TAG, + " Positive: %s\n" + " Negative: %s", + LOG_STR_ARG(input_to_str(this->channel_->channel_cfg.input_positive)), + LOG_STR_ARG(input_to_str(this->channel_->channel_cfg.input_negative))); + } else { + ESP_LOGV(TAG, " Positive: %s", LOG_STR_ARG(input_to_str(this->channel_->channel_cfg.input_positive))); + } +#endif + + LOG_UPDATE_INTERVAL(this); +} + +float ADCSensor::sample() { + auto aggr = Aggregator(this->sampling_mode_); + int err; + for (uint8_t sample = 0; sample < this->sample_count_; sample++) { + int16_t buf = 0; + struct adc_sequence sequence = { + .buffer = &buf, + /* buffer size in bytes, not number of samples */ + .buffer_size = sizeof(buf), + }; + int32_t val_raw; + + err = adc_sequence_init_dt(this->channel_, &sequence); + if (err < 0) { + ESP_LOGE(TAG, "Could sequence init %s (%d)", this->channel_->dev->name, err); + return 0.0; + } + + err = adc_read(this->channel_->dev, &sequence); + if (err < 0) { + ESP_LOGE(TAG, "Could not read %s (%d)", this->channel_->dev->name, err); + return 0.0; + } + + val_raw = (int32_t) buf; + if (!this->channel_->channel_cfg.differential) { + // https://github.com/adafruit/Adafruit_nRF52_Arduino/blob/0ed4d9ffc674ae407be7cacf5696a02f5e789861/cores/nRF5/wiring_analog_nRF52.c#L222 + if (val_raw < 0) { + val_raw = 0; + } + } + aggr.add_sample(val_raw); + } + + int32_t val_mv = aggr.aggregate(); + + if (this->output_raw_) { + return val_mv; + } + + err = adc_raw_to_millivolts_dt(this->channel_, &val_mv); + /* conversion to mV may not be supported, skip if not */ + if (err < 0) { + ESP_LOGE(TAG, "Value in mV not available %s (%d)", this->channel_->dev->name, err); + return 0.0; + } + + return val_mv / 1000.0f; +} + +} // namespace adc +} // namespace esphome +#endif diff --git a/esphome/components/adc/sensor.py b/esphome/components/adc/sensor.py index 3309bd04c5..607609bbc7 100644 --- a/esphome/components/adc/sensor.py +++ b/esphome/components/adc/sensor.py @@ -3,6 +3,13 @@ import logging import esphome.codegen as cg from esphome.components import sensor, voltage_sampler from esphome.components.esp32 import get_esp32_variant +from esphome.components.nrf52.const import AIN_TO_GPIO, EXTRA_ADC +from esphome.components.zephyr import ( + zephyr_add_overlay, + zephyr_add_prj_conf, + zephyr_add_user, +) +from esphome.config_helpers import filter_source_files_from_platform import esphome.config_validation as cv from esphome.const import ( CONF_ATTENUATION, @@ -10,13 +17,13 @@ from esphome.const import ( CONF_NUMBER, CONF_PIN, CONF_RAW, - CONF_WIFI, DEVICE_CLASS_VOLTAGE, + PLATFORM_NRF52, STATE_CLASS_MEASUREMENT, UNIT_VOLT, + PlatformFramework, ) from esphome.core import CORE -import esphome.final_validate as fv from . import ( ATTENUATION_MODES, @@ -24,6 +31,7 @@ from . import ( ESP32_VARIANT_ADC2_PIN_TO_CHANNEL, SAMPLING_MODES, adc_ns, + adc_unit_t, validate_adc_pin, ) @@ -57,25 +65,14 @@ def validate_config(config): return config -def final_validate_config(config): - if CORE.is_esp32: - variant = get_esp32_variant() - if ( - CONF_WIFI in fv.full_config.get() - and config[CONF_PIN][CONF_NUMBER] - in ESP32_VARIANT_ADC2_PIN_TO_CHANNEL[variant] - ): - raise cv.Invalid( - f"{variant} doesn't support ADC on this pin when Wi-Fi is configured" - ) - - return config - - ADCSensor = adc_ns.class_( "ADCSensor", sensor.Sensor, cg.PollingComponent, voltage_sampler.VoltageSampler ) +CONF_NRF_SAADC = "nrf_saadc" + +adc_dt_spec = cg.global_ns.class_("adc_dt_spec") + CONFIG_SCHEMA = cv.All( sensor.sensor_schema( ADCSensor, @@ -91,6 +88,7 @@ CONFIG_SCHEMA = cv.All( cv.SplitDefault(CONF_ATTENUATION, esp32="0db"): cv.All( cv.only_on_esp32, _attenuation ), + cv.OnlyWith(CONF_NRF_SAADC, PLATFORM_NRF52): cv.declare_id(adc_dt_spec), cv.Optional(CONF_SAMPLES, default=1): cv.int_range(min=1, max=255), cv.Optional(CONF_SAMPLING_MODE, default="avg"): _sampling_mode, } @@ -99,7 +97,7 @@ CONFIG_SCHEMA = cv.All( validate_config, ) -FINAL_VALIDATE_SCHEMA = final_validate_config +CONF_ADC_CHANNEL_ID = "adc_channel_id" async def to_code(config): @@ -111,7 +109,7 @@ async def to_code(config): cg.add_define("USE_ADC_SENSOR_VCC") elif config[CONF_PIN] == "TEMPERATURE": cg.add(var.set_is_temperature()) - else: + elif not CORE.is_nrf52 or config[CONF_PIN][CONF_NUMBER] not in EXTRA_ADC: pin = await cg.gpio_pin_expression(config[CONF_PIN]) cg.add(var.set_pin(pin)) @@ -119,13 +117,13 @@ async def to_code(config): cg.add(var.set_sample_count(config[CONF_SAMPLES])) cg.add(var.set_sampling_mode(config[CONF_SAMPLING_MODE])) - if attenuation := config.get(CONF_ATTENUATION): - if attenuation == "auto": - cg.add(var.set_autorange(cg.global_ns.true)) - else: - cg.add(var.set_attenuation(attenuation)) - if CORE.is_esp32: + if attenuation := config.get(CONF_ATTENUATION): + if attenuation == "auto": + cg.add(var.set_autorange(cg.global_ns.true)) + else: + cg.add(var.set_attenuation(attenuation)) + variant = get_esp32_variant() pin_num = config[CONF_PIN][CONF_NUMBER] if ( @@ -133,10 +131,66 @@ async def to_code(config): and pin_num in ESP32_VARIANT_ADC1_PIN_TO_CHANNEL[variant] ): chan = ESP32_VARIANT_ADC1_PIN_TO_CHANNEL[variant][pin_num] - cg.add(var.set_channel1(chan)) + cg.add(var.set_channel(adc_unit_t.ADC_UNIT_1, chan)) elif ( variant in ESP32_VARIANT_ADC2_PIN_TO_CHANNEL and pin_num in ESP32_VARIANT_ADC2_PIN_TO_CHANNEL[variant] ): chan = ESP32_VARIANT_ADC2_PIN_TO_CHANNEL[variant][pin_num] - cg.add(var.set_channel2(chan)) + cg.add(var.set_channel(adc_unit_t.ADC_UNIT_2, chan)) + + elif CORE.is_nrf52: + CORE.data.setdefault(CONF_ADC_CHANNEL_ID, 0) + channel_id = CORE.data[CONF_ADC_CHANNEL_ID] + CORE.data[CONF_ADC_CHANNEL_ID] = channel_id + 1 + zephyr_add_prj_conf("ADC", True) + nrf_saadc = config[CONF_NRF_SAADC] + rhs = cg.RawExpression( + f"ADC_DT_SPEC_GET_BY_IDX(DT_PATH(zephyr_user), {channel_id})" + ) + adc = cg.new_Pvariable(nrf_saadc, rhs) + cg.add(var.set_adc_channel(adc)) + gain = "ADC_GAIN_1_6" + pin_number = config[CONF_PIN][CONF_NUMBER] + if pin_number == "VDDHDIV5": + gain = "ADC_GAIN_1_2" + if isinstance(pin_number, int): + GPIO_TO_AIN = {v: k for k, v in AIN_TO_GPIO.items()} + pin_number = GPIO_TO_AIN[pin_number] + zephyr_add_user("io-channels", f"<&adc {channel_id}>") + zephyr_add_overlay( + f""" +&adc {{ + #address-cells = <1>; + #size-cells = <0>; + + channel@{channel_id} {{ + reg = <{channel_id}>; + zephyr,gain = "{gain}"; + zephyr,reference = "ADC_REF_INTERNAL"; + zephyr,acquisition-time = ; + zephyr,input-positive = ; + zephyr,resolution = <14>; + zephyr,oversampling = <8>; + }}; +}}; +""" + ) + + +FILTER_SOURCE_FILES = filter_source_files_from_platform( + { + "adc_sensor_esp32.cpp": { + PlatformFramework.ESP32_ARDUINO, + PlatformFramework.ESP32_IDF, + }, + "adc_sensor_esp8266.cpp": {PlatformFramework.ESP8266_ARDUINO}, + "adc_sensor_rp2040.cpp": {PlatformFramework.RP2040_ARDUINO}, + "adc_sensor_libretiny.cpp": { + PlatformFramework.BK72XX_ARDUINO, + PlatformFramework.RTL87XX_ARDUINO, + PlatformFramework.LN882X_ARDUINO, + }, + "adc_sensor_zephyr.cpp": {PlatformFramework.NRF52_ZEPHYR}, + } +) diff --git a/esphome/components/adc128s102/adc128s102.cpp b/esphome/components/adc128s102/adc128s102.cpp index c8e8edb359..935dbde8ea 100644 --- a/esphome/components/adc128s102/adc128s102.cpp +++ b/esphome/components/adc128s102/adc128s102.cpp @@ -8,10 +8,7 @@ static const char *const TAG = "adc128s102"; float ADC128S102::get_setup_priority() const { return setup_priority::HARDWARE; } -void ADC128S102::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->spi_setup(); -} +void ADC128S102::setup() { this->spi_setup(); } void ADC128S102::dump_config() { ESP_LOGCONFIG(TAG, "ADC128S102:"); diff --git a/esphome/components/ade7880/ade7880.cpp b/esphome/components/ade7880/ade7880.cpp index 55f834bf86..fd560e0676 100644 --- a/esphome/components/ade7880/ade7880.cpp +++ b/esphome/components/ade7880/ade7880.cpp @@ -113,7 +113,7 @@ void ADE7880::update() { if (this->channel_a_ != nullptr) { auto *chan = this->channel_a_; this->update_sensor_from_s24zp_register16_(chan->current, AIRMS, [](float val) { return val / 100000.0f; }); - this->update_sensor_from_s24zp_register16_(chan->voltage, BVRMS, [](float val) { return val / 10000.0f; }); + this->update_sensor_from_s24zp_register16_(chan->voltage, AVRMS, [](float val) { return val / 10000.0f; }); this->update_sensor_from_s24zp_register16_(chan->active_power, AWATT, [](float val) { return val / 100.0f; }); this->update_sensor_from_s24zp_register16_(chan->apparent_power, AVA, [](float val) { return val / 100.0f; }); this->update_sensor_from_s16_register16_(chan->power_factor, APF, diff --git a/esphome/components/ade7880/sensor.py b/esphome/components/ade7880/sensor.py index 3ef5e6bfff..39dbeb225f 100644 --- a/esphome/components/ade7880/sensor.py +++ b/esphome/components/ade7880/sensor.py @@ -36,6 +36,7 @@ from esphome.const import ( UNIT_WATT, UNIT_WATT_HOURS, ) +from esphome.types import ConfigType DEPENDENCIES = ["i2c"] @@ -51,6 +52,20 @@ CONF_POWER_GAIN = "power_gain" CONF_NEUTRAL = "neutral" +# Tuple of power channel phases +POWER_PHASES = (CONF_PHASE_A, CONF_PHASE_B, CONF_PHASE_C) + +# Tuple of sensor types that can be configured for power channels +POWER_SENSOR_TYPES = ( + CONF_CURRENT, + CONF_VOLTAGE, + CONF_ACTIVE_POWER, + CONF_APPARENT_POWER, + CONF_POWER_FACTOR, + CONF_FORWARD_ACTIVE_ENERGY, + CONF_REVERSE_ACTIVE_ENERGY, +) + NEUTRAL_CHANNEL_SCHEMA = cv.Schema( { cv.GenerateID(): cv.declare_id(NeutralChannel), @@ -150,7 +165,64 @@ POWER_CHANNEL_SCHEMA = cv.Schema( } ) -CONFIG_SCHEMA = ( + +def prefix_sensor_name( + sensor_conf: ConfigType, + channel_name: str, + channel_config: ConfigType, + sensor_type: str, +) -> None: + """Helper to prefix sensor name with channel name. + + Args: + sensor_conf: The sensor configuration (dict or string) + channel_name: The channel name to prefix with + channel_config: The channel configuration to update + sensor_type: The sensor type key in the channel config + """ + if isinstance(sensor_conf, dict) and CONF_NAME in sensor_conf: + sensor_name = sensor_conf[CONF_NAME] + if sensor_name and not sensor_name.startswith(channel_name): + sensor_conf[CONF_NAME] = f"{channel_name} {sensor_name}" + elif isinstance(sensor_conf, str): + # Simple value case - convert to dict with prefixed name + channel_config[sensor_type] = {CONF_NAME: f"{channel_name} {sensor_conf}"} + + +def process_channel_sensors( + config: ConfigType, channel_key: str, sensor_types: tuple +) -> None: + """Process sensors for a channel and prefix their names. + + Args: + config: The main configuration + channel_key: The channel key (e.g., CONF_PHASE_A, CONF_NEUTRAL) + sensor_types: Tuple of sensor types to process for this channel + """ + if not (channel_config := config.get(channel_key)) or not ( + channel_name := channel_config.get(CONF_NAME) + ): + return + + for sensor_type in sensor_types: + if sensor_conf := channel_config.get(sensor_type): + prefix_sensor_name(sensor_conf, channel_name, channel_config, sensor_type) + + +def preprocess_channels(config: ConfigType) -> ConfigType: + """Preprocess channel configurations to add channel name prefix to sensor names.""" + # Process power channels + for channel in POWER_PHASES: + process_channel_sensors(config, channel, POWER_SENSOR_TYPES) + + # Process neutral channel + process_channel_sensors(config, CONF_NEUTRAL, (CONF_CURRENT,)) + + return config + + +CONFIG_SCHEMA = cv.All( + preprocess_channels, cv.Schema( { cv.GenerateID(): cv.declare_id(ADE7880), @@ -167,7 +239,7 @@ CONFIG_SCHEMA = ( } ) .extend(cv.polling_component_schema("60s")) - .extend(i2c.i2c_device_schema(0x38)) + .extend(i2c.i2c_device_schema(0x38)), ) @@ -188,15 +260,7 @@ async def neutral_channel(config): async def power_channel(config): var = cg.new_Pvariable(config[CONF_ID]) - for sensor_type in [ - CONF_CURRENT, - CONF_VOLTAGE, - CONF_ACTIVE_POWER, - CONF_APPARENT_POWER, - CONF_POWER_FACTOR, - CONF_FORWARD_ACTIVE_ENERGY, - CONF_REVERSE_ACTIVE_ENERGY, - ]: + for sensor_type in POWER_SENSOR_TYPES: if conf := config.get(sensor_type): sens = await sensor.new_sensor(conf) cg.add(getattr(var, f"set_{sensor_type}")(sens)) @@ -216,44 +280,6 @@ async def power_channel(config): return var -def final_validate(config): - for channel in [CONF_PHASE_A, CONF_PHASE_B, CONF_PHASE_C]: - if channel := config.get(channel): - channel_name = channel.get(CONF_NAME) - - for sensor_type in [ - CONF_CURRENT, - CONF_VOLTAGE, - CONF_ACTIVE_POWER, - CONF_APPARENT_POWER, - CONF_POWER_FACTOR, - CONF_FORWARD_ACTIVE_ENERGY, - CONF_REVERSE_ACTIVE_ENERGY, - ]: - if conf := channel.get(sensor_type): - sensor_name = conf.get(CONF_NAME) - if ( - sensor_name - and channel_name - and not sensor_name.startswith(channel_name) - ): - conf[CONF_NAME] = f"{channel_name} {sensor_name}" - - if channel := config.get(CONF_NEUTRAL): - channel_name = channel.get(CONF_NAME) - if conf := channel.get(CONF_CURRENT): - sensor_name = conf.get(CONF_NAME) - if ( - sensor_name - and channel_name - and not sensor_name.startswith(channel_name) - ): - conf[CONF_NAME] = f"{channel_name} {sensor_name}" - - -FINAL_VALIDATE_SCHEMA = final_validate - - async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) diff --git a/esphome/components/ads1115/ads1115.cpp b/esphome/components/ads1115/ads1115.cpp index 11a5663ed1..f4996cd3b1 100644 --- a/esphome/components/ads1115/ads1115.cpp +++ b/esphome/components/ads1115/ads1115.cpp @@ -10,7 +10,6 @@ static const uint8_t ADS1115_REGISTER_CONVERSION = 0x00; static const uint8_t ADS1115_REGISTER_CONFIG = 0x01; void ADS1115Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint16_t value; if (!this->read_byte_16(ADS1115_REGISTER_CONVERSION, &value)) { this->mark_failed(); diff --git a/esphome/components/ads1118/ads1118.cpp b/esphome/components/ads1118/ads1118.cpp index 1daa8fdfd4..f7db9f93dd 100644 --- a/esphome/components/ads1118/ads1118.cpp +++ b/esphome/components/ads1118/ads1118.cpp @@ -9,7 +9,6 @@ static const char *const TAG = "ads1118"; static const uint8_t ADS1118_DATA_RATE_860_SPS = 0b111; void ADS1118::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); this->config_ = 0; diff --git a/esphome/components/ags10/ags10.cpp b/esphome/components/ags10/ags10.cpp index 797a07afa5..fa7170114c 100644 --- a/esphome/components/ags10/ags10.cpp +++ b/esphome/components/ags10/ags10.cpp @@ -24,8 +24,6 @@ static const uint16_t ZP_CURRENT = 0x0000; static const uint16_t ZP_DEFAULT = 0xFFFF; void AGS10Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - auto version = this->read_version_(); if (version) { ESP_LOGD(TAG, "AGS10 Sensor Version: 0x%02X", *version); @@ -45,8 +43,6 @@ void AGS10Component::setup() { } else { ESP_LOGE(TAG, "AGS10 Sensor Resistance: unknown"); } - - ESP_LOGD(TAG, "Sensor initialized"); } void AGS10Component::update() { @@ -93,7 +89,7 @@ void AGS10Component::dump_config() { bool AGS10Component::new_i2c_address(uint8_t newaddress) { uint8_t rev_newaddress = ~newaddress; std::array data{newaddress, rev_newaddress, newaddress, rev_newaddress, 0}; - data[4] = calc_crc8_(data, 4); + data[4] = crc8(data.data(), 4, 0xFF, 0x31, true); if (!this->write_bytes(REG_ADDRESS, data)) { this->error_code_ = COMMUNICATION_FAILED; this->status_set_warning(); @@ -113,7 +109,7 @@ bool AGS10Component::set_zero_point_with_current_resistance() { return this->set bool AGS10Component::set_zero_point_with(uint16_t value) { std::array data{0x00, 0x0C, (uint8_t) ((value >> 8) & 0xFF), (uint8_t) (value & 0xFF), 0}; - data[4] = calc_crc8_(data, 4); + data[4] = crc8(data.data(), 4, 0xFF, 0x31, true); if (!this->write_bytes(REG_CALIBRATION, data)) { this->error_code_ = COMMUNICATION_FAILED; this->status_set_warning(); @@ -188,7 +184,7 @@ template optional> AGS10Component::read_and_che auto res = *data; auto crc_byte = res[len]; - if (crc_byte != calc_crc8_(res, len)) { + if (crc_byte != crc8(res.data(), len, 0xFF, 0x31, true)) { this->error_code_ = CRC_CHECK_FAILED; ESP_LOGE(TAG, "Reading AGS10 version failed: crc error!"); return optional>(); @@ -196,20 +192,5 @@ template optional> AGS10Component::read_and_che return data; } - -template uint8_t AGS10Component::calc_crc8_(std::array dat, uint8_t num) { - uint8_t i, byte1, crc = 0xFF; - for (byte1 = 0; byte1 < num; byte1++) { - crc ^= (dat[byte1]); - for (i = 0; i < 8; i++) { - if (crc & 0x80) { - crc = (crc << 1) ^ 0x31; - } else { - crc = (crc << 1); - } - } - } - return crc; -} } // namespace ags10 } // namespace esphome diff --git a/esphome/components/ags10/ags10.h b/esphome/components/ags10/ags10.h index 3e184ae176..e0975f14bc 100644 --- a/esphome/components/ags10/ags10.h +++ b/esphome/components/ags10/ags10.h @@ -1,9 +1,9 @@ #pragma once +#include "esphome/components/i2c/i2c.h" +#include "esphome/components/sensor/sensor.h" #include "esphome/core/automation.h" #include "esphome/core/component.h" -#include "esphome/components/sensor/sensor.h" -#include "esphome/components/i2c/i2c.h" namespace esphome { namespace ags10 { @@ -99,16 +99,6 @@ class AGS10Component : public PollingComponent, public i2c::I2CDevice { * Read, checks and returns data from the sensor. */ template optional> read_and_check_(uint8_t a_register); - - /** - * Calculates CRC8 value. - * - * CRC8 calculation, initial value: 0xFF, polynomial: 0x31 (x8+ x5+ x4+1) - * - * @param[in] dat the data buffer - * @param num number of bytes in the buffer - */ - template uint8_t calc_crc8_(std::array dat, uint8_t num); }; template class AGS10NewI2cAddressAction : public Action, public Parented { diff --git a/esphome/components/aht10/aht10.cpp b/esphome/components/aht10/aht10.cpp index 7f17e1c0d6..53c712a7a7 100644 --- a/esphome/components/aht10/aht10.cpp +++ b/esphome/components/aht10/aht10.cpp @@ -38,8 +38,6 @@ static const uint8_t AHT10_STATUS_BUSY = 0x80; static const float AHT10_DIVISOR = 1048576.0f; // 2^20, used for temperature and humidity calculations void AHT10Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - if (this->write(AHT10_SOFTRESET_CMD, sizeof(AHT10_SOFTRESET_CMD)) != i2c::ERROR_OK) { ESP_LOGE(TAG, "Reset failed"); } @@ -80,8 +78,6 @@ void AHT10Component::setup() { this->mark_failed(); return; } - - ESP_LOGV(TAG, "Initialization complete"); } void AHT10Component::restart_read_() { @@ -100,7 +96,7 @@ void AHT10Component::read_data_() { ESP_LOGD(TAG, "Read attempt %d at %ums", this->read_count_, (unsigned) (millis() - this->start_time_)); } if (this->read(data, 6) != i2c::ERROR_OK) { - this->status_set_warning("Read failed, will retry"); + this->status_set_warning(LOG_STR("Read failed, will retry")); this->restart_read_(); return; } @@ -117,7 +113,7 @@ void AHT10Component::read_data_() { } else { ESP_LOGD(TAG, "Invalid humidity, retrying"); if (this->write(AHT10_MEASURE_CMD, sizeof(AHT10_MEASURE_CMD)) != i2c::ERROR_OK) { - this->status_set_warning(ESP_LOG_MSG_COMM_FAIL); + this->status_set_warning(LOG_STR(ESP_LOG_MSG_COMM_FAIL)); } this->restart_read_(); return; @@ -148,7 +144,7 @@ void AHT10Component::update() { return; this->start_time_ = millis(); if (this->write(AHT10_MEASURE_CMD, sizeof(AHT10_MEASURE_CMD)) != i2c::ERROR_OK) { - this->status_set_warning(ESP_LOG_MSG_COMM_FAIL); + this->status_set_warning(LOG_STR(ESP_LOG_MSG_COMM_FAIL)); return; } this->restart_read_(); diff --git a/esphome/components/aic3204/aic3204.cpp b/esphome/components/aic3204/aic3204.cpp index a004fb42ce..e1acf32f83 100644 --- a/esphome/components/aic3204/aic3204.cpp +++ b/esphome/components/aic3204/aic3204.cpp @@ -17,8 +17,6 @@ static const char *const TAG = "aic3204"; } void AIC3204::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // Set register page to 0 ERROR_CHECK(this->write_byte(AIC3204_PAGE_CTRL, 0x00), "Set page 0 failed"); // Initiate SW reset (PLL is powered off as part of reset) diff --git a/esphome/components/airthings_ble/__init__.py b/esphome/components/airthings_ble/__init__.py index eae400ab39..1545110798 100644 --- a/esphome/components/airthings_ble/__init__.py +++ b/esphome/components/airthings_ble/__init__.py @@ -18,6 +18,6 @@ CONFIG_SCHEMA = cv.Schema( ).extend(esp32_ble_tracker.ESP_BLE_DEVICE_SCHEMA) -def to_code(config): +async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) - yield esp32_ble_tracker.register_ble_device(var, config) + await esp32_ble_tracker.register_ble_device(var, config) diff --git a/esphome/components/alarm_control_panel/__init__.py b/esphome/components/alarm_control_panel/__init__.py index 6d37d53a4c..174a9d9e0a 100644 --- a/esphome/components/alarm_control_panel/__init__.py +++ b/esphome/components/alarm_control_panel/__init__.py @@ -13,7 +13,7 @@ from esphome.const import ( CONF_TRIGGER_ID, CONF_WEB_SERVER, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass @@ -301,8 +301,7 @@ async def alarm_action_disarm_to_code(config, action_id, template_arg, args): ) async def alarm_action_pending_to_code(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + return cg.new_Pvariable(action_id, template_arg, paren) @automation.register_action( @@ -310,8 +309,7 @@ async def alarm_action_pending_to_code(config, action_id, template_arg, args): ) async def alarm_action_trigger_to_code(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + return cg.new_Pvariable(action_id, template_arg, paren) @automation.register_action( @@ -319,8 +317,7 @@ async def alarm_action_trigger_to_code(config, action_id, template_arg, args): ) async def alarm_action_chime_to_code(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + return cg.new_Pvariable(action_id, template_arg, paren) @automation.register_action( @@ -333,8 +330,7 @@ async def alarm_action_chime_to_code(config, action_id, template_arg, args): ) async def alarm_action_ready_to_code(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + return cg.new_Pvariable(action_id, template_arg, paren) @automation.register_condition( @@ -349,7 +345,6 @@ async def alarm_control_panel_is_armed_to_code( return cg.new_Pvariable(condition_id, template_arg, paren) -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): cg.add_global(alarm_control_panel_ns.using) - cg.add_define("USE_ALARM_CONTROL_PANEL") diff --git a/esphome/components/am2315c/am2315c.cpp b/esphome/components/am2315c/am2315c.cpp index cea5263fd6..b20a8c6cbb 100644 --- a/esphome/components/am2315c/am2315c.cpp +++ b/esphome/components/am2315c/am2315c.cpp @@ -29,22 +29,6 @@ namespace am2315c { static const char *const TAG = "am2315c"; -uint8_t AM2315C::crc8_(uint8_t *data, uint8_t len) { - uint8_t crc = 0xFF; - while (len--) { - crc ^= *data++; - for (uint8_t i = 0; i < 8; i++) { - if (crc & 0x80) { - crc <<= 1; - crc ^= 0x31; - } else { - crc <<= 1; - } - } - } - return crc; -} - bool AM2315C::reset_register_(uint8_t reg) { // code based on demo code sent by www.aosong.com // no further documentation. @@ -86,12 +70,10 @@ bool AM2315C::convert_(uint8_t *data, float &humidity, float &temperature) { humidity = raw * 9.5367431640625e-5; raw = ((data[3] & 0x0F) << 16) | (data[4] << 8) | data[5]; temperature = raw * 1.9073486328125e-4 - 50; - return this->crc8_(data, 6) == data[6]; + return crc8(data, 6, 0xFF, 0x31, true) == data[6]; } void AM2315C::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // get status uint8_t status = 0; if (this->read(&status, 1) != i2c::ERROR_OK) { diff --git a/esphome/components/am2315c/am2315c.h b/esphome/components/am2315c/am2315c.h index 9cec40e4c2..c8d01beeaa 100644 --- a/esphome/components/am2315c/am2315c.h +++ b/esphome/components/am2315c/am2315c.h @@ -21,9 +21,9 @@ // SOFTWARE. #pragma once -#include "esphome/core/component.h" -#include "esphome/components/sensor/sensor.h" #include "esphome/components/i2c/i2c.h" +#include "esphome/components/sensor/sensor.h" +#include "esphome/core/component.h" namespace esphome { namespace am2315c { @@ -39,7 +39,6 @@ class AM2315C : public PollingComponent, public i2c::I2CDevice { void set_humidity_sensor(sensor::Sensor *humidity_sensor) { this->humidity_sensor_ = humidity_sensor; } protected: - uint8_t crc8_(uint8_t *data, uint8_t len); bool convert_(uint8_t *data, float &humidity, float &temperature); bool reset_register_(uint8_t reg); diff --git a/esphome/components/am2320/am2320.cpp b/esphome/components/am2320/am2320.cpp index 6400ecef4b..055be2aeee 100644 --- a/esphome/components/am2320/am2320.cpp +++ b/esphome/components/am2320/am2320.cpp @@ -34,7 +34,6 @@ void AM2320Component::update() { this->status_clear_warning(); } void AM2320Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t data[8]; data[0] = 0; data[1] = 4; diff --git a/esphome/components/animation/__init__.py b/esphome/components/animation/__init__.py index f73b8ef08f..c4ac7adb23 100644 --- a/esphome/components/animation/__init__.py +++ b/esphome/components/animation/__init__.py @@ -34,17 +34,20 @@ SetFrameAction = animation_ns.class_( "AnimationSetFrameAction", automation.Action, cg.Parented.template(Animation_) ) -CONFIG_SCHEMA = espImage.IMAGE_SCHEMA.extend( - { - cv.Required(CONF_ID): cv.declare_id(Animation_), - cv.Optional(CONF_LOOP): cv.All( - { - cv.Optional(CONF_START_FRAME, default=0): cv.positive_int, - cv.Optional(CONF_END_FRAME): cv.positive_int, - cv.Optional(CONF_REPEAT): cv.positive_int, - } - ), - }, +CONFIG_SCHEMA = cv.All( + espImage.IMAGE_SCHEMA.extend( + { + cv.Required(CONF_ID): cv.declare_id(Animation_), + cv.Optional(CONF_LOOP): cv.All( + { + cv.Optional(CONF_START_FRAME, default=0): cv.positive_int, + cv.Optional(CONF_END_FRAME): cv.positive_int, + cv.Optional(CONF_REPEAT): cv.positive_int, + } + ), + }, + ), + espImage.validate_settings, ) diff --git a/esphome/components/animation/animation.cpp b/esphome/components/animation/animation.cpp index 6db6f1a7bd..c2ae3b2f76 100644 --- a/esphome/components/animation/animation.cpp +++ b/esphome/components/animation/animation.cpp @@ -26,12 +26,12 @@ uint32_t Animation::get_animation_frame_count() const { return this->animation_f int Animation::get_current_frame() const { return this->current_frame_; } void Animation::next_frame() { this->current_frame_++; - if (loop_count_ && this->current_frame_ == loop_end_frame_ && + if (loop_count_ && static_cast(this->current_frame_) == loop_end_frame_ && (this->loop_current_iteration_ < loop_count_ || loop_count_ < 0)) { this->current_frame_ = loop_start_frame_; this->loop_current_iteration_++; } - if (this->current_frame_ >= animation_frame_count_) { + if (static_cast(this->current_frame_) >= animation_frame_count_) { this->loop_current_iteration_ = 1; this->current_frame_ = 0; } diff --git a/esphome/components/apds9306/apds9306.cpp b/esphome/components/apds9306/apds9306.cpp index 9799f54d3d..fb3adde868 100644 --- a/esphome/components/apds9306/apds9306.cpp +++ b/esphome/components/apds9306/apds9306.cpp @@ -54,8 +54,6 @@ enum { // APDS9306 registers } void APDS9306::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - uint8_t id; if (!this->read_byte(APDS9306_PART_ID, &id)) { // Part ID register this->error_code_ = COMMUNICATION_FAILED; @@ -86,8 +84,6 @@ void APDS9306::setup() { // Set to active mode APDS9306_WRITE_BYTE(APDS9306_MAIN_CTRL, 0x02); - - ESP_LOGCONFIG(TAG, "APDS9306 setup complete"); } void APDS9306::dump_config() { diff --git a/esphome/components/apds9960/apds9960.cpp b/esphome/components/apds9960/apds9960.cpp index b736e6b8b0..93038d3160 100644 --- a/esphome/components/apds9960/apds9960.cpp +++ b/esphome/components/apds9960/apds9960.cpp @@ -15,7 +15,6 @@ static const char *const TAG = "apds9960"; #define APDS9960_WRITE_BYTE(reg, value) APDS9960_ERROR_CHECK(this->write_byte(reg, value)); void APDS9960::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t id; if (!this->read_byte(0x92, &id)) { // ID register this->error_code_ = COMMUNICATION_FAILED; diff --git a/esphome/components/api/__init__.py b/esphome/components/api/__init__.py index 5b302760b1..e8dacf51bc 100644 --- a/esphome/components/api/__init__.py +++ b/esphome/components/api/__init__.py @@ -1,4 +1,5 @@ import base64 +import logging from esphome import automation from esphome.automation import Condition @@ -8,34 +9,59 @@ import esphome.config_validation as cv from esphome.const import ( CONF_ACTION, CONF_ACTIONS, + CONF_CAPTURE_RESPONSE, CONF_DATA, CONF_DATA_TEMPLATE, CONF_EVENT, CONF_ID, CONF_KEY, + CONF_MAX_CONNECTIONS, CONF_ON_CLIENT_CONNECTED, CONF_ON_CLIENT_DISCONNECTED, + CONF_ON_ERROR, + CONF_ON_SUCCESS, CONF_PASSWORD, CONF_PORT, CONF_REBOOT_TIMEOUT, + CONF_RESPONSE_TEMPLATE, CONF_SERVICE, CONF_SERVICES, CONF_TAG, CONF_TRIGGER_ID, CONF_VARIABLES, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, ID, CoroPriority, coroutine_with_priority +from esphome.cpp_generator import TemplateArgsType +from esphome.types import ConfigType + +_LOGGER = logging.getLogger(__name__) DOMAIN = "api" DEPENDENCIES = ["network"] -AUTO_LOAD = ["socket"] -CODEOWNERS = ["@OttoWinter"] +CODEOWNERS = ["@esphome/core"] + + +def AUTO_LOAD(config: ConfigType) -> list[str]: + """Conditionally auto-load json only when capture_response is used.""" + base = ["socket"] + + # Check if any homeassistant.action/homeassistant.service has capture_response: true + # This flag is set during config validation in _validate_response_config + if not config or CORE.data.get(DOMAIN, {}).get(CONF_CAPTURE_RESPONSE, False): + return base + ["json"] + + return base + api_ns = cg.esphome_ns.namespace("api") APIServer = api_ns.class_("APIServer", cg.Component, cg.Controller) HomeAssistantServiceCallAction = api_ns.class_( "HomeAssistantServiceCallAction", automation.Action ) +ActionResponse = api_ns.class_("ActionResponse") +HomeAssistantActionResponseTrigger = api_ns.class_( + "HomeAssistantActionResponseTrigger", automation.Trigger +) APIConnectedCondition = api_ns.class_("APIConnectedCondition", Condition) UserServiceTrigger = api_ns.class_("UserServiceTrigger", automation.Trigger) @@ -53,6 +79,10 @@ SERVICE_ARG_NATIVE_TYPES = { CONF_ENCRYPTION = "encryption" CONF_BATCH_DELAY = "batch_delay" CONF_CUSTOM_SERVICES = "custom_services" +CONF_HOMEASSISTANT_SERVICES = "homeassistant_services" +CONF_HOMEASSISTANT_STATES = "homeassistant_states" +CONF_LISTEN_BACKLOG = "listen_backlog" +CONF_MAX_SEND_QUEUE = "max_send_queue" def validate_encryption_key(value): @@ -99,6 +129,32 @@ def _encryption_schema(config): return ENCRYPTION_SCHEMA(config) +def _validate_api_config(config: ConfigType) -> ConfigType: + """Validate API configuration with mutual exclusivity check and deprecation warning.""" + # Check if both password and encryption are configured + has_password = CONF_PASSWORD in config and config[CONF_PASSWORD] + has_encryption = CONF_ENCRYPTION in config + + if has_password and has_encryption: + raise cv.Invalid( + "The 'password' and 'encryption' options are mutually exclusive. " + "The API client only supports one authentication method at a time. " + "Please remove one of them. " + "Note: 'password' authentication is deprecated and will be removed in version 2026.1.0. " + "We strongly recommend using 'encryption' instead for better security." + ) + + # Warn about password deprecation + if has_password: + _LOGGER.warning( + "API 'password' authentication has been deprecated since May 2022 and will be removed in version 2026.1.0. " + "Please migrate to the 'encryption' configuration. " + "See https://esphome.io/components/api.html#configuration-variables" + ) + + return config + + CONFIG_SCHEMA = cv.All( cv.Schema( { @@ -118,19 +174,58 @@ CONFIG_SCHEMA = cv.All( cv.Range(max=cv.TimePeriod(milliseconds=65535)), ), cv.Optional(CONF_CUSTOM_SERVICES, default=False): cv.boolean, + cv.Optional(CONF_HOMEASSISTANT_SERVICES, default=False): cv.boolean, + cv.Optional(CONF_HOMEASSISTANT_STATES, default=False): cv.boolean, cv.Optional(CONF_ON_CLIENT_CONNECTED): automation.validate_automation( single=True ), cv.Optional(CONF_ON_CLIENT_DISCONNECTED): automation.validate_automation( single=True ), + # Connection limits to prevent memory exhaustion on resource-constrained devices + # Each connection uses ~500-1000 bytes of RAM plus system resources + # Platform defaults based on available RAM and network stack implementation: + cv.SplitDefault( + CONF_LISTEN_BACKLOG, + esp8266=1, # Limited RAM (~40KB free), LWIP raw sockets + esp32=4, # More RAM (520KB), BSD sockets + rp2040=1, # Limited RAM (264KB), LWIP raw sockets like ESP8266 + bk72xx=4, # Moderate RAM, BSD-style sockets + rtl87xx=4, # Moderate RAM, BSD-style sockets + host=4, # Abundant resources + ln882x=4, # Moderate RAM + ): cv.int_range(min=1, max=10), + cv.SplitDefault( + CONF_MAX_CONNECTIONS, + esp8266=4, # ~40KB free RAM, each connection uses ~500-1000 bytes + esp32=8, # 520KB RAM available + rp2040=4, # 264KB RAM but LWIP constraints + bk72xx=8, # Moderate RAM + rtl87xx=8, # Moderate RAM + host=8, # Abundant resources + ln882x=8, # Moderate RAM + ): cv.int_range(min=1, max=20), + # Maximum queued send buffers per connection before dropping connection + # Each buffer uses ~8-12 bytes overhead plus actual message size + # Platform defaults based on available RAM and typical message rates: + cv.SplitDefault( + CONF_MAX_SEND_QUEUE, + esp8266=5, # Limited RAM, need to fail fast + esp32=8, # More RAM, can buffer more + rp2040=5, # Limited RAM + bk72xx=8, # Moderate RAM + rtl87xx=8, # Moderate RAM + host=16, # Abundant resources + ln882x=8, # Moderate RAM + ): cv.int_range(min=1, max=64), } ).extend(cv.COMPONENT_SCHEMA), cv.rename_key(CONF_SERVICES, CONF_ACTIONS), + _validate_api_config, ) -@coroutine_with_priority(40.0) +@coroutine_with_priority(CoroPriority.WEB) async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) @@ -141,11 +236,22 @@ async def to_code(config): cg.add(var.set_password(config[CONF_PASSWORD])) cg.add(var.set_reboot_timeout(config[CONF_REBOOT_TIMEOUT])) cg.add(var.set_batch_delay(config[CONF_BATCH_DELAY])) + if CONF_LISTEN_BACKLOG in config: + cg.add(var.set_listen_backlog(config[CONF_LISTEN_BACKLOG])) + if CONF_MAX_CONNECTIONS in config: + cg.add(var.set_max_connections(config[CONF_MAX_CONNECTIONS])) + cg.add_define("API_MAX_SEND_QUEUE", config[CONF_MAX_SEND_QUEUE]) # Set USE_API_SERVICES if any services are enabled if config.get(CONF_ACTIONS) or config[CONF_CUSTOM_SERVICES]: cg.add_define("USE_API_SERVICES") + if config[CONF_HOMEASSISTANT_SERVICES]: + cg.add_define("USE_API_HOMEASSISTANT_SERVICES") + + if config[CONF_HOMEASSISTANT_STATES]: + cg.add_define("USE_API_HOMEASSISTANT_STATES") + if actions := config.get(CONF_ACTIONS, []): for conf in actions: template_args = [] @@ -183,6 +289,7 @@ async def to_code(config): if key := encryption_config.get(CONF_KEY): decoded = base64.b64decode(key) cg.add(var.set_noise_psk(list(decoded))) + cg.add_define("USE_API_NOISE_PSK_FROM_YAML") else: # No key provided, but encryption desired # This will allow a plaintext client to provide a noise key, @@ -202,6 +309,29 @@ async def to_code(config): KEY_VALUE_SCHEMA = cv.Schema({cv.string: cv.templatable(cv.string_strict)}) +def _validate_response_config(config: ConfigType) -> ConfigType: + # Validate dependencies: + # - response_template requires capture_response: true + # - capture_response: true requires on_success + if CONF_RESPONSE_TEMPLATE in config and not config[CONF_CAPTURE_RESPONSE]: + raise cv.Invalid( + f"`{CONF_RESPONSE_TEMPLATE}` requires `{CONF_CAPTURE_RESPONSE}: true` to be set.", + path=[CONF_RESPONSE_TEMPLATE], + ) + + if config[CONF_CAPTURE_RESPONSE] and CONF_ON_SUCCESS not in config: + raise cv.Invalid( + f"`{CONF_CAPTURE_RESPONSE}: true` requires `{CONF_ON_SUCCESS}` to be set.", + path=[CONF_CAPTURE_RESPONSE], + ) + + # Track if any action uses capture_response for AUTO_LOAD + if config[CONF_CAPTURE_RESPONSE]: + CORE.data.setdefault(DOMAIN, {})[CONF_CAPTURE_RESPONSE] = True + + return config + + HOMEASSISTANT_ACTION_ACTION_SCHEMA = cv.All( cv.Schema( { @@ -217,10 +347,15 @@ HOMEASSISTANT_ACTION_ACTION_SCHEMA = cv.All( cv.Optional(CONF_VARIABLES, default={}): cv.Schema( {cv.string: cv.returning_lambda} ), + cv.Optional(CONF_RESPONSE_TEMPLATE): cv.templatable(cv.string), + cv.Optional(CONF_CAPTURE_RESPONSE, default=False): cv.boolean, + cv.Optional(CONF_ON_SUCCESS): automation.validate_automation(single=True), + cv.Optional(CONF_ON_ERROR): automation.validate_automation(single=True), } ), cv.has_exactly_one_key(CONF_SERVICE, CONF_ACTION), cv.rename_key(CONF_SERVICE, CONF_ACTION), + _validate_response_config, ) @@ -234,20 +369,67 @@ HOMEASSISTANT_ACTION_ACTION_SCHEMA = cv.All( HomeAssistantServiceCallAction, HOMEASSISTANT_ACTION_ACTION_SCHEMA, ) -async def homeassistant_service_to_code(config, action_id, template_arg, args): +async def homeassistant_service_to_code( + config: ConfigType, + action_id: ID, + template_arg: cg.TemplateArguments, + args: TemplateArgsType, +): + cg.add_define("USE_API_HOMEASSISTANT_SERVICES") serv = await cg.get_variable(config[CONF_ID]) var = cg.new_Pvariable(action_id, template_arg, serv, False) templ = await cg.templatable(config[CONF_ACTION], args, None) cg.add(var.set_service(templ)) + + # Initialize FixedVectors with exact sizes from config + cg.add(var.init_data(len(config[CONF_DATA]))) for key, value in config[CONF_DATA].items(): templ = await cg.templatable(value, args, None) cg.add(var.add_data(key, templ)) + + cg.add(var.init_data_template(len(config[CONF_DATA_TEMPLATE]))) for key, value in config[CONF_DATA_TEMPLATE].items(): templ = await cg.templatable(value, args, None) cg.add(var.add_data_template(key, templ)) + + cg.add(var.init_variables(len(config[CONF_VARIABLES]))) for key, value in config[CONF_VARIABLES].items(): templ = await cg.templatable(value, args, None) cg.add(var.add_variable(key, templ)) + + if on_error := config.get(CONF_ON_ERROR): + cg.add_define("USE_API_HOMEASSISTANT_ACTION_RESPONSES") + cg.add_define("USE_API_HOMEASSISTANT_ACTION_RESPONSES_ERRORS") + cg.add(var.set_wants_status()) + await automation.build_automation( + var.get_error_trigger(), + [(cg.std_string, "error"), *args], + on_error, + ) + + if on_success := config.get(CONF_ON_SUCCESS): + cg.add_define("USE_API_HOMEASSISTANT_ACTION_RESPONSES") + cg.add(var.set_wants_status()) + if config[CONF_CAPTURE_RESPONSE]: + cg.add(var.set_wants_response()) + cg.add_define("USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON") + await automation.build_automation( + var.get_success_trigger_with_response(), + [(cg.JsonObjectConst, "response"), *args], + on_success, + ) + + if response_template := config.get(CONF_RESPONSE_TEMPLATE): + templ = await cg.templatable(response_template, args, cg.std_string) + cg.add(var.set_response_template(templ)) + + else: + await automation.build_automation( + var.get_success_trigger(), + args, + on_success, + ) + return var @@ -278,19 +460,28 @@ HOMEASSISTANT_EVENT_ACTION_SCHEMA = cv.Schema( HOMEASSISTANT_EVENT_ACTION_SCHEMA, ) async def homeassistant_event_to_code(config, action_id, template_arg, args): + cg.add_define("USE_API_HOMEASSISTANT_SERVICES") serv = await cg.get_variable(config[CONF_ID]) var = cg.new_Pvariable(action_id, template_arg, serv, True) templ = await cg.templatable(config[CONF_EVENT], args, None) cg.add(var.set_service(templ)) + + # Initialize FixedVectors with exact sizes from config + cg.add(var.init_data(len(config[CONF_DATA]))) for key, value in config[CONF_DATA].items(): templ = await cg.templatable(value, args, None) cg.add(var.add_data(key, templ)) + + cg.add(var.init_data_template(len(config[CONF_DATA_TEMPLATE]))) for key, value in config[CONF_DATA_TEMPLATE].items(): templ = await cg.templatable(value, args, None) cg.add(var.add_data_template(key, templ)) + + cg.add(var.init_variables(len(config[CONF_VARIABLES]))) for key, value in config[CONF_VARIABLES].items(): templ = await cg.templatable(value, args, None) cg.add(var.add_variable(key, templ)) + return var @@ -309,9 +500,12 @@ HOMEASSISTANT_TAG_SCANNED_ACTION_SCHEMA = cv.maybe_simple_value( HOMEASSISTANT_TAG_SCANNED_ACTION_SCHEMA, ) async def homeassistant_tag_scanned_to_code(config, action_id, template_arg, args): + cg.add_define("USE_API_HOMEASSISTANT_SERVICES") serv = await cg.get_variable(config[CONF_ID]) var = cg.new_Pvariable(action_id, template_arg, serv, True) cg.add(var.set_service("esphome.tag_scanned")) + # Initialize FixedVector with exact size (1 data field) + cg.add(var.init_data(1)) templ = await cg.templatable(config[CONF_TAG], args, cg.std_string) cg.add(var.add_data("tag_id", templ)) return var @@ -323,9 +517,10 @@ async def api_connected_to_code(config, condition_id, template_arg, args): def FILTER_SOURCE_FILES() -> list[str]: - """Filter out api_pb2_dump.cpp when proto message dumping is not enabled - and user_services.cpp when no services are defined.""" - files_to_filter = [] + """Filter out api_pb2_dump.cpp when proto message dumping is not enabled, + user_services.cpp when no services are defined, and protocol-specific + implementations based on encryption configuration.""" + files_to_filter: list[str] = [] # api_pb2_dump.cpp is only needed when HAS_PROTO_MESSAGE_DUMP is defined # This is a particularly large file that still needs to be opened and read @@ -341,4 +536,16 @@ def FILTER_SOURCE_FILES() -> list[str]: if config and not config.get(CONF_ACTIONS) and not config[CONF_CUSTOM_SERVICES]: files_to_filter.append("user_services.cpp") + # Filter protocol-specific implementations based on encryption configuration + encryption_config = config.get(CONF_ENCRYPTION) if config else None + + # If encryption is not configured at all, we only need plaintext + if encryption_config is None: + files_to_filter.append("api_frame_helper_noise.cpp") + # If encryption is configured with a key, we only need noise + elif encryption_config.get(CONF_KEY): + files_to_filter.append("api_frame_helper_plaintext.cpp") + # If encryption is configured but no key is provided, we need both + # (this allows a plaintext client to provide a noise key) + return files_to_filter diff --git a/esphome/components/api/api.proto b/esphome/components/api/api.proto index 861b3471d7..753adc3592 100644 --- a/esphome/components/api/api.proto +++ b/esphome/components/api/api.proto @@ -7,7 +7,7 @@ service APIConnection { option (needs_setup_connection) = false; option (needs_authentication) = false; } - rpc connect (ConnectRequest) returns (ConnectResponse) { + rpc authenticate (AuthenticationRequest) returns (AuthenticationResponse) { option (needs_setup_connection) = false; option (needs_authentication) = false; } @@ -27,9 +27,6 @@ service APIConnection { rpc subscribe_logs (SubscribeLogsRequest) returns (void) {} rpc subscribe_homeassistant_services (SubscribeHomeassistantServicesRequest) returns (void) {} rpc subscribe_home_assistant_states (SubscribeHomeAssistantStatesRequest) returns (void) {} - rpc get_time (GetTimeRequest) returns (GetTimeResponse) { - option (needs_authentication) = false; - } rpc execute_service (ExecuteServiceRequest) returns (void) {} rpc noise_encryption_set_key (NoiseEncryptionSetKeyRequest) returns (NoiseEncryptionSetKeyResponse) {} @@ -69,6 +66,9 @@ service APIConnection { rpc voice_assistant_set_configuration(VoiceAssistantSetConfiguration) returns (void) {} rpc alarm_control_panel_command (AlarmControlPanelCommandRequest) returns (void) {} + + rpc zwave_proxy_frame(ZWaveProxyFrame) returns (void) {} + rpc zwave_proxy_request(ZWaveProxyRequest) returns (void) {} } @@ -102,7 +102,7 @@ message HelloRequest { // For example "Home Assistant" // Not strictly necessary to send but nice for debugging // purposes. - string client_info = 1; + string client_info = 1 [(pointer_to_buffer) = true]; uint32 api_version_major = 2; uint32 api_version_minor = 3; } @@ -132,21 +132,23 @@ message HelloResponse { // Message sent at the beginning of each connection to authenticate the client // Can only be sent by the client and only at the beginning of the connection -message ConnectRequest { +message AuthenticationRequest { option (id) = 3; option (source) = SOURCE_CLIENT; option (no_delay) = true; + option (ifdef) = "USE_API_PASSWORD"; // The password to log in with - string password = 1; + string password = 1 [(pointer_to_buffer) = true]; } // Confirmation of successful connection. After this the connection is available for all traffic. // Can only be sent by the server and only at the beginning of the connection -message ConnectResponse { +message AuthenticationResponse { option (id) = 4; option (source) = SOURCE_SERVER; option (no_delay) = true; + option (ifdef) = "USE_API_PASSWORD"; bool invalid_password = 1; } @@ -203,7 +205,7 @@ message DeviceInfoResponse { option (id) = 10; option (source) = SOURCE_SERVER; - bool uses_password = 1; + bool uses_password = 1 [(field_ifdef) = "USE_API_PASSWORD"]; // The name of the node, given by "App.set_name()" string name = 2; @@ -222,37 +224,43 @@ message DeviceInfoResponse { // The model of the board. For example NodeMCU string model = 6; - bool has_deep_sleep = 7; + bool has_deep_sleep = 7 [(field_ifdef) = "USE_DEEP_SLEEP"]; // The esphome project details if set - string project_name = 8; - string project_version = 9; + string project_name = 8 [(field_ifdef) = "ESPHOME_PROJECT_NAME"]; + string project_version = 9 [(field_ifdef) = "ESPHOME_PROJECT_NAME"]; - uint32 webserver_port = 10; + uint32 webserver_port = 10 [(field_ifdef) = "USE_WEBSERVER"]; - uint32 legacy_bluetooth_proxy_version = 11; - uint32 bluetooth_proxy_feature_flags = 15; + // Deprecated in API version 1.9 + uint32 legacy_bluetooth_proxy_version = 11 [deprecated=true, (field_ifdef) = "USE_BLUETOOTH_PROXY"]; + uint32 bluetooth_proxy_feature_flags = 15 [(field_ifdef) = "USE_BLUETOOTH_PROXY"]; string manufacturer = 12; string friendly_name = 13; - uint32 legacy_voice_assistant_version = 14; - uint32 voice_assistant_feature_flags = 17; + // Deprecated in API version 1.10 + uint32 legacy_voice_assistant_version = 14 [deprecated=true, (field_ifdef) = "USE_VOICE_ASSISTANT"]; + uint32 voice_assistant_feature_flags = 17 [(field_ifdef) = "USE_VOICE_ASSISTANT"]; - string suggested_area = 16; + string suggested_area = 16 [(field_ifdef) = "USE_AREAS"]; // The Bluetooth mac address of the device. For example "AC:BC:32:89:0E:AA" - string bluetooth_mac_address = 18; + string bluetooth_mac_address = 18 [(field_ifdef) = "USE_BLUETOOTH_PROXY"]; // Supports receiving and saving api encryption key - bool api_encryption_supported = 19; + bool api_encryption_supported = 19 [(field_ifdef) = "USE_API_NOISE"]; - repeated DeviceInfo devices = 20; - repeated AreaInfo areas = 21; + repeated DeviceInfo devices = 20 [(field_ifdef) = "USE_DEVICES", (fixed_array_size_define) = "ESPHOME_DEVICE_COUNT"]; + repeated AreaInfo areas = 21 [(field_ifdef) = "USE_AREAS", (fixed_array_size_define) = "ESPHOME_AREA_COUNT"]; // Top-level area info to phase out suggested_area - AreaInfo area = 22; + AreaInfo area = 22 [(field_ifdef) = "USE_AREAS"]; + + // Indicates if Z-Wave proxy support is available and features supported + uint32 zwave_proxy_feature_flags = 23 [(field_ifdef) = "USE_ZWAVE_PROXY"]; + uint32 zwave_home_id = 24 [(field_ifdef) = "USE_ZWAVE_PROXY"]; } message ListEntitiesRequest { @@ -290,14 +298,14 @@ message ListEntitiesBinarySensorResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id string device_class = 5; bool is_status_binary_sensor = 6; bool disabled_by_default = 7; - string icon = 8; + string icon = 8 [(field_ifdef) = "USE_ENTITY_ICON"]; EntityCategory entity_category = 9; - uint32 device_id = 10; + uint32 device_id = 10 [(field_ifdef) = "USE_DEVICES"]; } message BinarySensorStateResponse { option (id) = 21; @@ -311,7 +319,7 @@ message BinarySensorStateResponse { // If the binary sensor does not have a valid state yet. // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller bool missing_state = 3; - uint32 device_id = 4; + uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"]; } // ==================== COVER ==================== @@ -324,20 +332,22 @@ message ListEntitiesCoverResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id bool assumed_state = 5; bool supports_position = 6; bool supports_tilt = 7; string device_class = 8; bool disabled_by_default = 9; - string icon = 10; + string icon = 10 [(field_ifdef) = "USE_ENTITY_ICON"]; EntityCategory entity_category = 11; bool supports_stop = 12; - uint32 device_id = 13; + uint32 device_id = 13 [(field_ifdef) = "USE_DEVICES"]; } +// Deprecated in API version 1.1 enum LegacyCoverState { + option deprecated = true; LEGACY_COVER_STATE_OPEN = 0; LEGACY_COVER_STATE_CLOSED = 1; } @@ -356,15 +366,18 @@ message CoverStateResponse { fixed32 key = 1; // legacy: state has been removed in 1.13 // clients/servers must still send/accept it until the next protocol change - LegacyCoverState legacy_state = 2; + // Deprecated in API version 1.1 + LegacyCoverState legacy_state = 2 [deprecated=true]; float position = 3; float tilt = 4; CoverOperation current_operation = 5; - uint32 device_id = 6; + uint32 device_id = 6 [(field_ifdef) = "USE_DEVICES"]; } +// Deprecated in API version 1.1 enum LegacyCoverCommand { + option deprecated = true; LEGACY_COVER_COMMAND_OPEN = 0; LEGACY_COVER_COMMAND_CLOSE = 1; LEGACY_COVER_COMMAND_STOP = 2; @@ -380,15 +393,17 @@ message CoverCommandRequest { // legacy: command has been removed in 1.13 // clients/servers must still send/accept it until the next protocol change - bool has_legacy_command = 2; - LegacyCoverCommand legacy_command = 3; + // Deprecated in API version 1.1 + bool has_legacy_command = 2 [deprecated=true]; + // Deprecated in API version 1.1 + LegacyCoverCommand legacy_command = 3 [deprecated=true]; bool has_position = 4; float position = 5; bool has_tilt = 6; float tilt = 7; bool stop = 8; - uint32 device_id = 9; + uint32 device_id = 9 [(field_ifdef) = "USE_DEVICES"]; } // ==================== FAN ==================== @@ -401,19 +416,21 @@ message ListEntitiesFanResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id bool supports_oscillation = 5; bool supports_speed = 6; bool supports_direction = 7; int32 supported_speed_count = 8; bool disabled_by_default = 9; - string icon = 10; + string icon = 10 [(field_ifdef) = "USE_ENTITY_ICON"]; EntityCategory entity_category = 11; - repeated string supported_preset_modes = 12; - uint32 device_id = 13; + repeated string supported_preset_modes = 12 [(container_pointer) = "std::set"]; + uint32 device_id = 13 [(field_ifdef) = "USE_DEVICES"]; } +// Deprecated in API version 1.6 - only used in deprecated fields enum FanSpeed { + option deprecated = true; FAN_SPEED_LOW = 0; FAN_SPEED_MEDIUM = 1; FAN_SPEED_HIGH = 2; @@ -432,11 +449,12 @@ message FanStateResponse { fixed32 key = 1; bool state = 2; bool oscillating = 3; - FanSpeed speed = 4 [deprecated = true]; + // Deprecated in API version 1.6 + FanSpeed speed = 4 [deprecated=true]; FanDirection direction = 5; int32 speed_level = 6; string preset_mode = 7; - uint32 device_id = 8; + uint32 device_id = 8 [(field_ifdef) = "USE_DEVICES"]; } message FanCommandRequest { option (id) = 31; @@ -448,8 +466,10 @@ message FanCommandRequest { fixed32 key = 1; bool has_state = 2; bool state = 3; - bool has_speed = 4 [deprecated = true]; - FanSpeed speed = 5 [deprecated = true]; + // Deprecated in API version 1.6 + bool has_speed = 4 [deprecated=true]; + // Deprecated in API version 1.6 + FanSpeed speed = 5 [deprecated=true]; bool has_oscillating = 6; bool oscillating = 7; bool has_direction = 8; @@ -458,7 +478,7 @@ message FanCommandRequest { int32 speed_level = 11; bool has_preset_mode = 12; string preset_mode = 13; - uint32 device_id = 14; + uint32 device_id = 14 [(field_ifdef) = "USE_DEVICES"]; } // ==================== LIGHT ==================== @@ -484,21 +504,25 @@ message ListEntitiesLightResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - repeated ColorMode supported_color_modes = 12; + repeated ColorMode supported_color_modes = 12 [(container_pointer) = "std::set"]; // next four supports_* are for legacy clients, newer clients should use color modes + // Deprecated in API version 1.6 bool legacy_supports_brightness = 5 [deprecated=true]; + // Deprecated in API version 1.6 bool legacy_supports_rgb = 6 [deprecated=true]; + // Deprecated in API version 1.6 bool legacy_supports_white_value = 7 [deprecated=true]; + // Deprecated in API version 1.6 bool legacy_supports_color_temperature = 8 [deprecated=true]; float min_mireds = 9; float max_mireds = 10; repeated string effects = 11; bool disabled_by_default = 13; - string icon = 14; + string icon = 14 [(field_ifdef) = "USE_ENTITY_ICON"]; EntityCategory entity_category = 15; - uint32 device_id = 16; + uint32 device_id = 16 [(field_ifdef) = "USE_DEVICES"]; } message LightStateResponse { option (id) = 24; @@ -520,7 +544,7 @@ message LightStateResponse { float cold_white = 12; float warm_white = 13; string effect = 9; - uint32 device_id = 14; + uint32 device_id = 14 [(field_ifdef) = "USE_DEVICES"]; } message LightCommandRequest { option (id) = 32; @@ -556,7 +580,7 @@ message LightCommandRequest { uint32 flash_length = 17; bool has_effect = 18; string effect = 19; - uint32 device_id = 28; + uint32 device_id = 28 [(field_ifdef) = "USE_DEVICES"]; } // ==================== SENSOR ==================== @@ -567,7 +591,9 @@ enum SensorStateClass { STATE_CLASS_TOTAL = 3; } +// Deprecated in API version 1.5 enum SensorLastResetType { + option deprecated = true; LAST_RESET_NONE = 0; LAST_RESET_NEVER = 1; LAST_RESET_AUTO = 2; @@ -582,19 +608,20 @@ message ListEntitiesSensorResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; string unit_of_measurement = 6; int32 accuracy_decimals = 7; bool force_update = 8; string device_class = 9; SensorStateClass state_class = 10; // Last reset type removed in 2021.9.0 - SensorLastResetType legacy_last_reset_type = 11; + // Deprecated in API version 1.5 + SensorLastResetType legacy_last_reset_type = 11 [deprecated=true]; bool disabled_by_default = 12; EntityCategory entity_category = 13; - uint32 device_id = 14; + uint32 device_id = 14 [(field_ifdef) = "USE_DEVICES"]; } message SensorStateResponse { option (id) = 25; @@ -608,7 +635,7 @@ message SensorStateResponse { // If the sensor does not have a valid state yet. // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller bool missing_state = 3; - uint32 device_id = 4; + uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"]; } // ==================== SWITCH ==================== @@ -621,14 +648,14 @@ message ListEntitiesSwitchResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool assumed_state = 6; bool disabled_by_default = 7; EntityCategory entity_category = 8; string device_class = 9; - uint32 device_id = 10; + uint32 device_id = 10 [(field_ifdef) = "USE_DEVICES"]; } message SwitchStateResponse { option (id) = 26; @@ -639,7 +666,7 @@ message SwitchStateResponse { fixed32 key = 1; bool state = 2; - uint32 device_id = 3; + uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"]; } message SwitchCommandRequest { option (id) = 33; @@ -650,7 +677,7 @@ message SwitchCommandRequest { fixed32 key = 1; bool state = 2; - uint32 device_id = 3; + uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"]; } // ==================== TEXT SENSOR ==================== @@ -663,13 +690,13 @@ message ListEntitiesTextSensorResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool disabled_by_default = 6; EntityCategory entity_category = 7; string device_class = 8; - uint32 device_id = 9; + uint32 device_id = 9 [(field_ifdef) = "USE_DEVICES"]; } message TextSensorStateResponse { option (id) = 27; @@ -683,7 +710,7 @@ message TextSensorStateResponse { // If the text sensor does not have a valid state yet. // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller bool missing_state = 3; - uint32 device_id = 4; + uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"]; } // ==================== SUBSCRIBE LOGS ==================== @@ -711,7 +738,6 @@ message SubscribeLogsResponse { LogLevel level = 1; bytes message = 3; - bool send_failed = 4; } // ==================== NOISE ENCRYPTION ==================== @@ -735,23 +761,41 @@ message NoiseEncryptionSetKeyResponse { message SubscribeHomeassistantServicesRequest { option (id) = 34; option (source) = SOURCE_CLIENT; + option (ifdef) = "USE_API_HOMEASSISTANT_SERVICES"; } message HomeassistantServiceMap { string key = 1; - string value = 2; + string value = 2 [(no_zero_copy) = true]; } -message HomeassistantServiceResponse { +message HomeassistantActionRequest { option (id) = 35; option (source) = SOURCE_SERVER; option (no_delay) = true; + option (ifdef) = "USE_API_HOMEASSISTANT_SERVICES"; string service = 1; - repeated HomeassistantServiceMap data = 2; - repeated HomeassistantServiceMap data_template = 3; - repeated HomeassistantServiceMap variables = 4; + repeated HomeassistantServiceMap data = 2 [(fixed_vector) = true]; + repeated HomeassistantServiceMap data_template = 3 [(fixed_vector) = true]; + repeated HomeassistantServiceMap variables = 4 [(fixed_vector) = true]; bool is_event = 5; + uint32 call_id = 6 [(field_ifdef) = "USE_API_HOMEASSISTANT_ACTION_RESPONSES"]; + bool wants_response = 7 [(field_ifdef) = "USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON"]; + string response_template = 8 [(no_zero_copy) = true, (field_ifdef) = "USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON"]; +} + +// Message sent by Home Assistant to ESPHome with service call response data +message HomeassistantActionResponse { + option (id) = 130; + option (source) = SOURCE_CLIENT; + option (no_delay) = true; + option (ifdef) = "USE_API_HOMEASSISTANT_ACTION_RESPONSES"; + + uint32 call_id = 1; // Matches the call_id from HomeassistantActionRequest + bool success = 2; // Whether the service call succeeded + string error_message = 3; // Error message if success = false + bytes response_data = 4 [(pointer_to_buffer) = true, (field_ifdef) = "USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON"]; } // ==================== IMPORT HOME ASSISTANT STATES ==================== @@ -761,11 +805,13 @@ message HomeassistantServiceResponse { message SubscribeHomeAssistantStatesRequest { option (id) = 38; option (source) = SOURCE_CLIENT; + option (ifdef) = "USE_API_HOMEASSISTANT_STATES"; } message SubscribeHomeAssistantStateResponse { option (id) = 39; option (source) = SOURCE_SERVER; + option (ifdef) = "USE_API_HOMEASSISTANT_STATES"; string entity_id = 1; string attribute = 2; bool once = 3; @@ -775,6 +821,7 @@ message HomeAssistantStateResponse { option (id) = 40; option (source) = SOURCE_CLIENT; option (no_delay) = true; + option (ifdef) = "USE_API_HOMEASSISTANT_STATES"; string entity_id = 1; string state = 2; @@ -784,15 +831,16 @@ message HomeAssistantStateResponse { // ==================== IMPORT TIME ==================== message GetTimeRequest { option (id) = 36; - option (source) = SOURCE_BOTH; + option (source) = SOURCE_SERVER; } message GetTimeResponse { option (id) = 37; - option (source) = SOURCE_BOTH; + option (source) = SOURCE_CLIENT; option (no_delay) = true; fixed32 epoch_seconds = 1; + string timezone = 2 [(pointer_to_buffer) = true]; } // ==================== USER-DEFINES SERVICES ==================== @@ -818,7 +866,7 @@ message ListEntitiesServicesResponse { string name = 1; fixed32 key = 2; - repeated ListEntitiesServicesArgument args = 3; + repeated ListEntitiesServicesArgument args = 3 [(fixed_vector) = true]; } message ExecuteServiceArgument { option (ifdef) = "USE_API_SERVICES"; @@ -828,10 +876,10 @@ message ExecuteServiceArgument { string string_ = 4; // ESPHome 1.14 (api v1.3) make int a signed value sint32 int_ = 5; - repeated bool bool_array = 6 [packed=false]; - repeated sint32 int_array = 7 [packed=false]; - repeated float float_array = 8 [packed=false]; - repeated string string_array = 9; + repeated bool bool_array = 6 [packed=false, (fixed_vector) = true]; + repeated sint32 int_array = 7 [packed=false, (fixed_vector) = true]; + repeated float float_array = 8 [packed=false, (fixed_vector) = true]; + repeated string string_array = 9 [(fixed_vector) = true]; } message ExecuteServiceRequest { option (id) = 42; @@ -840,7 +888,7 @@ message ExecuteServiceRequest { option (ifdef) = "USE_API_SERVICES"; fixed32 key = 1; - repeated ExecuteServiceArgument args = 2; + repeated ExecuteServiceArgument args = 2 [(fixed_vector) = true]; } // ==================== CAMERA ==================== @@ -853,11 +901,11 @@ message ListEntitiesCameraResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id bool disabled_by_default = 5; - string icon = 6; + string icon = 6 [(field_ifdef) = "USE_ENTITY_ICON"]; EntityCategory entity_category = 7; - uint32 device_id = 8; + uint32 device_id = 8 [(field_ifdef) = "USE_DEVICES"]; } message CameraImageResponse { @@ -869,7 +917,7 @@ message CameraImageResponse { fixed32 key = 1; bytes data = 2; bool done = 3; - uint32 device_id = 4; + uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"]; } message CameraImageRequest { option (id) = 45; @@ -937,32 +985,34 @@ message ListEntitiesClimateResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - bool supports_current_temperature = 5; - bool supports_two_point_target_temperature = 6; - repeated ClimateMode supported_modes = 7; + bool supports_current_temperature = 5; // Deprecated: use feature_flags + bool supports_two_point_target_temperature = 6; // Deprecated: use feature_flags + repeated ClimateMode supported_modes = 7 [(container_pointer) = "std::set"]; float visual_min_temperature = 8; float visual_max_temperature = 9; float visual_target_temperature_step = 10; // for older peer versions - in new system this // is if CLIMATE_PRESET_AWAY exists is supported_presets - bool legacy_supports_away = 11; - bool supports_action = 12; - repeated ClimateFanMode supported_fan_modes = 13; - repeated ClimateSwingMode supported_swing_modes = 14; - repeated string supported_custom_fan_modes = 15; - repeated ClimatePreset supported_presets = 16; - repeated string supported_custom_presets = 17; + // Deprecated in API version 1.5 + bool legacy_supports_away = 11 [deprecated=true]; + bool supports_action = 12; // Deprecated: use feature_flags + repeated ClimateFanMode supported_fan_modes = 13 [(container_pointer) = "std::set"]; + repeated ClimateSwingMode supported_swing_modes = 14 [(container_pointer) = "std::set"]; + repeated string supported_custom_fan_modes = 15 [(container_pointer) = "std::set"]; + repeated ClimatePreset supported_presets = 16 [(container_pointer) = "std::set"]; + repeated string supported_custom_presets = 17 [(container_pointer) = "std::set"]; bool disabled_by_default = 18; - string icon = 19; + string icon = 19 [(field_ifdef) = "USE_ENTITY_ICON"]; EntityCategory entity_category = 20; float visual_current_temperature_step = 21; - bool supports_current_humidity = 22; - bool supports_target_humidity = 23; + bool supports_current_humidity = 22; // Deprecated: use feature_flags + bool supports_target_humidity = 23; // Deprecated: use feature_flags float visual_min_humidity = 24; float visual_max_humidity = 25; - uint32 device_id = 26; + uint32 device_id = 26 [(field_ifdef) = "USE_DEVICES"]; + uint32 feature_flags = 27; } message ClimateStateResponse { option (id) = 47; @@ -978,7 +1028,8 @@ message ClimateStateResponse { float target_temperature_low = 5; float target_temperature_high = 6; // For older peers, equal to preset == CLIMATE_PRESET_AWAY - bool unused_legacy_away = 7; + // Deprecated in API version 1.5 + bool unused_legacy_away = 7 [deprecated=true]; ClimateAction action = 8; ClimateFanMode fan_mode = 9; ClimateSwingMode swing_mode = 10; @@ -987,7 +1038,7 @@ message ClimateStateResponse { string custom_preset = 13; float current_humidity = 14; float target_humidity = 15; - uint32 device_id = 16; + uint32 device_id = 16 [(field_ifdef) = "USE_DEVICES"]; } message ClimateCommandRequest { option (id) = 48; @@ -1006,8 +1057,10 @@ message ClimateCommandRequest { bool has_target_temperature_high = 8; float target_temperature_high = 9; // legacy, for older peers, newer ones should use CLIMATE_PRESET_AWAY in preset - bool unused_has_legacy_away = 10; - bool unused_legacy_away = 11; + // Deprecated in API version 1.5 + bool unused_has_legacy_away = 10 [deprecated=true]; + // Deprecated in API version 1.5 + bool unused_legacy_away = 11 [deprecated=true]; bool has_fan_mode = 12; ClimateFanMode fan_mode = 13; bool has_swing_mode = 14; @@ -1020,7 +1073,7 @@ message ClimateCommandRequest { string custom_preset = 21; bool has_target_humidity = 22; float target_humidity = 23; - uint32 device_id = 24; + uint32 device_id = 24 [(field_ifdef) = "USE_DEVICES"]; } // ==================== NUMBER ==================== @@ -1038,9 +1091,9 @@ message ListEntitiesNumberResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; float min_value = 6; float max_value = 7; float step = 8; @@ -1049,7 +1102,7 @@ message ListEntitiesNumberResponse { string unit_of_measurement = 11; NumberMode mode = 12; string device_class = 13; - uint32 device_id = 14; + uint32 device_id = 14 [(field_ifdef) = "USE_DEVICES"]; } message NumberStateResponse { option (id) = 50; @@ -1063,7 +1116,7 @@ message NumberStateResponse { // If the number does not have a valid state yet. // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller bool missing_state = 3; - uint32 device_id = 4; + uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"]; } message NumberCommandRequest { option (id) = 51; @@ -1074,7 +1127,7 @@ message NumberCommandRequest { fixed32 key = 1; float state = 2; - uint32 device_id = 3; + uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"]; } // ==================== SELECT ==================== @@ -1087,13 +1140,13 @@ message ListEntitiesSelectResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; - repeated string options = 6; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; + repeated string options = 6 [(container_pointer) = "std::vector"]; bool disabled_by_default = 7; EntityCategory entity_category = 8; - uint32 device_id = 9; + uint32 device_id = 9 [(field_ifdef) = "USE_DEVICES"]; } message SelectStateResponse { option (id) = 53; @@ -1107,7 +1160,7 @@ message SelectStateResponse { // If the select does not have a valid state yet. // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller bool missing_state = 3; - uint32 device_id = 4; + uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"]; } message SelectCommandRequest { option (id) = 54; @@ -1118,7 +1171,7 @@ message SelectCommandRequest { fixed32 key = 1; string state = 2; - uint32 device_id = 3; + uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"]; } // ==================== SIREN ==================== @@ -1131,15 +1184,15 @@ message ListEntitiesSirenResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool disabled_by_default = 6; repeated string tones = 7; bool supports_duration = 8; bool supports_volume = 9; EntityCategory entity_category = 10; - uint32 device_id = 11; + uint32 device_id = 11 [(field_ifdef) = "USE_DEVICES"]; } message SirenStateResponse { option (id) = 56; @@ -1150,7 +1203,7 @@ message SirenStateResponse { fixed32 key = 1; bool state = 2; - uint32 device_id = 3; + uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"]; } message SirenCommandRequest { option (id) = 57; @@ -1168,7 +1221,7 @@ message SirenCommandRequest { uint32 duration = 7; bool has_volume = 8; float volume = 9; - uint32 device_id = 10; + uint32 device_id = 10 [(field_ifdef) = "USE_DEVICES"]; } // ==================== LOCK ==================== @@ -1194,9 +1247,9 @@ message ListEntitiesLockResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool disabled_by_default = 6; EntityCategory entity_category = 7; bool assumed_state = 8; @@ -1206,7 +1259,7 @@ message ListEntitiesLockResponse { // Not yet implemented: string code_format = 11; - uint32 device_id = 12; + uint32 device_id = 12 [(field_ifdef) = "USE_DEVICES"]; } message LockStateResponse { option (id) = 59; @@ -1216,7 +1269,7 @@ message LockStateResponse { option (no_delay) = true; fixed32 key = 1; LockState state = 2; - uint32 device_id = 3; + uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"]; } message LockCommandRequest { option (id) = 60; @@ -1230,7 +1283,7 @@ message LockCommandRequest { // Not yet implemented: bool has_code = 3; string code = 4; - uint32 device_id = 5; + uint32 device_id = 5 [(field_ifdef) = "USE_DEVICES"]; } // ==================== BUTTON ==================== @@ -1243,13 +1296,13 @@ message ListEntitiesButtonResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool disabled_by_default = 6; EntityCategory entity_category = 7; string device_class = 8; - uint32 device_id = 9; + uint32 device_id = 9 [(field_ifdef) = "USE_DEVICES"]; } message ButtonCommandRequest { option (id) = 62; @@ -1259,7 +1312,7 @@ message ButtonCommandRequest { option (base_class) = "CommandProtoMessage"; fixed32 key = 1; - uint32 device_id = 2; + uint32 device_id = 2 [(field_ifdef) = "USE_DEVICES"]; } // ==================== MEDIA PLAYER ==================== @@ -1268,6 +1321,9 @@ enum MediaPlayerState { MEDIA_PLAYER_STATE_IDLE = 1; MEDIA_PLAYER_STATE_PLAYING = 2; MEDIA_PLAYER_STATE_PAUSED = 3; + MEDIA_PLAYER_STATE_ANNOUNCING = 4; + MEDIA_PLAYER_STATE_OFF = 5; + MEDIA_PLAYER_STATE_ON = 6; } enum MediaPlayerCommand { MEDIA_PLAYER_COMMAND_PLAY = 0; @@ -1275,6 +1331,15 @@ enum MediaPlayerCommand { MEDIA_PLAYER_COMMAND_STOP = 2; MEDIA_PLAYER_COMMAND_MUTE = 3; MEDIA_PLAYER_COMMAND_UNMUTE = 4; + MEDIA_PLAYER_COMMAND_TOGGLE = 5; + MEDIA_PLAYER_COMMAND_VOLUME_UP = 6; + MEDIA_PLAYER_COMMAND_VOLUME_DOWN = 7; + MEDIA_PLAYER_COMMAND_ENQUEUE = 8; + MEDIA_PLAYER_COMMAND_REPEAT_ONE = 9; + MEDIA_PLAYER_COMMAND_REPEAT_OFF = 10; + MEDIA_PLAYER_COMMAND_CLEAR_PLAYLIST = 11; + MEDIA_PLAYER_COMMAND_TURN_ON = 12; + MEDIA_PLAYER_COMMAND_TURN_OFF = 13; } enum MediaPlayerFormatPurpose { MEDIA_PLAYER_FORMAT_PURPOSE_DEFAULT = 0; @@ -1298,9 +1363,9 @@ message ListEntitiesMediaPlayerResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool disabled_by_default = 6; EntityCategory entity_category = 7; @@ -1308,7 +1373,9 @@ message ListEntitiesMediaPlayerResponse { repeated MediaPlayerSupportedFormat supported_formats = 9; - uint32 device_id = 10; + uint32 device_id = 10 [(field_ifdef) = "USE_DEVICES"]; + + uint32 feature_flags = 11; } message MediaPlayerStateResponse { option (id) = 64; @@ -1320,7 +1387,7 @@ message MediaPlayerStateResponse { MediaPlayerState state = 2; float volume = 3; bool muted = 4; - uint32 device_id = 5; + uint32 device_id = 5 [(field_ifdef) = "USE_DEVICES"]; } message MediaPlayerCommandRequest { option (id) = 65; @@ -1342,7 +1409,7 @@ message MediaPlayerCommandRequest { bool has_announcement = 8; bool announcement = 9; - uint32 device_id = 10; + uint32 device_id = 10 [(field_ifdef) = "USE_DEVICES"]; } // ==================== BLUETOOTH ==================== @@ -1354,12 +1421,17 @@ message SubscribeBluetoothLEAdvertisementsRequest { uint32 flags = 1; } +// Deprecated - only used by deprecated BluetoothLEAdvertisementResponse message BluetoothServiceData { + option deprecated = true; string uuid = 1; - repeated uint32 legacy_data = 2 [deprecated = true]; // Removed in api version 1.7 + // Deprecated in API version 1.7 + repeated uint32 legacy_data = 2 [deprecated=true]; // Removed in api version 1.7 bytes data = 3; // Added in api version 1.7 } +// Removed in ESPHome 2025.8.0 - use BluetoothLERawAdvertisementsResponse instead message BluetoothLEAdvertisementResponse { + option deprecated = true; option (id) = 67; option (source) = SOURCE_SERVER; option (ifdef) = "USE_BLUETOOTH_PROXY"; @@ -1381,7 +1453,7 @@ message BluetoothLERawAdvertisement { sint32 rssi = 2; uint32 address_type = 3; - bytes data = 4; + bytes data = 4 [(fixed_array_size) = 62]; } message BluetoothLERawAdvertisementsResponse { @@ -1390,11 +1462,11 @@ message BluetoothLERawAdvertisementsResponse { option (ifdef) = "USE_BLUETOOTH_PROXY"; option (no_delay) = true; - repeated BluetoothLERawAdvertisement advertisements = 1; + repeated BluetoothLERawAdvertisement advertisements = 1 [(fixed_array_with_length_define) = "BLUETOOTH_PROXY_ADVERTISEMENT_BATCH_SIZE"]; } enum BluetoothDeviceRequestType { - BLUETOOTH_DEVICE_REQUEST_TYPE_CONNECT = 0; + BLUETOOTH_DEVICE_REQUEST_TYPE_CONNECT = 0 [deprecated = true]; // V1 removed, use V3 variants BLUETOOTH_DEVICE_REQUEST_TYPE_DISCONNECT = 1; BLUETOOTH_DEVICE_REQUEST_TYPE_PAIR = 2; BLUETOOTH_DEVICE_REQUEST_TYPE_UNPAIR = 3; @@ -1410,7 +1482,7 @@ message BluetoothDeviceRequest { uint64 address = 1; BluetoothDeviceRequestType request_type = 2; - bool has_address_type = 3; + bool has_address_type = 3; // Deprecated, should be removed in 2027.8 - https://github.com/esphome/esphome/pull/10318 uint32 address_type = 4; } @@ -1434,21 +1506,39 @@ message BluetoothGATTGetServicesRequest { } message BluetoothGATTDescriptor { - repeated uint64 uuid = 1; + repeated uint64 uuid = 1 [(fixed_array_size) = 2, (fixed_array_skip_zero) = true]; uint32 handle = 2; + + // New field for efficient UUID (v1.12+) + // Only one of uuid or short_uuid will be set. + // short_uuid is used for both 16-bit and 32-bit UUIDs with v1.12+ clients. + // 128-bit UUIDs always use the uuid field for backwards compatibility. + uint32 short_uuid = 3; // 16-bit or 32-bit UUID } message BluetoothGATTCharacteristic { - repeated uint64 uuid = 1; + repeated uint64 uuid = 1 [(fixed_array_size) = 2, (fixed_array_skip_zero) = true]; uint32 handle = 2; uint32 properties = 3; - repeated BluetoothGATTDescriptor descriptors = 4; + repeated BluetoothGATTDescriptor descriptors = 4 [(fixed_vector) = true]; + + // New field for efficient UUID (v1.12+) + // Only one of uuid or short_uuid will be set. + // short_uuid is used for both 16-bit and 32-bit UUIDs with v1.12+ clients. + // 128-bit UUIDs always use the uuid field for backwards compatibility. + uint32 short_uuid = 5; // 16-bit or 32-bit UUID } message BluetoothGATTService { - repeated uint64 uuid = 1; + repeated uint64 uuid = 1 [(fixed_array_size) = 2, (fixed_array_skip_zero) = true]; uint32 handle = 2; - repeated BluetoothGATTCharacteristic characteristics = 3; + repeated BluetoothGATTCharacteristic characteristics = 3 [(fixed_vector) = true]; + + // New field for efficient UUID (v1.12+) + // Only one of uuid or short_uuid will be set. + // short_uuid is used for both 16-bit and 32-bit UUIDs with v1.12+ clients. + // 128-bit UUIDs always use the uuid field for backwards compatibility. + uint32 short_uuid = 4; // 16-bit or 32-bit UUID } message BluetoothGATTGetServicesResponse { @@ -1498,7 +1588,7 @@ message BluetoothGATTWriteRequest { uint32 handle = 2; bool response = 3; - bytes data = 4; + bytes data = 4 [(pointer_to_buffer) = true]; } message BluetoothGATTReadDescriptorRequest { @@ -1518,7 +1608,7 @@ message BluetoothGATTWriteDescriptorRequest { uint64 address = 1; uint32 handle = 2; - bytes data = 3; + bytes data = 3 [(pointer_to_buffer) = true]; } message BluetoothGATTNotifyRequest { @@ -1555,7 +1645,10 @@ message BluetoothConnectionsFreeResponse { uint32 free = 1; uint32 limit = 2; - repeated uint64 allocated = 3; + repeated uint64 allocated = 3 [ + (fixed_array_size_define) = "BLUETOOTH_PROXY_MAX_CONNECTIONS", + (fixed_array_skip_zero) = true + ]; } message BluetoothGATTErrorResponse { @@ -1643,6 +1736,7 @@ message BluetoothScannerStateResponse { BluetoothScannerState state = 1; BluetoothScannerMode mode = 2; + BluetoothScannerMode configured_mode = 3; } message BluetoothScannerSetModeRequest { @@ -1788,10 +1882,22 @@ message VoiceAssistantWakeWord { repeated string trained_languages = 3; } +message VoiceAssistantExternalWakeWord { + string id = 1; + string wake_word = 2; + repeated string trained_languages = 3; + string model_type = 4; + uint32 model_size = 5; + string model_hash = 6; + string url = 7; +} + message VoiceAssistantConfigurationRequest { option (id) = 121; option (source) = SOURCE_CLIENT; option (ifdef) = "USE_VOICE_ASSISTANT"; + + repeated VoiceAssistantExternalWakeWord external_wake_words = 1; } message VoiceAssistantConfigurationResponse { @@ -1800,7 +1906,7 @@ message VoiceAssistantConfigurationResponse { option (ifdef) = "USE_VOICE_ASSISTANT"; repeated VoiceAssistantWakeWord available_wake_words = 1; - repeated string active_wake_words = 2; + repeated string active_wake_words = 2 [(container_pointer) = "std::vector"]; uint32 max_active_wake_words = 3; } @@ -1845,14 +1951,14 @@ message ListEntitiesAlarmControlPanelResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; - string icon = 5; + reserved 4; // Deprecated: was string unique_id + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool disabled_by_default = 6; EntityCategory entity_category = 7; uint32 supported_features = 8; bool requires_code = 9; bool requires_code_to_arm = 10; - uint32 device_id = 11; + uint32 device_id = 11 [(field_ifdef) = "USE_DEVICES"]; } message AlarmControlPanelStateResponse { @@ -1863,7 +1969,7 @@ message AlarmControlPanelStateResponse { option (no_delay) = true; fixed32 key = 1; AlarmControlPanelState state = 2; - uint32 device_id = 3; + uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"]; } message AlarmControlPanelCommandRequest { @@ -1875,7 +1981,7 @@ message AlarmControlPanelCommandRequest { fixed32 key = 1; AlarmControlPanelStateCommand command = 2; string code = 3; - uint32 device_id = 4; + uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"]; } // ===================== TEXT ===================== @@ -1892,8 +1998,8 @@ message ListEntitiesTextResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; - string icon = 5; + reserved 4; // Deprecated: was string unique_id + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool disabled_by_default = 6; EntityCategory entity_category = 7; @@ -1901,7 +2007,7 @@ message ListEntitiesTextResponse { uint32 max_length = 9; string pattern = 10; TextMode mode = 11; - uint32 device_id = 12; + uint32 device_id = 12 [(field_ifdef) = "USE_DEVICES"]; } message TextStateResponse { option (id) = 98; @@ -1915,7 +2021,7 @@ message TextStateResponse { // If the Text does not have a valid state yet. // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller bool missing_state = 3; - uint32 device_id = 4; + uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"]; } message TextCommandRequest { option (id) = 99; @@ -1926,7 +2032,7 @@ message TextCommandRequest { fixed32 key = 1; string state = 2; - uint32 device_id = 3; + uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"]; } @@ -1940,12 +2046,12 @@ message ListEntitiesDateResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool disabled_by_default = 6; EntityCategory entity_category = 7; - uint32 device_id = 8; + uint32 device_id = 8 [(field_ifdef) = "USE_DEVICES"]; } message DateStateResponse { option (id) = 101; @@ -1961,7 +2067,7 @@ message DateStateResponse { uint32 year = 3; uint32 month = 4; uint32 day = 5; - uint32 device_id = 6; + uint32 device_id = 6 [(field_ifdef) = "USE_DEVICES"]; } message DateCommandRequest { option (id) = 102; @@ -1974,7 +2080,7 @@ message DateCommandRequest { uint32 year = 2; uint32 month = 3; uint32 day = 4; - uint32 device_id = 5; + uint32 device_id = 5 [(field_ifdef) = "USE_DEVICES"]; } // ==================== DATETIME TIME ==================== @@ -1987,12 +2093,12 @@ message ListEntitiesTimeResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool disabled_by_default = 6; EntityCategory entity_category = 7; - uint32 device_id = 8; + uint32 device_id = 8 [(field_ifdef) = "USE_DEVICES"]; } message TimeStateResponse { option (id) = 104; @@ -2008,7 +2114,7 @@ message TimeStateResponse { uint32 hour = 3; uint32 minute = 4; uint32 second = 5; - uint32 device_id = 6; + uint32 device_id = 6 [(field_ifdef) = "USE_DEVICES"]; } message TimeCommandRequest { option (id) = 105; @@ -2021,7 +2127,7 @@ message TimeCommandRequest { uint32 hour = 2; uint32 minute = 3; uint32 second = 4; - uint32 device_id = 5; + uint32 device_id = 5 [(field_ifdef) = "USE_DEVICES"]; } // ==================== EVENT ==================== @@ -2034,15 +2140,15 @@ message ListEntitiesEventResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool disabled_by_default = 6; EntityCategory entity_category = 7; string device_class = 8; repeated string event_types = 9; - uint32 device_id = 10; + uint32 device_id = 10 [(field_ifdef) = "USE_DEVICES"]; } message EventResponse { option (id) = 108; @@ -2052,7 +2158,7 @@ message EventResponse { fixed32 key = 1; string event_type = 2; - uint32 device_id = 3; + uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"]; } // ==================== VALVE ==================== @@ -2065,9 +2171,9 @@ message ListEntitiesValveResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool disabled_by_default = 6; EntityCategory entity_category = 7; string device_class = 8; @@ -2075,7 +2181,7 @@ message ListEntitiesValveResponse { bool assumed_state = 9; bool supports_position = 10; bool supports_stop = 11; - uint32 device_id = 12; + uint32 device_id = 12 [(field_ifdef) = "USE_DEVICES"]; } enum ValveOperation { @@ -2093,7 +2199,7 @@ message ValveStateResponse { fixed32 key = 1; float position = 2; ValveOperation current_operation = 3; - uint32 device_id = 4; + uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"]; } message ValveCommandRequest { @@ -2107,7 +2213,7 @@ message ValveCommandRequest { bool has_position = 2; float position = 3; bool stop = 4; - uint32 device_id = 5; + uint32 device_id = 5 [(field_ifdef) = "USE_DEVICES"]; } // ==================== DATETIME DATETIME ==================== @@ -2120,12 +2226,12 @@ message ListEntitiesDateTimeResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool disabled_by_default = 6; EntityCategory entity_category = 7; - uint32 device_id = 8; + uint32 device_id = 8 [(field_ifdef) = "USE_DEVICES"]; } message DateTimeStateResponse { option (id) = 113; @@ -2139,7 +2245,7 @@ message DateTimeStateResponse { // Equivalent to `!obj->has_state()` - inverse logic to make state packets smaller bool missing_state = 2; fixed32 epoch_seconds = 3; - uint32 device_id = 4; + uint32 device_id = 4 [(field_ifdef) = "USE_DEVICES"]; } message DateTimeCommandRequest { option (id) = 114; @@ -2150,7 +2256,7 @@ message DateTimeCommandRequest { fixed32 key = 1; fixed32 epoch_seconds = 2; - uint32 device_id = 3; + uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"]; } // ==================== UPDATE ==================== @@ -2163,13 +2269,13 @@ message ListEntitiesUpdateResponse { string object_id = 1; fixed32 key = 2; string name = 3; - string unique_id = 4; + reserved 4; // Deprecated: was string unique_id - string icon = 5; + string icon = 5 [(field_ifdef) = "USE_ENTITY_ICON"]; bool disabled_by_default = 6; EntityCategory entity_category = 7; string device_class = 8; - uint32 device_id = 9; + uint32 device_id = 9 [(field_ifdef) = "USE_DEVICES"]; } message UpdateStateResponse { option (id) = 117; @@ -2188,7 +2294,7 @@ message UpdateStateResponse { string title = 8; string release_summary = 9; string release_url = 10; - uint32 device_id = 11; + uint32 device_id = 11 [(field_ifdef) = "USE_DEVICES"]; } enum UpdateCommand { UPDATE_COMMAND_NONE = 0; @@ -2204,5 +2310,30 @@ message UpdateCommandRequest { fixed32 key = 1; UpdateCommand command = 2; - uint32 device_id = 3; + uint32 device_id = 3 [(field_ifdef) = "USE_DEVICES"]; +} + +// ==================== Z-WAVE ==================== + +message ZWaveProxyFrame { + option (id) = 128; + option (source) = SOURCE_BOTH; + option (ifdef) = "USE_ZWAVE_PROXY"; + option (no_delay) = true; + + bytes data = 1 [(pointer_to_buffer) = true]; +} + +enum ZWaveProxyRequestType { + ZWAVE_PROXY_REQUEST_TYPE_SUBSCRIBE = 0; + ZWAVE_PROXY_REQUEST_TYPE_UNSUBSCRIBE = 1; + ZWAVE_PROXY_REQUEST_TYPE_HOME_ID_CHANGE = 2; +} +message ZWaveProxyRequest { + option (id) = 129; + option (source) = SOURCE_BOTH; + option (ifdef) = "USE_ZWAVE_PROXY"; + + ZWaveProxyRequestType type = 1; + bytes data = 2 [(pointer_to_buffer) = true]; } diff --git a/esphome/components/api/api_connection.cpp b/esphome/components/api/api_connection.cpp index ea3268a583..ea1e130092 100644 --- a/esphome/components/api/api_connection.cpp +++ b/esphome/components/api/api_connection.cpp @@ -1,10 +1,16 @@ #include "api_connection.h" #ifdef USE_API +#ifdef USE_API_NOISE +#include "api_frame_helper_noise.h" +#endif +#ifdef USE_API_PLAINTEXT +#include "api_frame_helper_plaintext.h" +#endif #include #include -#include #include #include +#include #include "esphome/components/network/util.h" #include "esphome/core/application.h" #include "esphome/core/entity_base.h" @@ -21,12 +27,17 @@ #ifdef USE_BLUETOOTH_PROXY #include "esphome/components/bluetooth_proxy/bluetooth_proxy.h" #endif +#ifdef USE_CLIMATE +#include "esphome/components/climate/climate_mode.h" +#endif #ifdef USE_VOICE_ASSISTANT #include "esphome/components/voice_assistant/voice_assistant.h" #endif +#ifdef USE_ZWAVE_PROXY +#include "esphome/components/zwave_proxy/zwave_proxy.h" +#endif -namespace esphome { -namespace api { +namespace esphome::api { // Read a maximum of 5 messages per loop iteration to prevent starving other components. // This is a balance between API responsiveness and allowing other components to run. @@ -37,37 +48,60 @@ static constexpr uint8_t MAX_PING_RETRIES = 60; static constexpr uint16_t PING_RETRY_INTERVAL = 1000; static constexpr uint32_t KEEPALIVE_DISCONNECT_TIMEOUT = (KEEPALIVE_TIMEOUT_MS * 5) / 2; +static constexpr auto ESPHOME_VERSION_REF = StringRef::from_lit(ESPHOME_VERSION); + static const char *const TAG = "api.connection"; #ifdef USE_CAMERA static const int CAMERA_STOP_STREAM = 5000; #endif -// Helper macro for entity command handlers - gets entity by key, returns if not found, and creates call object +#ifdef USE_DEVICES +// Helper macro for entity command handlers - gets entity by key and device_id, returns if not found, and creates call +// object +#define ENTITY_COMMAND_MAKE_CALL(entity_type, entity_var, getter_name) \ + entity_type *entity_var = App.get_##getter_name##_by_key(msg.key, msg.device_id); \ + if ((entity_var) == nullptr) \ + return; \ + auto call = (entity_var)->make_call(); + +// Helper macro for entity command handlers that don't use make_call() - gets entity by key and device_id and returns if +// not found +#define ENTITY_COMMAND_GET(entity_type, entity_var, getter_name) \ + entity_type *entity_var = App.get_##getter_name##_by_key(msg.key, msg.device_id); \ + if ((entity_var) == nullptr) \ + return; +#else // No device support, use simpler macros +// Helper macro for entity command handlers - gets entity by key, returns if not found, and creates call +// object #define ENTITY_COMMAND_MAKE_CALL(entity_type, entity_var, getter_name) \ entity_type *entity_var = App.get_##getter_name##_by_key(msg.key); \ if ((entity_var) == nullptr) \ return; \ auto call = (entity_var)->make_call(); -// Helper macro for entity command handlers that don't use make_call() - gets entity by key and returns if not found +// Helper macro for entity command handlers that don't use make_call() - gets entity by key and returns if +// not found #define ENTITY_COMMAND_GET(entity_type, entity_var, getter_name) \ entity_type *entity_var = App.get_##getter_name##_by_key(msg.key); \ if ((entity_var) == nullptr) \ return; +#endif // USE_DEVICES APIConnection::APIConnection(std::unique_ptr sock, APIServer *parent) : parent_(parent), initial_state_iterator_(this), list_entities_iterator_(this) { #if defined(USE_API_PLAINTEXT) && defined(USE_API_NOISE) auto noise_ctx = parent->get_noise_ctx(); if (noise_ctx->has_psk()) { - this->helper_ = std::unique_ptr{new APINoiseFrameHelper(std::move(sock), noise_ctx)}; + this->helper_ = + std::unique_ptr{new APINoiseFrameHelper(std::move(sock), noise_ctx, &this->client_info_)}; } else { - this->helper_ = std::unique_ptr{new APIPlaintextFrameHelper(std::move(sock))}; + this->helper_ = std::unique_ptr{new APIPlaintextFrameHelper(std::move(sock), &this->client_info_)}; } #elif defined(USE_API_PLAINTEXT) - this->helper_ = std::unique_ptr{new APIPlaintextFrameHelper(std::move(sock))}; + this->helper_ = std::unique_ptr{new APIPlaintextFrameHelper(std::move(sock), &this->client_info_)}; #elif defined(USE_API_NOISE) - this->helper_ = std::unique_ptr{new APINoiseFrameHelper(std::move(sock), parent->get_noise_ctx())}; + this->helper_ = std::unique_ptr{ + new APINoiseFrameHelper(std::move(sock), parent->get_noise_ctx(), &this->client_info_)}; #else #error "No frame helper defined" #endif @@ -85,14 +119,11 @@ void APIConnection::start() { APIError err = this->helper_->init(); if (err != APIError::OK) { - on_fatal_error(); - ESP_LOGW(TAG, "%s: Helper init failed: %s errno=%d", this->get_client_combined_info().c_str(), - api_error_to_str(err), errno); + this->fatal_error_with_log_(LOG_STR("Helper init failed"), err); return; } - this->client_info_ = helper_->getpeername(); - this->client_peername_ = this->client_info_; - this->helper_->set_log_info(this->client_info_); + this->client_info_.peername = helper_->getpeername(); + this->client_info_.name = this->client_info_.peername; } APIConnection::~APIConnection() { @@ -118,9 +149,7 @@ void APIConnection::loop() { APIError err = this->helper_->loop(); if (err != APIError::OK) { - on_fatal_error(); - ESP_LOGW(TAG, "%s: Socket operation failed: %s errno=%d", this->get_client_combined_info().c_str(), - api_error_to_str(err), errno); + this->fatal_error_with_log_(LOG_STR("Socket operation failed"), err); return; } @@ -135,24 +164,13 @@ void APIConnection::loop() { // No more data available break; } else if (err != APIError::OK) { - on_fatal_error(); - if (err == APIError::SOCKET_READ_FAILED && errno == ECONNRESET) { - ESP_LOGW(TAG, "%s: Connection reset", this->get_client_combined_info().c_str()); - } else if (err == APIError::CONNECTION_CLOSED) { - ESP_LOGW(TAG, "%s: Connection closed", this->get_client_combined_info().c_str()); - } else { - ESP_LOGW(TAG, "%s: Reading failed: %s errno=%d", this->get_client_combined_info().c_str(), - api_error_to_str(err), errno); - } + this->fatal_error_with_log_(LOG_STR("Reading failed"), err); return; } else { this->last_traffic_ = now; // read a packet - if (buffer.data_len > 0) { - this->read_message(buffer.data_len, buffer.type, &buffer.container[buffer.data_offset]); - } else { - this->read_message(0, buffer.type, nullptr); - } + this->read_message(buffer.data_len, buffer.type, + buffer.data_len > 0 ? &buffer.container[buffer.data_offset] : nullptr); if (this->flags_.remove) return; } @@ -184,11 +202,14 @@ void APIConnection::loop() { // Disconnect if not responded within 2.5*keepalive if (now - this->last_traffic_ > KEEPALIVE_DISCONNECT_TIMEOUT) { on_fatal_error(); - ESP_LOGW(TAG, "%s is unresponsive; disconnecting", this->get_client_combined_info().c_str()); + ESP_LOGW(TAG, "%s (%s) is unresponsive; disconnecting", this->client_info_.name.c_str(), + this->client_info_.peername.c_str()); } - } else if (now - this->last_traffic_ > KEEPALIVE_TIMEOUT_MS) { + } else if (now - this->last_traffic_ > KEEPALIVE_TIMEOUT_MS && !this->flags_.remove) { + // Only send ping if we're not disconnecting ESP_LOGVV(TAG, "Sending keepalive PING"); - this->flags_.sent_ping = this->send_message(PingRequest()); + PingRequest req; + this->flags_.sent_ping = this->send_message(req, PingRequest::MESSAGE_TYPE); if (!this->flags_.sent_ping) { // If we can't send the ping request directly (tx_buffer full), // schedule it at the front of the batch so it will be sent with priority @@ -203,24 +224,16 @@ void APIConnection::loop() { if (this->image_reader_ && this->image_reader_->available() && this->helper_->can_write_without_blocking()) { uint32_t to_send = std::min((size_t) MAX_BATCH_PACKET_SIZE, this->image_reader_->available()); bool done = this->image_reader_->available() == to_send; - uint32_t msg_size = 0; - ProtoSize::add_fixed_field<4>(msg_size, 1, true); - // partial message size calculated manually since its a special case - // 1 for the data field, varint for the data size, and the data itself - msg_size += 1 + ProtoSize::varint(to_send) + to_send; - ProtoSize::add_bool_field(msg_size, 1, done); - auto buffer = this->create_buffer(msg_size); - // fixed32 key = 1; - buffer.encode_fixed32(1, camera::Camera::instance()->get_object_id_hash()); - // bytes data = 2; - buffer.encode_bytes(2, this->image_reader_->peek_data_buffer(), to_send); - // bool done = 3; - buffer.encode_bool(3, done); + CameraImageResponse msg; + msg.key = camera::Camera::instance()->get_object_id_hash(); + msg.set_data(this->image_reader_->peek_data_buffer(), to_send); + msg.done = done; +#ifdef USE_DEVICES + msg.device_id = camera::Camera::instance()->get_device_id(); +#endif - bool success = this->send_buffer(buffer, CameraImageResponse::MESSAGE_TYPE); - - if (success) { + if (this->send_message_(msg, CameraImageResponse::MESSAGE_TYPE)) { this->image_reader_->consume_data(to_send); if (done) { this->image_reader_->return_image(); @@ -229,35 +242,21 @@ void APIConnection::loop() { } #endif +#ifdef USE_API_HOMEASSISTANT_STATES if (state_subs_at_ >= 0) { - const auto &subs = this->parent_->get_state_subs(); - if (state_subs_at_ < static_cast(subs.size())) { - auto &it = subs[state_subs_at_]; - SubscribeHomeAssistantStateResponse resp; - resp.entity_id = it.entity_id; - resp.attribute = it.attribute.value(); - resp.once = it.once; - if (this->send_message(resp)) { - state_subs_at_++; - } - } else { - state_subs_at_ = -1; - } + this->process_state_subscriptions_(); } +#endif } -std::string get_default_unique_id(const std::string &component_type, EntityBase *entity) { - return App.get_name() + component_type + entity->get_object_id(); -} - -DisconnectResponse APIConnection::disconnect(const DisconnectRequest &msg) { +bool APIConnection::send_disconnect_response(const DisconnectRequest &msg) { // remote initiated disconnect_client // don't close yet, we still need to send the disconnect response // close will happen on next loop - ESP_LOGD(TAG, "%s disconnected", this->get_client_combined_info().c_str()); + ESP_LOGD(TAG, "%s (%s) disconnected", this->client_info_.name.c_str(), this->client_info_.peername.c_str()); this->flags_.next_close = true; DisconnectResponse resp; - return resp; + return this->send_message(resp, DisconnectResponse::MESSAGE_TYPE); } void APIConnection::on_disconnect_response(const DisconnectResponse &value) { this->helper_->close(); @@ -277,8 +276,9 @@ uint16_t APIConnection::encode_message_to_buffer(ProtoMessage &msg, uint8_t mess #endif // Calculate size - uint32_t calculated_size = 0; - msg.calculate_size(calculated_size); + ProtoSize size_calc; + msg.calculate_size(size_calc); + uint32_t calculated_size = size_calc.get_size(); // Cache frame sizes to avoid repeated virtual calls const uint8_t header_padding = conn->helper_->frame_header_padding(); @@ -292,16 +292,26 @@ uint16_t APIConnection::encode_message_to_buffer(ProtoMessage &msg, uint8_t mess return 0; // Doesn't fit } - // Allocate buffer space - pass payload size, allocation functions add header/footer space - ProtoWriteBuffer buffer = is_single ? conn->allocate_single_message_buffer(calculated_size) - : conn->allocate_batch_message_buffer(calculated_size); - // Get buffer size after allocation (which includes header padding) std::vector &shared_buf = conn->parent_->get_shared_buffer_ref(); - size_t size_before_encode = shared_buf.size(); + + if (is_single || conn->flags_.batch_first_message) { + // Single message or first batch message + conn->prepare_first_message_buffer(shared_buf, header_padding, total_calculated_size); + if (conn->flags_.batch_first_message) { + conn->flags_.batch_first_message = false; + } + } else { + // Batch message second or later + // Add padding for previous message footer + this message header + size_t current_size = shared_buf.size(); + shared_buf.reserve(current_size + total_calculated_size); + shared_buf.resize(current_size + footer_size + header_padding); + } // Encode directly into buffer - msg.encode(buffer); + size_t size_before_encode = shared_buf.size(); + msg.encode({&shared_buf}); // Calculate actual encoded size (not including header that was already added) size_t actual_payload_size = shared_buf.size() - size_before_encode; @@ -326,19 +336,18 @@ uint16_t APIConnection::try_send_binary_sensor_state(EntityBase *entity, APIConn BinarySensorStateResponse resp; resp.state = binary_sensor->state; resp.missing_state = !binary_sensor->has_state(); - fill_entity_state_base(binary_sensor, resp); - return encode_message_to_buffer(resp, BinarySensorStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(binary_sensor, resp, BinarySensorStateResponse::MESSAGE_TYPE, conn, + remaining_size, is_single); } uint16_t APIConnection::try_send_binary_sensor_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { auto *binary_sensor = static_cast(entity); ListEntitiesBinarySensorResponse msg; - msg.device_class = binary_sensor->get_device_class(); + msg.set_device_class(binary_sensor->get_device_class_ref()); msg.is_status_binary_sensor = binary_sensor->is_status_binary_sensor(); - msg.unique_id = get_default_unique_id("binary_sensor", binary_sensor); - fill_entity_info_base(binary_sensor, msg); - return encode_message_to_buffer(msg, ListEntitiesBinarySensorResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_info(binary_sensor, msg, ListEntitiesBinarySensorResponse::MESSAGE_TYPE, conn, + remaining_size, is_single); } #endif @@ -352,14 +361,11 @@ uint16_t APIConnection::try_send_cover_state(EntityBase *entity, APIConnection * auto *cover = static_cast(entity); CoverStateResponse msg; auto traits = cover->get_traits(); - msg.legacy_state = - (cover->position == cover::COVER_OPEN) ? enums::LEGACY_COVER_STATE_OPEN : enums::LEGACY_COVER_STATE_CLOSED; msg.position = cover->position; if (traits.get_supports_tilt()) msg.tilt = cover->tilt; msg.current_operation = static_cast(cover->current_operation); - fill_entity_state_base(cover, msg); - return encode_message_to_buffer(msg, CoverStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(cover, msg, CoverStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } uint16_t APIConnection::try_send_cover_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { @@ -370,26 +376,12 @@ uint16_t APIConnection::try_send_cover_info(EntityBase *entity, APIConnection *c msg.supports_position = traits.get_supports_position(); msg.supports_tilt = traits.get_supports_tilt(); msg.supports_stop = traits.get_supports_stop(); - msg.device_class = cover->get_device_class(); - msg.unique_id = get_default_unique_id("cover", cover); - fill_entity_info_base(cover, msg); - return encode_message_to_buffer(msg, ListEntitiesCoverResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + msg.set_device_class(cover->get_device_class_ref()); + return fill_and_encode_entity_info(cover, msg, ListEntitiesCoverResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::cover_command(const CoverCommandRequest &msg) { ENTITY_COMMAND_MAKE_CALL(cover::Cover, cover, cover) - if (msg.has_legacy_command) { - switch (msg.legacy_command) { - case enums::LEGACY_COVER_COMMAND_OPEN: - call.set_command_open(); - break; - case enums::LEGACY_COVER_COMMAND_CLOSE: - call.set_command_close(); - break; - case enums::LEGACY_COVER_COMMAND_STOP: - call.set_command_stop(); - break; - } - } if (msg.has_position) call.set_position(msg.position); if (msg.has_tilt) @@ -419,9 +411,8 @@ uint16_t APIConnection::try_send_fan_state(EntityBase *entity, APIConnection *co if (traits.supports_direction()) msg.direction = static_cast(fan->direction); if (traits.supports_preset_modes()) - msg.preset_mode = fan->preset_mode; - fill_entity_state_base(fan, msg); - return encode_message_to_buffer(msg, FanStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + msg.set_preset_mode(StringRef(fan->preset_mode)); + return fill_and_encode_entity_state(fan, msg, FanStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } uint16_t APIConnection::try_send_fan_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { @@ -432,11 +423,8 @@ uint16_t APIConnection::try_send_fan_info(EntityBase *entity, APIConnection *con msg.supports_speed = traits.supports_speed(); msg.supports_direction = traits.supports_direction(); msg.supported_speed_count = traits.supported_speed_count(); - for (auto const &preset : traits.supported_preset_modes()) - msg.supported_preset_modes.push_back(preset); - msg.unique_id = get_default_unique_id("fan", fan); - fill_entity_info_base(fan, msg); - return encode_message_to_buffer(msg, ListEntitiesFanResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + msg.supported_preset_modes = &traits.supported_preset_modes_for_api_(); + return fill_and_encode_entity_info(fan, msg, ListEntitiesFanResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } void APIConnection::fan_command(const FanCommandRequest &msg) { ENTITY_COMMAND_MAKE_CALL(fan::Fan, fan, fan) @@ -479,26 +467,19 @@ uint16_t APIConnection::try_send_light_state(EntityBase *entity, APIConnection * resp.color_temperature = values.get_color_temperature(); resp.cold_white = values.get_cold_white(); resp.warm_white = values.get_warm_white(); - if (light->supports_effects()) - resp.effect = light->get_effect_name(); - fill_entity_state_base(light, resp); - return encode_message_to_buffer(resp, LightStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + if (light->supports_effects()) { + resp.set_effect(light->get_effect_name_ref()); + } + return fill_and_encode_entity_state(light, resp, LightStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } uint16_t APIConnection::try_send_light_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { auto *light = static_cast(entity); ListEntitiesLightResponse msg; auto traits = light->get_traits(); - for (auto mode : traits.get_supported_color_modes()) - msg.supported_color_modes.push_back(static_cast(mode)); - msg.legacy_supports_brightness = traits.supports_color_capability(light::ColorCapability::BRIGHTNESS); - msg.legacy_supports_rgb = traits.supports_color_capability(light::ColorCapability::RGB); - msg.legacy_supports_white_value = - msg.legacy_supports_rgb && (traits.supports_color_capability(light::ColorCapability::WHITE) || - traits.supports_color_capability(light::ColorCapability::COLD_WARM_WHITE)); - msg.legacy_supports_color_temperature = traits.supports_color_capability(light::ColorCapability::COLOR_TEMPERATURE) || - traits.supports_color_capability(light::ColorCapability::COLD_WARM_WHITE); - if (msg.legacy_supports_color_temperature) { + msg.supported_color_modes = &traits.get_supported_color_modes_for_api_(); + if (traits.supports_color_capability(light::ColorCapability::COLOR_TEMPERATURE) || + traits.supports_color_capability(light::ColorCapability::COLD_WARM_WHITE)) { msg.min_mireds = traits.get_min_mireds(); msg.max_mireds = traits.get_max_mireds(); } @@ -508,9 +489,8 @@ uint16_t APIConnection::try_send_light_info(EntityBase *entity, APIConnection *c msg.effects.push_back(effect->get_name()); } } - msg.unique_id = get_default_unique_id("light", light); - fill_entity_info_base(light, msg); - return encode_message_to_buffer(msg, ListEntitiesLightResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_info(light, msg, ListEntitiesLightResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::light_command(const LightCommandRequest &msg) { ENTITY_COMMAND_MAKE_CALL(light::LightState, light, light) @@ -557,24 +537,20 @@ uint16_t APIConnection::try_send_sensor_state(EntityBase *entity, APIConnection SensorStateResponse resp; resp.state = sensor->state; resp.missing_state = !sensor->has_state(); - fill_entity_state_base(sensor, resp); - return encode_message_to_buffer(resp, SensorStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(sensor, resp, SensorStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } uint16_t APIConnection::try_send_sensor_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { auto *sensor = static_cast(entity); ListEntitiesSensorResponse msg; - msg.unit_of_measurement = sensor->get_unit_of_measurement(); + msg.set_unit_of_measurement(sensor->get_unit_of_measurement_ref()); msg.accuracy_decimals = sensor->get_accuracy_decimals(); msg.force_update = sensor->get_force_update(); - msg.device_class = sensor->get_device_class(); + msg.set_device_class(sensor->get_device_class_ref()); msg.state_class = static_cast(sensor->get_state_class()); - msg.unique_id = sensor->unique_id(); - if (msg.unique_id.empty()) - msg.unique_id = get_default_unique_id("sensor", sensor); - fill_entity_info_base(sensor, msg); - return encode_message_to_buffer(msg, ListEntitiesSensorResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_info(sensor, msg, ListEntitiesSensorResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } #endif @@ -589,8 +565,8 @@ uint16_t APIConnection::try_send_switch_state(EntityBase *entity, APIConnection auto *a_switch = static_cast(entity); SwitchStateResponse resp; resp.state = a_switch->state; - fill_entity_state_base(a_switch, resp); - return encode_message_to_buffer(resp, SwitchStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(a_switch, resp, SwitchStateResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } uint16_t APIConnection::try_send_switch_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, @@ -598,10 +574,9 @@ uint16_t APIConnection::try_send_switch_info(EntityBase *entity, APIConnection * auto *a_switch = static_cast(entity); ListEntitiesSwitchResponse msg; msg.assumed_state = a_switch->assumed_state(); - msg.device_class = a_switch->get_device_class(); - msg.unique_id = get_default_unique_id("switch", a_switch); - fill_entity_info_base(a_switch, msg); - return encode_message_to_buffer(msg, ListEntitiesSwitchResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + msg.set_device_class(a_switch->get_device_class_ref()); + return fill_and_encode_entity_info(a_switch, msg, ListEntitiesSwitchResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::switch_command(const SwitchCommandRequest &msg) { ENTITY_COMMAND_GET(switch_::Switch, a_switch, switch) @@ -624,21 +599,18 @@ uint16_t APIConnection::try_send_text_sensor_state(EntityBase *entity, APIConnec bool is_single) { auto *text_sensor = static_cast(entity); TextSensorStateResponse resp; - resp.state = text_sensor->state; + resp.set_state(StringRef(text_sensor->state)); resp.missing_state = !text_sensor->has_state(); - fill_entity_state_base(text_sensor, resp); - return encode_message_to_buffer(resp, TextSensorStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(text_sensor, resp, TextSensorStateResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } uint16_t APIConnection::try_send_text_sensor_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { auto *text_sensor = static_cast(entity); ListEntitiesTextSensorResponse msg; - msg.device_class = text_sensor->get_device_class(); - msg.unique_id = text_sensor->unique_id(); - if (msg.unique_id.empty()) - msg.unique_id = get_default_unique_id("text_sensor", text_sensor); - fill_entity_info_base(text_sensor, msg); - return encode_message_to_buffer(msg, ListEntitiesTextSensorResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + msg.set_device_class(text_sensor->get_device_class_ref()); + return fill_and_encode_entity_info(text_sensor, msg, ListEntitiesTextSensorResponse::MESSAGE_TYPE, conn, + remaining_size, is_single); } #endif @@ -651,13 +623,13 @@ uint16_t APIConnection::try_send_climate_state(EntityBase *entity, APIConnection bool is_single) { auto *climate = static_cast(entity); ClimateStateResponse resp; - fill_entity_state_base(climate, resp); auto traits = climate->get_traits(); resp.mode = static_cast(climate->mode); resp.action = static_cast(climate->action); - if (traits.get_supports_current_temperature()) + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_TEMPERATURE)) resp.current_temperature = climate->current_temperature; - if (traits.get_supports_two_point_target_temperature()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | + climate::CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE)) { resp.target_temperature_low = climate->target_temperature_low; resp.target_temperature_high = climate->target_temperature_high; } else { @@ -665,53 +637,52 @@ uint16_t APIConnection::try_send_climate_state(EntityBase *entity, APIConnection } if (traits.get_supports_fan_modes() && climate->fan_mode.has_value()) resp.fan_mode = static_cast(climate->fan_mode.value()); - if (!traits.get_supported_custom_fan_modes().empty() && climate->custom_fan_mode.has_value()) - resp.custom_fan_mode = climate->custom_fan_mode.value(); + if (!traits.get_supported_custom_fan_modes().empty() && climate->custom_fan_mode.has_value()) { + resp.set_custom_fan_mode(StringRef(climate->custom_fan_mode.value())); + } if (traits.get_supports_presets() && climate->preset.has_value()) { resp.preset = static_cast(climate->preset.value()); } - if (!traits.get_supported_custom_presets().empty() && climate->custom_preset.has_value()) - resp.custom_preset = climate->custom_preset.value(); + if (!traits.get_supported_custom_presets().empty() && climate->custom_preset.has_value()) { + resp.set_custom_preset(StringRef(climate->custom_preset.value())); + } if (traits.get_supports_swing_modes()) resp.swing_mode = static_cast(climate->swing_mode); - if (traits.get_supports_current_humidity()) + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_HUMIDITY)) resp.current_humidity = climate->current_humidity; - if (traits.get_supports_target_humidity()) + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TARGET_HUMIDITY)) resp.target_humidity = climate->target_humidity; - return encode_message_to_buffer(resp, ClimateStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(climate, resp, ClimateStateResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } uint16_t APIConnection::try_send_climate_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { auto *climate = static_cast(entity); ListEntitiesClimateResponse msg; auto traits = climate->get_traits(); - msg.supports_current_temperature = traits.get_supports_current_temperature(); - msg.supports_current_humidity = traits.get_supports_current_humidity(); - msg.supports_two_point_target_temperature = traits.get_supports_two_point_target_temperature(); - msg.supports_target_humidity = traits.get_supports_target_humidity(); - for (auto mode : traits.get_supported_modes()) - msg.supported_modes.push_back(static_cast(mode)); + // Flags set for backward compatibility, deprecated in 2025.11.0 + msg.supports_current_temperature = traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_TEMPERATURE); + msg.supports_current_humidity = traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_HUMIDITY); + msg.supports_two_point_target_temperature = traits.has_feature_flags( + climate::CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | climate::CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE); + msg.supports_target_humidity = traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TARGET_HUMIDITY); + msg.supports_action = traits.has_feature_flags(climate::CLIMATE_SUPPORTS_ACTION); + // Current feature flags and other supported parameters + msg.feature_flags = traits.get_feature_flags(); + msg.supported_modes = &traits.get_supported_modes_for_api_(); msg.visual_min_temperature = traits.get_visual_min_temperature(); msg.visual_max_temperature = traits.get_visual_max_temperature(); msg.visual_target_temperature_step = traits.get_visual_target_temperature_step(); msg.visual_current_temperature_step = traits.get_visual_current_temperature_step(); msg.visual_min_humidity = traits.get_visual_min_humidity(); msg.visual_max_humidity = traits.get_visual_max_humidity(); - msg.legacy_supports_away = traits.supports_preset(climate::CLIMATE_PRESET_AWAY); - msg.supports_action = traits.get_supports_action(); - for (auto fan_mode : traits.get_supported_fan_modes()) - msg.supported_fan_modes.push_back(static_cast(fan_mode)); - for (auto const &custom_fan_mode : traits.get_supported_custom_fan_modes()) - msg.supported_custom_fan_modes.push_back(custom_fan_mode); - for (auto preset : traits.get_supported_presets()) - msg.supported_presets.push_back(static_cast(preset)); - for (auto const &custom_preset : traits.get_supported_custom_presets()) - msg.supported_custom_presets.push_back(custom_preset); - for (auto swing_mode : traits.get_supported_swing_modes()) - msg.supported_swing_modes.push_back(static_cast(swing_mode)); - msg.unique_id = get_default_unique_id("climate", climate); - fill_entity_info_base(climate, msg); - return encode_message_to_buffer(msg, ListEntitiesClimateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + msg.supported_fan_modes = &traits.get_supported_fan_modes_for_api_(); + msg.supported_custom_fan_modes = &traits.get_supported_custom_fan_modes_for_api_(); + msg.supported_presets = &traits.get_supported_presets_for_api_(); + msg.supported_custom_presets = &traits.get_supported_custom_presets_for_api_(); + msg.supported_swing_modes = &traits.get_supported_swing_modes_for_api_(); + return fill_and_encode_entity_info(climate, msg, ListEntitiesClimateResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::climate_command(const ClimateCommandRequest &msg) { ENTITY_COMMAND_MAKE_CALL(climate::Climate, climate, climate) @@ -751,23 +722,21 @@ uint16_t APIConnection::try_send_number_state(EntityBase *entity, APIConnection NumberStateResponse resp; resp.state = number->state; resp.missing_state = !number->has_state(); - fill_entity_state_base(number, resp); - return encode_message_to_buffer(resp, NumberStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(number, resp, NumberStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } uint16_t APIConnection::try_send_number_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { auto *number = static_cast(entity); ListEntitiesNumberResponse msg; - msg.unit_of_measurement = number->traits.get_unit_of_measurement(); + msg.set_unit_of_measurement(number->traits.get_unit_of_measurement_ref()); msg.mode = static_cast(number->traits.get_mode()); - msg.device_class = number->traits.get_device_class(); + msg.set_device_class(number->traits.get_device_class_ref()); msg.min_value = number->traits.get_min_value(); msg.max_value = number->traits.get_max_value(); msg.step = number->traits.get_step(); - msg.unique_id = get_default_unique_id("number", number); - fill_entity_info_base(number, msg); - return encode_message_to_buffer(msg, ListEntitiesNumberResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_info(number, msg, ListEntitiesNumberResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::number_command(const NumberCommandRequest &msg) { ENTITY_COMMAND_MAKE_CALL(number::Number, number, number) @@ -789,16 +758,14 @@ uint16_t APIConnection::try_send_date_state(EntityBase *entity, APIConnection *c resp.year = date->year; resp.month = date->month; resp.day = date->day; - fill_entity_state_base(date, resp); - return encode_message_to_buffer(resp, DateStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(date, resp, DateStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } uint16_t APIConnection::try_send_date_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { auto *date = static_cast(entity); ListEntitiesDateResponse msg; - msg.unique_id = get_default_unique_id("date", date); - fill_entity_info_base(date, msg); - return encode_message_to_buffer(msg, ListEntitiesDateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_info(date, msg, ListEntitiesDateResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::date_command(const DateCommandRequest &msg) { ENTITY_COMMAND_MAKE_CALL(datetime::DateEntity, date, date) @@ -820,16 +787,14 @@ uint16_t APIConnection::try_send_time_state(EntityBase *entity, APIConnection *c resp.hour = time->hour; resp.minute = time->minute; resp.second = time->second; - fill_entity_state_base(time, resp); - return encode_message_to_buffer(resp, TimeStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(time, resp, TimeStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } uint16_t APIConnection::try_send_time_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { auto *time = static_cast(entity); ListEntitiesTimeResponse msg; - msg.unique_id = get_default_unique_id("time", time); - fill_entity_info_base(time, msg); - return encode_message_to_buffer(msg, ListEntitiesTimeResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_info(time, msg, ListEntitiesTimeResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::time_command(const TimeCommandRequest &msg) { ENTITY_COMMAND_MAKE_CALL(datetime::TimeEntity, time, time) @@ -852,16 +817,15 @@ uint16_t APIConnection::try_send_datetime_state(EntityBase *entity, APIConnectio ESPTime state = datetime->state_as_esptime(); resp.epoch_seconds = state.timestamp; } - fill_entity_state_base(datetime, resp); - return encode_message_to_buffer(resp, DateTimeStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(datetime, resp, DateTimeStateResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } uint16_t APIConnection::try_send_datetime_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { auto *datetime = static_cast(entity); ListEntitiesDateTimeResponse msg; - msg.unique_id = get_default_unique_id("datetime", datetime); - fill_entity_info_base(datetime, msg); - return encode_message_to_buffer(msg, ListEntitiesDateTimeResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_info(datetime, msg, ListEntitiesDateTimeResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::datetime_command(const DateTimeCommandRequest &msg) { ENTITY_COMMAND_MAKE_CALL(datetime::DateTimeEntity, datetime, datetime) @@ -880,10 +844,9 @@ uint16_t APIConnection::try_send_text_state(EntityBase *entity, APIConnection *c bool is_single) { auto *text = static_cast(entity); TextStateResponse resp; - resp.state = text->state; + resp.set_state(StringRef(text->state)); resp.missing_state = !text->has_state(); - fill_entity_state_base(text, resp); - return encode_message_to_buffer(resp, TextStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(text, resp, TextStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } uint16_t APIConnection::try_send_text_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, @@ -893,10 +856,9 @@ uint16_t APIConnection::try_send_text_info(EntityBase *entity, APIConnection *co msg.mode = static_cast(text->traits.get_mode()); msg.min_length = text->traits.get_min_length(); msg.max_length = text->traits.get_max_length(); - msg.pattern = text->traits.get_pattern(); - msg.unique_id = get_default_unique_id("text", text); - fill_entity_info_base(text, msg); - return encode_message_to_buffer(msg, ListEntitiesTextResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + msg.set_pattern(text->traits.get_pattern_ref()); + return fill_and_encode_entity_info(text, msg, ListEntitiesTextResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::text_command(const TextCommandRequest &msg) { ENTITY_COMMAND_MAKE_CALL(text::Text, text, text) @@ -915,21 +877,18 @@ uint16_t APIConnection::try_send_select_state(EntityBase *entity, APIConnection bool is_single) { auto *select = static_cast(entity); SelectStateResponse resp; - resp.state = select->state; + resp.set_state(StringRef(select->state)); resp.missing_state = !select->has_state(); - fill_entity_state_base(select, resp); - return encode_message_to_buffer(resp, SelectStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(select, resp, SelectStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } uint16_t APIConnection::try_send_select_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { auto *select = static_cast(entity); ListEntitiesSelectResponse msg; - for (const auto &option : select->traits.get_options()) - msg.options.push_back(option); - msg.unique_id = get_default_unique_id("select", select); - fill_entity_info_base(select, msg); - return encode_message_to_buffer(msg, ListEntitiesSelectResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + msg.options = &select->traits.get_options(); + return fill_and_encode_entity_info(select, msg, ListEntitiesSelectResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::select_command(const SelectCommandRequest &msg) { ENTITY_COMMAND_MAKE_CALL(select::Select, select, select) @@ -943,10 +902,9 @@ uint16_t APIConnection::try_send_button_info(EntityBase *entity, APIConnection * bool is_single) { auto *button = static_cast(entity); ListEntitiesButtonResponse msg; - msg.device_class = button->get_device_class(); - msg.unique_id = get_default_unique_id("button", button); - fill_entity_info_base(button, msg); - return encode_message_to_buffer(msg, ListEntitiesButtonResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + msg.set_device_class(button->get_device_class_ref()); + return fill_and_encode_entity_info(button, msg, ListEntitiesButtonResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void esphome::api::APIConnection::button_command(const ButtonCommandRequest &msg) { ENTITY_COMMAND_GET(button::Button, button, button) @@ -965,8 +923,7 @@ uint16_t APIConnection::try_send_lock_state(EntityBase *entity, APIConnection *c auto *a_lock = static_cast(entity); LockStateResponse resp; resp.state = static_cast(a_lock->state); - fill_entity_state_base(a_lock, resp); - return encode_message_to_buffer(resp, LockStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(a_lock, resp, LockStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } uint16_t APIConnection::try_send_lock_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, @@ -976,9 +933,8 @@ uint16_t APIConnection::try_send_lock_info(EntityBase *entity, APIConnection *co msg.assumed_state = a_lock->traits.get_assumed_state(); msg.supports_open = a_lock->traits.get_supports_open(); msg.requires_code = a_lock->traits.get_requires_code(); - msg.unique_id = get_default_unique_id("lock", a_lock); - fill_entity_info_base(a_lock, msg); - return encode_message_to_buffer(msg, ListEntitiesLockResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_info(a_lock, msg, ListEntitiesLockResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::lock_command(const LockCommandRequest &msg) { ENTITY_COMMAND_GET(lock::Lock, a_lock, lock) @@ -1008,21 +964,19 @@ uint16_t APIConnection::try_send_valve_state(EntityBase *entity, APIConnection * ValveStateResponse resp; resp.position = valve->position; resp.current_operation = static_cast(valve->current_operation); - fill_entity_state_base(valve, resp); - return encode_message_to_buffer(resp, ValveStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(valve, resp, ValveStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } uint16_t APIConnection::try_send_valve_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { auto *valve = static_cast(entity); ListEntitiesValveResponse msg; auto traits = valve->get_traits(); - msg.device_class = valve->get_device_class(); + msg.set_device_class(valve->get_device_class_ref()); msg.assumed_state = traits.get_is_assumed_state(); msg.supports_position = traits.get_supports_position(); msg.supports_stop = traits.get_supports_stop(); - msg.unique_id = get_default_unique_id("valve", valve); - fill_entity_info_base(valve, msg); - return encode_message_to_buffer(msg, ListEntitiesValveResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_info(valve, msg, ListEntitiesValveResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::valve_command(const ValveCommandRequest &msg) { ENTITY_COMMAND_MAKE_CALL(valve::Valve, valve, valve) @@ -1049,8 +1003,8 @@ uint16_t APIConnection::try_send_media_player_state(EntityBase *entity, APIConne resp.state = static_cast(report_state); resp.volume = media_player->volume; resp.muted = media_player->is_muted(); - fill_entity_state_base(media_player, resp); - return encode_message_to_buffer(resp, MediaPlayerStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(media_player, resp, MediaPlayerStateResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } uint16_t APIConnection::try_send_media_player_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { @@ -1058,18 +1012,18 @@ uint16_t APIConnection::try_send_media_player_info(EntityBase *entity, APIConnec ListEntitiesMediaPlayerResponse msg; auto traits = media_player->get_traits(); msg.supports_pause = traits.get_supports_pause(); + msg.feature_flags = traits.get_feature_flags(); for (auto &supported_format : traits.get_supported_formats()) { - MediaPlayerSupportedFormat media_format; - media_format.format = supported_format.format; + msg.supported_formats.emplace_back(); + auto &media_format = msg.supported_formats.back(); + media_format.set_format(StringRef(supported_format.format)); media_format.sample_rate = supported_format.sample_rate; media_format.num_channels = supported_format.num_channels; media_format.purpose = static_cast(supported_format.purpose); media_format.sample_bytes = supported_format.sample_bytes; - msg.supported_formats.push_back(media_format); } - msg.unique_id = get_default_unique_id("media_player", media_player); - fill_entity_info_base(media_player, msg); - return encode_message_to_buffer(msg, ListEntitiesMediaPlayerResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_info(media_player, msg, ListEntitiesMediaPlayerResponse::MESSAGE_TYPE, conn, + remaining_size, is_single); } void APIConnection::media_player_command(const MediaPlayerCommandRequest &msg) { ENTITY_COMMAND_MAKE_CALL(media_player::MediaPlayer, media_player, media_player) @@ -1104,9 +1058,8 @@ uint16_t APIConnection::try_send_camera_info(EntityBase *entity, APIConnection * bool is_single) { auto *camera = static_cast(entity); ListEntitiesCameraResponse msg; - msg.unique_id = get_default_unique_id("camera", camera); - fill_entity_info_base(camera, msg); - return encode_message_to_buffer(msg, ListEntitiesCameraResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_info(camera, msg, ListEntitiesCameraResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::camera_image(const CameraImageRequest &msg) { if (camera::Camera::instance() == nullptr) @@ -1125,8 +1078,20 @@ void APIConnection::camera_image(const CameraImageRequest &msg) { #ifdef USE_HOMEASSISTANT_TIME void APIConnection::on_get_time_response(const GetTimeResponse &value) { - if (homeassistant::global_homeassistant_time != nullptr) + if (homeassistant::global_homeassistant_time != nullptr) { homeassistant::global_homeassistant_time->set_epoch_time(value.epoch_seconds); +#ifdef USE_TIME_TIMEZONE + if (value.timezone_len > 0) { + const std::string ¤t_tz = homeassistant::global_homeassistant_time->get_timezone(); + // Compare without allocating a string + if (current_tz.length() != value.timezone_len || + memcmp(current_tz.c_str(), value.timezone, value.timezone_len) != 0) { + homeassistant::global_homeassistant_time->set_timezone( + std::string(reinterpret_cast(value.timezone), value.timezone_len)); + } + } +#endif + } } #endif @@ -1137,21 +1102,6 @@ void APIConnection::subscribe_bluetooth_le_advertisements(const SubscribeBluetoo void APIConnection::unsubscribe_bluetooth_le_advertisements(const UnsubscribeBluetoothLEAdvertisementsRequest &msg) { bluetooth_proxy::global_bluetooth_proxy->unsubscribe_api_connection(this); } -bool APIConnection::send_bluetooth_le_advertisement(const BluetoothLEAdvertisementResponse &msg) { - if (this->client_api_version_major_ < 1 || this->client_api_version_minor_ < 7) { - BluetoothLEAdvertisementResponse resp = msg; - for (auto &service : resp.service_data) { - service.legacy_data.assign(service.data.begin(), service.data.end()); - service.data.clear(); - } - for (auto &manufacturer_data : resp.manufacturer_data) { - manufacturer_data.legacy_data.assign(manufacturer_data.data.begin(), manufacturer_data.data.end()); - manufacturer_data.data.clear(); - } - return this->send_message(resp); - } - return this->send_message(msg); -} void APIConnection::bluetooth_device_request(const BluetoothDeviceRequest &msg) { bluetooth_proxy::global_bluetooth_proxy->bluetooth_device_request(msg); } @@ -1175,12 +1125,10 @@ void APIConnection::bluetooth_gatt_notify(const BluetoothGATTNotifyRequest &msg) bluetooth_proxy::global_bluetooth_proxy->bluetooth_gatt_notify(msg); } -BluetoothConnectionsFreeResponse APIConnection::subscribe_bluetooth_connections_free( +bool APIConnection::send_subscribe_bluetooth_connections_free_response( const SubscribeBluetoothConnectionsFreeRequest &msg) { - BluetoothConnectionsFreeResponse resp; - resp.free = bluetooth_proxy::global_bluetooth_proxy->get_bluetooth_connections_free(); - resp.limit = bluetooth_proxy::global_bluetooth_proxy->get_bluetooth_connections_limit(); - return resp; + bluetooth_proxy::global_bluetooth_proxy->send_connections_free(this); + return true; } void APIConnection::bluetooth_scanner_set_mode(const BluetoothScannerSetModeRequest &msg) { @@ -1241,28 +1189,42 @@ void APIConnection::on_voice_assistant_announce_request(const VoiceAssistantAnno } } -VoiceAssistantConfigurationResponse APIConnection::voice_assistant_get_configuration( - const VoiceAssistantConfigurationRequest &msg) { +bool APIConnection::send_voice_assistant_get_configuration_response(const VoiceAssistantConfigurationRequest &msg) { VoiceAssistantConfigurationResponse resp; if (!this->check_voice_assistant_api_connection_()) { - return resp; + return this->send_message(resp, VoiceAssistantConfigurationResponse::MESSAGE_TYPE); } auto &config = voice_assistant::global_voice_assistant->get_configuration(); for (auto &wake_word : config.available_wake_words) { - VoiceAssistantWakeWord resp_wake_word; - resp_wake_word.id = wake_word.id; - resp_wake_word.wake_word = wake_word.wake_word; + resp.available_wake_words.emplace_back(); + auto &resp_wake_word = resp.available_wake_words.back(); + resp_wake_word.set_id(StringRef(wake_word.id)); + resp_wake_word.set_wake_word(StringRef(wake_word.wake_word)); for (const auto &lang : wake_word.trained_languages) { resp_wake_word.trained_languages.push_back(lang); } - resp.available_wake_words.push_back(std::move(resp_wake_word)); } - for (auto &wake_word_id : config.active_wake_words) { - resp.active_wake_words.push_back(wake_word_id); + + // Filter external wake words + for (auto &wake_word : msg.external_wake_words) { + if (wake_word.model_type != "micro") { + // microWakeWord only + continue; + } + + resp.available_wake_words.emplace_back(); + auto &resp_wake_word = resp.available_wake_words.back(); + resp_wake_word.set_id(StringRef(wake_word.id)); + resp_wake_word.set_wake_word(StringRef(wake_word.wake_word)); + for (const auto &lang : wake_word.trained_languages) { + resp_wake_word.trained_languages.push_back(lang); + } } + + resp.active_wake_words = &config.active_wake_words; resp.max_active_wake_words = config.max_active_wake_words; - return resp; + return this->send_message(resp, VoiceAssistantConfigurationResponse::MESSAGE_TYPE); } void APIConnection::voice_assistant_set_configuration(const VoiceAssistantSetConfiguration &msg) { @@ -1270,7 +1232,16 @@ void APIConnection::voice_assistant_set_configuration(const VoiceAssistantSetCon voice_assistant::global_voice_assistant->on_set_configuration(msg.active_wake_words); } } +#endif +#ifdef USE_ZWAVE_PROXY +void APIConnection::zwave_proxy_frame(const ZWaveProxyFrame &msg) { + zwave_proxy::global_zwave_proxy->send_frame(msg.data, msg.data_len); +} + +void APIConnection::zwave_proxy_request(const ZWaveProxyRequest &msg) { + zwave_proxy::global_zwave_proxy->zwave_proxy_request(this, msg.type); +} #endif #ifdef USE_ALARM_CONTROL_PANEL @@ -1284,8 +1255,8 @@ uint16_t APIConnection::try_send_alarm_control_panel_state(EntityBase *entity, A auto *a_alarm_control_panel = static_cast(entity); AlarmControlPanelStateResponse resp; resp.state = static_cast(a_alarm_control_panel->get_state()); - fill_entity_state_base(a_alarm_control_panel, resp); - return encode_message_to_buffer(resp, AlarmControlPanelStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(a_alarm_control_panel, resp, AlarmControlPanelStateResponse::MESSAGE_TYPE, conn, + remaining_size, is_single); } uint16_t APIConnection::try_send_alarm_control_panel_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { @@ -1294,10 +1265,8 @@ uint16_t APIConnection::try_send_alarm_control_panel_info(EntityBase *entity, AP msg.supported_features = a_alarm_control_panel->get_supported_features(); msg.requires_code = a_alarm_control_panel->get_requires_code(); msg.requires_code_to_arm = a_alarm_control_panel->get_requires_code_to_arm(); - msg.unique_id = get_default_unique_id("alarm_control_panel", a_alarm_control_panel); - fill_entity_info_base(a_alarm_control_panel, msg); - return encode_message_to_buffer(msg, ListEntitiesAlarmControlPanelResponse::MESSAGE_TYPE, conn, remaining_size, - is_single); + return fill_and_encode_entity_info(a_alarm_control_panel, msg, ListEntitiesAlarmControlPanelResponse::MESSAGE_TYPE, + conn, remaining_size, is_single); } void APIConnection::alarm_control_panel_command(const AlarmControlPanelCommandRequest &msg) { ENTITY_COMMAND_MAKE_CALL(alarm_control_panel::AlarmControlPanel, a_alarm_control_panel, alarm_control_panel) @@ -1337,21 +1306,19 @@ void APIConnection::send_event(event::Event *event, const std::string &event_typ uint16_t APIConnection::try_send_event_response(event::Event *event, const std::string &event_type, APIConnection *conn, uint32_t remaining_size, bool is_single) { EventResponse resp; - resp.event_type = event_type; - fill_entity_state_base(event, resp); - return encode_message_to_buffer(resp, EventResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + resp.set_event_type(StringRef(event_type)); + return fill_and_encode_entity_state(event, resp, EventResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } uint16_t APIConnection::try_send_event_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { auto *event = static_cast(entity); ListEntitiesEventResponse msg; - msg.device_class = event->get_device_class(); + msg.set_device_class(event->get_device_class_ref()); for (const auto &event_type : event->get_event_types()) msg.event_types.push_back(event_type); - msg.unique_id = get_default_unique_id("event", event); - fill_entity_info_base(event, msg); - return encode_message_to_buffer(msg, ListEntitiesEventResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_info(event, msg, ListEntitiesEventResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } #endif @@ -1371,23 +1338,21 @@ uint16_t APIConnection::try_send_update_state(EntityBase *entity, APIConnection resp.has_progress = true; resp.progress = update->update_info.progress; } - resp.current_version = update->update_info.current_version; - resp.latest_version = update->update_info.latest_version; - resp.title = update->update_info.title; - resp.release_summary = update->update_info.summary; - resp.release_url = update->update_info.release_url; + resp.set_current_version(StringRef(update->update_info.current_version)); + resp.set_latest_version(StringRef(update->update_info.latest_version)); + resp.set_title(StringRef(update->update_info.title)); + resp.set_release_summary(StringRef(update->update_info.summary)); + resp.set_release_url(StringRef(update->update_info.release_url)); } - fill_entity_state_base(update, resp); - return encode_message_to_buffer(resp, UpdateStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + return fill_and_encode_entity_state(update, resp, UpdateStateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); } uint16_t APIConnection::try_send_update_info(EntityBase *entity, APIConnection *conn, uint32_t remaining_size, bool is_single) { auto *update = static_cast(entity); ListEntitiesUpdateResponse msg; - msg.device_class = update->get_device_class(); - msg.unique_id = get_default_unique_id("update", update); - fill_entity_info_base(update, msg); - return encode_message_to_buffer(msg, ListEntitiesUpdateResponse::MESSAGE_TYPE, conn, remaining_size, is_single); + msg.set_device_class(update->get_device_class_ref()); + return fill_and_encode_entity_info(update, msg, ListEntitiesUpdateResponse::MESSAGE_TYPE, conn, remaining_size, + is_single); } void APIConnection::update_command(const UpdateCommandRequest &msg) { ENTITY_COMMAND_GET(update::UpdateEntity, update, update) @@ -1410,140 +1375,167 @@ void APIConnection::update_command(const UpdateCommandRequest &msg) { #endif bool APIConnection::try_send_log_message(int level, const char *tag, const char *line, size_t message_len) { - if (this->flags_.log_subscription < level) - return false; - - // Pre-calculate message size to avoid reallocations - uint32_t msg_size = 0; - - // Add size for level field (field ID 1, varint type) - // 1 byte for field tag + size of the level varint - msg_size += 1 + api::ProtoSize::varint(static_cast(level)); - - // Add size for string field (field ID 3, string type) - // 1 byte for field tag + size of length varint + string length - msg_size += 1 + api::ProtoSize::varint(static_cast(message_len)) + message_len; - - // Create a pre-sized buffer - auto buffer = this->create_buffer(msg_size); - - // Encode the message (SubscribeLogsResponse) - buffer.encode_uint32(1, static_cast(level)); // LogLevel level = 1 - buffer.encode_string(3, line, message_len); // string message = 3 - - // SubscribeLogsResponse - 29 - return this->send_buffer(buffer, SubscribeLogsResponse::MESSAGE_TYPE); + SubscribeLogsResponse msg; + msg.level = static_cast(level); + msg.set_message(reinterpret_cast(line), message_len); + return this->send_message_(msg, SubscribeLogsResponse::MESSAGE_TYPE); } -HelloResponse APIConnection::hello(const HelloRequest &msg) { - this->client_info_ = msg.client_info; - this->client_peername_ = this->helper_->getpeername(); - this->helper_->set_log_info(this->get_client_combined_info()); +void APIConnection::complete_authentication_() { + // Early return if already authenticated + if (this->flags_.connection_state == static_cast(ConnectionState::AUTHENTICATED)) { + return; + } + + this->flags_.connection_state = static_cast(ConnectionState::AUTHENTICATED); + ESP_LOGD(TAG, "%s (%s) connected", this->client_info_.name.c_str(), this->client_info_.peername.c_str()); +#ifdef USE_API_CLIENT_CONNECTED_TRIGGER + this->parent_->get_client_connected_trigger()->trigger(this->client_info_.name, this->client_info_.peername); +#endif +#ifdef USE_HOMEASSISTANT_TIME + if (homeassistant::global_homeassistant_time != nullptr) { + this->send_time_request(); + } +#endif +#ifdef USE_ZWAVE_PROXY + if (zwave_proxy::global_zwave_proxy != nullptr) { + zwave_proxy::global_zwave_proxy->api_connection_authenticated(this); + } +#endif +} + +bool APIConnection::send_hello_response(const HelloRequest &msg) { + this->client_info_.name.assign(reinterpret_cast(msg.client_info), msg.client_info_len); + this->client_info_.peername = this->helper_->getpeername(); this->client_api_version_major_ = msg.api_version_major; this->client_api_version_minor_ = msg.api_version_minor; - ESP_LOGV(TAG, "Hello from client: '%s' | %s | API Version %" PRIu32 ".%" PRIu32, this->client_info_.c_str(), - this->client_peername_.c_str(), this->client_api_version_major_, this->client_api_version_minor_); + ESP_LOGV(TAG, "Hello from client: '%s' | %s | API Version %" PRIu32 ".%" PRIu32, this->client_info_.name.c_str(), + this->client_info_.peername.c_str(), this->client_api_version_major_, this->client_api_version_minor_); HelloResponse resp; resp.api_version_major = 1; - resp.api_version_minor = 10; - resp.server_info = App.get_name() + " (esphome v" ESPHOME_VERSION ")"; - resp.name = App.get_name(); + resp.api_version_minor = 13; + // Send only the version string - the client only logs this for debugging and doesn't use it otherwise + resp.set_server_info(ESPHOME_VERSION_REF); + resp.set_name(StringRef(App.get_name())); - this->flags_.connection_state = static_cast(ConnectionState::CONNECTED); - return resp; -} -ConnectResponse APIConnection::connect(const ConnectRequest &msg) { - bool correct = true; #ifdef USE_API_PASSWORD - correct = this->parent_->check_password(msg.password); + // Password required - wait for authentication + this->flags_.connection_state = static_cast(ConnectionState::CONNECTED); +#else + // No password configured - auto-authenticate + this->complete_authentication_(); #endif - ConnectResponse resp; - // bool invalid_password = 1; - resp.invalid_password = !correct; - if (correct) { - ESP_LOGD(TAG, "%s connected", this->get_client_combined_info().c_str()); - this->flags_.connection_state = static_cast(ConnectionState::AUTHENTICATED); -#ifdef USE_API_CLIENT_CONNECTED_TRIGGER - this->parent_->get_client_connected_trigger()->trigger(this->client_info_, this->client_peername_); -#endif -#ifdef USE_HOMEASSISTANT_TIME - if (homeassistant::global_homeassistant_time != nullptr) { - this->send_time_request(); - } -#endif - } - return resp; + return this->send_message(resp, HelloResponse::MESSAGE_TYPE); } -DeviceInfoResponse APIConnection::device_info(const DeviceInfoRequest &msg) { +#ifdef USE_API_PASSWORD +bool APIConnection::send_authenticate_response(const AuthenticationRequest &msg) { + AuthenticationResponse resp; + // bool invalid_password = 1; + resp.invalid_password = !this->parent_->check_password(msg.password, msg.password_len); + if (!resp.invalid_password) { + this->complete_authentication_(); + } + return this->send_message(resp, AuthenticationResponse::MESSAGE_TYPE); +} +#endif // USE_API_PASSWORD + +bool APIConnection::send_ping_response(const PingRequest &msg) { + PingResponse resp; + return this->send_message(resp, PingResponse::MESSAGE_TYPE); +} + +bool APIConnection::send_device_info_response(const DeviceInfoRequest &msg) { DeviceInfoResponse resp{}; #ifdef USE_API_PASSWORD - resp.uses_password = this->parent_->uses_password(); -#else - resp.uses_password = false; + resp.uses_password = true; #endif - resp.name = App.get_name(); - resp.friendly_name = App.get_friendly_name(); - resp.suggested_area = App.get_area(); - resp.mac_address = get_mac_address_pretty(); - resp.esphome_version = ESPHOME_VERSION; - resp.compilation_time = App.get_compilation_time(); + resp.set_name(StringRef(App.get_name())); + resp.set_friendly_name(StringRef(App.get_friendly_name())); +#ifdef USE_AREAS + resp.set_suggested_area(StringRef(App.get_area())); +#endif + // mac_address must store temporary string - will be valid during send_message call + std::string mac_address = get_mac_address_pretty(); + resp.set_mac_address(StringRef(mac_address)); + + resp.set_esphome_version(ESPHOME_VERSION_REF); + + resp.set_compilation_time(App.get_compilation_time_ref()); + + // Compile-time StringRef constants for manufacturers #if defined(USE_ESP8266) || defined(USE_ESP32) - resp.manufacturer = "Espressif"; + static constexpr auto MANUFACTURER = StringRef::from_lit("Espressif"); #elif defined(USE_RP2040) - resp.manufacturer = "Raspberry Pi"; + static constexpr auto MANUFACTURER = StringRef::from_lit("Raspberry Pi"); #elif defined(USE_BK72XX) - resp.manufacturer = "Beken"; + static constexpr auto MANUFACTURER = StringRef::from_lit("Beken"); #elif defined(USE_LN882X) - resp.manufacturer = "Lightning"; + static constexpr auto MANUFACTURER = StringRef::from_lit("Lightning"); #elif defined(USE_RTL87XX) - resp.manufacturer = "Realtek"; + static constexpr auto MANUFACTURER = StringRef::from_lit("Realtek"); #elif defined(USE_HOST) - resp.manufacturer = "Host"; + static constexpr auto MANUFACTURER = StringRef::from_lit("Host"); #endif - resp.model = ESPHOME_BOARD; + resp.set_manufacturer(MANUFACTURER); + + static constexpr auto MODEL = StringRef::from_lit(ESPHOME_BOARD); + resp.set_model(MODEL); #ifdef USE_DEEP_SLEEP resp.has_deep_sleep = deep_sleep::global_has_deep_sleep; #endif #ifdef ESPHOME_PROJECT_NAME - resp.project_name = ESPHOME_PROJECT_NAME; - resp.project_version = ESPHOME_PROJECT_VERSION; + static constexpr auto PROJECT_NAME = StringRef::from_lit(ESPHOME_PROJECT_NAME); + static constexpr auto PROJECT_VERSION = StringRef::from_lit(ESPHOME_PROJECT_VERSION); + resp.set_project_name(PROJECT_NAME); + resp.set_project_version(PROJECT_VERSION); #endif #ifdef USE_WEBSERVER resp.webserver_port = USE_WEBSERVER_PORT; #endif #ifdef USE_BLUETOOTH_PROXY - resp.legacy_bluetooth_proxy_version = bluetooth_proxy::global_bluetooth_proxy->get_legacy_version(); resp.bluetooth_proxy_feature_flags = bluetooth_proxy::global_bluetooth_proxy->get_feature_flags(); - resp.bluetooth_mac_address = bluetooth_proxy::global_bluetooth_proxy->get_bluetooth_mac_address_pretty(); + // bt_mac must store temporary string - will be valid during send_message call + std::string bluetooth_mac = bluetooth_proxy::global_bluetooth_proxy->get_bluetooth_mac_address_pretty(); + resp.set_bluetooth_mac_address(StringRef(bluetooth_mac)); #endif #ifdef USE_VOICE_ASSISTANT - resp.legacy_voice_assistant_version = voice_assistant::global_voice_assistant->get_legacy_version(); resp.voice_assistant_feature_flags = voice_assistant::global_voice_assistant->get_feature_flags(); #endif +#ifdef USE_ZWAVE_PROXY + resp.zwave_proxy_feature_flags = zwave_proxy::global_zwave_proxy->get_feature_flags(); + resp.zwave_home_id = zwave_proxy::global_zwave_proxy->get_home_id(); +#endif #ifdef USE_API_NOISE resp.api_encryption_supported = true; #endif #ifdef USE_DEVICES + size_t device_index = 0; for (auto const &device : App.get_devices()) { - DeviceInfo device_info; + if (device_index >= ESPHOME_DEVICE_COUNT) + break; + auto &device_info = resp.devices[device_index++]; device_info.device_id = device->get_device_id(); - device_info.name = device->get_name(); + device_info.set_name(StringRef(device->get_name())); device_info.area_id = device->get_area_id(); - resp.devices.push_back(device_info); } #endif #ifdef USE_AREAS + size_t area_index = 0; for (auto const &area : App.get_areas()) { - AreaInfo area_info; + if (area_index >= ESPHOME_AREA_COUNT) + break; + auto &area_info = resp.areas[area_index++]; area_info.area_id = area->get_area_id(); - area_info.name = area->get_name(); - resp.areas.push_back(area_info); + area_info.set_name(StringRef(area->get_name())); } #endif - return resp; + + return this->send_message(resp, DeviceInfoResponse::MESSAGE_TYPE); } + +#ifdef USE_API_HOMEASSISTANT_STATES void APIConnection::on_home_assistant_state_response(const HomeAssistantStateResponse &msg) { for (auto &it : this->parent_->get_state_subs()) { if (it.entity_id == msg.entity_id && it.attribute.value() == msg.attribute) { @@ -1551,6 +1543,7 @@ void APIConnection::on_home_assistant_state_response(const HomeAssistantStateRes } } } +#endif #ifdef USE_API_SERVICES void APIConnection::execute_service(const ExecuteServiceRequest &msg) { bool found = false; @@ -1564,29 +1557,42 @@ void APIConnection::execute_service(const ExecuteServiceRequest &msg) { } } #endif + +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES +void APIConnection::on_homeassistant_action_response(const HomeassistantActionResponse &msg) { +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + if (msg.response_data_len > 0) { + this->parent_->handle_action_response(msg.call_id, msg.success, msg.error_message, msg.response_data, + msg.response_data_len); + } else +#endif + { + this->parent_->handle_action_response(msg.call_id, msg.success, msg.error_message); + } +}; +#endif #ifdef USE_API_NOISE -NoiseEncryptionSetKeyResponse APIConnection::noise_encryption_set_key(const NoiseEncryptionSetKeyRequest &msg) { - psk_t psk{}; +bool APIConnection::send_noise_encryption_set_key_response(const NoiseEncryptionSetKeyRequest &msg) { NoiseEncryptionSetKeyResponse resp; + resp.success = false; + + psk_t psk{}; if (base64_decode(msg.key, psk.data(), msg.key.size()) != psk.size()) { ESP_LOGW(TAG, "Invalid encryption key length"); - resp.success = false; - return resp; - } - - if (!this->parent_->save_noise_psk(psk, true)) { + } else if (!this->parent_->save_noise_psk(psk, true)) { ESP_LOGW(TAG, "Failed to save encryption key"); - resp.success = false; - return resp; + } else { + resp.success = true; } - resp.success = true; - return resp; + return this->send_message(resp, NoiseEncryptionSetKeyResponse::MESSAGE_TYPE); } #endif +#ifdef USE_API_HOMEASSISTANT_STATES void APIConnection::subscribe_home_assistant_states(const SubscribeHomeAssistantStatesRequest &msg) { state_subs_at_ = 0; } +#endif bool APIConnection::try_to_clear_buffer(bool log_out_of_space) { if (this->flags_.remove) return false; @@ -1595,9 +1601,7 @@ bool APIConnection::try_to_clear_buffer(bool log_out_of_space) { delay(0); APIError err = this->helper_->loop(); if (err != APIError::OK) { - on_fatal_error(); - ESP_LOGW(TAG, "%s: Socket operation failed: %s errno=%d", this->get_client_combined_info().c_str(), - api_error_to_str(err), errno); + this->fatal_error_with_log_(LOG_STR("Socket operation failed"), err); return false; } if (this->helper_->can_write_without_blocking()) @@ -1616,25 +1620,21 @@ bool APIConnection::send_buffer(ProtoWriteBuffer buffer, uint8_t message_type) { if (err == APIError::WOULD_BLOCK) return false; if (err != APIError::OK) { - on_fatal_error(); - if (err == APIError::SOCKET_WRITE_FAILED && errno == ECONNRESET) { - ESP_LOGW(TAG, "%s: Connection reset", this->get_client_combined_info().c_str()); - } else { - ESP_LOGW(TAG, "%s: Packet write failed %s errno=%d", this->get_client_combined_info().c_str(), - api_error_to_str(err), errno); - } + this->fatal_error_with_log_(LOG_STR("Packet write failed"), err); return false; } // Do not set last_traffic_ on send return true; } +#ifdef USE_API_PASSWORD void APIConnection::on_unauthenticated_access() { this->on_fatal_error(); - ESP_LOGD(TAG, "%s requested access without authentication", this->get_client_combined_info().c_str()); + ESP_LOGD(TAG, "%s (%s) no authentication", this->client_info_.name.c_str(), this->client_info_.peername.c_str()); } +#endif void APIConnection::on_no_setup_connection() { this->on_fatal_error(); - ESP_LOGD(TAG, "%s requested access without full connection", this->get_client_combined_info().c_str()); + ESP_LOGD(TAG, "%s (%s) no connection setup", this->client_info_.name.c_str(), this->client_info_.peername.c_str()); } void APIConnection::on_fatal_error() { this->helper_->close(); @@ -1662,8 +1662,15 @@ void APIConnection::DeferredBatch::add_item(EntityBase *entity, MessageCreator c void APIConnection::DeferredBatch::add_item_front(EntityBase *entity, MessageCreator creator, uint8_t message_type, uint8_t estimated_size) { - // Insert at front for high priority messages (no deduplication check) - items.insert(items.begin(), BatchItem(entity, std::move(creator), message_type, estimated_size)); + // Add high priority message and swap to front + // This avoids expensive vector::insert which shifts all elements + // Note: We only ever have one high-priority message at a time (ping OR disconnect) + // If we're disconnecting, pings are blocked, so this simple swap is sufficient + items.emplace_back(entity, std::move(creator), message_type, estimated_size); + if (items.size() > 1) { + // Swap the new high-priority item to the front + std::swap(items.front(), items.back()); + } } bool APIConnection::schedule_batch_() { @@ -1674,15 +1681,11 @@ bool APIConnection::schedule_batch_() { return true; } -ProtoWriteBuffer APIConnection::allocate_single_message_buffer(uint16_t size) { return this->create_buffer(size); } - -ProtoWriteBuffer APIConnection::allocate_batch_message_buffer(uint16_t size) { - ProtoWriteBuffer result = this->prepare_message_buffer(size, this->flags_.batch_first_message); - this->flags_.batch_first_message = false; - return result; -} - void APIConnection::process_batch_() { + // Ensure PacketInfo remains trivially destructible for our placement new approach + static_assert(std::is_trivially_destructible::value, + "PacketInfo must remain trivially destructible with this placement-new approach"); + if (this->deferred_batch_.empty()) { this->flags_.batch_scheduled = false; return; @@ -1694,6 +1697,8 @@ void APIConnection::process_batch_() { return; } + // Get shared buffer reference once to avoid multiple calls + auto &shared_buf = this->parent_->get_shared_buffer_ref(); size_t num_items = this->deferred_batch_.size(); // Fast path for single message - allocate exact size needed @@ -1704,8 +1709,7 @@ void APIConnection::process_batch_() { uint16_t payload_size = item.creator(item.entity, this, std::numeric_limits::max(), true, item.message_type); - if (payload_size > 0 && - this->send_buffer(ProtoWriteBuffer{&this->parent_->get_shared_buffer_ref()}, item.message_type)) { + if (payload_size > 0 && this->send_buffer(ProtoWriteBuffer{&shared_buf}, item.message_type)) { #ifdef HAS_PROTO_MESSAGE_DUMP // Log messages after send attempt for VV debugging // It's safe to use the buffer for logging at this point regardless of send result @@ -1720,29 +1724,30 @@ void APIConnection::process_batch_() { return; } - // Pre-allocate storage for packet info - std::vector packet_info; - packet_info.reserve(num_items); + size_t packets_to_process = std::min(num_items, MAX_PACKETS_PER_BATCH); + + // Stack-allocated array for packet info + alignas(PacketInfo) char packet_info_storage[MAX_PACKETS_PER_BATCH * sizeof(PacketInfo)]; + PacketInfo *packet_info = reinterpret_cast(packet_info_storage); + size_t packet_count = 0; // Cache these values to avoid repeated virtual calls const uint8_t header_padding = this->helper_->frame_header_padding(); const uint8_t footer_size = this->helper_->frame_footer_size(); // Initialize buffer and tracking variables - this->parent_->get_shared_buffer_ref().clear(); + shared_buf.clear(); // Pre-calculate exact buffer size needed based on message types - uint32_t total_estimated_size = 0; + uint32_t total_estimated_size = num_items * (header_padding + footer_size); for (size_t i = 0; i < this->deferred_batch_.size(); i++) { const auto &item = this->deferred_batch_[i]; total_estimated_size += item.estimated_size; } // Calculate total overhead for all messages - uint32_t total_overhead = (header_padding + footer_size) * num_items; - // Reserve based on estimated size (much more accurate than 24-byte worst-case) - this->parent_->get_shared_buffer_ref().reserve(total_estimated_size + total_overhead); + shared_buf.reserve(total_estimated_size); this->flags_.batch_first_message = true; size_t items_processed = 0; @@ -1754,8 +1759,8 @@ void APIConnection::process_batch_() { // The actual message data follows after the header padding uint32_t current_offset = 0; - // Process items and encode directly to buffer - for (size_t i = 0; i < this->deferred_batch_.size(); i++) { + // Process items and encode directly to buffer (up to our limit) + for (size_t i = 0; i < packets_to_process; i++) { const auto &item = this->deferred_batch_[i]; // Try to encode message // The creator will calculate overhead to determine if the message fits @@ -1769,7 +1774,11 @@ void APIConnection::process_batch_() { // Message was encoded successfully // payload_size is header_padding + actual payload size + footer_size uint16_t proto_payload_size = payload_size - header_padding - footer_size; - packet_info.emplace_back(item.message_type, current_offset, proto_payload_size); + // Use placement new to construct PacketInfo in pre-allocated stack array + // This avoids default-constructing all MAX_PACKETS_PER_BATCH elements + // Explicit destruction is not needed because PacketInfo is trivially destructible, + // as ensured by the static_assert in its definition. + new (&packet_info[packet_count++]) PacketInfo(item.message_type, current_offset, proto_payload_size); // Update tracking variables items_processed++; @@ -1779,8 +1788,8 @@ void APIConnection::process_batch_() { } remaining_size -= payload_size; // Calculate where the next message's header padding will start - // Current buffer size + footer space (that prepare_message_buffer will add for this message) - current_offset = this->parent_->get_shared_buffer_ref().size() + footer_size; + // Current buffer size + footer space for this message + current_offset = shared_buf.size() + footer_size; } if (items_processed == 0) { @@ -1790,21 +1799,14 @@ void APIConnection::process_batch_() { // Add footer space for the last message (for Noise protocol MAC) if (footer_size > 0) { - auto &shared_buf = this->parent_->get_shared_buffer_ref(); shared_buf.resize(shared_buf.size() + footer_size); } // Send all collected packets - APIError err = - this->helper_->write_protobuf_packets(ProtoWriteBuffer{&this->parent_->get_shared_buffer_ref()}, packet_info); + APIError err = this->helper_->write_protobuf_packets(ProtoWriteBuffer{&shared_buf}, + std::span(packet_info, packet_count)); if (err != APIError::OK && err != APIError::WOULD_BLOCK) { - on_fatal_error(); - if (err == APIError::SOCKET_WRITE_FAILED && errno == ECONNRESET) { - ESP_LOGW(TAG, "%s: Connection reset during batch write", this->get_client_combined_info().c_str()); - } else { - ESP_LOGW(TAG, "%s: Batch write failed %s errno=%d", this->get_client_combined_info().c_str(), - api_error_to_str(err), errno); - } + this->fatal_error_with_log_(LOG_STR("Batch write failed"), err); } #ifdef HAS_PROTO_MESSAGE_DUMP @@ -1860,6 +1862,32 @@ uint16_t APIConnection::try_send_ping_request(EntityBase *entity, APIConnection return encode_message_to_buffer(req, PingRequest::MESSAGE_TYPE, conn, remaining_size, is_single); } -} // namespace api -} // namespace esphome +#ifdef USE_API_HOMEASSISTANT_STATES +void APIConnection::process_state_subscriptions_() { + const auto &subs = this->parent_->get_state_subs(); + if (this->state_subs_at_ >= static_cast(subs.size())) { + this->state_subs_at_ = -1; + return; + } + + const auto &it = subs[this->state_subs_at_]; + SubscribeHomeAssistantStateResponse resp; + resp.set_entity_id(StringRef(it.entity_id)); + + // Avoid string copy by directly using the optional's value if it exists + resp.set_attribute(it.attribute.has_value() ? StringRef(it.attribute.value()) : StringRef("")); + + resp.once = it.once; + if (this->send_message(resp, SubscribeHomeAssistantStateResponse::MESSAGE_TYPE)) { + this->state_subs_at_++; + } +} +#endif // USE_API_HOMEASSISTANT_STATES + +void APIConnection::log_warning_(const LogString *message, APIError err) { + ESP_LOGW(TAG, "%s (%s): %s %s errno=%d", this->client_info_.name.c_str(), this->client_info_.peername.c_str(), + LOG_STR_ARG(message), LOG_STR_ARG(api_error_to_logstr(err)), errno); +} + +} // namespace esphome::api #endif diff --git a/esphome/components/api/api_connection.h b/esphome/components/api/api_connection.h index 0051a143de..284fa11a95 100644 --- a/esphome/components/api/api_connection.h +++ b/esphome/components/api/api_connection.h @@ -10,18 +10,33 @@ #include "esphome/core/component.h" #include "esphome/core/entity_base.h" -#include #include +#include -namespace esphome { -namespace api { +namespace esphome::api { + +// Client information structure +struct ClientInfo { + std::string name; // Client name from Hello message + std::string peername; // IP:port from socket +}; // Keepalive timeout in milliseconds static constexpr uint32_t KEEPALIVE_TIMEOUT_MS = 60000; // Maximum number of entities to process in a single batch during initial state/info sending -static constexpr size_t MAX_INITIAL_PER_BATCH = 20; +// This was increased from 20 to 24 after removing the unique_id field from entity info messages, +// which reduced message sizes allowing more entities per batch without exceeding packet limits +static constexpr size_t MAX_INITIAL_PER_BATCH = 24; +// Maximum number of packets to process in a single batch (platform-dependent) +// This limit exists to prevent stack overflow from the PacketInfo array in process_batch_ +// Each PacketInfo is 8 bytes, so 64 * 8 = 512 bytes, 32 * 8 = 256 bytes +#if defined(USE_ESP32) || defined(USE_HOST) +static constexpr size_t MAX_PACKETS_PER_BATCH = 64; // ESP32 has 8KB+ stack, HOST has plenty +#else +static constexpr size_t MAX_PACKETS_PER_BATCH = 32; // ESP8266/RP2040/etc have smaller stacks +#endif -class APIConnection : public APIServerConnection { +class APIConnection final : public APIServerConnection { public: friend class APIServer; friend class ListEntitiesIterator; @@ -108,15 +123,19 @@ class APIConnection : public APIServerConnection { void media_player_command(const MediaPlayerCommandRequest &msg) override; #endif bool try_send_log_message(int level, const char *tag, const char *line, size_t message_len); - void send_homeassistant_service_call(const HomeassistantServiceResponse &call) { +#ifdef USE_API_HOMEASSISTANT_SERVICES + void send_homeassistant_action(const HomeassistantActionRequest &call) { if (!this->flags_.service_call_subscription) return; - this->send_message(call); + this->send_message(call, HomeassistantActionRequest::MESSAGE_TYPE); } +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES + void on_homeassistant_action_response(const HomeassistantActionResponse &msg) override; +#endif // USE_API_HOMEASSISTANT_ACTION_RESPONSES +#endif // USE_API_HOMEASSISTANT_SERVICES #ifdef USE_BLUETOOTH_PROXY void subscribe_bluetooth_le_advertisements(const SubscribeBluetoothLEAdvertisementsRequest &msg) override; void unsubscribe_bluetooth_le_advertisements(const UnsubscribeBluetoothLEAdvertisementsRequest &msg) override; - bool send_bluetooth_le_advertisement(const BluetoothLEAdvertisementResponse &msg); void bluetooth_device_request(const BluetoothDeviceRequest &msg) override; void bluetooth_gatt_read(const BluetoothGATTReadRequest &msg) override; @@ -125,15 +144,14 @@ class APIConnection : public APIServerConnection { void bluetooth_gatt_write_descriptor(const BluetoothGATTWriteDescriptorRequest &msg) override; void bluetooth_gatt_get_services(const BluetoothGATTGetServicesRequest &msg) override; void bluetooth_gatt_notify(const BluetoothGATTNotifyRequest &msg) override; - BluetoothConnectionsFreeResponse subscribe_bluetooth_connections_free( - const SubscribeBluetoothConnectionsFreeRequest &msg) override; + bool send_subscribe_bluetooth_connections_free_response(const SubscribeBluetoothConnectionsFreeRequest &msg) override; void bluetooth_scanner_set_mode(const BluetoothScannerSetModeRequest &msg) override; #endif #ifdef USE_HOMEASSISTANT_TIME void send_time_request() { GetTimeRequest req; - this->send_message(req); + this->send_message(req, GetTimeRequest::MESSAGE_TYPE); } #endif @@ -144,11 +162,15 @@ class APIConnection : public APIServerConnection { void on_voice_assistant_audio(const VoiceAssistantAudio &msg) override; void on_voice_assistant_timer_event_response(const VoiceAssistantTimerEventResponse &msg) override; void on_voice_assistant_announce_request(const VoiceAssistantAnnounceRequest &msg) override; - VoiceAssistantConfigurationResponse voice_assistant_get_configuration( - const VoiceAssistantConfigurationRequest &msg) override; + bool send_voice_assistant_get_configuration_response(const VoiceAssistantConfigurationRequest &msg) override; void voice_assistant_set_configuration(const VoiceAssistantSetConfiguration &msg) override; #endif +#ifdef USE_ZWAVE_PROXY + void zwave_proxy_frame(const ZWaveProxyFrame &msg) override; + void zwave_proxy_request(const ZWaveProxyRequest &msg) override; +#endif + #ifdef USE_ALARM_CONTROL_PANEL bool send_alarm_control_panel_state(alarm_control_panel::AlarmControlPanel *a_alarm_control_panel); void alarm_control_panel_command(const AlarmControlPanelCommandRequest &msg) override; @@ -168,15 +190,19 @@ class APIConnection : public APIServerConnection { // we initiated ping this->flags_.sent_ping = false; } +#ifdef USE_API_HOMEASSISTANT_STATES void on_home_assistant_state_response(const HomeAssistantStateResponse &msg) override; +#endif #ifdef USE_HOMEASSISTANT_TIME void on_get_time_response(const GetTimeResponse &value) override; #endif - HelloResponse hello(const HelloRequest &msg) override; - ConnectResponse connect(const ConnectRequest &msg) override; - DisconnectResponse disconnect(const DisconnectRequest &msg) override; - PingResponse ping(const PingRequest &msg) override { return {}; } - DeviceInfoResponse device_info(const DeviceInfoRequest &msg) override; + bool send_hello_response(const HelloRequest &msg) override; +#ifdef USE_API_PASSWORD + bool send_authenticate_response(const AuthenticationRequest &msg) override; +#endif + bool send_disconnect_response(const DisconnectRequest &msg) override; + bool send_ping_response(const PingRequest &msg) override; + bool send_device_info_response(const DeviceInfoRequest &msg) override; void list_entities(const ListEntitiesRequest &msg) override { this->list_entities_iterator_.begin(); } void subscribe_states(const SubscribeStatesRequest &msg) override { this->flags_.state_subscription = true; @@ -187,19 +213,19 @@ class APIConnection : public APIServerConnection { if (msg.dump_config) App.schedule_dump_config(); } +#ifdef USE_API_HOMEASSISTANT_SERVICES void subscribe_homeassistant_services(const SubscribeHomeassistantServicesRequest &msg) override { this->flags_.service_call_subscription = true; } +#endif +#ifdef USE_API_HOMEASSISTANT_STATES void subscribe_home_assistant_states(const SubscribeHomeAssistantStatesRequest &msg) override; - GetTimeResponse get_time(const GetTimeRequest &msg) override { - // TODO - return {}; - } +#endif #ifdef USE_API_SERVICES void execute_service(const ExecuteServiceRequest &msg) override; #endif #ifdef USE_API_NOISE - NoiseEncryptionSetKeyResponse noise_encryption_set_key(const NoiseEncryptionSetKeyRequest &msg) override; + bool send_noise_encryption_set_key_response(const NoiseEncryptionSetKeyRequest &msg) override; #endif bool is_authenticated() override { @@ -209,100 +235,102 @@ class APIConnection : public APIServerConnection { return static_cast(this->flags_.connection_state) == ConnectionState::CONNECTED || this->is_authenticated(); } + uint8_t get_log_subscription_level() const { return this->flags_.log_subscription; } + + // Get client API version for feature detection + bool client_supports_api_version(uint16_t major, uint16_t minor) const { + return this->client_api_version_major_ > major || + (this->client_api_version_major_ == major && this->client_api_version_minor_ >= minor); + } + void on_fatal_error() override; +#ifdef USE_API_PASSWORD void on_unauthenticated_access() override; +#endif void on_no_setup_connection() override; ProtoWriteBuffer create_buffer(uint32_t reserve_size) override { // FIXME: ensure no recursive writes can happen // Get header padding size - used for both reserve and insert uint8_t header_padding = this->helper_->frame_header_padding(); - // Get shared buffer from parent server std::vector &shared_buf = this->parent_->get_shared_buffer_ref(); + this->prepare_first_message_buffer(shared_buf, header_padding, + reserve_size + header_padding + this->helper_->frame_footer_size()); + return {&shared_buf}; + } + + void prepare_first_message_buffer(std::vector &shared_buf, size_t header_padding, size_t total_size) { shared_buf.clear(); // Reserve space for header padding + message + footer // - Header padding: space for protocol headers (7 bytes for Noise, 6 for Plaintext) // - Footer: space for MAC (16 bytes for Noise, 0 for Plaintext) - shared_buf.reserve(reserve_size + header_padding + this->helper_->frame_footer_size()); + shared_buf.reserve(total_size); // Resize to add header padding so message encoding starts at the correct position shared_buf.resize(header_padding); - return {&shared_buf}; - } - - // Prepare buffer for next message in batch - ProtoWriteBuffer prepare_message_buffer(uint16_t message_size, bool is_first_message) { - // Get reference to shared buffer (it maintains state between batch messages) - std::vector &shared_buf = this->parent_->get_shared_buffer_ref(); - - if (is_first_message) { - shared_buf.clear(); - } - - size_t current_size = shared_buf.size(); - - // Calculate padding to add: - // - First message: just header padding - // - Subsequent messages: footer for previous message + header padding for this message - size_t padding_to_add = is_first_message - ? this->helper_->frame_header_padding() - : this->helper_->frame_header_padding() + this->helper_->frame_footer_size(); - - // Reserve space for padding + message - shared_buf.reserve(current_size + padding_to_add + message_size); - - // Resize to add the padding bytes - shared_buf.resize(current_size + padding_to_add); - - return {&shared_buf}; } bool try_to_clear_buffer(bool log_out_of_space); bool send_buffer(ProtoWriteBuffer buffer, uint8_t message_type) override; - std::string get_client_combined_info() const { - if (this->client_info_ == this->client_peername_) { - // Before Hello message, both are the same (just IP:port) - return this->client_info_; - } - return this->client_info_ + " (" + this->client_peername_ + ")"; - } - - // Buffer allocator methods for batch processing - ProtoWriteBuffer allocate_single_message_buffer(uint16_t size); - ProtoWriteBuffer allocate_batch_message_buffer(uint16_t size); + const std::string &get_name() const { return this->client_info_.name; } + const std::string &get_peername() const { return this->client_info_.peername; } protected: - // Helper function to fill common entity info fields - static void fill_entity_info_base(esphome::EntityBase *entity, InfoResponseProtoMessage &response) { - // Set common fields that are shared by all entity types - response.key = entity->get_object_id_hash(); - response.object_id = entity->get_object_id(); + // Helper function to handle authentication completion + void complete_authentication_(); - if (entity->has_own_name()) - response.name = entity->get_name(); - - // Set common EntityBase properties - response.icon = entity->get_icon(); - response.disabled_by_default = entity->is_disabled_by_default(); - response.entity_category = static_cast(entity->get_entity_category()); -#ifdef USE_DEVICES - response.device_id = entity->get_device_id(); +#ifdef USE_API_HOMEASSISTANT_STATES + void process_state_subscriptions_(); #endif - } - - // Helper function to fill common entity state fields - static void fill_entity_state_base(esphome::EntityBase *entity, StateResponseProtoMessage &response) { - response.key = entity->get_object_id_hash(); -#ifdef USE_DEVICES - response.device_id = entity->get_device_id(); -#endif - } // Non-template helper to encode any ProtoMessage static uint16_t encode_message_to_buffer(ProtoMessage &msg, uint8_t message_type, APIConnection *conn, uint32_t remaining_size, bool is_single); + // Helper to fill entity state base and encode message + static uint16_t fill_and_encode_entity_state(EntityBase *entity, StateResponseProtoMessage &msg, uint8_t message_type, + APIConnection *conn, uint32_t remaining_size, bool is_single) { + msg.key = entity->get_object_id_hash(); +#ifdef USE_DEVICES + msg.device_id = entity->get_device_id(); +#endif + return encode_message_to_buffer(msg, message_type, conn, remaining_size, is_single); + } + + // Helper to fill entity info base and encode message + static uint16_t fill_and_encode_entity_info(EntityBase *entity, InfoResponseProtoMessage &msg, uint8_t message_type, + APIConnection *conn, uint32_t remaining_size, bool is_single) { + // Set common fields that are shared by all entity types + msg.key = entity->get_object_id_hash(); + // Try to use static reference first to avoid allocation + StringRef static_ref = entity->get_object_id_ref_for_api_(); + // Store dynamic string outside the if-else to maintain lifetime + std::string object_id; + if (!static_ref.empty()) { + msg.set_object_id(static_ref); + } else { + // Dynamic case - need to allocate + object_id = entity->get_object_id(); + msg.set_object_id(StringRef(object_id)); + } + + if (entity->has_own_name()) { + msg.set_name(entity->get_name()); + } + + // Set common EntityBase properties +#ifdef USE_ENTITY_ICON + msg.set_icon(entity->get_icon_ref()); +#endif + msg.disabled_by_default = entity->is_disabled_by_default(); + msg.entity_category = static_cast(entity->get_entity_category()); +#ifdef USE_DEVICES + msg.device_id = entity->get_device_id(); +#endif + return encode_message_to_buffer(msg, message_type, conn, remaining_size, is_single); + } + #ifdef USE_VOICE_ASSISTANT // Helper to check voice assistant validity and connection ownership inline bool check_voice_assistant_api_connection_() const; @@ -463,13 +491,14 @@ class APIConnection : public APIServerConnection { std::unique_ptr image_reader_; #endif - // Group 3: Strings (12 bytes each on 32-bit, 4-byte aligned) - std::string client_info_; - std::string client_peername_; + // Group 3: Client info struct (24 bytes on 32-bit: 2 strings × 12 bytes each) + ClientInfo client_info_; // Group 4: 4-byte types uint32_t last_traffic_; +#ifdef USE_API_HOMEASSISTANT_STATES int state_subs_at_ = -1; +#endif // Function pointer type for message encoding using MessageCreatorPtr = uint16_t (*)(EntityBase *, APIConnection *, uint32_t remaining_size, bool is_single); @@ -657,10 +686,16 @@ class APIConnection : public APIServerConnection { bool send_message_smart_(EntityBase *entity, MessageCreatorPtr creator, uint8_t message_type, uint8_t estimated_size) { // Try to send immediately if: - // 1. We should try to send immediately (should_try_send_immediately = true) - // 2. Batch delay is 0 (user has opted in to immediate sending) - // 3. Buffer has space available - if (this->flags_.should_try_send_immediately && this->get_batch_delay_ms_() == 0 && + // 1. It's an UpdateStateResponse (always send immediately to handle cases where + // the main loop is blocked, e.g., during OTA updates) + // 2. OR: We should try to send immediately (should_try_send_immediately = true) + // AND Batch delay is 0 (user has opted in to immediate sending) + // 3. AND: Buffer has space available + if (( +#ifdef USE_UPDATE + message_type == UpdateStateResponse::MESSAGE_TYPE || +#endif + (this->flags_.should_try_send_immediately && this->get_batch_delay_ms_() == 0)) && this->helper_->can_write_without_blocking()) { // Now actually encode and send if (creator(entity, this, MAX_BATCH_PACKET_SIZE, true) && @@ -697,8 +732,15 @@ class APIConnection : public APIServerConnection { this->deferred_batch_.add_item_front(entity, MessageCreator(function_ptr), message_type, estimated_size); return this->schedule_batch_(); } + + // Helper function to log API errors with errno + void log_warning_(const LogString *message, APIError err); + // Helper to handle fatal errors with logging + inline void fatal_error_with_log_(const LogString *message, APIError err) { + this->on_fatal_error(); + this->log_warning_(message, err); + } }; -} // namespace api -} // namespace esphome +} // namespace esphome::api #endif diff --git a/esphome/components/api/api_frame_helper.cpp b/esphome/components/api/api_frame_helper.cpp index afd64e8981..20f8fcaf61 100644 --- a/esphome/components/api/api_frame_helper.cpp +++ b/esphome/components/api/api_frame_helper.cpp @@ -1,5 +1,6 @@ #include "api_frame_helper.h" #ifdef USE_API +#include "api_connection.h" // For ClientInfo struct #include "esphome/core/application.h" #include "esphome/core/hal.h" #include "esphome/core/helpers.h" @@ -8,66 +9,79 @@ #include #include -namespace esphome { -namespace api { +namespace esphome::api { -static const char *const TAG = "api.socket"; +static const char *const TAG = "api.frame_helper"; -const char *api_error_to_str(APIError err) { +#define HELPER_LOG(msg, ...) \ + ESP_LOGVV(TAG, "%s (%s): " msg, this->client_info_->name.c_str(), this->client_info_->peername.c_str(), ##__VA_ARGS__) + +#ifdef HELPER_LOG_PACKETS +#define LOG_PACKET_RECEIVED(buffer) ESP_LOGVV(TAG, "Received frame: %s", format_hex_pretty(buffer).c_str()) +#define LOG_PACKET_SENDING(data, len) ESP_LOGVV(TAG, "Sending raw: %s", format_hex_pretty(data, len).c_str()) +#else +#define LOG_PACKET_RECEIVED(buffer) ((void) 0) +#define LOG_PACKET_SENDING(data, len) ((void) 0) +#endif + +const LogString *api_error_to_logstr(APIError err) { // not using switch to ensure compiler doesn't try to build a big table out of it if (err == APIError::OK) { - return "OK"; + return LOG_STR("OK"); } else if (err == APIError::WOULD_BLOCK) { - return "WOULD_BLOCK"; - } else if (err == APIError::BAD_HANDSHAKE_PACKET_LEN) { - return "BAD_HANDSHAKE_PACKET_LEN"; + return LOG_STR("WOULD_BLOCK"); } else if (err == APIError::BAD_INDICATOR) { - return "BAD_INDICATOR"; + return LOG_STR("BAD_INDICATOR"); } else if (err == APIError::BAD_DATA_PACKET) { - return "BAD_DATA_PACKET"; + return LOG_STR("BAD_DATA_PACKET"); } else if (err == APIError::TCP_NODELAY_FAILED) { - return "TCP_NODELAY_FAILED"; + return LOG_STR("TCP_NODELAY_FAILED"); } else if (err == APIError::TCP_NONBLOCKING_FAILED) { - return "TCP_NONBLOCKING_FAILED"; + return LOG_STR("TCP_NONBLOCKING_FAILED"); } else if (err == APIError::CLOSE_FAILED) { - return "CLOSE_FAILED"; + return LOG_STR("CLOSE_FAILED"); } else if (err == APIError::SHUTDOWN_FAILED) { - return "SHUTDOWN_FAILED"; + return LOG_STR("SHUTDOWN_FAILED"); } else if (err == APIError::BAD_STATE) { - return "BAD_STATE"; + return LOG_STR("BAD_STATE"); } else if (err == APIError::BAD_ARG) { - return "BAD_ARG"; + return LOG_STR("BAD_ARG"); } else if (err == APIError::SOCKET_READ_FAILED) { - return "SOCKET_READ_FAILED"; + return LOG_STR("SOCKET_READ_FAILED"); } else if (err == APIError::SOCKET_WRITE_FAILED) { - return "SOCKET_WRITE_FAILED"; - } else if (err == APIError::HANDSHAKESTATE_READ_FAILED) { - return "HANDSHAKESTATE_READ_FAILED"; - } else if (err == APIError::HANDSHAKESTATE_WRITE_FAILED) { - return "HANDSHAKESTATE_WRITE_FAILED"; - } else if (err == APIError::HANDSHAKESTATE_BAD_STATE) { - return "HANDSHAKESTATE_BAD_STATE"; - } else if (err == APIError::CIPHERSTATE_DECRYPT_FAILED) { - return "CIPHERSTATE_DECRYPT_FAILED"; - } else if (err == APIError::CIPHERSTATE_ENCRYPT_FAILED) { - return "CIPHERSTATE_ENCRYPT_FAILED"; + return LOG_STR("SOCKET_WRITE_FAILED"); } else if (err == APIError::OUT_OF_MEMORY) { - return "OUT_OF_MEMORY"; - } else if (err == APIError::HANDSHAKESTATE_SETUP_FAILED) { - return "HANDSHAKESTATE_SETUP_FAILED"; - } else if (err == APIError::HANDSHAKESTATE_SPLIT_FAILED) { - return "HANDSHAKESTATE_SPLIT_FAILED"; - } else if (err == APIError::BAD_HANDSHAKE_ERROR_BYTE) { - return "BAD_HANDSHAKE_ERROR_BYTE"; + return LOG_STR("OUT_OF_MEMORY"); } else if (err == APIError::CONNECTION_CLOSED) { - return "CONNECTION_CLOSED"; + return LOG_STR("CONNECTION_CLOSED"); } - return "UNKNOWN"; +#ifdef USE_API_NOISE + else if (err == APIError::BAD_HANDSHAKE_PACKET_LEN) { + return LOG_STR("BAD_HANDSHAKE_PACKET_LEN"); + } else if (err == APIError::HANDSHAKESTATE_READ_FAILED) { + return LOG_STR("HANDSHAKESTATE_READ_FAILED"); + } else if (err == APIError::HANDSHAKESTATE_WRITE_FAILED) { + return LOG_STR("HANDSHAKESTATE_WRITE_FAILED"); + } else if (err == APIError::HANDSHAKESTATE_BAD_STATE) { + return LOG_STR("HANDSHAKESTATE_BAD_STATE"); + } else if (err == APIError::CIPHERSTATE_DECRYPT_FAILED) { + return LOG_STR("CIPHERSTATE_DECRYPT_FAILED"); + } else if (err == APIError::CIPHERSTATE_ENCRYPT_FAILED) { + return LOG_STR("CIPHERSTATE_ENCRYPT_FAILED"); + } else if (err == APIError::HANDSHAKESTATE_SETUP_FAILED) { + return LOG_STR("HANDSHAKESTATE_SETUP_FAILED"); + } else if (err == APIError::HANDSHAKESTATE_SPLIT_FAILED) { + return LOG_STR("HANDSHAKESTATE_SPLIT_FAILED"); + } else if (err == APIError::BAD_HANDSHAKE_ERROR_BYTE) { + return LOG_STR("BAD_HANDSHAKE_ERROR_BYTE"); + } +#endif + return LOG_STR("UNKNOWN"); } // Default implementation for loop - handles sending buffered data APIError APIFrameHelper::loop() { - if (!this->tx_buf_.empty()) { + if (this->tx_buf_count_ > 0) { APIError err = try_send_tx_buf_(); if (err != APIError::OK && err != APIError::WOULD_BLOCK) { return err; @@ -76,36 +90,72 @@ APIError APIFrameHelper::loop() { return APIError::OK; // Convert WOULD_BLOCK to OK to avoid connection termination } -// Helper method to buffer data from IOVs -void APIFrameHelper::buffer_data_from_iov_(const struct iovec *iov, int iovcnt, uint16_t total_write_len) { - SendBuffer buffer; - buffer.data.reserve(total_write_len); - for (int i = 0; i < iovcnt; i++) { - const uint8_t *data = reinterpret_cast(iov[i].iov_base); - buffer.data.insert(buffer.data.end(), data, data + iov[i].iov_len); +// Common socket write error handling +APIError APIFrameHelper::handle_socket_write_error_() { + if (errno == EWOULDBLOCK || errno == EAGAIN) { + return APIError::WOULD_BLOCK; } - this->tx_buf_.push_back(std::move(buffer)); + HELPER_LOG("Socket write failed with errno %d", errno); + this->state_ = State::FAILED; + return APIError::SOCKET_WRITE_FAILED; +} + +// Helper method to buffer data from IOVs +void APIFrameHelper::buffer_data_from_iov_(const struct iovec *iov, int iovcnt, uint16_t total_write_len, + uint16_t offset) { + // Check if queue is full + if (this->tx_buf_count_ >= API_MAX_SEND_QUEUE) { + HELPER_LOG("Send queue full (%u buffers), dropping connection", this->tx_buf_count_); + this->state_ = State::FAILED; + return; + } + + uint16_t buffer_size = total_write_len - offset; + auto &buffer = this->tx_buf_[this->tx_buf_tail_]; + buffer = std::make_unique(SendBuffer{ + .data = std::make_unique(buffer_size), + .size = buffer_size, + .offset = 0, + }); + + uint16_t to_skip = offset; + uint16_t write_pos = 0; + + for (int i = 0; i < iovcnt; i++) { + if (to_skip >= iov[i].iov_len) { + // Skip this entire segment + to_skip -= static_cast(iov[i].iov_len); + } else { + // Include this segment (partially or fully) + const uint8_t *src = reinterpret_cast(iov[i].iov_base) + to_skip; + uint16_t len = static_cast(iov[i].iov_len) - to_skip; + std::memcpy(buffer->data.get() + write_pos, src, len); + write_pos += len; + to_skip = 0; + } + } + + // Update circular buffer tracking + this->tx_buf_tail_ = (this->tx_buf_tail_ + 1) % API_MAX_SEND_QUEUE; + this->tx_buf_count_++; } // This method writes data to socket or buffers it -APIError APIFrameHelper::write_raw_(const struct iovec *iov, int iovcnt) { +APIError APIFrameHelper::write_raw_(const struct iovec *iov, int iovcnt, uint16_t total_write_len) { // Returns APIError::OK if successful (or would block, but data has been buffered) // Returns APIError::SOCKET_WRITE_FAILED if socket write failed, and sets state to FAILED if (iovcnt == 0) return APIError::OK; // Nothing to do, success - uint16_t total_write_len = 0; - for (int i = 0; i < iovcnt; i++) { #ifdef HELPER_LOG_PACKETS - ESP_LOGVV(TAG, "Sending raw: %s", - format_hex_pretty(reinterpret_cast(iov[i].iov_base), iov[i].iov_len).c_str()); -#endif - total_write_len += static_cast(iov[i].iov_len); + for (int i = 0; i < iovcnt; i++) { + LOG_PACKET_SENDING(reinterpret_cast(iov[i].iov_base), iov[i].iov_len); } +#endif // Try to send any existing buffered data first if there is any - if (!this->tx_buf_.empty()) { + if (this->tx_buf_count_ > 0) { APIError send_result = try_send_tx_buf_(); // If real error occurred (not just WOULD_BLOCK), return it if (send_result != APIError::OK && send_result != APIError::WOULD_BLOCK) { @@ -114,86 +164,59 @@ APIError APIFrameHelper::write_raw_(const struct iovec *iov, int iovcnt) { // If there is still data in the buffer, we can't send, buffer // the new data and return - if (!this->tx_buf_.empty()) { - this->buffer_data_from_iov_(iov, iovcnt, total_write_len); + if (this->tx_buf_count_ > 0) { + this->buffer_data_from_iov_(iov, iovcnt, total_write_len, 0); return APIError::OK; // Success, data buffered } } // Try to send directly if no buffered data - ssize_t sent = this->socket_->writev(iov, iovcnt); + // Optimize for single iovec case (common for plaintext API) + ssize_t sent = + (iovcnt == 1) ? this->socket_->write(iov[0].iov_base, iov[0].iov_len) : this->socket_->writev(iov, iovcnt); if (sent == -1) { - if (errno == EWOULDBLOCK || errno == EAGAIN) { + APIError err = this->handle_socket_write_error_(); + if (err == APIError::WOULD_BLOCK) { // Socket would block, buffer the data - this->buffer_data_from_iov_(iov, iovcnt, total_write_len); + this->buffer_data_from_iov_(iov, iovcnt, total_write_len, 0); return APIError::OK; // Success, data buffered } - // Socket error - ESP_LOGVV(TAG, "%s: Socket write failed with errno %d", this->info_.c_str(), errno); - this->state_ = State::FAILED; - return APIError::SOCKET_WRITE_FAILED; // Socket write failed + return err; // Socket write failed } else if (static_cast(sent) < total_write_len) { // Partially sent, buffer the remaining data - SendBuffer buffer; - uint16_t to_consume = static_cast(sent); - uint16_t remaining = total_write_len - static_cast(sent); - - buffer.data.reserve(remaining); - - for (int i = 0; i < iovcnt; i++) { - if (to_consume >= iov[i].iov_len) { - // This segment was fully sent - to_consume -= static_cast(iov[i].iov_len); - } else { - // This segment was partially sent or not sent at all - const uint8_t *data = reinterpret_cast(iov[i].iov_base) + to_consume; - uint16_t len = static_cast(iov[i].iov_len) - to_consume; - buffer.data.insert(buffer.data.end(), data, data + len); - to_consume = 0; - } - } - - this->tx_buf_.push_back(std::move(buffer)); + this->buffer_data_from_iov_(iov, iovcnt, total_write_len, static_cast(sent)); } return APIError::OK; // Success, all data sent or buffered } // Common implementation for trying to send buffered data -// IMPORTANT: Caller MUST ensure tx_buf_ is not empty before calling this method +// IMPORTANT: Caller MUST ensure tx_buf_count_ > 0 before calling this method APIError APIFrameHelper::try_send_tx_buf_() { // Try to send from tx_buf - we assume it's not empty as it's the caller's responsibility to check - bool tx_buf_empty = false; - while (!tx_buf_empty) { + while (this->tx_buf_count_ > 0) { // Get the first buffer in the queue - SendBuffer &front_buffer = this->tx_buf_.front(); + SendBuffer *front_buffer = this->tx_buf_[this->tx_buf_head_].get(); // Try to send the remaining data in this buffer - ssize_t sent = this->socket_->write(front_buffer.current_data(), front_buffer.remaining()); + ssize_t sent = this->socket_->write(front_buffer->current_data(), front_buffer->remaining()); if (sent == -1) { - if (errno != EWOULDBLOCK && errno != EAGAIN) { - // Real socket error (not just would block) - ESP_LOGVV(TAG, "%s: Socket write failed with errno %d", this->info_.c_str(), errno); - this->state_ = State::FAILED; - return APIError::SOCKET_WRITE_FAILED; // Socket write failed - } - // Socket would block, we'll try again later - return APIError::WOULD_BLOCK; + return this->handle_socket_write_error_(); } else if (sent == 0) { // Nothing sent but not an error return APIError::WOULD_BLOCK; - } else if (static_cast(sent) < front_buffer.remaining()) { + } else if (static_cast(sent) < front_buffer->remaining()) { // Partially sent, update offset // Cast to ensure no overflow issues with uint16_t - front_buffer.offset += static_cast(sent); + front_buffer->offset += static_cast(sent); return APIError::WOULD_BLOCK; // Stop processing more buffers if we couldn't send a complete buffer } else { // Buffer completely sent, remove it from the queue - this->tx_buf_.pop_front(); - // Update empty status for the loop condition - tx_buf_empty = this->tx_buf_.empty(); + this->tx_buf_[this->tx_buf_head_].reset(); + this->tx_buf_head_ = (this->tx_buf_head_ + 1) % API_MAX_SEND_QUEUE; + this->tx_buf_count_--; // Continue loop to try sending the next buffer } } @@ -203,13 +226,13 @@ APIError APIFrameHelper::try_send_tx_buf_() { APIError APIFrameHelper::init_common_() { if (state_ != State::INITIALIZE || this->socket_ == nullptr) { - ESP_LOGVV(TAG, "%s: Bad state for init %d", this->info_.c_str(), (int) state_); + HELPER_LOG("Bad state for init %d", (int) state_); return APIError::BAD_STATE; } int err = this->socket_->setblocking(false); if (err != 0) { state_ = State::FAILED; - ESP_LOGVV(TAG, "%s: Setting nonblocking failed with errno %d", this->info_.c_str(), errno); + HELPER_LOG("Setting nonblocking failed with errno %d", errno); return APIError::TCP_NONBLOCKING_FAILED; } @@ -217,14 +240,12 @@ APIError APIFrameHelper::init_common_() { err = this->socket_->setsockopt(IPPROTO_TCP, TCP_NODELAY, &enable, sizeof(int)); if (err != 0) { state_ = State::FAILED; - ESP_LOGVV(TAG, "%s: Setting nodelay failed with errno %d", this->info_.c_str(), errno); + HELPER_LOG("Setting nodelay failed with errno %d", errno); return APIError::TCP_NODELAY_FAILED; } return APIError::OK; } -#define HELPER_LOG(msg, ...) ESP_LOGVV(TAG, "%s: " msg, this->info_.c_str(), ##__VA_ARGS__) - APIError APIFrameHelper::handle_socket_read_result_(ssize_t received) { if (received == -1) { if (errno == EWOULDBLOCK || errno == EAGAIN) { @@ -240,840 +261,6 @@ APIError APIFrameHelper::handle_socket_read_result_(ssize_t received) { } return APIError::OK; } -// uncomment to log raw packets -//#define HELPER_LOG_PACKETS -#ifdef USE_API_NOISE -static const char *const PROLOGUE_INIT = "NoiseAPIInit"; - -/// Convert a noise error code to a readable error -std::string noise_err_to_str(int err) { - if (err == NOISE_ERROR_NO_MEMORY) - return "NO_MEMORY"; - if (err == NOISE_ERROR_UNKNOWN_ID) - return "UNKNOWN_ID"; - if (err == NOISE_ERROR_UNKNOWN_NAME) - return "UNKNOWN_NAME"; - if (err == NOISE_ERROR_MAC_FAILURE) - return "MAC_FAILURE"; - if (err == NOISE_ERROR_NOT_APPLICABLE) - return "NOT_APPLICABLE"; - if (err == NOISE_ERROR_SYSTEM) - return "SYSTEM"; - if (err == NOISE_ERROR_REMOTE_KEY_REQUIRED) - return "REMOTE_KEY_REQUIRED"; - if (err == NOISE_ERROR_LOCAL_KEY_REQUIRED) - return "LOCAL_KEY_REQUIRED"; - if (err == NOISE_ERROR_PSK_REQUIRED) - return "PSK_REQUIRED"; - if (err == NOISE_ERROR_INVALID_LENGTH) - return "INVALID_LENGTH"; - if (err == NOISE_ERROR_INVALID_PARAM) - return "INVALID_PARAM"; - if (err == NOISE_ERROR_INVALID_STATE) - return "INVALID_STATE"; - if (err == NOISE_ERROR_INVALID_NONCE) - return "INVALID_NONCE"; - if (err == NOISE_ERROR_INVALID_PRIVATE_KEY) - return "INVALID_PRIVATE_KEY"; - if (err == NOISE_ERROR_INVALID_PUBLIC_KEY) - return "INVALID_PUBLIC_KEY"; - if (err == NOISE_ERROR_INVALID_FORMAT) - return "INVALID_FORMAT"; - if (err == NOISE_ERROR_INVALID_SIGNATURE) - return "INVALID_SIGNATURE"; - return to_string(err); -} - -/// Initialize the frame helper, returns OK if successful. -APIError APINoiseFrameHelper::init() { - APIError err = init_common_(); - if (err != APIError::OK) { - return err; - } - - // init prologue - prologue_.insert(prologue_.end(), PROLOGUE_INIT, PROLOGUE_INIT + strlen(PROLOGUE_INIT)); - - state_ = State::CLIENT_HELLO; - return APIError::OK; -} -/// Run through handshake messages (if in that phase) -APIError APINoiseFrameHelper::loop() { - // During handshake phase, process as many actions as possible until we can't progress - // socket_->ready() stays true until next main loop, but state_action() will return - // WOULD_BLOCK when no more data is available to read - while (state_ != State::DATA && this->socket_->ready()) { - APIError err = state_action_(); - if (err != APIError::OK && err != APIError::WOULD_BLOCK) { - return err; - } - if (err == APIError::WOULD_BLOCK) { - break; - } - } - - // Use base class implementation for buffer sending - return APIFrameHelper::loop(); -} - -/** Read a packet into the rx_buf_. If successful, stores frame data in the frame parameter - * - * @param frame: The struct to hold the frame information in. - * msg_start: points to the start of the payload - this pointer is only valid until the next - * try_receive_raw_ call - * - * @return 0 if a full packet is in rx_buf_ - * @return -1 if error, check errno. - * - * errno EWOULDBLOCK: Packet could not be read without blocking. Try again later. - * errno ENOMEM: Not enough memory for reading packet. - * errno API_ERROR_BAD_INDICATOR: Bad indicator byte at start of frame. - * errno API_ERROR_HANDSHAKE_PACKET_LEN: Packet too big for this phase. - */ -APIError APINoiseFrameHelper::try_read_frame_(ParsedFrame *frame) { - if (frame == nullptr) { - HELPER_LOG("Bad argument for try_read_frame_"); - return APIError::BAD_ARG; - } - - // read header - if (rx_header_buf_len_ < 3) { - // no header information yet - uint8_t to_read = 3 - rx_header_buf_len_; - ssize_t received = this->socket_->read(&rx_header_buf_[rx_header_buf_len_], to_read); - APIError err = handle_socket_read_result_(received); - if (err != APIError::OK) { - return err; - } - rx_header_buf_len_ += static_cast(received); - if (static_cast(received) != to_read) { - // not a full read - return APIError::WOULD_BLOCK; - } - - if (rx_header_buf_[0] != 0x01) { - state_ = State::FAILED; - HELPER_LOG("Bad indicator byte %u", rx_header_buf_[0]); - return APIError::BAD_INDICATOR; - } - // header reading done - } - - // read body - uint16_t msg_size = (((uint16_t) rx_header_buf_[1]) << 8) | rx_header_buf_[2]; - - if (state_ != State::DATA && msg_size > 128) { - // for handshake message only permit up to 128 bytes - state_ = State::FAILED; - HELPER_LOG("Bad packet len for handshake: %d", msg_size); - return APIError::BAD_HANDSHAKE_PACKET_LEN; - } - - // reserve space for body - if (rx_buf_.size() != msg_size) { - rx_buf_.resize(msg_size); - } - - if (rx_buf_len_ < msg_size) { - // more data to read - uint16_t to_read = msg_size - rx_buf_len_; - ssize_t received = this->socket_->read(&rx_buf_[rx_buf_len_], to_read); - APIError err = handle_socket_read_result_(received); - if (err != APIError::OK) { - return err; - } - rx_buf_len_ += static_cast(received); - if (static_cast(received) != to_read) { - // not all read - return APIError::WOULD_BLOCK; - } - } - - // uncomment for even more debugging -#ifdef HELPER_LOG_PACKETS - ESP_LOGVV(TAG, "Received frame: %s", format_hex_pretty(rx_buf_).c_str()); -#endif - frame->msg = std::move(rx_buf_); - // consume msg - rx_buf_ = {}; - rx_buf_len_ = 0; - rx_header_buf_len_ = 0; - return APIError::OK; -} - -/** To be called from read/write methods. - * - * This method runs through the internal handshake methods, if in that state. - * - * If the handshake is still active when this method returns and a read/write can't take place at - * the moment, returns WOULD_BLOCK. - * If an error occurred, returns that error. Only returns OK if the transport is ready for data - * traffic. - */ -APIError APINoiseFrameHelper::state_action_() { - int err; - APIError aerr; - if (state_ == State::INITIALIZE) { - HELPER_LOG("Bad state for method: %d", (int) state_); - return APIError::BAD_STATE; - } - if (state_ == State::CLIENT_HELLO) { - // waiting for client hello - ParsedFrame frame; - aerr = try_read_frame_(&frame); - if (aerr == APIError::BAD_INDICATOR) { - send_explicit_handshake_reject_("Bad indicator byte"); - return aerr; - } - if (aerr == APIError::BAD_HANDSHAKE_PACKET_LEN) { - send_explicit_handshake_reject_("Bad handshake packet len"); - return aerr; - } - if (aerr != APIError::OK) - return aerr; - // ignore contents, may be used in future for flags - // Reserve space for: existing prologue + 2 size bytes + frame data - prologue_.reserve(prologue_.size() + 2 + frame.msg.size()); - prologue_.push_back((uint8_t) (frame.msg.size() >> 8)); - prologue_.push_back((uint8_t) frame.msg.size()); - prologue_.insert(prologue_.end(), frame.msg.begin(), frame.msg.end()); - - state_ = State::SERVER_HELLO; - } - if (state_ == State::SERVER_HELLO) { - // send server hello - const std::string &name = App.get_name(); - const std::string &mac = get_mac_address(); - - std::vector msg; - // Reserve space for: 1 byte proto + name + null + mac + null - msg.reserve(1 + name.size() + 1 + mac.size() + 1); - - // chosen proto - msg.push_back(0x01); - - // node name, terminated by null byte - const uint8_t *name_ptr = reinterpret_cast(name.c_str()); - msg.insert(msg.end(), name_ptr, name_ptr + name.size() + 1); - // node mac, terminated by null byte - const uint8_t *mac_ptr = reinterpret_cast(mac.c_str()); - msg.insert(msg.end(), mac_ptr, mac_ptr + mac.size() + 1); - - aerr = write_frame_(msg.data(), msg.size()); - if (aerr != APIError::OK) - return aerr; - - // start handshake - aerr = init_handshake_(); - if (aerr != APIError::OK) - return aerr; - - state_ = State::HANDSHAKE; - } - if (state_ == State::HANDSHAKE) { - int action = noise_handshakestate_get_action(handshake_); - if (action == NOISE_ACTION_READ_MESSAGE) { - // waiting for handshake msg - ParsedFrame frame; - aerr = try_read_frame_(&frame); - if (aerr == APIError::BAD_INDICATOR) { - send_explicit_handshake_reject_("Bad indicator byte"); - return aerr; - } - if (aerr == APIError::BAD_HANDSHAKE_PACKET_LEN) { - send_explicit_handshake_reject_("Bad handshake packet len"); - return aerr; - } - if (aerr != APIError::OK) - return aerr; - - if (frame.msg.empty()) { - send_explicit_handshake_reject_("Empty handshake message"); - return APIError::BAD_HANDSHAKE_ERROR_BYTE; - } else if (frame.msg[0] != 0x00) { - HELPER_LOG("Bad handshake error byte: %u", frame.msg[0]); - send_explicit_handshake_reject_("Bad handshake error byte"); - return APIError::BAD_HANDSHAKE_ERROR_BYTE; - } - - NoiseBuffer mbuf; - noise_buffer_init(mbuf); - noise_buffer_set_input(mbuf, frame.msg.data() + 1, frame.msg.size() - 1); - err = noise_handshakestate_read_message(handshake_, &mbuf, nullptr); - if (err != 0) { - state_ = State::FAILED; - HELPER_LOG("noise_handshakestate_read_message failed: %s", noise_err_to_str(err).c_str()); - if (err == NOISE_ERROR_MAC_FAILURE) { - send_explicit_handshake_reject_("Handshake MAC failure"); - } else { - send_explicit_handshake_reject_("Handshake error"); - } - return APIError::HANDSHAKESTATE_READ_FAILED; - } - - aerr = check_handshake_finished_(); - if (aerr != APIError::OK) - return aerr; - } else if (action == NOISE_ACTION_WRITE_MESSAGE) { - uint8_t buffer[65]; - NoiseBuffer mbuf; - noise_buffer_init(mbuf); - noise_buffer_set_output(mbuf, buffer + 1, sizeof(buffer) - 1); - - err = noise_handshakestate_write_message(handshake_, &mbuf, nullptr); - if (err != 0) { - state_ = State::FAILED; - HELPER_LOG("noise_handshakestate_write_message failed: %s", noise_err_to_str(err).c_str()); - return APIError::HANDSHAKESTATE_WRITE_FAILED; - } - buffer[0] = 0x00; // success - - aerr = write_frame_(buffer, mbuf.size + 1); - if (aerr != APIError::OK) - return aerr; - aerr = check_handshake_finished_(); - if (aerr != APIError::OK) - return aerr; - } else { - // bad state for action - state_ = State::FAILED; - HELPER_LOG("Bad action for handshake: %d", action); - return APIError::HANDSHAKESTATE_BAD_STATE; - } - } - if (state_ == State::CLOSED || state_ == State::FAILED) { - return APIError::BAD_STATE; - } - return APIError::OK; -} -void APINoiseFrameHelper::send_explicit_handshake_reject_(const std::string &reason) { - std::vector data; - data.resize(reason.length() + 1); - data[0] = 0x01; // failure - - // Copy error message in bulk - if (!reason.empty()) { - std::memcpy(data.data() + 1, reason.c_str(), reason.length()); - } - - // temporarily remove failed state - auto orig_state = state_; - state_ = State::EXPLICIT_REJECT; - write_frame_(data.data(), data.size()); - state_ = orig_state; -} -APIError APINoiseFrameHelper::read_packet(ReadPacketBuffer *buffer) { - int err; - APIError aerr; - aerr = state_action_(); - if (aerr != APIError::OK) { - return aerr; - } - - if (state_ != State::DATA) { - return APIError::WOULD_BLOCK; - } - - ParsedFrame frame; - aerr = try_read_frame_(&frame); - if (aerr != APIError::OK) - return aerr; - - NoiseBuffer mbuf; - noise_buffer_init(mbuf); - noise_buffer_set_inout(mbuf, frame.msg.data(), frame.msg.size(), frame.msg.size()); - err = noise_cipherstate_decrypt(recv_cipher_, &mbuf); - if (err != 0) { - state_ = State::FAILED; - HELPER_LOG("noise_cipherstate_decrypt failed: %s", noise_err_to_str(err).c_str()); - return APIError::CIPHERSTATE_DECRYPT_FAILED; - } - - uint16_t msg_size = mbuf.size; - uint8_t *msg_data = frame.msg.data(); - if (msg_size < 4) { - state_ = State::FAILED; - HELPER_LOG("Bad data packet: size %d too short", msg_size); - return APIError::BAD_DATA_PACKET; - } - - uint16_t type = (((uint16_t) msg_data[0]) << 8) | msg_data[1]; - uint16_t data_len = (((uint16_t) msg_data[2]) << 8) | msg_data[3]; - if (data_len > msg_size - 4) { - state_ = State::FAILED; - HELPER_LOG("Bad data packet: data_len %u greater than msg_size %u", data_len, msg_size); - return APIError::BAD_DATA_PACKET; - } - - buffer->container = std::move(frame.msg); - buffer->data_offset = 4; - buffer->data_len = data_len; - buffer->type = type; - return APIError::OK; -} -APIError APINoiseFrameHelper::write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) { - // Resize to include MAC space (required for Noise encryption) - buffer.get_buffer()->resize(buffer.get_buffer()->size() + frame_footer_size_); - PacketInfo packet{type, 0, - static_cast(buffer.get_buffer()->size() - frame_header_padding_ - frame_footer_size_)}; - return write_protobuf_packets(buffer, std::span(&packet, 1)); -} - -APIError APINoiseFrameHelper::write_protobuf_packets(ProtoWriteBuffer buffer, std::span packets) { - APIError aerr = state_action_(); - if (aerr != APIError::OK) { - return aerr; - } - - if (state_ != State::DATA) { - return APIError::WOULD_BLOCK; - } - - if (packets.empty()) { - return APIError::OK; - } - - std::vector *raw_buffer = buffer.get_buffer(); - uint8_t *buffer_data = raw_buffer->data(); // Cache buffer pointer - - this->reusable_iovs_.clear(); - this->reusable_iovs_.reserve(packets.size()); - - // We need to encrypt each packet in place - for (const auto &packet : packets) { - // The buffer already has padding at offset - uint8_t *buf_start = buffer_data + packet.offset; - - // Write noise header - buf_start[0] = 0x01; // indicator - // buf_start[1], buf_start[2] to be set after encryption - - // Write message header (to be encrypted) - const uint8_t msg_offset = 3; - buf_start[msg_offset] = static_cast(packet.message_type >> 8); // type high byte - buf_start[msg_offset + 1] = static_cast(packet.message_type); // type low byte - buf_start[msg_offset + 2] = static_cast(packet.payload_size >> 8); // data_len high byte - buf_start[msg_offset + 3] = static_cast(packet.payload_size); // data_len low byte - // payload data is already in the buffer starting at offset + 7 - - // Make sure we have space for MAC - // The buffer should already have been sized appropriately - - // Encrypt the message in place - NoiseBuffer mbuf; - noise_buffer_init(mbuf); - noise_buffer_set_inout(mbuf, buf_start + msg_offset, 4 + packet.payload_size, - 4 + packet.payload_size + frame_footer_size_); - - int err = noise_cipherstate_encrypt(send_cipher_, &mbuf); - if (err != 0) { - state_ = State::FAILED; - HELPER_LOG("noise_cipherstate_encrypt failed: %s", noise_err_to_str(err).c_str()); - return APIError::CIPHERSTATE_ENCRYPT_FAILED; - } - - // Fill in the encrypted size - buf_start[1] = static_cast(mbuf.size >> 8); - buf_start[2] = static_cast(mbuf.size); - - // Add iovec for this encrypted packet - this->reusable_iovs_.push_back( - {buf_start, static_cast(3 + mbuf.size)}); // indicator + size + encrypted data - } - - // Send all encrypted packets in one writev call - return this->write_raw_(this->reusable_iovs_.data(), this->reusable_iovs_.size()); -} - -APIError APINoiseFrameHelper::write_frame_(const uint8_t *data, uint16_t len) { - uint8_t header[3]; - header[0] = 0x01; // indicator - header[1] = (uint8_t) (len >> 8); - header[2] = (uint8_t) len; - - struct iovec iov[2]; - iov[0].iov_base = header; - iov[0].iov_len = 3; - if (len == 0) { - return this->write_raw_(iov, 1); - } - iov[1].iov_base = const_cast(data); - iov[1].iov_len = len; - - return this->write_raw_(iov, 2); -} - -/** Initiate the data structures for the handshake. - * - * @return 0 on success, -1 on error (check errno) - */ -APIError APINoiseFrameHelper::init_handshake_() { - int err; - memset(&nid_, 0, sizeof(nid_)); - // const char *proto = "Noise_NNpsk0_25519_ChaChaPoly_SHA256"; - // err = noise_protocol_name_to_id(&nid_, proto, strlen(proto)); - nid_.pattern_id = NOISE_PATTERN_NN; - nid_.cipher_id = NOISE_CIPHER_CHACHAPOLY; - nid_.dh_id = NOISE_DH_CURVE25519; - nid_.prefix_id = NOISE_PREFIX_STANDARD; - nid_.hybrid_id = NOISE_DH_NONE; - nid_.hash_id = NOISE_HASH_SHA256; - nid_.modifier_ids[0] = NOISE_MODIFIER_PSK0; - - err = noise_handshakestate_new_by_id(&handshake_, &nid_, NOISE_ROLE_RESPONDER); - if (err != 0) { - state_ = State::FAILED; - HELPER_LOG("noise_handshakestate_new_by_id failed: %s", noise_err_to_str(err).c_str()); - return APIError::HANDSHAKESTATE_SETUP_FAILED; - } - - const auto &psk = ctx_->get_psk(); - err = noise_handshakestate_set_pre_shared_key(handshake_, psk.data(), psk.size()); - if (err != 0) { - state_ = State::FAILED; - HELPER_LOG("noise_handshakestate_set_pre_shared_key failed: %s", noise_err_to_str(err).c_str()); - return APIError::HANDSHAKESTATE_SETUP_FAILED; - } - - err = noise_handshakestate_set_prologue(handshake_, prologue_.data(), prologue_.size()); - if (err != 0) { - state_ = State::FAILED; - HELPER_LOG("noise_handshakestate_set_prologue failed: %s", noise_err_to_str(err).c_str()); - return APIError::HANDSHAKESTATE_SETUP_FAILED; - } - // set_prologue copies it into handshakestate, so we can get rid of it now - prologue_ = {}; - - err = noise_handshakestate_start(handshake_); - if (err != 0) { - state_ = State::FAILED; - HELPER_LOG("noise_handshakestate_start failed: %s", noise_err_to_str(err).c_str()); - return APIError::HANDSHAKESTATE_SETUP_FAILED; - } - return APIError::OK; -} - -APIError APINoiseFrameHelper::check_handshake_finished_() { - assert(state_ == State::HANDSHAKE); - - int action = noise_handshakestate_get_action(handshake_); - if (action == NOISE_ACTION_READ_MESSAGE || action == NOISE_ACTION_WRITE_MESSAGE) - return APIError::OK; - if (action != NOISE_ACTION_SPLIT) { - state_ = State::FAILED; - HELPER_LOG("Bad action for handshake: %d", action); - return APIError::HANDSHAKESTATE_BAD_STATE; - } - int err = noise_handshakestate_split(handshake_, &send_cipher_, &recv_cipher_); - if (err != 0) { - state_ = State::FAILED; - HELPER_LOG("noise_handshakestate_split failed: %s", noise_err_to_str(err).c_str()); - return APIError::HANDSHAKESTATE_SPLIT_FAILED; - } - - frame_footer_size_ = noise_cipherstate_get_mac_length(send_cipher_); - - HELPER_LOG("Handshake complete!"); - noise_handshakestate_free(handshake_); - handshake_ = nullptr; - state_ = State::DATA; - return APIError::OK; -} - -APINoiseFrameHelper::~APINoiseFrameHelper() { - if (handshake_ != nullptr) { - noise_handshakestate_free(handshake_); - handshake_ = nullptr; - } - if (send_cipher_ != nullptr) { - noise_cipherstate_free(send_cipher_); - send_cipher_ = nullptr; - } - if (recv_cipher_ != nullptr) { - noise_cipherstate_free(recv_cipher_); - recv_cipher_ = nullptr; - } -} - -extern "C" { -// declare how noise generates random bytes (here with a good HWRNG based on the RF system) -void noise_rand_bytes(void *output, size_t len) { - if (!esphome::random_bytes(reinterpret_cast(output), len)) { - ESP_LOGE(TAG, "Acquiring random bytes failed; rebooting"); - arch_restart(); - } -} -} - -#endif // USE_API_NOISE - -#ifdef USE_API_PLAINTEXT - -/// Initialize the frame helper, returns OK if successful. -APIError APIPlaintextFrameHelper::init() { - APIError err = init_common_(); - if (err != APIError::OK) { - return err; - } - - state_ = State::DATA; - return APIError::OK; -} -APIError APIPlaintextFrameHelper::loop() { - if (state_ != State::DATA) { - return APIError::BAD_STATE; - } - // Use base class implementation for buffer sending - return APIFrameHelper::loop(); -} - -/** Read a packet into the rx_buf_. If successful, stores frame data in the frame parameter - * - * @param frame: The struct to hold the frame information in. - * msg: store the parsed frame in that struct - * - * @return See APIError - * - * error API_ERROR_BAD_INDICATOR: Bad indicator byte at start of frame. - */ -APIError APIPlaintextFrameHelper::try_read_frame_(ParsedFrame *frame) { - if (frame == nullptr) { - HELPER_LOG("Bad argument for try_read_frame_"); - return APIError::BAD_ARG; - } - - // read header - while (!rx_header_parsed_) { - // Now that we know when the socket is ready, we can read up to 3 bytes - // into the rx_header_buf_ before we have to switch back to reading - // one byte at a time to ensure we don't read past the message and - // into the next one. - - // Read directly into rx_header_buf_ at the current position - // Try to get to at least 3 bytes total (indicator + 2 varint bytes), then read one byte at a time - ssize_t received = - this->socket_->read(&rx_header_buf_[rx_header_buf_pos_], rx_header_buf_pos_ < 3 ? 3 - rx_header_buf_pos_ : 1); - APIError err = handle_socket_read_result_(received); - if (err != APIError::OK) { - return err; - } - - // If this was the first read, validate the indicator byte - if (rx_header_buf_pos_ == 0 && received > 0) { - if (rx_header_buf_[0] != 0x00) { - state_ = State::FAILED; - HELPER_LOG("Bad indicator byte %u", rx_header_buf_[0]); - return APIError::BAD_INDICATOR; - } - } - - rx_header_buf_pos_ += received; - - // Check for buffer overflow - if (rx_header_buf_pos_ >= sizeof(rx_header_buf_)) { - state_ = State::FAILED; - HELPER_LOG("Header buffer overflow"); - return APIError::BAD_DATA_PACKET; - } - - // Need at least 3 bytes total (indicator + 2 varint bytes) before trying to parse - if (rx_header_buf_pos_ < 3) { - continue; - } - - // At this point, we have at least 3 bytes total: - // - Validated indicator byte (0x00) stored at position 0 - // - At least 2 bytes in the buffer for the varints - // Buffer layout: - // [0]: indicator byte (0x00) - // [1-3]: Message size varint (variable length) - // - 2 bytes would only allow up to 16383, which is less than noise's UINT16_MAX (65535) - // - 3 bytes allows up to 2097151, ensuring we support at least as much as noise - // [2-5]: Message type varint (variable length) - // We now attempt to parse both varints. If either is incomplete, - // we'll continue reading more bytes. - - // Skip indicator byte at position 0 - uint8_t varint_pos = 1; - uint32_t consumed = 0; - - auto msg_size_varint = ProtoVarInt::parse(&rx_header_buf_[varint_pos], rx_header_buf_pos_ - varint_pos, &consumed); - if (!msg_size_varint.has_value()) { - // not enough data there yet - continue; - } - - if (msg_size_varint->as_uint32() > std::numeric_limits::max()) { - state_ = State::FAILED; - HELPER_LOG("Bad packet: message size %" PRIu32 " exceeds maximum %u", msg_size_varint->as_uint32(), - std::numeric_limits::max()); - return APIError::BAD_DATA_PACKET; - } - rx_header_parsed_len_ = msg_size_varint->as_uint16(); - - // Move to next varint position - varint_pos += consumed; - - auto msg_type_varint = ProtoVarInt::parse(&rx_header_buf_[varint_pos], rx_header_buf_pos_ - varint_pos, &consumed); - if (!msg_type_varint.has_value()) { - // not enough data there yet - continue; - } - if (msg_type_varint->as_uint32() > std::numeric_limits::max()) { - state_ = State::FAILED; - HELPER_LOG("Bad packet: message type %" PRIu32 " exceeds maximum %u", msg_type_varint->as_uint32(), - std::numeric_limits::max()); - return APIError::BAD_DATA_PACKET; - } - rx_header_parsed_type_ = msg_type_varint->as_uint16(); - rx_header_parsed_ = true; - } - // header reading done - - // reserve space for body - if (rx_buf_.size() != rx_header_parsed_len_) { - rx_buf_.resize(rx_header_parsed_len_); - } - - if (rx_buf_len_ < rx_header_parsed_len_) { - // more data to read - uint16_t to_read = rx_header_parsed_len_ - rx_buf_len_; - ssize_t received = this->socket_->read(&rx_buf_[rx_buf_len_], to_read); - APIError err = handle_socket_read_result_(received); - if (err != APIError::OK) { - return err; - } - rx_buf_len_ += static_cast(received); - if (static_cast(received) != to_read) { - // not all read - return APIError::WOULD_BLOCK; - } - } - - // uncomment for even more debugging -#ifdef HELPER_LOG_PACKETS - ESP_LOGVV(TAG, "Received frame: %s", format_hex_pretty(rx_buf_).c_str()); -#endif - frame->msg = std::move(rx_buf_); - // consume msg - rx_buf_ = {}; - rx_buf_len_ = 0; - rx_header_buf_pos_ = 0; - rx_header_parsed_ = false; - return APIError::OK; -} -APIError APIPlaintextFrameHelper::read_packet(ReadPacketBuffer *buffer) { - APIError aerr; - - if (state_ != State::DATA) { - return APIError::WOULD_BLOCK; - } - - ParsedFrame frame; - aerr = try_read_frame_(&frame); - if (aerr != APIError::OK) { - if (aerr == APIError::BAD_INDICATOR) { - // Make sure to tell the remote that we don't - // understand the indicator byte so it knows - // we do not support it. - struct iovec iov[1]; - // The \x00 first byte is the marker for plaintext. - // - // The remote will know how to handle the indicator byte, - // but it likely won't understand the rest of the message. - // - // We must send at least 3 bytes to be read, so we add - // a message after the indicator byte to ensures its long - // enough and can aid in debugging. - const char msg[] = "\x00" - "Bad indicator byte"; - iov[0].iov_base = (void *) msg; - iov[0].iov_len = 19; - this->write_raw_(iov, 1); - } - return aerr; - } - - buffer->container = std::move(frame.msg); - buffer->data_offset = 0; - buffer->data_len = rx_header_parsed_len_; - buffer->type = rx_header_parsed_type_; - return APIError::OK; -} -APIError APIPlaintextFrameHelper::write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) { - PacketInfo packet{type, 0, static_cast(buffer.get_buffer()->size() - frame_header_padding_)}; - return write_protobuf_packets(buffer, std::span(&packet, 1)); -} - -APIError APIPlaintextFrameHelper::write_protobuf_packets(ProtoWriteBuffer buffer, std::span packets) { - if (state_ != State::DATA) { - return APIError::BAD_STATE; - } - - if (packets.empty()) { - return APIError::OK; - } - - std::vector *raw_buffer = buffer.get_buffer(); - uint8_t *buffer_data = raw_buffer->data(); // Cache buffer pointer - - this->reusable_iovs_.clear(); - this->reusable_iovs_.reserve(packets.size()); - - for (const auto &packet : packets) { - // Calculate varint sizes for header layout - uint8_t size_varint_len = api::ProtoSize::varint(static_cast(packet.payload_size)); - uint8_t type_varint_len = api::ProtoSize::varint(static_cast(packet.message_type)); - uint8_t total_header_len = 1 + size_varint_len + type_varint_len; - - // Calculate where to start writing the header - // The header starts at the latest possible position to minimize unused padding - // - // Example 1 (small values): total_header_len = 3, header_offset = 6 - 3 = 3 - // [0-2] - Unused padding - // [3] - 0x00 indicator byte - // [4] - Payload size varint (1 byte, for sizes 0-127) - // [5] - Message type varint (1 byte, for types 0-127) - // [6...] - Actual payload data - // - // Example 2 (medium values): total_header_len = 4, header_offset = 6 - 4 = 2 - // [0-1] - Unused padding - // [2] - 0x00 indicator byte - // [3-4] - Payload size varint (2 bytes, for sizes 128-16383) - // [5] - Message type varint (1 byte, for types 0-127) - // [6...] - Actual payload data - // - // Example 3 (large values): total_header_len = 6, header_offset = 6 - 6 = 0 - // [0] - 0x00 indicator byte - // [1-3] - Payload size varint (3 bytes, for sizes 16384-2097151) - // [4-5] - Message type varint (2 bytes, for types 128-32767) - // [6...] - Actual payload data - // - // The message starts at offset + frame_header_padding_ - // So we write the header starting at offset + frame_header_padding_ - total_header_len - uint8_t *buf_start = buffer_data + packet.offset; - uint32_t header_offset = frame_header_padding_ - total_header_len; - - // Write the plaintext header - buf_start[header_offset] = 0x00; // indicator - - // Encode varints directly into buffer - ProtoVarInt(packet.payload_size).encode_to_buffer_unchecked(buf_start + header_offset + 1, size_varint_len); - ProtoVarInt(packet.message_type) - .encode_to_buffer_unchecked(buf_start + header_offset + 1 + size_varint_len, type_varint_len); - - // Add iovec for this packet (header + payload) - this->reusable_iovs_.push_back( - {buf_start + header_offset, static_cast(total_header_len + packet.payload_size)}); - } - - // Send all packets in one writev call - return write_raw_(this->reusable_iovs_.data(), this->reusable_iovs_.size()); -} - -#endif // USE_API_PLAINTEXT - -} // namespace api -} // namespace esphome +} // namespace esphome::api #endif diff --git a/esphome/components/api/api_frame_helper.h b/esphome/components/api/api_frame_helper.h index 4bcc4acd61..9aaada3cf7 100644 --- a/esphome/components/api/api_frame_helper.h +++ b/esphome/components/api/api_frame_helper.h @@ -1,23 +1,36 @@ #pragma once +#include #include -#include #include +#include #include #include #include #include "esphome/core/defines.h" #ifdef USE_API -#ifdef USE_API_NOISE -#include "noise/protocol.h" -#endif - -#include "api_noise_context.h" #include "esphome/components/socket/socket.h" #include "esphome/core/application.h" +#include "esphome/core/log.h" -namespace esphome { -namespace api { +namespace esphome::api { + +// uncomment to log raw packets +//#define HELPER_LOG_PACKETS + +// Maximum message size limits to prevent OOM on constrained devices +// Handshake messages are limited to a small size for security +static constexpr uint16_t MAX_HANDSHAKE_SIZE = 128; + +// Data message limits vary by platform based on available memory +#ifdef USE_ESP8266 +static constexpr uint16_t MAX_MESSAGE_SIZE = 8192; // 8 KiB for ESP8266 +#else +static constexpr uint16_t MAX_MESSAGE_SIZE = 32768; // 32 KiB for ESP32 and other platforms +#endif + +// Forward declaration +struct ClientInfo; class ProtoWriteBuffer; @@ -40,7 +53,6 @@ struct PacketInfo { enum class APIError : uint16_t { OK = 0, WOULD_BLOCK = 1001, - BAD_HANDSHAKE_PACKET_LEN = 1002, BAD_INDICATOR = 1003, BAD_DATA_PACKET = 1004, TCP_NODELAY_FAILED = 1005, @@ -51,31 +63,35 @@ enum class APIError : uint16_t { BAD_ARG = 1010, SOCKET_READ_FAILED = 1011, SOCKET_WRITE_FAILED = 1012, + OUT_OF_MEMORY = 1018, + CONNECTION_CLOSED = 1022, +#ifdef USE_API_NOISE + BAD_HANDSHAKE_PACKET_LEN = 1002, HANDSHAKESTATE_READ_FAILED = 1013, HANDSHAKESTATE_WRITE_FAILED = 1014, HANDSHAKESTATE_BAD_STATE = 1015, CIPHERSTATE_DECRYPT_FAILED = 1016, CIPHERSTATE_ENCRYPT_FAILED = 1017, - OUT_OF_MEMORY = 1018, HANDSHAKESTATE_SETUP_FAILED = 1019, HANDSHAKESTATE_SPLIT_FAILED = 1020, BAD_HANDSHAKE_ERROR_BYTE = 1021, - CONNECTION_CLOSED = 1022, +#endif }; -const char *api_error_to_str(APIError err); +const LogString *api_error_to_logstr(APIError err); class APIFrameHelper { public: APIFrameHelper() = default; - explicit APIFrameHelper(std::unique_ptr socket) : socket_owned_(std::move(socket)) { + explicit APIFrameHelper(std::unique_ptr socket, const ClientInfo *client_info) + : socket_owned_(std::move(socket)), client_info_(client_info) { socket_ = socket_owned_.get(); } virtual ~APIFrameHelper() = default; virtual APIError init() = 0; virtual APIError loop(); virtual APIError read_packet(ReadPacketBuffer *buffer) = 0; - bool can_write_without_blocking() { return state_ == State::DATA && tx_buf_.empty(); } + bool can_write_without_blocking() { return this->state_ == State::DATA && this->tx_buf_count_ == 0; } std::string getpeername() { return socket_->getpeername(); } int getpeername(struct sockaddr *addr, socklen_t *addrlen) { return socket_->getpeername(addr, addrlen); } APIError close() { @@ -94,44 +110,41 @@ class APIFrameHelper { } return APIError::OK; } - // Give this helper a name for logging - void set_log_info(std::string info) { info_ = std::move(info); } virtual APIError write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) = 0; // Write multiple protobuf packets in a single operation // packets contains (message_type, offset, length) for each message in the buffer // The buffer contains all messages with appropriate padding before each virtual APIError write_protobuf_packets(ProtoWriteBuffer buffer, std::span packets) = 0; // Get the frame header padding required by this protocol - virtual uint8_t frame_header_padding() = 0; + uint8_t frame_header_padding() const { return frame_header_padding_; } // Get the frame footer size required by this protocol - virtual uint8_t frame_footer_size() = 0; + uint8_t frame_footer_size() const { return frame_footer_size_; } // Check if socket has data ready to read bool is_socket_ready() const { return socket_ != nullptr && socket_->ready(); } protected: - // Struct for holding parsed frame data - struct ParsedFrame { - std::vector msg; - }; - // Buffer containing data to be sent struct SendBuffer { - std::vector data; - uint16_t offset{0}; // Current offset within the buffer (uint16_t to reduce memory usage) + std::unique_ptr data; + uint16_t size{0}; // Total size of the buffer + uint16_t offset{0}; // Current offset within the buffer // Using uint16_t reduces memory usage since ESPHome API messages are limited to UINT16_MAX (65535) bytes - uint16_t remaining() const { return static_cast(data.size()) - offset; } - const uint8_t *current_data() const { return data.data() + offset; } + uint16_t remaining() const { return size - offset; } + const uint8_t *current_data() const { return data.get() + offset; } }; // Common implementation for writing raw data to socket - APIError write_raw_(const struct iovec *iov, int iovcnt); + APIError write_raw_(const struct iovec *iov, int iovcnt, uint16_t total_write_len); // Try to send data from the tx buffer APIError try_send_tx_buf_(); // Helper method to buffer data from IOVs - void buffer_data_from_iov_(const struct iovec *iov, int iovcnt, uint16_t total_write_len); + void buffer_data_from_iov_(const struct iovec *iov, int iovcnt, uint16_t total_write_len, uint16_t offset); + + // Common socket write error handling + APIError handle_socket_write_error_(); template APIError write_raw_(const struct iovec *iov, int iovcnt, socket::Socket *socket, std::vector &tx_buf, const std::string &info, StateEnum &state, StateEnum failed_state); @@ -160,17 +173,23 @@ class APIFrameHelper { }; // Containers (size varies, but typically 12+ bytes on 32-bit) - std::deque tx_buf_; - std::string info_; + std::array, API_MAX_SEND_QUEUE> tx_buf_; std::vector reusable_iovs_; std::vector rx_buf_; + // Pointer to client info (4 bytes on 32-bit) + // Note: The pointed-to ClientInfo object must outlive this APIFrameHelper instance. + const ClientInfo *client_info_{nullptr}; + // Group smaller types together uint16_t rx_buf_len_ = 0; State state_{State::INITIALIZE}; uint8_t frame_header_padding_{0}; uint8_t frame_footer_size_{0}; - // 5 bytes total, 3 bytes padding + uint8_t tx_buf_head_{0}; + uint8_t tx_buf_tail_{0}; + uint8_t tx_buf_count_{0}; + // 8 bytes total, 0 bytes padding // Common initialization for both plaintext and noise protocols APIError init_common_(); @@ -179,105 +198,6 @@ class APIFrameHelper { APIError handle_socket_read_result_(ssize_t received); }; -#ifdef USE_API_NOISE -class APINoiseFrameHelper : public APIFrameHelper { - public: - APINoiseFrameHelper(std::unique_ptr socket, std::shared_ptr ctx) - : APIFrameHelper(std::move(socket)), ctx_(std::move(ctx)) { - // Noise header structure: - // Pos 0: indicator (0x01) - // Pos 1-2: encrypted payload size (16-bit big-endian) - // Pos 3-6: encrypted type (16-bit) + data_len (16-bit) - // Pos 7+: actual payload data - frame_header_padding_ = 7; - } - ~APINoiseFrameHelper() override; - APIError init() override; - APIError loop() override; - APIError read_packet(ReadPacketBuffer *buffer) override; - APIError write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) override; - APIError write_protobuf_packets(ProtoWriteBuffer buffer, std::span packets) override; - // Get the frame header padding required by this protocol - uint8_t frame_header_padding() override { return frame_header_padding_; } - // Get the frame footer size required by this protocol - uint8_t frame_footer_size() override { return frame_footer_size_; } +} // namespace esphome::api - protected: - APIError state_action_(); - APIError try_read_frame_(ParsedFrame *frame); - APIError write_frame_(const uint8_t *data, uint16_t len); - APIError init_handshake_(); - APIError check_handshake_finished_(); - void send_explicit_handshake_reject_(const std::string &reason); - - // Pointers first (4 bytes each) - NoiseHandshakeState *handshake_{nullptr}; - NoiseCipherState *send_cipher_{nullptr}; - NoiseCipherState *recv_cipher_{nullptr}; - - // Shared pointer (8 bytes on 32-bit = 4 bytes control block pointer + 4 bytes object pointer) - std::shared_ptr ctx_; - - // Vector (12 bytes on 32-bit) - std::vector prologue_; - - // NoiseProtocolId (size depends on implementation) - NoiseProtocolId nid_; - - // Group small types together - // Fixed-size header buffer for noise protocol: - // 1 byte for indicator + 2 bytes for message size (16-bit value, not varint) - // Note: Maximum message size is UINT16_MAX (65535), with a limit of 128 bytes during handshake phase - uint8_t rx_header_buf_[3]; - uint8_t rx_header_buf_len_ = 0; - // 4 bytes total, no padding -}; -#endif // USE_API_NOISE - -#ifdef USE_API_PLAINTEXT -class APIPlaintextFrameHelper : public APIFrameHelper { - public: - APIPlaintextFrameHelper(std::unique_ptr socket) : APIFrameHelper(std::move(socket)) { - // Plaintext header structure (worst case): - // Pos 0: indicator (0x00) - // Pos 1-3: payload size varint (up to 3 bytes) - // Pos 4-5: message type varint (up to 2 bytes) - // Pos 6+: actual payload data - frame_header_padding_ = 6; - } - ~APIPlaintextFrameHelper() override = default; - APIError init() override; - APIError loop() override; - APIError read_packet(ReadPacketBuffer *buffer) override; - APIError write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) override; - APIError write_protobuf_packets(ProtoWriteBuffer buffer, std::span packets) override; - uint8_t frame_header_padding() override { return frame_header_padding_; } - // Get the frame footer size required by this protocol - uint8_t frame_footer_size() override { return frame_footer_size_; } - - protected: - APIError try_read_frame_(ParsedFrame *frame); - - // Group 2-byte aligned types - uint16_t rx_header_parsed_type_ = 0; - uint16_t rx_header_parsed_len_ = 0; - - // Group 1-byte types together - // Fixed-size header buffer for plaintext protocol: - // We now store the indicator byte + the two varints. - // To match noise protocol's maximum message size (UINT16_MAX = 65535), we need: - // 1 byte for indicator + 3 bytes for message size varint (supports up to 2097151) + 2 bytes for message type varint - // - // While varints could theoretically be up to 10 bytes each for 64-bit values, - // attempting to process messages with headers that large would likely crash the - // ESP32 due to memory constraints. - uint8_t rx_header_buf_[6]; // 1 byte indicator + 5 bytes for varints (3 for size + 2 for type) - uint8_t rx_header_buf_pos_ = 0; - bool rx_header_parsed_ = false; - // 8 bytes total, no padding needed -}; -#endif - -} // namespace api -} // namespace esphome -#endif +#endif // USE_API diff --git a/esphome/components/api/api_frame_helper_noise.cpp b/esphome/components/api/api_frame_helper_noise.cpp new file mode 100644 index 0000000000..e952ea670b --- /dev/null +++ b/esphome/components/api/api_frame_helper_noise.cpp @@ -0,0 +1,605 @@ +#include "api_frame_helper_noise.h" +#ifdef USE_API +#ifdef USE_API_NOISE +#include "api_connection.h" // For ClientInfo struct +#include "esphome/core/application.h" +#include "esphome/core/hal.h" +#include "esphome/core/helpers.h" +#include "esphome/core/log.h" +#include "proto.h" +#include +#include + +#ifdef USE_ESP8266 +#include +#endif + +namespace esphome::api { + +static const char *const TAG = "api.noise"; +#ifdef USE_ESP8266 +static const char PROLOGUE_INIT[] PROGMEM = "NoiseAPIInit"; +#else +static const char *const PROLOGUE_INIT = "NoiseAPIInit"; +#endif +static constexpr size_t PROLOGUE_INIT_LEN = 12; // strlen("NoiseAPIInit") + +#define HELPER_LOG(msg, ...) \ + ESP_LOGVV(TAG, "%s (%s): " msg, this->client_info_->name.c_str(), this->client_info_->peername.c_str(), ##__VA_ARGS__) + +#ifdef HELPER_LOG_PACKETS +#define LOG_PACKET_RECEIVED(buffer) ESP_LOGVV(TAG, "Received frame: %s", format_hex_pretty(buffer).c_str()) +#define LOG_PACKET_SENDING(data, len) ESP_LOGVV(TAG, "Sending raw: %s", format_hex_pretty(data, len).c_str()) +#else +#define LOG_PACKET_RECEIVED(buffer) ((void) 0) +#define LOG_PACKET_SENDING(data, len) ((void) 0) +#endif + +/// Convert a noise error code to a readable error +const LogString *noise_err_to_logstr(int err) { + if (err == NOISE_ERROR_NO_MEMORY) + return LOG_STR("NO_MEMORY"); + if (err == NOISE_ERROR_UNKNOWN_ID) + return LOG_STR("UNKNOWN_ID"); + if (err == NOISE_ERROR_UNKNOWN_NAME) + return LOG_STR("UNKNOWN_NAME"); + if (err == NOISE_ERROR_MAC_FAILURE) + return LOG_STR("MAC_FAILURE"); + if (err == NOISE_ERROR_NOT_APPLICABLE) + return LOG_STR("NOT_APPLICABLE"); + if (err == NOISE_ERROR_SYSTEM) + return LOG_STR("SYSTEM"); + if (err == NOISE_ERROR_REMOTE_KEY_REQUIRED) + return LOG_STR("REMOTE_KEY_REQUIRED"); + if (err == NOISE_ERROR_LOCAL_KEY_REQUIRED) + return LOG_STR("LOCAL_KEY_REQUIRED"); + if (err == NOISE_ERROR_PSK_REQUIRED) + return LOG_STR("PSK_REQUIRED"); + if (err == NOISE_ERROR_INVALID_LENGTH) + return LOG_STR("INVALID_LENGTH"); + if (err == NOISE_ERROR_INVALID_PARAM) + return LOG_STR("INVALID_PARAM"); + if (err == NOISE_ERROR_INVALID_STATE) + return LOG_STR("INVALID_STATE"); + if (err == NOISE_ERROR_INVALID_NONCE) + return LOG_STR("INVALID_NONCE"); + if (err == NOISE_ERROR_INVALID_PRIVATE_KEY) + return LOG_STR("INVALID_PRIVATE_KEY"); + if (err == NOISE_ERROR_INVALID_PUBLIC_KEY) + return LOG_STR("INVALID_PUBLIC_KEY"); + if (err == NOISE_ERROR_INVALID_FORMAT) + return LOG_STR("INVALID_FORMAT"); + if (err == NOISE_ERROR_INVALID_SIGNATURE) + return LOG_STR("INVALID_SIGNATURE"); + return LOG_STR("UNKNOWN"); +} + +/// Initialize the frame helper, returns OK if successful. +APIError APINoiseFrameHelper::init() { + APIError err = init_common_(); + if (err != APIError::OK) { + return err; + } + + // init prologue + size_t old_size = prologue_.size(); + prologue_.resize(old_size + PROLOGUE_INIT_LEN); +#ifdef USE_ESP8266 + memcpy_P(prologue_.data() + old_size, PROLOGUE_INIT, PROLOGUE_INIT_LEN); +#else + std::memcpy(prologue_.data() + old_size, PROLOGUE_INIT, PROLOGUE_INIT_LEN); +#endif + + state_ = State::CLIENT_HELLO; + return APIError::OK; +} +// Helper for handling handshake frame errors +APIError APINoiseFrameHelper::handle_handshake_frame_error_(APIError aerr) { + if (aerr == APIError::BAD_INDICATOR) { + send_explicit_handshake_reject_(LOG_STR("Bad indicator byte")); + } else if (aerr == APIError::BAD_HANDSHAKE_PACKET_LEN) { + send_explicit_handshake_reject_(LOG_STR("Bad handshake packet len")); + } + return aerr; +} + +// Helper for handling noise library errors +APIError APINoiseFrameHelper::handle_noise_error_(int err, const LogString *func_name, APIError api_err) { + if (err != 0) { + state_ = State::FAILED; + HELPER_LOG("%s failed: %s", LOG_STR_ARG(func_name), LOG_STR_ARG(noise_err_to_logstr(err))); + return api_err; + } + return APIError::OK; +} + +/// Run through handshake messages (if in that phase) +APIError APINoiseFrameHelper::loop() { + // During handshake phase, process as many actions as possible until we can't progress + // socket_->ready() stays true until next main loop, but state_action() will return + // WOULD_BLOCK when no more data is available to read + while (state_ != State::DATA && this->socket_->ready()) { + APIError err = state_action_(); + if (err == APIError::WOULD_BLOCK) { + break; + } + if (err != APIError::OK) { + return err; + } + } + + // Use base class implementation for buffer sending + return APIFrameHelper::loop(); +} + +/** Read a packet into the rx_buf_. + * + * @return APIError::OK if a full packet is in rx_buf_ + * + * errno EWOULDBLOCK: Packet could not be read without blocking. Try again later. + * errno ENOMEM: Not enough memory for reading packet. + * errno API_ERROR_BAD_INDICATOR: Bad indicator byte at start of frame. + * errno API_ERROR_HANDSHAKE_PACKET_LEN: Packet too big for this phase. + */ +APIError APINoiseFrameHelper::try_read_frame_() { + // read header + if (rx_header_buf_len_ < 3) { + // no header information yet + uint8_t to_read = 3 - rx_header_buf_len_; + ssize_t received = this->socket_->read(&rx_header_buf_[rx_header_buf_len_], to_read); + APIError err = handle_socket_read_result_(received); + if (err != APIError::OK) { + return err; + } + rx_header_buf_len_ += static_cast(received); + if (static_cast(received) != to_read) { + // not a full read + return APIError::WOULD_BLOCK; + } + + if (rx_header_buf_[0] != 0x01) { + state_ = State::FAILED; + HELPER_LOG("Bad indicator byte %u", rx_header_buf_[0]); + return APIError::BAD_INDICATOR; + } + // header reading done + } + + // read body + uint16_t msg_size = (((uint16_t) rx_header_buf_[1]) << 8) | rx_header_buf_[2]; + + // Check against size limits to prevent OOM: MAX_HANDSHAKE_SIZE for handshake, MAX_MESSAGE_SIZE for data + uint16_t limit = (state_ == State::DATA) ? MAX_MESSAGE_SIZE : MAX_HANDSHAKE_SIZE; + if (msg_size > limit) { + state_ = State::FAILED; + HELPER_LOG("Bad packet: message size %u exceeds maximum %u", msg_size, limit); + return (state_ == State::DATA) ? APIError::BAD_DATA_PACKET : APIError::BAD_HANDSHAKE_PACKET_LEN; + } + + // Reserve space for body + if (this->rx_buf_.size() != msg_size) { + this->rx_buf_.resize(msg_size); + } + + if (rx_buf_len_ < msg_size) { + // more data to read + uint16_t to_read = msg_size - rx_buf_len_; + ssize_t received = this->socket_->read(&rx_buf_[rx_buf_len_], to_read); + APIError err = handle_socket_read_result_(received); + if (err != APIError::OK) { + return err; + } + rx_buf_len_ += static_cast(received); + if (static_cast(received) != to_read) { + // not all read + return APIError::WOULD_BLOCK; + } + } + + LOG_PACKET_RECEIVED(this->rx_buf_); + + // Clear state for next frame (rx_buf_ still contains data for caller) + this->rx_buf_len_ = 0; + this->rx_header_buf_len_ = 0; + + return APIError::OK; +} + +/** To be called from read/write methods. + * + * This method runs through the internal handshake methods, if in that state. + * + * If the handshake is still active when this method returns and a read/write can't take place at + * the moment, returns WOULD_BLOCK. + * If an error occurred, returns that error. Only returns OK if the transport is ready for data + * traffic. + */ +APIError APINoiseFrameHelper::state_action_() { + int err; + APIError aerr; + if (state_ == State::INITIALIZE) { + HELPER_LOG("Bad state for method: %d", (int) state_); + return APIError::BAD_STATE; + } + if (state_ == State::CLIENT_HELLO) { + // waiting for client hello + aerr = this->try_read_frame_(); + if (aerr != APIError::OK) { + return handle_handshake_frame_error_(aerr); + } + // ignore contents, may be used in future for flags + // Resize for: existing prologue + 2 size bytes + frame data + size_t old_size = this->prologue_.size(); + this->prologue_.resize(old_size + 2 + this->rx_buf_.size()); + this->prologue_[old_size] = (uint8_t) (this->rx_buf_.size() >> 8); + this->prologue_[old_size + 1] = (uint8_t) this->rx_buf_.size(); + std::memcpy(this->prologue_.data() + old_size + 2, this->rx_buf_.data(), this->rx_buf_.size()); + + state_ = State::SERVER_HELLO; + } + if (state_ == State::SERVER_HELLO) { + // send server hello + const std::string &name = App.get_name(); + const std::string &mac = get_mac_address(); + + // Calculate positions and sizes + size_t name_len = name.size() + 1; // including null terminator + size_t mac_len = mac.size() + 1; // including null terminator + size_t name_offset = 1; + size_t mac_offset = name_offset + name_len; + size_t total_size = 1 + name_len + mac_len; + + auto msg = std::make_unique(total_size); + + // chosen proto + msg[0] = 0x01; + + // node name, terminated by null byte + std::memcpy(msg.get() + name_offset, name.c_str(), name_len); + // node mac, terminated by null byte + std::memcpy(msg.get() + mac_offset, mac.c_str(), mac_len); + + aerr = write_frame_(msg.get(), total_size); + if (aerr != APIError::OK) + return aerr; + + // start handshake + aerr = init_handshake_(); + if (aerr != APIError::OK) + return aerr; + + state_ = State::HANDSHAKE; + } + if (state_ == State::HANDSHAKE) { + int action = noise_handshakestate_get_action(handshake_); + if (action == NOISE_ACTION_READ_MESSAGE) { + // waiting for handshake msg + aerr = this->try_read_frame_(); + if (aerr != APIError::OK) { + return handle_handshake_frame_error_(aerr); + } + + if (this->rx_buf_.empty()) { + send_explicit_handshake_reject_(LOG_STR("Empty handshake message")); + return APIError::BAD_HANDSHAKE_ERROR_BYTE; + } else if (this->rx_buf_[0] != 0x00) { + HELPER_LOG("Bad handshake error byte: %u", this->rx_buf_[0]); + send_explicit_handshake_reject_(LOG_STR("Bad handshake error byte")); + return APIError::BAD_HANDSHAKE_ERROR_BYTE; + } + + NoiseBuffer mbuf; + noise_buffer_init(mbuf); + noise_buffer_set_input(mbuf, this->rx_buf_.data() + 1, this->rx_buf_.size() - 1); + err = noise_handshakestate_read_message(handshake_, &mbuf, nullptr); + if (err != 0) { + // Special handling for MAC failure + send_explicit_handshake_reject_(err == NOISE_ERROR_MAC_FAILURE ? LOG_STR("Handshake MAC failure") + : LOG_STR("Handshake error")); + return handle_noise_error_(err, LOG_STR("noise_handshakestate_read_message"), + APIError::HANDSHAKESTATE_READ_FAILED); + } + + aerr = check_handshake_finished_(); + if (aerr != APIError::OK) + return aerr; + } else if (action == NOISE_ACTION_WRITE_MESSAGE) { + uint8_t buffer[65]; + NoiseBuffer mbuf; + noise_buffer_init(mbuf); + noise_buffer_set_output(mbuf, buffer + 1, sizeof(buffer) - 1); + + err = noise_handshakestate_write_message(handshake_, &mbuf, nullptr); + APIError aerr_write = handle_noise_error_(err, LOG_STR("noise_handshakestate_write_message"), + APIError::HANDSHAKESTATE_WRITE_FAILED); + if (aerr_write != APIError::OK) + return aerr_write; + buffer[0] = 0x00; // success + + aerr = write_frame_(buffer, mbuf.size + 1); + if (aerr != APIError::OK) + return aerr; + aerr = check_handshake_finished_(); + if (aerr != APIError::OK) + return aerr; + } else { + // bad state for action + state_ = State::FAILED; + HELPER_LOG("Bad action for handshake: %d", action); + return APIError::HANDSHAKESTATE_BAD_STATE; + } + } + if (state_ == State::CLOSED || state_ == State::FAILED) { + return APIError::BAD_STATE; + } + return APIError::OK; +} +void APINoiseFrameHelper::send_explicit_handshake_reject_(const LogString *reason) { +#ifdef USE_STORE_LOG_STR_IN_FLASH + // On ESP8266 with flash strings, we need to use PROGMEM-aware functions + size_t reason_len = strlen_P(reinterpret_cast(reason)); + size_t data_size = reason_len + 1; + auto data = std::make_unique(data_size); + data[0] = 0x01; // failure + + // Copy error message from PROGMEM + if (reason_len > 0) { + memcpy_P(data.get() + 1, reinterpret_cast(reason), reason_len); + } +#else + // Normal memory access + const char *reason_str = LOG_STR_ARG(reason); + size_t reason_len = strlen(reason_str); + size_t data_size = reason_len + 1; + auto data = std::make_unique(data_size); + data[0] = 0x01; // failure + + // Copy error message in bulk + if (reason_len > 0) { + std::memcpy(data.get() + 1, reason_str, reason_len); + } +#endif + + // temporarily remove failed state + auto orig_state = state_; + state_ = State::EXPLICIT_REJECT; + write_frame_(data.get(), data_size); + state_ = orig_state; +} +APIError APINoiseFrameHelper::read_packet(ReadPacketBuffer *buffer) { + APIError aerr = this->state_action_(); + if (aerr != APIError::OK) { + return aerr; + } + + if (this->state_ != State::DATA) { + return APIError::WOULD_BLOCK; + } + + aerr = this->try_read_frame_(); + if (aerr != APIError::OK) + return aerr; + + NoiseBuffer mbuf; + noise_buffer_init(mbuf); + noise_buffer_set_inout(mbuf, this->rx_buf_.data(), this->rx_buf_.size(), this->rx_buf_.size()); + int err = noise_cipherstate_decrypt(this->recv_cipher_, &mbuf); + APIError decrypt_err = + handle_noise_error_(err, LOG_STR("noise_cipherstate_decrypt"), APIError::CIPHERSTATE_DECRYPT_FAILED); + if (decrypt_err != APIError::OK) { + return decrypt_err; + } + + uint16_t msg_size = mbuf.size; + uint8_t *msg_data = this->rx_buf_.data(); + if (msg_size < 4) { + this->state_ = State::FAILED; + HELPER_LOG("Bad data packet: size %d too short", msg_size); + return APIError::BAD_DATA_PACKET; + } + + uint16_t type = (((uint16_t) msg_data[0]) << 8) | msg_data[1]; + uint16_t data_len = (((uint16_t) msg_data[2]) << 8) | msg_data[3]; + if (data_len > msg_size - 4) { + this->state_ = State::FAILED; + HELPER_LOG("Bad data packet: data_len %u greater than msg_size %u", data_len, msg_size); + return APIError::BAD_DATA_PACKET; + } + + buffer->container = std::move(this->rx_buf_); + buffer->data_offset = 4; + buffer->data_len = data_len; + buffer->type = type; + return APIError::OK; +} +APIError APINoiseFrameHelper::write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) { + // Resize to include MAC space (required for Noise encryption) + buffer.get_buffer()->resize(buffer.get_buffer()->size() + frame_footer_size_); + PacketInfo packet{type, 0, + static_cast(buffer.get_buffer()->size() - frame_header_padding_ - frame_footer_size_)}; + return write_protobuf_packets(buffer, std::span(&packet, 1)); +} + +APIError APINoiseFrameHelper::write_protobuf_packets(ProtoWriteBuffer buffer, std::span packets) { + APIError aerr = state_action_(); + if (aerr != APIError::OK) { + return aerr; + } + + if (state_ != State::DATA) { + return APIError::WOULD_BLOCK; + } + + if (packets.empty()) { + return APIError::OK; + } + + std::vector *raw_buffer = buffer.get_buffer(); + uint8_t *buffer_data = raw_buffer->data(); // Cache buffer pointer + + this->reusable_iovs_.clear(); + this->reusable_iovs_.reserve(packets.size()); + uint16_t total_write_len = 0; + + // We need to encrypt each packet in place + for (const auto &packet : packets) { + // The buffer already has padding at offset + uint8_t *buf_start = buffer_data + packet.offset; + + // Write noise header + buf_start[0] = 0x01; // indicator + // buf_start[1], buf_start[2] to be set after encryption + + // Write message header (to be encrypted) + const uint8_t msg_offset = 3; + buf_start[msg_offset] = static_cast(packet.message_type >> 8); // type high byte + buf_start[msg_offset + 1] = static_cast(packet.message_type); // type low byte + buf_start[msg_offset + 2] = static_cast(packet.payload_size >> 8); // data_len high byte + buf_start[msg_offset + 3] = static_cast(packet.payload_size); // data_len low byte + // payload data is already in the buffer starting at offset + 7 + + // Make sure we have space for MAC + // The buffer should already have been sized appropriately + + // Encrypt the message in place + NoiseBuffer mbuf; + noise_buffer_init(mbuf); + noise_buffer_set_inout(mbuf, buf_start + msg_offset, 4 + packet.payload_size, + 4 + packet.payload_size + frame_footer_size_); + + int err = noise_cipherstate_encrypt(send_cipher_, &mbuf); + APIError aerr = + handle_noise_error_(err, LOG_STR("noise_cipherstate_encrypt"), APIError::CIPHERSTATE_ENCRYPT_FAILED); + if (aerr != APIError::OK) + return aerr; + + // Fill in the encrypted size + buf_start[1] = static_cast(mbuf.size >> 8); + buf_start[2] = static_cast(mbuf.size); + + // Add iovec for this encrypted packet + size_t packet_len = static_cast(3 + mbuf.size); // indicator + size + encrypted data + this->reusable_iovs_.push_back({buf_start, packet_len}); + total_write_len += packet_len; + } + + // Send all encrypted packets in one writev call + return this->write_raw_(this->reusable_iovs_.data(), this->reusable_iovs_.size(), total_write_len); +} + +APIError APINoiseFrameHelper::write_frame_(const uint8_t *data, uint16_t len) { + uint8_t header[3]; + header[0] = 0x01; // indicator + header[1] = (uint8_t) (len >> 8); + header[2] = (uint8_t) len; + + struct iovec iov[2]; + iov[0].iov_base = header; + iov[0].iov_len = 3; + if (len == 0) { + return this->write_raw_(iov, 1, 3); // Just header + } + iov[1].iov_base = const_cast(data); + iov[1].iov_len = len; + + return this->write_raw_(iov, 2, 3 + len); // Header + data +} + +/** Initiate the data structures for the handshake. + * + * @return 0 on success, -1 on error (check errno) + */ +APIError APINoiseFrameHelper::init_handshake_() { + int err; + memset(&nid_, 0, sizeof(nid_)); + // const char *proto = "Noise_NNpsk0_25519_ChaChaPoly_SHA256"; + // err = noise_protocol_name_to_id(&nid_, proto, strlen(proto)); + nid_.pattern_id = NOISE_PATTERN_NN; + nid_.cipher_id = NOISE_CIPHER_CHACHAPOLY; + nid_.dh_id = NOISE_DH_CURVE25519; + nid_.prefix_id = NOISE_PREFIX_STANDARD; + nid_.hybrid_id = NOISE_DH_NONE; + nid_.hash_id = NOISE_HASH_SHA256; + nid_.modifier_ids[0] = NOISE_MODIFIER_PSK0; + + err = noise_handshakestate_new_by_id(&handshake_, &nid_, NOISE_ROLE_RESPONDER); + APIError aerr = + handle_noise_error_(err, LOG_STR("noise_handshakestate_new_by_id"), APIError::HANDSHAKESTATE_SETUP_FAILED); + if (aerr != APIError::OK) + return aerr; + + const auto &psk = ctx_->get_psk(); + err = noise_handshakestate_set_pre_shared_key(handshake_, psk.data(), psk.size()); + aerr = handle_noise_error_(err, LOG_STR("noise_handshakestate_set_pre_shared_key"), + APIError::HANDSHAKESTATE_SETUP_FAILED); + if (aerr != APIError::OK) + return aerr; + + err = noise_handshakestate_set_prologue(handshake_, prologue_.data(), prologue_.size()); + aerr = handle_noise_error_(err, LOG_STR("noise_handshakestate_set_prologue"), APIError::HANDSHAKESTATE_SETUP_FAILED); + if (aerr != APIError::OK) + return aerr; + // set_prologue copies it into handshakestate, so we can get rid of it now + prologue_ = {}; + + err = noise_handshakestate_start(handshake_); + aerr = handle_noise_error_(err, LOG_STR("noise_handshakestate_start"), APIError::HANDSHAKESTATE_SETUP_FAILED); + if (aerr != APIError::OK) + return aerr; + return APIError::OK; +} + +APIError APINoiseFrameHelper::check_handshake_finished_() { + assert(state_ == State::HANDSHAKE); + + int action = noise_handshakestate_get_action(handshake_); + if (action == NOISE_ACTION_READ_MESSAGE || action == NOISE_ACTION_WRITE_MESSAGE) + return APIError::OK; + if (action != NOISE_ACTION_SPLIT) { + state_ = State::FAILED; + HELPER_LOG("Bad action for handshake: %d", action); + return APIError::HANDSHAKESTATE_BAD_STATE; + } + int err = noise_handshakestate_split(handshake_, &send_cipher_, &recv_cipher_); + APIError aerr = + handle_noise_error_(err, LOG_STR("noise_handshakestate_split"), APIError::HANDSHAKESTATE_SPLIT_FAILED); + if (aerr != APIError::OK) + return aerr; + + frame_footer_size_ = noise_cipherstate_get_mac_length(send_cipher_); + + HELPER_LOG("Handshake complete!"); + noise_handshakestate_free(handshake_); + handshake_ = nullptr; + state_ = State::DATA; + return APIError::OK; +} + +APINoiseFrameHelper::~APINoiseFrameHelper() { + if (handshake_ != nullptr) { + noise_handshakestate_free(handshake_); + handshake_ = nullptr; + } + if (send_cipher_ != nullptr) { + noise_cipherstate_free(send_cipher_); + send_cipher_ = nullptr; + } + if (recv_cipher_ != nullptr) { + noise_cipherstate_free(recv_cipher_); + recv_cipher_ = nullptr; + } +} + +extern "C" { +// declare how noise generates random bytes (here with a good HWRNG based on the RF system) +void noise_rand_bytes(void *output, size_t len) { + if (!esphome::random_bytes(reinterpret_cast(output), len)) { + ESP_LOGE(TAG, "Acquiring random bytes failed; rebooting"); + arch_restart(); + } +} +} + +} // namespace esphome::api +#endif // USE_API_NOISE +#endif // USE_API diff --git a/esphome/components/api/api_frame_helper_noise.h b/esphome/components/api/api_frame_helper_noise.h new file mode 100644 index 0000000000..e3243e4fa5 --- /dev/null +++ b/esphome/components/api/api_frame_helper_noise.h @@ -0,0 +1,64 @@ +#pragma once +#include "api_frame_helper.h" +#ifdef USE_API +#ifdef USE_API_NOISE +#include "noise/protocol.h" +#include "api_noise_context.h" + +namespace esphome::api { + +class APINoiseFrameHelper final : public APIFrameHelper { + public: + APINoiseFrameHelper(std::unique_ptr socket, std::shared_ptr ctx, + const ClientInfo *client_info) + : APIFrameHelper(std::move(socket), client_info), ctx_(std::move(ctx)) { + // Noise header structure: + // Pos 0: indicator (0x01) + // Pos 1-2: encrypted payload size (16-bit big-endian) + // Pos 3-6: encrypted type (16-bit) + data_len (16-bit) + // Pos 7+: actual payload data + frame_header_padding_ = 7; + } + ~APINoiseFrameHelper() override; + APIError init() override; + APIError loop() override; + APIError read_packet(ReadPacketBuffer *buffer) override; + APIError write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) override; + APIError write_protobuf_packets(ProtoWriteBuffer buffer, std::span packets) override; + + protected: + APIError state_action_(); + APIError try_read_frame_(); + APIError write_frame_(const uint8_t *data, uint16_t len); + APIError init_handshake_(); + APIError check_handshake_finished_(); + void send_explicit_handshake_reject_(const LogString *reason); + APIError handle_handshake_frame_error_(APIError aerr); + APIError handle_noise_error_(int err, const LogString *func_name, APIError api_err); + + // Pointers first (4 bytes each) + NoiseHandshakeState *handshake_{nullptr}; + NoiseCipherState *send_cipher_{nullptr}; + NoiseCipherState *recv_cipher_{nullptr}; + + // Shared pointer (8 bytes on 32-bit = 4 bytes control block pointer + 4 bytes object pointer) + std::shared_ptr ctx_; + + // Vector (12 bytes on 32-bit) + std::vector prologue_; + + // NoiseProtocolId (size depends on implementation) + NoiseProtocolId nid_; + + // Group small types together + // Fixed-size header buffer for noise protocol: + // 1 byte for indicator + 2 bytes for message size (16-bit value, not varint) + // Note: Maximum message size is UINT16_MAX (65535), with a limit of 128 bytes during handshake phase + uint8_t rx_header_buf_[3]; + uint8_t rx_header_buf_len_ = 0; + // 4 bytes total, no padding +}; + +} // namespace esphome::api +#endif // USE_API_NOISE +#endif // USE_API diff --git a/esphome/components/api/api_frame_helper_plaintext.cpp b/esphome/components/api/api_frame_helper_plaintext.cpp new file mode 100644 index 0000000000..471e6c5404 --- /dev/null +++ b/esphome/components/api/api_frame_helper_plaintext.cpp @@ -0,0 +1,294 @@ +#include "api_frame_helper_plaintext.h" +#ifdef USE_API +#ifdef USE_API_PLAINTEXT +#include "api_connection.h" // For ClientInfo struct +#include "esphome/core/application.h" +#include "esphome/core/hal.h" +#include "esphome/core/helpers.h" +#include "esphome/core/log.h" +#include "proto.h" +#include +#include + +#ifdef USE_ESP8266 +#include +#endif + +namespace esphome::api { + +static const char *const TAG = "api.plaintext"; + +#define HELPER_LOG(msg, ...) \ + ESP_LOGVV(TAG, "%s (%s): " msg, this->client_info_->name.c_str(), this->client_info_->peername.c_str(), ##__VA_ARGS__) + +#ifdef HELPER_LOG_PACKETS +#define LOG_PACKET_RECEIVED(buffer) ESP_LOGVV(TAG, "Received frame: %s", format_hex_pretty(buffer).c_str()) +#define LOG_PACKET_SENDING(data, len) ESP_LOGVV(TAG, "Sending raw: %s", format_hex_pretty(data, len).c_str()) +#else +#define LOG_PACKET_RECEIVED(buffer) ((void) 0) +#define LOG_PACKET_SENDING(data, len) ((void) 0) +#endif + +/// Initialize the frame helper, returns OK if successful. +APIError APIPlaintextFrameHelper::init() { + APIError err = init_common_(); + if (err != APIError::OK) { + return err; + } + + state_ = State::DATA; + return APIError::OK; +} +APIError APIPlaintextFrameHelper::loop() { + if (state_ != State::DATA) { + return APIError::BAD_STATE; + } + // Use base class implementation for buffer sending + return APIFrameHelper::loop(); +} + +/** Read a packet into the rx_buf_. + * + * @return See APIError + * + * error API_ERROR_BAD_INDICATOR: Bad indicator byte at start of frame. + */ +APIError APIPlaintextFrameHelper::try_read_frame_() { + // read header + while (!rx_header_parsed_) { + // Now that we know when the socket is ready, we can read up to 3 bytes + // into the rx_header_buf_ before we have to switch back to reading + // one byte at a time to ensure we don't read past the message and + // into the next one. + + // Read directly into rx_header_buf_ at the current position + // Try to get to at least 3 bytes total (indicator + 2 varint bytes), then read one byte at a time + ssize_t received = + this->socket_->read(&rx_header_buf_[rx_header_buf_pos_], rx_header_buf_pos_ < 3 ? 3 - rx_header_buf_pos_ : 1); + APIError err = handle_socket_read_result_(received); + if (err != APIError::OK) { + return err; + } + + // If this was the first read, validate the indicator byte + if (rx_header_buf_pos_ == 0 && received > 0) { + if (rx_header_buf_[0] != 0x00) { + state_ = State::FAILED; + HELPER_LOG("Bad indicator byte %u", rx_header_buf_[0]); + return APIError::BAD_INDICATOR; + } + } + + rx_header_buf_pos_ += received; + + // Check for buffer overflow + if (rx_header_buf_pos_ >= sizeof(rx_header_buf_)) { + state_ = State::FAILED; + HELPER_LOG("Header buffer overflow"); + return APIError::BAD_DATA_PACKET; + } + + // Need at least 3 bytes total (indicator + 2 varint bytes) before trying to parse + if (rx_header_buf_pos_ < 3) { + continue; + } + + // At this point, we have at least 3 bytes total: + // - Validated indicator byte (0x00) stored at position 0 + // - At least 2 bytes in the buffer for the varints + // Buffer layout: + // [0]: indicator byte (0x00) + // [1-3]: Message size varint (variable length) + // - 2 bytes would only allow up to 16383, which is less than noise's UINT16_MAX (65535) + // - 3 bytes allows up to 2097151, ensuring we support at least as much as noise + // [2-5]: Message type varint (variable length) + // We now attempt to parse both varints. If either is incomplete, + // we'll continue reading more bytes. + + // Skip indicator byte at position 0 + uint8_t varint_pos = 1; + uint32_t consumed = 0; + + auto msg_size_varint = ProtoVarInt::parse(&rx_header_buf_[varint_pos], rx_header_buf_pos_ - varint_pos, &consumed); + if (!msg_size_varint.has_value()) { + // not enough data there yet + continue; + } + + if (msg_size_varint->as_uint32() > MAX_MESSAGE_SIZE) { + state_ = State::FAILED; + HELPER_LOG("Bad packet: message size %" PRIu32 " exceeds maximum %u", msg_size_varint->as_uint32(), + MAX_MESSAGE_SIZE); + return APIError::BAD_DATA_PACKET; + } + rx_header_parsed_len_ = msg_size_varint->as_uint16(); + + // Move to next varint position + varint_pos += consumed; + + auto msg_type_varint = ProtoVarInt::parse(&rx_header_buf_[varint_pos], rx_header_buf_pos_ - varint_pos, &consumed); + if (!msg_type_varint.has_value()) { + // not enough data there yet + continue; + } + if (msg_type_varint->as_uint32() > std::numeric_limits::max()) { + state_ = State::FAILED; + HELPER_LOG("Bad packet: message type %" PRIu32 " exceeds maximum %u", msg_type_varint->as_uint32(), + std::numeric_limits::max()); + return APIError::BAD_DATA_PACKET; + } + rx_header_parsed_type_ = msg_type_varint->as_uint16(); + rx_header_parsed_ = true; + } + // header reading done + + // Reserve space for body + if (this->rx_buf_.size() != this->rx_header_parsed_len_) { + this->rx_buf_.resize(this->rx_header_parsed_len_); + } + + if (rx_buf_len_ < rx_header_parsed_len_) { + // more data to read + uint16_t to_read = rx_header_parsed_len_ - rx_buf_len_; + ssize_t received = this->socket_->read(&rx_buf_[rx_buf_len_], to_read); + APIError err = handle_socket_read_result_(received); + if (err != APIError::OK) { + return err; + } + rx_buf_len_ += static_cast(received); + if (static_cast(received) != to_read) { + // not all read + return APIError::WOULD_BLOCK; + } + } + + LOG_PACKET_RECEIVED(this->rx_buf_); + + // Clear state for next frame (rx_buf_ still contains data for caller) + this->rx_buf_len_ = 0; + this->rx_header_buf_pos_ = 0; + this->rx_header_parsed_ = false; + + return APIError::OK; +} + +APIError APIPlaintextFrameHelper::read_packet(ReadPacketBuffer *buffer) { + if (this->state_ != State::DATA) { + return APIError::WOULD_BLOCK; + } + + APIError aerr = this->try_read_frame_(); + if (aerr != APIError::OK) { + if (aerr == APIError::BAD_INDICATOR) { + // Make sure to tell the remote that we don't + // understand the indicator byte so it knows + // we do not support it. + struct iovec iov[1]; + // The \x00 first byte is the marker for plaintext. + // + // The remote will know how to handle the indicator byte, + // but it likely won't understand the rest of the message. + // + // We must send at least 3 bytes to be read, so we add + // a message after the indicator byte to ensures its long + // enough and can aid in debugging. + static constexpr uint8_t INDICATOR_MSG_SIZE = 19; +#ifdef USE_ESP8266 + static const char MSG_PROGMEM[] PROGMEM = "\x00" + "Bad indicator byte"; + char msg[INDICATOR_MSG_SIZE]; + memcpy_P(msg, MSG_PROGMEM, INDICATOR_MSG_SIZE); + iov[0].iov_base = (void *) msg; +#else + static const char MSG[] = "\x00" + "Bad indicator byte"; + iov[0].iov_base = (void *) MSG; +#endif + iov[0].iov_len = INDICATOR_MSG_SIZE; + this->write_raw_(iov, 1, INDICATOR_MSG_SIZE); + } + return aerr; + } + + buffer->container = std::move(this->rx_buf_); + buffer->data_offset = 0; + buffer->data_len = this->rx_header_parsed_len_; + buffer->type = this->rx_header_parsed_type_; + return APIError::OK; +} +APIError APIPlaintextFrameHelper::write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) { + PacketInfo packet{type, 0, static_cast(buffer.get_buffer()->size() - frame_header_padding_)}; + return write_protobuf_packets(buffer, std::span(&packet, 1)); +} + +APIError APIPlaintextFrameHelper::write_protobuf_packets(ProtoWriteBuffer buffer, std::span packets) { + if (state_ != State::DATA) { + return APIError::BAD_STATE; + } + + if (packets.empty()) { + return APIError::OK; + } + + std::vector *raw_buffer = buffer.get_buffer(); + uint8_t *buffer_data = raw_buffer->data(); // Cache buffer pointer + + this->reusable_iovs_.clear(); + this->reusable_iovs_.reserve(packets.size()); + uint16_t total_write_len = 0; + + for (const auto &packet : packets) { + // Calculate varint sizes for header layout + uint8_t size_varint_len = api::ProtoSize::varint(static_cast(packet.payload_size)); + uint8_t type_varint_len = api::ProtoSize::varint(static_cast(packet.message_type)); + uint8_t total_header_len = 1 + size_varint_len + type_varint_len; + + // Calculate where to start writing the header + // The header starts at the latest possible position to minimize unused padding + // + // Example 1 (small values): total_header_len = 3, header_offset = 6 - 3 = 3 + // [0-2] - Unused padding + // [3] - 0x00 indicator byte + // [4] - Payload size varint (1 byte, for sizes 0-127) + // [5] - Message type varint (1 byte, for types 0-127) + // [6...] - Actual payload data + // + // Example 2 (medium values): total_header_len = 4, header_offset = 6 - 4 = 2 + // [0-1] - Unused padding + // [2] - 0x00 indicator byte + // [3-4] - Payload size varint (2 bytes, for sizes 128-16383) + // [5] - Message type varint (1 byte, for types 0-127) + // [6...] - Actual payload data + // + // Example 3 (large values): total_header_len = 6, header_offset = 6 - 6 = 0 + // [0] - 0x00 indicator byte + // [1-3] - Payload size varint (3 bytes, for sizes 16384-2097151) + // [4-5] - Message type varint (2 bytes, for types 128-32767) + // [6...] - Actual payload data + // + // The message starts at offset + frame_header_padding_ + // So we write the header starting at offset + frame_header_padding_ - total_header_len + uint8_t *buf_start = buffer_data + packet.offset; + uint32_t header_offset = frame_header_padding_ - total_header_len; + + // Write the plaintext header + buf_start[header_offset] = 0x00; // indicator + + // Encode varints directly into buffer + ProtoVarInt(packet.payload_size).encode_to_buffer_unchecked(buf_start + header_offset + 1, size_varint_len); + ProtoVarInt(packet.message_type) + .encode_to_buffer_unchecked(buf_start + header_offset + 1 + size_varint_len, type_varint_len); + + // Add iovec for this packet (header + payload) + size_t packet_len = static_cast(total_header_len + packet.payload_size); + this->reusable_iovs_.push_back({buf_start + header_offset, packet_len}); + total_write_len += packet_len; + } + + // Send all packets in one writev call + return write_raw_(this->reusable_iovs_.data(), this->reusable_iovs_.size(), total_write_len); +} + +} // namespace esphome::api +#endif // USE_API_PLAINTEXT +#endif // USE_API diff --git a/esphome/components/api/api_frame_helper_plaintext.h b/esphome/components/api/api_frame_helper_plaintext.h new file mode 100644 index 0000000000..bba981d26b --- /dev/null +++ b/esphome/components/api/api_frame_helper_plaintext.h @@ -0,0 +1,50 @@ +#pragma once +#include "api_frame_helper.h" +#ifdef USE_API +#ifdef USE_API_PLAINTEXT + +namespace esphome::api { + +class APIPlaintextFrameHelper final : public APIFrameHelper { + public: + APIPlaintextFrameHelper(std::unique_ptr socket, const ClientInfo *client_info) + : APIFrameHelper(std::move(socket), client_info) { + // Plaintext header structure (worst case): + // Pos 0: indicator (0x00) + // Pos 1-3: payload size varint (up to 3 bytes) + // Pos 4-5: message type varint (up to 2 bytes) + // Pos 6+: actual payload data + frame_header_padding_ = 6; + } + ~APIPlaintextFrameHelper() override = default; + APIError init() override; + APIError loop() override; + APIError read_packet(ReadPacketBuffer *buffer) override; + APIError write_protobuf_packet(uint8_t type, ProtoWriteBuffer buffer) override; + APIError write_protobuf_packets(ProtoWriteBuffer buffer, std::span packets) override; + + protected: + APIError try_read_frame_(); + + // Group 2-byte aligned types + uint16_t rx_header_parsed_type_ = 0; + uint16_t rx_header_parsed_len_ = 0; + + // Group 1-byte types together + // Fixed-size header buffer for plaintext protocol: + // We now store the indicator byte + the two varints. + // To match noise protocol's maximum message size (UINT16_MAX = 65535), we need: + // 1 byte for indicator + 3 bytes for message size varint (supports up to 2097151) + 2 bytes for message type varint + // + // While varints could theoretically be up to 10 bytes each for 64-bit values, + // attempting to process messages with headers that large would likely crash the + // ESP32 due to memory constraints. + uint8_t rx_header_buf_[6]; // 1 byte indicator + 5 bytes for varints (3 for size + 2 for type) + uint8_t rx_header_buf_pos_ = 0; + bool rx_header_parsed_ = false; + // 8 bytes total, no padding needed +}; + +} // namespace esphome::api +#endif // USE_API_PLAINTEXT +#endif // USE_API diff --git a/esphome/components/api/api_noise_context.h b/esphome/components/api/api_noise_context.h index fa4435e570..b5f7016689 100644 --- a/esphome/components/api/api_noise_context.h +++ b/esphome/components/api/api_noise_context.h @@ -3,8 +3,7 @@ #include #include "esphome/core/defines.h" -namespace esphome { -namespace api { +namespace esphome::api { #ifdef USE_API_NOISE using psk_t = std::array; @@ -28,5 +27,4 @@ class APINoiseContext { }; #endif // USE_API_NOISE -} // namespace api -} // namespace esphome +} // namespace esphome::api diff --git a/esphome/components/api/api_options.proto b/esphome/components/api/api_options.proto index 3a547b8688..ead8ac0bbc 100644 --- a/esphome/components/api/api_options.proto +++ b/esphome/components/api/api_options.proto @@ -23,3 +23,51 @@ extend google.protobuf.MessageOptions { optional bool no_delay = 1040 [default=false]; optional string base_class = 1041; } + +extend google.protobuf.FieldOptions { + optional string field_ifdef = 1042; + optional uint32 fixed_array_size = 50007; + optional bool no_zero_copy = 50008 [default=false]; + optional bool fixed_array_skip_zero = 50009 [default=false]; + optional string fixed_array_size_define = 50010; + optional string fixed_array_with_length_define = 50011; + + // pointer_to_buffer: Use pointer instead of array for fixed-size byte fields + // When set, the field will be declared as a pointer (const uint8_t *data) + // instead of an array (uint8_t data[N]). This allows zero-copy on decode + // by pointing directly to the protobuf buffer. The buffer must remain valid + // until the message is processed (which is guaranteed for stack-allocated messages). + optional bool pointer_to_buffer = 50012 [default=false]; + + // container_pointer: Zero-copy optimization for repeated fields. + // + // When container_pointer is set on a repeated field, the generated message will + // store a pointer to an existing container instead of copying the data into the + // message's own repeated field. This eliminates heap allocations and improves performance. + // + // Requirements for safe usage: + // 1. The source container must remain valid until the message is encoded + // 2. Messages must be encoded immediately (which ESPHome does by default) + // 3. The container type must match the field type exactly + // + // Supported container types: + // - "std::vector" for most repeated fields + // - "std::set" for unique/sorted data + // - Full type specification required for enums (e.g., "std::set") + // + // Example usage in .proto file: + // repeated string supported_modes = 12 [(container_pointer) = "std::set"]; + // repeated ColorMode color_modes = 13 [(container_pointer) = "std::set"]; + // + // The corresponding C++ code must provide const reference access to a container + // that matches the specified type and remains valid during message encoding. + // This is typically done through methods returning const T& or special accessor + // methods like get_options() or supported_modes_for_api_(). + optional string container_pointer = 50001; + + // fixed_vector: Use FixedVector instead of std::vector for repeated fields + // When set, the repeated field will use FixedVector which requires calling + // init(size) before adding elements. This eliminates std::vector template overhead + // and is ideal when the exact size is known before populating the array. + optional bool fixed_vector = 50013 [default=false]; +} diff --git a/esphome/components/api/api_pb2.cpp b/esphome/components/api/api_pb2.cpp index 4c0e20e0f0..37bcf5d8a0 100644 --- a/esphome/components/api/api_pb2.cpp +++ b/esphome/components/api/api_pb2.cpp @@ -3,488 +3,515 @@ #include "api_pb2.h" #include "esphome/core/log.h" #include "esphome/core/helpers.h" +#include -namespace esphome { -namespace api { +namespace esphome::api { bool HelloRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->api_version_major = value.as_uint32(); - return true; - } - case 3: { + break; + case 3: this->api_version_minor = value.as_uint32(); - return true; - } + break; default: return false; } + return true; } bool HelloRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { case 1: { - this->client_info = value.as_string(); - return true; + // Use raw data directly to avoid allocation + this->client_info = value.data(); + this->client_info_len = value.size(); + break; } default: return false; } + return true; } void HelloResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint32(1, this->api_version_major); buffer.encode_uint32(2, this->api_version_minor); - buffer.encode_string(3, this->server_info); - buffer.encode_string(4, this->name); + buffer.encode_string(3, this->server_info_ref_); + buffer.encode_string(4, this->name_ref_); } -void HelloResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint32_field(total_size, 1, this->api_version_major); - ProtoSize::add_uint32_field(total_size, 1, this->api_version_minor); - ProtoSize::add_string_field(total_size, 1, this->server_info); - ProtoSize::add_string_field(total_size, 1, this->name); +void HelloResponse::calculate_size(ProtoSize &size) const { + size.add_uint32(1, this->api_version_major); + size.add_uint32(1, this->api_version_minor); + size.add_length(1, this->server_info_ref_.size()); + size.add_length(1, this->name_ref_.size()); } -bool ConnectRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { +#ifdef USE_API_PASSWORD +bool AuthenticationRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { case 1: { - this->password = value.as_string(); - return true; - } - default: - return false; - } -} -void ConnectResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_bool(1, this->invalid_password); } -void ConnectResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_bool_field(total_size, 1, this->invalid_password); -} -bool AreaInfo::decode_varint(uint32_t field_id, ProtoVarInt value) { - switch (field_id) { - case 1: { - this->area_id = value.as_uint32(); - return true; - } - default: - return false; - } -} -bool AreaInfo::decode_length(uint32_t field_id, ProtoLengthDelimited value) { - switch (field_id) { - case 2: { - this->name = value.as_string(); - return true; + // Use raw data directly to avoid allocation + this->password = value.data(); + this->password_len = value.size(); + break; } default: return false; } + return true; } +void AuthenticationResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_bool(1, this->invalid_password); } +void AuthenticationResponse::calculate_size(ProtoSize &size) const { size.add_bool(1, this->invalid_password); } +#endif +#ifdef USE_AREAS void AreaInfo::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint32(1, this->area_id); - buffer.encode_string(2, this->name); + buffer.encode_string(2, this->name_ref_); } -void AreaInfo::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint32_field(total_size, 1, this->area_id); - ProtoSize::add_string_field(total_size, 1, this->name); -} -bool DeviceInfo::decode_varint(uint32_t field_id, ProtoVarInt value) { - switch (field_id) { - case 1: { - this->device_id = value.as_uint32(); - return true; - } - case 3: { - this->area_id = value.as_uint32(); - return true; - } - default: - return false; - } -} -bool DeviceInfo::decode_length(uint32_t field_id, ProtoLengthDelimited value) { - switch (field_id) { - case 2: { - this->name = value.as_string(); - return true; - } - default: - return false; - } +void AreaInfo::calculate_size(ProtoSize &size) const { + size.add_uint32(1, this->area_id); + size.add_length(1, this->name_ref_.size()); } +#endif +#ifdef USE_DEVICES void DeviceInfo::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint32(1, this->device_id); - buffer.encode_string(2, this->name); + buffer.encode_string(2, this->name_ref_); buffer.encode_uint32(3, this->area_id); } -void DeviceInfo::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint32_field(total_size, 1, this->device_id); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_uint32_field(total_size, 1, this->area_id); +void DeviceInfo::calculate_size(ProtoSize &size) const { + size.add_uint32(1, this->device_id); + size.add_length(1, this->name_ref_.size()); + size.add_uint32(1, this->area_id); } +#endif void DeviceInfoResponse::encode(ProtoWriteBuffer buffer) const { +#ifdef USE_API_PASSWORD buffer.encode_bool(1, this->uses_password); - buffer.encode_string(2, this->name); - buffer.encode_string(3, this->mac_address); - buffer.encode_string(4, this->esphome_version); - buffer.encode_string(5, this->compilation_time); - buffer.encode_string(6, this->model); +#endif + buffer.encode_string(2, this->name_ref_); + buffer.encode_string(3, this->mac_address_ref_); + buffer.encode_string(4, this->esphome_version_ref_); + buffer.encode_string(5, this->compilation_time_ref_); + buffer.encode_string(6, this->model_ref_); +#ifdef USE_DEEP_SLEEP buffer.encode_bool(7, this->has_deep_sleep); - buffer.encode_string(8, this->project_name); - buffer.encode_string(9, this->project_version); +#endif +#ifdef ESPHOME_PROJECT_NAME + buffer.encode_string(8, this->project_name_ref_); +#endif +#ifdef ESPHOME_PROJECT_NAME + buffer.encode_string(9, this->project_version_ref_); +#endif +#ifdef USE_WEBSERVER buffer.encode_uint32(10, this->webserver_port); - buffer.encode_uint32(11, this->legacy_bluetooth_proxy_version); +#endif +#ifdef USE_BLUETOOTH_PROXY buffer.encode_uint32(15, this->bluetooth_proxy_feature_flags); - buffer.encode_string(12, this->manufacturer); - buffer.encode_string(13, this->friendly_name); - buffer.encode_uint32(14, this->legacy_voice_assistant_version); +#endif + buffer.encode_string(12, this->manufacturer_ref_); + buffer.encode_string(13, this->friendly_name_ref_); +#ifdef USE_VOICE_ASSISTANT buffer.encode_uint32(17, this->voice_assistant_feature_flags); - buffer.encode_string(16, this->suggested_area); - buffer.encode_string(18, this->bluetooth_mac_address); +#endif +#ifdef USE_AREAS + buffer.encode_string(16, this->suggested_area_ref_); +#endif +#ifdef USE_BLUETOOTH_PROXY + buffer.encode_string(18, this->bluetooth_mac_address_ref_); +#endif +#ifdef USE_API_NOISE buffer.encode_bool(19, this->api_encryption_supported); - for (auto &it : this->devices) { +#endif +#ifdef USE_DEVICES + for (const auto &it : this->devices) { buffer.encode_message(20, it, true); } - for (auto &it : this->areas) { +#endif +#ifdef USE_AREAS + for (const auto &it : this->areas) { buffer.encode_message(21, it, true); } +#endif +#ifdef USE_AREAS buffer.encode_message(22, this->area); +#endif +#ifdef USE_ZWAVE_PROXY + buffer.encode_uint32(23, this->zwave_proxy_feature_flags); +#endif +#ifdef USE_ZWAVE_PROXY + buffer.encode_uint32(24, this->zwave_home_id); +#endif } -void DeviceInfoResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_bool_field(total_size, 1, this->uses_password); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->mac_address); - ProtoSize::add_string_field(total_size, 1, this->esphome_version); - ProtoSize::add_string_field(total_size, 1, this->compilation_time); - ProtoSize::add_string_field(total_size, 1, this->model); - ProtoSize::add_bool_field(total_size, 1, this->has_deep_sleep); - ProtoSize::add_string_field(total_size, 1, this->project_name); - ProtoSize::add_string_field(total_size, 1, this->project_version); - ProtoSize::add_uint32_field(total_size, 1, this->webserver_port); - ProtoSize::add_uint32_field(total_size, 1, this->legacy_bluetooth_proxy_version); - ProtoSize::add_uint32_field(total_size, 1, this->bluetooth_proxy_feature_flags); - ProtoSize::add_string_field(total_size, 1, this->manufacturer); - ProtoSize::add_string_field(total_size, 1, this->friendly_name); - ProtoSize::add_uint32_field(total_size, 1, this->legacy_voice_assistant_version); - ProtoSize::add_uint32_field(total_size, 2, this->voice_assistant_feature_flags); - ProtoSize::add_string_field(total_size, 2, this->suggested_area); - ProtoSize::add_string_field(total_size, 2, this->bluetooth_mac_address); - ProtoSize::add_bool_field(total_size, 2, this->api_encryption_supported); - ProtoSize::add_repeated_message(total_size, 2, this->devices); - ProtoSize::add_repeated_message(total_size, 2, this->areas); - ProtoSize::add_message_object(total_size, 2, this->area); +void DeviceInfoResponse::calculate_size(ProtoSize &size) const { +#ifdef USE_API_PASSWORD + size.add_bool(1, this->uses_password); +#endif + size.add_length(1, this->name_ref_.size()); + size.add_length(1, this->mac_address_ref_.size()); + size.add_length(1, this->esphome_version_ref_.size()); + size.add_length(1, this->compilation_time_ref_.size()); + size.add_length(1, this->model_ref_.size()); +#ifdef USE_DEEP_SLEEP + size.add_bool(1, this->has_deep_sleep); +#endif +#ifdef ESPHOME_PROJECT_NAME + size.add_length(1, this->project_name_ref_.size()); +#endif +#ifdef ESPHOME_PROJECT_NAME + size.add_length(1, this->project_version_ref_.size()); +#endif +#ifdef USE_WEBSERVER + size.add_uint32(1, this->webserver_port); +#endif +#ifdef USE_BLUETOOTH_PROXY + size.add_uint32(1, this->bluetooth_proxy_feature_flags); +#endif + size.add_length(1, this->manufacturer_ref_.size()); + size.add_length(1, this->friendly_name_ref_.size()); +#ifdef USE_VOICE_ASSISTANT + size.add_uint32(2, this->voice_assistant_feature_flags); +#endif +#ifdef USE_AREAS + size.add_length(2, this->suggested_area_ref_.size()); +#endif +#ifdef USE_BLUETOOTH_PROXY + size.add_length(2, this->bluetooth_mac_address_ref_.size()); +#endif +#ifdef USE_API_NOISE + size.add_bool(2, this->api_encryption_supported); +#endif +#ifdef USE_DEVICES + for (const auto &it : this->devices) { + size.add_message_object_force(2, it); + } +#endif +#ifdef USE_AREAS + for (const auto &it : this->areas) { + size.add_message_object_force(2, it); + } +#endif +#ifdef USE_AREAS + size.add_message_object(2, this->area); +#endif +#ifdef USE_ZWAVE_PROXY + size.add_uint32(2, this->zwave_proxy_feature_flags); +#endif +#ifdef USE_ZWAVE_PROXY + size.add_uint32(2, this->zwave_home_id); +#endif } #ifdef USE_BINARY_SENSOR void ListEntitiesBinarySensorResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->device_class); + buffer.encode_string(3, this->name_ref_); + buffer.encode_string(5, this->device_class_ref_); buffer.encode_bool(6, this->is_status_binary_sensor); buffer.encode_bool(7, this->disabled_by_default); - buffer.encode_string(8, this->icon); +#ifdef USE_ENTITY_ICON + buffer.encode_string(8, this->icon_ref_); +#endif buffer.encode_uint32(9, static_cast(this->entity_category)); +#ifdef USE_DEVICES buffer.encode_uint32(10, this->device_id); +#endif } -void ListEntitiesBinarySensorResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->device_class); - ProtoSize::add_bool_field(total_size, 1, this->is_status_binary_sensor); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesBinarySensorResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); + size.add_length(1, this->device_class_ref_.size()); + size.add_bool(1, this->is_status_binary_sensor); + size.add_bool(1, this->disabled_by_default); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_uint32(1, static_cast(this->entity_category)); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void BinarySensorStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); buffer.encode_bool(2, this->state); buffer.encode_bool(3, this->missing_state); +#ifdef USE_DEVICES buffer.encode_uint32(4, this->device_id); +#endif } -void BinarySensorStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_bool_field(total_size, 1, this->state); - ProtoSize::add_bool_field(total_size, 1, this->missing_state); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void BinarySensorStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_bool(1, this->state); + size.add_bool(1, this->missing_state); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } #endif #ifdef USE_COVER void ListEntitiesCoverResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); + buffer.encode_string(3, this->name_ref_); buffer.encode_bool(5, this->assumed_state); buffer.encode_bool(6, this->supports_position); buffer.encode_bool(7, this->supports_tilt); - buffer.encode_string(8, this->device_class); + buffer.encode_string(8, this->device_class_ref_); buffer.encode_bool(9, this->disabled_by_default); - buffer.encode_string(10, this->icon); +#ifdef USE_ENTITY_ICON + buffer.encode_string(10, this->icon_ref_); +#endif buffer.encode_uint32(11, static_cast(this->entity_category)); buffer.encode_bool(12, this->supports_stop); +#ifdef USE_DEVICES buffer.encode_uint32(13, this->device_id); +#endif } -void ListEntitiesCoverResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_bool_field(total_size, 1, this->assumed_state); - ProtoSize::add_bool_field(total_size, 1, this->supports_position); - ProtoSize::add_bool_field(total_size, 1, this->supports_tilt); - ProtoSize::add_string_field(total_size, 1, this->device_class); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_bool_field(total_size, 1, this->supports_stop); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesCoverResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); + size.add_bool(1, this->assumed_state); + size.add_bool(1, this->supports_position); + size.add_bool(1, this->supports_tilt); + size.add_length(1, this->device_class_ref_.size()); + size.add_bool(1, this->disabled_by_default); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_uint32(1, static_cast(this->entity_category)); + size.add_bool(1, this->supports_stop); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void CoverStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); - buffer.encode_uint32(2, static_cast(this->legacy_state)); buffer.encode_float(3, this->position); buffer.encode_float(4, this->tilt); buffer.encode_uint32(5, static_cast(this->current_operation)); +#ifdef USE_DEVICES buffer.encode_uint32(6, this->device_id); +#endif } -void CoverStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->legacy_state)); - ProtoSize::add_fixed_field<4>(total_size, 1, this->position != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->tilt != 0.0f); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->current_operation)); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void CoverStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_float(1, this->position); + size.add_float(1, this->tilt); + size.add_uint32(1, static_cast(this->current_operation)); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool CoverCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { - this->has_legacy_command = value.as_bool(); - return true; - } - case 3: { - this->legacy_command = static_cast(value.as_uint32()); - return true; - } - case 4: { + case 4: this->has_position = value.as_bool(); - return true; - } - case 6: { + break; + case 6: this->has_tilt = value.as_bool(); - return true; - } - case 8: { + break; + case 8: this->stop = value.as_bool(); - return true; - } - case 9: { + break; +#ifdef USE_DEVICES + case 9: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool CoverCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } - case 5: { + break; + case 5: this->position = value.as_float(); - return true; - } - case 7: { + break; + case 7: this->tilt = value.as_float(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_FAN void ListEntitiesFanResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); + buffer.encode_string(3, this->name_ref_); buffer.encode_bool(5, this->supports_oscillation); buffer.encode_bool(6, this->supports_speed); buffer.encode_bool(7, this->supports_direction); buffer.encode_int32(8, this->supported_speed_count); buffer.encode_bool(9, this->disabled_by_default); - buffer.encode_string(10, this->icon); +#ifdef USE_ENTITY_ICON + buffer.encode_string(10, this->icon_ref_); +#endif buffer.encode_uint32(11, static_cast(this->entity_category)); - for (auto &it : this->supported_preset_modes) { + for (const auto &it : *this->supported_preset_modes) { buffer.encode_string(12, it, true); } +#ifdef USE_DEVICES buffer.encode_uint32(13, this->device_id); +#endif } -void ListEntitiesFanResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_bool_field(total_size, 1, this->supports_oscillation); - ProtoSize::add_bool_field(total_size, 1, this->supports_speed); - ProtoSize::add_bool_field(total_size, 1, this->supports_direction); - ProtoSize::add_int32_field(total_size, 1, this->supported_speed_count); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - if (!this->supported_preset_modes.empty()) { - for (const auto &it : this->supported_preset_modes) { - ProtoSize::add_string_field_repeated(total_size, 1, it); +void ListEntitiesFanResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); + size.add_bool(1, this->supports_oscillation); + size.add_bool(1, this->supports_speed); + size.add_bool(1, this->supports_direction); + size.add_int32(1, this->supported_speed_count); + size.add_bool(1, this->disabled_by_default); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_uint32(1, static_cast(this->entity_category)); + if (!this->supported_preset_modes->empty()) { + for (const auto &it : *this->supported_preset_modes) { + size.add_length_force(1, it.size()); } } - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void FanStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); buffer.encode_bool(2, this->state); buffer.encode_bool(3, this->oscillating); - buffer.encode_uint32(4, static_cast(this->speed)); buffer.encode_uint32(5, static_cast(this->direction)); buffer.encode_int32(6, this->speed_level); - buffer.encode_string(7, this->preset_mode); + buffer.encode_string(7, this->preset_mode_ref_); +#ifdef USE_DEVICES buffer.encode_uint32(8, this->device_id); +#endif } -void FanStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_bool_field(total_size, 1, this->state); - ProtoSize::add_bool_field(total_size, 1, this->oscillating); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->speed)); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->direction)); - ProtoSize::add_int32_field(total_size, 1, this->speed_level); - ProtoSize::add_string_field(total_size, 1, this->preset_mode); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void FanStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_bool(1, this->state); + size.add_bool(1, this->oscillating); + size.add_uint32(1, static_cast(this->direction)); + size.add_int32(1, this->speed_level); + size.add_length(1, this->preset_mode_ref_.size()); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool FanCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->has_state = value.as_bool(); - return true; - } - case 3: { + break; + case 3: this->state = value.as_bool(); - return true; - } - case 4: { - this->has_speed = value.as_bool(); - return true; - } - case 5: { - this->speed = static_cast(value.as_uint32()); - return true; - } - case 6: { + break; + case 6: this->has_oscillating = value.as_bool(); - return true; - } - case 7: { + break; + case 7: this->oscillating = value.as_bool(); - return true; - } - case 8: { + break; + case 8: this->has_direction = value.as_bool(); - return true; - } - case 9: { + break; + case 9: this->direction = static_cast(value.as_uint32()); - return true; - } - case 10: { + break; + case 10: this->has_speed_level = value.as_bool(); - return true; - } - case 11: { + break; + case 11: this->speed_level = value.as_int32(); - return true; - } - case 12: { + break; + case 12: this->has_preset_mode = value.as_bool(); - return true; - } - case 14: { + break; +#ifdef USE_DEVICES + case 14: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool FanCommandRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 13: { + case 13: this->preset_mode = value.as_string(); - return true; - } + break; default: return false; } + return true; } bool FanCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_LIGHT void ListEntitiesLightResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - for (auto &it : this->supported_color_modes) { + buffer.encode_string(3, this->name_ref_); + for (const auto &it : *this->supported_color_modes) { buffer.encode_uint32(12, static_cast(it), true); } - buffer.encode_bool(5, this->legacy_supports_brightness); - buffer.encode_bool(6, this->legacy_supports_rgb); - buffer.encode_bool(7, this->legacy_supports_white_value); - buffer.encode_bool(8, this->legacy_supports_color_temperature); buffer.encode_float(9, this->min_mireds); buffer.encode_float(10, this->max_mireds); for (auto &it : this->effects) { buffer.encode_string(11, it, true); } buffer.encode_bool(13, this->disabled_by_default); - buffer.encode_string(14, this->icon); +#ifdef USE_ENTITY_ICON + buffer.encode_string(14, this->icon_ref_); +#endif buffer.encode_uint32(15, static_cast(this->entity_category)); +#ifdef USE_DEVICES buffer.encode_uint32(16, this->device_id); +#endif } -void ListEntitiesLightResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - if (!this->supported_color_modes.empty()) { - for (const auto &it : this->supported_color_modes) { - ProtoSize::add_enum_field_repeated(total_size, 1, static_cast(it)); +void ListEntitiesLightResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); + if (!this->supported_color_modes->empty()) { + for (const auto &it : *this->supported_color_modes) { + size.add_uint32_force(1, static_cast(it)); } } - ProtoSize::add_bool_field(total_size, 1, this->legacy_supports_brightness); - ProtoSize::add_bool_field(total_size, 1, this->legacy_supports_rgb); - ProtoSize::add_bool_field(total_size, 1, this->legacy_supports_white_value); - ProtoSize::add_bool_field(total_size, 1, this->legacy_supports_color_temperature); - ProtoSize::add_fixed_field<4>(total_size, 1, this->min_mireds != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->max_mireds != 0.0f); + size.add_float(1, this->min_mireds); + size.add_float(1, this->max_mireds); if (!this->effects.empty()) { for (const auto &it : this->effects) { - ProtoSize::add_string_field_repeated(total_size, 1, it); + size.add_length_force(1, it.size()); } } - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_uint32_field(total_size, 2, this->device_id); + size.add_bool(1, this->disabled_by_default); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_uint32(1, static_cast(this->entity_category)); +#ifdef USE_DEVICES + size.add_uint32(2, this->device_id); +#endif } void LightStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); @@ -499,362 +526,354 @@ void LightStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_float(8, this->color_temperature); buffer.encode_float(12, this->cold_white); buffer.encode_float(13, this->warm_white); - buffer.encode_string(9, this->effect); + buffer.encode_string(9, this->effect_ref_); +#ifdef USE_DEVICES buffer.encode_uint32(14, this->device_id); +#endif } -void LightStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_bool_field(total_size, 1, this->state); - ProtoSize::add_fixed_field<4>(total_size, 1, this->brightness != 0.0f); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->color_mode)); - ProtoSize::add_fixed_field<4>(total_size, 1, this->color_brightness != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->red != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->green != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->blue != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->white != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->color_temperature != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->cold_white != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->warm_white != 0.0f); - ProtoSize::add_string_field(total_size, 1, this->effect); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void LightStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_bool(1, this->state); + size.add_float(1, this->brightness); + size.add_uint32(1, static_cast(this->color_mode)); + size.add_float(1, this->color_brightness); + size.add_float(1, this->red); + size.add_float(1, this->green); + size.add_float(1, this->blue); + size.add_float(1, this->white); + size.add_float(1, this->color_temperature); + size.add_float(1, this->cold_white); + size.add_float(1, this->warm_white); + size.add_length(1, this->effect_ref_.size()); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool LightCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->has_state = value.as_bool(); - return true; - } - case 3: { + break; + case 3: this->state = value.as_bool(); - return true; - } - case 4: { + break; + case 4: this->has_brightness = value.as_bool(); - return true; - } - case 22: { + break; + case 22: this->has_color_mode = value.as_bool(); - return true; - } - case 23: { + break; + case 23: this->color_mode = static_cast(value.as_uint32()); - return true; - } - case 20: { + break; + case 20: this->has_color_brightness = value.as_bool(); - return true; - } - case 6: { + break; + case 6: this->has_rgb = value.as_bool(); - return true; - } - case 10: { + break; + case 10: this->has_white = value.as_bool(); - return true; - } - case 12: { + break; + case 12: this->has_color_temperature = value.as_bool(); - return true; - } - case 24: { + break; + case 24: this->has_cold_white = value.as_bool(); - return true; - } - case 26: { + break; + case 26: this->has_warm_white = value.as_bool(); - return true; - } - case 14: { + break; + case 14: this->has_transition_length = value.as_bool(); - return true; - } - case 15: { + break; + case 15: this->transition_length = value.as_uint32(); - return true; - } - case 16: { + break; + case 16: this->has_flash_length = value.as_bool(); - return true; - } - case 17: { + break; + case 17: this->flash_length = value.as_uint32(); - return true; - } - case 18: { + break; + case 18: this->has_effect = value.as_bool(); - return true; - } - case 28: { + break; +#ifdef USE_DEVICES + case 28: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool LightCommandRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 19: { + case 19: this->effect = value.as_string(); - return true; - } + break; default: return false; } + return true; } bool LightCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } - case 5: { + break; + case 5: this->brightness = value.as_float(); - return true; - } - case 21: { + break; + case 21: this->color_brightness = value.as_float(); - return true; - } - case 7: { + break; + case 7: this->red = value.as_float(); - return true; - } - case 8: { + break; + case 8: this->green = value.as_float(); - return true; - } - case 9: { + break; + case 9: this->blue = value.as_float(); - return true; - } - case 11: { + break; + case 11: this->white = value.as_float(); - return true; - } - case 13: { + break; + case 13: this->color_temperature = value.as_float(); - return true; - } - case 25: { + break; + case 25: this->cold_white = value.as_float(); - return true; - } - case 27: { + break; + case 27: this->warm_white = value.as_float(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_SENSOR void ListEntitiesSensorResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); - buffer.encode_string(6, this->unit_of_measurement); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif + buffer.encode_string(6, this->unit_of_measurement_ref_); buffer.encode_int32(7, this->accuracy_decimals); buffer.encode_bool(8, this->force_update); - buffer.encode_string(9, this->device_class); + buffer.encode_string(9, this->device_class_ref_); buffer.encode_uint32(10, static_cast(this->state_class)); - buffer.encode_uint32(11, static_cast(this->legacy_last_reset_type)); buffer.encode_bool(12, this->disabled_by_default); buffer.encode_uint32(13, static_cast(this->entity_category)); +#ifdef USE_DEVICES buffer.encode_uint32(14, this->device_id); +#endif } -void ListEntitiesSensorResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_string_field(total_size, 1, this->unit_of_measurement); - ProtoSize::add_int32_field(total_size, 1, this->accuracy_decimals); - ProtoSize::add_bool_field(total_size, 1, this->force_update); - ProtoSize::add_string_field(total_size, 1, this->device_class); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->state_class)); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->legacy_last_reset_type)); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesSensorResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_length(1, this->unit_of_measurement_ref_.size()); + size.add_int32(1, this->accuracy_decimals); + size.add_bool(1, this->force_update); + size.add_length(1, this->device_class_ref_.size()); + size.add_uint32(1, static_cast(this->state_class)); + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void SensorStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); buffer.encode_float(2, this->state); buffer.encode_bool(3, this->missing_state); +#ifdef USE_DEVICES buffer.encode_uint32(4, this->device_id); +#endif } -void SensorStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_fixed_field<4>(total_size, 1, this->state != 0.0f); - ProtoSize::add_bool_field(total_size, 1, this->missing_state); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void SensorStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_float(1, this->state); + size.add_bool(1, this->missing_state); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } #endif #ifdef USE_SWITCH void ListEntitiesSwitchResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->assumed_state); buffer.encode_bool(7, this->disabled_by_default); buffer.encode_uint32(8, static_cast(this->entity_category)); - buffer.encode_string(9, this->device_class); + buffer.encode_string(9, this->device_class_ref_); +#ifdef USE_DEVICES buffer.encode_uint32(10, this->device_id); +#endif } -void ListEntitiesSwitchResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->assumed_state); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_string_field(total_size, 1, this->device_class); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesSwitchResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->assumed_state); + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); + size.add_length(1, this->device_class_ref_.size()); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void SwitchStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); buffer.encode_bool(2, this->state); +#ifdef USE_DEVICES buffer.encode_uint32(3, this->device_id); +#endif } -void SwitchStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_bool_field(total_size, 1, this->state); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void SwitchStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_bool(1, this->state); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool SwitchCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->state = value.as_bool(); - return true; - } - case 3: { + break; +#ifdef USE_DEVICES + case 3: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool SwitchCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_TEXT_SENSOR void ListEntitiesTextSensorResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->disabled_by_default); buffer.encode_uint32(7, static_cast(this->entity_category)); - buffer.encode_string(8, this->device_class); + buffer.encode_string(8, this->device_class_ref_); +#ifdef USE_DEVICES buffer.encode_uint32(9, this->device_id); +#endif } -void ListEntitiesTextSensorResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_string_field(total_size, 1, this->device_class); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesTextSensorResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); + size.add_length(1, this->device_class_ref_.size()); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void TextSensorStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); - buffer.encode_string(2, this->state); + buffer.encode_string(2, this->state_ref_); buffer.encode_bool(3, this->missing_state); +#ifdef USE_DEVICES buffer.encode_uint32(4, this->device_id); +#endif } -void TextSensorStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->state); - ProtoSize::add_bool_field(total_size, 1, this->missing_state); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void TextSensorStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_length(1, this->state_ref_.size()); + size.add_bool(1, this->missing_state); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } #endif bool SubscribeLogsRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->level = static_cast(value.as_uint32()); - return true; - } - case 2: { + break; + case 2: this->dump_config = value.as_bool(); - return true; - } + break; default: return false; } + return true; } void SubscribeLogsResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint32(1, static_cast(this->level)); - buffer.encode_bytes(3, reinterpret_cast(this->message.data()), this->message.size()); - buffer.encode_bool(4, this->send_failed); + buffer.encode_bytes(3, this->message_ptr_, this->message_len_); } -void SubscribeLogsResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_enum_field(total_size, 1, static_cast(this->level)); - ProtoSize::add_string_field(total_size, 1, this->message); - ProtoSize::add_bool_field(total_size, 1, this->send_failed); +void SubscribeLogsResponse::calculate_size(ProtoSize &size) const { + size.add_uint32(1, static_cast(this->level)); + size.add_length(1, this->message_len_); } #ifdef USE_API_NOISE bool NoiseEncryptionSetKeyRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_string(); - return true; - } + break; default: return false; } + return true; } void NoiseEncryptionSetKeyResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_bool(1, this->success); } -void NoiseEncryptionSetKeyResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_bool_field(total_size, 1, this->success); -} +void NoiseEncryptionSetKeyResponse::calculate_size(ProtoSize &size) const { size.add_bool(1, this->success); } #endif -bool HomeassistantServiceMap::decode_length(uint32_t field_id, ProtoLengthDelimited value) { - switch (field_id) { - case 1: { - this->key = value.as_string(); - return true; - } - case 2: { - this->value = value.as_string(); - return true; - } - default: - return false; - } -} +#ifdef USE_API_HOMEASSISTANT_SERVICES void HomeassistantServiceMap::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->key); + buffer.encode_string(1, this->key_ref_); buffer.encode_string(2, this->value); } -void HomeassistantServiceMap::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->key); - ProtoSize::add_string_field(total_size, 1, this->value); +void HomeassistantServiceMap::calculate_size(ProtoSize &size) const { + size.add_length(1, this->key_ref_.size()); + size.add_length(1, this->value.size()); } -void HomeassistantServiceResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->service); +void HomeassistantActionRequest::encode(ProtoWriteBuffer buffer) const { + buffer.encode_string(1, this->service_ref_); for (auto &it : this->data) { buffer.encode_message(2, it, true); } @@ -865,357 +884,379 @@ void HomeassistantServiceResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_message(4, it, true); } buffer.encode_bool(5, this->is_event); +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES + buffer.encode_uint32(6, this->call_id); +#endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + buffer.encode_bool(7, this->wants_response); +#endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + buffer.encode_string(8, this->response_template); +#endif } -void HomeassistantServiceResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->service); - ProtoSize::add_repeated_message(total_size, 1, this->data); - ProtoSize::add_repeated_message(total_size, 1, this->data_template); - ProtoSize::add_repeated_message(total_size, 1, this->variables); - ProtoSize::add_bool_field(total_size, 1, this->is_event); +void HomeassistantActionRequest::calculate_size(ProtoSize &size) const { + size.add_length(1, this->service_ref_.size()); + size.add_repeated_message(1, this->data); + size.add_repeated_message(1, this->data_template); + size.add_repeated_message(1, this->variables); + size.add_bool(1, this->is_event); +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES + size.add_uint32(1, this->call_id); +#endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + size.add_bool(1, this->wants_response); +#endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + size.add_length(1, this->response_template.size()); +#endif } +#endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES +bool HomeassistantActionResponse::decode_varint(uint32_t field_id, ProtoVarInt value) { + switch (field_id) { + case 1: + this->call_id = value.as_uint32(); + break; + case 2: + this->success = value.as_bool(); + break; + default: + return false; + } + return true; +} +bool HomeassistantActionResponse::decode_length(uint32_t field_id, ProtoLengthDelimited value) { + switch (field_id) { + case 3: + this->error_message = value.as_string(); + break; +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + case 4: { + // Use raw data directly to avoid allocation + this->response_data = value.data(); + this->response_data_len = value.size(); + break; + } +#endif + default: + return false; + } + return true; +} +#endif +#ifdef USE_API_HOMEASSISTANT_STATES void SubscribeHomeAssistantStateResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->entity_id); - buffer.encode_string(2, this->attribute); + buffer.encode_string(1, this->entity_id_ref_); + buffer.encode_string(2, this->attribute_ref_); buffer.encode_bool(3, this->once); } -void SubscribeHomeAssistantStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->entity_id); - ProtoSize::add_string_field(total_size, 1, this->attribute); - ProtoSize::add_bool_field(total_size, 1, this->once); +void SubscribeHomeAssistantStateResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->entity_id_ref_.size()); + size.add_length(1, this->attribute_ref_.size()); + size.add_bool(1, this->once); } bool HomeAssistantStateResponse::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 1: { + case 1: this->entity_id = value.as_string(); - return true; - } - case 2: { + break; + case 2: this->state = value.as_string(); - return true; - } - case 3: { + break; + case 3: this->attribute = value.as_string(); - return true; + break; + default: + return false; + } + return true; +} +#endif +bool GetTimeResponse::decode_length(uint32_t field_id, ProtoLengthDelimited value) { + switch (field_id) { + case 2: { + // Use raw data directly to avoid allocation + this->timezone = value.data(); + this->timezone_len = value.size(); + break; } default: return false; } + return true; } bool GetTimeResponse::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->epoch_seconds = value.as_fixed32(); - return true; - } + break; default: return false; } -} -void GetTimeResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->epoch_seconds); } -void GetTimeResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->epoch_seconds != 0); + return true; } #ifdef USE_API_SERVICES -bool ListEntitiesServicesArgument::decode_varint(uint32_t field_id, ProtoVarInt value) { - switch (field_id) { - case 2: { - this->type = static_cast(value.as_uint32()); - return true; - } - default: - return false; - } -} -bool ListEntitiesServicesArgument::decode_length(uint32_t field_id, ProtoLengthDelimited value) { - switch (field_id) { - case 1: { - this->name = value.as_string(); - return true; - } - default: - return false; - } -} void ListEntitiesServicesArgument::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->name); + buffer.encode_string(1, this->name_ref_); buffer.encode_uint32(2, static_cast(this->type)); } -void ListEntitiesServicesArgument::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->type)); +void ListEntitiesServicesArgument::calculate_size(ProtoSize &size) const { + size.add_length(1, this->name_ref_.size()); + size.add_uint32(1, static_cast(this->type)); } void ListEntitiesServicesResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->name); + buffer.encode_string(1, this->name_ref_); buffer.encode_fixed32(2, this->key); for (auto &it : this->args) { buffer.encode_message(3, it, true); } } -void ListEntitiesServicesResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_repeated_message(total_size, 1, this->args); +void ListEntitiesServicesResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->name_ref_.size()); + size.add_fixed32(1, this->key); + size.add_repeated_message(1, this->args); } bool ExecuteServiceArgument::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->bool_ = value.as_bool(); - return true; - } - case 2: { + break; + case 2: this->legacy_int = value.as_int32(); - return true; - } - case 5: { + break; + case 5: this->int_ = value.as_sint32(); - return true; - } - case 6: { + break; + case 6: this->bool_array.push_back(value.as_bool()); - return true; - } - case 7: { + break; + case 7: this->int_array.push_back(value.as_sint32()); - return true; - } + break; default: return false; } + return true; } bool ExecuteServiceArgument::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 4: { + case 4: this->string_ = value.as_string(); - return true; - } - case 9: { + break; + case 9: this->string_array.push_back(value.as_string()); - return true; - } + break; default: return false; } + return true; } bool ExecuteServiceArgument::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 3: { + case 3: this->float_ = value.as_float(); - return true; - } - case 8: { + break; + case 8: this->float_array.push_back(value.as_float()); - return true; - } + break; default: return false; } + return true; } -void ExecuteServiceArgument::encode(ProtoWriteBuffer buffer) const { - buffer.encode_bool(1, this->bool_); - buffer.encode_int32(2, this->legacy_int); - buffer.encode_float(3, this->float_); - buffer.encode_string(4, this->string_); - buffer.encode_sint32(5, this->int_); - for (auto it : this->bool_array) { - buffer.encode_bool(6, it, true); - } - for (auto &it : this->int_array) { - buffer.encode_sint32(7, it, true); - } - for (auto &it : this->float_array) { - buffer.encode_float(8, it, true); - } - for (auto &it : this->string_array) { - buffer.encode_string(9, it, true); - } -} -void ExecuteServiceArgument::calculate_size(uint32_t &total_size) const { - ProtoSize::add_bool_field(total_size, 1, this->bool_); - ProtoSize::add_int32_field(total_size, 1, this->legacy_int); - ProtoSize::add_fixed_field<4>(total_size, 1, this->float_ != 0.0f); - ProtoSize::add_string_field(total_size, 1, this->string_); - ProtoSize::add_sint32_field(total_size, 1, this->int_); - if (!this->bool_array.empty()) { - for (const auto it : this->bool_array) { - ProtoSize::add_bool_field_repeated(total_size, 1, it); - } - } - if (!this->int_array.empty()) { - for (const auto &it : this->int_array) { - ProtoSize::add_sint32_field_repeated(total_size, 1, it); - } - } - if (!this->float_array.empty()) { - total_size += this->float_array.size() * 5; - } - if (!this->string_array.empty()) { - for (const auto &it : this->string_array) { - ProtoSize::add_string_field_repeated(total_size, 1, it); - } - } +void ExecuteServiceArgument::decode(const uint8_t *buffer, size_t length) { + uint32_t count_bool_array = ProtoDecodableMessage::count_repeated_field(buffer, length, 6); + this->bool_array.init(count_bool_array); + uint32_t count_int_array = ProtoDecodableMessage::count_repeated_field(buffer, length, 7); + this->int_array.init(count_int_array); + uint32_t count_float_array = ProtoDecodableMessage::count_repeated_field(buffer, length, 8); + this->float_array.init(count_float_array); + uint32_t count_string_array = ProtoDecodableMessage::count_repeated_field(buffer, length, 9); + this->string_array.init(count_string_array); + ProtoDecodableMessage::decode(buffer, length); } bool ExecuteServiceRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 2: { + case 2: this->args.emplace_back(); value.decode_to_message(this->args.back()); - return true; - } + break; default: return false; } + return true; } bool ExecuteServiceRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } + break; default: return false; } + return true; +} +void ExecuteServiceRequest::decode(const uint8_t *buffer, size_t length) { + uint32_t count_args = ProtoDecodableMessage::count_repeated_field(buffer, length, 2); + this->args.init(count_args); + ProtoDecodableMessage::decode(buffer, length); } #endif #ifdef USE_CAMERA void ListEntitiesCameraResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); + buffer.encode_string(3, this->name_ref_); buffer.encode_bool(5, this->disabled_by_default); - buffer.encode_string(6, this->icon); +#ifdef USE_ENTITY_ICON + buffer.encode_string(6, this->icon_ref_); +#endif buffer.encode_uint32(7, static_cast(this->entity_category)); +#ifdef USE_DEVICES buffer.encode_uint32(8, this->device_id); +#endif } -void ListEntitiesCameraResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesCameraResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); + size.add_bool(1, this->disabled_by_default); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_uint32(1, static_cast(this->entity_category)); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void CameraImageResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); - buffer.encode_bytes(2, reinterpret_cast(this->data.data()), this->data.size()); + buffer.encode_bytes(2, this->data_ptr_, this->data_len_); buffer.encode_bool(3, this->done); +#ifdef USE_DEVICES buffer.encode_uint32(4, this->device_id); +#endif } -void CameraImageResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->data); - ProtoSize::add_bool_field(total_size, 1, this->done); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void CameraImageResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_length(1, this->data_len_); + size.add_bool(1, this->done); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool CameraImageRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->single = value.as_bool(); - return true; - } - case 2: { + break; + case 2: this->stream = value.as_bool(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_CLIMATE void ListEntitiesClimateResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); + buffer.encode_string(3, this->name_ref_); buffer.encode_bool(5, this->supports_current_temperature); buffer.encode_bool(6, this->supports_two_point_target_temperature); - for (auto &it : this->supported_modes) { + for (const auto &it : *this->supported_modes) { buffer.encode_uint32(7, static_cast(it), true); } buffer.encode_float(8, this->visual_min_temperature); buffer.encode_float(9, this->visual_max_temperature); buffer.encode_float(10, this->visual_target_temperature_step); - buffer.encode_bool(11, this->legacy_supports_away); buffer.encode_bool(12, this->supports_action); - for (auto &it : this->supported_fan_modes) { + for (const auto &it : *this->supported_fan_modes) { buffer.encode_uint32(13, static_cast(it), true); } - for (auto &it : this->supported_swing_modes) { + for (const auto &it : *this->supported_swing_modes) { buffer.encode_uint32(14, static_cast(it), true); } - for (auto &it : this->supported_custom_fan_modes) { + for (const auto &it : *this->supported_custom_fan_modes) { buffer.encode_string(15, it, true); } - for (auto &it : this->supported_presets) { + for (const auto &it : *this->supported_presets) { buffer.encode_uint32(16, static_cast(it), true); } - for (auto &it : this->supported_custom_presets) { + for (const auto &it : *this->supported_custom_presets) { buffer.encode_string(17, it, true); } buffer.encode_bool(18, this->disabled_by_default); - buffer.encode_string(19, this->icon); +#ifdef USE_ENTITY_ICON + buffer.encode_string(19, this->icon_ref_); +#endif buffer.encode_uint32(20, static_cast(this->entity_category)); buffer.encode_float(21, this->visual_current_temperature_step); buffer.encode_bool(22, this->supports_current_humidity); buffer.encode_bool(23, this->supports_target_humidity); buffer.encode_float(24, this->visual_min_humidity); buffer.encode_float(25, this->visual_max_humidity); +#ifdef USE_DEVICES buffer.encode_uint32(26, this->device_id); +#endif + buffer.encode_uint32(27, this->feature_flags); } -void ListEntitiesClimateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_bool_field(total_size, 1, this->supports_current_temperature); - ProtoSize::add_bool_field(total_size, 1, this->supports_two_point_target_temperature); - if (!this->supported_modes.empty()) { - for (const auto &it : this->supported_modes) { - ProtoSize::add_enum_field_repeated(total_size, 1, static_cast(it)); +void ListEntitiesClimateResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); + size.add_bool(1, this->supports_current_temperature); + size.add_bool(1, this->supports_two_point_target_temperature); + if (!this->supported_modes->empty()) { + for (const auto &it : *this->supported_modes) { + size.add_uint32_force(1, static_cast(it)); } } - ProtoSize::add_fixed_field<4>(total_size, 1, this->visual_min_temperature != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->visual_max_temperature != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->visual_target_temperature_step != 0.0f); - ProtoSize::add_bool_field(total_size, 1, this->legacy_supports_away); - ProtoSize::add_bool_field(total_size, 1, this->supports_action); - if (!this->supported_fan_modes.empty()) { - for (const auto &it : this->supported_fan_modes) { - ProtoSize::add_enum_field_repeated(total_size, 1, static_cast(it)); + size.add_float(1, this->visual_min_temperature); + size.add_float(1, this->visual_max_temperature); + size.add_float(1, this->visual_target_temperature_step); + size.add_bool(1, this->supports_action); + if (!this->supported_fan_modes->empty()) { + for (const auto &it : *this->supported_fan_modes) { + size.add_uint32_force(1, static_cast(it)); } } - if (!this->supported_swing_modes.empty()) { - for (const auto &it : this->supported_swing_modes) { - ProtoSize::add_enum_field_repeated(total_size, 1, static_cast(it)); + if (!this->supported_swing_modes->empty()) { + for (const auto &it : *this->supported_swing_modes) { + size.add_uint32_force(1, static_cast(it)); } } - if (!this->supported_custom_fan_modes.empty()) { - for (const auto &it : this->supported_custom_fan_modes) { - ProtoSize::add_string_field_repeated(total_size, 1, it); + if (!this->supported_custom_fan_modes->empty()) { + for (const auto &it : *this->supported_custom_fan_modes) { + size.add_length_force(1, it.size()); } } - if (!this->supported_presets.empty()) { - for (const auto &it : this->supported_presets) { - ProtoSize::add_enum_field_repeated(total_size, 2, static_cast(it)); + if (!this->supported_presets->empty()) { + for (const auto &it : *this->supported_presets) { + size.add_uint32_force(2, static_cast(it)); } } - if (!this->supported_custom_presets.empty()) { - for (const auto &it : this->supported_custom_presets) { - ProtoSize::add_string_field_repeated(total_size, 2, it); + if (!this->supported_custom_presets->empty()) { + for (const auto &it : *this->supported_custom_presets) { + size.add_length_force(2, it.size()); } } - ProtoSize::add_bool_field(total_size, 2, this->disabled_by_default); - ProtoSize::add_string_field(total_size, 2, this->icon); - ProtoSize::add_enum_field(total_size, 2, static_cast(this->entity_category)); - ProtoSize::add_fixed_field<4>(total_size, 2, this->visual_current_temperature_step != 0.0f); - ProtoSize::add_bool_field(total_size, 2, this->supports_current_humidity); - ProtoSize::add_bool_field(total_size, 2, this->supports_target_humidity); - ProtoSize::add_fixed_field<4>(total_size, 2, this->visual_min_humidity != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 2, this->visual_max_humidity != 0.0f); - ProtoSize::add_uint32_field(total_size, 2, this->device_id); + size.add_bool(2, this->disabled_by_default); +#ifdef USE_ENTITY_ICON + size.add_length(2, this->icon_ref_.size()); +#endif + size.add_uint32(2, static_cast(this->entity_category)); + size.add_float(2, this->visual_current_temperature_step); + size.add_bool(2, this->supports_current_humidity); + size.add_bool(2, this->supports_target_humidity); + size.add_float(2, this->visual_min_humidity); + size.add_float(2, this->visual_max_humidity); +#ifdef USE_DEVICES + size.add_uint32(2, this->device_id); +#endif + size.add_uint32(2, this->feature_flags); } void ClimateStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); @@ -1224,299 +1265,300 @@ void ClimateStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_float(4, this->target_temperature); buffer.encode_float(5, this->target_temperature_low); buffer.encode_float(6, this->target_temperature_high); - buffer.encode_bool(7, this->unused_legacy_away); buffer.encode_uint32(8, static_cast(this->action)); buffer.encode_uint32(9, static_cast(this->fan_mode)); buffer.encode_uint32(10, static_cast(this->swing_mode)); - buffer.encode_string(11, this->custom_fan_mode); + buffer.encode_string(11, this->custom_fan_mode_ref_); buffer.encode_uint32(12, static_cast(this->preset)); - buffer.encode_string(13, this->custom_preset); + buffer.encode_string(13, this->custom_preset_ref_); buffer.encode_float(14, this->current_humidity); buffer.encode_float(15, this->target_humidity); +#ifdef USE_DEVICES buffer.encode_uint32(16, this->device_id); +#endif } -void ClimateStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->mode)); - ProtoSize::add_fixed_field<4>(total_size, 1, this->current_temperature != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->target_temperature != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->target_temperature_low != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->target_temperature_high != 0.0f); - ProtoSize::add_bool_field(total_size, 1, this->unused_legacy_away); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->action)); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->fan_mode)); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->swing_mode)); - ProtoSize::add_string_field(total_size, 1, this->custom_fan_mode); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->preset)); - ProtoSize::add_string_field(total_size, 1, this->custom_preset); - ProtoSize::add_fixed_field<4>(total_size, 1, this->current_humidity != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->target_humidity != 0.0f); - ProtoSize::add_uint32_field(total_size, 2, this->device_id); +void ClimateStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_uint32(1, static_cast(this->mode)); + size.add_float(1, this->current_temperature); + size.add_float(1, this->target_temperature); + size.add_float(1, this->target_temperature_low); + size.add_float(1, this->target_temperature_high); + size.add_uint32(1, static_cast(this->action)); + size.add_uint32(1, static_cast(this->fan_mode)); + size.add_uint32(1, static_cast(this->swing_mode)); + size.add_length(1, this->custom_fan_mode_ref_.size()); + size.add_uint32(1, static_cast(this->preset)); + size.add_length(1, this->custom_preset_ref_.size()); + size.add_float(1, this->current_humidity); + size.add_float(1, this->target_humidity); +#ifdef USE_DEVICES + size.add_uint32(2, this->device_id); +#endif } bool ClimateCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->has_mode = value.as_bool(); - return true; - } - case 3: { + break; + case 3: this->mode = static_cast(value.as_uint32()); - return true; - } - case 4: { + break; + case 4: this->has_target_temperature = value.as_bool(); - return true; - } - case 6: { + break; + case 6: this->has_target_temperature_low = value.as_bool(); - return true; - } - case 8: { + break; + case 8: this->has_target_temperature_high = value.as_bool(); - return true; - } - case 10: { - this->unused_has_legacy_away = value.as_bool(); - return true; - } - case 11: { - this->unused_legacy_away = value.as_bool(); - return true; - } - case 12: { + break; + case 12: this->has_fan_mode = value.as_bool(); - return true; - } - case 13: { + break; + case 13: this->fan_mode = static_cast(value.as_uint32()); - return true; - } - case 14: { + break; + case 14: this->has_swing_mode = value.as_bool(); - return true; - } - case 15: { + break; + case 15: this->swing_mode = static_cast(value.as_uint32()); - return true; - } - case 16: { + break; + case 16: this->has_custom_fan_mode = value.as_bool(); - return true; - } - case 18: { + break; + case 18: this->has_preset = value.as_bool(); - return true; - } - case 19: { + break; + case 19: this->preset = static_cast(value.as_uint32()); - return true; - } - case 20: { + break; + case 20: this->has_custom_preset = value.as_bool(); - return true; - } - case 22: { + break; + case 22: this->has_target_humidity = value.as_bool(); - return true; - } - case 24: { + break; +#ifdef USE_DEVICES + case 24: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool ClimateCommandRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 17: { + case 17: this->custom_fan_mode = value.as_string(); - return true; - } - case 21: { + break; + case 21: this->custom_preset = value.as_string(); - return true; - } + break; default: return false; } + return true; } bool ClimateCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } - case 5: { + break; + case 5: this->target_temperature = value.as_float(); - return true; - } - case 7: { + break; + case 7: this->target_temperature_low = value.as_float(); - return true; - } - case 9: { + break; + case 9: this->target_temperature_high = value.as_float(); - return true; - } - case 23: { + break; + case 23: this->target_humidity = value.as_float(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_NUMBER void ListEntitiesNumberResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_float(6, this->min_value); buffer.encode_float(7, this->max_value); buffer.encode_float(8, this->step); buffer.encode_bool(9, this->disabled_by_default); buffer.encode_uint32(10, static_cast(this->entity_category)); - buffer.encode_string(11, this->unit_of_measurement); + buffer.encode_string(11, this->unit_of_measurement_ref_); buffer.encode_uint32(12, static_cast(this->mode)); - buffer.encode_string(13, this->device_class); + buffer.encode_string(13, this->device_class_ref_); +#ifdef USE_DEVICES buffer.encode_uint32(14, this->device_id); +#endif } -void ListEntitiesNumberResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_fixed_field<4>(total_size, 1, this->min_value != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->max_value != 0.0f); - ProtoSize::add_fixed_field<4>(total_size, 1, this->step != 0.0f); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_string_field(total_size, 1, this->unit_of_measurement); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->mode)); - ProtoSize::add_string_field(total_size, 1, this->device_class); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesNumberResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_float(1, this->min_value); + size.add_float(1, this->max_value); + size.add_float(1, this->step); + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); + size.add_length(1, this->unit_of_measurement_ref_.size()); + size.add_uint32(1, static_cast(this->mode)); + size.add_length(1, this->device_class_ref_.size()); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void NumberStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); buffer.encode_float(2, this->state); buffer.encode_bool(3, this->missing_state); +#ifdef USE_DEVICES buffer.encode_uint32(4, this->device_id); +#endif } -void NumberStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_fixed_field<4>(total_size, 1, this->state != 0.0f); - ProtoSize::add_bool_field(total_size, 1, this->missing_state); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void NumberStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_float(1, this->state); + size.add_bool(1, this->missing_state); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool NumberCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 3: { +#ifdef USE_DEVICES + case 3: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool NumberCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } - case 2: { + break; + case 2: this->state = value.as_float(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_SELECT void ListEntitiesSelectResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); - for (auto &it : this->options) { + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif + for (const auto &it : *this->options) { buffer.encode_string(6, it, true); } buffer.encode_bool(7, this->disabled_by_default); buffer.encode_uint32(8, static_cast(this->entity_category)); +#ifdef USE_DEVICES buffer.encode_uint32(9, this->device_id); +#endif } -void ListEntitiesSelectResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - if (!this->options.empty()) { - for (const auto &it : this->options) { - ProtoSize::add_string_field_repeated(total_size, 1, it); +void ListEntitiesSelectResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + if (!this->options->empty()) { + for (const auto &it : *this->options) { + size.add_length_force(1, it.size()); } } - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void SelectStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); - buffer.encode_string(2, this->state); + buffer.encode_string(2, this->state_ref_); buffer.encode_bool(3, this->missing_state); +#ifdef USE_DEVICES buffer.encode_uint32(4, this->device_id); +#endif } -void SelectStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->state); - ProtoSize::add_bool_field(total_size, 1, this->missing_state); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void SelectStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_length(1, this->state_ref_.size()); + size.add_bool(1, this->missing_state); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool SelectCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 3: { +#ifdef USE_DEVICES + case 3: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool SelectCommandRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 2: { + case 2: this->state = value.as_string(); - return true; - } + break; default: return false; } + return true; } bool SelectCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_SIREN void ListEntitiesSirenResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->disabled_by_default); for (auto &it : this->tones) { buffer.encode_string(7, it, true); @@ -1524,510 +1566,410 @@ void ListEntitiesSirenResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_bool(8, this->supports_duration); buffer.encode_bool(9, this->supports_volume); buffer.encode_uint32(10, static_cast(this->entity_category)); +#ifdef USE_DEVICES buffer.encode_uint32(11, this->device_id); +#endif } -void ListEntitiesSirenResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); +void ListEntitiesSirenResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->disabled_by_default); if (!this->tones.empty()) { for (const auto &it : this->tones) { - ProtoSize::add_string_field_repeated(total_size, 1, it); + size.add_length_force(1, it.size()); } } - ProtoSize::add_bool_field(total_size, 1, this->supports_duration); - ProtoSize::add_bool_field(total_size, 1, this->supports_volume); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); + size.add_bool(1, this->supports_duration); + size.add_bool(1, this->supports_volume); + size.add_uint32(1, static_cast(this->entity_category)); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void SirenStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); buffer.encode_bool(2, this->state); +#ifdef USE_DEVICES buffer.encode_uint32(3, this->device_id); +#endif } -void SirenStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_bool_field(total_size, 1, this->state); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void SirenStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_bool(1, this->state); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool SirenCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->has_state = value.as_bool(); - return true; - } - case 3: { + break; + case 3: this->state = value.as_bool(); - return true; - } - case 4: { + break; + case 4: this->has_tone = value.as_bool(); - return true; - } - case 6: { + break; + case 6: this->has_duration = value.as_bool(); - return true; - } - case 7: { + break; + case 7: this->duration = value.as_uint32(); - return true; - } - case 8: { + break; + case 8: this->has_volume = value.as_bool(); - return true; - } - case 10: { + break; +#ifdef USE_DEVICES + case 10: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool SirenCommandRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 5: { + case 5: this->tone = value.as_string(); - return true; - } + break; default: return false; } + return true; } bool SirenCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } - case 9: { + break; + case 9: this->volume = value.as_float(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_LOCK void ListEntitiesLockResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->disabled_by_default); buffer.encode_uint32(7, static_cast(this->entity_category)); buffer.encode_bool(8, this->assumed_state); buffer.encode_bool(9, this->supports_open); buffer.encode_bool(10, this->requires_code); - buffer.encode_string(11, this->code_format); + buffer.encode_string(11, this->code_format_ref_); +#ifdef USE_DEVICES buffer.encode_uint32(12, this->device_id); +#endif } -void ListEntitiesLockResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_bool_field(total_size, 1, this->assumed_state); - ProtoSize::add_bool_field(total_size, 1, this->supports_open); - ProtoSize::add_bool_field(total_size, 1, this->requires_code); - ProtoSize::add_string_field(total_size, 1, this->code_format); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesLockResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); + size.add_bool(1, this->assumed_state); + size.add_bool(1, this->supports_open); + size.add_bool(1, this->requires_code); + size.add_length(1, this->code_format_ref_.size()); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void LockStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); buffer.encode_uint32(2, static_cast(this->state)); +#ifdef USE_DEVICES buffer.encode_uint32(3, this->device_id); +#endif } -void LockStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->state)); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void LockStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_uint32(1, static_cast(this->state)); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool LockCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->command = static_cast(value.as_uint32()); - return true; - } - case 3: { + break; + case 3: this->has_code = value.as_bool(); - return true; - } - case 5: { + break; +#ifdef USE_DEVICES + case 5: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool LockCommandRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 4: { + case 4: this->code = value.as_string(); - return true; - } + break; default: return false; } + return true; } bool LockCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_BUTTON void ListEntitiesButtonResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->disabled_by_default); buffer.encode_uint32(7, static_cast(this->entity_category)); - buffer.encode_string(8, this->device_class); + buffer.encode_string(8, this->device_class_ref_); +#ifdef USE_DEVICES buffer.encode_uint32(9, this->device_id); +#endif } -void ListEntitiesButtonResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_string_field(total_size, 1, this->device_class); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesButtonResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); + size.add_length(1, this->device_class_ref_.size()); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool ButtonCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { +#ifdef USE_DEVICES + case 2: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool ButtonCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_MEDIA_PLAYER -bool MediaPlayerSupportedFormat::decode_varint(uint32_t field_id, ProtoVarInt value) { - switch (field_id) { - case 2: { - this->sample_rate = value.as_uint32(); - return true; - } - case 3: { - this->num_channels = value.as_uint32(); - return true; - } - case 4: { - this->purpose = static_cast(value.as_uint32()); - return true; - } - case 5: { - this->sample_bytes = value.as_uint32(); - return true; - } - default: - return false; - } -} -bool MediaPlayerSupportedFormat::decode_length(uint32_t field_id, ProtoLengthDelimited value) { - switch (field_id) { - case 1: { - this->format = value.as_string(); - return true; - } - default: - return false; - } -} void MediaPlayerSupportedFormat::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->format); + buffer.encode_string(1, this->format_ref_); buffer.encode_uint32(2, this->sample_rate); buffer.encode_uint32(3, this->num_channels); buffer.encode_uint32(4, static_cast(this->purpose)); buffer.encode_uint32(5, this->sample_bytes); } -void MediaPlayerSupportedFormat::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->format); - ProtoSize::add_uint32_field(total_size, 1, this->sample_rate); - ProtoSize::add_uint32_field(total_size, 1, this->num_channels); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->purpose)); - ProtoSize::add_uint32_field(total_size, 1, this->sample_bytes); +void MediaPlayerSupportedFormat::calculate_size(ProtoSize &size) const { + size.add_length(1, this->format_ref_.size()); + size.add_uint32(1, this->sample_rate); + size.add_uint32(1, this->num_channels); + size.add_uint32(1, static_cast(this->purpose)); + size.add_uint32(1, this->sample_bytes); } void ListEntitiesMediaPlayerResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->disabled_by_default); buffer.encode_uint32(7, static_cast(this->entity_category)); buffer.encode_bool(8, this->supports_pause); for (auto &it : this->supported_formats) { buffer.encode_message(9, it, true); } +#ifdef USE_DEVICES buffer.encode_uint32(10, this->device_id); +#endif + buffer.encode_uint32(11, this->feature_flags); } -void ListEntitiesMediaPlayerResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_bool_field(total_size, 1, this->supports_pause); - ProtoSize::add_repeated_message(total_size, 1, this->supported_formats); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesMediaPlayerResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); + size.add_bool(1, this->supports_pause); + size.add_repeated_message(1, this->supported_formats); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif + size.add_uint32(1, this->feature_flags); } void MediaPlayerStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); buffer.encode_uint32(2, static_cast(this->state)); buffer.encode_float(3, this->volume); buffer.encode_bool(4, this->muted); +#ifdef USE_DEVICES buffer.encode_uint32(5, this->device_id); +#endif } -void MediaPlayerStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->state)); - ProtoSize::add_fixed_field<4>(total_size, 1, this->volume != 0.0f); - ProtoSize::add_bool_field(total_size, 1, this->muted); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void MediaPlayerStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_uint32(1, static_cast(this->state)); + size.add_float(1, this->volume); + size.add_bool(1, this->muted); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool MediaPlayerCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->has_command = value.as_bool(); - return true; - } - case 3: { + break; + case 3: this->command = static_cast(value.as_uint32()); - return true; - } - case 4: { + break; + case 4: this->has_volume = value.as_bool(); - return true; - } - case 6: { + break; + case 6: this->has_media_url = value.as_bool(); - return true; - } - case 8: { + break; + case 8: this->has_announcement = value.as_bool(); - return true; - } - case 9: { + break; + case 9: this->announcement = value.as_bool(); - return true; - } - case 10: { + break; +#ifdef USE_DEVICES + case 10: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool MediaPlayerCommandRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 7: { + case 7: this->media_url = value.as_string(); - return true; - } + break; default: return false; } + return true; } bool MediaPlayerCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } - case 5: { + break; + case 5: this->volume = value.as_float(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_BLUETOOTH_PROXY bool SubscribeBluetoothLEAdvertisementsRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->flags = value.as_uint32(); - return true; - } - default: - return false; - } -} -bool BluetoothServiceData::decode_varint(uint32_t field_id, ProtoVarInt value) { - switch (field_id) { - case 2: { - this->legacy_data.push_back(value.as_uint32()); - return true; - } - default: - return false; - } -} -bool BluetoothServiceData::decode_length(uint32_t field_id, ProtoLengthDelimited value) { - switch (field_id) { - case 1: { - this->uuid = value.as_string(); - return true; - } - case 3: { - this->data = value.as_string(); - return true; - } - default: - return false; - } -} -void BluetoothServiceData::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->uuid); - for (auto &it : this->legacy_data) { - buffer.encode_uint32(2, it, true); - } - buffer.encode_bytes(3, reinterpret_cast(this->data.data()), this->data.size()); -} -void BluetoothServiceData::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->uuid); - if (!this->legacy_data.empty()) { - for (const auto &it : this->legacy_data) { - ProtoSize::add_uint32_field_repeated(total_size, 1, it); - } - } - ProtoSize::add_string_field(total_size, 1, this->data); -} -void BluetoothLEAdvertisementResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_uint64(1, this->address); - buffer.encode_bytes(2, reinterpret_cast(this->name.data()), this->name.size()); - buffer.encode_sint32(3, this->rssi); - for (auto &it : this->service_uuids) { - buffer.encode_string(4, it, true); - } - for (auto &it : this->service_data) { - buffer.encode_message(5, it, true); - } - for (auto &it : this->manufacturer_data) { - buffer.encode_message(6, it, true); - } - buffer.encode_uint32(7, this->address_type); -} -void BluetoothLEAdvertisementResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint64_field(total_size, 1, this->address); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_sint32_field(total_size, 1, this->rssi); - if (!this->service_uuids.empty()) { - for (const auto &it : this->service_uuids) { - ProtoSize::add_string_field_repeated(total_size, 1, it); - } - } - ProtoSize::add_repeated_message(total_size, 1, this->service_data); - ProtoSize::add_repeated_message(total_size, 1, this->manufacturer_data); - ProtoSize::add_uint32_field(total_size, 1, this->address_type); -} -bool BluetoothLERawAdvertisement::decode_varint(uint32_t field_id, ProtoVarInt value) { - switch (field_id) { - case 1: { - this->address = value.as_uint64(); - return true; - } - case 2: { - this->rssi = value.as_sint32(); - return true; - } - case 3: { - this->address_type = value.as_uint32(); - return true; - } - default: - return false; - } -} -bool BluetoothLERawAdvertisement::decode_length(uint32_t field_id, ProtoLengthDelimited value) { - switch (field_id) { - case 4: { - this->data = value.as_string(); - return true; - } + break; default: return false; } + return true; } void BluetoothLERawAdvertisement::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint64(1, this->address); buffer.encode_sint32(2, this->rssi); buffer.encode_uint32(3, this->address_type); - buffer.encode_bytes(4, reinterpret_cast(this->data.data()), this->data.size()); + buffer.encode_bytes(4, this->data, this->data_len); } -void BluetoothLERawAdvertisement::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint64_field(total_size, 1, this->address); - ProtoSize::add_sint32_field(total_size, 1, this->rssi); - ProtoSize::add_uint32_field(total_size, 1, this->address_type); - ProtoSize::add_string_field(total_size, 1, this->data); +void BluetoothLERawAdvertisement::calculate_size(ProtoSize &size) const { + size.add_uint64(1, this->address); + size.add_sint32(1, this->rssi); + size.add_uint32(1, this->address_type); + size.add_length(1, this->data_len); } void BluetoothLERawAdvertisementsResponse::encode(ProtoWriteBuffer buffer) const { - for (auto &it : this->advertisements) { - buffer.encode_message(1, it, true); + for (uint16_t i = 0; i < this->advertisements_len; i++) { + buffer.encode_message(1, this->advertisements[i], true); } } -void BluetoothLERawAdvertisementsResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_repeated_message(total_size, 1, this->advertisements); +void BluetoothLERawAdvertisementsResponse::calculate_size(ProtoSize &size) const { + for (uint16_t i = 0; i < this->advertisements_len; i++) { + size.add_message_object_force(1, this->advertisements[i]); + } } bool BluetoothDeviceRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->address = value.as_uint64(); - return true; - } - case 2: { + break; + case 2: this->request_type = static_cast(value.as_uint32()); - return true; - } - case 3: { + break; + case 3: this->has_address_type = value.as_bool(); - return true; - } - case 4: { + break; + case 4: this->address_type = value.as_uint32(); - return true; - } + break; default: return false; } + return true; } void BluetoothDeviceConnectionResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint64(1, this->address); @@ -2035,141 +1977,79 @@ void BluetoothDeviceConnectionResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint32(3, this->mtu); buffer.encode_int32(4, this->error); } -void BluetoothDeviceConnectionResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint64_field(total_size, 1, this->address); - ProtoSize::add_bool_field(total_size, 1, this->connected); - ProtoSize::add_uint32_field(total_size, 1, this->mtu); - ProtoSize::add_int32_field(total_size, 1, this->error); +void BluetoothDeviceConnectionResponse::calculate_size(ProtoSize &size) const { + size.add_uint64(1, this->address); + size.add_bool(1, this->connected); + size.add_uint32(1, this->mtu); + size.add_int32(1, this->error); } bool BluetoothGATTGetServicesRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->address = value.as_uint64(); - return true; - } - default: - return false; - } -} -bool BluetoothGATTDescriptor::decode_varint(uint32_t field_id, ProtoVarInt value) { - switch (field_id) { - case 1: { - this->uuid.push_back(value.as_uint64()); - return true; - } - case 2: { - this->handle = value.as_uint32(); - return true; - } + break; default: return false; } + return true; } void BluetoothGATTDescriptor::encode(ProtoWriteBuffer buffer) const { - for (auto &it : this->uuid) { - buffer.encode_uint64(1, it, true); + if (this->uuid[0] != 0 || this->uuid[1] != 0) { + buffer.encode_uint64(1, this->uuid[0], true); + buffer.encode_uint64(1, this->uuid[1], true); } buffer.encode_uint32(2, this->handle); + buffer.encode_uint32(3, this->short_uuid); } -void BluetoothGATTDescriptor::calculate_size(uint32_t &total_size) const { - if (!this->uuid.empty()) { - for (const auto &it : this->uuid) { - ProtoSize::add_uint64_field_repeated(total_size, 1, it); - } - } - ProtoSize::add_uint32_field(total_size, 1, this->handle); -} -bool BluetoothGATTCharacteristic::decode_varint(uint32_t field_id, ProtoVarInt value) { - switch (field_id) { - case 1: { - this->uuid.push_back(value.as_uint64()); - return true; - } - case 2: { - this->handle = value.as_uint32(); - return true; - } - case 3: { - this->properties = value.as_uint32(); - return true; - } - default: - return false; - } -} -bool BluetoothGATTCharacteristic::decode_length(uint32_t field_id, ProtoLengthDelimited value) { - switch (field_id) { - case 4: { - this->descriptors.emplace_back(); - value.decode_to_message(this->descriptors.back()); - return true; - } - default: - return false; +void BluetoothGATTDescriptor::calculate_size(ProtoSize &size) const { + if (this->uuid[0] != 0 || this->uuid[1] != 0) { + size.add_uint64_force(1, this->uuid[0]); + size.add_uint64_force(1, this->uuid[1]); } + size.add_uint32(1, this->handle); + size.add_uint32(1, this->short_uuid); } void BluetoothGATTCharacteristic::encode(ProtoWriteBuffer buffer) const { - for (auto &it : this->uuid) { - buffer.encode_uint64(1, it, true); + if (this->uuid[0] != 0 || this->uuid[1] != 0) { + buffer.encode_uint64(1, this->uuid[0], true); + buffer.encode_uint64(1, this->uuid[1], true); } buffer.encode_uint32(2, this->handle); buffer.encode_uint32(3, this->properties); for (auto &it : this->descriptors) { buffer.encode_message(4, it, true); } + buffer.encode_uint32(5, this->short_uuid); } -void BluetoothGATTCharacteristic::calculate_size(uint32_t &total_size) const { - if (!this->uuid.empty()) { - for (const auto &it : this->uuid) { - ProtoSize::add_uint64_field_repeated(total_size, 1, it); - } - } - ProtoSize::add_uint32_field(total_size, 1, this->handle); - ProtoSize::add_uint32_field(total_size, 1, this->properties); - ProtoSize::add_repeated_message(total_size, 1, this->descriptors); -} -bool BluetoothGATTService::decode_varint(uint32_t field_id, ProtoVarInt value) { - switch (field_id) { - case 1: { - this->uuid.push_back(value.as_uint64()); - return true; - } - case 2: { - this->handle = value.as_uint32(); - return true; - } - default: - return false; - } -} -bool BluetoothGATTService::decode_length(uint32_t field_id, ProtoLengthDelimited value) { - switch (field_id) { - case 3: { - this->characteristics.emplace_back(); - value.decode_to_message(this->characteristics.back()); - return true; - } - default: - return false; +void BluetoothGATTCharacteristic::calculate_size(ProtoSize &size) const { + if (this->uuid[0] != 0 || this->uuid[1] != 0) { + size.add_uint64_force(1, this->uuid[0]); + size.add_uint64_force(1, this->uuid[1]); } + size.add_uint32(1, this->handle); + size.add_uint32(1, this->properties); + size.add_repeated_message(1, this->descriptors); + size.add_uint32(1, this->short_uuid); } void BluetoothGATTService::encode(ProtoWriteBuffer buffer) const { - for (auto &it : this->uuid) { - buffer.encode_uint64(1, it, true); + if (this->uuid[0] != 0 || this->uuid[1] != 0) { + buffer.encode_uint64(1, this->uuid[0], true); + buffer.encode_uint64(1, this->uuid[1], true); } buffer.encode_uint32(2, this->handle); for (auto &it : this->characteristics) { buffer.encode_message(3, it, true); } + buffer.encode_uint32(4, this->short_uuid); } -void BluetoothGATTService::calculate_size(uint32_t &total_size) const { - if (!this->uuid.empty()) { - for (const auto &it : this->uuid) { - ProtoSize::add_uint64_field_repeated(total_size, 1, it); - } +void BluetoothGATTService::calculate_size(ProtoSize &size) const { + if (this->uuid[0] != 0 || this->uuid[1] != 0) { + size.add_uint64_force(1, this->uuid[0]); + size.add_uint64_force(1, this->uuid[1]); } - ProtoSize::add_uint32_field(total_size, 1, this->handle); - ProtoSize::add_repeated_message(total_size, 1, this->characteristics); + size.add_uint32(1, this->handle); + size.add_repeated_message(1, this->characteristics); + size.add_uint32(1, this->short_uuid); } void BluetoothGATTGetServicesResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint64(1, this->address); @@ -2177,147 +2057,146 @@ void BluetoothGATTGetServicesResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_message(2, it, true); } } -void BluetoothGATTGetServicesResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint64_field(total_size, 1, this->address); - ProtoSize::add_repeated_message(total_size, 1, this->services); +void BluetoothGATTGetServicesResponse::calculate_size(ProtoSize &size) const { + size.add_uint64(1, this->address); + size.add_repeated_message(1, this->services); } void BluetoothGATTGetServicesDoneResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint64(1, this->address); } -void BluetoothGATTGetServicesDoneResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint64_field(total_size, 1, this->address); -} +void BluetoothGATTGetServicesDoneResponse::calculate_size(ProtoSize &size) const { size.add_uint64(1, this->address); } bool BluetoothGATTReadRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->address = value.as_uint64(); - return true; - } - case 2: { + break; + case 2: this->handle = value.as_uint32(); - return true; - } + break; default: return false; } + return true; } void BluetoothGATTReadResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint64(1, this->address); buffer.encode_uint32(2, this->handle); - buffer.encode_bytes(3, reinterpret_cast(this->data.data()), this->data.size()); + buffer.encode_bytes(3, this->data_ptr_, this->data_len_); } -void BluetoothGATTReadResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint64_field(total_size, 1, this->address); - ProtoSize::add_uint32_field(total_size, 1, this->handle); - ProtoSize::add_string_field(total_size, 1, this->data); +void BluetoothGATTReadResponse::calculate_size(ProtoSize &size) const { + size.add_uint64(1, this->address); + size.add_uint32(1, this->handle); + size.add_length(1, this->data_len_); } bool BluetoothGATTWriteRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->address = value.as_uint64(); - return true; - } - case 2: { + break; + case 2: this->handle = value.as_uint32(); - return true; - } - case 3: { + break; + case 3: this->response = value.as_bool(); - return true; - } + break; default: return false; } + return true; } bool BluetoothGATTWriteRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { case 4: { - this->data = value.as_string(); - return true; + // Use raw data directly to avoid allocation + this->data = value.data(); + this->data_len = value.size(); + break; } default: return false; } + return true; } bool BluetoothGATTReadDescriptorRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->address = value.as_uint64(); - return true; - } - case 2: { + break; + case 2: this->handle = value.as_uint32(); - return true; - } + break; default: return false; } + return true; } bool BluetoothGATTWriteDescriptorRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->address = value.as_uint64(); - return true; - } - case 2: { + break; + case 2: this->handle = value.as_uint32(); - return true; - } + break; default: return false; } + return true; } bool BluetoothGATTWriteDescriptorRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { case 3: { - this->data = value.as_string(); - return true; + // Use raw data directly to avoid allocation + this->data = value.data(); + this->data_len = value.size(); + break; } default: return false; } + return true; } bool BluetoothGATTNotifyRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->address = value.as_uint64(); - return true; - } - case 2: { + break; + case 2: this->handle = value.as_uint32(); - return true; - } - case 3: { + break; + case 3: this->enable = value.as_bool(); - return true; - } + break; default: return false; } + return true; } void BluetoothGATTNotifyDataResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint64(1, this->address); buffer.encode_uint32(2, this->handle); - buffer.encode_bytes(3, reinterpret_cast(this->data.data()), this->data.size()); + buffer.encode_bytes(3, this->data_ptr_, this->data_len_); } -void BluetoothGATTNotifyDataResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint64_field(total_size, 1, this->address); - ProtoSize::add_uint32_field(total_size, 1, this->handle); - ProtoSize::add_string_field(total_size, 1, this->data); +void BluetoothGATTNotifyDataResponse::calculate_size(ProtoSize &size) const { + size.add_uint64(1, this->address); + size.add_uint32(1, this->handle); + size.add_length(1, this->data_len_); } void BluetoothConnectionsFreeResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint32(1, this->free); buffer.encode_uint32(2, this->limit); - for (auto &it : this->allocated) { - buffer.encode_uint64(3, it, true); + for (const auto &it : this->allocated) { + if (it != 0) { + buffer.encode_uint64(3, it, true); + } } } -void BluetoothConnectionsFreeResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint32_field(total_size, 1, this->free); - ProtoSize::add_uint32_field(total_size, 1, this->limit); - if (!this->allocated.empty()) { - for (const auto &it : this->allocated) { - ProtoSize::add_uint64_field_repeated(total_size, 1, it); +void BluetoothConnectionsFreeResponse::calculate_size(ProtoSize &size) const { + size.add_uint32(1, this->free); + size.add_uint32(1, this->limit); + for (const auto &it : this->allocated) { + if (it != 0) { + size.add_uint64_force(1, it); } } } @@ -2326,519 +2205,535 @@ void BluetoothGATTErrorResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint32(2, this->handle); buffer.encode_int32(3, this->error); } -void BluetoothGATTErrorResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint64_field(total_size, 1, this->address); - ProtoSize::add_uint32_field(total_size, 1, this->handle); - ProtoSize::add_int32_field(total_size, 1, this->error); +void BluetoothGATTErrorResponse::calculate_size(ProtoSize &size) const { + size.add_uint64(1, this->address); + size.add_uint32(1, this->handle); + size.add_int32(1, this->error); } void BluetoothGATTWriteResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint64(1, this->address); buffer.encode_uint32(2, this->handle); } -void BluetoothGATTWriteResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint64_field(total_size, 1, this->address); - ProtoSize::add_uint32_field(total_size, 1, this->handle); +void BluetoothGATTWriteResponse::calculate_size(ProtoSize &size) const { + size.add_uint64(1, this->address); + size.add_uint32(1, this->handle); } void BluetoothGATTNotifyResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint64(1, this->address); buffer.encode_uint32(2, this->handle); } -void BluetoothGATTNotifyResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint64_field(total_size, 1, this->address); - ProtoSize::add_uint32_field(total_size, 1, this->handle); +void BluetoothGATTNotifyResponse::calculate_size(ProtoSize &size) const { + size.add_uint64(1, this->address); + size.add_uint32(1, this->handle); } void BluetoothDevicePairingResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint64(1, this->address); buffer.encode_bool(2, this->paired); buffer.encode_int32(3, this->error); } -void BluetoothDevicePairingResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint64_field(total_size, 1, this->address); - ProtoSize::add_bool_field(total_size, 1, this->paired); - ProtoSize::add_int32_field(total_size, 1, this->error); +void BluetoothDevicePairingResponse::calculate_size(ProtoSize &size) const { + size.add_uint64(1, this->address); + size.add_bool(1, this->paired); + size.add_int32(1, this->error); } void BluetoothDeviceUnpairingResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint64(1, this->address); buffer.encode_bool(2, this->success); buffer.encode_int32(3, this->error); } -void BluetoothDeviceUnpairingResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint64_field(total_size, 1, this->address); - ProtoSize::add_bool_field(total_size, 1, this->success); - ProtoSize::add_int32_field(total_size, 1, this->error); +void BluetoothDeviceUnpairingResponse::calculate_size(ProtoSize &size) const { + size.add_uint64(1, this->address); + size.add_bool(1, this->success); + size.add_int32(1, this->error); } void BluetoothDeviceClearCacheResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint64(1, this->address); buffer.encode_bool(2, this->success); buffer.encode_int32(3, this->error); } -void BluetoothDeviceClearCacheResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint64_field(total_size, 1, this->address); - ProtoSize::add_bool_field(total_size, 1, this->success); - ProtoSize::add_int32_field(total_size, 1, this->error); +void BluetoothDeviceClearCacheResponse::calculate_size(ProtoSize &size) const { + size.add_uint64(1, this->address); + size.add_bool(1, this->success); + size.add_int32(1, this->error); } void BluetoothScannerStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint32(1, static_cast(this->state)); buffer.encode_uint32(2, static_cast(this->mode)); + buffer.encode_uint32(3, static_cast(this->configured_mode)); } -void BluetoothScannerStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_enum_field(total_size, 1, static_cast(this->state)); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->mode)); +void BluetoothScannerStateResponse::calculate_size(ProtoSize &size) const { + size.add_uint32(1, static_cast(this->state)); + size.add_uint32(1, static_cast(this->mode)); + size.add_uint32(1, static_cast(this->configured_mode)); } bool BluetoothScannerSetModeRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->mode = static_cast(value.as_uint32()); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_VOICE_ASSISTANT bool SubscribeVoiceAssistantRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->subscribe = value.as_bool(); - return true; - } - case 2: { + break; + case 2: this->flags = value.as_uint32(); - return true; - } - default: - return false; - } -} -bool VoiceAssistantAudioSettings::decode_varint(uint32_t field_id, ProtoVarInt value) { - switch (field_id) { - case 1: { - this->noise_suppression_level = value.as_uint32(); - return true; - } - case 2: { - this->auto_gain = value.as_uint32(); - return true; - } - default: - return false; - } -} -bool VoiceAssistantAudioSettings::decode_32bit(uint32_t field_id, Proto32Bit value) { - switch (field_id) { - case 3: { - this->volume_multiplier = value.as_float(); - return true; - } + break; default: return false; } + return true; } void VoiceAssistantAudioSettings::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint32(1, this->noise_suppression_level); buffer.encode_uint32(2, this->auto_gain); buffer.encode_float(3, this->volume_multiplier); } -void VoiceAssistantAudioSettings::calculate_size(uint32_t &total_size) const { - ProtoSize::add_uint32_field(total_size, 1, this->noise_suppression_level); - ProtoSize::add_uint32_field(total_size, 1, this->auto_gain); - ProtoSize::add_fixed_field<4>(total_size, 1, this->volume_multiplier != 0.0f); +void VoiceAssistantAudioSettings::calculate_size(ProtoSize &size) const { + size.add_uint32(1, this->noise_suppression_level); + size.add_uint32(1, this->auto_gain); + size.add_float(1, this->volume_multiplier); } void VoiceAssistantRequest::encode(ProtoWriteBuffer buffer) const { buffer.encode_bool(1, this->start); - buffer.encode_string(2, this->conversation_id); + buffer.encode_string(2, this->conversation_id_ref_); buffer.encode_uint32(3, this->flags); buffer.encode_message(4, this->audio_settings); - buffer.encode_string(5, this->wake_word_phrase); + buffer.encode_string(5, this->wake_word_phrase_ref_); } -void VoiceAssistantRequest::calculate_size(uint32_t &total_size) const { - ProtoSize::add_bool_field(total_size, 1, this->start); - ProtoSize::add_string_field(total_size, 1, this->conversation_id); - ProtoSize::add_uint32_field(total_size, 1, this->flags); - ProtoSize::add_message_object(total_size, 1, this->audio_settings); - ProtoSize::add_string_field(total_size, 1, this->wake_word_phrase); +void VoiceAssistantRequest::calculate_size(ProtoSize &size) const { + size.add_bool(1, this->start); + size.add_length(1, this->conversation_id_ref_.size()); + size.add_uint32(1, this->flags); + size.add_message_object(1, this->audio_settings); + size.add_length(1, this->wake_word_phrase_ref_.size()); } bool VoiceAssistantResponse::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->port = value.as_uint32(); - return true; - } - case 2: { + break; + case 2: this->error = value.as_bool(); - return true; - } + break; default: return false; } + return true; } bool VoiceAssistantEventData::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 1: { + case 1: this->name = value.as_string(); - return true; - } - case 2: { + break; + case 2: this->value = value.as_string(); - return true; - } + break; default: return false; } -} -void VoiceAssistantEventData::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->name); - buffer.encode_string(2, this->value); -} -void VoiceAssistantEventData::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->value); + return true; } bool VoiceAssistantEventResponse::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->event_type = static_cast(value.as_uint32()); - return true; - } + break; default: return false; } + return true; } bool VoiceAssistantEventResponse::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 2: { + case 2: this->data.emplace_back(); value.decode_to_message(this->data.back()); - return true; - } + break; default: return false; } + return true; } bool VoiceAssistantAudio::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->end = value.as_bool(); - return true; - } + break; default: return false; } + return true; } bool VoiceAssistantAudio::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 1: { + case 1: this->data = value.as_string(); - return true; - } + break; default: return false; } + return true; } void VoiceAssistantAudio::encode(ProtoWriteBuffer buffer) const { - buffer.encode_bytes(1, reinterpret_cast(this->data.data()), this->data.size()); + buffer.encode_bytes(1, this->data_ptr_, this->data_len_); buffer.encode_bool(2, this->end); } -void VoiceAssistantAudio::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->data); - ProtoSize::add_bool_field(total_size, 1, this->end); +void VoiceAssistantAudio::calculate_size(ProtoSize &size) const { + size.add_length(1, this->data_len_); + size.add_bool(1, this->end); } bool VoiceAssistantTimerEventResponse::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 1: { + case 1: this->event_type = static_cast(value.as_uint32()); - return true; - } - case 4: { + break; + case 4: this->total_seconds = value.as_uint32(); - return true; - } - case 5: { + break; + case 5: this->seconds_left = value.as_uint32(); - return true; - } - case 6: { + break; + case 6: this->is_active = value.as_bool(); - return true; - } + break; default: return false; } + return true; } bool VoiceAssistantTimerEventResponse::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 2: { + case 2: this->timer_id = value.as_string(); - return true; - } - case 3: { + break; + case 3: this->name = value.as_string(); - return true; - } + break; default: return false; } + return true; } bool VoiceAssistantAnnounceRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 4: { + case 4: this->start_conversation = value.as_bool(); - return true; - } + break; default: return false; } + return true; } bool VoiceAssistantAnnounceRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 1: { + case 1: this->media_id = value.as_string(); - return true; - } - case 2: { + break; + case 2: this->text = value.as_string(); - return true; - } - case 3: { + break; + case 3: this->preannounce_media_id = value.as_string(); - return true; - } + break; default: return false; } + return true; } void VoiceAssistantAnnounceFinished::encode(ProtoWriteBuffer buffer) const { buffer.encode_bool(1, this->success); } -void VoiceAssistantAnnounceFinished::calculate_size(uint32_t &total_size) const { - ProtoSize::add_bool_field(total_size, 1, this->success); -} -bool VoiceAssistantWakeWord::decode_length(uint32_t field_id, ProtoLengthDelimited value) { - switch (field_id) { - case 1: { - this->id = value.as_string(); - return true; - } - case 2: { - this->wake_word = value.as_string(); - return true; - } - case 3: { - this->trained_languages.push_back(value.as_string()); - return true; - } - default: - return false; - } -} +void VoiceAssistantAnnounceFinished::calculate_size(ProtoSize &size) const { size.add_bool(1, this->success); } void VoiceAssistantWakeWord::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->id); - buffer.encode_string(2, this->wake_word); + buffer.encode_string(1, this->id_ref_); + buffer.encode_string(2, this->wake_word_ref_); for (auto &it : this->trained_languages) { buffer.encode_string(3, it, true); } } -void VoiceAssistantWakeWord::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->id); - ProtoSize::add_string_field(total_size, 1, this->wake_word); +void VoiceAssistantWakeWord::calculate_size(ProtoSize &size) const { + size.add_length(1, this->id_ref_.size()); + size.add_length(1, this->wake_word_ref_.size()); if (!this->trained_languages.empty()) { for (const auto &it : this->trained_languages) { - ProtoSize::add_string_field_repeated(total_size, 1, it); + size.add_length_force(1, it.size()); } } } +bool VoiceAssistantExternalWakeWord::decode_varint(uint32_t field_id, ProtoVarInt value) { + switch (field_id) { + case 5: + this->model_size = value.as_uint32(); + break; + default: + return false; + } + return true; +} +bool VoiceAssistantExternalWakeWord::decode_length(uint32_t field_id, ProtoLengthDelimited value) { + switch (field_id) { + case 1: + this->id = value.as_string(); + break; + case 2: + this->wake_word = value.as_string(); + break; + case 3: + this->trained_languages.push_back(value.as_string()); + break; + case 4: + this->model_type = value.as_string(); + break; + case 6: + this->model_hash = value.as_string(); + break; + case 7: + this->url = value.as_string(); + break; + default: + return false; + } + return true; +} +bool VoiceAssistantConfigurationRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { + switch (field_id) { + case 1: + this->external_wake_words.emplace_back(); + value.decode_to_message(this->external_wake_words.back()); + break; + default: + return false; + } + return true; +} void VoiceAssistantConfigurationResponse::encode(ProtoWriteBuffer buffer) const { for (auto &it : this->available_wake_words) { buffer.encode_message(1, it, true); } - for (auto &it : this->active_wake_words) { + for (const auto &it : *this->active_wake_words) { buffer.encode_string(2, it, true); } buffer.encode_uint32(3, this->max_active_wake_words); } -void VoiceAssistantConfigurationResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_repeated_message(total_size, 1, this->available_wake_words); - if (!this->active_wake_words.empty()) { - for (const auto &it : this->active_wake_words) { - ProtoSize::add_string_field_repeated(total_size, 1, it); +void VoiceAssistantConfigurationResponse::calculate_size(ProtoSize &size) const { + size.add_repeated_message(1, this->available_wake_words); + if (!this->active_wake_words->empty()) { + for (const auto &it : *this->active_wake_words) { + size.add_length_force(1, it.size()); } } - ProtoSize::add_uint32_field(total_size, 1, this->max_active_wake_words); + size.add_uint32(1, this->max_active_wake_words); } bool VoiceAssistantSetConfiguration::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 1: { + case 1: this->active_wake_words.push_back(value.as_string()); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_ALARM_CONTROL_PANEL void ListEntitiesAlarmControlPanelResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->disabled_by_default); buffer.encode_uint32(7, static_cast(this->entity_category)); buffer.encode_uint32(8, this->supported_features); buffer.encode_bool(9, this->requires_code); buffer.encode_bool(10, this->requires_code_to_arm); +#ifdef USE_DEVICES buffer.encode_uint32(11, this->device_id); +#endif } -void ListEntitiesAlarmControlPanelResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_uint32_field(total_size, 1, this->supported_features); - ProtoSize::add_bool_field(total_size, 1, this->requires_code); - ProtoSize::add_bool_field(total_size, 1, this->requires_code_to_arm); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesAlarmControlPanelResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); + size.add_uint32(1, this->supported_features); + size.add_bool(1, this->requires_code); + size.add_bool(1, this->requires_code_to_arm); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void AlarmControlPanelStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); buffer.encode_uint32(2, static_cast(this->state)); +#ifdef USE_DEVICES buffer.encode_uint32(3, this->device_id); +#endif } -void AlarmControlPanelStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->state)); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void AlarmControlPanelStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_uint32(1, static_cast(this->state)); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool AlarmControlPanelCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->command = static_cast(value.as_uint32()); - return true; - } - case 4: { + break; +#ifdef USE_DEVICES + case 4: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool AlarmControlPanelCommandRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 3: { + case 3: this->code = value.as_string(); - return true; - } + break; default: return false; } + return true; } bool AlarmControlPanelCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_TEXT void ListEntitiesTextResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->disabled_by_default); buffer.encode_uint32(7, static_cast(this->entity_category)); buffer.encode_uint32(8, this->min_length); buffer.encode_uint32(9, this->max_length); - buffer.encode_string(10, this->pattern); + buffer.encode_string(10, this->pattern_ref_); buffer.encode_uint32(11, static_cast(this->mode)); +#ifdef USE_DEVICES buffer.encode_uint32(12, this->device_id); +#endif } -void ListEntitiesTextResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_uint32_field(total_size, 1, this->min_length); - ProtoSize::add_uint32_field(total_size, 1, this->max_length); - ProtoSize::add_string_field(total_size, 1, this->pattern); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->mode)); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesTextResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); + size.add_uint32(1, this->min_length); + size.add_uint32(1, this->max_length); + size.add_length(1, this->pattern_ref_.size()); + size.add_uint32(1, static_cast(this->mode)); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void TextStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); - buffer.encode_string(2, this->state); + buffer.encode_string(2, this->state_ref_); buffer.encode_bool(3, this->missing_state); +#ifdef USE_DEVICES buffer.encode_uint32(4, this->device_id); +#endif } -void TextStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->state); - ProtoSize::add_bool_field(total_size, 1, this->missing_state); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void TextStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_length(1, this->state_ref_.size()); + size.add_bool(1, this->missing_state); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool TextCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 3: { +#ifdef USE_DEVICES + case 3: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool TextCommandRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { switch (field_id) { - case 2: { + case 2: this->state = value.as_string(); - return true; - } + break; default: return false; } + return true; } bool TextCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_DATETIME_DATE void ListEntitiesDateResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->disabled_by_default); buffer.encode_uint32(7, static_cast(this->entity_category)); +#ifdef USE_DEVICES buffer.encode_uint32(8, this->device_id); +#endif } -void ListEntitiesDateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesDateResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void DateStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); @@ -2846,69 +2741,78 @@ void DateStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint32(3, this->year); buffer.encode_uint32(4, this->month); buffer.encode_uint32(5, this->day); +#ifdef USE_DEVICES buffer.encode_uint32(6, this->device_id); +#endif } -void DateStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_bool_field(total_size, 1, this->missing_state); - ProtoSize::add_uint32_field(total_size, 1, this->year); - ProtoSize::add_uint32_field(total_size, 1, this->month); - ProtoSize::add_uint32_field(total_size, 1, this->day); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void DateStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_bool(1, this->missing_state); + size.add_uint32(1, this->year); + size.add_uint32(1, this->month); + size.add_uint32(1, this->day); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool DateCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->year = value.as_uint32(); - return true; - } - case 3: { + break; + case 3: this->month = value.as_uint32(); - return true; - } - case 4: { + break; + case 4: this->day = value.as_uint32(); - return true; - } - case 5: { + break; +#ifdef USE_DEVICES + case 5: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool DateCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_DATETIME_TIME void ListEntitiesTimeResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->disabled_by_default); buffer.encode_uint32(7, static_cast(this->entity_category)); +#ifdef USE_DEVICES buffer.encode_uint32(8, this->device_id); +#endif } -void ListEntitiesTimeResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesTimeResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void TimeStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); @@ -2916,245 +2820,284 @@ void TimeStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_uint32(3, this->hour); buffer.encode_uint32(4, this->minute); buffer.encode_uint32(5, this->second); +#ifdef USE_DEVICES buffer.encode_uint32(6, this->device_id); +#endif } -void TimeStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_bool_field(total_size, 1, this->missing_state); - ProtoSize::add_uint32_field(total_size, 1, this->hour); - ProtoSize::add_uint32_field(total_size, 1, this->minute); - ProtoSize::add_uint32_field(total_size, 1, this->second); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void TimeStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_bool(1, this->missing_state); + size.add_uint32(1, this->hour); + size.add_uint32(1, this->minute); + size.add_uint32(1, this->second); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool TimeCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->hour = value.as_uint32(); - return true; - } - case 3: { + break; + case 3: this->minute = value.as_uint32(); - return true; - } - case 4: { + break; + case 4: this->second = value.as_uint32(); - return true; - } - case 5: { + break; +#ifdef USE_DEVICES + case 5: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool TimeCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_EVENT void ListEntitiesEventResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->disabled_by_default); buffer.encode_uint32(7, static_cast(this->entity_category)); - buffer.encode_string(8, this->device_class); + buffer.encode_string(8, this->device_class_ref_); for (auto &it : this->event_types) { buffer.encode_string(9, it, true); } +#ifdef USE_DEVICES buffer.encode_uint32(10, this->device_id); +#endif } -void ListEntitiesEventResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_string_field(total_size, 1, this->device_class); +void ListEntitiesEventResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); + size.add_length(1, this->device_class_ref_.size()); if (!this->event_types.empty()) { for (const auto &it : this->event_types) { - ProtoSize::add_string_field_repeated(total_size, 1, it); + size.add_length_force(1, it.size()); } } - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void EventResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); - buffer.encode_string(2, this->event_type); + buffer.encode_string(2, this->event_type_ref_); +#ifdef USE_DEVICES buffer.encode_uint32(3, this->device_id); +#endif } -void EventResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->event_type); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void EventResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_length(1, this->event_type_ref_.size()); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } #endif #ifdef USE_VALVE void ListEntitiesValveResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->disabled_by_default); buffer.encode_uint32(7, static_cast(this->entity_category)); - buffer.encode_string(8, this->device_class); + buffer.encode_string(8, this->device_class_ref_); buffer.encode_bool(9, this->assumed_state); buffer.encode_bool(10, this->supports_position); buffer.encode_bool(11, this->supports_stop); +#ifdef USE_DEVICES buffer.encode_uint32(12, this->device_id); +#endif } -void ListEntitiesValveResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_string_field(total_size, 1, this->device_class); - ProtoSize::add_bool_field(total_size, 1, this->assumed_state); - ProtoSize::add_bool_field(total_size, 1, this->supports_position); - ProtoSize::add_bool_field(total_size, 1, this->supports_stop); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesValveResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); + size.add_length(1, this->device_class_ref_.size()); + size.add_bool(1, this->assumed_state); + size.add_bool(1, this->supports_position); + size.add_bool(1, this->supports_stop); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void ValveStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); buffer.encode_float(2, this->position); buffer.encode_uint32(3, static_cast(this->current_operation)); +#ifdef USE_DEVICES buffer.encode_uint32(4, this->device_id); +#endif } -void ValveStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_fixed_field<4>(total_size, 1, this->position != 0.0f); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->current_operation)); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ValveStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_float(1, this->position); + size.add_uint32(1, static_cast(this->current_operation)); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool ValveCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->has_position = value.as_bool(); - return true; - } - case 4: { + break; + case 4: this->stop = value.as_bool(); - return true; - } - case 5: { + break; +#ifdef USE_DEVICES + case 5: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool ValveCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } - case 3: { + break; + case 3: this->position = value.as_float(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_DATETIME_DATETIME void ListEntitiesDateTimeResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->disabled_by_default); buffer.encode_uint32(7, static_cast(this->entity_category)); +#ifdef USE_DEVICES buffer.encode_uint32(8, this->device_id); +#endif } -void ListEntitiesDateTimeResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesDateTimeResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void DateTimeStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); buffer.encode_bool(2, this->missing_state); buffer.encode_fixed32(3, this->epoch_seconds); +#ifdef USE_DEVICES buffer.encode_uint32(4, this->device_id); +#endif } -void DateTimeStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_bool_field(total_size, 1, this->missing_state); - ProtoSize::add_fixed_field<4>(total_size, 1, this->epoch_seconds != 0); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void DateTimeStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_bool(1, this->missing_state); + size.add_fixed32(1, this->epoch_seconds); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool DateTimeCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 3: { +#ifdef USE_DEVICES + case 3: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool DateTimeCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; - } - case 2: { + break; + case 2: this->epoch_seconds = value.as_fixed32(); - return true; - } + break; default: return false; } + return true; } #endif #ifdef USE_UPDATE void ListEntitiesUpdateResponse::encode(ProtoWriteBuffer buffer) const { - buffer.encode_string(1, this->object_id); + buffer.encode_string(1, this->object_id_ref_); buffer.encode_fixed32(2, this->key); - buffer.encode_string(3, this->name); - buffer.encode_string(4, this->unique_id); - buffer.encode_string(5, this->icon); + buffer.encode_string(3, this->name_ref_); +#ifdef USE_ENTITY_ICON + buffer.encode_string(5, this->icon_ref_); +#endif buffer.encode_bool(6, this->disabled_by_default); buffer.encode_uint32(7, static_cast(this->entity_category)); - buffer.encode_string(8, this->device_class); + buffer.encode_string(8, this->device_class_ref_); +#ifdef USE_DEVICES buffer.encode_uint32(9, this->device_id); +#endif } -void ListEntitiesUpdateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_string_field(total_size, 1, this->object_id); - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_string_field(total_size, 1, this->name); - ProtoSize::add_string_field(total_size, 1, this->unique_id); - ProtoSize::add_string_field(total_size, 1, this->icon); - ProtoSize::add_bool_field(total_size, 1, this->disabled_by_default); - ProtoSize::add_enum_field(total_size, 1, static_cast(this->entity_category)); - ProtoSize::add_string_field(total_size, 1, this->device_class); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void ListEntitiesUpdateResponse::calculate_size(ProtoSize &size) const { + size.add_length(1, this->object_id_ref_.size()); + size.add_fixed32(1, this->key); + size.add_length(1, this->name_ref_.size()); +#ifdef USE_ENTITY_ICON + size.add_length(1, this->icon_ref_.size()); +#endif + size.add_bool(1, this->disabled_by_default); + size.add_uint32(1, static_cast(this->entity_category)); + size.add_length(1, this->device_class_ref_.size()); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } void UpdateStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_fixed32(1, this->key); @@ -3162,51 +3105,103 @@ void UpdateStateResponse::encode(ProtoWriteBuffer buffer) const { buffer.encode_bool(3, this->in_progress); buffer.encode_bool(4, this->has_progress); buffer.encode_float(5, this->progress); - buffer.encode_string(6, this->current_version); - buffer.encode_string(7, this->latest_version); - buffer.encode_string(8, this->title); - buffer.encode_string(9, this->release_summary); - buffer.encode_string(10, this->release_url); + buffer.encode_string(6, this->current_version_ref_); + buffer.encode_string(7, this->latest_version_ref_); + buffer.encode_string(8, this->title_ref_); + buffer.encode_string(9, this->release_summary_ref_); + buffer.encode_string(10, this->release_url_ref_); +#ifdef USE_DEVICES buffer.encode_uint32(11, this->device_id); +#endif } -void UpdateStateResponse::calculate_size(uint32_t &total_size) const { - ProtoSize::add_fixed_field<4>(total_size, 1, this->key != 0); - ProtoSize::add_bool_field(total_size, 1, this->missing_state); - ProtoSize::add_bool_field(total_size, 1, this->in_progress); - ProtoSize::add_bool_field(total_size, 1, this->has_progress); - ProtoSize::add_fixed_field<4>(total_size, 1, this->progress != 0.0f); - ProtoSize::add_string_field(total_size, 1, this->current_version); - ProtoSize::add_string_field(total_size, 1, this->latest_version); - ProtoSize::add_string_field(total_size, 1, this->title); - ProtoSize::add_string_field(total_size, 1, this->release_summary); - ProtoSize::add_string_field(total_size, 1, this->release_url); - ProtoSize::add_uint32_field(total_size, 1, this->device_id); +void UpdateStateResponse::calculate_size(ProtoSize &size) const { + size.add_fixed32(1, this->key); + size.add_bool(1, this->missing_state); + size.add_bool(1, this->in_progress); + size.add_bool(1, this->has_progress); + size.add_float(1, this->progress); + size.add_length(1, this->current_version_ref_.size()); + size.add_length(1, this->latest_version_ref_.size()); + size.add_length(1, this->title_ref_.size()); + size.add_length(1, this->release_summary_ref_.size()); + size.add_length(1, this->release_url_ref_.size()); +#ifdef USE_DEVICES + size.add_uint32(1, this->device_id); +#endif } bool UpdateCommandRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { switch (field_id) { - case 2: { + case 2: this->command = static_cast(value.as_uint32()); - return true; - } - case 3: { + break; +#ifdef USE_DEVICES + case 3: this->device_id = value.as_uint32(); - return true; - } + break; +#endif default: return false; } + return true; } bool UpdateCommandRequest::decode_32bit(uint32_t field_id, Proto32Bit value) { switch (field_id) { - case 1: { + case 1: this->key = value.as_fixed32(); - return true; + break; + default: + return false; + } + return true; +} +#endif +#ifdef USE_ZWAVE_PROXY +bool ZWaveProxyFrame::decode_length(uint32_t field_id, ProtoLengthDelimited value) { + switch (field_id) { + case 1: { + // Use raw data directly to avoid allocation + this->data = value.data(); + this->data_len = value.size(); + break; } default: return false; } + return true; +} +void ZWaveProxyFrame::encode(ProtoWriteBuffer buffer) const { buffer.encode_bytes(1, this->data, this->data_len); } +void ZWaveProxyFrame::calculate_size(ProtoSize &size) const { size.add_length(1, this->data_len); } +bool ZWaveProxyRequest::decode_varint(uint32_t field_id, ProtoVarInt value) { + switch (field_id) { + case 1: + this->type = static_cast(value.as_uint32()); + break; + default: + return false; + } + return true; +} +bool ZWaveProxyRequest::decode_length(uint32_t field_id, ProtoLengthDelimited value) { + switch (field_id) { + case 2: { + // Use raw data directly to avoid allocation + this->data = value.data(); + this->data_len = value.size(); + break; + } + default: + return false; + } + return true; +} +void ZWaveProxyRequest::encode(ProtoWriteBuffer buffer) const { + buffer.encode_uint32(1, static_cast(this->type)); + buffer.encode_bytes(2, this->data, this->data_len); +} +void ZWaveProxyRequest::calculate_size(ProtoSize &size) const { + size.add_uint32(1, static_cast(this->type)); + size.add_length(2, this->data_len); } #endif -} // namespace api -} // namespace esphome +} // namespace esphome::api diff --git a/esphome/components/api/api_pb2.h b/esphome/components/api/api_pb2.h index 3f2d4afad3..5603204801 100644 --- a/esphome/components/api/api_pb2.h +++ b/esphome/components/api/api_pb2.h @@ -3,11 +3,12 @@ #pragma once #include "esphome/core/defines.h" +#include "esphome/core/string_ref.h" #include "proto.h" +#include "api_pb2_includes.h" -namespace esphome { -namespace api { +namespace esphome::api { namespace enums { @@ -17,27 +18,13 @@ enum EntityCategory : uint32_t { ENTITY_CATEGORY_DIAGNOSTIC = 2, }; #ifdef USE_COVER -enum LegacyCoverState : uint32_t { - LEGACY_COVER_STATE_OPEN = 0, - LEGACY_COVER_STATE_CLOSED = 1, -}; enum CoverOperation : uint32_t { COVER_OPERATION_IDLE = 0, COVER_OPERATION_IS_OPENING = 1, COVER_OPERATION_IS_CLOSING = 2, }; -enum LegacyCoverCommand : uint32_t { - LEGACY_COVER_COMMAND_OPEN = 0, - LEGACY_COVER_COMMAND_CLOSE = 1, - LEGACY_COVER_COMMAND_STOP = 2, -}; #endif #ifdef USE_FAN -enum FanSpeed : uint32_t { - FAN_SPEED_LOW = 0, - FAN_SPEED_MEDIUM = 1, - FAN_SPEED_HIGH = 2, -}; enum FanDirection : uint32_t { FAN_DIRECTION_FORWARD = 0, FAN_DIRECTION_REVERSE = 1, @@ -65,11 +52,6 @@ enum SensorStateClass : uint32_t { STATE_CLASS_TOTAL_INCREASING = 2, STATE_CLASS_TOTAL = 3, }; -enum SensorLastResetType : uint32_t { - LAST_RESET_NONE = 0, - LAST_RESET_NEVER = 1, - LAST_RESET_AUTO = 2, -}; #endif enum LogLevel : uint32_t { LOG_LEVEL_NONE = 0, @@ -168,6 +150,9 @@ enum MediaPlayerState : uint32_t { MEDIA_PLAYER_STATE_IDLE = 1, MEDIA_PLAYER_STATE_PLAYING = 2, MEDIA_PLAYER_STATE_PAUSED = 3, + MEDIA_PLAYER_STATE_ANNOUNCING = 4, + MEDIA_PLAYER_STATE_OFF = 5, + MEDIA_PLAYER_STATE_ON = 6, }; enum MediaPlayerCommand : uint32_t { MEDIA_PLAYER_COMMAND_PLAY = 0, @@ -175,6 +160,15 @@ enum MediaPlayerCommand : uint32_t { MEDIA_PLAYER_COMMAND_STOP = 2, MEDIA_PLAYER_COMMAND_MUTE = 3, MEDIA_PLAYER_COMMAND_UNMUTE = 4, + MEDIA_PLAYER_COMMAND_TOGGLE = 5, + MEDIA_PLAYER_COMMAND_VOLUME_UP = 6, + MEDIA_PLAYER_COMMAND_VOLUME_DOWN = 7, + MEDIA_PLAYER_COMMAND_ENQUEUE = 8, + MEDIA_PLAYER_COMMAND_REPEAT_ONE = 9, + MEDIA_PLAYER_COMMAND_REPEAT_OFF = 10, + MEDIA_PLAYER_COMMAND_CLEAR_PLAYLIST = 11, + MEDIA_PLAYER_COMMAND_TURN_ON = 12, + MEDIA_PLAYER_COMMAND_TURN_OFF = 13, }; enum MediaPlayerFormatPurpose : uint32_t { MEDIA_PLAYER_FORMAT_PURPOSE_DEFAULT = 0, @@ -282,20 +276,33 @@ enum UpdateCommand : uint32_t { UPDATE_COMMAND_CHECK = 2, }; #endif +#ifdef USE_ZWAVE_PROXY +enum ZWaveProxyRequestType : uint32_t { + ZWAVE_PROXY_REQUEST_TYPE_SUBSCRIBE = 0, + ZWAVE_PROXY_REQUEST_TYPE_UNSUBSCRIBE = 1, + ZWAVE_PROXY_REQUEST_TYPE_HOME_ID_CHANGE = 2, +}; +#endif } // namespace enums class InfoResponseProtoMessage : public ProtoMessage { public: ~InfoResponseProtoMessage() override = default; - std::string object_id{}; + StringRef object_id_ref_{}; + void set_object_id(const StringRef &ref) { this->object_id_ref_ = ref; } uint32_t key{0}; - std::string name{}; - std::string unique_id{}; + StringRef name_ref_{}; + void set_name(const StringRef &ref) { this->name_ref_ = ref; } bool disabled_by_default{false}; - std::string icon{}; +#ifdef USE_ENTITY_ICON + StringRef icon_ref_{}; + void set_icon(const StringRef &ref) { this->icon_ref_ = ref; } +#endif enums::EntityCategory entity_category{}; +#ifdef USE_DEVICES uint32_t device_id{0}; +#endif protected: }; @@ -304,27 +311,32 @@ class StateResponseProtoMessage : public ProtoMessage { public: ~StateResponseProtoMessage() override = default; uint32_t key{0}; +#ifdef USE_DEVICES uint32_t device_id{0}; +#endif protected: }; -class CommandProtoMessage : public ProtoMessage { +class CommandProtoMessage : public ProtoDecodableMessage { public: ~CommandProtoMessage() override = default; uint32_t key{0}; +#ifdef USE_DEVICES uint32_t device_id{0}; +#endif protected: }; -class HelloRequest : public ProtoMessage { +class HelloRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 1; - static constexpr uint8_t ESTIMATED_SIZE = 17; + static constexpr uint8_t ESTIMATED_SIZE = 27; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "hello_request"; } #endif - std::string client_info{}; + const uint8_t *client_info{nullptr}; + uint16_t client_info_len{0}; uint32_t api_version_major{0}; uint32_t api_version_minor{0}; #ifdef HAS_PROTO_MESSAGE_DUMP @@ -335,7 +347,7 @@ class HelloRequest : public ProtoMessage { bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class HelloResponse : public ProtoMessage { +class HelloResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 2; static constexpr uint8_t ESTIMATED_SIZE = 26; @@ -344,24 +356,28 @@ class HelloResponse : public ProtoMessage { #endif uint32_t api_version_major{0}; uint32_t api_version_minor{0}; - std::string server_info{}; - std::string name{}; + StringRef server_info_ref_{}; + void set_server_info(const StringRef &ref) { this->server_info_ref_ = ref; } + StringRef name_ref_{}; + void set_name(const StringRef &ref) { this->name_ref_ = ref; } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class ConnectRequest : public ProtoMessage { +#ifdef USE_API_PASSWORD +class AuthenticationRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 3; - static constexpr uint8_t ESTIMATED_SIZE = 9; + static constexpr uint8_t ESTIMATED_SIZE = 19; #ifdef HAS_PROTO_MESSAGE_DUMP - const char *message_name() const override { return "connect_request"; } + const char *message_name() const override { return "authentication_request"; } #endif - std::string password{}; + const uint8_t *password{nullptr}; + uint16_t password_len{0}; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -369,23 +385,24 @@ class ConnectRequest : public ProtoMessage { protected: bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; }; -class ConnectResponse : public ProtoMessage { +class AuthenticationResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 4; static constexpr uint8_t ESTIMATED_SIZE = 2; #ifdef HAS_PROTO_MESSAGE_DUMP - const char *message_name() const override { return "connect_response"; } + const char *message_name() const override { return "authentication_response"; } #endif bool invalid_password{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class DisconnectRequest : public ProtoMessage { +#endif +class DisconnectRequest final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 5; static constexpr uint8_t ESTIMATED_SIZE = 0; @@ -398,7 +415,7 @@ class DisconnectRequest : public ProtoMessage { protected: }; -class DisconnectResponse : public ProtoMessage { +class DisconnectResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 6; static constexpr uint8_t ESTIMATED_SIZE = 0; @@ -411,7 +428,7 @@ class DisconnectResponse : public ProtoMessage { protected: }; -class PingRequest : public ProtoMessage { +class PingRequest final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 7; static constexpr uint8_t ESTIMATED_SIZE = 0; @@ -424,7 +441,7 @@ class PingRequest : public ProtoMessage { protected: }; -class PingResponse : public ProtoMessage { +class PingResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 8; static constexpr uint8_t ESTIMATED_SIZE = 0; @@ -437,7 +454,7 @@ class PingResponse : public ProtoMessage { protected: }; -class DeviceInfoRequest : public ProtoMessage { +class DeviceInfoRequest final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 9; static constexpr uint8_t ESTIMATED_SIZE = 0; @@ -450,73 +467,116 @@ class DeviceInfoRequest : public ProtoMessage { protected: }; -class AreaInfo : public ProtoMessage { +#ifdef USE_AREAS +class AreaInfo final : public ProtoMessage { public: uint32_t area_id{0}; - std::string name{}; + StringRef name_ref_{}; + void set_name(const StringRef &ref) { this->name_ref_ = ref; } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: - bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; - bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class DeviceInfo : public ProtoMessage { +#endif +#ifdef USE_DEVICES +class DeviceInfo final : public ProtoMessage { public: uint32_t device_id{0}; - std::string name{}; + StringRef name_ref_{}; + void set_name(const StringRef &ref) { this->name_ref_ = ref; } uint32_t area_id{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: - bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; - bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class DeviceInfoResponse : public ProtoMessage { +#endif +class DeviceInfoResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 10; - static constexpr uint8_t ESTIMATED_SIZE = 219; + static constexpr uint16_t ESTIMATED_SIZE = 257; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "device_info_response"; } #endif +#ifdef USE_API_PASSWORD bool uses_password{false}; - std::string name{}; - std::string mac_address{}; - std::string esphome_version{}; - std::string compilation_time{}; - std::string model{}; +#endif + StringRef name_ref_{}; + void set_name(const StringRef &ref) { this->name_ref_ = ref; } + StringRef mac_address_ref_{}; + void set_mac_address(const StringRef &ref) { this->mac_address_ref_ = ref; } + StringRef esphome_version_ref_{}; + void set_esphome_version(const StringRef &ref) { this->esphome_version_ref_ = ref; } + StringRef compilation_time_ref_{}; + void set_compilation_time(const StringRef &ref) { this->compilation_time_ref_ = ref; } + StringRef model_ref_{}; + void set_model(const StringRef &ref) { this->model_ref_ = ref; } +#ifdef USE_DEEP_SLEEP bool has_deep_sleep{false}; - std::string project_name{}; - std::string project_version{}; +#endif +#ifdef ESPHOME_PROJECT_NAME + StringRef project_name_ref_{}; + void set_project_name(const StringRef &ref) { this->project_name_ref_ = ref; } +#endif +#ifdef ESPHOME_PROJECT_NAME + StringRef project_version_ref_{}; + void set_project_version(const StringRef &ref) { this->project_version_ref_ = ref; } +#endif +#ifdef USE_WEBSERVER uint32_t webserver_port{0}; - uint32_t legacy_bluetooth_proxy_version{0}; +#endif +#ifdef USE_BLUETOOTH_PROXY uint32_t bluetooth_proxy_feature_flags{0}; - std::string manufacturer{}; - std::string friendly_name{}; - uint32_t legacy_voice_assistant_version{0}; +#endif + StringRef manufacturer_ref_{}; + void set_manufacturer(const StringRef &ref) { this->manufacturer_ref_ = ref; } + StringRef friendly_name_ref_{}; + void set_friendly_name(const StringRef &ref) { this->friendly_name_ref_ = ref; } +#ifdef USE_VOICE_ASSISTANT uint32_t voice_assistant_feature_flags{0}; - std::string suggested_area{}; - std::string bluetooth_mac_address{}; +#endif +#ifdef USE_AREAS + StringRef suggested_area_ref_{}; + void set_suggested_area(const StringRef &ref) { this->suggested_area_ref_ = ref; } +#endif +#ifdef USE_BLUETOOTH_PROXY + StringRef bluetooth_mac_address_ref_{}; + void set_bluetooth_mac_address(const StringRef &ref) { this->bluetooth_mac_address_ref_ = ref; } +#endif +#ifdef USE_API_NOISE bool api_encryption_supported{false}; - std::vector devices{}; - std::vector areas{}; +#endif +#ifdef USE_DEVICES + std::array devices{}; +#endif +#ifdef USE_AREAS + std::array areas{}; +#endif +#ifdef USE_AREAS AreaInfo area{}; +#endif +#ifdef USE_ZWAVE_PROXY + uint32_t zwave_proxy_feature_flags{0}; +#endif +#ifdef USE_ZWAVE_PROXY + uint32_t zwave_home_id{0}; +#endif void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class ListEntitiesRequest : public ProtoMessage { +class ListEntitiesRequest final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 11; static constexpr uint8_t ESTIMATED_SIZE = 0; @@ -529,7 +589,7 @@ class ListEntitiesRequest : public ProtoMessage { protected: }; -class ListEntitiesDoneResponse : public ProtoMessage { +class ListEntitiesDoneResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 19; static constexpr uint8_t ESTIMATED_SIZE = 0; @@ -542,7 +602,7 @@ class ListEntitiesDoneResponse : public ProtoMessage { protected: }; -class SubscribeStatesRequest : public ProtoMessage { +class SubscribeStatesRequest final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 20; static constexpr uint8_t ESTIMATED_SIZE = 0; @@ -556,24 +616,25 @@ class SubscribeStatesRequest : public ProtoMessage { protected: }; #ifdef USE_BINARY_SENSOR -class ListEntitiesBinarySensorResponse : public InfoResponseProtoMessage { +class ListEntitiesBinarySensorResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 12; - static constexpr uint8_t ESTIMATED_SIZE = 60; + static constexpr uint8_t ESTIMATED_SIZE = 51; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_binary_sensor_response"; } #endif - std::string device_class{}; + StringRef device_class_ref_{}; + void set_device_class(const StringRef &ref) { this->device_class_ref_ = ref; } bool is_status_binary_sensor{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class BinarySensorStateResponse : public StateResponseProtoMessage { +class BinarySensorStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 21; static constexpr uint8_t ESTIMATED_SIZE = 13; @@ -583,7 +644,7 @@ class BinarySensorStateResponse : public StateResponseProtoMessage { bool state{false}; bool missing_state{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -592,54 +653,52 @@ class BinarySensorStateResponse : public StateResponseProtoMessage { }; #endif #ifdef USE_COVER -class ListEntitiesCoverResponse : public InfoResponseProtoMessage { +class ListEntitiesCoverResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 13; - static constexpr uint8_t ESTIMATED_SIZE = 66; + static constexpr uint8_t ESTIMATED_SIZE = 57; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_cover_response"; } #endif bool assumed_state{false}; bool supports_position{false}; bool supports_tilt{false}; - std::string device_class{}; + StringRef device_class_ref_{}; + void set_device_class(const StringRef &ref) { this->device_class_ref_ = ref; } bool supports_stop{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class CoverStateResponse : public StateResponseProtoMessage { +class CoverStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 22; - static constexpr uint8_t ESTIMATED_SIZE = 23; + static constexpr uint8_t ESTIMATED_SIZE = 21; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "cover_state_response"; } #endif - enums::LegacyCoverState legacy_state{}; float position{0.0f}; float tilt{0.0f}; enums::CoverOperation current_operation{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class CoverCommandRequest : public CommandProtoMessage { +class CoverCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 30; - static constexpr uint8_t ESTIMATED_SIZE = 29; + static constexpr uint8_t ESTIMATED_SIZE = 25; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "cover_command_request"; } #endif - bool has_legacy_command{false}; - enums::LegacyCoverCommand legacy_command{}; bool has_position{false}; float position{0.0f}; bool has_tilt{false}; @@ -655,10 +714,10 @@ class CoverCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_FAN -class ListEntitiesFanResponse : public InfoResponseProtoMessage { +class ListEntitiesFanResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 14; - static constexpr uint8_t ESTIMATED_SIZE = 77; + static constexpr uint8_t ESTIMATED_SIZE = 68; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_fan_response"; } #endif @@ -666,47 +725,45 @@ class ListEntitiesFanResponse : public InfoResponseProtoMessage { bool supports_speed{false}; bool supports_direction{false}; int32_t supported_speed_count{0}; - std::vector supported_preset_modes{}; + const std::set *supported_preset_modes{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class FanStateResponse : public StateResponseProtoMessage { +class FanStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 23; - static constexpr uint8_t ESTIMATED_SIZE = 30; + static constexpr uint8_t ESTIMATED_SIZE = 28; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "fan_state_response"; } #endif bool state{false}; bool oscillating{false}; - enums::FanSpeed speed{}; enums::FanDirection direction{}; int32_t speed_level{0}; - std::string preset_mode{}; + StringRef preset_mode_ref_{}; + void set_preset_mode(const StringRef &ref) { this->preset_mode_ref_ = ref; } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class FanCommandRequest : public CommandProtoMessage { +class FanCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 31; - static constexpr uint8_t ESTIMATED_SIZE = 42; + static constexpr uint8_t ESTIMATED_SIZE = 38; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "fan_command_request"; } #endif bool has_state{false}; bool state{false}; - bool has_speed{false}; - enums::FanSpeed speed{}; bool has_oscillating{false}; bool oscillating{false}; bool has_direction{false}; @@ -726,30 +783,26 @@ class FanCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_LIGHT -class ListEntitiesLightResponse : public InfoResponseProtoMessage { +class ListEntitiesLightResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 15; - static constexpr uint8_t ESTIMATED_SIZE = 90; + static constexpr uint8_t ESTIMATED_SIZE = 73; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_light_response"; } #endif - std::vector supported_color_modes{}; - bool legacy_supports_brightness{false}; - bool legacy_supports_rgb{false}; - bool legacy_supports_white_value{false}; - bool legacy_supports_color_temperature{false}; + const std::set *supported_color_modes{}; float min_mireds{0.0f}; float max_mireds{0.0f}; std::vector effects{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class LightStateResponse : public StateResponseProtoMessage { +class LightStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 24; static constexpr uint8_t ESTIMATED_SIZE = 67; @@ -767,16 +820,17 @@ class LightStateResponse : public StateResponseProtoMessage { float color_temperature{0.0f}; float cold_white{0.0f}; float warm_white{0.0f}; - std::string effect{}; + StringRef effect_ref_{}; + void set_effect(const StringRef &ref) { this->effect_ref_ = ref; } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class LightCommandRequest : public CommandProtoMessage { +class LightCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 32; static constexpr uint8_t ESTIMATED_SIZE = 112; @@ -820,28 +874,29 @@ class LightCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_SENSOR -class ListEntitiesSensorResponse : public InfoResponseProtoMessage { +class ListEntitiesSensorResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 16; - static constexpr uint8_t ESTIMATED_SIZE = 77; + static constexpr uint8_t ESTIMATED_SIZE = 66; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_sensor_response"; } #endif - std::string unit_of_measurement{}; + StringRef unit_of_measurement_ref_{}; + void set_unit_of_measurement(const StringRef &ref) { this->unit_of_measurement_ref_ = ref; } int32_t accuracy_decimals{0}; bool force_update{false}; - std::string device_class{}; + StringRef device_class_ref_{}; + void set_device_class(const StringRef &ref) { this->device_class_ref_ = ref; } enums::SensorStateClass state_class{}; - enums::SensorLastResetType legacy_last_reset_type{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class SensorStateResponse : public StateResponseProtoMessage { +class SensorStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 25; static constexpr uint8_t ESTIMATED_SIZE = 16; @@ -851,7 +906,7 @@ class SensorStateResponse : public StateResponseProtoMessage { float state{0.0f}; bool missing_state{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -860,24 +915,25 @@ class SensorStateResponse : public StateResponseProtoMessage { }; #endif #ifdef USE_SWITCH -class ListEntitiesSwitchResponse : public InfoResponseProtoMessage { +class ListEntitiesSwitchResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 17; - static constexpr uint8_t ESTIMATED_SIZE = 60; + static constexpr uint8_t ESTIMATED_SIZE = 51; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_switch_response"; } #endif bool assumed_state{false}; - std::string device_class{}; + StringRef device_class_ref_{}; + void set_device_class(const StringRef &ref) { this->device_class_ref_ = ref; } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class SwitchStateResponse : public StateResponseProtoMessage { +class SwitchStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 26; static constexpr uint8_t ESTIMATED_SIZE = 11; @@ -886,14 +942,14 @@ class SwitchStateResponse : public StateResponseProtoMessage { #endif bool state{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class SwitchCommandRequest : public CommandProtoMessage { +class SwitchCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 33; static constexpr uint8_t ESTIMATED_SIZE = 11; @@ -911,33 +967,35 @@ class SwitchCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_TEXT_SENSOR -class ListEntitiesTextSensorResponse : public InfoResponseProtoMessage { +class ListEntitiesTextSensorResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 18; - static constexpr uint8_t ESTIMATED_SIZE = 58; + static constexpr uint8_t ESTIMATED_SIZE = 49; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_text_sensor_response"; } #endif - std::string device_class{}; + StringRef device_class_ref_{}; + void set_device_class(const StringRef &ref) { this->device_class_ref_ = ref; } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class TextSensorStateResponse : public StateResponseProtoMessage { +class TextSensorStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 27; static constexpr uint8_t ESTIMATED_SIZE = 20; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "text_sensor_state_response"; } #endif - std::string state{}; + StringRef state_ref_{}; + void set_state(const StringRef &ref) { this->state_ref_ = ref; } bool missing_state{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -945,7 +1003,7 @@ class TextSensorStateResponse : public StateResponseProtoMessage { protected: }; #endif -class SubscribeLogsRequest : public ProtoMessage { +class SubscribeLogsRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 28; static constexpr uint8_t ESTIMATED_SIZE = 4; @@ -961,18 +1019,22 @@ class SubscribeLogsRequest : public ProtoMessage { protected: bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class SubscribeLogsResponse : public ProtoMessage { +class SubscribeLogsResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 29; - static constexpr uint8_t ESTIMATED_SIZE = 13; + static constexpr uint8_t ESTIMATED_SIZE = 11; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "subscribe_logs_response"; } #endif enums::LogLevel level{}; - std::string message{}; - bool send_failed{false}; + const uint8_t *message_ptr_{nullptr}; + size_t message_len_{0}; + void set_message(const uint8_t *data, size_t len) { + this->message_ptr_ = data; + this->message_len_ = len; + } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -980,7 +1042,7 @@ class SubscribeLogsResponse : public ProtoMessage { protected: }; #ifdef USE_API_NOISE -class NoiseEncryptionSetKeyRequest : public ProtoMessage { +class NoiseEncryptionSetKeyRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 124; static constexpr uint8_t ESTIMATED_SIZE = 9; @@ -995,7 +1057,7 @@ class NoiseEncryptionSetKeyRequest : public ProtoMessage { protected: bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; }; -class NoiseEncryptionSetKeyResponse : public ProtoMessage { +class NoiseEncryptionSetKeyResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 125; static constexpr uint8_t ESTIMATED_SIZE = 2; @@ -1004,7 +1066,7 @@ class NoiseEncryptionSetKeyResponse : public ProtoMessage { #endif bool success{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -1012,7 +1074,8 @@ class NoiseEncryptionSetKeyResponse : public ProtoMessage { protected: }; #endif -class SubscribeHomeassistantServicesRequest : public ProtoMessage { +#ifdef USE_API_HOMEASSISTANT_SERVICES +class SubscribeHomeassistantServicesRequest final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 34; static constexpr uint8_t ESTIMATED_SIZE = 0; @@ -1025,40 +1088,76 @@ class SubscribeHomeassistantServicesRequest : public ProtoMessage { protected: }; -class HomeassistantServiceMap : public ProtoMessage { +class HomeassistantServiceMap final : public ProtoMessage { public: - std::string key{}; + StringRef key_ref_{}; + void set_key(const StringRef &ref) { this->key_ref_ = ref; } std::string value{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; +#ifdef HAS_PROTO_MESSAGE_DUMP + void dump_to(std::string &out) const override; +#endif + + protected: +}; +class HomeassistantActionRequest final : public ProtoMessage { + public: + static constexpr uint8_t MESSAGE_TYPE = 35; + static constexpr uint8_t ESTIMATED_SIZE = 128; +#ifdef HAS_PROTO_MESSAGE_DUMP + const char *message_name() const override { return "homeassistant_action_request"; } +#endif + StringRef service_ref_{}; + void set_service(const StringRef &ref) { this->service_ref_ = ref; } + FixedVector data{}; + FixedVector data_template{}; + FixedVector variables{}; + bool is_event{false}; +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES + uint32_t call_id{0}; +#endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + bool wants_response{false}; +#endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + std::string response_template{}; +#endif + void encode(ProtoWriteBuffer buffer) const override; + void calculate_size(ProtoSize &size) const override; +#ifdef HAS_PROTO_MESSAGE_DUMP + void dump_to(std::string &out) const override; +#endif + + protected: +}; +#endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES +class HomeassistantActionResponse final : public ProtoDecodableMessage { + public: + static constexpr uint8_t MESSAGE_TYPE = 130; + static constexpr uint8_t ESTIMATED_SIZE = 34; +#ifdef HAS_PROTO_MESSAGE_DUMP + const char *message_name() const override { return "homeassistant_action_response"; } +#endif + uint32_t call_id{0}; + bool success{false}; + std::string error_message{}; +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + const uint8_t *response_data{nullptr}; + uint16_t response_data_len{0}; +#endif #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; + bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class HomeassistantServiceResponse : public ProtoMessage { - public: - static constexpr uint8_t MESSAGE_TYPE = 35; - static constexpr uint8_t ESTIMATED_SIZE = 113; -#ifdef HAS_PROTO_MESSAGE_DUMP - const char *message_name() const override { return "homeassistant_service_response"; } #endif - std::string service{}; - std::vector data{}; - std::vector data_template{}; - std::vector variables{}; - bool is_event{false}; - void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; -#ifdef HAS_PROTO_MESSAGE_DUMP - void dump_to(std::string &out) const override; -#endif - - protected: -}; -class SubscribeHomeAssistantStatesRequest : public ProtoMessage { +#ifdef USE_API_HOMEASSISTANT_STATES +class SubscribeHomeAssistantStatesRequest final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 38; static constexpr uint8_t ESTIMATED_SIZE = 0; @@ -1071,25 +1170,27 @@ class SubscribeHomeAssistantStatesRequest : public ProtoMessage { protected: }; -class SubscribeHomeAssistantStateResponse : public ProtoMessage { +class SubscribeHomeAssistantStateResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 39; static constexpr uint8_t ESTIMATED_SIZE = 20; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "subscribe_home_assistant_state_response"; } #endif - std::string entity_id{}; - std::string attribute{}; + StringRef entity_id_ref_{}; + void set_entity_id(const StringRef &ref) { this->entity_id_ref_ = ref; } + StringRef attribute_ref_{}; + void set_attribute(const StringRef &ref) { this->attribute_ref_ = ref; } bool once{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class HomeAssistantStateResponse : public ProtoMessage { +class HomeAssistantStateResponse final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 40; static constexpr uint8_t ESTIMATED_SIZE = 27; @@ -1106,7 +1207,8 @@ class HomeAssistantStateResponse : public ProtoMessage { protected: bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; }; -class GetTimeRequest : public ProtoMessage { +#endif +class GetTimeRequest final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 36; static constexpr uint8_t ESTIMATED_SIZE = 0; @@ -1119,69 +1221,69 @@ class GetTimeRequest : public ProtoMessage { protected: }; -class GetTimeResponse : public ProtoMessage { +class GetTimeResponse final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 37; - static constexpr uint8_t ESTIMATED_SIZE = 5; + static constexpr uint8_t ESTIMATED_SIZE = 24; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "get_time_response"; } #endif uint32_t epoch_seconds{0}; - void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + const uint8_t *timezone{nullptr}; + uint16_t timezone_len{0}; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: bool decode_32bit(uint32_t field_id, Proto32Bit value) override; + bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; }; #ifdef USE_API_SERVICES -class ListEntitiesServicesArgument : public ProtoMessage { +class ListEntitiesServicesArgument final : public ProtoMessage { public: - std::string name{}; + StringRef name_ref_{}; + void set_name(const StringRef &ref) { this->name_ref_ = ref; } enums::ServiceArgType type{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: - bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; - bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class ListEntitiesServicesResponse : public ProtoMessage { +class ListEntitiesServicesResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 41; static constexpr uint8_t ESTIMATED_SIZE = 48; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_services_response"; } #endif - std::string name{}; + StringRef name_ref_{}; + void set_name(const StringRef &ref) { this->name_ref_ = ref; } uint32_t key{0}; - std::vector args{}; + FixedVector args{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class ExecuteServiceArgument : public ProtoMessage { +class ExecuteServiceArgument final : public ProtoDecodableMessage { public: bool bool_{false}; int32_t legacy_int{0}; float float_{0.0f}; std::string string_{}; int32_t int_{0}; - std::vector bool_array{}; - std::vector int_array{}; - std::vector float_array{}; - std::vector string_array{}; - void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + FixedVector bool_array{}; + FixedVector int_array{}; + FixedVector float_array{}; + FixedVector string_array{}; + void decode(const uint8_t *buffer, size_t length) override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -1191,7 +1293,7 @@ class ExecuteServiceArgument : public ProtoMessage { bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class ExecuteServiceRequest : public ProtoMessage { +class ExecuteServiceRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 42; static constexpr uint8_t ESTIMATED_SIZE = 39; @@ -1199,7 +1301,8 @@ class ExecuteServiceRequest : public ProtoMessage { const char *message_name() const override { return "execute_service_request"; } #endif uint32_t key{0}; - std::vector args{}; + FixedVector args{}; + void decode(const uint8_t *buffer, size_t length) override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -1210,39 +1313,44 @@ class ExecuteServiceRequest : public ProtoMessage { }; #endif #ifdef USE_CAMERA -class ListEntitiesCameraResponse : public InfoResponseProtoMessage { +class ListEntitiesCameraResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 43; - static constexpr uint8_t ESTIMATED_SIZE = 49; + static constexpr uint8_t ESTIMATED_SIZE = 40; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_camera_response"; } #endif void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class CameraImageResponse : public StateResponseProtoMessage { +class CameraImageResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 44; static constexpr uint8_t ESTIMATED_SIZE = 20; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "camera_image_response"; } #endif - std::string data{}; + const uint8_t *data_ptr_{nullptr}; + size_t data_len_{0}; + void set_data(const uint8_t *data, size_t len) { + this->data_ptr_ = data; + this->data_len_ = len; + } bool done{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class CameraImageRequest : public ProtoMessage { +class CameraImageRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 45; static constexpr uint8_t ESTIMATED_SIZE = 4; @@ -1260,43 +1368,43 @@ class CameraImageRequest : public ProtoMessage { }; #endif #ifdef USE_CLIMATE -class ListEntitiesClimateResponse : public InfoResponseProtoMessage { +class ListEntitiesClimateResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 46; - static constexpr uint8_t ESTIMATED_SIZE = 156; + static constexpr uint8_t ESTIMATED_SIZE = 150; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_climate_response"; } #endif bool supports_current_temperature{false}; bool supports_two_point_target_temperature{false}; - std::vector supported_modes{}; + const std::set *supported_modes{}; float visual_min_temperature{0.0f}; float visual_max_temperature{0.0f}; float visual_target_temperature_step{0.0f}; - bool legacy_supports_away{false}; bool supports_action{false}; - std::vector supported_fan_modes{}; - std::vector supported_swing_modes{}; - std::vector supported_custom_fan_modes{}; - std::vector supported_presets{}; - std::vector supported_custom_presets{}; + const std::set *supported_fan_modes{}; + const std::set *supported_swing_modes{}; + const std::set *supported_custom_fan_modes{}; + const std::set *supported_presets{}; + const std::set *supported_custom_presets{}; float visual_current_temperature_step{0.0f}; bool supports_current_humidity{false}; bool supports_target_humidity{false}; float visual_min_humidity{0.0f}; float visual_max_humidity{0.0f}; + uint32_t feature_flags{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class ClimateStateResponse : public StateResponseProtoMessage { +class ClimateStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 47; - static constexpr uint8_t ESTIMATED_SIZE = 70; + static constexpr uint8_t ESTIMATED_SIZE = 68; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "climate_state_response"; } #endif @@ -1305,27 +1413,28 @@ class ClimateStateResponse : public StateResponseProtoMessage { float target_temperature{0.0f}; float target_temperature_low{0.0f}; float target_temperature_high{0.0f}; - bool unused_legacy_away{false}; enums::ClimateAction action{}; enums::ClimateFanMode fan_mode{}; enums::ClimateSwingMode swing_mode{}; - std::string custom_fan_mode{}; + StringRef custom_fan_mode_ref_{}; + void set_custom_fan_mode(const StringRef &ref) { this->custom_fan_mode_ref_ = ref; } enums::ClimatePreset preset{}; - std::string custom_preset{}; + StringRef custom_preset_ref_{}; + void set_custom_preset(const StringRef &ref) { this->custom_preset_ref_ = ref; } float current_humidity{0.0f}; float target_humidity{0.0f}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class ClimateCommandRequest : public CommandProtoMessage { +class ClimateCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 48; - static constexpr uint8_t ESTIMATED_SIZE = 88; + static constexpr uint8_t ESTIMATED_SIZE = 84; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "climate_command_request"; } #endif @@ -1337,8 +1446,6 @@ class ClimateCommandRequest : public CommandProtoMessage { float target_temperature_low{0.0f}; bool has_target_temperature_high{false}; float target_temperature_high{0.0f}; - bool unused_has_legacy_away{false}; - bool unused_legacy_away{false}; bool has_fan_mode{false}; enums::ClimateFanMode fan_mode{}; bool has_swing_mode{false}; @@ -1362,28 +1469,30 @@ class ClimateCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_NUMBER -class ListEntitiesNumberResponse : public InfoResponseProtoMessage { +class ListEntitiesNumberResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 49; - static constexpr uint8_t ESTIMATED_SIZE = 84; + static constexpr uint8_t ESTIMATED_SIZE = 75; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_number_response"; } #endif float min_value{0.0f}; float max_value{0.0f}; float step{0.0f}; - std::string unit_of_measurement{}; + StringRef unit_of_measurement_ref_{}; + void set_unit_of_measurement(const StringRef &ref) { this->unit_of_measurement_ref_ = ref; } enums::NumberMode mode{}; - std::string device_class{}; + StringRef device_class_ref_{}; + void set_device_class(const StringRef &ref) { this->device_class_ref_ = ref; } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class NumberStateResponse : public StateResponseProtoMessage { +class NumberStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 50; static constexpr uint8_t ESTIMATED_SIZE = 16; @@ -1393,14 +1502,14 @@ class NumberStateResponse : public StateResponseProtoMessage { float state{0.0f}; bool missing_state{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class NumberCommandRequest : public CommandProtoMessage { +class NumberCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 51; static constexpr uint8_t ESTIMATED_SIZE = 14; @@ -1418,40 +1527,41 @@ class NumberCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_SELECT -class ListEntitiesSelectResponse : public InfoResponseProtoMessage { +class ListEntitiesSelectResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 52; - static constexpr uint8_t ESTIMATED_SIZE = 67; + static constexpr uint8_t ESTIMATED_SIZE = 58; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_select_response"; } #endif - std::vector options{}; + const std::vector *options{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class SelectStateResponse : public StateResponseProtoMessage { +class SelectStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 53; static constexpr uint8_t ESTIMATED_SIZE = 20; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "select_state_response"; } #endif - std::string state{}; + StringRef state_ref_{}; + void set_state(const StringRef &ref) { this->state_ref_ = ref; } bool missing_state{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class SelectCommandRequest : public CommandProtoMessage { +class SelectCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 54; static constexpr uint8_t ESTIMATED_SIZE = 18; @@ -1470,10 +1580,10 @@ class SelectCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_SIREN -class ListEntitiesSirenResponse : public InfoResponseProtoMessage { +class ListEntitiesSirenResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 55; - static constexpr uint8_t ESTIMATED_SIZE = 71; + static constexpr uint8_t ESTIMATED_SIZE = 62; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_siren_response"; } #endif @@ -1481,14 +1591,14 @@ class ListEntitiesSirenResponse : public InfoResponseProtoMessage { bool supports_duration{false}; bool supports_volume{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class SirenStateResponse : public StateResponseProtoMessage { +class SirenStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 56; static constexpr uint8_t ESTIMATED_SIZE = 11; @@ -1497,14 +1607,14 @@ class SirenStateResponse : public StateResponseProtoMessage { #endif bool state{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class SirenCommandRequest : public CommandProtoMessage { +class SirenCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 57; static constexpr uint8_t ESTIMATED_SIZE = 37; @@ -1530,26 +1640,27 @@ class SirenCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_LOCK -class ListEntitiesLockResponse : public InfoResponseProtoMessage { +class ListEntitiesLockResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 58; - static constexpr uint8_t ESTIMATED_SIZE = 64; + static constexpr uint8_t ESTIMATED_SIZE = 55; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_lock_response"; } #endif bool assumed_state{false}; bool supports_open{false}; bool requires_code{false}; - std::string code_format{}; + StringRef code_format_ref_{}; + void set_code_format(const StringRef &ref) { this->code_format_ref_ = ref; } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class LockStateResponse : public StateResponseProtoMessage { +class LockStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 59; static constexpr uint8_t ESTIMATED_SIZE = 11; @@ -1558,14 +1669,14 @@ class LockStateResponse : public StateResponseProtoMessage { #endif enums::LockState state{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class LockCommandRequest : public CommandProtoMessage { +class LockCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 60; static constexpr uint8_t ESTIMATED_SIZE = 22; @@ -1586,23 +1697,24 @@ class LockCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_BUTTON -class ListEntitiesButtonResponse : public InfoResponseProtoMessage { +class ListEntitiesButtonResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 61; - static constexpr uint8_t ESTIMATED_SIZE = 58; + static constexpr uint8_t ESTIMATED_SIZE = 49; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_button_response"; } #endif - std::string device_class{}; + StringRef device_class_ref_{}; + void set_device_class(const StringRef &ref) { this->device_class_ref_ = ref; } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class ButtonCommandRequest : public CommandProtoMessage { +class ButtonCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 62; static constexpr uint8_t ESTIMATED_SIZE = 9; @@ -1619,41 +1731,41 @@ class ButtonCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_MEDIA_PLAYER -class MediaPlayerSupportedFormat : public ProtoMessage { +class MediaPlayerSupportedFormat final : public ProtoMessage { public: - std::string format{}; + StringRef format_ref_{}; + void set_format(const StringRef &ref) { this->format_ref_ = ref; } uint32_t sample_rate{0}; uint32_t num_channels{0}; enums::MediaPlayerFormatPurpose purpose{}; uint32_t sample_bytes{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: - bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; - bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class ListEntitiesMediaPlayerResponse : public InfoResponseProtoMessage { +class ListEntitiesMediaPlayerResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 63; - static constexpr uint8_t ESTIMATED_SIZE = 85; + static constexpr uint8_t ESTIMATED_SIZE = 80; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_media_player_response"; } #endif bool supports_pause{false}; std::vector supported_formats{}; + uint32_t feature_flags{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class MediaPlayerStateResponse : public StateResponseProtoMessage { +class MediaPlayerStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 64; static constexpr uint8_t ESTIMATED_SIZE = 18; @@ -1664,14 +1776,14 @@ class MediaPlayerStateResponse : public StateResponseProtoMessage { float volume{0.0f}; bool muted{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class MediaPlayerCommandRequest : public CommandProtoMessage { +class MediaPlayerCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 65; static constexpr uint8_t ESTIMATED_SIZE = 35; @@ -1697,7 +1809,7 @@ class MediaPlayerCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_BLUETOOTH_PROXY -class SubscribeBluetoothLEAdvertisementsRequest : public ProtoMessage { +class SubscribeBluetoothLEAdvertisementsRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 66; static constexpr uint8_t ESTIMATED_SIZE = 4; @@ -1712,76 +1824,39 @@ class SubscribeBluetoothLEAdvertisementsRequest : public ProtoMessage { protected: bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class BluetoothServiceData : public ProtoMessage { - public: - std::string uuid{}; - std::vector legacy_data{}; - std::string data{}; - void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; -#ifdef HAS_PROTO_MESSAGE_DUMP - void dump_to(std::string &out) const override; -#endif - - protected: - bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; - bool decode_varint(uint32_t field_id, ProtoVarInt value) override; -}; -class BluetoothLEAdvertisementResponse : public ProtoMessage { - public: - static constexpr uint8_t MESSAGE_TYPE = 67; - static constexpr uint8_t ESTIMATED_SIZE = 107; -#ifdef HAS_PROTO_MESSAGE_DUMP - const char *message_name() const override { return "bluetooth_le_advertisement_response"; } -#endif - uint64_t address{0}; - std::string name{}; - int32_t rssi{0}; - std::vector service_uuids{}; - std::vector service_data{}; - std::vector manufacturer_data{}; - uint32_t address_type{0}; - void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; -#ifdef HAS_PROTO_MESSAGE_DUMP - void dump_to(std::string &out) const override; -#endif - - protected: -}; -class BluetoothLERawAdvertisement : public ProtoMessage { +class BluetoothLERawAdvertisement final : public ProtoMessage { public: uint64_t address{0}; int32_t rssi{0}; uint32_t address_type{0}; - std::string data{}; + uint8_t data[62]{}; + uint8_t data_len{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: - bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; - bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class BluetoothLERawAdvertisementsResponse : public ProtoMessage { +class BluetoothLERawAdvertisementsResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 93; - static constexpr uint8_t ESTIMATED_SIZE = 34; + static constexpr uint8_t ESTIMATED_SIZE = 136; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "bluetooth_le_raw_advertisements_response"; } #endif - std::vector advertisements{}; + std::array advertisements{}; + uint16_t advertisements_len{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class BluetoothDeviceRequest : public ProtoMessage { +class BluetoothDeviceRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 68; static constexpr uint8_t ESTIMATED_SIZE = 12; @@ -1799,7 +1874,7 @@ class BluetoothDeviceRequest : public ProtoMessage { protected: bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class BluetoothDeviceConnectionResponse : public ProtoMessage { +class BluetoothDeviceConnectionResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 69; static constexpr uint8_t ESTIMATED_SIZE = 14; @@ -1811,14 +1886,14 @@ class BluetoothDeviceConnectionResponse : public ProtoMessage { uint32_t mtu{0}; int32_t error{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class BluetoothGATTGetServicesRequest : public ProtoMessage { +class BluetoothGATTGetServicesRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 70; static constexpr uint8_t ESTIMATED_SIZE = 4; @@ -1833,51 +1908,49 @@ class BluetoothGATTGetServicesRequest : public ProtoMessage { protected: bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class BluetoothGATTDescriptor : public ProtoMessage { +class BluetoothGATTDescriptor final : public ProtoMessage { public: - std::vector uuid{}; + std::array uuid{}; uint32_t handle{0}; + uint32_t short_uuid{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: - bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class BluetoothGATTCharacteristic : public ProtoMessage { +class BluetoothGATTCharacteristic final : public ProtoMessage { public: - std::vector uuid{}; + std::array uuid{}; uint32_t handle{0}; uint32_t properties{0}; - std::vector descriptors{}; + FixedVector descriptors{}; + uint32_t short_uuid{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: - bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; - bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class BluetoothGATTService : public ProtoMessage { +class BluetoothGATTService final : public ProtoMessage { public: - std::vector uuid{}; + std::array uuid{}; uint32_t handle{0}; - std::vector characteristics{}; + FixedVector characteristics{}; + uint32_t short_uuid{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: - bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; - bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class BluetoothGATTGetServicesResponse : public ProtoMessage { +class BluetoothGATTGetServicesResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 71; static constexpr uint8_t ESTIMATED_SIZE = 38; @@ -1887,14 +1960,14 @@ class BluetoothGATTGetServicesResponse : public ProtoMessage { uint64_t address{0}; std::vector services{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class BluetoothGATTGetServicesDoneResponse : public ProtoMessage { +class BluetoothGATTGetServicesDoneResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 72; static constexpr uint8_t ESTIMATED_SIZE = 4; @@ -1903,14 +1976,14 @@ class BluetoothGATTGetServicesDoneResponse : public ProtoMessage { #endif uint64_t address{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class BluetoothGATTReadRequest : public ProtoMessage { +class BluetoothGATTReadRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 73; static constexpr uint8_t ESTIMATED_SIZE = 8; @@ -1926,7 +1999,7 @@ class BluetoothGATTReadRequest : public ProtoMessage { protected: bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class BluetoothGATTReadResponse : public ProtoMessage { +class BluetoothGATTReadResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 74; static constexpr uint8_t ESTIMATED_SIZE = 17; @@ -1935,26 +2008,32 @@ class BluetoothGATTReadResponse : public ProtoMessage { #endif uint64_t address{0}; uint32_t handle{0}; - std::string data{}; + const uint8_t *data_ptr_{nullptr}; + size_t data_len_{0}; + void set_data(const uint8_t *data, size_t len) { + this->data_ptr_ = data; + this->data_len_ = len; + } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class BluetoothGATTWriteRequest : public ProtoMessage { +class BluetoothGATTWriteRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 75; - static constexpr uint8_t ESTIMATED_SIZE = 19; + static constexpr uint8_t ESTIMATED_SIZE = 29; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "bluetooth_gatt_write_request"; } #endif uint64_t address{0}; uint32_t handle{0}; bool response{false}; - std::string data{}; + const uint8_t *data{nullptr}; + uint16_t data_len{0}; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -1963,7 +2042,7 @@ class BluetoothGATTWriteRequest : public ProtoMessage { bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class BluetoothGATTReadDescriptorRequest : public ProtoMessage { +class BluetoothGATTReadDescriptorRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 76; static constexpr uint8_t ESTIMATED_SIZE = 8; @@ -1979,16 +2058,17 @@ class BluetoothGATTReadDescriptorRequest : public ProtoMessage { protected: bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class BluetoothGATTWriteDescriptorRequest : public ProtoMessage { +class BluetoothGATTWriteDescriptorRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 77; - static constexpr uint8_t ESTIMATED_SIZE = 17; + static constexpr uint8_t ESTIMATED_SIZE = 27; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "bluetooth_gatt_write_descriptor_request"; } #endif uint64_t address{0}; uint32_t handle{0}; - std::string data{}; + const uint8_t *data{nullptr}; + uint16_t data_len{0}; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -1997,7 +2077,7 @@ class BluetoothGATTWriteDescriptorRequest : public ProtoMessage { bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class BluetoothGATTNotifyRequest : public ProtoMessage { +class BluetoothGATTNotifyRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 78; static constexpr uint8_t ESTIMATED_SIZE = 10; @@ -2014,7 +2094,7 @@ class BluetoothGATTNotifyRequest : public ProtoMessage { protected: bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class BluetoothGATTNotifyDataResponse : public ProtoMessage { +class BluetoothGATTNotifyDataResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 79; static constexpr uint8_t ESTIMATED_SIZE = 17; @@ -2023,16 +2103,21 @@ class BluetoothGATTNotifyDataResponse : public ProtoMessage { #endif uint64_t address{0}; uint32_t handle{0}; - std::string data{}; + const uint8_t *data_ptr_{nullptr}; + size_t data_len_{0}; + void set_data(const uint8_t *data, size_t len) { + this->data_ptr_ = data; + this->data_len_ = len; + } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class SubscribeBluetoothConnectionsFreeRequest : public ProtoMessage { +class SubscribeBluetoothConnectionsFreeRequest final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 80; static constexpr uint8_t ESTIMATED_SIZE = 0; @@ -2045,25 +2130,25 @@ class SubscribeBluetoothConnectionsFreeRequest : public ProtoMessage { protected: }; -class BluetoothConnectionsFreeResponse : public ProtoMessage { +class BluetoothConnectionsFreeResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 81; - static constexpr uint8_t ESTIMATED_SIZE = 16; + static constexpr uint8_t ESTIMATED_SIZE = 20; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "bluetooth_connections_free_response"; } #endif uint32_t free{0}; uint32_t limit{0}; - std::vector allocated{}; + std::array allocated{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class BluetoothGATTErrorResponse : public ProtoMessage { +class BluetoothGATTErrorResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 82; static constexpr uint8_t ESTIMATED_SIZE = 12; @@ -2074,14 +2159,14 @@ class BluetoothGATTErrorResponse : public ProtoMessage { uint32_t handle{0}; int32_t error{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class BluetoothGATTWriteResponse : public ProtoMessage { +class BluetoothGATTWriteResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 83; static constexpr uint8_t ESTIMATED_SIZE = 8; @@ -2091,14 +2176,14 @@ class BluetoothGATTWriteResponse : public ProtoMessage { uint64_t address{0}; uint32_t handle{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class BluetoothGATTNotifyResponse : public ProtoMessage { +class BluetoothGATTNotifyResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 84; static constexpr uint8_t ESTIMATED_SIZE = 8; @@ -2108,14 +2193,14 @@ class BluetoothGATTNotifyResponse : public ProtoMessage { uint64_t address{0}; uint32_t handle{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class BluetoothDevicePairingResponse : public ProtoMessage { +class BluetoothDevicePairingResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 85; static constexpr uint8_t ESTIMATED_SIZE = 10; @@ -2126,14 +2211,14 @@ class BluetoothDevicePairingResponse : public ProtoMessage { bool paired{false}; int32_t error{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class BluetoothDeviceUnpairingResponse : public ProtoMessage { +class BluetoothDeviceUnpairingResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 86; static constexpr uint8_t ESTIMATED_SIZE = 10; @@ -2144,14 +2229,14 @@ class BluetoothDeviceUnpairingResponse : public ProtoMessage { bool success{false}; int32_t error{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class UnsubscribeBluetoothLEAdvertisementsRequest : public ProtoMessage { +class UnsubscribeBluetoothLEAdvertisementsRequest final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 87; static constexpr uint8_t ESTIMATED_SIZE = 0; @@ -2164,7 +2249,7 @@ class UnsubscribeBluetoothLEAdvertisementsRequest : public ProtoMessage { protected: }; -class BluetoothDeviceClearCacheResponse : public ProtoMessage { +class BluetoothDeviceClearCacheResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 88; static constexpr uint8_t ESTIMATED_SIZE = 10; @@ -2175,31 +2260,32 @@ class BluetoothDeviceClearCacheResponse : public ProtoMessage { bool success{false}; int32_t error{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class BluetoothScannerStateResponse : public ProtoMessage { +class BluetoothScannerStateResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 126; - static constexpr uint8_t ESTIMATED_SIZE = 4; + static constexpr uint8_t ESTIMATED_SIZE = 6; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "bluetooth_scanner_state_response"; } #endif enums::BluetoothScannerState state{}; enums::BluetoothScannerMode mode{}; + enums::BluetoothScannerMode configured_mode{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class BluetoothScannerSetModeRequest : public ProtoMessage { +class BluetoothScannerSetModeRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 127; static constexpr uint8_t ESTIMATED_SIZE = 2; @@ -2216,7 +2302,7 @@ class BluetoothScannerSetModeRequest : public ProtoMessage { }; #endif #ifdef USE_VOICE_ASSISTANT -class SubscribeVoiceAssistantRequest : public ProtoMessage { +class SubscribeVoiceAssistantRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 89; static constexpr uint8_t ESTIMATED_SIZE = 6; @@ -2232,22 +2318,20 @@ class SubscribeVoiceAssistantRequest : public ProtoMessage { protected: bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class VoiceAssistantAudioSettings : public ProtoMessage { +class VoiceAssistantAudioSettings final : public ProtoMessage { public: uint32_t noise_suppression_level{0}; uint32_t auto_gain{0}; float volume_multiplier{0.0f}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: - bool decode_32bit(uint32_t field_id, Proto32Bit value) override; - bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class VoiceAssistantRequest : public ProtoMessage { +class VoiceAssistantRequest final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 90; static constexpr uint8_t ESTIMATED_SIZE = 41; @@ -2255,19 +2339,21 @@ class VoiceAssistantRequest : public ProtoMessage { const char *message_name() const override { return "voice_assistant_request"; } #endif bool start{false}; - std::string conversation_id{}; + StringRef conversation_id_ref_{}; + void set_conversation_id(const StringRef &ref) { this->conversation_id_ref_ = ref; } uint32_t flags{0}; VoiceAssistantAudioSettings audio_settings{}; - std::string wake_word_phrase{}; + StringRef wake_word_phrase_ref_{}; + void set_wake_word_phrase(const StringRef &ref) { this->wake_word_phrase_ref_ = ref; } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class VoiceAssistantResponse : public ProtoMessage { +class VoiceAssistantResponse final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 91; static constexpr uint8_t ESTIMATED_SIZE = 6; @@ -2283,12 +2369,10 @@ class VoiceAssistantResponse : public ProtoMessage { protected: bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class VoiceAssistantEventData : public ProtoMessage { +class VoiceAssistantEventData final : public ProtoDecodableMessage { public: std::string name{}; std::string value{}; - void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -2296,7 +2380,7 @@ class VoiceAssistantEventData : public ProtoMessage { protected: bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; }; -class VoiceAssistantEventResponse : public ProtoMessage { +class VoiceAssistantEventResponse final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 92; static constexpr uint8_t ESTIMATED_SIZE = 36; @@ -2313,7 +2397,7 @@ class VoiceAssistantEventResponse : public ProtoMessage { bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class VoiceAssistantAudio : public ProtoMessage { +class VoiceAssistantAudio final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 106; static constexpr uint8_t ESTIMATED_SIZE = 11; @@ -2321,9 +2405,15 @@ class VoiceAssistantAudio : public ProtoMessage { const char *message_name() const override { return "voice_assistant_audio"; } #endif std::string data{}; + const uint8_t *data_ptr_{nullptr}; + size_t data_len_{0}; + void set_data(const uint8_t *data, size_t len) { + this->data_ptr_ = data; + this->data_len_ = len; + } bool end{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -2332,7 +2422,7 @@ class VoiceAssistantAudio : public ProtoMessage { bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class VoiceAssistantTimerEventResponse : public ProtoMessage { +class VoiceAssistantTimerEventResponse final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 115; static constexpr uint8_t ESTIMATED_SIZE = 30; @@ -2353,7 +2443,7 @@ class VoiceAssistantTimerEventResponse : public ProtoMessage { bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class VoiceAssistantAnnounceRequest : public ProtoMessage { +class VoiceAssistantAnnounceRequest final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 119; static constexpr uint8_t ESTIMATED_SIZE = 29; @@ -2372,7 +2462,7 @@ class VoiceAssistantAnnounceRequest : public ProtoMessage { bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; -class VoiceAssistantAnnounceFinished : public ProtoMessage { +class VoiceAssistantAnnounceFinished final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 120; static constexpr uint8_t ESTIMATED_SIZE = 2; @@ -2381,20 +2471,53 @@ class VoiceAssistantAnnounceFinished : public ProtoMessage { #endif bool success{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class VoiceAssistantWakeWord : public ProtoMessage { +class VoiceAssistantWakeWord final : public ProtoMessage { + public: + StringRef id_ref_{}; + void set_id(const StringRef &ref) { this->id_ref_ = ref; } + StringRef wake_word_ref_{}; + void set_wake_word(const StringRef &ref) { this->wake_word_ref_ = ref; } + std::vector trained_languages{}; + void encode(ProtoWriteBuffer buffer) const override; + void calculate_size(ProtoSize &size) const override; +#ifdef HAS_PROTO_MESSAGE_DUMP + void dump_to(std::string &out) const override; +#endif + + protected: +}; +class VoiceAssistantExternalWakeWord final : public ProtoDecodableMessage { public: std::string id{}; std::string wake_word{}; std::vector trained_languages{}; - void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + std::string model_type{}; + uint32_t model_size{0}; + std::string model_hash{}; + std::string url{}; +#ifdef HAS_PROTO_MESSAGE_DUMP + void dump_to(std::string &out) const override; +#endif + + protected: + bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; + bool decode_varint(uint32_t field_id, ProtoVarInt value) override; +}; +class VoiceAssistantConfigurationRequest final : public ProtoDecodableMessage { + public: + static constexpr uint8_t MESSAGE_TYPE = 121; + static constexpr uint8_t ESTIMATED_SIZE = 34; +#ifdef HAS_PROTO_MESSAGE_DUMP + const char *message_name() const override { return "voice_assistant_configuration_request"; } +#endif + std::vector external_wake_words{}; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -2402,20 +2525,7 @@ class VoiceAssistantWakeWord : public ProtoMessage { protected: bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; }; -class VoiceAssistantConfigurationRequest : public ProtoMessage { - public: - static constexpr uint8_t MESSAGE_TYPE = 121; - static constexpr uint8_t ESTIMATED_SIZE = 0; -#ifdef HAS_PROTO_MESSAGE_DUMP - const char *message_name() const override { return "voice_assistant_configuration_request"; } -#endif -#ifdef HAS_PROTO_MESSAGE_DUMP - void dump_to(std::string &out) const override; -#endif - - protected: -}; -class VoiceAssistantConfigurationResponse : public ProtoMessage { +class VoiceAssistantConfigurationResponse final : public ProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 122; static constexpr uint8_t ESTIMATED_SIZE = 56; @@ -2423,17 +2533,17 @@ class VoiceAssistantConfigurationResponse : public ProtoMessage { const char *message_name() const override { return "voice_assistant_configuration_response"; } #endif std::vector available_wake_words{}; - std::vector active_wake_words{}; + const std::vector *active_wake_words{}; uint32_t max_active_wake_words{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class VoiceAssistantSetConfiguration : public ProtoMessage { +class VoiceAssistantSetConfiguration final : public ProtoDecodableMessage { public: static constexpr uint8_t MESSAGE_TYPE = 123; static constexpr uint8_t ESTIMATED_SIZE = 18; @@ -2450,10 +2560,10 @@ class VoiceAssistantSetConfiguration : public ProtoMessage { }; #endif #ifdef USE_ALARM_CONTROL_PANEL -class ListEntitiesAlarmControlPanelResponse : public InfoResponseProtoMessage { +class ListEntitiesAlarmControlPanelResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 94; - static constexpr uint8_t ESTIMATED_SIZE = 57; + static constexpr uint8_t ESTIMATED_SIZE = 48; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_alarm_control_panel_response"; } #endif @@ -2461,14 +2571,14 @@ class ListEntitiesAlarmControlPanelResponse : public InfoResponseProtoMessage { bool requires_code{false}; bool requires_code_to_arm{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class AlarmControlPanelStateResponse : public StateResponseProtoMessage { +class AlarmControlPanelStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 95; static constexpr uint8_t ESTIMATED_SIZE = 11; @@ -2477,14 +2587,14 @@ class AlarmControlPanelStateResponse : public StateResponseProtoMessage { #endif enums::AlarmControlPanelState state{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class AlarmControlPanelCommandRequest : public CommandProtoMessage { +class AlarmControlPanelCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 96; static constexpr uint8_t ESTIMATED_SIZE = 20; @@ -2504,43 +2614,45 @@ class AlarmControlPanelCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_TEXT -class ListEntitiesTextResponse : public InfoResponseProtoMessage { +class ListEntitiesTextResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 97; - static constexpr uint8_t ESTIMATED_SIZE = 68; + static constexpr uint8_t ESTIMATED_SIZE = 59; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_text_response"; } #endif uint32_t min_length{0}; uint32_t max_length{0}; - std::string pattern{}; + StringRef pattern_ref_{}; + void set_pattern(const StringRef &ref) { this->pattern_ref_ = ref; } enums::TextMode mode{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class TextStateResponse : public StateResponseProtoMessage { +class TextStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 98; static constexpr uint8_t ESTIMATED_SIZE = 20; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "text_state_response"; } #endif - std::string state{}; + StringRef state_ref_{}; + void set_state(const StringRef &ref) { this->state_ref_ = ref; } bool missing_state{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class TextCommandRequest : public CommandProtoMessage { +class TextCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 99; static constexpr uint8_t ESTIMATED_SIZE = 18; @@ -2559,22 +2671,22 @@ class TextCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_DATETIME_DATE -class ListEntitiesDateResponse : public InfoResponseProtoMessage { +class ListEntitiesDateResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 100; - static constexpr uint8_t ESTIMATED_SIZE = 49; + static constexpr uint8_t ESTIMATED_SIZE = 40; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_date_response"; } #endif void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class DateStateResponse : public StateResponseProtoMessage { +class DateStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 101; static constexpr uint8_t ESTIMATED_SIZE = 23; @@ -2586,14 +2698,14 @@ class DateStateResponse : public StateResponseProtoMessage { uint32_t month{0}; uint32_t day{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class DateCommandRequest : public CommandProtoMessage { +class DateCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 102; static constexpr uint8_t ESTIMATED_SIZE = 21; @@ -2613,22 +2725,22 @@ class DateCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_DATETIME_TIME -class ListEntitiesTimeResponse : public InfoResponseProtoMessage { +class ListEntitiesTimeResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 103; - static constexpr uint8_t ESTIMATED_SIZE = 49; + static constexpr uint8_t ESTIMATED_SIZE = 40; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_time_response"; } #endif void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class TimeStateResponse : public StateResponseProtoMessage { +class TimeStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 104; static constexpr uint8_t ESTIMATED_SIZE = 23; @@ -2640,14 +2752,14 @@ class TimeStateResponse : public StateResponseProtoMessage { uint32_t minute{0}; uint32_t second{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class TimeCommandRequest : public CommandProtoMessage { +class TimeCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 105; static constexpr uint8_t ESTIMATED_SIZE = 21; @@ -2667,33 +2779,35 @@ class TimeCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_EVENT -class ListEntitiesEventResponse : public InfoResponseProtoMessage { +class ListEntitiesEventResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 107; - static constexpr uint8_t ESTIMATED_SIZE = 76; + static constexpr uint8_t ESTIMATED_SIZE = 67; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_event_response"; } #endif - std::string device_class{}; + StringRef device_class_ref_{}; + void set_device_class(const StringRef &ref) { this->device_class_ref_ = ref; } std::vector event_types{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class EventResponse : public StateResponseProtoMessage { +class EventResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 108; static constexpr uint8_t ESTIMATED_SIZE = 18; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "event_response"; } #endif - std::string event_type{}; + StringRef event_type_ref_{}; + void set_event_type(const StringRef &ref) { this->event_type_ref_ = ref; } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif @@ -2702,26 +2816,27 @@ class EventResponse : public StateResponseProtoMessage { }; #endif #ifdef USE_VALVE -class ListEntitiesValveResponse : public InfoResponseProtoMessage { +class ListEntitiesValveResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 109; - static constexpr uint8_t ESTIMATED_SIZE = 64; + static constexpr uint8_t ESTIMATED_SIZE = 55; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_valve_response"; } #endif - std::string device_class{}; + StringRef device_class_ref_{}; + void set_device_class(const StringRef &ref) { this->device_class_ref_ = ref; } bool assumed_state{false}; bool supports_position{false}; bool supports_stop{false}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class ValveStateResponse : public StateResponseProtoMessage { +class ValveStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 110; static constexpr uint8_t ESTIMATED_SIZE = 16; @@ -2731,14 +2846,14 @@ class ValveStateResponse : public StateResponseProtoMessage { float position{0.0f}; enums::ValveOperation current_operation{}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class ValveCommandRequest : public CommandProtoMessage { +class ValveCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 111; static constexpr uint8_t ESTIMATED_SIZE = 18; @@ -2758,22 +2873,22 @@ class ValveCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_DATETIME_DATETIME -class ListEntitiesDateTimeResponse : public InfoResponseProtoMessage { +class ListEntitiesDateTimeResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 112; - static constexpr uint8_t ESTIMATED_SIZE = 49; + static constexpr uint8_t ESTIMATED_SIZE = 40; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_date_time_response"; } #endif void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class DateTimeStateResponse : public StateResponseProtoMessage { +class DateTimeStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 113; static constexpr uint8_t ESTIMATED_SIZE = 16; @@ -2783,14 +2898,14 @@ class DateTimeStateResponse : public StateResponseProtoMessage { bool missing_state{false}; uint32_t epoch_seconds{0}; void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class DateTimeCommandRequest : public CommandProtoMessage { +class DateTimeCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 114; static constexpr uint8_t ESTIMATED_SIZE = 14; @@ -2808,23 +2923,24 @@ class DateTimeCommandRequest : public CommandProtoMessage { }; #endif #ifdef USE_UPDATE -class ListEntitiesUpdateResponse : public InfoResponseProtoMessage { +class ListEntitiesUpdateResponse final : public InfoResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 116; - static constexpr uint8_t ESTIMATED_SIZE = 58; + static constexpr uint8_t ESTIMATED_SIZE = 49; #ifdef HAS_PROTO_MESSAGE_DUMP const char *message_name() const override { return "list_entities_update_response"; } #endif - std::string device_class{}; + StringRef device_class_ref_{}; + void set_device_class(const StringRef &ref) { this->device_class_ref_ = ref; } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class UpdateStateResponse : public StateResponseProtoMessage { +class UpdateStateResponse final : public StateResponseProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 117; static constexpr uint8_t ESTIMATED_SIZE = 65; @@ -2835,20 +2951,25 @@ class UpdateStateResponse : public StateResponseProtoMessage { bool in_progress{false}; bool has_progress{false}; float progress{0.0f}; - std::string current_version{}; - std::string latest_version{}; - std::string title{}; - std::string release_summary{}; - std::string release_url{}; + StringRef current_version_ref_{}; + void set_current_version(const StringRef &ref) { this->current_version_ref_ = ref; } + StringRef latest_version_ref_{}; + void set_latest_version(const StringRef &ref) { this->latest_version_ref_ = ref; } + StringRef title_ref_{}; + void set_title(const StringRef &ref) { this->title_ref_ = ref; } + StringRef release_summary_ref_{}; + void set_release_summary(const StringRef &ref) { this->release_summary_ref_ = ref; } + StringRef release_url_ref_{}; + void set_release_url(const StringRef &ref) { this->release_url_ref_ = ref; } void encode(ProtoWriteBuffer buffer) const override; - void calculate_size(uint32_t &total_size) const override; + void calculate_size(ProtoSize &size) const override; #ifdef HAS_PROTO_MESSAGE_DUMP void dump_to(std::string &out) const override; #endif protected: }; -class UpdateCommandRequest : public CommandProtoMessage { +class UpdateCommandRequest final : public CommandProtoMessage { public: static constexpr uint8_t MESSAGE_TYPE = 118; static constexpr uint8_t ESTIMATED_SIZE = 11; @@ -2864,7 +2985,46 @@ class UpdateCommandRequest : public CommandProtoMessage { bool decode_32bit(uint32_t field_id, Proto32Bit value) override; bool decode_varint(uint32_t field_id, ProtoVarInt value) override; }; +#endif +#ifdef USE_ZWAVE_PROXY +class ZWaveProxyFrame final : public ProtoDecodableMessage { + public: + static constexpr uint8_t MESSAGE_TYPE = 128; + static constexpr uint8_t ESTIMATED_SIZE = 19; +#ifdef HAS_PROTO_MESSAGE_DUMP + const char *message_name() const override { return "z_wave_proxy_frame"; } +#endif + const uint8_t *data{nullptr}; + uint16_t data_len{0}; + void encode(ProtoWriteBuffer buffer) const override; + void calculate_size(ProtoSize &size) const override; +#ifdef HAS_PROTO_MESSAGE_DUMP + void dump_to(std::string &out) const override; #endif -} // namespace api -} // namespace esphome + protected: + bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; +}; +class ZWaveProxyRequest final : public ProtoDecodableMessage { + public: + static constexpr uint8_t MESSAGE_TYPE = 129; + static constexpr uint8_t ESTIMATED_SIZE = 21; +#ifdef HAS_PROTO_MESSAGE_DUMP + const char *message_name() const override { return "z_wave_proxy_request"; } +#endif + enums::ZWaveProxyRequestType type{}; + const uint8_t *data{nullptr}; + uint16_t data_len{0}; + void encode(ProtoWriteBuffer buffer) const override; + void calculate_size(ProtoSize &size) const override; +#ifdef HAS_PROTO_MESSAGE_DUMP + void dump_to(std::string &out) const override; +#endif + + protected: + bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override; + bool decode_varint(uint32_t field_id, ProtoVarInt value) override; +}; +#endif + +} // namespace esphome::api diff --git a/esphome/components/api/api_pb2_dump.cpp b/esphome/components/api/api_pb2_dump.cpp index f6509f47cc..e803125f53 100644 --- a/esphome/components/api/api_pb2_dump.cpp +++ b/esphome/components/api/api_pb2_dump.cpp @@ -7,8 +7,92 @@ #ifdef HAS_PROTO_MESSAGE_DUMP -namespace esphome { -namespace api { +namespace esphome::api { + +// Helper function to append a quoted string, handling empty StringRef +static inline void append_quoted_string(std::string &out, const StringRef &ref) { + out.append("'"); + if (!ref.empty()) { + out.append(ref.c_str()); + } + out.append("'"); +} + +// Common helpers for dump_field functions +static inline void append_field_prefix(std::string &out, const char *field_name, int indent) { + out.append(indent, ' ').append(field_name).append(": "); +} + +static inline void append_with_newline(std::string &out, const char *str) { + out.append(str); + out.append("\n"); +} + +// RAII helper for message dump formatting +class MessageDumpHelper { + public: + MessageDumpHelper(std::string &out, const char *message_name) : out_(out) { + out_.append(message_name); + out_.append(" {\n"); + } + ~MessageDumpHelper() { out_.append(" }"); } + + private: + std::string &out_; +}; + +// Helper functions to reduce code duplication in dump methods +static void dump_field(std::string &out, const char *field_name, int32_t value, int indent = 2) { + char buffer[64]; + append_field_prefix(out, field_name, indent); + snprintf(buffer, 64, "%" PRId32, value); + append_with_newline(out, buffer); +} + +static void dump_field(std::string &out, const char *field_name, uint32_t value, int indent = 2) { + char buffer[64]; + append_field_prefix(out, field_name, indent); + snprintf(buffer, 64, "%" PRIu32, value); + append_with_newline(out, buffer); +} + +static void dump_field(std::string &out, const char *field_name, float value, int indent = 2) { + char buffer[64]; + append_field_prefix(out, field_name, indent); + snprintf(buffer, 64, "%g", value); + append_with_newline(out, buffer); +} + +static void dump_field(std::string &out, const char *field_name, uint64_t value, int indent = 2) { + char buffer[64]; + append_field_prefix(out, field_name, indent); + snprintf(buffer, 64, "%llu", value); + append_with_newline(out, buffer); +} + +static void dump_field(std::string &out, const char *field_name, bool value, int indent = 2) { + append_field_prefix(out, field_name, indent); + out.append(YESNO(value)); + out.append("\n"); +} + +static void dump_field(std::string &out, const char *field_name, const std::string &value, int indent = 2) { + append_field_prefix(out, field_name, indent); + out.append("'").append(value).append("'"); + out.append("\n"); +} + +static void dump_field(std::string &out, const char *field_name, StringRef value, int indent = 2) { + append_field_prefix(out, field_name, indent); + append_quoted_string(out, value); + out.append("\n"); +} + +template static void dump_field(std::string &out, const char *field_name, T value, int indent = 2) { + append_field_prefix(out, field_name, indent); + out.append(proto_enum_to_string(value)); + out.append("\n"); +} template<> const char *proto_enum_to_string(enums::EntityCategory value) { switch (value) { @@ -23,16 +107,6 @@ template<> const char *proto_enum_to_string(enums::Entity } } #ifdef USE_COVER -template<> const char *proto_enum_to_string(enums::LegacyCoverState value) { - switch (value) { - case enums::LEGACY_COVER_STATE_OPEN: - return "LEGACY_COVER_STATE_OPEN"; - case enums::LEGACY_COVER_STATE_CLOSED: - return "LEGACY_COVER_STATE_CLOSED"; - default: - return "UNKNOWN"; - } -} template<> const char *proto_enum_to_string(enums::CoverOperation value) { switch (value) { case enums::COVER_OPERATION_IDLE: @@ -45,32 +119,8 @@ template<> const char *proto_enum_to_string(enums::CoverO return "UNKNOWN"; } } -template<> const char *proto_enum_to_string(enums::LegacyCoverCommand value) { - switch (value) { - case enums::LEGACY_COVER_COMMAND_OPEN: - return "LEGACY_COVER_COMMAND_OPEN"; - case enums::LEGACY_COVER_COMMAND_CLOSE: - return "LEGACY_COVER_COMMAND_CLOSE"; - case enums::LEGACY_COVER_COMMAND_STOP: - return "LEGACY_COVER_COMMAND_STOP"; - default: - return "UNKNOWN"; - } -} #endif #ifdef USE_FAN -template<> const char *proto_enum_to_string(enums::FanSpeed value) { - switch (value) { - case enums::FAN_SPEED_LOW: - return "FAN_SPEED_LOW"; - case enums::FAN_SPEED_MEDIUM: - return "FAN_SPEED_MEDIUM"; - case enums::FAN_SPEED_HIGH: - return "FAN_SPEED_HIGH"; - default: - return "UNKNOWN"; - } -} template<> const char *proto_enum_to_string(enums::FanDirection value) { switch (value) { case enums::FAN_DIRECTION_FORWARD: @@ -127,18 +177,6 @@ template<> const char *proto_enum_to_string(enums::Sens return "UNKNOWN"; } } -template<> const char *proto_enum_to_string(enums::SensorLastResetType value) { - switch (value) { - case enums::LAST_RESET_NONE: - return "LAST_RESET_NONE"; - case enums::LAST_RESET_NEVER: - return "LAST_RESET_NEVER"; - case enums::LAST_RESET_AUTO: - return "LAST_RESET_AUTO"; - default: - return "UNKNOWN"; - } -} #endif template<> const char *proto_enum_to_string(enums::LogLevel value) { switch (value) { @@ -345,6 +383,12 @@ template<> const char *proto_enum_to_string(enums::Medi return "MEDIA_PLAYER_STATE_PLAYING"; case enums::MEDIA_PLAYER_STATE_PAUSED: return "MEDIA_PLAYER_STATE_PAUSED"; + case enums::MEDIA_PLAYER_STATE_ANNOUNCING: + return "MEDIA_PLAYER_STATE_ANNOUNCING"; + case enums::MEDIA_PLAYER_STATE_OFF: + return "MEDIA_PLAYER_STATE_OFF"; + case enums::MEDIA_PLAYER_STATE_ON: + return "MEDIA_PLAYER_STATE_ON"; default: return "UNKNOWN"; } @@ -361,6 +405,24 @@ template<> const char *proto_enum_to_string(enums::Me return "MEDIA_PLAYER_COMMAND_MUTE"; case enums::MEDIA_PLAYER_COMMAND_UNMUTE: return "MEDIA_PLAYER_COMMAND_UNMUTE"; + case enums::MEDIA_PLAYER_COMMAND_TOGGLE: + return "MEDIA_PLAYER_COMMAND_TOGGLE"; + case enums::MEDIA_PLAYER_COMMAND_VOLUME_UP: + return "MEDIA_PLAYER_COMMAND_VOLUME_UP"; + case enums::MEDIA_PLAYER_COMMAND_VOLUME_DOWN: + return "MEDIA_PLAYER_COMMAND_VOLUME_DOWN"; + case enums::MEDIA_PLAYER_COMMAND_ENQUEUE: + return "MEDIA_PLAYER_COMMAND_ENQUEUE"; + case enums::MEDIA_PLAYER_COMMAND_REPEAT_ONE: + return "MEDIA_PLAYER_COMMAND_REPEAT_ONE"; + case enums::MEDIA_PLAYER_COMMAND_REPEAT_OFF: + return "MEDIA_PLAYER_COMMAND_REPEAT_OFF"; + case enums::MEDIA_PLAYER_COMMAND_CLEAR_PLAYLIST: + return "MEDIA_PLAYER_COMMAND_CLEAR_PLAYLIST"; + case enums::MEDIA_PLAYER_COMMAND_TURN_ON: + return "MEDIA_PLAYER_COMMAND_TURN_ON"; + case enums::MEDIA_PLAYER_COMMAND_TURN_OFF: + return "MEDIA_PLAYER_COMMAND_TURN_OFF"; default: return "UNKNOWN"; } @@ -593,3840 +655,1543 @@ template<> const char *proto_enum_to_string(enums::UpdateC } } #endif +#ifdef USE_ZWAVE_PROXY +template<> const char *proto_enum_to_string(enums::ZWaveProxyRequestType value) { + switch (value) { + case enums::ZWAVE_PROXY_REQUEST_TYPE_SUBSCRIBE: + return "ZWAVE_PROXY_REQUEST_TYPE_SUBSCRIBE"; + case enums::ZWAVE_PROXY_REQUEST_TYPE_UNSUBSCRIBE: + return "ZWAVE_PROXY_REQUEST_TYPE_UNSUBSCRIBE"; + case enums::ZWAVE_PROXY_REQUEST_TYPE_HOME_ID_CHANGE: + return "ZWAVE_PROXY_REQUEST_TYPE_HOME_ID_CHANGE"; + default: + return "UNKNOWN"; + } +} +#endif void HelloRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("HelloRequest {\n"); + MessageDumpHelper helper(out, "HelloRequest"); out.append(" client_info: "); - out.append("'").append(this->client_info).append("'"); + out.append(format_hex_pretty(this->client_info, this->client_info_len)); out.append("\n"); - - out.append(" api_version_major: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->api_version_major); - out.append(buffer); - out.append("\n"); - - out.append(" api_version_minor: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->api_version_minor); - out.append(buffer); - out.append("\n"); - out.append("}"); + dump_field(out, "api_version_major", this->api_version_major); + dump_field(out, "api_version_minor", this->api_version_minor); } void HelloResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("HelloResponse {\n"); - out.append(" api_version_major: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->api_version_major); - out.append(buffer); - out.append("\n"); - - out.append(" api_version_minor: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->api_version_minor); - out.append(buffer); - out.append("\n"); - - out.append(" server_info: "); - out.append("'").append(this->server_info).append("'"); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "HelloResponse"); + dump_field(out, "api_version_major", this->api_version_major); + dump_field(out, "api_version_minor", this->api_version_minor); + dump_field(out, "server_info", this->server_info_ref_); + dump_field(out, "name", this->name_ref_); } -void ConnectRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ConnectRequest {\n"); +#ifdef USE_API_PASSWORD +void AuthenticationRequest::dump_to(std::string &out) const { + MessageDumpHelper helper(out, "AuthenticationRequest"); out.append(" password: "); - out.append("'").append(this->password).append("'"); + out.append(format_hex_pretty(this->password, this->password_len)); out.append("\n"); - out.append("}"); } -void ConnectResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ConnectResponse {\n"); - out.append(" invalid_password: "); - out.append(YESNO(this->invalid_password)); - out.append("\n"); - out.append("}"); +void AuthenticationResponse::dump_to(std::string &out) const { + MessageDumpHelper helper(out, "AuthenticationResponse"); + dump_field(out, "invalid_password", this->invalid_password); } +#endif void DisconnectRequest::dump_to(std::string &out) const { out.append("DisconnectRequest {}"); } void DisconnectResponse::dump_to(std::string &out) const { out.append("DisconnectResponse {}"); } void PingRequest::dump_to(std::string &out) const { out.append("PingRequest {}"); } void PingResponse::dump_to(std::string &out) const { out.append("PingResponse {}"); } void DeviceInfoRequest::dump_to(std::string &out) const { out.append("DeviceInfoRequest {}"); } +#ifdef USE_AREAS void AreaInfo::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("AreaInfo {\n"); - out.append(" area_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->area_id); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "AreaInfo"); + dump_field(out, "area_id", this->area_id); + dump_field(out, "name", this->name_ref_); } +#endif +#ifdef USE_DEVICES void DeviceInfo::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("DeviceInfo {\n"); - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" area_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->area_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "DeviceInfo"); + dump_field(out, "device_id", this->device_id); + dump_field(out, "name", this->name_ref_); + dump_field(out, "area_id", this->area_id); } +#endif void DeviceInfoResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("DeviceInfoResponse {\n"); - out.append(" uses_password: "); - out.append(YESNO(this->uses_password)); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" mac_address: "); - out.append("'").append(this->mac_address).append("'"); - out.append("\n"); - - out.append(" esphome_version: "); - out.append("'").append(this->esphome_version).append("'"); - out.append("\n"); - - out.append(" compilation_time: "); - out.append("'").append(this->compilation_time).append("'"); - out.append("\n"); - - out.append(" model: "); - out.append("'").append(this->model).append("'"); - out.append("\n"); - - out.append(" has_deep_sleep: "); - out.append(YESNO(this->has_deep_sleep)); - out.append("\n"); - - out.append(" project_name: "); - out.append("'").append(this->project_name).append("'"); - out.append("\n"); - - out.append(" project_version: "); - out.append("'").append(this->project_version).append("'"); - out.append("\n"); - - out.append(" webserver_port: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->webserver_port); - out.append(buffer); - out.append("\n"); - - out.append(" legacy_bluetooth_proxy_version: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->legacy_bluetooth_proxy_version); - out.append(buffer); - out.append("\n"); - - out.append(" bluetooth_proxy_feature_flags: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->bluetooth_proxy_feature_flags); - out.append(buffer); - out.append("\n"); - - out.append(" manufacturer: "); - out.append("'").append(this->manufacturer).append("'"); - out.append("\n"); - - out.append(" friendly_name: "); - out.append("'").append(this->friendly_name).append("'"); - out.append("\n"); - - out.append(" legacy_voice_assistant_version: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->legacy_voice_assistant_version); - out.append(buffer); - out.append("\n"); - - out.append(" voice_assistant_feature_flags: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->voice_assistant_feature_flags); - out.append(buffer); - out.append("\n"); - - out.append(" suggested_area: "); - out.append("'").append(this->suggested_area).append("'"); - out.append("\n"); - - out.append(" bluetooth_mac_address: "); - out.append("'").append(this->bluetooth_mac_address).append("'"); - out.append("\n"); - - out.append(" api_encryption_supported: "); - out.append(YESNO(this->api_encryption_supported)); - out.append("\n"); - + MessageDumpHelper helper(out, "DeviceInfoResponse"); +#ifdef USE_API_PASSWORD + dump_field(out, "uses_password", this->uses_password); +#endif + dump_field(out, "name", this->name_ref_); + dump_field(out, "mac_address", this->mac_address_ref_); + dump_field(out, "esphome_version", this->esphome_version_ref_); + dump_field(out, "compilation_time", this->compilation_time_ref_); + dump_field(out, "model", this->model_ref_); +#ifdef USE_DEEP_SLEEP + dump_field(out, "has_deep_sleep", this->has_deep_sleep); +#endif +#ifdef ESPHOME_PROJECT_NAME + dump_field(out, "project_name", this->project_name_ref_); +#endif +#ifdef ESPHOME_PROJECT_NAME + dump_field(out, "project_version", this->project_version_ref_); +#endif +#ifdef USE_WEBSERVER + dump_field(out, "webserver_port", this->webserver_port); +#endif +#ifdef USE_BLUETOOTH_PROXY + dump_field(out, "bluetooth_proxy_feature_flags", this->bluetooth_proxy_feature_flags); +#endif + dump_field(out, "manufacturer", this->manufacturer_ref_); + dump_field(out, "friendly_name", this->friendly_name_ref_); +#ifdef USE_VOICE_ASSISTANT + dump_field(out, "voice_assistant_feature_flags", this->voice_assistant_feature_flags); +#endif +#ifdef USE_AREAS + dump_field(out, "suggested_area", this->suggested_area_ref_); +#endif +#ifdef USE_BLUETOOTH_PROXY + dump_field(out, "bluetooth_mac_address", this->bluetooth_mac_address_ref_); +#endif +#ifdef USE_API_NOISE + dump_field(out, "api_encryption_supported", this->api_encryption_supported); +#endif +#ifdef USE_DEVICES for (const auto &it : this->devices) { out.append(" devices: "); it.dump_to(out); out.append("\n"); } - +#endif +#ifdef USE_AREAS for (const auto &it : this->areas) { out.append(" areas: "); it.dump_to(out); out.append("\n"); } - +#endif +#ifdef USE_AREAS out.append(" area: "); this->area.dump_to(out); out.append("\n"); - out.append("}"); +#endif +#ifdef USE_ZWAVE_PROXY + dump_field(out, "zwave_proxy_feature_flags", this->zwave_proxy_feature_flags); +#endif +#ifdef USE_ZWAVE_PROXY + dump_field(out, "zwave_home_id", this->zwave_home_id); +#endif } void ListEntitiesRequest::dump_to(std::string &out) const { out.append("ListEntitiesRequest {}"); } void ListEntitiesDoneResponse::dump_to(std::string &out) const { out.append("ListEntitiesDoneResponse {}"); } void SubscribeStatesRequest::dump_to(std::string &out) const { out.append("SubscribeStatesRequest {}"); } #ifdef USE_BINARY_SENSOR void ListEntitiesBinarySensorResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesBinarySensorResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" device_class: "); - out.append("'").append(this->device_class).append("'"); - out.append("\n"); - - out.append(" is_status_binary_sensor: "); - out.append(YESNO(this->is_status_binary_sensor)); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesBinarySensorResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); + dump_field(out, "device_class", this->device_class_ref_); + dump_field(out, "is_status_binary_sensor", this->is_status_binary_sensor); + dump_field(out, "disabled_by_default", this->disabled_by_default); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "entity_category", static_cast(this->entity_category)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void BinarySensorStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BinarySensorStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append(YESNO(this->state)); - out.append("\n"); - - out.append(" missing_state: "); - out.append(YESNO(this->missing_state)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BinarySensorStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state); + dump_field(out, "missing_state", this->missing_state); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_COVER void ListEntitiesCoverResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesCoverResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" assumed_state: "); - out.append(YESNO(this->assumed_state)); - out.append("\n"); - - out.append(" supports_position: "); - out.append(YESNO(this->supports_position)); - out.append("\n"); - - out.append(" supports_tilt: "); - out.append(YESNO(this->supports_tilt)); - out.append("\n"); - - out.append(" device_class: "); - out.append("'").append(this->device_class).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" supports_stop: "); - out.append(YESNO(this->supports_stop)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesCoverResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); + dump_field(out, "assumed_state", this->assumed_state); + dump_field(out, "supports_position", this->supports_position); + dump_field(out, "supports_tilt", this->supports_tilt); + dump_field(out, "device_class", this->device_class_ref_); + dump_field(out, "disabled_by_default", this->disabled_by_default); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "entity_category", static_cast(this->entity_category)); + dump_field(out, "supports_stop", this->supports_stop); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void CoverStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("CoverStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" legacy_state: "); - out.append(proto_enum_to_string(this->legacy_state)); - out.append("\n"); - - out.append(" position: "); - snprintf(buffer, sizeof(buffer), "%g", this->position); - out.append(buffer); - out.append("\n"); - - out.append(" tilt: "); - snprintf(buffer, sizeof(buffer), "%g", this->tilt); - out.append(buffer); - out.append("\n"); - - out.append(" current_operation: "); - out.append(proto_enum_to_string(this->current_operation)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "CoverStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "position", this->position); + dump_field(out, "tilt", this->tilt); + dump_field(out, "current_operation", static_cast(this->current_operation)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void CoverCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("CoverCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" has_legacy_command: "); - out.append(YESNO(this->has_legacy_command)); - out.append("\n"); - - out.append(" legacy_command: "); - out.append(proto_enum_to_string(this->legacy_command)); - out.append("\n"); - - out.append(" has_position: "); - out.append(YESNO(this->has_position)); - out.append("\n"); - - out.append(" position: "); - snprintf(buffer, sizeof(buffer), "%g", this->position); - out.append(buffer); - out.append("\n"); - - out.append(" has_tilt: "); - out.append(YESNO(this->has_tilt)); - out.append("\n"); - - out.append(" tilt: "); - snprintf(buffer, sizeof(buffer), "%g", this->tilt); - out.append(buffer); - out.append("\n"); - - out.append(" stop: "); - out.append(YESNO(this->stop)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "CoverCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "has_position", this->has_position); + dump_field(out, "position", this->position); + dump_field(out, "has_tilt", this->has_tilt); + dump_field(out, "tilt", this->tilt); + dump_field(out, "stop", this->stop); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_FAN void ListEntitiesFanResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesFanResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" supports_oscillation: "); - out.append(YESNO(this->supports_oscillation)); - out.append("\n"); - - out.append(" supports_speed: "); - out.append(YESNO(this->supports_speed)); - out.append("\n"); - - out.append(" supports_direction: "); - out.append(YESNO(this->supports_direction)); - out.append("\n"); - - out.append(" supported_speed_count: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, this->supported_speed_count); - out.append(buffer); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - for (const auto &it : this->supported_preset_modes) { - out.append(" supported_preset_modes: "); - out.append("'").append(it).append("'"); - out.append("\n"); + MessageDumpHelper helper(out, "ListEntitiesFanResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); + dump_field(out, "supports_oscillation", this->supports_oscillation); + dump_field(out, "supports_speed", this->supports_speed); + dump_field(out, "supports_direction", this->supports_direction); + dump_field(out, "supported_speed_count", this->supported_speed_count); + dump_field(out, "disabled_by_default", this->disabled_by_default); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "entity_category", static_cast(this->entity_category)); + for (const auto &it : *this->supported_preset_modes) { + dump_field(out, "supported_preset_modes", it, 4); } - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void FanStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("FanStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append(YESNO(this->state)); - out.append("\n"); - - out.append(" oscillating: "); - out.append(YESNO(this->oscillating)); - out.append("\n"); - - out.append(" speed: "); - out.append(proto_enum_to_string(this->speed)); - out.append("\n"); - - out.append(" direction: "); - out.append(proto_enum_to_string(this->direction)); - out.append("\n"); - - out.append(" speed_level: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, this->speed_level); - out.append(buffer); - out.append("\n"); - - out.append(" preset_mode: "); - out.append("'").append(this->preset_mode).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "FanStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state); + dump_field(out, "oscillating", this->oscillating); + dump_field(out, "direction", static_cast(this->direction)); + dump_field(out, "speed_level", this->speed_level); + dump_field(out, "preset_mode", this->preset_mode_ref_); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void FanCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("FanCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" has_state: "); - out.append(YESNO(this->has_state)); - out.append("\n"); - - out.append(" state: "); - out.append(YESNO(this->state)); - out.append("\n"); - - out.append(" has_speed: "); - out.append(YESNO(this->has_speed)); - out.append("\n"); - - out.append(" speed: "); - out.append(proto_enum_to_string(this->speed)); - out.append("\n"); - - out.append(" has_oscillating: "); - out.append(YESNO(this->has_oscillating)); - out.append("\n"); - - out.append(" oscillating: "); - out.append(YESNO(this->oscillating)); - out.append("\n"); - - out.append(" has_direction: "); - out.append(YESNO(this->has_direction)); - out.append("\n"); - - out.append(" direction: "); - out.append(proto_enum_to_string(this->direction)); - out.append("\n"); - - out.append(" has_speed_level: "); - out.append(YESNO(this->has_speed_level)); - out.append("\n"); - - out.append(" speed_level: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, this->speed_level); - out.append(buffer); - out.append("\n"); - - out.append(" has_preset_mode: "); - out.append(YESNO(this->has_preset_mode)); - out.append("\n"); - - out.append(" preset_mode: "); - out.append("'").append(this->preset_mode).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "FanCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "has_state", this->has_state); + dump_field(out, "state", this->state); + dump_field(out, "has_oscillating", this->has_oscillating); + dump_field(out, "oscillating", this->oscillating); + dump_field(out, "has_direction", this->has_direction); + dump_field(out, "direction", static_cast(this->direction)); + dump_field(out, "has_speed_level", this->has_speed_level); + dump_field(out, "speed_level", this->speed_level); + dump_field(out, "has_preset_mode", this->has_preset_mode); + dump_field(out, "preset_mode", this->preset_mode); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_LIGHT void ListEntitiesLightResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesLightResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - for (const auto &it : this->supported_color_modes) { - out.append(" supported_color_modes: "); - out.append(proto_enum_to_string(it)); - out.append("\n"); + MessageDumpHelper helper(out, "ListEntitiesLightResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); + for (const auto &it : *this->supported_color_modes) { + dump_field(out, "supported_color_modes", static_cast(it), 4); } - - out.append(" legacy_supports_brightness: "); - out.append(YESNO(this->legacy_supports_brightness)); - out.append("\n"); - - out.append(" legacy_supports_rgb: "); - out.append(YESNO(this->legacy_supports_rgb)); - out.append("\n"); - - out.append(" legacy_supports_white_value: "); - out.append(YESNO(this->legacy_supports_white_value)); - out.append("\n"); - - out.append(" legacy_supports_color_temperature: "); - out.append(YESNO(this->legacy_supports_color_temperature)); - out.append("\n"); - - out.append(" min_mireds: "); - snprintf(buffer, sizeof(buffer), "%g", this->min_mireds); - out.append(buffer); - out.append("\n"); - - out.append(" max_mireds: "); - snprintf(buffer, sizeof(buffer), "%g", this->max_mireds); - out.append(buffer); - out.append("\n"); - + dump_field(out, "min_mireds", this->min_mireds); + dump_field(out, "max_mireds", this->max_mireds); for (const auto &it : this->effects) { - out.append(" effects: "); - out.append("'").append(it).append("'"); - out.append("\n"); + dump_field(out, "effects", it, 4); } - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + dump_field(out, "disabled_by_default", this->disabled_by_default); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "entity_category", static_cast(this->entity_category)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void LightStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("LightStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append(YESNO(this->state)); - out.append("\n"); - - out.append(" brightness: "); - snprintf(buffer, sizeof(buffer), "%g", this->brightness); - out.append(buffer); - out.append("\n"); - - out.append(" color_mode: "); - out.append(proto_enum_to_string(this->color_mode)); - out.append("\n"); - - out.append(" color_brightness: "); - snprintf(buffer, sizeof(buffer), "%g", this->color_brightness); - out.append(buffer); - out.append("\n"); - - out.append(" red: "); - snprintf(buffer, sizeof(buffer), "%g", this->red); - out.append(buffer); - out.append("\n"); - - out.append(" green: "); - snprintf(buffer, sizeof(buffer), "%g", this->green); - out.append(buffer); - out.append("\n"); - - out.append(" blue: "); - snprintf(buffer, sizeof(buffer), "%g", this->blue); - out.append(buffer); - out.append("\n"); - - out.append(" white: "); - snprintf(buffer, sizeof(buffer), "%g", this->white); - out.append(buffer); - out.append("\n"); - - out.append(" color_temperature: "); - snprintf(buffer, sizeof(buffer), "%g", this->color_temperature); - out.append(buffer); - out.append("\n"); - - out.append(" cold_white: "); - snprintf(buffer, sizeof(buffer), "%g", this->cold_white); - out.append(buffer); - out.append("\n"); - - out.append(" warm_white: "); - snprintf(buffer, sizeof(buffer), "%g", this->warm_white); - out.append(buffer); - out.append("\n"); - - out.append(" effect: "); - out.append("'").append(this->effect).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "LightStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state); + dump_field(out, "brightness", this->brightness); + dump_field(out, "color_mode", static_cast(this->color_mode)); + dump_field(out, "color_brightness", this->color_brightness); + dump_field(out, "red", this->red); + dump_field(out, "green", this->green); + dump_field(out, "blue", this->blue); + dump_field(out, "white", this->white); + dump_field(out, "color_temperature", this->color_temperature); + dump_field(out, "cold_white", this->cold_white); + dump_field(out, "warm_white", this->warm_white); + dump_field(out, "effect", this->effect_ref_); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void LightCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("LightCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" has_state: "); - out.append(YESNO(this->has_state)); - out.append("\n"); - - out.append(" state: "); - out.append(YESNO(this->state)); - out.append("\n"); - - out.append(" has_brightness: "); - out.append(YESNO(this->has_brightness)); - out.append("\n"); - - out.append(" brightness: "); - snprintf(buffer, sizeof(buffer), "%g", this->brightness); - out.append(buffer); - out.append("\n"); - - out.append(" has_color_mode: "); - out.append(YESNO(this->has_color_mode)); - out.append("\n"); - - out.append(" color_mode: "); - out.append(proto_enum_to_string(this->color_mode)); - out.append("\n"); - - out.append(" has_color_brightness: "); - out.append(YESNO(this->has_color_brightness)); - out.append("\n"); - - out.append(" color_brightness: "); - snprintf(buffer, sizeof(buffer), "%g", this->color_brightness); - out.append(buffer); - out.append("\n"); - - out.append(" has_rgb: "); - out.append(YESNO(this->has_rgb)); - out.append("\n"); - - out.append(" red: "); - snprintf(buffer, sizeof(buffer), "%g", this->red); - out.append(buffer); - out.append("\n"); - - out.append(" green: "); - snprintf(buffer, sizeof(buffer), "%g", this->green); - out.append(buffer); - out.append("\n"); - - out.append(" blue: "); - snprintf(buffer, sizeof(buffer), "%g", this->blue); - out.append(buffer); - out.append("\n"); - - out.append(" has_white: "); - out.append(YESNO(this->has_white)); - out.append("\n"); - - out.append(" white: "); - snprintf(buffer, sizeof(buffer), "%g", this->white); - out.append(buffer); - out.append("\n"); - - out.append(" has_color_temperature: "); - out.append(YESNO(this->has_color_temperature)); - out.append("\n"); - - out.append(" color_temperature: "); - snprintf(buffer, sizeof(buffer), "%g", this->color_temperature); - out.append(buffer); - out.append("\n"); - - out.append(" has_cold_white: "); - out.append(YESNO(this->has_cold_white)); - out.append("\n"); - - out.append(" cold_white: "); - snprintf(buffer, sizeof(buffer), "%g", this->cold_white); - out.append(buffer); - out.append("\n"); - - out.append(" has_warm_white: "); - out.append(YESNO(this->has_warm_white)); - out.append("\n"); - - out.append(" warm_white: "); - snprintf(buffer, sizeof(buffer), "%g", this->warm_white); - out.append(buffer); - out.append("\n"); - - out.append(" has_transition_length: "); - out.append(YESNO(this->has_transition_length)); - out.append("\n"); - - out.append(" transition_length: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->transition_length); - out.append(buffer); - out.append("\n"); - - out.append(" has_flash_length: "); - out.append(YESNO(this->has_flash_length)); - out.append("\n"); - - out.append(" flash_length: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->flash_length); - out.append(buffer); - out.append("\n"); - - out.append(" has_effect: "); - out.append(YESNO(this->has_effect)); - out.append("\n"); - - out.append(" effect: "); - out.append("'").append(this->effect).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "LightCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "has_state", this->has_state); + dump_field(out, "state", this->state); + dump_field(out, "has_brightness", this->has_brightness); + dump_field(out, "brightness", this->brightness); + dump_field(out, "has_color_mode", this->has_color_mode); + dump_field(out, "color_mode", static_cast(this->color_mode)); + dump_field(out, "has_color_brightness", this->has_color_brightness); + dump_field(out, "color_brightness", this->color_brightness); + dump_field(out, "has_rgb", this->has_rgb); + dump_field(out, "red", this->red); + dump_field(out, "green", this->green); + dump_field(out, "blue", this->blue); + dump_field(out, "has_white", this->has_white); + dump_field(out, "white", this->white); + dump_field(out, "has_color_temperature", this->has_color_temperature); + dump_field(out, "color_temperature", this->color_temperature); + dump_field(out, "has_cold_white", this->has_cold_white); + dump_field(out, "cold_white", this->cold_white); + dump_field(out, "has_warm_white", this->has_warm_white); + dump_field(out, "warm_white", this->warm_white); + dump_field(out, "has_transition_length", this->has_transition_length); + dump_field(out, "transition_length", this->transition_length); + dump_field(out, "has_flash_length", this->has_flash_length); + dump_field(out, "flash_length", this->flash_length); + dump_field(out, "has_effect", this->has_effect); + dump_field(out, "effect", this->effect); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_SENSOR void ListEntitiesSensorResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesSensorResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" unit_of_measurement: "); - out.append("'").append(this->unit_of_measurement).append("'"); - out.append("\n"); - - out.append(" accuracy_decimals: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, this->accuracy_decimals); - out.append(buffer); - out.append("\n"); - - out.append(" force_update: "); - out.append(YESNO(this->force_update)); - out.append("\n"); - - out.append(" device_class: "); - out.append("'").append(this->device_class).append("'"); - out.append("\n"); - - out.append(" state_class: "); - out.append(proto_enum_to_string(this->state_class)); - out.append("\n"); - - out.append(" legacy_last_reset_type: "); - out.append(proto_enum_to_string(this->legacy_last_reset_type)); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesSensorResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "unit_of_measurement", this->unit_of_measurement_ref_); + dump_field(out, "accuracy_decimals", this->accuracy_decimals); + dump_field(out, "force_update", this->force_update); + dump_field(out, "device_class", this->device_class_ref_); + dump_field(out, "state_class", static_cast(this->state_class)); + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void SensorStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("SensorStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - snprintf(buffer, sizeof(buffer), "%g", this->state); - out.append(buffer); - out.append("\n"); - - out.append(" missing_state: "); - out.append(YESNO(this->missing_state)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "SensorStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state); + dump_field(out, "missing_state", this->missing_state); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_SWITCH void ListEntitiesSwitchResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesSwitchResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" assumed_state: "); - out.append(YESNO(this->assumed_state)); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_class: "); - out.append("'").append(this->device_class).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesSwitchResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "assumed_state", this->assumed_state); + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); + dump_field(out, "device_class", this->device_class_ref_); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void SwitchStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("SwitchStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append(YESNO(this->state)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "SwitchStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void SwitchCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("SwitchCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append(YESNO(this->state)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "SwitchCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_TEXT_SENSOR void ListEntitiesTextSensorResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesTextSensorResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_class: "); - out.append("'").append(this->device_class).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesTextSensorResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); + dump_field(out, "device_class", this->device_class_ref_); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void TextSensorStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("TextSensorStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append("'").append(this->state).append("'"); - out.append("\n"); - - out.append(" missing_state: "); - out.append(YESNO(this->missing_state)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "TextSensorStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state_ref_); + dump_field(out, "missing_state", this->missing_state); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif void SubscribeLogsRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("SubscribeLogsRequest {\n"); - out.append(" level: "); - out.append(proto_enum_to_string(this->level)); - out.append("\n"); - - out.append(" dump_config: "); - out.append(YESNO(this->dump_config)); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "SubscribeLogsRequest"); + dump_field(out, "level", static_cast(this->level)); + dump_field(out, "dump_config", this->dump_config); } void SubscribeLogsResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("SubscribeLogsResponse {\n"); - out.append(" level: "); - out.append(proto_enum_to_string(this->level)); - out.append("\n"); - + MessageDumpHelper helper(out, "SubscribeLogsResponse"); + dump_field(out, "level", static_cast(this->level)); out.append(" message: "); - out.append(format_hex_pretty(this->message)); + out.append(format_hex_pretty(this->message_ptr_, this->message_len_)); out.append("\n"); - - out.append(" send_failed: "); - out.append(YESNO(this->send_failed)); - out.append("\n"); - out.append("}"); } #ifdef USE_API_NOISE void NoiseEncryptionSetKeyRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("NoiseEncryptionSetKeyRequest {\n"); + MessageDumpHelper helper(out, "NoiseEncryptionSetKeyRequest"); out.append(" key: "); - out.append(format_hex_pretty(this->key)); + out.append(format_hex_pretty(reinterpret_cast(this->key.data()), this->key.size())); out.append("\n"); - out.append("}"); -} -void NoiseEncryptionSetKeyResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("NoiseEncryptionSetKeyResponse {\n"); - out.append(" success: "); - out.append(YESNO(this->success)); - out.append("\n"); - out.append("}"); } +void NoiseEncryptionSetKeyResponse::dump_to(std::string &out) const { dump_field(out, "success", this->success); } #endif +#ifdef USE_API_HOMEASSISTANT_SERVICES void SubscribeHomeassistantServicesRequest::dump_to(std::string &out) const { out.append("SubscribeHomeassistantServicesRequest {}"); } void HomeassistantServiceMap::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("HomeassistantServiceMap {\n"); - out.append(" key: "); - out.append("'").append(this->key).append("'"); - out.append("\n"); - - out.append(" value: "); - out.append("'").append(this->value).append("'"); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "HomeassistantServiceMap"); + dump_field(out, "key", this->key_ref_); + dump_field(out, "value", this->value); } -void HomeassistantServiceResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("HomeassistantServiceResponse {\n"); - out.append(" service: "); - out.append("'").append(this->service).append("'"); - out.append("\n"); - +void HomeassistantActionRequest::dump_to(std::string &out) const { + MessageDumpHelper helper(out, "HomeassistantActionRequest"); + dump_field(out, "service", this->service_ref_); for (const auto &it : this->data) { out.append(" data: "); it.dump_to(out); out.append("\n"); } - for (const auto &it : this->data_template) { out.append(" data_template: "); it.dump_to(out); out.append("\n"); } - for (const auto &it : this->variables) { out.append(" variables: "); it.dump_to(out); out.append("\n"); } - - out.append(" is_event: "); - out.append(YESNO(this->is_event)); - out.append("\n"); - out.append("}"); + dump_field(out, "is_event", this->is_event); +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES + dump_field(out, "call_id", this->call_id); +#endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + dump_field(out, "wants_response", this->wants_response); +#endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + dump_field(out, "response_template", this->response_template); +#endif } +#endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES +void HomeassistantActionResponse::dump_to(std::string &out) const { + MessageDumpHelper helper(out, "HomeassistantActionResponse"); + dump_field(out, "call_id", this->call_id); + dump_field(out, "success", this->success); + dump_field(out, "error_message", this->error_message); +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + out.append(" response_data: "); + out.append(format_hex_pretty(this->response_data, this->response_data_len)); + out.append("\n"); +#endif +} +#endif +#ifdef USE_API_HOMEASSISTANT_STATES void SubscribeHomeAssistantStatesRequest::dump_to(std::string &out) const { out.append("SubscribeHomeAssistantStatesRequest {}"); } void SubscribeHomeAssistantStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("SubscribeHomeAssistantStateResponse {\n"); - out.append(" entity_id: "); - out.append("'").append(this->entity_id).append("'"); - out.append("\n"); - - out.append(" attribute: "); - out.append("'").append(this->attribute).append("'"); - out.append("\n"); - - out.append(" once: "); - out.append(YESNO(this->once)); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "SubscribeHomeAssistantStateResponse"); + dump_field(out, "entity_id", this->entity_id_ref_); + dump_field(out, "attribute", this->attribute_ref_); + dump_field(out, "once", this->once); } void HomeAssistantStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("HomeAssistantStateResponse {\n"); - out.append(" entity_id: "); - out.append("'").append(this->entity_id).append("'"); - out.append("\n"); - - out.append(" state: "); - out.append("'").append(this->state).append("'"); - out.append("\n"); - - out.append(" attribute: "); - out.append("'").append(this->attribute).append("'"); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "HomeAssistantStateResponse"); + dump_field(out, "entity_id", this->entity_id); + dump_field(out, "state", this->state); + dump_field(out, "attribute", this->attribute); } +#endif void GetTimeRequest::dump_to(std::string &out) const { out.append("GetTimeRequest {}"); } void GetTimeResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("GetTimeResponse {\n"); - out.append(" epoch_seconds: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->epoch_seconds); - out.append(buffer); + MessageDumpHelper helper(out, "GetTimeResponse"); + dump_field(out, "epoch_seconds", this->epoch_seconds); + out.append(" timezone: "); + out.append(format_hex_pretty(this->timezone, this->timezone_len)); out.append("\n"); - out.append("}"); } #ifdef USE_API_SERVICES void ListEntitiesServicesArgument::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesServicesArgument {\n"); - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" type: "); - out.append(proto_enum_to_string(this->type)); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesServicesArgument"); + dump_field(out, "name", this->name_ref_); + dump_field(out, "type", static_cast(this->type)); } void ListEntitiesServicesResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesServicesResponse {\n"); - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - + MessageDumpHelper helper(out, "ListEntitiesServicesResponse"); + dump_field(out, "name", this->name_ref_); + dump_field(out, "key", this->key); for (const auto &it : this->args) { out.append(" args: "); it.dump_to(out); out.append("\n"); } - out.append("}"); } void ExecuteServiceArgument::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ExecuteServiceArgument {\n"); - out.append(" bool_: "); - out.append(YESNO(this->bool_)); - out.append("\n"); - - out.append(" legacy_int: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, this->legacy_int); - out.append(buffer); - out.append("\n"); - - out.append(" float_: "); - snprintf(buffer, sizeof(buffer), "%g", this->float_); - out.append(buffer); - out.append("\n"); - - out.append(" string_: "); - out.append("'").append(this->string_).append("'"); - out.append("\n"); - - out.append(" int_: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, this->int_); - out.append(buffer); - out.append("\n"); - + MessageDumpHelper helper(out, "ExecuteServiceArgument"); + dump_field(out, "bool_", this->bool_); + dump_field(out, "legacy_int", this->legacy_int); + dump_field(out, "float_", this->float_); + dump_field(out, "string_", this->string_); + dump_field(out, "int_", this->int_); for (const auto it : this->bool_array) { - out.append(" bool_array: "); - out.append(YESNO(it)); - out.append("\n"); + dump_field(out, "bool_array", static_cast(it), 4); } - for (const auto &it : this->int_array) { - out.append(" int_array: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, it); - out.append(buffer); - out.append("\n"); + dump_field(out, "int_array", it, 4); } - for (const auto &it : this->float_array) { - out.append(" float_array: "); - snprintf(buffer, sizeof(buffer), "%g", it); - out.append(buffer); - out.append("\n"); + dump_field(out, "float_array", it, 4); } - for (const auto &it : this->string_array) { - out.append(" string_array: "); - out.append("'").append(it).append("'"); - out.append("\n"); + dump_field(out, "string_array", it, 4); } - out.append("}"); } void ExecuteServiceRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ExecuteServiceRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - + MessageDumpHelper helper(out, "ExecuteServiceRequest"); + dump_field(out, "key", this->key); for (const auto &it : this->args) { out.append(" args: "); it.dump_to(out); out.append("\n"); } - out.append("}"); } #endif #ifdef USE_CAMERA void ListEntitiesCameraResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesCameraResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesCameraResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); + dump_field(out, "disabled_by_default", this->disabled_by_default); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "entity_category", static_cast(this->entity_category)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void CameraImageResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("CameraImageResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - + MessageDumpHelper helper(out, "CameraImageResponse"); + dump_field(out, "key", this->key); out.append(" data: "); - out.append(format_hex_pretty(this->data)); + out.append(format_hex_pretty(this->data_ptr_, this->data_len_)); out.append("\n"); - - out.append(" done: "); - out.append(YESNO(this->done)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + dump_field(out, "done", this->done); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void CameraImageRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("CameraImageRequest {\n"); - out.append(" single: "); - out.append(YESNO(this->single)); - out.append("\n"); - - out.append(" stream: "); - out.append(YESNO(this->stream)); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "CameraImageRequest"); + dump_field(out, "single", this->single); + dump_field(out, "stream", this->stream); } #endif #ifdef USE_CLIMATE void ListEntitiesClimateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesClimateResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" supports_current_temperature: "); - out.append(YESNO(this->supports_current_temperature)); - out.append("\n"); - - out.append(" supports_two_point_target_temperature: "); - out.append(YESNO(this->supports_two_point_target_temperature)); - out.append("\n"); - - for (const auto &it : this->supported_modes) { - out.append(" supported_modes: "); - out.append(proto_enum_to_string(it)); - out.append("\n"); + MessageDumpHelper helper(out, "ListEntitiesClimateResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); + dump_field(out, "supports_current_temperature", this->supports_current_temperature); + dump_field(out, "supports_two_point_target_temperature", this->supports_two_point_target_temperature); + for (const auto &it : *this->supported_modes) { + dump_field(out, "supported_modes", static_cast(it), 4); } - - out.append(" visual_min_temperature: "); - snprintf(buffer, sizeof(buffer), "%g", this->visual_min_temperature); - out.append(buffer); - out.append("\n"); - - out.append(" visual_max_temperature: "); - snprintf(buffer, sizeof(buffer), "%g", this->visual_max_temperature); - out.append(buffer); - out.append("\n"); - - out.append(" visual_target_temperature_step: "); - snprintf(buffer, sizeof(buffer), "%g", this->visual_target_temperature_step); - out.append(buffer); - out.append("\n"); - - out.append(" legacy_supports_away: "); - out.append(YESNO(this->legacy_supports_away)); - out.append("\n"); - - out.append(" supports_action: "); - out.append(YESNO(this->supports_action)); - out.append("\n"); - - for (const auto &it : this->supported_fan_modes) { - out.append(" supported_fan_modes: "); - out.append(proto_enum_to_string(it)); - out.append("\n"); + dump_field(out, "visual_min_temperature", this->visual_min_temperature); + dump_field(out, "visual_max_temperature", this->visual_max_temperature); + dump_field(out, "visual_target_temperature_step", this->visual_target_temperature_step); + dump_field(out, "supports_action", this->supports_action); + for (const auto &it : *this->supported_fan_modes) { + dump_field(out, "supported_fan_modes", static_cast(it), 4); } - - for (const auto &it : this->supported_swing_modes) { - out.append(" supported_swing_modes: "); - out.append(proto_enum_to_string(it)); - out.append("\n"); + for (const auto &it : *this->supported_swing_modes) { + dump_field(out, "supported_swing_modes", static_cast(it), 4); } - - for (const auto &it : this->supported_custom_fan_modes) { - out.append(" supported_custom_fan_modes: "); - out.append("'").append(it).append("'"); - out.append("\n"); + for (const auto &it : *this->supported_custom_fan_modes) { + dump_field(out, "supported_custom_fan_modes", it, 4); } - - for (const auto &it : this->supported_presets) { - out.append(" supported_presets: "); - out.append(proto_enum_to_string(it)); - out.append("\n"); + for (const auto &it : *this->supported_presets) { + dump_field(out, "supported_presets", static_cast(it), 4); } - - for (const auto &it : this->supported_custom_presets) { - out.append(" supported_custom_presets: "); - out.append("'").append(it).append("'"); - out.append("\n"); + for (const auto &it : *this->supported_custom_presets) { + dump_field(out, "supported_custom_presets", it, 4); } - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" visual_current_temperature_step: "); - snprintf(buffer, sizeof(buffer), "%g", this->visual_current_temperature_step); - out.append(buffer); - out.append("\n"); - - out.append(" supports_current_humidity: "); - out.append(YESNO(this->supports_current_humidity)); - out.append("\n"); - - out.append(" supports_target_humidity: "); - out.append(YESNO(this->supports_target_humidity)); - out.append("\n"); - - out.append(" visual_min_humidity: "); - snprintf(buffer, sizeof(buffer), "%g", this->visual_min_humidity); - out.append(buffer); - out.append("\n"); - - out.append(" visual_max_humidity: "); - snprintf(buffer, sizeof(buffer), "%g", this->visual_max_humidity); - out.append(buffer); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + dump_field(out, "disabled_by_default", this->disabled_by_default); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "entity_category", static_cast(this->entity_category)); + dump_field(out, "visual_current_temperature_step", this->visual_current_temperature_step); + dump_field(out, "supports_current_humidity", this->supports_current_humidity); + dump_field(out, "supports_target_humidity", this->supports_target_humidity); + dump_field(out, "visual_min_humidity", this->visual_min_humidity); + dump_field(out, "visual_max_humidity", this->visual_max_humidity); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif + dump_field(out, "feature_flags", this->feature_flags); } void ClimateStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ClimateStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" mode: "); - out.append(proto_enum_to_string(this->mode)); - out.append("\n"); - - out.append(" current_temperature: "); - snprintf(buffer, sizeof(buffer), "%g", this->current_temperature); - out.append(buffer); - out.append("\n"); - - out.append(" target_temperature: "); - snprintf(buffer, sizeof(buffer), "%g", this->target_temperature); - out.append(buffer); - out.append("\n"); - - out.append(" target_temperature_low: "); - snprintf(buffer, sizeof(buffer), "%g", this->target_temperature_low); - out.append(buffer); - out.append("\n"); - - out.append(" target_temperature_high: "); - snprintf(buffer, sizeof(buffer), "%g", this->target_temperature_high); - out.append(buffer); - out.append("\n"); - - out.append(" unused_legacy_away: "); - out.append(YESNO(this->unused_legacy_away)); - out.append("\n"); - - out.append(" action: "); - out.append(proto_enum_to_string(this->action)); - out.append("\n"); - - out.append(" fan_mode: "); - out.append(proto_enum_to_string(this->fan_mode)); - out.append("\n"); - - out.append(" swing_mode: "); - out.append(proto_enum_to_string(this->swing_mode)); - out.append("\n"); - - out.append(" custom_fan_mode: "); - out.append("'").append(this->custom_fan_mode).append("'"); - out.append("\n"); - - out.append(" preset: "); - out.append(proto_enum_to_string(this->preset)); - out.append("\n"); - - out.append(" custom_preset: "); - out.append("'").append(this->custom_preset).append("'"); - out.append("\n"); - - out.append(" current_humidity: "); - snprintf(buffer, sizeof(buffer), "%g", this->current_humidity); - out.append(buffer); - out.append("\n"); - - out.append(" target_humidity: "); - snprintf(buffer, sizeof(buffer), "%g", this->target_humidity); - out.append(buffer); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ClimateStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "mode", static_cast(this->mode)); + dump_field(out, "current_temperature", this->current_temperature); + dump_field(out, "target_temperature", this->target_temperature); + dump_field(out, "target_temperature_low", this->target_temperature_low); + dump_field(out, "target_temperature_high", this->target_temperature_high); + dump_field(out, "action", static_cast(this->action)); + dump_field(out, "fan_mode", static_cast(this->fan_mode)); + dump_field(out, "swing_mode", static_cast(this->swing_mode)); + dump_field(out, "custom_fan_mode", this->custom_fan_mode_ref_); + dump_field(out, "preset", static_cast(this->preset)); + dump_field(out, "custom_preset", this->custom_preset_ref_); + dump_field(out, "current_humidity", this->current_humidity); + dump_field(out, "target_humidity", this->target_humidity); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void ClimateCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ClimateCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" has_mode: "); - out.append(YESNO(this->has_mode)); - out.append("\n"); - - out.append(" mode: "); - out.append(proto_enum_to_string(this->mode)); - out.append("\n"); - - out.append(" has_target_temperature: "); - out.append(YESNO(this->has_target_temperature)); - out.append("\n"); - - out.append(" target_temperature: "); - snprintf(buffer, sizeof(buffer), "%g", this->target_temperature); - out.append(buffer); - out.append("\n"); - - out.append(" has_target_temperature_low: "); - out.append(YESNO(this->has_target_temperature_low)); - out.append("\n"); - - out.append(" target_temperature_low: "); - snprintf(buffer, sizeof(buffer), "%g", this->target_temperature_low); - out.append(buffer); - out.append("\n"); - - out.append(" has_target_temperature_high: "); - out.append(YESNO(this->has_target_temperature_high)); - out.append("\n"); - - out.append(" target_temperature_high: "); - snprintf(buffer, sizeof(buffer), "%g", this->target_temperature_high); - out.append(buffer); - out.append("\n"); - - out.append(" unused_has_legacy_away: "); - out.append(YESNO(this->unused_has_legacy_away)); - out.append("\n"); - - out.append(" unused_legacy_away: "); - out.append(YESNO(this->unused_legacy_away)); - out.append("\n"); - - out.append(" has_fan_mode: "); - out.append(YESNO(this->has_fan_mode)); - out.append("\n"); - - out.append(" fan_mode: "); - out.append(proto_enum_to_string(this->fan_mode)); - out.append("\n"); - - out.append(" has_swing_mode: "); - out.append(YESNO(this->has_swing_mode)); - out.append("\n"); - - out.append(" swing_mode: "); - out.append(proto_enum_to_string(this->swing_mode)); - out.append("\n"); - - out.append(" has_custom_fan_mode: "); - out.append(YESNO(this->has_custom_fan_mode)); - out.append("\n"); - - out.append(" custom_fan_mode: "); - out.append("'").append(this->custom_fan_mode).append("'"); - out.append("\n"); - - out.append(" has_preset: "); - out.append(YESNO(this->has_preset)); - out.append("\n"); - - out.append(" preset: "); - out.append(proto_enum_to_string(this->preset)); - out.append("\n"); - - out.append(" has_custom_preset: "); - out.append(YESNO(this->has_custom_preset)); - out.append("\n"); - - out.append(" custom_preset: "); - out.append("'").append(this->custom_preset).append("'"); - out.append("\n"); - - out.append(" has_target_humidity: "); - out.append(YESNO(this->has_target_humidity)); - out.append("\n"); - - out.append(" target_humidity: "); - snprintf(buffer, sizeof(buffer), "%g", this->target_humidity); - out.append(buffer); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ClimateCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "has_mode", this->has_mode); + dump_field(out, "mode", static_cast(this->mode)); + dump_field(out, "has_target_temperature", this->has_target_temperature); + dump_field(out, "target_temperature", this->target_temperature); + dump_field(out, "has_target_temperature_low", this->has_target_temperature_low); + dump_field(out, "target_temperature_low", this->target_temperature_low); + dump_field(out, "has_target_temperature_high", this->has_target_temperature_high); + dump_field(out, "target_temperature_high", this->target_temperature_high); + dump_field(out, "has_fan_mode", this->has_fan_mode); + dump_field(out, "fan_mode", static_cast(this->fan_mode)); + dump_field(out, "has_swing_mode", this->has_swing_mode); + dump_field(out, "swing_mode", static_cast(this->swing_mode)); + dump_field(out, "has_custom_fan_mode", this->has_custom_fan_mode); + dump_field(out, "custom_fan_mode", this->custom_fan_mode); + dump_field(out, "has_preset", this->has_preset); + dump_field(out, "preset", static_cast(this->preset)); + dump_field(out, "has_custom_preset", this->has_custom_preset); + dump_field(out, "custom_preset", this->custom_preset); + dump_field(out, "has_target_humidity", this->has_target_humidity); + dump_field(out, "target_humidity", this->target_humidity); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_NUMBER void ListEntitiesNumberResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesNumberResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" min_value: "); - snprintf(buffer, sizeof(buffer), "%g", this->min_value); - out.append(buffer); - out.append("\n"); - - out.append(" max_value: "); - snprintf(buffer, sizeof(buffer), "%g", this->max_value); - out.append(buffer); - out.append("\n"); - - out.append(" step: "); - snprintf(buffer, sizeof(buffer), "%g", this->step); - out.append(buffer); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" unit_of_measurement: "); - out.append("'").append(this->unit_of_measurement).append("'"); - out.append("\n"); - - out.append(" mode: "); - out.append(proto_enum_to_string(this->mode)); - out.append("\n"); - - out.append(" device_class: "); - out.append("'").append(this->device_class).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesNumberResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "min_value", this->min_value); + dump_field(out, "max_value", this->max_value); + dump_field(out, "step", this->step); + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); + dump_field(out, "unit_of_measurement", this->unit_of_measurement_ref_); + dump_field(out, "mode", static_cast(this->mode)); + dump_field(out, "device_class", this->device_class_ref_); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void NumberStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("NumberStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - snprintf(buffer, sizeof(buffer), "%g", this->state); - out.append(buffer); - out.append("\n"); - - out.append(" missing_state: "); - out.append(YESNO(this->missing_state)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "NumberStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state); + dump_field(out, "missing_state", this->missing_state); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void NumberCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("NumberCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - snprintf(buffer, sizeof(buffer), "%g", this->state); - out.append(buffer); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "NumberCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_SELECT void ListEntitiesSelectResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesSelectResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - for (const auto &it : this->options) { - out.append(" options: "); - out.append("'").append(it).append("'"); - out.append("\n"); + MessageDumpHelper helper(out, "ListEntitiesSelectResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + for (const auto &it : *this->options) { + dump_field(out, "options", it, 4); } - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void SelectStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("SelectStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append("'").append(this->state).append("'"); - out.append("\n"); - - out.append(" missing_state: "); - out.append(YESNO(this->missing_state)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "SelectStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state_ref_); + dump_field(out, "missing_state", this->missing_state); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void SelectCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("SelectCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append("'").append(this->state).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "SelectCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_SIREN void ListEntitiesSirenResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesSirenResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - + MessageDumpHelper helper(out, "ListEntitiesSirenResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "disabled_by_default", this->disabled_by_default); for (const auto &it : this->tones) { - out.append(" tones: "); - out.append("'").append(it).append("'"); - out.append("\n"); + dump_field(out, "tones", it, 4); } - - out.append(" supports_duration: "); - out.append(YESNO(this->supports_duration)); - out.append("\n"); - - out.append(" supports_volume: "); - out.append(YESNO(this->supports_volume)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + dump_field(out, "supports_duration", this->supports_duration); + dump_field(out, "supports_volume", this->supports_volume); + dump_field(out, "entity_category", static_cast(this->entity_category)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void SirenStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("SirenStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append(YESNO(this->state)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "SirenStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void SirenCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("SirenCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" has_state: "); - out.append(YESNO(this->has_state)); - out.append("\n"); - - out.append(" state: "); - out.append(YESNO(this->state)); - out.append("\n"); - - out.append(" has_tone: "); - out.append(YESNO(this->has_tone)); - out.append("\n"); - - out.append(" tone: "); - out.append("'").append(this->tone).append("'"); - out.append("\n"); - - out.append(" has_duration: "); - out.append(YESNO(this->has_duration)); - out.append("\n"); - - out.append(" duration: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->duration); - out.append(buffer); - out.append("\n"); - - out.append(" has_volume: "); - out.append(YESNO(this->has_volume)); - out.append("\n"); - - out.append(" volume: "); - snprintf(buffer, sizeof(buffer), "%g", this->volume); - out.append(buffer); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "SirenCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "has_state", this->has_state); + dump_field(out, "state", this->state); + dump_field(out, "has_tone", this->has_tone); + dump_field(out, "tone", this->tone); + dump_field(out, "has_duration", this->has_duration); + dump_field(out, "duration", this->duration); + dump_field(out, "has_volume", this->has_volume); + dump_field(out, "volume", this->volume); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_LOCK void ListEntitiesLockResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesLockResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" assumed_state: "); - out.append(YESNO(this->assumed_state)); - out.append("\n"); - - out.append(" supports_open: "); - out.append(YESNO(this->supports_open)); - out.append("\n"); - - out.append(" requires_code: "); - out.append(YESNO(this->requires_code)); - out.append("\n"); - - out.append(" code_format: "); - out.append("'").append(this->code_format).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesLockResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); + dump_field(out, "assumed_state", this->assumed_state); + dump_field(out, "supports_open", this->supports_open); + dump_field(out, "requires_code", this->requires_code); + dump_field(out, "code_format", this->code_format_ref_); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void LockStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("LockStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append(proto_enum_to_string(this->state)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "LockStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "state", static_cast(this->state)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void LockCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("LockCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" command: "); - out.append(proto_enum_to_string(this->command)); - out.append("\n"); - - out.append(" has_code: "); - out.append(YESNO(this->has_code)); - out.append("\n"); - - out.append(" code: "); - out.append("'").append(this->code).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "LockCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "command", static_cast(this->command)); + dump_field(out, "has_code", this->has_code); + dump_field(out, "code", this->code); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_BUTTON void ListEntitiesButtonResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesButtonResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_class: "); - out.append("'").append(this->device_class).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesButtonResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); + dump_field(out, "device_class", this->device_class_ref_); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void ButtonCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ButtonCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ButtonCommandRequest"); + dump_field(out, "key", this->key); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_MEDIA_PLAYER void MediaPlayerSupportedFormat::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("MediaPlayerSupportedFormat {\n"); - out.append(" format: "); - out.append("'").append(this->format).append("'"); - out.append("\n"); - - out.append(" sample_rate: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->sample_rate); - out.append(buffer); - out.append("\n"); - - out.append(" num_channels: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->num_channels); - out.append(buffer); - out.append("\n"); - - out.append(" purpose: "); - out.append(proto_enum_to_string(this->purpose)); - out.append("\n"); - - out.append(" sample_bytes: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->sample_bytes); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "MediaPlayerSupportedFormat"); + dump_field(out, "format", this->format_ref_); + dump_field(out, "sample_rate", this->sample_rate); + dump_field(out, "num_channels", this->num_channels); + dump_field(out, "purpose", static_cast(this->purpose)); + dump_field(out, "sample_bytes", this->sample_bytes); } void ListEntitiesMediaPlayerResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesMediaPlayerResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" supports_pause: "); - out.append(YESNO(this->supports_pause)); - out.append("\n"); - + MessageDumpHelper helper(out, "ListEntitiesMediaPlayerResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); + dump_field(out, "supports_pause", this->supports_pause); for (const auto &it : this->supported_formats) { out.append(" supported_formats: "); it.dump_to(out); out.append("\n"); } - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif + dump_field(out, "feature_flags", this->feature_flags); } void MediaPlayerStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("MediaPlayerStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append(proto_enum_to_string(this->state)); - out.append("\n"); - - out.append(" volume: "); - snprintf(buffer, sizeof(buffer), "%g", this->volume); - out.append(buffer); - out.append("\n"); - - out.append(" muted: "); - out.append(YESNO(this->muted)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "MediaPlayerStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "state", static_cast(this->state)); + dump_field(out, "volume", this->volume); + dump_field(out, "muted", this->muted); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void MediaPlayerCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("MediaPlayerCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" has_command: "); - out.append(YESNO(this->has_command)); - out.append("\n"); - - out.append(" command: "); - out.append(proto_enum_to_string(this->command)); - out.append("\n"); - - out.append(" has_volume: "); - out.append(YESNO(this->has_volume)); - out.append("\n"); - - out.append(" volume: "); - snprintf(buffer, sizeof(buffer), "%g", this->volume); - out.append(buffer); - out.append("\n"); - - out.append(" has_media_url: "); - out.append(YESNO(this->has_media_url)); - out.append("\n"); - - out.append(" media_url: "); - out.append("'").append(this->media_url).append("'"); - out.append("\n"); - - out.append(" has_announcement: "); - out.append(YESNO(this->has_announcement)); - out.append("\n"); - - out.append(" announcement: "); - out.append(YESNO(this->announcement)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "MediaPlayerCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "has_command", this->has_command); + dump_field(out, "command", static_cast(this->command)); + dump_field(out, "has_volume", this->has_volume); + dump_field(out, "volume", this->volume); + dump_field(out, "has_media_url", this->has_media_url); + dump_field(out, "media_url", this->media_url); + dump_field(out, "has_announcement", this->has_announcement); + dump_field(out, "announcement", this->announcement); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_BLUETOOTH_PROXY void SubscribeBluetoothLEAdvertisementsRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("SubscribeBluetoothLEAdvertisementsRequest {\n"); - out.append(" flags: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->flags); - out.append(buffer); - out.append("\n"); - out.append("}"); -} -void BluetoothServiceData::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothServiceData {\n"); - out.append(" uuid: "); - out.append("'").append(this->uuid).append("'"); - out.append("\n"); - - for (const auto &it : this->legacy_data) { - out.append(" legacy_data: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, it); - out.append(buffer); - out.append("\n"); - } - - out.append(" data: "); - out.append(format_hex_pretty(this->data)); - out.append("\n"); - out.append("}"); -} -void BluetoothLEAdvertisementResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothLEAdvertisementResponse {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append(format_hex_pretty(this->name)); - out.append("\n"); - - out.append(" rssi: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, this->rssi); - out.append(buffer); - out.append("\n"); - - for (const auto &it : this->service_uuids) { - out.append(" service_uuids: "); - out.append("'").append(it).append("'"); - out.append("\n"); - } - - for (const auto &it : this->service_data) { - out.append(" service_data: "); - it.dump_to(out); - out.append("\n"); - } - - for (const auto &it : this->manufacturer_data) { - out.append(" manufacturer_data: "); - it.dump_to(out); - out.append("\n"); - } - - out.append(" address_type: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->address_type); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "SubscribeBluetoothLEAdvertisementsRequest"); + dump_field(out, "flags", this->flags); } void BluetoothLERawAdvertisement::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothLERawAdvertisement {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" rssi: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, this->rssi); - out.append(buffer); - out.append("\n"); - - out.append(" address_type: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->address_type); - out.append(buffer); - out.append("\n"); - + MessageDumpHelper helper(out, "BluetoothLERawAdvertisement"); + dump_field(out, "address", this->address); + dump_field(out, "rssi", this->rssi); + dump_field(out, "address_type", this->address_type); out.append(" data: "); - out.append(format_hex_pretty(this->data)); + out.append(format_hex_pretty(this->data, this->data_len)); out.append("\n"); - out.append("}"); } void BluetoothLERawAdvertisementsResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothLERawAdvertisementsResponse {\n"); - for (const auto &it : this->advertisements) { + MessageDumpHelper helper(out, "BluetoothLERawAdvertisementsResponse"); + for (uint16_t i = 0; i < this->advertisements_len; i++) { out.append(" advertisements: "); - it.dump_to(out); + this->advertisements[i].dump_to(out); out.append("\n"); } - out.append("}"); } void BluetoothDeviceRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothDeviceRequest {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" request_type: "); - out.append(proto_enum_to_string(this->request_type)); - out.append("\n"); - - out.append(" has_address_type: "); - out.append(YESNO(this->has_address_type)); - out.append("\n"); - - out.append(" address_type: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->address_type); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothDeviceRequest"); + dump_field(out, "address", this->address); + dump_field(out, "request_type", static_cast(this->request_type)); + dump_field(out, "has_address_type", this->has_address_type); + dump_field(out, "address_type", this->address_type); } void BluetoothDeviceConnectionResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothDeviceConnectionResponse {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" connected: "); - out.append(YESNO(this->connected)); - out.append("\n"); - - out.append(" mtu: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->mtu); - out.append(buffer); - out.append("\n"); - - out.append(" error: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, this->error); - out.append(buffer); - out.append("\n"); - out.append("}"); -} -void BluetoothGATTGetServicesRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTGetServicesRequest {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothDeviceConnectionResponse"); + dump_field(out, "address", this->address); + dump_field(out, "connected", this->connected); + dump_field(out, "mtu", this->mtu); + dump_field(out, "error", this->error); } +void BluetoothGATTGetServicesRequest::dump_to(std::string &out) const { dump_field(out, "address", this->address); } void BluetoothGATTDescriptor::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTDescriptor {\n"); + MessageDumpHelper helper(out, "BluetoothGATTDescriptor"); for (const auto &it : this->uuid) { - out.append(" uuid: "); - snprintf(buffer, sizeof(buffer), "%llu", it); - out.append(buffer); - out.append("\n"); + dump_field(out, "uuid", it, 4); } - - out.append(" handle: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->handle); - out.append(buffer); - out.append("\n"); - out.append("}"); + dump_field(out, "handle", this->handle); + dump_field(out, "short_uuid", this->short_uuid); } void BluetoothGATTCharacteristic::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTCharacteristic {\n"); + MessageDumpHelper helper(out, "BluetoothGATTCharacteristic"); for (const auto &it : this->uuid) { - out.append(" uuid: "); - snprintf(buffer, sizeof(buffer), "%llu", it); - out.append(buffer); - out.append("\n"); + dump_field(out, "uuid", it, 4); } - - out.append(" handle: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->handle); - out.append(buffer); - out.append("\n"); - - out.append(" properties: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->properties); - out.append(buffer); - out.append("\n"); - + dump_field(out, "handle", this->handle); + dump_field(out, "properties", this->properties); for (const auto &it : this->descriptors) { out.append(" descriptors: "); it.dump_to(out); out.append("\n"); } - out.append("}"); + dump_field(out, "short_uuid", this->short_uuid); } void BluetoothGATTService::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTService {\n"); + MessageDumpHelper helper(out, "BluetoothGATTService"); for (const auto &it : this->uuid) { - out.append(" uuid: "); - snprintf(buffer, sizeof(buffer), "%llu", it); - out.append(buffer); - out.append("\n"); + dump_field(out, "uuid", it, 4); } - - out.append(" handle: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->handle); - out.append(buffer); - out.append("\n"); - + dump_field(out, "handle", this->handle); for (const auto &it : this->characteristics) { out.append(" characteristics: "); it.dump_to(out); out.append("\n"); } - out.append("}"); + dump_field(out, "short_uuid", this->short_uuid); } void BluetoothGATTGetServicesResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTGetServicesResponse {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - + MessageDumpHelper helper(out, "BluetoothGATTGetServicesResponse"); + dump_field(out, "address", this->address); for (const auto &it : this->services) { out.append(" services: "); it.dump_to(out); out.append("\n"); } - out.append("}"); } void BluetoothGATTGetServicesDoneResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTGetServicesDoneResponse {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothGATTGetServicesDoneResponse"); + dump_field(out, "address", this->address); } void BluetoothGATTReadRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTReadRequest {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" handle: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->handle); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothGATTReadRequest"); + dump_field(out, "address", this->address); + dump_field(out, "handle", this->handle); } void BluetoothGATTReadResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTReadResponse {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" handle: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->handle); - out.append(buffer); - out.append("\n"); - + MessageDumpHelper helper(out, "BluetoothGATTReadResponse"); + dump_field(out, "address", this->address); + dump_field(out, "handle", this->handle); out.append(" data: "); - out.append(format_hex_pretty(this->data)); + out.append(format_hex_pretty(this->data_ptr_, this->data_len_)); out.append("\n"); - out.append("}"); } void BluetoothGATTWriteRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTWriteRequest {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" handle: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->handle); - out.append(buffer); - out.append("\n"); - - out.append(" response: "); - out.append(YESNO(this->response)); - out.append("\n"); - + MessageDumpHelper helper(out, "BluetoothGATTWriteRequest"); + dump_field(out, "address", this->address); + dump_field(out, "handle", this->handle); + dump_field(out, "response", this->response); out.append(" data: "); - out.append(format_hex_pretty(this->data)); + out.append(format_hex_pretty(this->data, this->data_len)); out.append("\n"); - out.append("}"); } void BluetoothGATTReadDescriptorRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTReadDescriptorRequest {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" handle: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->handle); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothGATTReadDescriptorRequest"); + dump_field(out, "address", this->address); + dump_field(out, "handle", this->handle); } void BluetoothGATTWriteDescriptorRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTWriteDescriptorRequest {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" handle: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->handle); - out.append(buffer); - out.append("\n"); - + MessageDumpHelper helper(out, "BluetoothGATTWriteDescriptorRequest"); + dump_field(out, "address", this->address); + dump_field(out, "handle", this->handle); out.append(" data: "); - out.append(format_hex_pretty(this->data)); + out.append(format_hex_pretty(this->data, this->data_len)); out.append("\n"); - out.append("}"); } void BluetoothGATTNotifyRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTNotifyRequest {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" handle: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->handle); - out.append(buffer); - out.append("\n"); - - out.append(" enable: "); - out.append(YESNO(this->enable)); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothGATTNotifyRequest"); + dump_field(out, "address", this->address); + dump_field(out, "handle", this->handle); + dump_field(out, "enable", this->enable); } void BluetoothGATTNotifyDataResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTNotifyDataResponse {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" handle: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->handle); - out.append(buffer); - out.append("\n"); - + MessageDumpHelper helper(out, "BluetoothGATTNotifyDataResponse"); + dump_field(out, "address", this->address); + dump_field(out, "handle", this->handle); out.append(" data: "); - out.append(format_hex_pretty(this->data)); + out.append(format_hex_pretty(this->data_ptr_, this->data_len_)); out.append("\n"); - out.append("}"); } void SubscribeBluetoothConnectionsFreeRequest::dump_to(std::string &out) const { out.append("SubscribeBluetoothConnectionsFreeRequest {}"); } void BluetoothConnectionsFreeResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothConnectionsFreeResponse {\n"); - out.append(" free: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->free); - out.append(buffer); - out.append("\n"); - - out.append(" limit: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->limit); - out.append(buffer); - out.append("\n"); - + MessageDumpHelper helper(out, "BluetoothConnectionsFreeResponse"); + dump_field(out, "free", this->free); + dump_field(out, "limit", this->limit); for (const auto &it : this->allocated) { - out.append(" allocated: "); - snprintf(buffer, sizeof(buffer), "%llu", it); - out.append(buffer); - out.append("\n"); + dump_field(out, "allocated", it, 4); } - out.append("}"); } void BluetoothGATTErrorResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTErrorResponse {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" handle: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->handle); - out.append(buffer); - out.append("\n"); - - out.append(" error: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, this->error); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothGATTErrorResponse"); + dump_field(out, "address", this->address); + dump_field(out, "handle", this->handle); + dump_field(out, "error", this->error); } void BluetoothGATTWriteResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTWriteResponse {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" handle: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->handle); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothGATTWriteResponse"); + dump_field(out, "address", this->address); + dump_field(out, "handle", this->handle); } void BluetoothGATTNotifyResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothGATTNotifyResponse {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" handle: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->handle); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothGATTNotifyResponse"); + dump_field(out, "address", this->address); + dump_field(out, "handle", this->handle); } void BluetoothDevicePairingResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothDevicePairingResponse {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" paired: "); - out.append(YESNO(this->paired)); - out.append("\n"); - - out.append(" error: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, this->error); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothDevicePairingResponse"); + dump_field(out, "address", this->address); + dump_field(out, "paired", this->paired); + dump_field(out, "error", this->error); } void BluetoothDeviceUnpairingResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothDeviceUnpairingResponse {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" success: "); - out.append(YESNO(this->success)); - out.append("\n"); - - out.append(" error: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, this->error); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothDeviceUnpairingResponse"); + dump_field(out, "address", this->address); + dump_field(out, "success", this->success); + dump_field(out, "error", this->error); } void UnsubscribeBluetoothLEAdvertisementsRequest::dump_to(std::string &out) const { out.append("UnsubscribeBluetoothLEAdvertisementsRequest {}"); } void BluetoothDeviceClearCacheResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothDeviceClearCacheResponse {\n"); - out.append(" address: "); - snprintf(buffer, sizeof(buffer), "%llu", this->address); - out.append(buffer); - out.append("\n"); - - out.append(" success: "); - out.append(YESNO(this->success)); - out.append("\n"); - - out.append(" error: "); - snprintf(buffer, sizeof(buffer), "%" PRId32, this->error); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothDeviceClearCacheResponse"); + dump_field(out, "address", this->address); + dump_field(out, "success", this->success); + dump_field(out, "error", this->error); } void BluetoothScannerStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothScannerStateResponse {\n"); - out.append(" state: "); - out.append(proto_enum_to_string(this->state)); - out.append("\n"); - - out.append(" mode: "); - out.append(proto_enum_to_string(this->mode)); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothScannerStateResponse"); + dump_field(out, "state", static_cast(this->state)); + dump_field(out, "mode", static_cast(this->mode)); + dump_field(out, "configured_mode", static_cast(this->configured_mode)); } void BluetoothScannerSetModeRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("BluetoothScannerSetModeRequest {\n"); - out.append(" mode: "); - out.append(proto_enum_to_string(this->mode)); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "BluetoothScannerSetModeRequest"); + dump_field(out, "mode", static_cast(this->mode)); } #endif #ifdef USE_VOICE_ASSISTANT void SubscribeVoiceAssistantRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("SubscribeVoiceAssistantRequest {\n"); - out.append(" subscribe: "); - out.append(YESNO(this->subscribe)); - out.append("\n"); - - out.append(" flags: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->flags); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "SubscribeVoiceAssistantRequest"); + dump_field(out, "subscribe", this->subscribe); + dump_field(out, "flags", this->flags); } void VoiceAssistantAudioSettings::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("VoiceAssistantAudioSettings {\n"); - out.append(" noise_suppression_level: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->noise_suppression_level); - out.append(buffer); - out.append("\n"); - - out.append(" auto_gain: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->auto_gain); - out.append(buffer); - out.append("\n"); - - out.append(" volume_multiplier: "); - snprintf(buffer, sizeof(buffer), "%g", this->volume_multiplier); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "VoiceAssistantAudioSettings"); + dump_field(out, "noise_suppression_level", this->noise_suppression_level); + dump_field(out, "auto_gain", this->auto_gain); + dump_field(out, "volume_multiplier", this->volume_multiplier); } void VoiceAssistantRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("VoiceAssistantRequest {\n"); - out.append(" start: "); - out.append(YESNO(this->start)); - out.append("\n"); - - out.append(" conversation_id: "); - out.append("'").append(this->conversation_id).append("'"); - out.append("\n"); - - out.append(" flags: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->flags); - out.append(buffer); - out.append("\n"); - + MessageDumpHelper helper(out, "VoiceAssistantRequest"); + dump_field(out, "start", this->start); + dump_field(out, "conversation_id", this->conversation_id_ref_); + dump_field(out, "flags", this->flags); out.append(" audio_settings: "); this->audio_settings.dump_to(out); out.append("\n"); - - out.append(" wake_word_phrase: "); - out.append("'").append(this->wake_word_phrase).append("'"); - out.append("\n"); - out.append("}"); + dump_field(out, "wake_word_phrase", this->wake_word_phrase_ref_); } void VoiceAssistantResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("VoiceAssistantResponse {\n"); - out.append(" port: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->port); - out.append(buffer); - out.append("\n"); - - out.append(" error: "); - out.append(YESNO(this->error)); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "VoiceAssistantResponse"); + dump_field(out, "port", this->port); + dump_field(out, "error", this->error); } void VoiceAssistantEventData::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("VoiceAssistantEventData {\n"); - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" value: "); - out.append("'").append(this->value).append("'"); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "VoiceAssistantEventData"); + dump_field(out, "name", this->name); + dump_field(out, "value", this->value); } void VoiceAssistantEventResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("VoiceAssistantEventResponse {\n"); - out.append(" event_type: "); - out.append(proto_enum_to_string(this->event_type)); - out.append("\n"); - + MessageDumpHelper helper(out, "VoiceAssistantEventResponse"); + dump_field(out, "event_type", static_cast(this->event_type)); for (const auto &it : this->data) { out.append(" data: "); it.dump_to(out); out.append("\n"); } - out.append("}"); } void VoiceAssistantAudio::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("VoiceAssistantAudio {\n"); + MessageDumpHelper helper(out, "VoiceAssistantAudio"); out.append(" data: "); - out.append(format_hex_pretty(this->data)); + if (this->data_ptr_ != nullptr) { + out.append(format_hex_pretty(this->data_ptr_, this->data_len_)); + } else { + out.append(format_hex_pretty(reinterpret_cast(this->data.data()), this->data.size())); + } out.append("\n"); - - out.append(" end: "); - out.append(YESNO(this->end)); - out.append("\n"); - out.append("}"); + dump_field(out, "end", this->end); } void VoiceAssistantTimerEventResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("VoiceAssistantTimerEventResponse {\n"); - out.append(" event_type: "); - out.append(proto_enum_to_string(this->event_type)); - out.append("\n"); - - out.append(" timer_id: "); - out.append("'").append(this->timer_id).append("'"); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" total_seconds: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->total_seconds); - out.append(buffer); - out.append("\n"); - - out.append(" seconds_left: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->seconds_left); - out.append(buffer); - out.append("\n"); - - out.append(" is_active: "); - out.append(YESNO(this->is_active)); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "VoiceAssistantTimerEventResponse"); + dump_field(out, "event_type", static_cast(this->event_type)); + dump_field(out, "timer_id", this->timer_id); + dump_field(out, "name", this->name); + dump_field(out, "total_seconds", this->total_seconds); + dump_field(out, "seconds_left", this->seconds_left); + dump_field(out, "is_active", this->is_active); } void VoiceAssistantAnnounceRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("VoiceAssistantAnnounceRequest {\n"); - out.append(" media_id: "); - out.append("'").append(this->media_id).append("'"); - out.append("\n"); - - out.append(" text: "); - out.append("'").append(this->text).append("'"); - out.append("\n"); - - out.append(" preannounce_media_id: "); - out.append("'").append(this->preannounce_media_id).append("'"); - out.append("\n"); - - out.append(" start_conversation: "); - out.append(YESNO(this->start_conversation)); - out.append("\n"); - out.append("}"); -} -void VoiceAssistantAnnounceFinished::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("VoiceAssistantAnnounceFinished {\n"); - out.append(" success: "); - out.append(YESNO(this->success)); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "VoiceAssistantAnnounceRequest"); + dump_field(out, "media_id", this->media_id); + dump_field(out, "text", this->text); + dump_field(out, "preannounce_media_id", this->preannounce_media_id); + dump_field(out, "start_conversation", this->start_conversation); } +void VoiceAssistantAnnounceFinished::dump_to(std::string &out) const { dump_field(out, "success", this->success); } void VoiceAssistantWakeWord::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("VoiceAssistantWakeWord {\n"); - out.append(" id: "); - out.append("'").append(this->id).append("'"); - out.append("\n"); - - out.append(" wake_word: "); - out.append("'").append(this->wake_word).append("'"); - out.append("\n"); - + MessageDumpHelper helper(out, "VoiceAssistantWakeWord"); + dump_field(out, "id", this->id_ref_); + dump_field(out, "wake_word", this->wake_word_ref_); for (const auto &it : this->trained_languages) { - out.append(" trained_languages: "); - out.append("'").append(it).append("'"); - out.append("\n"); + dump_field(out, "trained_languages", it, 4); } - out.append("}"); +} +void VoiceAssistantExternalWakeWord::dump_to(std::string &out) const { + MessageDumpHelper helper(out, "VoiceAssistantExternalWakeWord"); + dump_field(out, "id", this->id); + dump_field(out, "wake_word", this->wake_word); + for (const auto &it : this->trained_languages) { + dump_field(out, "trained_languages", it, 4); + } + dump_field(out, "model_type", this->model_type); + dump_field(out, "model_size", this->model_size); + dump_field(out, "model_hash", this->model_hash); + dump_field(out, "url", this->url); } void VoiceAssistantConfigurationRequest::dump_to(std::string &out) const { - out.append("VoiceAssistantConfigurationRequest {}"); + MessageDumpHelper helper(out, "VoiceAssistantConfigurationRequest"); + for (const auto &it : this->external_wake_words) { + out.append(" external_wake_words: "); + it.dump_to(out); + out.append("\n"); + } } void VoiceAssistantConfigurationResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("VoiceAssistantConfigurationResponse {\n"); + MessageDumpHelper helper(out, "VoiceAssistantConfigurationResponse"); for (const auto &it : this->available_wake_words) { out.append(" available_wake_words: "); it.dump_to(out); out.append("\n"); } - - for (const auto &it : this->active_wake_words) { - out.append(" active_wake_words: "); - out.append("'").append(it).append("'"); - out.append("\n"); + for (const auto &it : *this->active_wake_words) { + dump_field(out, "active_wake_words", it, 4); } - - out.append(" max_active_wake_words: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->max_active_wake_words); - out.append(buffer); - out.append("\n"); - out.append("}"); + dump_field(out, "max_active_wake_words", this->max_active_wake_words); } void VoiceAssistantSetConfiguration::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("VoiceAssistantSetConfiguration {\n"); + MessageDumpHelper helper(out, "VoiceAssistantSetConfiguration"); for (const auto &it : this->active_wake_words) { - out.append(" active_wake_words: "); - out.append("'").append(it).append("'"); - out.append("\n"); + dump_field(out, "active_wake_words", it, 4); } - out.append("}"); } #endif #ifdef USE_ALARM_CONTROL_PANEL void ListEntitiesAlarmControlPanelResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesAlarmControlPanelResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" supported_features: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->supported_features); - out.append(buffer); - out.append("\n"); - - out.append(" requires_code: "); - out.append(YESNO(this->requires_code)); - out.append("\n"); - - out.append(" requires_code_to_arm: "); - out.append(YESNO(this->requires_code_to_arm)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesAlarmControlPanelResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); + dump_field(out, "supported_features", this->supported_features); + dump_field(out, "requires_code", this->requires_code); + dump_field(out, "requires_code_to_arm", this->requires_code_to_arm); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void AlarmControlPanelStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("AlarmControlPanelStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append(proto_enum_to_string(this->state)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "AlarmControlPanelStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "state", static_cast(this->state)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void AlarmControlPanelCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("AlarmControlPanelCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" command: "); - out.append(proto_enum_to_string(this->command)); - out.append("\n"); - - out.append(" code: "); - out.append("'").append(this->code).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "AlarmControlPanelCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "command", static_cast(this->command)); + dump_field(out, "code", this->code); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_TEXT void ListEntitiesTextResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesTextResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" min_length: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->min_length); - out.append(buffer); - out.append("\n"); - - out.append(" max_length: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->max_length); - out.append(buffer); - out.append("\n"); - - out.append(" pattern: "); - out.append("'").append(this->pattern).append("'"); - out.append("\n"); - - out.append(" mode: "); - out.append(proto_enum_to_string(this->mode)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesTextResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); + dump_field(out, "min_length", this->min_length); + dump_field(out, "max_length", this->max_length); + dump_field(out, "pattern", this->pattern_ref_); + dump_field(out, "mode", static_cast(this->mode)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void TextStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("TextStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append("'").append(this->state).append("'"); - out.append("\n"); - - out.append(" missing_state: "); - out.append(YESNO(this->missing_state)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "TextStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state_ref_); + dump_field(out, "missing_state", this->missing_state); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void TextCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("TextCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" state: "); - out.append("'").append(this->state).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "TextCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "state", this->state); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_DATETIME_DATE void ListEntitiesDateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesDateResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesDateResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void DateStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("DateStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" missing_state: "); - out.append(YESNO(this->missing_state)); - out.append("\n"); - - out.append(" year: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->year); - out.append(buffer); - out.append("\n"); - - out.append(" month: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->month); - out.append(buffer); - out.append("\n"); - - out.append(" day: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->day); - out.append(buffer); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "DateStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "missing_state", this->missing_state); + dump_field(out, "year", this->year); + dump_field(out, "month", this->month); + dump_field(out, "day", this->day); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void DateCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("DateCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" year: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->year); - out.append(buffer); - out.append("\n"); - - out.append(" month: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->month); - out.append(buffer); - out.append("\n"); - - out.append(" day: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->day); - out.append(buffer); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "DateCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "year", this->year); + dump_field(out, "month", this->month); + dump_field(out, "day", this->day); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_DATETIME_TIME void ListEntitiesTimeResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesTimeResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesTimeResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void TimeStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("TimeStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" missing_state: "); - out.append(YESNO(this->missing_state)); - out.append("\n"); - - out.append(" hour: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->hour); - out.append(buffer); - out.append("\n"); - - out.append(" minute: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->minute); - out.append(buffer); - out.append("\n"); - - out.append(" second: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->second); - out.append(buffer); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "TimeStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "missing_state", this->missing_state); + dump_field(out, "hour", this->hour); + dump_field(out, "minute", this->minute); + dump_field(out, "second", this->second); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void TimeCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("TimeCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" hour: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->hour); - out.append(buffer); - out.append("\n"); - - out.append(" minute: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->minute); - out.append(buffer); - out.append("\n"); - - out.append(" second: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->second); - out.append(buffer); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "TimeCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "hour", this->hour); + dump_field(out, "minute", this->minute); + dump_field(out, "second", this->second); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_EVENT void ListEntitiesEventResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesEventResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_class: "); - out.append("'").append(this->device_class).append("'"); - out.append("\n"); - + MessageDumpHelper helper(out, "ListEntitiesEventResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); + dump_field(out, "device_class", this->device_class_ref_); for (const auto &it : this->event_types) { - out.append(" event_types: "); - out.append("'").append(it).append("'"); - out.append("\n"); + dump_field(out, "event_types", it, 4); } - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void EventResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("EventResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" event_type: "); - out.append("'").append(this->event_type).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "EventResponse"); + dump_field(out, "key", this->key); + dump_field(out, "event_type", this->event_type_ref_); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_VALVE void ListEntitiesValveResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesValveResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_class: "); - out.append("'").append(this->device_class).append("'"); - out.append("\n"); - - out.append(" assumed_state: "); - out.append(YESNO(this->assumed_state)); - out.append("\n"); - - out.append(" supports_position: "); - out.append(YESNO(this->supports_position)); - out.append("\n"); - - out.append(" supports_stop: "); - out.append(YESNO(this->supports_stop)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesValveResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); + dump_field(out, "device_class", this->device_class_ref_); + dump_field(out, "assumed_state", this->assumed_state); + dump_field(out, "supports_position", this->supports_position); + dump_field(out, "supports_stop", this->supports_stop); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void ValveStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ValveStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" position: "); - snprintf(buffer, sizeof(buffer), "%g", this->position); - out.append(buffer); - out.append("\n"); - - out.append(" current_operation: "); - out.append(proto_enum_to_string(this->current_operation)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ValveStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "position", this->position); + dump_field(out, "current_operation", static_cast(this->current_operation)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void ValveCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ValveCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" has_position: "); - out.append(YESNO(this->has_position)); - out.append("\n"); - - out.append(" position: "); - snprintf(buffer, sizeof(buffer), "%g", this->position); - out.append(buffer); - out.append("\n"); - - out.append(" stop: "); - out.append(YESNO(this->stop)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ValveCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "has_position", this->has_position); + dump_field(out, "position", this->position); + dump_field(out, "stop", this->stop); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_DATETIME_DATETIME void ListEntitiesDateTimeResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesDateTimeResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesDateTimeResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void DateTimeStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("DateTimeStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" missing_state: "); - out.append(YESNO(this->missing_state)); - out.append("\n"); - - out.append(" epoch_seconds: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->epoch_seconds); - out.append(buffer); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "DateTimeStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "missing_state", this->missing_state); + dump_field(out, "epoch_seconds", this->epoch_seconds); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void DateTimeCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("DateTimeCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" epoch_seconds: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->epoch_seconds); - out.append(buffer); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "DateTimeCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "epoch_seconds", this->epoch_seconds); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } #endif #ifdef USE_UPDATE void ListEntitiesUpdateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("ListEntitiesUpdateResponse {\n"); - out.append(" object_id: "); - out.append("'").append(this->object_id).append("'"); - out.append("\n"); - - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" name: "); - out.append("'").append(this->name).append("'"); - out.append("\n"); - - out.append(" unique_id: "); - out.append("'").append(this->unique_id).append("'"); - out.append("\n"); - - out.append(" icon: "); - out.append("'").append(this->icon).append("'"); - out.append("\n"); - - out.append(" disabled_by_default: "); - out.append(YESNO(this->disabled_by_default)); - out.append("\n"); - - out.append(" entity_category: "); - out.append(proto_enum_to_string(this->entity_category)); - out.append("\n"); - - out.append(" device_class: "); - out.append("'").append(this->device_class).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "ListEntitiesUpdateResponse"); + dump_field(out, "object_id", this->object_id_ref_); + dump_field(out, "key", this->key); + dump_field(out, "name", this->name_ref_); +#ifdef USE_ENTITY_ICON + dump_field(out, "icon", this->icon_ref_); +#endif + dump_field(out, "disabled_by_default", this->disabled_by_default); + dump_field(out, "entity_category", static_cast(this->entity_category)); + dump_field(out, "device_class", this->device_class_ref_); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void UpdateStateResponse::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("UpdateStateResponse {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); - out.append("\n"); - - out.append(" missing_state: "); - out.append(YESNO(this->missing_state)); - out.append("\n"); - - out.append(" in_progress: "); - out.append(YESNO(this->in_progress)); - out.append("\n"); - - out.append(" has_progress: "); - out.append(YESNO(this->has_progress)); - out.append("\n"); - - out.append(" progress: "); - snprintf(buffer, sizeof(buffer), "%g", this->progress); - out.append(buffer); - out.append("\n"); - - out.append(" current_version: "); - out.append("'").append(this->current_version).append("'"); - out.append("\n"); - - out.append(" latest_version: "); - out.append("'").append(this->latest_version).append("'"); - out.append("\n"); - - out.append(" title: "); - out.append("'").append(this->title).append("'"); - out.append("\n"); - - out.append(" release_summary: "); - out.append("'").append(this->release_summary).append("'"); - out.append("\n"); - - out.append(" release_url: "); - out.append("'").append(this->release_url).append("'"); - out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); + MessageDumpHelper helper(out, "UpdateStateResponse"); + dump_field(out, "key", this->key); + dump_field(out, "missing_state", this->missing_state); + dump_field(out, "in_progress", this->in_progress); + dump_field(out, "has_progress", this->has_progress); + dump_field(out, "progress", this->progress); + dump_field(out, "current_version", this->current_version_ref_); + dump_field(out, "latest_version", this->latest_version_ref_); + dump_field(out, "title", this->title_ref_); + dump_field(out, "release_summary", this->release_summary_ref_); + dump_field(out, "release_url", this->release_url_ref_); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif } void UpdateCommandRequest::dump_to(std::string &out) const { - __attribute__((unused)) char buffer[64]; - out.append("UpdateCommandRequest {\n"); - out.append(" key: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->key); - out.append(buffer); + MessageDumpHelper helper(out, "UpdateCommandRequest"); + dump_field(out, "key", this->key); + dump_field(out, "command", static_cast(this->command)); +#ifdef USE_DEVICES + dump_field(out, "device_id", this->device_id); +#endif +} +#endif +#ifdef USE_ZWAVE_PROXY +void ZWaveProxyFrame::dump_to(std::string &out) const { + MessageDumpHelper helper(out, "ZWaveProxyFrame"); + out.append(" data: "); + out.append(format_hex_pretty(this->data, this->data_len)); out.append("\n"); - - out.append(" command: "); - out.append(proto_enum_to_string(this->command)); +} +void ZWaveProxyRequest::dump_to(std::string &out) const { + MessageDumpHelper helper(out, "ZWaveProxyRequest"); + dump_field(out, "type", static_cast(this->type)); + out.append(" data: "); + out.append(format_hex_pretty(this->data, this->data_len)); out.append("\n"); - - out.append(" device_id: "); - snprintf(buffer, sizeof(buffer), "%" PRIu32, this->device_id); - out.append(buffer); - out.append("\n"); - out.append("}"); } #endif -} // namespace api -} // namespace esphome +} // namespace esphome::api #endif // HAS_PROTO_MESSAGE_DUMP diff --git a/esphome/components/api/api_pb2_includes.h b/esphome/components/api/api_pb2_includes.h new file mode 100644 index 0000000000..55d95304b1 --- /dev/null +++ b/esphome/components/api/api_pb2_includes.h @@ -0,0 +1,34 @@ +#pragma once + +#include "esphome/core/defines.h" + +// This file provides includes needed by the generated protobuf code +// when using pointer optimizations for component-specific types + +#ifdef USE_CLIMATE +#include "esphome/components/climate/climate_mode.h" +#include "esphome/components/climate/climate_traits.h" +#endif + +#ifdef USE_LIGHT +#include "esphome/components/light/light_traits.h" +#endif + +#ifdef USE_FAN +#include "esphome/components/fan/fan_traits.h" +#endif + +#ifdef USE_SELECT +#include "esphome/components/select/select_traits.h" +#endif + +// Standard library includes that might be needed +#include +#include +#include + +namespace esphome::api { + +// This file only provides includes, no actual code + +} // namespace esphome::api diff --git a/esphome/components/api/api_pb2_service.cpp b/esphome/components/api/api_pb2_service.cpp index b96e5736a4..9d227af0a3 100644 --- a/esphome/components/api/api_pb2_service.cpp +++ b/esphome/components/api/api_pb2_service.cpp @@ -3,8 +3,7 @@ #include "api_pb2_service.h" #include "esphome/core/log.h" -namespace esphome { -namespace api { +namespace esphome::api { static const char *const TAG = "api.service"; @@ -16,7 +15,7 @@ void APIServerConnectionBase::log_send_message_(const char *name, const std::str void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, uint8_t *msg_data) { switch (msg_type) { - case 1: { + case HelloRequest::MESSAGE_TYPE: { HelloRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -25,79 +24,81 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, this->on_hello_request(msg); break; } - case 3: { - ConnectRequest msg; +#ifdef USE_API_PASSWORD + case AuthenticationRequest::MESSAGE_TYPE: { + AuthenticationRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP - ESP_LOGVV(TAG, "on_connect_request: %s", msg.dump().c_str()); + ESP_LOGVV(TAG, "on_authentication_request: %s", msg.dump().c_str()); #endif - this->on_connect_request(msg); + this->on_authentication_request(msg); break; } - case 5: { +#endif + case DisconnectRequest::MESSAGE_TYPE: { DisconnectRequest msg; - msg.decode(msg_data, msg_size); + // Empty message: no decode needed #ifdef HAS_PROTO_MESSAGE_DUMP ESP_LOGVV(TAG, "on_disconnect_request: %s", msg.dump().c_str()); #endif this->on_disconnect_request(msg); break; } - case 6: { + case DisconnectResponse::MESSAGE_TYPE: { DisconnectResponse msg; - msg.decode(msg_data, msg_size); + // Empty message: no decode needed #ifdef HAS_PROTO_MESSAGE_DUMP ESP_LOGVV(TAG, "on_disconnect_response: %s", msg.dump().c_str()); #endif this->on_disconnect_response(msg); break; } - case 7: { + case PingRequest::MESSAGE_TYPE: { PingRequest msg; - msg.decode(msg_data, msg_size); + // Empty message: no decode needed #ifdef HAS_PROTO_MESSAGE_DUMP ESP_LOGVV(TAG, "on_ping_request: %s", msg.dump().c_str()); #endif this->on_ping_request(msg); break; } - case 8: { + case PingResponse::MESSAGE_TYPE: { PingResponse msg; - msg.decode(msg_data, msg_size); + // Empty message: no decode needed #ifdef HAS_PROTO_MESSAGE_DUMP ESP_LOGVV(TAG, "on_ping_response: %s", msg.dump().c_str()); #endif this->on_ping_response(msg); break; } - case 9: { + case DeviceInfoRequest::MESSAGE_TYPE: { DeviceInfoRequest msg; - msg.decode(msg_data, msg_size); + // Empty message: no decode needed #ifdef HAS_PROTO_MESSAGE_DUMP ESP_LOGVV(TAG, "on_device_info_request: %s", msg.dump().c_str()); #endif this->on_device_info_request(msg); break; } - case 11: { + case ListEntitiesRequest::MESSAGE_TYPE: { ListEntitiesRequest msg; - msg.decode(msg_data, msg_size); + // Empty message: no decode needed #ifdef HAS_PROTO_MESSAGE_DUMP ESP_LOGVV(TAG, "on_list_entities_request: %s", msg.dump().c_str()); #endif this->on_list_entities_request(msg); break; } - case 20: { + case SubscribeStatesRequest::MESSAGE_TYPE: { SubscribeStatesRequest msg; - msg.decode(msg_data, msg_size); + // Empty message: no decode needed #ifdef HAS_PROTO_MESSAGE_DUMP ESP_LOGVV(TAG, "on_subscribe_states_request: %s", msg.dump().c_str()); #endif this->on_subscribe_states_request(msg); break; } - case 28: { + case SubscribeLogsRequest::MESSAGE_TYPE: { SubscribeLogsRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -107,7 +108,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, break; } #ifdef USE_COVER - case 30: { + case CoverCommandRequest::MESSAGE_TYPE: { CoverCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -118,7 +119,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_FAN - case 31: { + case FanCommandRequest::MESSAGE_TYPE: { FanCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -129,7 +130,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_LIGHT - case 32: { + case LightCommandRequest::MESSAGE_TYPE: { LightCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -140,7 +141,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_SWITCH - case 33: { + case SwitchCommandRequest::MESSAGE_TYPE: { SwitchCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -150,25 +151,18 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, break; } #endif - case 34: { +#ifdef USE_API_HOMEASSISTANT_SERVICES + case SubscribeHomeassistantServicesRequest::MESSAGE_TYPE: { SubscribeHomeassistantServicesRequest msg; - msg.decode(msg_data, msg_size); + // Empty message: no decode needed #ifdef HAS_PROTO_MESSAGE_DUMP ESP_LOGVV(TAG, "on_subscribe_homeassistant_services_request: %s", msg.dump().c_str()); #endif this->on_subscribe_homeassistant_services_request(msg); break; } - case 36: { - GetTimeRequest msg; - msg.decode(msg_data, msg_size); -#ifdef HAS_PROTO_MESSAGE_DUMP - ESP_LOGVV(TAG, "on_get_time_request: %s", msg.dump().c_str()); #endif - this->on_get_time_request(msg); - break; - } - case 37: { + case GetTimeResponse::MESSAGE_TYPE: { GetTimeResponse msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -177,16 +171,19 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, this->on_get_time_response(msg); break; } - case 38: { +#ifdef USE_API_HOMEASSISTANT_STATES + case SubscribeHomeAssistantStatesRequest::MESSAGE_TYPE: { SubscribeHomeAssistantStatesRequest msg; - msg.decode(msg_data, msg_size); + // Empty message: no decode needed #ifdef HAS_PROTO_MESSAGE_DUMP ESP_LOGVV(TAG, "on_subscribe_home_assistant_states_request: %s", msg.dump().c_str()); #endif this->on_subscribe_home_assistant_states_request(msg); break; } - case 40: { +#endif +#ifdef USE_API_HOMEASSISTANT_STATES + case HomeAssistantStateResponse::MESSAGE_TYPE: { HomeAssistantStateResponse msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -195,8 +192,9 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, this->on_home_assistant_state_response(msg); break; } +#endif #ifdef USE_API_SERVICES - case 42: { + case ExecuteServiceRequest::MESSAGE_TYPE: { ExecuteServiceRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -207,7 +205,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_CAMERA - case 45: { + case CameraImageRequest::MESSAGE_TYPE: { CameraImageRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -218,7 +216,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_CLIMATE - case 48: { + case ClimateCommandRequest::MESSAGE_TYPE: { ClimateCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -229,7 +227,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_NUMBER - case 51: { + case NumberCommandRequest::MESSAGE_TYPE: { NumberCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -240,7 +238,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_SELECT - case 54: { + case SelectCommandRequest::MESSAGE_TYPE: { SelectCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -251,7 +249,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_SIREN - case 57: { + case SirenCommandRequest::MESSAGE_TYPE: { SirenCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -262,7 +260,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_LOCK - case 60: { + case LockCommandRequest::MESSAGE_TYPE: { LockCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -273,7 +271,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_BUTTON - case 62: { + case ButtonCommandRequest::MESSAGE_TYPE: { ButtonCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -284,7 +282,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_MEDIA_PLAYER - case 65: { + case MediaPlayerCommandRequest::MESSAGE_TYPE: { MediaPlayerCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -295,7 +293,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_BLUETOOTH_PROXY - case 66: { + case SubscribeBluetoothLEAdvertisementsRequest::MESSAGE_TYPE: { SubscribeBluetoothLEAdvertisementsRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -306,7 +304,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_BLUETOOTH_PROXY - case 68: { + case BluetoothDeviceRequest::MESSAGE_TYPE: { BluetoothDeviceRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -317,7 +315,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_BLUETOOTH_PROXY - case 70: { + case BluetoothGATTGetServicesRequest::MESSAGE_TYPE: { BluetoothGATTGetServicesRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -328,7 +326,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_BLUETOOTH_PROXY - case 73: { + case BluetoothGATTReadRequest::MESSAGE_TYPE: { BluetoothGATTReadRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -339,7 +337,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_BLUETOOTH_PROXY - case 75: { + case BluetoothGATTWriteRequest::MESSAGE_TYPE: { BluetoothGATTWriteRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -350,7 +348,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_BLUETOOTH_PROXY - case 76: { + case BluetoothGATTReadDescriptorRequest::MESSAGE_TYPE: { BluetoothGATTReadDescriptorRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -361,7 +359,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_BLUETOOTH_PROXY - case 77: { + case BluetoothGATTWriteDescriptorRequest::MESSAGE_TYPE: { BluetoothGATTWriteDescriptorRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -372,7 +370,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_BLUETOOTH_PROXY - case 78: { + case BluetoothGATTNotifyRequest::MESSAGE_TYPE: { BluetoothGATTNotifyRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -383,9 +381,9 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_BLUETOOTH_PROXY - case 80: { + case SubscribeBluetoothConnectionsFreeRequest::MESSAGE_TYPE: { SubscribeBluetoothConnectionsFreeRequest msg; - msg.decode(msg_data, msg_size); + // Empty message: no decode needed #ifdef HAS_PROTO_MESSAGE_DUMP ESP_LOGVV(TAG, "on_subscribe_bluetooth_connections_free_request: %s", msg.dump().c_str()); #endif @@ -394,9 +392,9 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_BLUETOOTH_PROXY - case 87: { + case UnsubscribeBluetoothLEAdvertisementsRequest::MESSAGE_TYPE: { UnsubscribeBluetoothLEAdvertisementsRequest msg; - msg.decode(msg_data, msg_size); + // Empty message: no decode needed #ifdef HAS_PROTO_MESSAGE_DUMP ESP_LOGVV(TAG, "on_unsubscribe_bluetooth_le_advertisements_request: %s", msg.dump().c_str()); #endif @@ -405,7 +403,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_VOICE_ASSISTANT - case 89: { + case SubscribeVoiceAssistantRequest::MESSAGE_TYPE: { SubscribeVoiceAssistantRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -416,7 +414,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_VOICE_ASSISTANT - case 91: { + case VoiceAssistantResponse::MESSAGE_TYPE: { VoiceAssistantResponse msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -427,7 +425,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_VOICE_ASSISTANT - case 92: { + case VoiceAssistantEventResponse::MESSAGE_TYPE: { VoiceAssistantEventResponse msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -438,7 +436,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_ALARM_CONTROL_PANEL - case 96: { + case AlarmControlPanelCommandRequest::MESSAGE_TYPE: { AlarmControlPanelCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -449,7 +447,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_TEXT - case 99: { + case TextCommandRequest::MESSAGE_TYPE: { TextCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -460,7 +458,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_DATETIME_DATE - case 102: { + case DateCommandRequest::MESSAGE_TYPE: { DateCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -471,7 +469,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_DATETIME_TIME - case 105: { + case TimeCommandRequest::MESSAGE_TYPE: { TimeCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -482,7 +480,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_VOICE_ASSISTANT - case 106: { + case VoiceAssistantAudio::MESSAGE_TYPE: { VoiceAssistantAudio msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -493,7 +491,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_VALVE - case 111: { + case ValveCommandRequest::MESSAGE_TYPE: { ValveCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -504,7 +502,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_DATETIME_DATETIME - case 114: { + case DateTimeCommandRequest::MESSAGE_TYPE: { DateTimeCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -515,7 +513,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_VOICE_ASSISTANT - case 115: { + case VoiceAssistantTimerEventResponse::MESSAGE_TYPE: { VoiceAssistantTimerEventResponse msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -526,7 +524,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_UPDATE - case 118: { + case UpdateCommandRequest::MESSAGE_TYPE: { UpdateCommandRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -537,7 +535,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_VOICE_ASSISTANT - case 119: { + case VoiceAssistantAnnounceRequest::MESSAGE_TYPE: { VoiceAssistantAnnounceRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -548,7 +546,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_VOICE_ASSISTANT - case 121: { + case VoiceAssistantConfigurationRequest::MESSAGE_TYPE: { VoiceAssistantConfigurationRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -559,7 +557,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_VOICE_ASSISTANT - case 123: { + case VoiceAssistantSetConfiguration::MESSAGE_TYPE: { VoiceAssistantSetConfiguration msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -570,7 +568,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_API_NOISE - case 124: { + case NoiseEncryptionSetKeyRequest::MESSAGE_TYPE: { NoiseEncryptionSetKeyRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -581,7 +579,7 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } #endif #ifdef USE_BLUETOOTH_PROXY - case 127: { + case BluetoothScannerSetModeRequest::MESSAGE_TYPE: { BluetoothScannerSetModeRequest msg; msg.decode(msg_data, msg_size); #ifdef HAS_PROTO_MESSAGE_DUMP @@ -590,6 +588,39 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, this->on_bluetooth_scanner_set_mode_request(msg); break; } +#endif +#ifdef USE_ZWAVE_PROXY + case ZWaveProxyFrame::MESSAGE_TYPE: { + ZWaveProxyFrame msg; + msg.decode(msg_data, msg_size); +#ifdef HAS_PROTO_MESSAGE_DUMP + ESP_LOGVV(TAG, "on_z_wave_proxy_frame: %s", msg.dump().c_str()); +#endif + this->on_z_wave_proxy_frame(msg); + break; + } +#endif +#ifdef USE_ZWAVE_PROXY + case ZWaveProxyRequest::MESSAGE_TYPE: { + ZWaveProxyRequest msg; + msg.decode(msg_data, msg_size); +#ifdef HAS_PROTO_MESSAGE_DUMP + ESP_LOGVV(TAG, "on_z_wave_proxy_request: %s", msg.dump().c_str()); +#endif + this->on_z_wave_proxy_request(msg); + break; + } +#endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES + case HomeassistantActionResponse::MESSAGE_TYPE: { + HomeassistantActionResponse msg; + msg.decode(msg_data, msg_size); +#ifdef HAS_PROTO_MESSAGE_DUMP + ESP_LOGVV(TAG, "on_homeassistant_action_response: %s", msg.dump().c_str()); +#endif + this->on_homeassistant_action_response(msg); + break; + } #endif default: break; @@ -597,328 +628,230 @@ void APIServerConnectionBase::read_message(uint32_t msg_size, uint32_t msg_type, } void APIServerConnection::on_hello_request(const HelloRequest &msg) { - HelloResponse ret = this->hello(msg); - if (!this->send_message(ret)) { + if (!this->send_hello_response(msg)) { this->on_fatal_error(); } } -void APIServerConnection::on_connect_request(const ConnectRequest &msg) { - ConnectResponse ret = this->connect(msg); - if (!this->send_message(ret)) { +#ifdef USE_API_PASSWORD +void APIServerConnection::on_authentication_request(const AuthenticationRequest &msg) { + if (!this->send_authenticate_response(msg)) { this->on_fatal_error(); } } +#endif void APIServerConnection::on_disconnect_request(const DisconnectRequest &msg) { - DisconnectResponse ret = this->disconnect(msg); - if (!this->send_message(ret)) { + if (!this->send_disconnect_response(msg)) { this->on_fatal_error(); } } void APIServerConnection::on_ping_request(const PingRequest &msg) { - PingResponse ret = this->ping(msg); - if (!this->send_message(ret)) { + if (!this->send_ping_response(msg)) { this->on_fatal_error(); } } void APIServerConnection::on_device_info_request(const DeviceInfoRequest &msg) { - if (this->check_connection_setup_()) { - DeviceInfoResponse ret = this->device_info(msg); - if (!this->send_message(ret)) { - this->on_fatal_error(); - } - } -} -void APIServerConnection::on_list_entities_request(const ListEntitiesRequest &msg) { - if (this->check_authenticated_()) { - this->list_entities(msg); + if (!this->send_device_info_response(msg)) { + this->on_fatal_error(); } } +void APIServerConnection::on_list_entities_request(const ListEntitiesRequest &msg) { this->list_entities(msg); } void APIServerConnection::on_subscribe_states_request(const SubscribeStatesRequest &msg) { - if (this->check_authenticated_()) { - this->subscribe_states(msg); - } -} -void APIServerConnection::on_subscribe_logs_request(const SubscribeLogsRequest &msg) { - if (this->check_authenticated_()) { - this->subscribe_logs(msg); - } + this->subscribe_states(msg); } +void APIServerConnection::on_subscribe_logs_request(const SubscribeLogsRequest &msg) { this->subscribe_logs(msg); } +#ifdef USE_API_HOMEASSISTANT_SERVICES void APIServerConnection::on_subscribe_homeassistant_services_request( const SubscribeHomeassistantServicesRequest &msg) { - if (this->check_authenticated_()) { - this->subscribe_homeassistant_services(msg); - } + this->subscribe_homeassistant_services(msg); } +#endif +#ifdef USE_API_HOMEASSISTANT_STATES void APIServerConnection::on_subscribe_home_assistant_states_request(const SubscribeHomeAssistantStatesRequest &msg) { - if (this->check_authenticated_()) { - this->subscribe_home_assistant_states(msg); - } -} -void APIServerConnection::on_get_time_request(const GetTimeRequest &msg) { - if (this->check_connection_setup_()) { - GetTimeResponse ret = this->get_time(msg); - if (!this->send_message(ret)) { - this->on_fatal_error(); - } - } + this->subscribe_home_assistant_states(msg); } +#endif #ifdef USE_API_SERVICES -void APIServerConnection::on_execute_service_request(const ExecuteServiceRequest &msg) { - if (this->check_authenticated_()) { - this->execute_service(msg); - } -} +void APIServerConnection::on_execute_service_request(const ExecuteServiceRequest &msg) { this->execute_service(msg); } #endif #ifdef USE_API_NOISE void APIServerConnection::on_noise_encryption_set_key_request(const NoiseEncryptionSetKeyRequest &msg) { - if (this->check_authenticated_()) { - NoiseEncryptionSetKeyResponse ret = this->noise_encryption_set_key(msg); - if (!this->send_message(ret)) { - this->on_fatal_error(); - } + if (!this->send_noise_encryption_set_key_response(msg)) { + this->on_fatal_error(); } } #endif #ifdef USE_BUTTON -void APIServerConnection::on_button_command_request(const ButtonCommandRequest &msg) { - if (this->check_authenticated_()) { - this->button_command(msg); - } -} +void APIServerConnection::on_button_command_request(const ButtonCommandRequest &msg) { this->button_command(msg); } #endif #ifdef USE_CAMERA -void APIServerConnection::on_camera_image_request(const CameraImageRequest &msg) { - if (this->check_authenticated_()) { - this->camera_image(msg); - } -} +void APIServerConnection::on_camera_image_request(const CameraImageRequest &msg) { this->camera_image(msg); } #endif #ifdef USE_CLIMATE -void APIServerConnection::on_climate_command_request(const ClimateCommandRequest &msg) { - if (this->check_authenticated_()) { - this->climate_command(msg); - } -} +void APIServerConnection::on_climate_command_request(const ClimateCommandRequest &msg) { this->climate_command(msg); } #endif #ifdef USE_COVER -void APIServerConnection::on_cover_command_request(const CoverCommandRequest &msg) { - if (this->check_authenticated_()) { - this->cover_command(msg); - } -} +void APIServerConnection::on_cover_command_request(const CoverCommandRequest &msg) { this->cover_command(msg); } #endif #ifdef USE_DATETIME_DATE -void APIServerConnection::on_date_command_request(const DateCommandRequest &msg) { - if (this->check_authenticated_()) { - this->date_command(msg); - } -} +void APIServerConnection::on_date_command_request(const DateCommandRequest &msg) { this->date_command(msg); } #endif #ifdef USE_DATETIME_DATETIME void APIServerConnection::on_date_time_command_request(const DateTimeCommandRequest &msg) { - if (this->check_authenticated_()) { - this->datetime_command(msg); - } + this->datetime_command(msg); } #endif #ifdef USE_FAN -void APIServerConnection::on_fan_command_request(const FanCommandRequest &msg) { - if (this->check_authenticated_()) { - this->fan_command(msg); - } -} +void APIServerConnection::on_fan_command_request(const FanCommandRequest &msg) { this->fan_command(msg); } #endif #ifdef USE_LIGHT -void APIServerConnection::on_light_command_request(const LightCommandRequest &msg) { - if (this->check_authenticated_()) { - this->light_command(msg); - } -} +void APIServerConnection::on_light_command_request(const LightCommandRequest &msg) { this->light_command(msg); } #endif #ifdef USE_LOCK -void APIServerConnection::on_lock_command_request(const LockCommandRequest &msg) { - if (this->check_authenticated_()) { - this->lock_command(msg); - } -} +void APIServerConnection::on_lock_command_request(const LockCommandRequest &msg) { this->lock_command(msg); } #endif #ifdef USE_MEDIA_PLAYER void APIServerConnection::on_media_player_command_request(const MediaPlayerCommandRequest &msg) { - if (this->check_authenticated_()) { - this->media_player_command(msg); - } + this->media_player_command(msg); } #endif #ifdef USE_NUMBER -void APIServerConnection::on_number_command_request(const NumberCommandRequest &msg) { - if (this->check_authenticated_()) { - this->number_command(msg); - } -} +void APIServerConnection::on_number_command_request(const NumberCommandRequest &msg) { this->number_command(msg); } #endif #ifdef USE_SELECT -void APIServerConnection::on_select_command_request(const SelectCommandRequest &msg) { - if (this->check_authenticated_()) { - this->select_command(msg); - } -} +void APIServerConnection::on_select_command_request(const SelectCommandRequest &msg) { this->select_command(msg); } #endif #ifdef USE_SIREN -void APIServerConnection::on_siren_command_request(const SirenCommandRequest &msg) { - if (this->check_authenticated_()) { - this->siren_command(msg); - } -} +void APIServerConnection::on_siren_command_request(const SirenCommandRequest &msg) { this->siren_command(msg); } #endif #ifdef USE_SWITCH -void APIServerConnection::on_switch_command_request(const SwitchCommandRequest &msg) { - if (this->check_authenticated_()) { - this->switch_command(msg); - } -} +void APIServerConnection::on_switch_command_request(const SwitchCommandRequest &msg) { this->switch_command(msg); } #endif #ifdef USE_TEXT -void APIServerConnection::on_text_command_request(const TextCommandRequest &msg) { - if (this->check_authenticated_()) { - this->text_command(msg); - } -} +void APIServerConnection::on_text_command_request(const TextCommandRequest &msg) { this->text_command(msg); } #endif #ifdef USE_DATETIME_TIME -void APIServerConnection::on_time_command_request(const TimeCommandRequest &msg) { - if (this->check_authenticated_()) { - this->time_command(msg); - } -} +void APIServerConnection::on_time_command_request(const TimeCommandRequest &msg) { this->time_command(msg); } #endif #ifdef USE_UPDATE -void APIServerConnection::on_update_command_request(const UpdateCommandRequest &msg) { - if (this->check_authenticated_()) { - this->update_command(msg); - } -} +void APIServerConnection::on_update_command_request(const UpdateCommandRequest &msg) { this->update_command(msg); } #endif #ifdef USE_VALVE -void APIServerConnection::on_valve_command_request(const ValveCommandRequest &msg) { - if (this->check_authenticated_()) { - this->valve_command(msg); - } -} +void APIServerConnection::on_valve_command_request(const ValveCommandRequest &msg) { this->valve_command(msg); } #endif #ifdef USE_BLUETOOTH_PROXY void APIServerConnection::on_subscribe_bluetooth_le_advertisements_request( const SubscribeBluetoothLEAdvertisementsRequest &msg) { - if (this->check_authenticated_()) { - this->subscribe_bluetooth_le_advertisements(msg); - } + this->subscribe_bluetooth_le_advertisements(msg); } #endif #ifdef USE_BLUETOOTH_PROXY void APIServerConnection::on_bluetooth_device_request(const BluetoothDeviceRequest &msg) { - if (this->check_authenticated_()) { - this->bluetooth_device_request(msg); - } + this->bluetooth_device_request(msg); } #endif #ifdef USE_BLUETOOTH_PROXY void APIServerConnection::on_bluetooth_gatt_get_services_request(const BluetoothGATTGetServicesRequest &msg) { - if (this->check_authenticated_()) { - this->bluetooth_gatt_get_services(msg); - } + this->bluetooth_gatt_get_services(msg); } #endif #ifdef USE_BLUETOOTH_PROXY void APIServerConnection::on_bluetooth_gatt_read_request(const BluetoothGATTReadRequest &msg) { - if (this->check_authenticated_()) { - this->bluetooth_gatt_read(msg); - } + this->bluetooth_gatt_read(msg); } #endif #ifdef USE_BLUETOOTH_PROXY void APIServerConnection::on_bluetooth_gatt_write_request(const BluetoothGATTWriteRequest &msg) { - if (this->check_authenticated_()) { - this->bluetooth_gatt_write(msg); - } + this->bluetooth_gatt_write(msg); } #endif #ifdef USE_BLUETOOTH_PROXY void APIServerConnection::on_bluetooth_gatt_read_descriptor_request(const BluetoothGATTReadDescriptorRequest &msg) { - if (this->check_authenticated_()) { - this->bluetooth_gatt_read_descriptor(msg); - } + this->bluetooth_gatt_read_descriptor(msg); } #endif #ifdef USE_BLUETOOTH_PROXY void APIServerConnection::on_bluetooth_gatt_write_descriptor_request(const BluetoothGATTWriteDescriptorRequest &msg) { - if (this->check_authenticated_()) { - this->bluetooth_gatt_write_descriptor(msg); - } + this->bluetooth_gatt_write_descriptor(msg); } #endif #ifdef USE_BLUETOOTH_PROXY void APIServerConnection::on_bluetooth_gatt_notify_request(const BluetoothGATTNotifyRequest &msg) { - if (this->check_authenticated_()) { - this->bluetooth_gatt_notify(msg); - } + this->bluetooth_gatt_notify(msg); } #endif #ifdef USE_BLUETOOTH_PROXY void APIServerConnection::on_subscribe_bluetooth_connections_free_request( const SubscribeBluetoothConnectionsFreeRequest &msg) { - if (this->check_authenticated_()) { - BluetoothConnectionsFreeResponse ret = this->subscribe_bluetooth_connections_free(msg); - if (!this->send_message(ret)) { - this->on_fatal_error(); - } + if (!this->send_subscribe_bluetooth_connections_free_response(msg)) { + this->on_fatal_error(); } } #endif #ifdef USE_BLUETOOTH_PROXY void APIServerConnection::on_unsubscribe_bluetooth_le_advertisements_request( const UnsubscribeBluetoothLEAdvertisementsRequest &msg) { - if (this->check_authenticated_()) { - this->unsubscribe_bluetooth_le_advertisements(msg); - } + this->unsubscribe_bluetooth_le_advertisements(msg); } #endif #ifdef USE_BLUETOOTH_PROXY void APIServerConnection::on_bluetooth_scanner_set_mode_request(const BluetoothScannerSetModeRequest &msg) { - if (this->check_authenticated_()) { - this->bluetooth_scanner_set_mode(msg); - } + this->bluetooth_scanner_set_mode(msg); } #endif #ifdef USE_VOICE_ASSISTANT void APIServerConnection::on_subscribe_voice_assistant_request(const SubscribeVoiceAssistantRequest &msg) { - if (this->check_authenticated_()) { - this->subscribe_voice_assistant(msg); - } + this->subscribe_voice_assistant(msg); } #endif #ifdef USE_VOICE_ASSISTANT void APIServerConnection::on_voice_assistant_configuration_request(const VoiceAssistantConfigurationRequest &msg) { - if (this->check_authenticated_()) { - VoiceAssistantConfigurationResponse ret = this->voice_assistant_get_configuration(msg); - if (!this->send_message(ret)) { - this->on_fatal_error(); - } + if (!this->send_voice_assistant_get_configuration_response(msg)) { + this->on_fatal_error(); } } #endif #ifdef USE_VOICE_ASSISTANT void APIServerConnection::on_voice_assistant_set_configuration(const VoiceAssistantSetConfiguration &msg) { - if (this->check_authenticated_()) { - this->voice_assistant_set_configuration(msg); - } + this->voice_assistant_set_configuration(msg); } #endif #ifdef USE_ALARM_CONTROL_PANEL void APIServerConnection::on_alarm_control_panel_command_request(const AlarmControlPanelCommandRequest &msg) { - if (this->check_authenticated_()) { - this->alarm_control_panel_command(msg); - } + this->alarm_control_panel_command(msg); } #endif +#ifdef USE_ZWAVE_PROXY +void APIServerConnection::on_z_wave_proxy_frame(const ZWaveProxyFrame &msg) { this->zwave_proxy_frame(msg); } +#endif +#ifdef USE_ZWAVE_PROXY +void APIServerConnection::on_z_wave_proxy_request(const ZWaveProxyRequest &msg) { this->zwave_proxy_request(msg); } +#endif -} // namespace api -} // namespace esphome +void APIServerConnection::read_message(uint32_t msg_size, uint32_t msg_type, uint8_t *msg_data) { + // Check authentication/connection requirements for messages + switch (msg_type) { + case HelloRequest::MESSAGE_TYPE: // No setup required +#ifdef USE_API_PASSWORD + case AuthenticationRequest::MESSAGE_TYPE: // No setup required +#endif + case DisconnectRequest::MESSAGE_TYPE: // No setup required + case PingRequest::MESSAGE_TYPE: // No setup required + break; // Skip all checks for these messages + case DeviceInfoRequest::MESSAGE_TYPE: // Connection setup only + if (!this->check_connection_setup_()) { + return; // Connection not setup + } + break; + default: + // All other messages require authentication (which includes connection check) + if (!this->check_authenticated_()) { + return; // Authentication failed + } + break; + } + + // Call base implementation to process the message + APIServerConnectionBase::read_message(msg_size, msg_type, msg_data); +} + +} // namespace esphome::api diff --git a/esphome/components/api/api_pb2_service.h b/esphome/components/api/api_pb2_service.h index 9c5dc244fe..549b00ee6a 100644 --- a/esphome/components/api/api_pb2_service.h +++ b/esphome/components/api/api_pb2_service.h @@ -6,8 +6,7 @@ #include "api_pb2.h" -namespace esphome { -namespace api { +namespace esphome::api { class APIServerConnectionBase : public ProtoService { public: @@ -18,16 +17,18 @@ class APIServerConnectionBase : public ProtoService { public: #endif - template bool send_message(const T &msg) { + bool send_message(const ProtoMessage &msg, uint8_t message_type) { #ifdef HAS_PROTO_MESSAGE_DUMP this->log_send_message_(msg.message_name(), msg.dump()); #endif - return this->send_message_(msg, T::MESSAGE_TYPE); + return this->send_message_(msg, message_type); } virtual void on_hello_request(const HelloRequest &value){}; - virtual void on_connect_request(const ConnectRequest &value){}; +#ifdef USE_API_PASSWORD + virtual void on_authentication_request(const AuthenticationRequest &value){}; +#endif virtual void on_disconnect_request(const DisconnectRequest &value){}; virtual void on_disconnect_response(const DisconnectResponse &value){}; @@ -61,12 +62,21 @@ class APIServerConnectionBase : public ProtoService { virtual void on_noise_encryption_set_key_request(const NoiseEncryptionSetKeyRequest &value){}; #endif +#ifdef USE_API_HOMEASSISTANT_SERVICES virtual void on_subscribe_homeassistant_services_request(const SubscribeHomeassistantServicesRequest &value){}; +#endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES + virtual void on_homeassistant_action_response(const HomeassistantActionResponse &value){}; +#endif +#ifdef USE_API_HOMEASSISTANT_STATES virtual void on_subscribe_home_assistant_states_request(const SubscribeHomeAssistantStatesRequest &value){}; +#endif +#ifdef USE_API_HOMEASSISTANT_STATES virtual void on_home_assistant_state_response(const HomeAssistantStateResponse &value){}; - virtual void on_get_time_request(const GetTimeRequest &value){}; +#endif + virtual void on_get_time_response(const GetTimeResponse &value){}; #ifdef USE_API_SERVICES @@ -200,6 +210,12 @@ class APIServerConnectionBase : public ProtoService { #ifdef USE_UPDATE virtual void on_update_command_request(const UpdateCommandRequest &value){}; +#endif +#ifdef USE_ZWAVE_PROXY + virtual void on_z_wave_proxy_frame(const ZWaveProxyFrame &value){}; +#endif +#ifdef USE_ZWAVE_PROXY + virtual void on_z_wave_proxy_request(const ZWaveProxyRequest &value){}; #endif protected: void read_message(uint32_t msg_size, uint32_t msg_type, uint8_t *msg_data) override; @@ -207,22 +223,27 @@ class APIServerConnectionBase : public ProtoService { class APIServerConnection : public APIServerConnectionBase { public: - virtual HelloResponse hello(const HelloRequest &msg) = 0; - virtual ConnectResponse connect(const ConnectRequest &msg) = 0; - virtual DisconnectResponse disconnect(const DisconnectRequest &msg) = 0; - virtual PingResponse ping(const PingRequest &msg) = 0; - virtual DeviceInfoResponse device_info(const DeviceInfoRequest &msg) = 0; + virtual bool send_hello_response(const HelloRequest &msg) = 0; +#ifdef USE_API_PASSWORD + virtual bool send_authenticate_response(const AuthenticationRequest &msg) = 0; +#endif + virtual bool send_disconnect_response(const DisconnectRequest &msg) = 0; + virtual bool send_ping_response(const PingRequest &msg) = 0; + virtual bool send_device_info_response(const DeviceInfoRequest &msg) = 0; virtual void list_entities(const ListEntitiesRequest &msg) = 0; virtual void subscribe_states(const SubscribeStatesRequest &msg) = 0; virtual void subscribe_logs(const SubscribeLogsRequest &msg) = 0; +#ifdef USE_API_HOMEASSISTANT_SERVICES virtual void subscribe_homeassistant_services(const SubscribeHomeassistantServicesRequest &msg) = 0; +#endif +#ifdef USE_API_HOMEASSISTANT_STATES virtual void subscribe_home_assistant_states(const SubscribeHomeAssistantStatesRequest &msg) = 0; - virtual GetTimeResponse get_time(const GetTimeRequest &msg) = 0; +#endif #ifdef USE_API_SERVICES virtual void execute_service(const ExecuteServiceRequest &msg) = 0; #endif #ifdef USE_API_NOISE - virtual NoiseEncryptionSetKeyResponse noise_encryption_set_key(const NoiseEncryptionSetKeyRequest &msg) = 0; + virtual bool send_noise_encryption_set_key_response(const NoiseEncryptionSetKeyRequest &msg) = 0; #endif #ifdef USE_BUTTON virtual void button_command(const ButtonCommandRequest &msg) = 0; @@ -303,7 +324,7 @@ class APIServerConnection : public APIServerConnectionBase { virtual void bluetooth_gatt_notify(const BluetoothGATTNotifyRequest &msg) = 0; #endif #ifdef USE_BLUETOOTH_PROXY - virtual BluetoothConnectionsFreeResponse subscribe_bluetooth_connections_free( + virtual bool send_subscribe_bluetooth_connections_free_response( const SubscribeBluetoothConnectionsFreeRequest &msg) = 0; #endif #ifdef USE_BLUETOOTH_PROXY @@ -316,27 +337,37 @@ class APIServerConnection : public APIServerConnectionBase { virtual void subscribe_voice_assistant(const SubscribeVoiceAssistantRequest &msg) = 0; #endif #ifdef USE_VOICE_ASSISTANT - virtual VoiceAssistantConfigurationResponse voice_assistant_get_configuration( - const VoiceAssistantConfigurationRequest &msg) = 0; + virtual bool send_voice_assistant_get_configuration_response(const VoiceAssistantConfigurationRequest &msg) = 0; #endif #ifdef USE_VOICE_ASSISTANT virtual void voice_assistant_set_configuration(const VoiceAssistantSetConfiguration &msg) = 0; #endif #ifdef USE_ALARM_CONTROL_PANEL virtual void alarm_control_panel_command(const AlarmControlPanelCommandRequest &msg) = 0; +#endif +#ifdef USE_ZWAVE_PROXY + virtual void zwave_proxy_frame(const ZWaveProxyFrame &msg) = 0; +#endif +#ifdef USE_ZWAVE_PROXY + virtual void zwave_proxy_request(const ZWaveProxyRequest &msg) = 0; #endif protected: void on_hello_request(const HelloRequest &msg) override; - void on_connect_request(const ConnectRequest &msg) override; +#ifdef USE_API_PASSWORD + void on_authentication_request(const AuthenticationRequest &msg) override; +#endif void on_disconnect_request(const DisconnectRequest &msg) override; void on_ping_request(const PingRequest &msg) override; void on_device_info_request(const DeviceInfoRequest &msg) override; void on_list_entities_request(const ListEntitiesRequest &msg) override; void on_subscribe_states_request(const SubscribeStatesRequest &msg) override; void on_subscribe_logs_request(const SubscribeLogsRequest &msg) override; +#ifdef USE_API_HOMEASSISTANT_SERVICES void on_subscribe_homeassistant_services_request(const SubscribeHomeassistantServicesRequest &msg) override; +#endif +#ifdef USE_API_HOMEASSISTANT_STATES void on_subscribe_home_assistant_states_request(const SubscribeHomeAssistantStatesRequest &msg) override; - void on_get_time_request(const GetTimeRequest &msg) override; +#endif #ifdef USE_API_SERVICES void on_execute_service_request(const ExecuteServiceRequest &msg) override; #endif @@ -443,7 +474,13 @@ class APIServerConnection : public APIServerConnectionBase { #ifdef USE_ALARM_CONTROL_PANEL void on_alarm_control_panel_command_request(const AlarmControlPanelCommandRequest &msg) override; #endif +#ifdef USE_ZWAVE_PROXY + void on_z_wave_proxy_frame(const ZWaveProxyFrame &msg) override; +#endif +#ifdef USE_ZWAVE_PROXY + void on_z_wave_proxy_request(const ZWaveProxyRequest &msg) override; +#endif + void read_message(uint32_t msg_size, uint32_t msg_type, uint8_t *msg_data) override; }; -} // namespace api -} // namespace esphome +} // namespace esphome::api diff --git a/esphome/components/api/api_server.cpp b/esphome/components/api/api_server.cpp index f5be672c9a..778d9389ef 100644 --- a/esphome/components/api/api_server.cpp +++ b/esphome/components/api/api_server.cpp @@ -9,15 +9,18 @@ #include "esphome/core/log.h" #include "esphome/core/util.h" #include "esphome/core/version.h" +#ifdef USE_API_HOMEASSISTANT_SERVICES +#include "homeassistant_service.h" +#endif #ifdef USE_LOGGER #include "esphome/components/logger/logger.h" #endif #include +#include -namespace esphome { -namespace api { +namespace esphome::api { static const char *const TAG = "api"; @@ -31,7 +34,6 @@ APIServer::APIServer() { } void APIServer::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->setup_controller(); #ifdef USE_API_NOISE @@ -39,12 +41,14 @@ void APIServer::setup() { this->noise_pref_ = global_preferences->make_preference(hash, true); +#ifndef USE_API_NOISE_PSK_FROM_YAML + // Only load saved PSK if not set from YAML SavedNoisePsk noise_pref_saved{}; if (this->noise_pref_.load(&noise_pref_saved)) { ESP_LOGD(TAG, "Loaded saved Noise PSK"); - this->set_noise_psk(noise_pref_saved.psk); } +#endif #endif // Schedule reboot if no clients connect within timeout @@ -87,7 +91,7 @@ void APIServer::setup() { return; } - err = this->socket_->listen(4); + err = this->socket_->listen(this->listen_backlog_); if (err != 0) { ESP_LOGW(TAG, "Socket unable to listen: errno %d", errno); this->mark_failed(); @@ -105,7 +109,7 @@ void APIServer::setup() { return; } for (auto &c : this->clients_) { - if (!c->flags_.remove) + if (!c->flags_.remove && c->get_log_subscription_level() >= level) c->try_send_log_message(level, tag, message, message_len); } }); @@ -140,9 +144,19 @@ void APIServer::loop() { while (true) { struct sockaddr_storage source_addr; socklen_t addr_len = sizeof(source_addr); + auto sock = this->socket_->accept_loop_monitored((struct sockaddr *) &source_addr, &addr_len); if (!sock) break; + + // Check if we're at the connection limit + if (this->clients_.size() >= this->max_connections_) { + ESP_LOGW(TAG, "Max connections (%d), rejecting %s", this->max_connections_, sock->getpeername().c_str()); + // Immediately close - socket destructor will handle cleanup + sock.reset(); + continue; + } + ESP_LOGD(TAG, "Accept %s", sock->getpeername().c_str()); auto *conn = new APIConnection(std::move(sock), this); @@ -167,7 +181,8 @@ void APIServer::loop() { // Network is down - disconnect all clients for (auto &client : this->clients_) { client->on_fatal_error(); - ESP_LOGW(TAG, "%s: Network down; disconnect", client->get_client_combined_info().c_str()); + ESP_LOGW(TAG, "%s (%s): Network down; disconnect", client->client_info_.name.c_str(), + client->client_info_.peername.c_str()); } // Continue to process and clean up the clients below } @@ -185,9 +200,9 @@ void APIServer::loop() { // Rare case: handle disconnection #ifdef USE_API_CLIENT_DISCONNECTED_TRIGGER - this->client_disconnected_trigger_->trigger(client->client_info_, client->client_peername_); + this->client_disconnected_trigger_->trigger(client->client_info_.name, client->client_info_.peername); #endif - ESP_LOGV(TAG, "Remove connection %s", client->client_info_.c_str()); + ESP_LOGV(TAG, "Remove connection %s", client->client_info_.name.c_str()); // Swap with the last element and pop (avoids expensive vector shifts) if (client_index < this->clients_.size() - 1) { @@ -205,28 +220,28 @@ void APIServer::loop() { void APIServer::dump_config() { ESP_LOGCONFIG(TAG, - "API Server:\n" - " Address: %s:%u", - network::get_use_address().c_str(), this->port_); + "Server:\n" + " Address: %s:%u\n" + " Listen backlog: %u\n" + " Max connections: %u", + network::get_use_address().c_str(), this->port_, this->listen_backlog_, this->max_connections_); #ifdef USE_API_NOISE - ESP_LOGCONFIG(TAG, " Using noise encryption: %s", YESNO(this->noise_ctx_->has_psk())); + ESP_LOGCONFIG(TAG, " Noise encryption: %s", YESNO(this->noise_ctx_->has_psk())); if (!this->noise_ctx_->has_psk()) { - ESP_LOGCONFIG(TAG, " Supports noise encryption: YES"); + ESP_LOGCONFIG(TAG, " Supports encryption: YES"); } #else - ESP_LOGCONFIG(TAG, " Using noise encryption: NO"); + ESP_LOGCONFIG(TAG, " Noise encryption: NO"); #endif } #ifdef USE_API_PASSWORD -bool APIServer::uses_password() const { return !this->password_.empty(); } - -bool APIServer::check_password(const std::string &password) const { +bool APIServer::check_password(const uint8_t *password_data, size_t password_len) const { // depend only on input password length const char *a = this->password_.c_str(); uint32_t len_a = this->password_.length(); - const char *b = password.c_str(); - uint32_t len_b = password.length(); + const char *b = reinterpret_cast(password_data); + uint32_t len_b = password_len; // disable optimization with volatile volatile uint32_t length = len_b; @@ -249,6 +264,7 @@ bool APIServer::check_password(const std::string &password) const { return result == 0; } + #endif void APIServer::handle_disconnect(APIConnection *conn) {} @@ -359,6 +375,15 @@ void APIServer::on_update(update::UpdateEntity *obj) { } #endif +#ifdef USE_ZWAVE_PROXY +void APIServer::on_zwave_proxy_request(const esphome::api::ProtoMessage &msg) { + // We could add code to manage a second subscription type, but, since this message type is + // very infrequent and small, we simply send it to all clients + for (auto &c : this->clients_) + c->send_message(msg, api::ZWaveProxyRequest::MESSAGE_TYPE); +} +#endif + #ifdef USE_ALARM_CONTROL_PANEL API_DISPATCH_UPDATE(alarm_control_panel::AlarmControlPanel, alarm_control_panel) #endif @@ -373,12 +398,46 @@ void APIServer::set_password(const std::string &password) { this->password_ = pa void APIServer::set_batch_delay(uint16_t batch_delay) { this->batch_delay_ = batch_delay; } -void APIServer::send_homeassistant_service_call(const HomeassistantServiceResponse &call) { +#ifdef USE_API_HOMEASSISTANT_SERVICES +void APIServer::send_homeassistant_action(const HomeassistantActionRequest &call) { for (auto &client : this->clients_) { - client->send_homeassistant_service_call(call); + client->send_homeassistant_action(call); } } +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES +void APIServer::register_action_response_callback(uint32_t call_id, ActionResponseCallback callback) { + this->action_response_callbacks_.push_back({call_id, std::move(callback)}); +} +void APIServer::handle_action_response(uint32_t call_id, bool success, const std::string &error_message) { + for (auto it = this->action_response_callbacks_.begin(); it != this->action_response_callbacks_.end(); ++it) { + if (it->call_id == call_id) { + auto callback = std::move(it->callback); + this->action_response_callbacks_.erase(it); + ActionResponse response(success, error_message); + callback(response); + return; + } + } +} +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON +void APIServer::handle_action_response(uint32_t call_id, bool success, const std::string &error_message, + const uint8_t *response_data, size_t response_data_len) { + for (auto it = this->action_response_callbacks_.begin(); it != this->action_response_callbacks_.end(); ++it) { + if (it->call_id == call_id) { + auto callback = std::move(it->callback); + this->action_response_callbacks_.erase(it); + ActionResponse response(success, error_message, response_data, response_data_len); + callback(response); + return; + } + } +} +#endif // USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON +#endif // USE_API_HOMEASSISTANT_ACTION_RESPONSES +#endif // USE_API_HOMEASSISTANT_SERVICES + +#ifdef USE_API_HOMEASSISTANT_STATES void APIServer::subscribe_home_assistant_state(std::string entity_id, optional attribute, std::function f) { this->state_subs_.push_back(HomeAssistantStateSubscription{ @@ -402,6 +461,7 @@ void APIServer::get_home_assistant_state(std::string entity_id, optional &APIServer::get_state_subs() const { return this->state_subs_; } +#endif uint16_t APIServer::get_port() const { return this->port_; } @@ -409,6 +469,12 @@ void APIServer::set_reboot_timeout(uint32_t reboot_timeout) { this->reboot_timeo #ifdef USE_API_NOISE bool APIServer::save_noise_psk(psk_t psk, bool make_active) { +#ifdef USE_API_NOISE_PSK_FROM_YAML + // When PSK is set from YAML, this function should never be called + // but if it is, reject the change + ESP_LOGW(TAG, "Key set in YAML"); + return false; +#else auto &old_psk = this->noise_ctx_->get_psk(); if (std::equal(old_psk.begin(), old_psk.end(), psk.begin())) { ESP_LOGW(TAG, "New PSK matches old"); @@ -428,14 +494,16 @@ bool APIServer::save_noise_psk(psk_t psk, bool make_active) { ESP_LOGD(TAG, "Noise PSK saved"); if (make_active) { this->set_timeout(100, [this, psk]() { - ESP_LOGW(TAG, "Disconnecting all clients to reset connections"); + ESP_LOGW(TAG, "Disconnecting all clients to reset PSK"); this->set_noise_psk(psk); for (auto &c : this->clients_) { - c->send_message(DisconnectRequest()); + DisconnectRequest req; + c->send_message(req, DisconnectRequest::MESSAGE_TYPE); } }); } return true; +#endif } #endif @@ -464,7 +532,8 @@ void APIServer::on_shutdown() { // Send disconnect requests to all connected clients for (auto &c : this->clients_) { - if (!c->send_message(DisconnectRequest())) { + DisconnectRequest req; + if (!c->send_message(req, DisconnectRequest::MESSAGE_TYPE)) { // If we can't send the disconnect request directly (tx_buffer full), // schedule it at the front of the batch so it will be sent with priority c->schedule_message_front_(nullptr, &APIConnection::try_send_disconnect_request, DisconnectRequest::MESSAGE_TYPE, @@ -484,6 +553,5 @@ bool APIServer::teardown() { return this->clients_.empty(); } -} // namespace api -} // namespace esphome +} // namespace esphome::api #endif diff --git a/esphome/components/api/api_server.h b/esphome/components/api/api_server.h index f41064b62b..5d038e5ddd 100644 --- a/esphome/components/api/api_server.h +++ b/esphome/components/api/api_server.h @@ -16,10 +16,10 @@ #include "user_services.h" #endif +#include #include -namespace esphome { -namespace api { +namespace esphome::api { #ifdef USE_API_NOISE struct SavedNoisePsk { @@ -38,14 +38,15 @@ class APIServer : public Component, public Controller { void on_shutdown() override; bool teardown() override; #ifdef USE_API_PASSWORD - bool check_password(const std::string &password) const; - bool uses_password() const; + bool check_password(const uint8_t *password_data, size_t password_len) const; void set_password(const std::string &password); #endif void set_port(uint16_t port); void set_reboot_timeout(uint32_t reboot_timeout); void set_batch_delay(uint16_t batch_delay); uint16_t get_batch_delay() const { return batch_delay_; } + void set_listen_backlog(uint8_t listen_backlog) { this->listen_backlog_ = listen_backlog; } + void set_max_connections(uint8_t max_connections) { this->max_connections_ = max_connections; } // Get reference to shared buffer for API connections std::vector &get_shared_buffer_ref() { return shared_write_buffer_; } @@ -108,7 +109,20 @@ class APIServer : public Component, public Controller { #ifdef USE_MEDIA_PLAYER void on_media_player_update(media_player::MediaPlayer *obj) override; #endif - void send_homeassistant_service_call(const HomeassistantServiceResponse &call); +#ifdef USE_API_HOMEASSISTANT_SERVICES + void send_homeassistant_action(const HomeassistantActionRequest &call); + +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES + // Action response handling + using ActionResponseCallback = std::function; + void register_action_response_callback(uint32_t call_id, ActionResponseCallback callback); + void handle_action_response(uint32_t call_id, bool success, const std::string &error_message); +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + void handle_action_response(uint32_t call_id, bool success, const std::string &error_message, + const uint8_t *response_data, size_t response_data_len); +#endif // USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON +#endif // USE_API_HOMEASSISTANT_ACTION_RESPONSES +#endif // USE_API_HOMEASSISTANT_SERVICES #ifdef USE_API_SERVICES void register_user_service(UserServiceDescriptor *descriptor) { this->user_services_.push_back(descriptor); } #endif @@ -125,9 +139,13 @@ class APIServer : public Component, public Controller { #ifdef USE_UPDATE void on_update(update::UpdateEntity *obj) override; #endif +#ifdef USE_ZWAVE_PROXY + void on_zwave_proxy_request(const esphome::api::ProtoMessage &msg); +#endif bool is_connected() const; +#ifdef USE_API_HOMEASSISTANT_STATES struct HomeAssistantStateSubscription { std::string entity_id; optional attribute; @@ -140,6 +158,7 @@ class APIServer : public Component, public Controller { void get_home_assistant_state(std::string entity_id, optional attribute, std::function f); const std::vector &get_state_subs() const; +#endif #ifdef USE_API_SERVICES const std::vector &get_user_services() const { return this->user_services_; } #endif @@ -173,16 +192,29 @@ class APIServer : public Component, public Controller { std::string password_; #endif std::vector shared_write_buffer_; // Shared proto write buffer for all connections +#ifdef USE_API_HOMEASSISTANT_STATES std::vector state_subs_; +#endif #ifdef USE_API_SERVICES std::vector user_services_; #endif +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES + struct PendingActionResponse { + uint32_t call_id; + ActionResponseCallback callback; + }; + std::vector action_response_callbacks_; +#endif // Group smaller types together uint16_t port_{6053}; uint16_t batch_delay_{100}; + // Connection limits - these defaults will be overridden by config values + // from cv.SplitDefault in __init__.py which sets platform-specific defaults + uint8_t listen_backlog_{4}; + uint8_t max_connections_{8}; bool shutting_down_ = false; - // 5 bytes used, 3 bytes padding + // 7 bytes used, 1 byte padding #ifdef USE_API_NOISE std::shared_ptr noise_ctx_ = std::make_shared(); @@ -197,6 +229,5 @@ template class APIConnectedCondition : public Condition { bool check(Ts... x) override { return global_api_server->is_connected(); } }; -} // namespace api -} // namespace esphome +} // namespace esphome::api #endif diff --git a/esphome/components/api/client.py b/esphome/components/api/client.py index 2d4bc37c89..ca1fc089fa 100644 --- a/esphome/components/api/client.py +++ b/esphome/components/api/client.py @@ -14,6 +14,8 @@ with warnings.catch_warnings(): from aioesphomeapi import APIClient, parse_log_message from aioesphomeapi.log_runner import async_run +import contextlib + from esphome.const import CONF_KEY, CONF_PASSWORD, CONF_PORT, __version__ from esphome.core import CORE @@ -28,7 +30,7 @@ if TYPE_CHECKING: _LOGGER = logging.getLogger(__name__) -async def async_run_logs(config: dict[str, Any], address: str) -> None: +async def async_run_logs(config: dict[str, Any], addresses: list[str]) -> None: """Run the logs command in the event loop.""" conf = config["api"] name = config["esphome"]["name"] @@ -37,13 +39,21 @@ async def async_run_logs(config: dict[str, Any], address: str) -> None: noise_psk: str | None = None if (encryption := conf.get(CONF_ENCRYPTION)) and (key := encryption.get(CONF_KEY)): noise_psk = key - _LOGGER.info("Starting log output from %s using esphome API", address) + + if len(addresses) == 1: + _LOGGER.info("Starting log output from %s using esphome API", addresses[0]) + else: + _LOGGER.info( + "Starting log output from %s using esphome API", " or ".join(addresses) + ) + cli = APIClient( - address, + addresses[0], # Primary address for compatibility port, password, client_info=f"ESPHome Logs {__version__}", noise_psk=noise_psk, + addresses=addresses, # Pass all addresses for automatic retry ) dashboard = CORE.dashboard @@ -52,9 +62,11 @@ async def async_run_logs(config: dict[str, Any], address: str) -> None: time_ = datetime.now() message: bytes = msg.message text = message.decode("utf8", "backslashreplace") - for parsed_msg in parse_log_message( - text, f"[{time_.hour:02}:{time_.minute:02}:{time_.second:02}]" - ): + nanoseconds = time_.microsecond // 1000 + timestamp = ( + f"[{time_.hour:02}:{time_.minute:02}:{time_.second:02}.{nanoseconds:03}]" + ) + for parsed_msg in parse_log_message(text, timestamp): print(parsed_msg.replace("\033", "\\033") if dashboard else parsed_msg) stop = await async_run(cli, on_log, name=name) @@ -64,9 +76,7 @@ async def async_run_logs(config: dict[str, Any], address: str) -> None: await stop() -def run_logs(config: dict[str, Any], address: str) -> None: +def run_logs(config: dict[str, Any], addresses: list[str]) -> None: """Run the logs command.""" - try: - asyncio.run(async_run_logs(config, address)) - except KeyboardInterrupt: - pass + with contextlib.suppress(KeyboardInterrupt): + asyncio.run(async_run_logs(config, addresses)) diff --git a/esphome/components/api/custom_api_device.h b/esphome/components/api/custom_api_device.h index 35329c4a5e..711eba2444 100644 --- a/esphome/components/api/custom_api_device.h +++ b/esphome/components/api/custom_api_device.h @@ -6,8 +6,7 @@ #ifdef USE_API_SERVICES #include "user_services.h" #endif -namespace esphome { -namespace api { +namespace esphome::api { #ifdef USE_API_SERVICES template class CustomAPIDeviceService : public UserServiceBase { @@ -57,6 +56,14 @@ class CustomAPIDevice { auto *service = new CustomAPIDeviceService(name, arg_names, (T *) this, callback); // NOLINT global_api_server->register_user_service(service); } +#else + template + void register_service(void (T::*callback)(Ts...), const std::string &name, + const std::array &arg_names) { + static_assert( + sizeof(T) == 0, + "register_service() requires 'custom_services: true' in the 'api:' section of your YAML configuration"); + } #endif /** Register a custom native API service that will show up in Home Assistant. @@ -82,8 +89,15 @@ class CustomAPIDevice { auto *service = new CustomAPIDeviceService(name, {}, (T *) this, callback); // NOLINT global_api_server->register_user_service(service); } +#else + template void register_service(void (T::*callback)(), const std::string &name) { + static_assert( + sizeof(T) == 0, + "register_service() requires 'custom_services: true' in the 'api:' section of your YAML configuration"); + } #endif +#ifdef USE_API_HOMEASSISTANT_STATES /** Subscribe to the state (or attribute state) of an entity from Home Assistant. * * Usage: @@ -135,7 +149,25 @@ class CustomAPIDevice { auto f = std::bind(callback, (T *) this, entity_id, std::placeholders::_1); global_api_server->subscribe_home_assistant_state(entity_id, optional(attribute), f); } +#else + template + void subscribe_homeassistant_state(void (T::*callback)(std::string), const std::string &entity_id, + const std::string &attribute = "") { + static_assert(sizeof(T) == 0, + "subscribe_homeassistant_state() requires 'homeassistant_states: true' in the 'api:' section " + "of your YAML configuration"); + } + template + void subscribe_homeassistant_state(void (T::*callback)(std::string, std::string), const std::string &entity_id, + const std::string &attribute = "") { + static_assert(sizeof(T) == 0, + "subscribe_homeassistant_state() requires 'homeassistant_states: true' in the 'api:' section " + "of your YAML configuration"); + } +#endif + +#ifdef USE_API_HOMEASSISTANT_SERVICES /** Call a Home Assistant service from ESPHome. * * Usage: @@ -147,9 +179,9 @@ class CustomAPIDevice { * @param service_name The service to call. */ void call_homeassistant_service(const std::string &service_name) { - HomeassistantServiceResponse resp; - resp.service = service_name; - global_api_server->send_homeassistant_service_call(resp); + HomeassistantActionRequest resp; + resp.set_service(StringRef(service_name)); + global_api_server->send_homeassistant_action(resp); } /** Call a Home Assistant service from ESPHome. @@ -167,15 +199,15 @@ class CustomAPIDevice { * @param data The data for the service call, mapping from string to string. */ void call_homeassistant_service(const std::string &service_name, const std::map &data) { - HomeassistantServiceResponse resp; - resp.service = service_name; + HomeassistantActionRequest resp; + resp.set_service(StringRef(service_name)); + resp.data.init(data.size()); for (auto &it : data) { - HomeassistantServiceMap kv; - kv.key = it.first; + auto &kv = resp.data.emplace_back(); + kv.set_key(StringRef(it.first)); kv.value = it.second; - resp.data.push_back(kv); } - global_api_server->send_homeassistant_service_call(resp); + global_api_server->send_homeassistant_action(resp); } /** Fire an ESPHome event in Home Assistant. @@ -189,10 +221,10 @@ class CustomAPIDevice { * @param event_name The event to fire. */ void fire_homeassistant_event(const std::string &event_name) { - HomeassistantServiceResponse resp; - resp.service = event_name; + HomeassistantActionRequest resp; + resp.set_service(StringRef(event_name)); resp.is_event = true; - global_api_server->send_homeassistant_service_call(resp); + global_api_server->send_homeassistant_action(resp); } /** Fire an ESPHome event in Home Assistant. @@ -209,19 +241,41 @@ class CustomAPIDevice { * @param data The data for the event, mapping from string to string. */ void fire_homeassistant_event(const std::string &service_name, const std::map &data) { - HomeassistantServiceResponse resp; - resp.service = service_name; + HomeassistantActionRequest resp; + resp.set_service(StringRef(service_name)); resp.is_event = true; + resp.data.init(data.size()); for (auto &it : data) { - HomeassistantServiceMap kv; - kv.key = it.first; + auto &kv = resp.data.emplace_back(); + kv.set_key(StringRef(it.first)); kv.value = it.second; - resp.data.push_back(kv); } - global_api_server->send_homeassistant_service_call(resp); + global_api_server->send_homeassistant_action(resp); } +#else + template void call_homeassistant_service(const std::string &service_name) { + static_assert(sizeof(T) == 0, "call_homeassistant_service() requires 'homeassistant_services: true' in the 'api:' " + "section of your YAML configuration"); + } + + template + void call_homeassistant_service(const std::string &service_name, const std::map &data) { + static_assert(sizeof(T) == 0, "call_homeassistant_service() requires 'homeassistant_services: true' in the 'api:' " + "section of your YAML configuration"); + } + + template void fire_homeassistant_event(const std::string &event_name) { + static_assert(sizeof(T) == 0, "fire_homeassistant_event() requires 'homeassistant_services: true' in the 'api:' " + "section of your YAML configuration"); + } + + template + void fire_homeassistant_event(const std::string &service_name, const std::map &data) { + static_assert(sizeof(T) == 0, "fire_homeassistant_event() requires 'homeassistant_services: true' in the 'api:' " + "section of your YAML configuration"); + } +#endif }; -} // namespace api -} // namespace esphome +} // namespace esphome::api #endif diff --git a/esphome/components/api/homeassistant_service.h b/esphome/components/api/homeassistant_service.h index 32d13b69ae..4343fcd0bb 100644 --- a/esphome/components/api/homeassistant_service.h +++ b/esphome/components/api/homeassistant_service.h @@ -2,15 +2,32 @@ #include "api_server.h" #ifdef USE_API +#ifdef USE_API_HOMEASSISTANT_SERVICES +#include +#include +#include #include "api_pb2.h" +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON +#include "esphome/components/json/json_util.h" +#endif #include "esphome/core/automation.h" #include "esphome/core/helpers.h" -#include -namespace esphome { -namespace api { +namespace esphome::api { template class TemplatableStringValue : public TemplatableValue { + private: + // Helper to convert value to string - handles the case where value is already a string + template static std::string value_to_string(T &&val) { return to_string(std::forward(val)); } + + // Overloads for string types - needed because std::to_string doesn't support them + static std::string value_to_string(char *val) { + return val ? std::string(val) : std::string(); + } // For lambdas returning char* (e.g., itoa) + static std::string value_to_string(const char *val) { return std::string(val); } // For lambdas returning .c_str() + static std::string value_to_string(const std::string &val) { return val; } + static std::string value_to_string(std::string &&val) { return std::move(val); } + public: TemplatableStringValue() : TemplatableValue() {} @@ -19,66 +36,196 @@ template class TemplatableStringValue : public TemplatableValue::value, int> = 0> TemplatableStringValue(F f) - : TemplatableValue([f](X... x) -> std::string { return to_string(f(x...)); }) {} + : TemplatableValue([f](X... x) -> std::string { return value_to_string(f(x...)); }) {} }; template class TemplatableKeyValuePair { public: + // Default constructor needed for FixedVector::emplace_back() + TemplatableKeyValuePair() = default; + + // Keys are always string literals from YAML dictionary keys (e.g., "code", "event") + // and never templatable values or lambdas. Only the value parameter can be a lambda/template. + // Using pass-by-value with std::move allows optimal performance for both lvalues and rvalues. template TemplatableKeyValuePair(std::string key, T value) : key(std::move(key)), value(value) {} + std::string key; TemplatableStringValue value; }; +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES +// Represents the response data from a Home Assistant action +class ActionResponse { + public: + ActionResponse(bool success, std::string error_message = "") + : success_(success), error_message_(std::move(error_message)) {} + +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + ActionResponse(bool success, std::string error_message, const uint8_t *data, size_t data_len) + : success_(success), error_message_(std::move(error_message)) { + if (data == nullptr || data_len == 0) + return; + this->json_document_ = json::parse_json(data, data_len); + } +#endif + + bool is_success() const { return this->success_; } + const std::string &get_error_message() const { return this->error_message_; } + +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + // Get data as parsed JSON object (const version returns read-only view) + JsonObjectConst get_json() const { return this->json_document_.as(); } +#endif + + protected: + bool success_; + std::string error_message_; +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + JsonDocument json_document_; +#endif +}; + +// Callback type for action responses +template using ActionResponseCallback = std::function; +#endif + template class HomeAssistantServiceCallAction : public Action { public: - explicit HomeAssistantServiceCallAction(APIServer *parent, bool is_event) : parent_(parent), is_event_(is_event) {} + explicit HomeAssistantServiceCallAction(APIServer *parent, bool is_event) : parent_(parent) { + this->flags_.is_event = is_event; + } template void set_service(T service) { this->service_ = service; } - template void add_data(std::string key, T value) { - this->data_.push_back(TemplatableKeyValuePair(key, value)); + // Initialize FixedVector members - called from Python codegen with compile-time known sizes. + // Must be called before any add_* methods; capacity must match the number of subsequent add_* calls. + void init_data(size_t count) { this->data_.init(count); } + void init_data_template(size_t count) { this->data_template_.init(count); } + void init_variables(size_t count) { this->variables_.init(count); } + + // Keys are always string literals from the Python code generation (e.g., cg.add(var.add_data("tag_id", templ))). + // The value parameter can be a lambda/template, but keys are never templatable. + template void add_data(K &&key, V &&value) { + this->add_kv_(this->data_, std::forward(key), std::forward(value)); } - template void add_data_template(std::string key, T value) { - this->data_template_.push_back(TemplatableKeyValuePair(key, value)); + template void add_data_template(K &&key, V &&value) { + this->add_kv_(this->data_template_, std::forward(key), std::forward(value)); } - template void add_variable(std::string key, T value) { - this->variables_.push_back(TemplatableKeyValuePair(key, value)); + template void add_variable(K &&key, V &&value) { + this->add_kv_(this->variables_, std::forward(key), std::forward(value)); } +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES + template void set_response_template(T response_template) { + this->response_template_ = response_template; + this->flags_.has_response_template = true; + } + + void set_wants_status() { this->flags_.wants_status = true; } + void set_wants_response() { this->flags_.wants_response = true; } + +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + Trigger *get_success_trigger_with_response() const { + return this->success_trigger_with_response_; + } +#endif + Trigger *get_success_trigger() const { return this->success_trigger_; } + Trigger *get_error_trigger() const { return this->error_trigger_; } +#endif // USE_API_HOMEASSISTANT_ACTION_RESPONSES + void play(Ts... x) override { - HomeassistantServiceResponse resp; - resp.service = this->service_.value(x...); - resp.is_event = this->is_event_; - for (auto &it : this->data_) { - HomeassistantServiceMap kv; - kv.key = it.key; - kv.value = it.value.value(x...); - resp.data.push_back(kv); + HomeassistantActionRequest resp; + std::string service_value = this->service_.value(x...); + resp.set_service(StringRef(service_value)); + resp.is_event = this->flags_.is_event; + this->populate_service_map(resp.data, this->data_, x...); + this->populate_service_map(resp.data_template, this->data_template_, x...); + this->populate_service_map(resp.variables, this->variables_, x...); + +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES + if (this->flags_.wants_status) { + // Generate a unique call ID for this service call + static uint32_t call_id_counter = 1; + uint32_t call_id = call_id_counter++; + resp.call_id = call_id; +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + if (this->flags_.wants_response) { + resp.wants_response = true; + // Set response template if provided + if (this->flags_.has_response_template) { + std::string response_template_value = this->response_template_.value(x...); + resp.response_template = response_template_value; + } + } +#endif + + auto captured_args = std::make_tuple(x...); + this->parent_->register_action_response_callback(call_id, [this, captured_args](const ActionResponse &response) { + std::apply( + [this, &response](auto &&...args) { + if (response.is_success()) { +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + if (this->flags_.wants_response) { + this->success_trigger_with_response_->trigger(response.get_json(), args...); + } else +#endif + { + this->success_trigger_->trigger(args...); + } + } else { + this->error_trigger_->trigger(response.get_error_message(), args...); + } + }, + captured_args); + }); } - for (auto &it : this->data_template_) { - HomeassistantServiceMap kv; - kv.key = it.key; - kv.value = it.value.value(x...); - resp.data_template.push_back(kv); - } - for (auto &it : this->variables_) { - HomeassistantServiceMap kv; - kv.key = it.key; - kv.value = it.value.value(x...); - resp.variables.push_back(kv); - } - this->parent_->send_homeassistant_service_call(resp); +#endif + + this->parent_->send_homeassistant_action(resp); } protected: + // Helper to add key-value pairs to FixedVectors with perfect forwarding to avoid copies + template void add_kv_(FixedVector> &vec, K &&key, V &&value) { + auto &kv = vec.emplace_back(); + kv.key = std::forward(key); + kv.value = std::forward(value); + } + + template + static void populate_service_map(VectorType &dest, SourceType &source, Ts... x) { + dest.init(source.size()); + for (auto &it : source) { + auto &kv = dest.emplace_back(); + kv.set_key(StringRef(it.key)); + kv.value = it.value.value(x...); + } + } + APIServer *parent_; - bool is_event_; TemplatableStringValue service_{}; - std::vector> data_; - std::vector> data_template_; - std::vector> variables_; + FixedVector> data_; + FixedVector> data_template_; + FixedVector> variables_; +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES +#ifdef USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + TemplatableStringValue response_template_{""}; + Trigger *success_trigger_with_response_ = new Trigger(); +#endif // USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON + Trigger *success_trigger_ = new Trigger(); + Trigger *error_trigger_ = new Trigger(); +#endif // USE_API_HOMEASSISTANT_ACTION_RESPONSES + + struct Flags { + uint8_t is_event : 1; + uint8_t wants_status : 1; + uint8_t wants_response : 1; + uint8_t has_response_template : 1; + uint8_t reserved : 5; + } flags_{0}; }; -} // namespace api -} // namespace esphome +} // namespace esphome::api + +#endif #endif diff --git a/esphome/components/api/list_entities.cpp b/esphome/components/api/list_entities.cpp index 1fbe68117b..da4800a45e 100644 --- a/esphome/components/api/list_entities.cpp +++ b/esphome/components/api/list_entities.cpp @@ -6,8 +6,7 @@ #include "esphome/core/log.h" #include "esphome/core/util.h" -namespace esphome { -namespace api { +namespace esphome::api { // Generate entity handler implementations using macros #ifdef USE_BINARY_SENSOR @@ -86,10 +85,9 @@ ListEntitiesIterator::ListEntitiesIterator(APIConnection *client) : client_(clie #ifdef USE_API_SERVICES bool ListEntitiesIterator::on_service(UserServiceDescriptor *service) { auto resp = service->encode_list_service_response(); - return this->client_->send_message(resp); + return this->client_->send_message(resp, ListEntitiesServicesResponse::MESSAGE_TYPE); } #endif -} // namespace api -} // namespace esphome +} // namespace esphome::api #endif diff --git a/esphome/components/api/list_entities.h b/esphome/components/api/list_entities.h index b4cbf6c489..769d7b9b6e 100644 --- a/esphome/components/api/list_entities.h +++ b/esphome/components/api/list_entities.h @@ -4,8 +4,7 @@ #ifdef USE_API #include "esphome/core/component.h" #include "esphome/core/component_iterator.h" -namespace esphome { -namespace api { +namespace esphome::api { class APIConnection; @@ -96,6 +95,5 @@ class ListEntitiesIterator : public ComponentIterator { APIConnection *client_; }; -} // namespace api -} // namespace esphome +} // namespace esphome::api #endif diff --git a/esphome/components/api/proto.cpp b/esphome/components/api/proto.cpp index 25daf17ccc..4f0d0846d7 100644 --- a/esphome/components/api/proto.cpp +++ b/esphome/components/api/proto.cpp @@ -3,80 +3,138 @@ #include "esphome/core/helpers.h" #include "esphome/core/log.h" -namespace esphome { -namespace api { +namespace esphome::api { static const char *const TAG = "api.proto"; -void ProtoMessage::decode(const uint8_t *buffer, size_t length) { - uint32_t i = 0; - bool error = false; - while (i < length) { +uint32_t ProtoDecodableMessage::count_repeated_field(const uint8_t *buffer, size_t length, uint32_t target_field_id) { + uint32_t count = 0; + const uint8_t *ptr = buffer; + const uint8_t *end = buffer + length; + + while (ptr < end) { uint32_t consumed; - auto res = ProtoVarInt::parse(&buffer[i], length - i, &consumed); + + // Parse field header (tag) + auto res = ProtoVarInt::parse(ptr, end - ptr, &consumed); if (!res.has_value()) { - ESP_LOGV(TAG, "Invalid field start at %" PRIu32, i); - break; + break; // Invalid data, stop counting } - uint32_t field_type = (res->as_uint32()) & 0b111; - uint32_t field_id = (res->as_uint32()) >> 3; - i += consumed; + uint32_t tag = res->as_uint32(); + uint32_t field_type = tag & WIRE_TYPE_MASK; + uint32_t field_id = tag >> 3; + ptr += consumed; + + // Count if this is the target field + if (field_id == target_field_id) { + count++; + } + + // Skip field data based on wire type + switch (field_type) { + case WIRE_TYPE_VARINT: { // VarInt - parse and skip + res = ProtoVarInt::parse(ptr, end - ptr, &consumed); + if (!res.has_value()) { + return count; // Invalid data, return what we have + } + ptr += consumed; + break; + } + case WIRE_TYPE_LENGTH_DELIMITED: { // Length-delimited - parse length and skip data + res = ProtoVarInt::parse(ptr, end - ptr, &consumed); + if (!res.has_value()) { + return count; + } + uint32_t field_length = res->as_uint32(); + ptr += consumed; + if (ptr + field_length > end) { + return count; // Out of bounds + } + ptr += field_length; + break; + } + case WIRE_TYPE_FIXED32: { // 32-bit - skip 4 bytes + if (ptr + 4 > end) { + return count; + } + ptr += 4; + break; + } + default: + // Unknown wire type, can't continue + return count; + } + } + + return count; +} + +void ProtoDecodableMessage::decode(const uint8_t *buffer, size_t length) { + const uint8_t *ptr = buffer; + const uint8_t *end = buffer + length; + + while (ptr < end) { + uint32_t consumed; + + // Parse field header + auto res = ProtoVarInt::parse(ptr, end - ptr, &consumed); + if (!res.has_value()) { + ESP_LOGV(TAG, "Invalid field start at offset %ld", (long) (ptr - buffer)); + return; + } + + uint32_t tag = res->as_uint32(); + uint32_t field_type = tag & WIRE_TYPE_MASK; + uint32_t field_id = tag >> 3; + ptr += consumed; switch (field_type) { - case 0: { // VarInt - res = ProtoVarInt::parse(&buffer[i], length - i, &consumed); + case WIRE_TYPE_VARINT: { // VarInt + res = ProtoVarInt::parse(ptr, end - ptr, &consumed); if (!res.has_value()) { - ESP_LOGV(TAG, "Invalid VarInt at %" PRIu32, i); - error = true; - break; + ESP_LOGV(TAG, "Invalid VarInt at offset %ld", (long) (ptr - buffer)); + return; } if (!this->decode_varint(field_id, *res)) { ESP_LOGV(TAG, "Cannot decode VarInt field %" PRIu32 " with value %" PRIu32 "!", field_id, res->as_uint32()); } - i += consumed; + ptr += consumed; break; } - case 2: { // Length-delimited - res = ProtoVarInt::parse(&buffer[i], length - i, &consumed); + case WIRE_TYPE_LENGTH_DELIMITED: { // Length-delimited + res = ProtoVarInt::parse(ptr, end - ptr, &consumed); if (!res.has_value()) { - ESP_LOGV(TAG, "Invalid Length Delimited at %" PRIu32, i); - error = true; - break; + ESP_LOGV(TAG, "Invalid Length Delimited at offset %ld", (long) (ptr - buffer)); + return; } uint32_t field_length = res->as_uint32(); - i += consumed; - if (field_length > length - i) { - ESP_LOGV(TAG, "Out-of-bounds Length Delimited at %" PRIu32, i); - error = true; - break; + ptr += consumed; + if (ptr + field_length > end) { + ESP_LOGV(TAG, "Out-of-bounds Length Delimited at offset %ld", (long) (ptr - buffer)); + return; } - if (!this->decode_length(field_id, ProtoLengthDelimited(&buffer[i], field_length))) { + if (!this->decode_length(field_id, ProtoLengthDelimited(ptr, field_length))) { ESP_LOGV(TAG, "Cannot decode Length Delimited field %" PRIu32 "!", field_id); } - i += field_length; + ptr += field_length; break; } - case 5: { // 32-bit - if (length - i < 4) { - ESP_LOGV(TAG, "Out-of-bounds Fixed32-bit at %" PRIu32, i); - error = true; - break; + case WIRE_TYPE_FIXED32: { // 32-bit + if (ptr + 4 > end) { + ESP_LOGV(TAG, "Out-of-bounds Fixed32-bit at offset %ld", (long) (ptr - buffer)); + return; } - uint32_t val = encode_uint32(buffer[i + 3], buffer[i + 2], buffer[i + 1], buffer[i]); + uint32_t val = encode_uint32(ptr[3], ptr[2], ptr[1], ptr[0]); if (!this->decode_32bit(field_id, Proto32Bit(val))) { ESP_LOGV(TAG, "Cannot decode 32-bit field %" PRIu32 " with value %" PRIu32 "!", field_id, val); } - i += 4; + ptr += 4; break; } default: - ESP_LOGV(TAG, "Invalid field type at %" PRIu32, i); - error = true; - break; - } - if (error) { - break; + ESP_LOGV(TAG, "Invalid field type %u at offset %ld", field_type, (long) (ptr - buffer)); + return; } } } @@ -89,5 +147,4 @@ std::string ProtoMessage::dump() const { } #endif -} // namespace api -} // namespace esphome +} // namespace esphome::api diff --git a/esphome/components/api/proto.h b/esphome/components/api/proto.h index a435168821..e7585924a5 100644 --- a/esphome/components/api/proto.h +++ b/esphome/components/api/proto.h @@ -3,16 +3,71 @@ #include "esphome/core/component.h" #include "esphome/core/helpers.h" #include "esphome/core/log.h" +#include "esphome/core/string_ref.h" #include +#include #include #ifdef ESPHOME_LOG_HAS_VERY_VERBOSE #define HAS_PROTO_MESSAGE_DUMP #endif -namespace esphome { -namespace api { +namespace esphome::api { + +// Protocol Buffer wire type constants +// See https://protobuf.dev/programming-guides/encoding/#structure +constexpr uint8_t WIRE_TYPE_VARINT = 0; // int32, int64, uint32, uint64, sint32, sint64, bool, enum +constexpr uint8_t WIRE_TYPE_LENGTH_DELIMITED = 2; // string, bytes, embedded messages, packed repeated fields +constexpr uint8_t WIRE_TYPE_FIXED32 = 5; // fixed32, sfixed32, float +constexpr uint8_t WIRE_TYPE_MASK = 0b111; // Mask to extract wire type from tag + +// Helper functions for ZigZag encoding/decoding +inline constexpr uint32_t encode_zigzag32(int32_t value) { + return (static_cast(value) << 1) ^ (static_cast(value >> 31)); +} + +inline constexpr uint64_t encode_zigzag64(int64_t value) { + return (static_cast(value) << 1) ^ (static_cast(value >> 63)); +} + +inline constexpr int32_t decode_zigzag32(uint32_t value) { + return (value & 1) ? static_cast(~(value >> 1)) : static_cast(value >> 1); +} + +inline constexpr int64_t decode_zigzag64(uint64_t value) { + return (value & 1) ? static_cast(~(value >> 1)) : static_cast(value >> 1); +} + +/* + * StringRef Ownership Model for API Protocol Messages + * =================================================== + * + * StringRef is used for zero-copy string handling in outgoing (SOURCE_SERVER) messages. + * It holds a pointer and length to existing string data without copying. + * + * CRITICAL: The referenced string data MUST remain valid until message encoding completes. + * + * Safe StringRef Patterns: + * 1. String literals: StringRef("literal") - Always safe (static storage duration) + * 2. Member variables: StringRef(this->member_string_) - Safe if object outlives encoding + * 3. Global/static strings: StringRef(GLOBAL_CONSTANT) - Always safe + * 4. Local variables: Safe ONLY if encoding happens before function returns: + * std::string temp = compute_value(); + * msg.set_field(StringRef(temp)); + * return this->send_message(msg); // temp is valid during encoding + * + * Unsafe Patterns (WILL cause crashes/corruption): + * 1. Temporaries: msg.set_field(StringRef(obj.get_string())) // get_string() returns by value + * 2. Concatenation: msg.set_field(StringRef(str1 + str2)) // Result is temporary + * + * For unsafe patterns, store in a local variable first: + * std::string temp = get_string(); // or str1 + str2 + * msg.set_field(StringRef(temp)); + * + * The send_*_response pattern ensures proper lifetime management by encoding + * within the same function scope where temporaries are created. + */ /// Representation of a VarInt - in ProtoBuf should be 64bit but we only use 32bit class ProtoVarInt { @@ -56,33 +111,25 @@ class ProtoVarInt { return {}; // Incomplete or invalid varint } - uint16_t as_uint16() const { return this->value_; } - uint32_t as_uint32() const { return this->value_; } - uint64_t as_uint64() const { return this->value_; } - bool as_bool() const { return this->value_; } - int32_t as_int32() const { + constexpr uint16_t as_uint16() const { return this->value_; } + constexpr uint32_t as_uint32() const { return this->value_; } + constexpr uint64_t as_uint64() const { return this->value_; } + constexpr bool as_bool() const { return this->value_; } + constexpr int32_t as_int32() const { // Not ZigZag encoded return static_cast(this->as_int64()); } - int64_t as_int64() const { + constexpr int64_t as_int64() const { // Not ZigZag encoded return static_cast(this->value_); } - int32_t as_sint32() const { + constexpr int32_t as_sint32() const { // with ZigZag encoding - if (this->value_ & 1) { - return static_cast(~(this->value_ >> 1)); - } else { - return static_cast(this->value_ >> 1); - } + return decode_zigzag32(static_cast(this->value_)); } - int64_t as_sint64() const { + constexpr int64_t as_sint64() const { // with ZigZag encoding - if (this->value_ & 1) { - return static_cast(~(this->value_ >> 1)); - } else { - return static_cast(this->value_ >> 1); - } + return decode_zigzag64(this->value_); } /** * Encode the varint value to a pre-allocated buffer without bounds checking. @@ -135,22 +182,27 @@ class ProtoVarInt { // Forward declaration for decode_to_message and encode_to_writer class ProtoMessage; +class ProtoDecodableMessage; class ProtoLengthDelimited { public: explicit ProtoLengthDelimited(const uint8_t *value, size_t length) : value_(value), length_(length) {} std::string as_string() const { return std::string(reinterpret_cast(this->value_), this->length_); } + // Direct access to raw data without string allocation + const uint8_t *data() const { return this->value_; } + size_t size() const { return this->length_; } + /** - * Decode the length-delimited data into an existing ProtoMessage instance. + * Decode the length-delimited data into an existing ProtoDecodableMessage instance. * * This method allows decoding without templates, enabling use in contexts - * where the message type is not known at compile time. The ProtoMessage's + * where the message type is not known at compile time. The ProtoDecodableMessage's * decode() method will be called with the raw data and length. * - * @param msg The ProtoMessage instance to decode into + * @param msg The ProtoDecodableMessage instance to decode into */ - void decode_to_message(ProtoMessage &msg) const; + void decode_to_message(ProtoDecodableMessage &msg) const; protected: const uint8_t *const value_; @@ -175,23 +227,7 @@ class Proto32Bit { const uint32_t value_; }; -class Proto64Bit { - public: - explicit Proto64Bit(uint64_t value) : value_(value) {} - uint64_t as_fixed64() const { return this->value_; } - int64_t as_sfixed64() const { return static_cast(this->value_); } - double as_double() const { - union { - uint64_t raw; - double value; - } s{}; - s.raw = this->value_; - return s.value; - } - - protected: - const uint64_t value_; -}; +// NOTE: Proto64Bit class removed - wire type 1 (64-bit fixed) not supported class ProtoWriteBuffer { public: @@ -205,14 +241,14 @@ class ProtoWriteBuffer { * @param field_id Field number (tag) in the protobuf message * @param type Wire type value: * - 0: Varint (int32, int64, uint32, uint64, sint32, sint64, bool, enum) - * - 1: 64-bit (fixed64, sfixed64, double) * - 2: Length-delimited (string, bytes, embedded messages, packed repeated fields) * - 5: 32-bit (fixed32, sfixed32, float) + * - Note: Wire type 1 (64-bit fixed) is not supported * * Following https://protobuf.dev/programming-guides/encoding/#structure */ void encode_field_raw(uint32_t field_id, uint32_t type) { - uint32_t val = (field_id << 3) | (type & 0b111); + uint32_t val = (field_id << 3) | (type & WIRE_TYPE_MASK); this->encode_varint_raw(val); } void encode_string(uint32_t field_id, const char *string, size_t len, bool force = false) { @@ -221,12 +257,20 @@ class ProtoWriteBuffer { this->encode_field_raw(field_id, 2); // type 2: Length-delimited string this->encode_varint_raw(len); - auto *data = reinterpret_cast(string); - this->buffer_->insert(this->buffer_->end(), data, data + len); + + // Using resize + memcpy instead of insert provides significant performance improvement: + // ~10-11x faster for 16-32 byte strings, ~3x faster for 64-byte strings + // as it avoids iterator checks and potential element moves that insert performs + size_t old_size = this->buffer_->size(); + this->buffer_->resize(old_size + len); + std::memcpy(this->buffer_->data() + old_size, string, len); } void encode_string(uint32_t field_id, const std::string &value, bool force = false) { this->encode_string(field_id, value.data(), value.size(), force); } + void encode_string(uint32_t field_id, const StringRef &ref, bool force = false) { + this->encode_string(field_id, ref.c_str(), ref.size(), force); + } void encode_bytes(uint32_t field_id, const uint8_t *data, size_t len, bool force = false) { this->encode_string(field_id, reinterpret_cast(data), len, force); } @@ -258,20 +302,10 @@ class ProtoWriteBuffer { this->write((value >> 16) & 0xFF); this->write((value >> 24) & 0xFF); } - void encode_fixed64(uint32_t field_id, uint64_t value, bool force = false) { - if (value == 0 && !force) - return; - - this->encode_field_raw(field_id, 1); // type 1: 64-bit fixed64 - this->write((value >> 0) & 0xFF); - this->write((value >> 8) & 0xFF); - this->write((value >> 16) & 0xFF); - this->write((value >> 24) & 0xFF); - this->write((value >> 32) & 0xFF); - this->write((value >> 40) & 0xFF); - this->write((value >> 48) & 0xFF); - this->write((value >> 56) & 0xFF); - } + // NOTE: Wire type 1 (64-bit fixed: double, fixed64, sfixed64) is intentionally + // not supported to reduce overhead on embedded systems. All ESPHome devices are + // 32-bit microcontrollers where 64-bit operations are expensive. If 64-bit support + // is needed in the future, the necessary encoding/decoding functions must be added. void encode_float(uint32_t field_id, float value, bool force = false) { if (value == 0.0f && !force) return; @@ -295,22 +329,10 @@ class ProtoWriteBuffer { this->encode_uint64(field_id, static_cast(value), force); } void encode_sint32(uint32_t field_id, int32_t value, bool force = false) { - uint32_t uvalue; - if (value < 0) { - uvalue = ~(value << 1); - } else { - uvalue = value << 1; - } - this->encode_uint32(field_id, uvalue, force); + this->encode_uint32(field_id, encode_zigzag32(value), force); } void encode_sint64(uint32_t field_id, int64_t value, bool force = false) { - uint64_t uvalue; - if (value < 0) { - uvalue = ~(value << 1); - } else { - uvalue = value << 1; - } - this->encode_uint64(field_id, uvalue, force); + this->encode_uint64(field_id, encode_zigzag64(value), force); } void encode_message(uint32_t field_id, const ProtoMessage &value, bool force = false); std::vector *get_buffer() const { return buffer_; } @@ -319,53 +341,80 @@ class ProtoWriteBuffer { std::vector *buffer_; }; +// Forward declaration +class ProtoSize; + class ProtoMessage { public: virtual ~ProtoMessage() = default; // Default implementation for messages with no fields virtual void encode(ProtoWriteBuffer buffer) const {} - void decode(const uint8_t *buffer, size_t length); // Default implementation for messages with no fields - virtual void calculate_size(uint32_t &total_size) const {} + virtual void calculate_size(ProtoSize &size) const {} #ifdef HAS_PROTO_MESSAGE_DUMP std::string dump() const; virtual void dump_to(std::string &out) const = 0; virtual const char *message_name() const { return "unknown"; } #endif +}; + +// Base class for messages that support decoding +class ProtoDecodableMessage : public ProtoMessage { + public: + virtual void decode(const uint8_t *buffer, size_t length); + + /** + * Count occurrences of a repeated field in a protobuf buffer. + * This is a lightweight scan that only parses tags and skips field data. + * + * @param buffer Pointer to the protobuf buffer + * @param length Length of the buffer in bytes + * @param target_field_id The field ID to count + * @return Number of times the field appears in the buffer + */ + static uint32_t count_repeated_field(const uint8_t *buffer, size_t length, uint32_t target_field_id); protected: virtual bool decode_varint(uint32_t field_id, ProtoVarInt value) { return false; } virtual bool decode_length(uint32_t field_id, ProtoLengthDelimited value) { return false; } virtual bool decode_32bit(uint32_t field_id, Proto32Bit value) { return false; } - virtual bool decode_64bit(uint32_t field_id, Proto64Bit value) { return false; } + // NOTE: decode_64bit removed - wire type 1 not supported }; class ProtoSize { + private: + uint32_t total_size_ = 0; + public: /** * @brief ProtoSize class for Protocol Buffer serialization size calculation * - * This class provides static methods to calculate the exact byte counts needed - * for encoding various Protocol Buffer field types. All methods are designed to be - * efficient for the common case where many fields have default values. + * This class provides methods to calculate the exact byte counts needed + * for encoding various Protocol Buffer field types. The class now uses an + * object-based approach to reduce parameter passing overhead while keeping + * varint calculation methods static for external use. * * Implements Protocol Buffer encoding size calculation according to: * https://protobuf.dev/programming-guides/encoding/ * * Key features: + * - Object-based approach reduces flash usage by eliminating parameter passing * - Early-return optimization for zero/default values - * - Direct total_size updates to avoid unnecessary additions + * - Static varint methods for external callers * - Specialized handling for different field types according to protobuf spec - * - Templated helpers for repeated fields and messages */ + ProtoSize() = default; + + uint32_t get_size() const { return total_size_; } + /** * @brief Calculates the size in bytes needed to encode a uint32_t value as a varint * * @param value The uint32_t value to calculate size for * @return The number of bytes needed to encode the value */ - static inline uint32_t varint(uint32_t value) { + static constexpr uint32_t varint(uint32_t value) { // Optimized varint size calculation using leading zeros // Each 7 bits requires one byte in the varint encoding if (value < 128) @@ -389,7 +438,7 @@ class ProtoSize { * @param value The uint64_t value to calculate size for * @return The number of bytes needed to encode the value */ - static inline uint32_t varint(uint64_t value) { + static constexpr uint32_t varint(uint64_t value) { // Handle common case of values fitting in uint32_t (vast majority of use cases) if (value <= UINT32_MAX) { return varint(static_cast(value)); @@ -420,7 +469,7 @@ class ProtoSize { * @param value The int32_t value to calculate size for * @return The number of bytes needed to encode the value */ - static inline uint32_t varint(int32_t value) { + static constexpr uint32_t varint(int32_t value) { // Negative values are sign-extended to 64 bits in protocol buffers, // which always results in a 10-byte varint for negative int32 if (value < 0) { @@ -436,7 +485,7 @@ class ProtoSize { * @param value The int64_t value to calculate size for * @return The number of bytes needed to encode the value */ - static inline uint32_t varint(int64_t value) { + static constexpr uint32_t varint(int64_t value) { // For int64_t, we convert to uint64_t and calculate the size // This works because the bit pattern determines the encoding size, // and we've handled negative int32 values as a special case above @@ -450,8 +499,8 @@ class ProtoSize { * @param type The wire type value (from the WireType enum in the protobuf spec) * @return The number of bytes needed to encode the field ID and wire type */ - static inline uint32_t field(uint32_t field_id, uint32_t type) { - uint32_t tag = (field_id << 3) | (type & 0b111); + static constexpr uint32_t field(uint32_t field_id, uint32_t type) { + uint32_t tag = (field_id << 3) | (type & WIRE_TYPE_MASK); return varint(tag); } @@ -459,9 +508,7 @@ class ProtoSize { * @brief Common parameters for all add_*_field methods * * All add_*_field methods follow these common patterns: - * - * @param total_size Reference to the total message size to update - * @param field_id_size Pre-calculated size of the field ID in bytes + * * @param field_id_size Pre-calculated size of the field ID in bytes * @param value The value to calculate size for (type varies) * @param force Whether to calculate size even if the value is default/zero/empty * @@ -474,244 +521,180 @@ class ProtoSize { /** * @brief Calculates and adds the size of an int32 field to the total message size */ - static inline void add_int32_field(uint32_t &total_size, uint32_t field_id_size, int32_t value) { - // Skip calculation if value is zero - if (value == 0) { - return; // No need to update total_size - } - - // Calculate and directly add to total_size - if (value < 0) { - // Negative values are encoded as 10-byte varints in protobuf - total_size += field_id_size + 10; - } else { - // For non-negative values, use the standard varint size - total_size += field_id_size + varint(static_cast(value)); + inline void add_int32(uint32_t field_id_size, int32_t value) { + if (value != 0) { + add_int32_force(field_id_size, value); } } /** - * @brief Calculates and adds the size of an int32 field to the total message size (repeated field version) + * @brief Calculates and adds the size of an int32 field to the total message size (force version) */ - static inline void add_int32_field_repeated(uint32_t &total_size, uint32_t field_id_size, int32_t value) { - // Always calculate size for repeated fields - if (value < 0) { - // Negative values are encoded as 10-byte varints in protobuf - total_size += field_id_size + 10; - } else { - // For non-negative values, use the standard varint size - total_size += field_id_size + varint(static_cast(value)); - } + inline void add_int32_force(uint32_t field_id_size, int32_t value) { + // Always calculate size when forced + // Negative values are encoded as 10-byte varints in protobuf + total_size_ += field_id_size + (value < 0 ? 10 : varint(static_cast(value))); } /** * @brief Calculates and adds the size of a uint32 field to the total message size */ - static inline void add_uint32_field(uint32_t &total_size, uint32_t field_id_size, uint32_t value) { - // Skip calculation if value is zero - if (value == 0) { - return; // No need to update total_size + inline void add_uint32(uint32_t field_id_size, uint32_t value) { + if (value != 0) { + add_uint32_force(field_id_size, value); } - - // Calculate and directly add to total_size - total_size += field_id_size + varint(value); } /** - * @brief Calculates and adds the size of a uint32 field to the total message size (repeated field version) + * @brief Calculates and adds the size of a uint32 field to the total message size (force version) */ - static inline void add_uint32_field_repeated(uint32_t &total_size, uint32_t field_id_size, uint32_t value) { - // Always calculate size for repeated fields - total_size += field_id_size + varint(value); + inline void add_uint32_force(uint32_t field_id_size, uint32_t value) { + // Always calculate size when force is true + total_size_ += field_id_size + varint(value); } /** * @brief Calculates and adds the size of a boolean field to the total message size */ - static inline void add_bool_field(uint32_t &total_size, uint32_t field_id_size, bool value) { - // Skip calculation if value is false - if (!value) { - return; // No need to update total_size + inline void add_bool(uint32_t field_id_size, bool value) { + if (value) { + // Boolean fields always use 1 byte when true + total_size_ += field_id_size + 1; } - - // Boolean fields always use 1 byte when true - total_size += field_id_size + 1; } /** - * @brief Calculates and adds the size of a boolean field to the total message size (repeated field version) + * @brief Calculates and adds the size of a boolean field to the total message size (force version) */ - static inline void add_bool_field_repeated(uint32_t &total_size, uint32_t field_id_size, bool value) { - // Always calculate size for repeated fields + inline void add_bool_force(uint32_t field_id_size, bool value) { + // Always calculate size when force is true // Boolean fields always use 1 byte - total_size += field_id_size + 1; + total_size_ += field_id_size + 1; } /** - * @brief Calculates and adds the size of a fixed field to the total message size - * - * Fixed fields always take exactly N bytes (4 for fixed32/float, 8 for fixed64/double). - * - * @tparam NumBytes The number of bytes for this fixed field (4 or 8) - * @param is_nonzero Whether the value is non-zero + * @brief Calculates and adds the size of a float field to the total message size */ - template - static inline void add_fixed_field(uint32_t &total_size, uint32_t field_id_size, bool is_nonzero) { - // Skip calculation if value is zero - if (!is_nonzero) { - return; // No need to update total_size + inline void add_float(uint32_t field_id_size, float value) { + if (value != 0.0f) { + total_size_ += field_id_size + 4; } - - // Fixed fields always take exactly NumBytes - total_size += field_id_size + NumBytes; } + // NOTE: add_double_field removed - wire type 1 (64-bit: double) not supported + // to reduce overhead on embedded systems + /** - * @brief Calculates and adds the size of an enum field to the total message size - * - * Enum fields are encoded as uint32 varints. + * @brief Calculates and adds the size of a fixed32 field to the total message size */ - static inline void add_enum_field(uint32_t &total_size, uint32_t field_id_size, uint32_t value) { - // Skip calculation if value is zero - if (value == 0) { - return; // No need to update total_size + inline void add_fixed32(uint32_t field_id_size, uint32_t value) { + if (value != 0) { + total_size_ += field_id_size + 4; } - - // Enums are encoded as uint32 - total_size += field_id_size + varint(value); } + // NOTE: add_fixed64_field removed - wire type 1 (64-bit: fixed64) not supported + // to reduce overhead on embedded systems + /** - * @brief Calculates and adds the size of an enum field to the total message size (repeated field version) - * - * Enum fields are encoded as uint32 varints. + * @brief Calculates and adds the size of a sfixed32 field to the total message size */ - static inline void add_enum_field_repeated(uint32_t &total_size, uint32_t field_id_size, uint32_t value) { - // Always calculate size for repeated fields - // Enums are encoded as uint32 - total_size += field_id_size + varint(value); + inline void add_sfixed32(uint32_t field_id_size, int32_t value) { + if (value != 0) { + total_size_ += field_id_size + 4; + } } + // NOTE: add_sfixed64_field removed - wire type 1 (64-bit: sfixed64) not supported + // to reduce overhead on embedded systems + /** * @brief Calculates and adds the size of a sint32 field to the total message size * * Sint32 fields use ZigZag encoding, which is more efficient for negative values. */ - static inline void add_sint32_field(uint32_t &total_size, uint32_t field_id_size, int32_t value) { - // Skip calculation if value is zero - if (value == 0) { - return; // No need to update total_size + inline void add_sint32(uint32_t field_id_size, int32_t value) { + if (value != 0) { + add_sint32_force(field_id_size, value); } - - // ZigZag encoding for sint32: (n << 1) ^ (n >> 31) - uint32_t zigzag = (static_cast(value) << 1) ^ (static_cast(value >> 31)); - total_size += field_id_size + varint(zigzag); } /** - * @brief Calculates and adds the size of a sint32 field to the total message size (repeated field version) + * @brief Calculates and adds the size of a sint32 field to the total message size (force version) * * Sint32 fields use ZigZag encoding, which is more efficient for negative values. */ - static inline void add_sint32_field_repeated(uint32_t &total_size, uint32_t field_id_size, int32_t value) { - // Always calculate size for repeated fields - // ZigZag encoding for sint32: (n << 1) ^ (n >> 31) - uint32_t zigzag = (static_cast(value) << 1) ^ (static_cast(value >> 31)); - total_size += field_id_size + varint(zigzag); + inline void add_sint32_force(uint32_t field_id_size, int32_t value) { + // Always calculate size when force is true + // ZigZag encoding for sint32 + total_size_ += field_id_size + varint(encode_zigzag32(value)); } /** * @brief Calculates and adds the size of an int64 field to the total message size */ - static inline void add_int64_field(uint32_t &total_size, uint32_t field_id_size, int64_t value) { - // Skip calculation if value is zero - if (value == 0) { - return; // No need to update total_size + inline void add_int64(uint32_t field_id_size, int64_t value) { + if (value != 0) { + add_int64_force(field_id_size, value); } - - // Calculate and directly add to total_size - total_size += field_id_size + varint(value); } /** - * @brief Calculates and adds the size of an int64 field to the total message size (repeated field version) + * @brief Calculates and adds the size of an int64 field to the total message size (force version) */ - static inline void add_int64_field_repeated(uint32_t &total_size, uint32_t field_id_size, int64_t value) { - // Always calculate size for repeated fields - total_size += field_id_size + varint(value); + inline void add_int64_force(uint32_t field_id_size, int64_t value) { + // Always calculate size when force is true + total_size_ += field_id_size + varint(value); } /** * @brief Calculates and adds the size of a uint64 field to the total message size */ - static inline void add_uint64_field(uint32_t &total_size, uint32_t field_id_size, uint64_t value) { - // Skip calculation if value is zero - if (value == 0) { - return; // No need to update total_size + inline void add_uint64(uint32_t field_id_size, uint64_t value) { + if (value != 0) { + add_uint64_force(field_id_size, value); } - - // Calculate and directly add to total_size - total_size += field_id_size + varint(value); } /** - * @brief Calculates and adds the size of a uint64 field to the total message size (repeated field version) + * @brief Calculates and adds the size of a uint64 field to the total message size (force version) */ - static inline void add_uint64_field_repeated(uint32_t &total_size, uint32_t field_id_size, uint64_t value) { - // Always calculate size for repeated fields - total_size += field_id_size + varint(value); + inline void add_uint64_force(uint32_t field_id_size, uint64_t value) { + // Always calculate size when force is true + total_size_ += field_id_size + varint(value); + } + + // NOTE: sint64 support functions (add_sint64_field, add_sint64_field_force) removed + // sint64 type is not supported by ESPHome API to reduce overhead on embedded systems + + /** + * @brief Calculates and adds the size of a length-delimited field (string/bytes) to the total message size + */ + inline void add_length(uint32_t field_id_size, size_t len) { + if (len != 0) { + add_length_force(field_id_size, len); + } } /** - * @brief Calculates and adds the size of a sint64 field to the total message size + * @brief Calculates and adds the size of a length-delimited field (string/bytes) to the total message size (repeated + * field version) + */ + inline void add_length_force(uint32_t field_id_size, size_t len) { + // Always calculate size when force is true + // Field ID + length varint + data bytes + total_size_ += field_id_size + varint(static_cast(len)) + static_cast(len); + } + + /** + * @brief Adds a pre-calculated size directly to the total * - * Sint64 fields use ZigZag encoding, which is more efficient for negative values. - */ - static inline void add_sint64_field(uint32_t &total_size, uint32_t field_id_size, int64_t value) { - // Skip calculation if value is zero - if (value == 0) { - return; // No need to update total_size - } - - // ZigZag encoding for sint64: (n << 1) ^ (n >> 63) - uint64_t zigzag = (static_cast(value) << 1) ^ (static_cast(value >> 63)); - total_size += field_id_size + varint(zigzag); - } - - /** - * @brief Calculates and adds the size of a sint64 field to the total message size (repeated field version) + * This is used when we can calculate the total size by multiplying the number + * of elements by the bytes per element (for repeated fixed-size types like float, fixed32, etc.) * - * Sint64 fields use ZigZag encoding, which is more efficient for negative values. + * @param size The pre-calculated total size to add */ - static inline void add_sint64_field_repeated(uint32_t &total_size, uint32_t field_id_size, int64_t value) { - // Always calculate size for repeated fields - // ZigZag encoding for sint64: (n << 1) ^ (n >> 63) - uint64_t zigzag = (static_cast(value) << 1) ^ (static_cast(value >> 63)); - total_size += field_id_size + varint(zigzag); - } - - /** - * @brief Calculates and adds the size of a string/bytes field to the total message size - */ - static inline void add_string_field(uint32_t &total_size, uint32_t field_id_size, const std::string &str) { - // Skip calculation if string is empty - if (str.empty()) { - return; // No need to update total_size - } - - // Calculate and directly add to total_size - const uint32_t str_size = static_cast(str.size()); - total_size += field_id_size + varint(str_size) + str_size; - } - - /** - * @brief Calculates and adds the size of a string/bytes field to the total message size (repeated field version) - */ - static inline void add_string_field_repeated(uint32_t &total_size, uint32_t field_id_size, const std::string &str) { - // Always calculate size for repeated fields - const uint32_t str_size = static_cast(str.size()); - total_size += field_id_size + varint(str_size) + str_size; - } + inline void add_precalculated_size(uint32_t size) { total_size_ += size; } /** * @brief Calculates and adds the size of a nested message field to the total message size @@ -721,26 +704,21 @@ class ProtoSize { * * @param nested_size The pre-calculated size of the nested message */ - static inline void add_message_field(uint32_t &total_size, uint32_t field_id_size, uint32_t nested_size) { - // Skip calculation if nested message is empty - if (nested_size == 0) { - return; // No need to update total_size + inline void add_message_field(uint32_t field_id_size, uint32_t nested_size) { + if (nested_size != 0) { + add_message_field_force(field_id_size, nested_size); } - - // Calculate and directly add to total_size - // Field ID + length varint + nested message content - total_size += field_id_size + varint(nested_size) + nested_size; } /** - * @brief Calculates and adds the size of a nested message field to the total message size (repeated field version) + * @brief Calculates and adds the size of a nested message field to the total message size (force version) * * @param nested_size The pre-calculated size of the nested message */ - static inline void add_message_field_repeated(uint32_t &total_size, uint32_t field_id_size, uint32_t nested_size) { - // Always calculate size for repeated fields + inline void add_message_field_force(uint32_t field_id_size, uint32_t nested_size) { + // Always calculate size when force is true // Field ID + length varint + nested message content - total_size += field_id_size + varint(nested_size) + nested_size; + total_size_ += field_id_size + varint(nested_size) + nested_size; } /** @@ -752,26 +730,29 @@ class ProtoSize { * * @param message The nested message object */ - static inline void add_message_object(uint32_t &total_size, uint32_t field_id_size, const ProtoMessage &message) { - uint32_t nested_size = 0; - message.calculate_size(nested_size); + inline void add_message_object(uint32_t field_id_size, const ProtoMessage &message) { + // Calculate nested message size by creating a temporary ProtoSize + ProtoSize nested_calc; + message.calculate_size(nested_calc); + uint32_t nested_size = nested_calc.get_size(); // Use the base implementation with the calculated nested_size - add_message_field(total_size, field_id_size, nested_size); + add_message_field(field_id_size, nested_size); } /** - * @brief Calculates and adds the size of a nested message field to the total message size (repeated field version) + * @brief Calculates and adds the size of a nested message field to the total message size (force version) * * @param message The nested message object */ - static inline void add_message_object_repeated(uint32_t &total_size, uint32_t field_id_size, - const ProtoMessage &message) { - uint32_t nested_size = 0; - message.calculate_size(nested_size); + inline void add_message_object_force(uint32_t field_id_size, const ProtoMessage &message) { + // Calculate nested message size by creating a temporary ProtoSize + ProtoSize nested_calc; + message.calculate_size(nested_calc); + uint32_t nested_size = nested_calc.get_size(); // Use the base implementation with the calculated nested_size - add_message_field_repeated(total_size, field_id_size, nested_size); + add_message_field_force(field_id_size, nested_size); } /** @@ -784,16 +765,31 @@ class ProtoSize { * @param messages Vector of message objects */ template - static inline void add_repeated_message(uint32_t &total_size, uint32_t field_id_size, - const std::vector &messages) { + inline void add_repeated_message(uint32_t field_id_size, const std::vector &messages) { // Skip if the vector is empty - if (messages.empty()) { - return; + if (!messages.empty()) { + // Use the force version for all messages in the repeated field + for (const auto &message : messages) { + add_message_object_force(field_id_size, message); + } } + } - // Use the repeated field version for all messages - for (const auto &message : messages) { - add_message_object_repeated(total_size, field_id_size, message); + /** + * @brief Calculates and adds the sizes of all messages in a repeated field to the total message size (FixedVector + * version) + * + * @tparam MessageType The type of the nested messages in the FixedVector + * @param messages FixedVector of message objects + */ + template + inline void add_repeated_message(uint32_t field_id_size, const FixedVector &messages) { + // Skip if the fixed vector is empty + if (!messages.empty()) { + // Use the force version for all messages in the repeated field + for (const auto &message : messages) { + add_message_object_force(field_id_size, message); + } } } }; @@ -803,8 +799,9 @@ inline void ProtoWriteBuffer::encode_message(uint32_t field_id, const ProtoMessa this->encode_field_raw(field_id, 2); // type 2: Length-delimited message // Calculate the message size first - uint32_t msg_length_bytes = 0; - value.calculate_size(msg_length_bytes); + ProtoSize msg_size; + value.calculate_size(msg_size); + uint32_t msg_length_bytes = msg_size.get_size(); // Calculate how many bytes the length varint needs uint32_t varint_length_bytes = ProtoSize::varint(msg_length_bytes); @@ -823,8 +820,8 @@ inline void ProtoWriteBuffer::encode_message(uint32_t field_id, const ProtoMessa assert(this->buffer_->size() == begin + varint_length_bytes + msg_length_bytes); } -// Implementation of decode_to_message - must be after ProtoMessage is defined -inline void ProtoLengthDelimited::decode_to_message(ProtoMessage &msg) const { +// Implementation of decode_to_message - must be after ProtoDecodableMessage is defined +inline void ProtoLengthDelimited::decode_to_message(ProtoDecodableMessage &msg) const { msg.decode(this->value_, this->length_); } @@ -836,7 +833,9 @@ class ProtoService { virtual bool is_authenticated() = 0; virtual bool is_connection_setup() = 0; virtual void on_fatal_error() = 0; +#ifdef USE_API_PASSWORD virtual void on_unauthenticated_access() = 0; +#endif virtual void on_no_setup_connection() = 0; /** * Create a buffer with a reserved size. @@ -851,8 +850,9 @@ class ProtoService { // Optimized method that pre-allocates buffer based on message size bool send_message_(const ProtoMessage &msg, uint8_t message_type) { - uint32_t msg_size = 0; - msg.calculate_size(msg_size); + ProtoSize size; + msg.calculate_size(size); + uint32_t msg_size = size.get_size(); // Create a pre-sized buffer auto buffer = this->create_buffer(msg_size); @@ -865,7 +865,7 @@ class ProtoService { } // Authentication helper methods - bool check_connection_setup_() { + inline bool check_connection_setup_() { if (!this->is_connection_setup()) { this->on_no_setup_connection(); return false; @@ -873,7 +873,8 @@ class ProtoService { return true; } - bool check_authenticated_() { + inline bool check_authenticated_() { +#ifdef USE_API_PASSWORD if (!this->check_connection_setup_()) { return false; } @@ -882,8 +883,10 @@ class ProtoService { return false; } return true; +#else + return this->check_connection_setup_(); +#endif } }; -} // namespace api -} // namespace esphome +} // namespace esphome::api diff --git a/esphome/components/api/subscribe_state.cpp b/esphome/components/api/subscribe_state.cpp index 12accf4613..3a563f2221 100644 --- a/esphome/components/api/subscribe_state.cpp +++ b/esphome/components/api/subscribe_state.cpp @@ -3,8 +3,7 @@ #include "api_connection.h" #include "esphome/core/log.h" -namespace esphome { -namespace api { +namespace esphome::api { // Generate entity handler implementations using macros #ifdef USE_BINARY_SENSOR @@ -69,6 +68,5 @@ INITIAL_STATE_HANDLER(update, update::UpdateEntity) InitialStateIterator::InitialStateIterator(APIConnection *client) : client_(client) {} -} // namespace api -} // namespace esphome +} // namespace esphome::api #endif diff --git a/esphome/components/api/subscribe_state.h b/esphome/components/api/subscribe_state.h index 2b7b508056..2c22c322ec 100644 --- a/esphome/components/api/subscribe_state.h +++ b/esphome/components/api/subscribe_state.h @@ -5,8 +5,7 @@ #include "esphome/core/component.h" #include "esphome/core/component_iterator.h" #include "esphome/core/controller.h" -namespace esphome { -namespace api { +namespace esphome::api { class APIConnection; @@ -89,6 +88,5 @@ class InitialStateIterator : public ComponentIterator { APIConnection *client_; }; -} // namespace api -} // namespace esphome +} // namespace esphome::api #endif diff --git a/esphome/components/api/user_services.cpp b/esphome/components/api/user_services.cpp index 7e73722a92..3cbf2ab5f9 100644 --- a/esphome/components/api/user_services.cpp +++ b/esphome/components/api/user_services.cpp @@ -1,8 +1,7 @@ #include "user_services.h" #include "esphome/core/log.h" -namespace esphome { -namespace api { +namespace esphome::api { template<> bool get_execute_arg_value(const ExecuteServiceArgument &arg) { return arg.bool_; } template<> int32_t get_execute_arg_value(const ExecuteServiceArgument &arg) { @@ -13,16 +12,16 @@ template<> int32_t get_execute_arg_value(const ExecuteServiceArgument & template<> float get_execute_arg_value(const ExecuteServiceArgument &arg) { return arg.float_; } template<> std::string get_execute_arg_value(const ExecuteServiceArgument &arg) { return arg.string_; } template<> std::vector get_execute_arg_value>(const ExecuteServiceArgument &arg) { - return arg.bool_array; + return std::vector(arg.bool_array.begin(), arg.bool_array.end()); } template<> std::vector get_execute_arg_value>(const ExecuteServiceArgument &arg) { - return arg.int_array; + return std::vector(arg.int_array.begin(), arg.int_array.end()); } template<> std::vector get_execute_arg_value>(const ExecuteServiceArgument &arg) { - return arg.float_array; + return std::vector(arg.float_array.begin(), arg.float_array.end()); } template<> std::vector get_execute_arg_value>(const ExecuteServiceArgument &arg) { - return arg.string_array; + return std::vector(arg.string_array.begin(), arg.string_array.end()); } template<> enums::ServiceArgType to_service_arg_type() { return enums::SERVICE_ARG_TYPE_BOOL; } @@ -40,5 +39,4 @@ template<> enums::ServiceArgType to_service_arg_type>() return enums::SERVICE_ARG_TYPE_STRING_ARRAY; } -} // namespace api -} // namespace esphome +} // namespace esphome::api diff --git a/esphome/components/api/user_services.h b/esphome/components/api/user_services.h index 93cea8133f..9ca5e1093e 100644 --- a/esphome/components/api/user_services.h +++ b/esphome/components/api/user_services.h @@ -8,14 +8,15 @@ #include "api_pb2.h" #ifdef USE_API_SERVICES -namespace esphome { -namespace api { +namespace esphome::api { class UserServiceDescriptor { public: virtual ListEntitiesServicesResponse encode_list_service_response() = 0; virtual bool execute_service(const ExecuteServiceRequest &req) = 0; + + bool is_internal() { return false; } }; template T get_execute_arg_value(const ExecuteServiceArgument &arg); @@ -31,14 +32,14 @@ template class UserServiceBase : public UserServiceDescriptor { ListEntitiesServicesResponse encode_list_service_response() override { ListEntitiesServicesResponse msg; - msg.name = this->name_; + msg.set_name(StringRef(this->name_)); msg.key = this->key_; std::array arg_types = {to_service_arg_type()...}; - for (int i = 0; i < sizeof...(Ts); i++) { - ListEntitiesServicesArgument arg; + msg.args.init(sizeof...(Ts)); + for (size_t i = 0; i < sizeof...(Ts); i++) { + auto &arg = msg.args.emplace_back(); arg.type = arg_types[i]; - arg.name = this->arg_names_[i]; - msg.args.push_back(arg); + arg.set_name(StringRef(this->arg_names_[i])); } return msg; } @@ -54,7 +55,7 @@ template class UserServiceBase : public UserServiceDescriptor { protected: virtual void execute(Ts... x) = 0; - template void execute_(std::vector args, seq type) { + template void execute_(const ArgsContainer &args, seq type) { this->execute((get_execute_arg_value(args[S]))...); } @@ -72,6 +73,5 @@ template class UserServiceTrigger : public UserServiceBasetrigger(x...); } // NOLINT }; -} // namespace api -} // namespace esphome +} // namespace esphome::api #endif // USE_API_SERVICES diff --git a/esphome/components/as3935/as3935.cpp b/esphome/components/as3935/as3935.cpp index 5e6d62b284..2609af07d3 100644 --- a/esphome/components/as3935/as3935.cpp +++ b/esphome/components/as3935/as3935.cpp @@ -7,8 +7,6 @@ namespace as3935 { static const char *const TAG = "as3935"; void AS3935Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->irq_pin_->setup(); LOG_PIN(" IRQ Pin: ", this->irq_pin_); diff --git a/esphome/components/as3935_spi/as3935_spi.cpp b/esphome/components/as3935_spi/as3935_spi.cpp index 3a517df56d..1b2e9ccd3f 100644 --- a/esphome/components/as3935_spi/as3935_spi.cpp +++ b/esphome/components/as3935_spi/as3935_spi.cpp @@ -7,9 +7,7 @@ namespace as3935_spi { static const char *const TAG = "as3935_spi"; void SPIAS3935Component::setup() { - ESP_LOGI(TAG, "SPIAS3935Component setup started!"); this->spi_setup(); - ESP_LOGI(TAG, "SPI setup finished!"); AS3935Component::setup(); } diff --git a/esphome/components/as3935_spi/as3935_spi.h b/esphome/components/as3935_spi/as3935_spi.h index 073f5c09a4..e5422f9b37 100644 --- a/esphome/components/as3935_spi/as3935_spi.h +++ b/esphome/components/as3935_spi/as3935_spi.h @@ -3,8 +3,6 @@ #include "esphome/core/component.h" #include "esphome/components/as3935/as3935.h" #include "esphome/components/spi/spi.h" -#include "esphome/components/sensor/sensor.h" -#include "esphome/components/binary_sensor/binary_sensor.h" namespace esphome { namespace as3935_spi { diff --git a/esphome/components/as5600/__init__.py b/esphome/components/as5600/__init__.py index 1a437a68a2..acb1c4d9db 100644 --- a/esphome/components/as5600/__init__.py +++ b/esphome/components/as5600/__init__.py @@ -7,6 +7,7 @@ from esphome.const import ( CONF_DIRECTION, CONF_HYSTERESIS, CONF_ID, + CONF_POWER_MODE, CONF_RANGE, ) @@ -57,7 +58,6 @@ FAST_FILTER = { CONF_RAW_ANGLE = "raw_angle" CONF_RAW_POSITION = "raw_position" CONF_WATCHDOG = "watchdog" -CONF_POWER_MODE = "power_mode" CONF_SLOW_FILTER = "slow_filter" CONF_FAST_FILTER = "fast_filter" CONF_START_POSITION = "start_position" diff --git a/esphome/components/as5600/as5600.cpp b/esphome/components/as5600/as5600.cpp index ff29ae5cd4..ee3083d561 100644 --- a/esphome/components/as5600/as5600.cpp +++ b/esphome/components/as5600/as5600.cpp @@ -23,8 +23,6 @@ static const uint8_t REGISTER_AGC = 0x1A; // 8 bytes / R static const uint8_t REGISTER_MAGNITUDE = 0x1B; // 16 bytes / R void AS5600Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - if (!this->read_byte(REGISTER_STATUS).has_value()) { this->mark_failed(); return; diff --git a/esphome/components/as5600/sensor/__init__.py b/esphome/components/as5600/sensor/__init__.py index cfc38d796d..1491852e07 100644 --- a/esphome/components/as5600/sensor/__init__.py +++ b/esphome/components/as5600/sensor/__init__.py @@ -24,7 +24,6 @@ AS5600Sensor = as5600_ns.class_("AS5600Sensor", sensor.Sensor, cg.PollingCompone CONF_RAW_ANGLE = "raw_angle" CONF_RAW_POSITION = "raw_position" CONF_WATCHDOG = "watchdog" -CONF_POWER_MODE = "power_mode" CONF_SLOW_FILTER = "slow_filter" CONF_FAST_FILTER = "fast_filter" CONF_PWM_FREQUENCY = "pwm_frequency" diff --git a/esphome/components/as7341/as7341.cpp b/esphome/components/as7341/as7341.cpp index 1e335f43ad..893eaa850f 100644 --- a/esphome/components/as7341/as7341.cpp +++ b/esphome/components/as7341/as7341.cpp @@ -8,7 +8,6 @@ namespace as7341 { static const char *const TAG = "as7341"; void AS7341Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); LOG_I2C_DEVICE(this); // Verify device ID diff --git a/esphome/components/as7341/sensor.py b/esphome/components/as7341/sensor.py index 2832b7c3df..fa51a1cdfa 100644 --- a/esphome/components/as7341/sensor.py +++ b/esphome/components/as7341/sensor.py @@ -2,6 +2,7 @@ import esphome.codegen as cg from esphome.components import i2c, sensor import esphome.config_validation as cv from esphome.const import ( + CONF_CLEAR, CONF_GAIN, CONF_ID, DEVICE_CLASS_ILLUMINANCE, @@ -29,7 +30,6 @@ CONF_F5 = "f5" CONF_F6 = "f6" CONF_F7 = "f7" CONF_F8 = "f8" -CONF_CLEAR = "clear" CONF_NIR = "nir" UNIT_COUNTS = "#" diff --git a/esphome/components/async_tcp/__init__.py b/esphome/components/async_tcp/__init__.py index 4a469fa0e0..f2d8895b39 100644 --- a/esphome/components/async_tcp/__init__.py +++ b/esphome/components/async_tcp/__init__.py @@ -8,9 +8,9 @@ from esphome.const import ( PLATFORM_LN882X, PLATFORM_RTL87XX, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority -CODEOWNERS = ["@OttoWinter"] +CODEOWNERS = ["@esphome/core"] CONFIG_SCHEMA = cv.All( cv.Schema({}), @@ -27,7 +27,7 @@ CONFIG_SCHEMA = cv.All( ) -@coroutine_with_priority(200.0) +@coroutine_with_priority(CoroPriority.NETWORK_TRANSPORT) async def to_code(config): if CORE.is_esp32 or CORE.is_libretiny: # https://github.com/ESP32Async/AsyncTCP diff --git a/esphome/components/at581x/at581x.cpp b/esphome/components/at581x/at581x.cpp index b4f817b737..6804a7f4b5 100644 --- a/esphome/components/at581x/at581x.cpp +++ b/esphome/components/at581x/at581x.cpp @@ -71,7 +71,7 @@ bool AT581XComponent::i2c_read_reg(uint8_t addr, uint8_t &data) { return this->read_register(addr, &data, 1) == esphome::i2c::NO_ERROR; } -void AT581XComponent::setup() { ESP_LOGCONFIG(TAG, "Running setup"); } +void AT581XComponent::setup() {} void AT581XComponent::dump_config() { LOG_I2C_DEVICE(this); } #define ARRAY_SIZE(X) (sizeof(X) / sizeof((X)[0])) bool AT581XComponent::i2c_write_config() { diff --git a/esphome/components/atm90e26/atm90e26.cpp b/esphome/components/atm90e26/atm90e26.cpp index ce254f9532..cadc06ac6b 100644 --- a/esphome/components/atm90e26/atm90e26.cpp +++ b/esphome/components/atm90e26/atm90e26.cpp @@ -41,7 +41,6 @@ void ATM90E26Component::update() { } void ATM90E26Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); uint16_t mmode = 0x422; // default values for everything but L/N line current gains diff --git a/esphome/components/atm90e26/sensor.py b/esphome/components/atm90e26/sensor.py index 42ef259100..4522e94846 100644 --- a/esphome/components/atm90e26/sensor.py +++ b/esphome/components/atm90e26/sensor.py @@ -16,6 +16,7 @@ from esphome.const import ( DEVICE_CLASS_ENERGY, DEVICE_CLASS_POWER, DEVICE_CLASS_POWER_FACTOR, + DEVICE_CLASS_REACTIVE_POWER, DEVICE_CLASS_VOLTAGE, ICON_CURRENT_AC, ICON_LIGHTBULB, @@ -78,6 +79,7 @@ CONFIG_SCHEMA = ( unit_of_measurement=UNIT_VOLT_AMPS_REACTIVE, icon=ICON_LIGHTBULB, accuracy_decimals=2, + device_class=DEVICE_CLASS_REACTIVE_POWER, state_class=STATE_CLASS_MEASUREMENT, ), cv.Optional(CONF_POWER_FACTOR): sensor.sensor_schema( diff --git a/esphome/components/atm90e32/atm90e32.cpp b/esphome/components/atm90e32/atm90e32.cpp index 4669a59e39..634260b5e9 100644 --- a/esphome/components/atm90e32/atm90e32.cpp +++ b/esphome/components/atm90e32/atm90e32.cpp @@ -109,8 +109,9 @@ void ATM90E32Component::update() { } void ATM90E32Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); + this->cs_summary_ = this->cs_->dump_summary(); + const char *cs = this->cs_summary_.c_str(); uint16_t mmode0 = 0x87; // 3P4W 50Hz uint16_t high_thresh = 0; @@ -131,9 +132,9 @@ void ATM90E32Component::setup() { mmode0 |= 0 << 1; // sets 1st bit to 0, phase b is not counted into the all-phase sum energy/power (P/Q/S) } - this->write16_(ATM90E32_REGISTER_SOFTRESET, 0x789A); // Perform soft reset - delay(6); // Wait for the minimum 5ms + 1ms - this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x55AA); // enable register config access + this->write16_(ATM90E32_REGISTER_SOFTRESET, 0x789A, false); // Perform soft reset + delay(6); // Wait for the minimum 5ms + 1ms + this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x55AA); // enable register config access if (!this->validate_spi_read_(0x55AA, "setup()")) { ESP_LOGW(TAG, "Could not initialize ATM90E32 IC, check SPI settings"); this->mark_failed(); @@ -157,16 +158,17 @@ void ATM90E32Component::setup() { if (this->enable_offset_calibration_) { // Initialize flash storage for offset calibrations - uint32_t o_hash = fnv1_hash(std::string("_offset_calibration_") + this->cs_->dump_summary()); + uint32_t o_hash = fnv1_hash(std::string("_offset_calibration_") + this->cs_summary_); this->offset_pref_ = global_preferences->make_preference(o_hash, true); this->restore_offset_calibrations_(); // Initialize flash storage for power offset calibrations - uint32_t po_hash = fnv1_hash(std::string("_power_offset_calibration_") + this->cs_->dump_summary()); + uint32_t po_hash = fnv1_hash(std::string("_power_offset_calibration_") + this->cs_summary_); this->power_offset_pref_ = global_preferences->make_preference(po_hash, true); this->restore_power_offset_calibrations_(); } else { - ESP_LOGI(TAG, "[CALIBRATION] Power & Voltage/Current offset calibration is disabled. Using config file values."); + ESP_LOGI(TAG, "[CALIBRATION][%s] Power & Voltage/Current offset calibration is disabled. Using config file values.", + cs); for (uint8_t phase = 0; phase < 3; ++phase) { this->write16_(this->voltage_offset_registers[phase], static_cast(this->offset_phase_[phase].voltage_offset_)); @@ -181,21 +183,18 @@ void ATM90E32Component::setup() { if (this->enable_gain_calibration_) { // Initialize flash storage for gain calibration - uint32_t g_hash = fnv1_hash(std::string("_gain_calibration_") + this->cs_->dump_summary()); + uint32_t g_hash = fnv1_hash(std::string("_gain_calibration_") + this->cs_summary_); this->gain_calibration_pref_ = global_preferences->make_preference(g_hash, true); this->restore_gain_calibrations_(); - if (this->using_saved_calibrations_) { - ESP_LOGI(TAG, "[CALIBRATION] Successfully restored gain calibration from memory."); - } else { + if (!this->using_saved_calibrations_) { for (uint8_t phase = 0; phase < 3; ++phase) { this->write16_(voltage_gain_registers[phase], this->phase_[phase].voltage_gain_); this->write16_(current_gain_registers[phase], this->phase_[phase].ct_gain_); } } } else { - ESP_LOGI(TAG, "[CALIBRATION] Gain calibration is disabled. Using config file values."); - + ESP_LOGI(TAG, "[CALIBRATION][%s] Gain calibration is disabled. Using config file values.", cs); for (uint8_t phase = 0; phase < 3; ++phase) { this->write16_(voltage_gain_registers[phase], this->phase_[phase].voltage_gain_); this->write16_(current_gain_registers[phase], this->phase_[phase].ct_gain_); @@ -214,6 +213,122 @@ void ATM90E32Component::setup() { this->write16_(ATM90E32_REGISTER_CFGREGACCEN, 0x0000); // end configuration } +void ATM90E32Component::log_calibration_status_() { + const char *cs = this->cs_summary_.c_str(); + + bool offset_mismatch = false; + bool power_mismatch = false; + bool gain_mismatch = false; + + for (uint8_t phase = 0; phase < 3; ++phase) { + offset_mismatch |= this->offset_calibration_mismatch_[phase]; + power_mismatch |= this->power_offset_calibration_mismatch_[phase]; + gain_mismatch |= this->gain_calibration_mismatch_[phase]; + } + + if (offset_mismatch) { + ESP_LOGW(TAG, "[CALIBRATION][%s] ", cs); + ESP_LOGW(TAG, + "[CALIBRATION][%s] ===================== Offset mismatch: using flash values =====================", cs); + ESP_LOGW(TAG, "[CALIBRATION][%s] ------------------------------------------------------------------------------", + cs); + ESP_LOGW(TAG, "[CALIBRATION][%s] | Phase | offset_voltage | offset_current |", cs); + ESP_LOGW(TAG, "[CALIBRATION][%s] | | config | flash | config | flash |", cs); + ESP_LOGW(TAG, "[CALIBRATION][%s] ------------------------------------------------------------------------------", + cs); + for (uint8_t phase = 0; phase < 3; ++phase) { + ESP_LOGW(TAG, "[CALIBRATION][%s] | %c | %6d | %6d | %6d | %6d |", cs, 'A' + phase, + this->config_offset_phase_[phase].voltage_offset_, this->offset_phase_[phase].voltage_offset_, + this->config_offset_phase_[phase].current_offset_, this->offset_phase_[phase].current_offset_); + } + ESP_LOGW(TAG, + "[CALIBRATION][%s] ===============================================================================", cs); + } + if (power_mismatch) { + ESP_LOGW(TAG, "[CALIBRATION][%s] ", cs); + ESP_LOGW(TAG, + "[CALIBRATION][%s] ================= Power offset mismatch: using flash values =================", cs); + ESP_LOGW(TAG, "[CALIBRATION][%s] ------------------------------------------------------------------------------", + cs); + ESP_LOGW(TAG, "[CALIBRATION][%s] | Phase | offset_active_power|offset_reactive_power|", cs); + ESP_LOGW(TAG, "[CALIBRATION][%s] | | config | flash | config | flash |", cs); + ESP_LOGW(TAG, "[CALIBRATION][%s] ------------------------------------------------------------------------------", + cs); + for (uint8_t phase = 0; phase < 3; ++phase) { + ESP_LOGW(TAG, "[CALIBRATION][%s] | %c | %6d | %6d | %6d | %6d |", cs, 'A' + phase, + this->config_power_offset_phase_[phase].active_power_offset, + this->power_offset_phase_[phase].active_power_offset, + this->config_power_offset_phase_[phase].reactive_power_offset, + this->power_offset_phase_[phase].reactive_power_offset); + } + ESP_LOGW(TAG, + "[CALIBRATION][%s] ===============================================================================", cs); + } + if (gain_mismatch) { + ESP_LOGW(TAG, "[CALIBRATION][%s] ", cs); + ESP_LOGW(TAG, + "[CALIBRATION][%s] ====================== Gain mismatch: using flash values =====================", cs); + ESP_LOGW(TAG, "[CALIBRATION][%s] ------------------------------------------------------------------------------", + cs); + ESP_LOGW(TAG, "[CALIBRATION][%s] | Phase | voltage_gain | current_gain |", cs); + ESP_LOGW(TAG, "[CALIBRATION][%s] | | config | flash | config | flash |", cs); + ESP_LOGW(TAG, "[CALIBRATION][%s] ------------------------------------------------------------------------------", + cs); + for (uint8_t phase = 0; phase < 3; ++phase) { + ESP_LOGW(TAG, "[CALIBRATION][%s] | %c | %6u | %6u | %6u | %6u |", cs, 'A' + phase, + this->config_gain_phase_[phase].voltage_gain, this->gain_phase_[phase].voltage_gain, + this->config_gain_phase_[phase].current_gain, this->gain_phase_[phase].current_gain); + } + ESP_LOGW(TAG, + "[CALIBRATION][%s] ===============================================================================", cs); + } + if (!this->enable_offset_calibration_) { + ESP_LOGI(TAG, "[CALIBRATION][%s] Power & Voltage/Current offset calibration is disabled. Using config file values.", + cs); + } else if (this->restored_offset_calibration_ && !offset_mismatch) { + ESP_LOGI(TAG, "[CALIBRATION][%s] ", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ============== Restored offset calibration from memory ==============", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] --------------------------------------------------------------", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_voltage | offset_current |", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] --------------------------------------------------------------", cs); + for (uint8_t phase = 0; phase < 3; phase++) { + ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, + this->offset_phase_[phase].voltage_offset_, this->offset_phase_[phase].current_offset_); + } + ESP_LOGI(TAG, "[CALIBRATION][%s] ==============================================================\\n", cs); + } + + if (this->restored_power_offset_calibration_ && !power_mismatch) { + ESP_LOGI(TAG, "[CALIBRATION][%s] ", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ============ Restored power offset calibration from memory ============", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_active_power | offset_reactive_power |", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); + for (uint8_t phase = 0; phase < 3; phase++) { + ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, + this->power_offset_phase_[phase].active_power_offset, + this->power_offset_phase_[phase].reactive_power_offset); + } + ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\n", cs); + } + if (!this->enable_gain_calibration_) { + ESP_LOGI(TAG, "[CALIBRATION][%s] Gain calibration is disabled. Using config file values.", cs); + } else if (this->restored_gain_calibration_ && !gain_mismatch) { + ESP_LOGI(TAG, "[CALIBRATION][%s] ", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ============ Restoring saved gain calibrations to registers ============", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | voltage_gain | current_gain |", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); + for (uint8_t phase = 0; phase < 3; phase++) { + ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6u | %6u |", cs, 'A' + phase, + this->gain_phase_[phase].voltage_gain, this->gain_phase_[phase].current_gain); + } + ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\\n", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] Gain calibration loaded and verified successfully.\n", cs); + } + this->calibration_message_printed_ = true; +} + void ATM90E32Component::dump_config() { ESP_LOGCONFIG("", "ATM90E32:"); LOG_PIN(" CS Pin: ", this->cs_); @@ -256,6 +371,10 @@ void ATM90E32Component::dump_config() { LOG_SENSOR(" ", "Peak Current C", this->phase_[PHASEC].peak_current_sensor_); LOG_SENSOR(" ", "Frequency", this->freq_sensor_); LOG_SENSOR(" ", "Chip Temp", this->chip_temperature_sensor_); + if (this->restored_offset_calibration_ || this->restored_power_offset_calibration_ || + this->restored_gain_calibration_ || !this->enable_offset_calibration_ || !this->enable_gain_calibration_) { + this->log_calibration_status_(); + } } float ATM90E32Component::get_setup_priority() const { return setup_priority::IO; } @@ -264,19 +383,17 @@ float ATM90E32Component::get_setup_priority() const { return setup_priority::IO; // Peakdetect period: 05H. Bit 15:8 are PeakDet_period in ms. 7:0 are Sag_period // Default is 143FH (20ms, 63ms) uint16_t ATM90E32Component::read16_(uint16_t a_register) { + this->enable(); + delay_microseconds_safe(1); // min delay between CS low and first SCK is 200ns - 1us is plenty uint8_t addrh = (1 << 7) | ((a_register >> 8) & 0x03); uint8_t addrl = (a_register & 0xFF); - uint8_t data[2]; - uint16_t output; - this->enable(); - delay_microseconds_safe(1); // min delay between CS low and first SCK is 200ns - 1ms is plenty - this->write_byte(addrh); - this->write_byte(addrl); - this->read_array(data, 2); - this->disable(); - - output = (uint16_t(data[0] & 0xFF) << 8) | (data[1] & 0xFF); + uint8_t data[4] = {addrh, addrl, 0x00, 0x00}; + this->transfer_array(data, 4); + uint16_t output = encode_uint16(data[2], data[3]); ESP_LOGVV(TAG, "read16_ 0x%04" PRIX16 " output 0x%04" PRIX16, a_register, output); + delay_microseconds_safe(1); // allow the last clock to propagate before releasing CS + this->disable(); + delay_microseconds_safe(1); // meet minimum CS high time before next transaction return output; } @@ -293,13 +410,19 @@ int ATM90E32Component::read32_(uint16_t addr_h, uint16_t addr_l) { return val; } -void ATM90E32Component::write16_(uint16_t a_register, uint16_t val) { +void ATM90E32Component::write16_(uint16_t a_register, uint16_t val, bool validate) { ESP_LOGVV(TAG, "write16_ 0x%04" PRIX16 " val 0x%04" PRIX16, a_register, val); + uint8_t addrh = ((a_register >> 8) & 0x03); + uint8_t addrl = (a_register & 0xFF); + uint8_t data[4] = {addrh, addrl, uint8_t((val >> 8) & 0xFF), uint8_t(val & 0xFF)}; this->enable(); - this->write_byte16(a_register); - this->write_byte16(val); + delay_microseconds_safe(1); // ensure CS setup time + this->write_array(data, 4); + delay_microseconds_safe(1); // allow clock to settle before raising CS this->disable(); - this->validate_spi_read_(val, "write16()"); + delay_microseconds_safe(1); // ensure minimum CS high time + if (validate) + this->validate_spi_read_(val, "write16()"); } float ATM90E32Component::get_local_phase_voltage_(uint8_t phase) { return this->phase_[phase].voltage_; } @@ -442,8 +565,10 @@ float ATM90E32Component::get_chip_temperature_() { } void ATM90E32Component::run_gain_calibrations() { + const char *cs = this->cs_summary_.c_str(); if (!this->enable_gain_calibration_) { - ESP_LOGW(TAG, "[CALIBRATION] Gain calibration is disabled! Enable it first with enable_gain_calibration: true"); + ESP_LOGW(TAG, "[CALIBRATION][%s] Gain calibration is disabled! Enable it first with enable_gain_calibration: true", + cs); return; } @@ -455,12 +580,14 @@ void ATM90E32Component::run_gain_calibrations() { float ref_currents[3] = {this->get_reference_current(0), this->get_reference_current(1), this->get_reference_current(2)}; - ESP_LOGI(TAG, "[CALIBRATION] "); - ESP_LOGI(TAG, "[CALIBRATION] ========================= Gain Calibration ========================="); - ESP_LOGI(TAG, "[CALIBRATION] ---------------------------------------------------------------------"); - ESP_LOGI(TAG, - "[CALIBRATION] | Phase | V_meas (V) | I_meas (A) | V_ref | I_ref | V_gain (old→new) | I_gain (old→new) |"); - ESP_LOGI(TAG, "[CALIBRATION] ---------------------------------------------------------------------"); + ESP_LOGI(TAG, "[CALIBRATION][%s] ", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ========================= Gain Calibration =========================", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); + ESP_LOGI( + TAG, + "[CALIBRATION][%s] | Phase | V_meas (V) | I_meas (A) | V_ref | I_ref | V_gain (old→new) | I_gain (old→new) |", + cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); for (uint8_t phase = 0; phase < 3; phase++) { float measured_voltage = this->get_phase_voltage_avg_(phase); @@ -477,22 +604,22 @@ void ATM90E32Component::run_gain_calibrations() { // Voltage calibration if (ref_voltage <= 0.0f) { - ESP_LOGW(TAG, "[CALIBRATION] Phase %s - Skipping voltage calibration: reference voltage is 0.", + ESP_LOGW(TAG, "[CALIBRATION][%s] Phase %s - Skipping voltage calibration: reference voltage is 0.", cs, phase_labels[phase]); } else if (measured_voltage == 0.0f) { - ESP_LOGW(TAG, "[CALIBRATION] Phase %s - Skipping voltage calibration: measured voltage is 0.", + ESP_LOGW(TAG, "[CALIBRATION][%s] Phase %s - Skipping voltage calibration: measured voltage is 0.", cs, phase_labels[phase]); } else { uint32_t new_voltage_gain = static_cast((ref_voltage / measured_voltage) * current_voltage_gain); if (new_voltage_gain == 0) { - ESP_LOGW(TAG, "[CALIBRATION] Phase %s - Voltage gain would be 0. Check reference and measured voltage.", + ESP_LOGW(TAG, "[CALIBRATION][%s] Phase %s - Voltage gain would be 0. Check reference and measured voltage.", cs, phase_labels[phase]); } else { if (new_voltage_gain >= 65535) { - ESP_LOGW( - TAG, - "[CALIBRATION] Phase %s - Voltage gain exceeds 65535. You may need a higher output voltage transformer.", - phase_labels[phase]); + ESP_LOGW(TAG, + "[CALIBRATION][%s] Phase %s - Voltage gain exceeds 65535. You may need a higher output voltage " + "transformer.", + cs, phase_labels[phase]); new_voltage_gain = 65535; } this->gain_phase_[phase].voltage_gain = static_cast(new_voltage_gain); @@ -502,20 +629,20 @@ void ATM90E32Component::run_gain_calibrations() { // Current calibration if (ref_current == 0.0f) { - ESP_LOGW(TAG, "[CALIBRATION] Phase %s - Skipping current calibration: reference current is 0.", + ESP_LOGW(TAG, "[CALIBRATION][%s] Phase %s - Skipping current calibration: reference current is 0.", cs, phase_labels[phase]); } else if (measured_current == 0.0f) { - ESP_LOGW(TAG, "[CALIBRATION] Phase %s - Skipping current calibration: measured current is 0.", + ESP_LOGW(TAG, "[CALIBRATION][%s] Phase %s - Skipping current calibration: measured current is 0.", cs, phase_labels[phase]); } else { uint32_t new_current_gain = static_cast((ref_current / measured_current) * current_current_gain); if (new_current_gain == 0) { - ESP_LOGW(TAG, "[CALIBRATION] Phase %s - Current gain would be 0. Check reference and measured current.", + ESP_LOGW(TAG, "[CALIBRATION][%s] Phase %s - Current gain would be 0. Check reference and measured current.", cs, phase_labels[phase]); } else { if (new_current_gain >= 65535) { - ESP_LOGW(TAG, "[CALIBRATION] Phase %s - Current gain exceeds 65535. You may need to turn up pga gain.", - phase_labels[phase]); + ESP_LOGW(TAG, "[CALIBRATION][%s] Phase %s - Current gain exceeds 65535. You may need to turn up pga gain.", + cs, phase_labels[phase]); new_current_gain = 65535; } this->gain_phase_[phase].current_gain = static_cast(new_current_gain); @@ -524,13 +651,13 @@ void ATM90E32Component::run_gain_calibrations() { } // Final row output - ESP_LOGI(TAG, "[CALIBRATION] | %c | %9.2f | %9.4f | %5.2f | %6.4f | %5u → %-5u | %5u → %-5u |", + ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %9.2f | %9.4f | %5.2f | %6.4f | %5u → %-5u | %5u → %-5u |", cs, 'A' + phase, measured_voltage, measured_current, ref_voltage, ref_current, current_voltage_gain, did_voltage ? this->gain_phase_[phase].voltage_gain : current_voltage_gain, current_current_gain, did_current ? this->gain_phase_[phase].current_gain : current_current_gain); } - ESP_LOGI(TAG, "[CALIBRATION] =====================================================================\n"); + ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\n", cs); this->save_gain_calibration_to_memory_(); this->write_gains_to_registers_(); @@ -538,54 +665,108 @@ void ATM90E32Component::run_gain_calibrations() { } void ATM90E32Component::save_gain_calibration_to_memory_() { + const char *cs = this->cs_summary_.c_str(); bool success = this->gain_calibration_pref_.save(&this->gain_phase_); + global_preferences->sync(); if (success) { this->using_saved_calibrations_ = true; - ESP_LOGI(TAG, "[CALIBRATION] Gain calibration saved to memory."); + ESP_LOGI(TAG, "[CALIBRATION][%s] Gain calibration saved to memory.", cs); } else { this->using_saved_calibrations_ = false; - ESP_LOGE(TAG, "[CALIBRATION] Failed to save gain calibration to memory!"); + ESP_LOGE(TAG, "[CALIBRATION][%s] Failed to save gain calibration to memory!", cs); + } +} + +void ATM90E32Component::save_offset_calibration_to_memory_() { + const char *cs = this->cs_summary_.c_str(); + bool success = this->offset_pref_.save(&this->offset_phase_); + global_preferences->sync(); + if (success) { + this->using_saved_calibrations_ = true; + this->restored_offset_calibration_ = true; + for (bool &phase : this->offset_calibration_mismatch_) + phase = false; + ESP_LOGI(TAG, "[CALIBRATION][%s] Offset calibration saved to memory.", cs); + } else { + this->using_saved_calibrations_ = false; + ESP_LOGE(TAG, "[CALIBRATION][%s] Failed to save offset calibration to memory!", cs); + } +} + +void ATM90E32Component::save_power_offset_calibration_to_memory_() { + const char *cs = this->cs_summary_.c_str(); + bool success = this->power_offset_pref_.save(&this->power_offset_phase_); + global_preferences->sync(); + if (success) { + this->using_saved_calibrations_ = true; + this->restored_power_offset_calibration_ = true; + for (bool &phase : this->power_offset_calibration_mismatch_) + phase = false; + ESP_LOGI(TAG, "[CALIBRATION][%s] Power offset calibration saved to memory.", cs); + } else { + this->using_saved_calibrations_ = false; + ESP_LOGE(TAG, "[CALIBRATION][%s] Failed to save power offset calibration to memory!", cs); } } void ATM90E32Component::run_offset_calibrations() { + const char *cs = this->cs_summary_.c_str(); if (!this->enable_offset_calibration_) { - ESP_LOGW(TAG, "[CALIBRATION] Offset calibration is disabled! Enable it first with enable_offset_calibration: true"); + ESP_LOGW(TAG, + "[CALIBRATION][%s] Offset calibration is disabled! Enable it first with enable_offset_calibration: true", + cs); return; } + ESP_LOGI(TAG, "[CALIBRATION][%s] ", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ======================== Offset Calibration ========================", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ------------------------------------------------------------------", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_voltage | offset_current |", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ------------------------------------------------------------------", cs); + for (uint8_t phase = 0; phase < 3; phase++) { int16_t voltage_offset = calibrate_offset(phase, true); int16_t current_offset = calibrate_offset(phase, false); this->write_offsets_to_registers_(phase, voltage_offset, current_offset); - ESP_LOGI(TAG, "[CALIBRATION] Phase %c - offset_voltage: %d, offset_current: %d", 'A' + phase, voltage_offset, + ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, voltage_offset, current_offset); } - this->offset_pref_.save(&this->offset_phase_); // Save to flash + ESP_LOGI(TAG, "[CALIBRATION][%s] ==================================================================\n", cs); + + this->save_offset_calibration_to_memory_(); } void ATM90E32Component::run_power_offset_calibrations() { + const char *cs = this->cs_summary_.c_str(); if (!this->enable_offset_calibration_) { ESP_LOGW( TAG, - "[CALIBRATION] Offset power calibration is disabled! Enable it first with enable_offset_calibration: true"); + "[CALIBRATION][%s] Offset power calibration is disabled! Enable it first with enable_offset_calibration: true", + cs); return; } + ESP_LOGI(TAG, "[CALIBRATION][%s] ", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ===================== Power Offset Calibration =====================", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_active_power | offset_reactive_power |", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); + for (uint8_t phase = 0; phase < 3; ++phase) { int16_t active_offset = calibrate_power_offset(phase, false); int16_t reactive_offset = calibrate_power_offset(phase, true); this->write_power_offsets_to_registers_(phase, active_offset, reactive_offset); - ESP_LOGI(TAG, "[CALIBRATION] Phase %c - offset_active_power: %d, offset_reactive_power: %d", 'A' + phase, - active_offset, reactive_offset); + ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, active_offset, + reactive_offset); } + ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\n", cs); - this->power_offset_pref_.save(&this->power_offset_phase_); // Save to flash + this->save_power_offset_calibration_to_memory_(); } void ATM90E32Component::write_gains_to_registers_() { @@ -632,102 +813,276 @@ void ATM90E32Component::write_power_offsets_to_registers_(uint8_t phase, int16_t } void ATM90E32Component::restore_gain_calibrations_() { - if (this->gain_calibration_pref_.load(&this->gain_phase_)) { - ESP_LOGI(TAG, "[CALIBRATION] Restoring saved gain calibrations to registers:"); - - for (uint8_t phase = 0; phase < 3; phase++) { - uint16_t v_gain = this->gain_phase_[phase].voltage_gain; - uint16_t i_gain = this->gain_phase_[phase].current_gain; - ESP_LOGI(TAG, "[CALIBRATION] Phase %c - Voltage Gain: %u, Current Gain: %u", 'A' + phase, v_gain, i_gain); - } - - this->write_gains_to_registers_(); - - if (this->verify_gain_writes_()) { - this->using_saved_calibrations_ = true; - ESP_LOGI(TAG, "[CALIBRATION] Gain calibration loaded and verified successfully."); - } else { - this->using_saved_calibrations_ = false; - ESP_LOGE(TAG, "[CALIBRATION] Gain verification failed! Calibration may not be applied correctly."); - } - } else { - this->using_saved_calibrations_ = false; - ESP_LOGW(TAG, "[CALIBRATION] No stored gain calibrations found. Using config file values."); + const char *cs = this->cs_summary_.c_str(); + for (uint8_t i = 0; i < 3; ++i) { + this->config_gain_phase_[i].voltage_gain = this->phase_[i].voltage_gain_; + this->config_gain_phase_[i].current_gain = this->phase_[i].ct_gain_; + this->gain_phase_[i] = this->config_gain_phase_[i]; } + + if (this->gain_calibration_pref_.load(&this->gain_phase_)) { + bool all_zero = true; + bool same_as_config = true; + for (uint8_t phase = 0; phase < 3; ++phase) { + const auto &cfg = this->config_gain_phase_[phase]; + const auto &saved = this->gain_phase_[phase]; + if (saved.voltage_gain != 0 || saved.current_gain != 0) + all_zero = false; + if (saved.voltage_gain != cfg.voltage_gain || saved.current_gain != cfg.current_gain) + same_as_config = false; + } + + if (!all_zero && !same_as_config) { + for (uint8_t phase = 0; phase < 3; ++phase) { + bool mismatch = false; + if (this->has_config_voltage_gain_[phase] && + this->gain_phase_[phase].voltage_gain != this->config_gain_phase_[phase].voltage_gain) + mismatch = true; + if (this->has_config_current_gain_[phase] && + this->gain_phase_[phase].current_gain != this->config_gain_phase_[phase].current_gain) + mismatch = true; + if (mismatch) + this->gain_calibration_mismatch_[phase] = true; + } + + this->write_gains_to_registers_(); + + if (this->verify_gain_writes_()) { + this->using_saved_calibrations_ = true; + this->restored_gain_calibration_ = true; + return; + } + + this->using_saved_calibrations_ = false; + ESP_LOGE(TAG, "[CALIBRATION][%s] Gain verification failed! Calibration may not be applied correctly.", cs); + } + } + + this->using_saved_calibrations_ = false; + for (uint8_t i = 0; i < 3; ++i) + this->gain_phase_[i] = this->config_gain_phase_[i]; + this->write_gains_to_registers_(); + + ESP_LOGW(TAG, "[CALIBRATION][%s] No stored gain calibrations found. Using config file values.", cs); } void ATM90E32Component::restore_offset_calibrations_() { - if (this->offset_pref_.load(&this->offset_phase_)) { - ESP_LOGI(TAG, "[CALIBRATION] Successfully restored offset calibration from memory."); + const char *cs = this->cs_summary_.c_str(); + for (uint8_t i = 0; i < 3; ++i) + this->config_offset_phase_[i] = this->offset_phase_[i]; + bool have_data = this->offset_pref_.load(&this->offset_phase_); + bool all_zero = true; + if (have_data) { + for (auto &phase : this->offset_phase_) { + if (phase.voltage_offset_ != 0 || phase.current_offset_ != 0) { + all_zero = false; + break; + } + } + } + + if (have_data && !all_zero) { + this->restored_offset_calibration_ = true; for (uint8_t phase = 0; phase < 3; phase++) { auto &offset = this->offset_phase_[phase]; - write_offsets_to_registers_(phase, offset.voltage_offset_, offset.current_offset_); - ESP_LOGI(TAG, "[CALIBRATION] Phase %c - offset_voltage:: %d, offset_current: %d", 'A' + phase, - offset.voltage_offset_, offset.current_offset_); + bool mismatch = false; + if (this->has_config_voltage_offset_[phase] && + offset.voltage_offset_ != this->config_offset_phase_[phase].voltage_offset_) + mismatch = true; + if (this->has_config_current_offset_[phase] && + offset.current_offset_ != this->config_offset_phase_[phase].current_offset_) + mismatch = true; + if (mismatch) + this->offset_calibration_mismatch_[phase] = true; } } else { - ESP_LOGW(TAG, "[CALIBRATION] No stored offset calibrations found. Using default values."); + for (uint8_t phase = 0; phase < 3; phase++) + this->offset_phase_[phase] = this->config_offset_phase_[phase]; + ESP_LOGW(TAG, "[CALIBRATION][%s] No stored offset calibrations found. Using default values.", cs); + } + + for (uint8_t phase = 0; phase < 3; phase++) { + write_offsets_to_registers_(phase, this->offset_phase_[phase].voltage_offset_, + this->offset_phase_[phase].current_offset_); } } void ATM90E32Component::restore_power_offset_calibrations_() { - if (this->power_offset_pref_.load(&this->power_offset_phase_)) { - ESP_LOGI(TAG, "[CALIBRATION] Successfully restored power offset calibration from memory."); + const char *cs = this->cs_summary_.c_str(); + for (uint8_t i = 0; i < 3; ++i) + this->config_power_offset_phase_[i] = this->power_offset_phase_[i]; + bool have_data = this->power_offset_pref_.load(&this->power_offset_phase_); + bool all_zero = true; + if (have_data) { + for (auto &phase : this->power_offset_phase_) { + if (phase.active_power_offset != 0 || phase.reactive_power_offset != 0) { + all_zero = false; + break; + } + } + } + + if (have_data && !all_zero) { + this->restored_power_offset_calibration_ = true; for (uint8_t phase = 0; phase < 3; ++phase) { auto &offset = this->power_offset_phase_[phase]; - write_power_offsets_to_registers_(phase, offset.active_power_offset, offset.reactive_power_offset); - ESP_LOGI(TAG, "[CALIBRATION] Phase %c - offset_active_power: %d, offset_reactive_power: %d", 'A' + phase, - offset.active_power_offset, offset.reactive_power_offset); + bool mismatch = false; + if (this->has_config_active_power_offset_[phase] && + offset.active_power_offset != this->config_power_offset_phase_[phase].active_power_offset) + mismatch = true; + if (this->has_config_reactive_power_offset_[phase] && + offset.reactive_power_offset != this->config_power_offset_phase_[phase].reactive_power_offset) + mismatch = true; + if (mismatch) + this->power_offset_calibration_mismatch_[phase] = true; } } else { - ESP_LOGW(TAG, "[CALIBRATION] No stored power offsets found. Using default values."); + for (uint8_t phase = 0; phase < 3; ++phase) + this->power_offset_phase_[phase] = this->config_power_offset_phase_[phase]; + ESP_LOGW(TAG, "[CALIBRATION][%s] No stored power offsets found. Using default values.", cs); + } + + for (uint8_t phase = 0; phase < 3; ++phase) { + write_power_offsets_to_registers_(phase, this->power_offset_phase_[phase].active_power_offset, + this->power_offset_phase_[phase].reactive_power_offset); } } void ATM90E32Component::clear_gain_calibrations() { - ESP_LOGI(TAG, "[CALIBRATION] Clearing stored gain calibrations and restoring config-defined values"); - - for (int phase = 0; phase < 3; phase++) { - gain_phase_[phase].voltage_gain = this->phase_[phase].voltage_gain_; - gain_phase_[phase].current_gain = this->phase_[phase].ct_gain_; + const char *cs = this->cs_summary_.c_str(); + if (!this->using_saved_calibrations_) { + ESP_LOGI(TAG, "[CALIBRATION][%s] No stored gain calibrations to clear. Current values:", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ----------------------------------------------------------", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | voltage_gain | current_gain |", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ----------------------------------------------------------", cs); + for (int phase = 0; phase < 3; phase++) { + ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6u | %6u |", cs, 'A' + phase, + this->gain_phase_[phase].voltage_gain, this->gain_phase_[phase].current_gain); + } + ESP_LOGI(TAG, "[CALIBRATION][%s] ==========================================================\n", cs); + return; } - bool success = this->gain_calibration_pref_.save(&this->gain_phase_); - this->using_saved_calibrations_ = false; + ESP_LOGI(TAG, "[CALIBRATION][%s] Clearing stored gain calibrations and restoring config-defined values", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ----------------------------------------------------------", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | voltage_gain | current_gain |", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ----------------------------------------------------------", cs); - if (success) { - ESP_LOGI(TAG, "[CALIBRATION] Gain calibrations cleared. Config values restored:"); - for (int phase = 0; phase < 3; phase++) { - ESP_LOGI(TAG, "[CALIBRATION] Phase %c - Voltage Gain: %u, Current Gain: %u", 'A' + phase, - gain_phase_[phase].voltage_gain, gain_phase_[phase].current_gain); - } - } else { - ESP_LOGE(TAG, "[CALIBRATION] Failed to clear gain calibrations!"); + for (int phase = 0; phase < 3; phase++) { + uint16_t voltage_gain = this->phase_[phase].voltage_gain_; + uint16_t current_gain = this->phase_[phase].ct_gain_; + + this->config_gain_phase_[phase].voltage_gain = voltage_gain; + this->config_gain_phase_[phase].current_gain = current_gain; + this->gain_phase_[phase].voltage_gain = voltage_gain; + this->gain_phase_[phase].current_gain = current_gain; + + ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6u | %6u |", cs, 'A' + phase, voltage_gain, current_gain); + } + ESP_LOGI(TAG, "[CALIBRATION][%s] ==========================================================\n", cs); + + GainCalibration zero_gains[3]{{0, 0}, {0, 0}, {0, 0}}; + bool success = this->gain_calibration_pref_.save(&zero_gains); + global_preferences->sync(); + + this->using_saved_calibrations_ = false; + this->restored_gain_calibration_ = false; + for (bool &phase : this->gain_calibration_mismatch_) + phase = false; + + if (!success) { + ESP_LOGE(TAG, "[CALIBRATION][%s] Failed to clear gain calibrations!", cs); } this->write_gains_to_registers_(); // Apply them to the chip immediately } void ATM90E32Component::clear_offset_calibrations() { - for (uint8_t phase = 0; phase < 3; phase++) { - this->write_offsets_to_registers_(phase, 0, 0); + const char *cs = this->cs_summary_.c_str(); + if (!this->restored_offset_calibration_) { + ESP_LOGI(TAG, "[CALIBRATION][%s] No stored offset calibrations to clear. Current values:", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] --------------------------------------------------------------", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_voltage | offset_current |", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] --------------------------------------------------------------", cs); + for (uint8_t phase = 0; phase < 3; phase++) { + ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, + this->offset_phase_[phase].voltage_offset_, this->offset_phase_[phase].current_offset_); + } + ESP_LOGI(TAG, "[CALIBRATION][%s] ==============================================================\n", cs); + return; } - this->offset_pref_.save(&this->offset_phase_); // Save cleared values to flash memory + ESP_LOGI(TAG, "[CALIBRATION][%s] Clearing stored offset calibrations and restoring config-defined values", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] --------------------------------------------------------------", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_voltage | offset_current |", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] --------------------------------------------------------------", cs); - ESP_LOGI(TAG, "[CALIBRATION] Offsets cleared."); + for (uint8_t phase = 0; phase < 3; phase++) { + int16_t voltage_offset = + this->has_config_voltage_offset_[phase] ? this->config_offset_phase_[phase].voltage_offset_ : 0; + int16_t current_offset = + this->has_config_current_offset_[phase] ? this->config_offset_phase_[phase].current_offset_ : 0; + this->write_offsets_to_registers_(phase, voltage_offset, current_offset); + ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, voltage_offset, + current_offset); + } + ESP_LOGI(TAG, "[CALIBRATION][%s] ==============================================================\n", cs); + + OffsetCalibration zero_offsets[3]{{0, 0}, {0, 0}, {0, 0}}; + this->offset_pref_.save(&zero_offsets); // Clear stored values in flash + global_preferences->sync(); + + this->restored_offset_calibration_ = false; + for (bool &phase : this->offset_calibration_mismatch_) + phase = false; + + ESP_LOGI(TAG, "[CALIBRATION][%s] Offsets cleared.", cs); } void ATM90E32Component::clear_power_offset_calibrations() { - for (uint8_t phase = 0; phase < 3; phase++) { - this->write_power_offsets_to_registers_(phase, 0, 0); + const char *cs = this->cs_summary_.c_str(); + if (!this->restored_power_offset_calibration_) { + ESP_LOGI(TAG, "[CALIBRATION][%s] No stored power offsets to clear. Current values:", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_active_power | offset_reactive_power |", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); + for (uint8_t phase = 0; phase < 3; phase++) { + ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, + this->power_offset_phase_[phase].active_power_offset, + this->power_offset_phase_[phase].reactive_power_offset); + } + ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\n", cs); + return; } - this->power_offset_pref_.save(&this->power_offset_phase_); + ESP_LOGI(TAG, "[CALIBRATION][%s] Clearing stored power offsets and restoring config-defined values", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] | Phase | offset_active_power | offset_reactive_power |", cs); + ESP_LOGI(TAG, "[CALIBRATION][%s] ---------------------------------------------------------------------", cs); - ESP_LOGI(TAG, "[CALIBRATION] Power offsets cleared."); + for (uint8_t phase = 0; phase < 3; phase++) { + int16_t active_offset = + this->has_config_active_power_offset_[phase] ? this->config_power_offset_phase_[phase].active_power_offset : 0; + int16_t reactive_offset = this->has_config_reactive_power_offset_[phase] + ? this->config_power_offset_phase_[phase].reactive_power_offset + : 0; + this->write_power_offsets_to_registers_(phase, active_offset, reactive_offset); + ESP_LOGI(TAG, "[CALIBRATION][%s] | %c | %6d | %6d |", cs, 'A' + phase, active_offset, + reactive_offset); + } + ESP_LOGI(TAG, "[CALIBRATION][%s] =====================================================================\n", cs); + + PowerOffsetCalibration zero_power_offsets[3]{{0, 0}, {0, 0}, {0, 0}}; + this->power_offset_pref_.save(&zero_power_offsets); + global_preferences->sync(); + + this->restored_power_offset_calibration_ = false; + for (bool &phase : this->power_offset_calibration_mismatch_) + phase = false; + + ESP_LOGI(TAG, "[CALIBRATION][%s] Power offsets cleared.", cs); } int16_t ATM90E32Component::calibrate_offset(uint8_t phase, bool voltage) { @@ -748,20 +1103,21 @@ int16_t ATM90E32Component::calibrate_offset(uint8_t phase, bool voltage) { int16_t ATM90E32Component::calibrate_power_offset(uint8_t phase, bool reactive) { const uint8_t num_reads = 5; - uint64_t total_value = 0; + int64_t total_value = 0; for (uint8_t i = 0; i < num_reads; ++i) { - uint32_t reading = reactive ? this->read32_(ATM90E32_REGISTER_QMEAN + phase, ATM90E32_REGISTER_QMEANLSB + phase) - : this->read32_(ATM90E32_REGISTER_PMEAN + phase, ATM90E32_REGISTER_PMEANLSB + phase); + int32_t reading = reactive ? this->read32_(ATM90E32_REGISTER_QMEAN + phase, ATM90E32_REGISTER_QMEANLSB + phase) + : this->read32_(ATM90E32_REGISTER_PMEAN + phase, ATM90E32_REGISTER_PMEANLSB + phase); total_value += reading; } - const uint32_t average_value = total_value / num_reads; - const uint32_t power_offset = ~average_value + 1; + int32_t average_value = total_value / num_reads; + int32_t power_offset = -average_value; return static_cast(power_offset); // Takes the lower 16 bits } bool ATM90E32Component::verify_gain_writes_() { + const char *cs = this->cs_summary_.c_str(); bool success = true; for (uint8_t phase = 0; phase < 3; phase++) { uint16_t read_voltage = this->read16_(voltage_gain_registers[phase]); @@ -769,7 +1125,7 @@ bool ATM90E32Component::verify_gain_writes_() { if (read_voltage != this->gain_phase_[phase].voltage_gain || read_current != this->gain_phase_[phase].current_gain) { - ESP_LOGE(TAG, "[CALIBRATION] Mismatch detected for Phase %s!", phase_labels[phase]); + ESP_LOGE(TAG, "[CALIBRATION][%s] Mismatch detected for Phase %s!", cs, phase_labels[phase]); success = false; } } @@ -792,16 +1148,16 @@ void ATM90E32Component::check_phase_status() { status += "Phase Loss; "; auto *sensor = this->phase_status_text_sensor_[phase]; - const char *phase_name = sensor ? sensor->get_name().c_str() : "Unknown Phase"; + if (sensor == nullptr) + continue; + if (!status.empty()) { status.pop_back(); // remove space status.pop_back(); // remove semicolon - ESP_LOGW(TAG, "%s: %s", phase_name, status.c_str()); - if (sensor != nullptr) - sensor->publish_state(status); + ESP_LOGW(TAG, "%s: %s", sensor->get_name().c_str(), status.c_str()); + sensor->publish_state(status); } else { - if (sensor != nullptr) - sensor->publish_state("Okay"); + sensor->publish_state("Okay"); } } } @@ -818,9 +1174,12 @@ void ATM90E32Component::check_freq_status() { } else { freq_status = "Normal"; } - ESP_LOGW(TAG, "Frequency status: %s", freq_status.c_str()); - if (this->freq_status_text_sensor_ != nullptr) { + if (freq_status == "Normal") { + ESP_LOGD(TAG, "Frequency status: %s", freq_status.c_str()); + } else { + ESP_LOGW(TAG, "Frequency status: %s", freq_status.c_str()); + } this->freq_status_text_sensor_->publish_state(freq_status); } } diff --git a/esphome/components/atm90e32/atm90e32.h b/esphome/components/atm90e32/atm90e32.h index 0703c40ae0..938ce512ce 100644 --- a/esphome/components/atm90e32/atm90e32.h +++ b/esphome/components/atm90e32/atm90e32.h @@ -61,15 +61,29 @@ class ATM90E32Component : public PollingComponent, this->phase_[phase].harmonic_active_power_sensor_ = obj; } void set_peak_current_sensor(int phase, sensor::Sensor *obj) { this->phase_[phase].peak_current_sensor_ = obj; } - void set_volt_gain(int phase, uint16_t gain) { this->phase_[phase].voltage_gain_ = gain; } - void set_ct_gain(int phase, uint16_t gain) { this->phase_[phase].ct_gain_ = gain; } - void set_voltage_offset(uint8_t phase, int16_t offset) { this->offset_phase_[phase].voltage_offset_ = offset; } - void set_current_offset(uint8_t phase, int16_t offset) { this->offset_phase_[phase].current_offset_ = offset; } + void set_volt_gain(int phase, uint16_t gain) { + this->phase_[phase].voltage_gain_ = gain; + this->has_config_voltage_gain_[phase] = true; + } + void set_ct_gain(int phase, uint16_t gain) { + this->phase_[phase].ct_gain_ = gain; + this->has_config_current_gain_[phase] = true; + } + void set_voltage_offset(uint8_t phase, int16_t offset) { + this->offset_phase_[phase].voltage_offset_ = offset; + this->has_config_voltage_offset_[phase] = true; + } + void set_current_offset(uint8_t phase, int16_t offset) { + this->offset_phase_[phase].current_offset_ = offset; + this->has_config_current_offset_[phase] = true; + } void set_active_power_offset(uint8_t phase, int16_t offset) { this->power_offset_phase_[phase].active_power_offset = offset; + this->has_config_active_power_offset_[phase] = true; } void set_reactive_power_offset(uint8_t phase, int16_t offset) { this->power_offset_phase_[phase].reactive_power_offset = offset; + this->has_config_reactive_power_offset_[phase] = true; } void set_freq_sensor(sensor::Sensor *freq_sensor) { freq_sensor_ = freq_sensor; } void set_peak_current_signed(bool flag) { peak_current_signed_ = flag; } @@ -127,7 +141,7 @@ class ATM90E32Component : public PollingComponent, #endif uint16_t read16_(uint16_t a_register); int read32_(uint16_t addr_h, uint16_t addr_l); - void write16_(uint16_t a_register, uint16_t val); + void write16_(uint16_t a_register, uint16_t val, bool validate = true); float get_local_phase_voltage_(uint8_t phase); float get_local_phase_current_(uint8_t phase); float get_local_phase_active_power_(uint8_t phase); @@ -159,12 +173,15 @@ class ATM90E32Component : public PollingComponent, void restore_offset_calibrations_(); void restore_power_offset_calibrations_(); void restore_gain_calibrations_(); + void save_offset_calibration_to_memory_(); void save_gain_calibration_to_memory_(); + void save_power_offset_calibration_to_memory_(); void write_offsets_to_registers_(uint8_t phase, int16_t voltage_offset, int16_t current_offset); void write_power_offsets_to_registers_(uint8_t phase, int16_t p_offset, int16_t q_offset); void write_gains_to_registers_(); bool verify_gain_writes_(); bool validate_spi_read_(uint16_t expected, const char *context = nullptr); + void log_calibration_status_(); struct ATM90E32Phase { uint16_t voltage_gain_{0}; @@ -204,19 +221,33 @@ class ATM90E32Component : public PollingComponent, int16_t current_offset_{0}; } offset_phase_[3]; + OffsetCalibration config_offset_phase_[3]; + struct PowerOffsetCalibration { int16_t active_power_offset{0}; int16_t reactive_power_offset{0}; } power_offset_phase_[3]; + PowerOffsetCalibration config_power_offset_phase_[3]; + struct GainCalibration { uint16_t voltage_gain{1}; uint16_t current_gain{1}; } gain_phase_[3]; + GainCalibration config_gain_phase_[3]; + + bool has_config_voltage_offset_[3]{false, false, false}; + bool has_config_current_offset_[3]{false, false, false}; + bool has_config_active_power_offset_[3]{false, false, false}; + bool has_config_reactive_power_offset_[3]{false, false, false}; + bool has_config_voltage_gain_[3]{false, false, false}; + bool has_config_current_gain_[3]{false, false, false}; + ESPPreferenceObject offset_pref_; ESPPreferenceObject power_offset_pref_; ESPPreferenceObject gain_calibration_pref_; + std::string cs_summary_; sensor::Sensor *freq_sensor_{nullptr}; #ifdef USE_TEXT_SENSOR @@ -231,6 +262,13 @@ class ATM90E32Component : public PollingComponent, bool peak_current_signed_{false}; bool enable_offset_calibration_{false}; bool enable_gain_calibration_{false}; + bool restored_offset_calibration_{false}; + bool restored_power_offset_calibration_{false}; + bool restored_gain_calibration_{false}; + bool calibration_message_printed_{false}; + bool offset_calibration_mismatch_[3]{false, false, false}; + bool power_offset_calibration_mismatch_[3]{false, false, false}; + bool gain_calibration_mismatch_[3]{false, false, false}; }; } // namespace atm90e32 diff --git a/esphome/components/atm90e32/sensor.py b/esphome/components/atm90e32/sensor.py index 7cdbd69f56..a510095217 100644 --- a/esphome/components/atm90e32/sensor.py +++ b/esphome/components/atm90e32/sensor.py @@ -17,10 +17,12 @@ from esphome.const import ( CONF_REACTIVE_POWER, CONF_REVERSE_ACTIVE_ENERGY, CONF_VOLTAGE, + DEVICE_CLASS_APPARENT_POWER, DEVICE_CLASS_CURRENT, DEVICE_CLASS_ENERGY, DEVICE_CLASS_POWER, DEVICE_CLASS_POWER_FACTOR, + DEVICE_CLASS_REACTIVE_POWER, DEVICE_CLASS_TEMPERATURE, DEVICE_CLASS_VOLTAGE, ENTITY_CATEGORY_DIAGNOSTIC, @@ -100,13 +102,13 @@ ATM90E32_PHASE_SCHEMA = cv.Schema( unit_of_measurement=UNIT_VOLT_AMPS_REACTIVE, icon=ICON_LIGHTBULB, accuracy_decimals=2, - device_class=DEVICE_CLASS_POWER, + device_class=DEVICE_CLASS_REACTIVE_POWER, state_class=STATE_CLASS_MEASUREMENT, ), cv.Optional(CONF_APPARENT_POWER): sensor.sensor_schema( unit_of_measurement=UNIT_VOLT_AMPS, accuracy_decimals=2, - device_class=DEVICE_CLASS_POWER, + device_class=DEVICE_CLASS_APPARENT_POWER, state_class=STATE_CLASS_MEASUREMENT, ), cv.Optional(CONF_POWER_FACTOR): sensor.sensor_schema( diff --git a/esphome/components/audio/__init__.py b/esphome/components/audio/__init__.py index f657cb5da3..7b03e4b6a7 100644 --- a/esphome/components/audio/__init__.py +++ b/esphome/components/audio/__init__.py @@ -165,4 +165,4 @@ def final_validate_audio_schema( async def to_code(config): - cg.add_library("esphome/esp-audio-libs", "1.1.4") + cg.add_library("esphome/esp-audio-libs", "2.0.1") diff --git a/esphome/components/audio/audio.cpp b/esphome/components/audio/audio.cpp index 2a58c38ac7..9cc9b7d0da 100644 --- a/esphome/components/audio/audio.cpp +++ b/esphome/components/audio/audio.cpp @@ -57,7 +57,7 @@ const char *audio_file_type_to_string(AudioFileType file_type) { void scale_audio_samples(const int16_t *audio_samples, int16_t *output_buffer, int16_t scale_factor, size_t samples_to_scale) { // Note the assembly dsps_mulc function has audio glitches if the input and output buffers are the same. - for (int i = 0; i < samples_to_scale; i++) { + for (size_t i = 0; i < samples_to_scale; i++) { int32_t acc = (int32_t) audio_samples[i] * (int32_t) scale_factor; output_buffer[i] = (int16_t) (acc >> 15); } diff --git a/esphome/components/audio/audio.h b/esphome/components/audio/audio.h index 95c31872e3..e01d7eb101 100644 --- a/esphome/components/audio/audio.h +++ b/esphome/components/audio/audio.h @@ -15,7 +15,7 @@ class AudioStreamInfo { * - An audio sample represents a unit of audio for one channel. * - A frame represents a unit of audio with a sample for every channel. * - * In gneneral, converting between bytes, samples, and frames shouldn't result in rounding errors so long as frames + * In general, converting between bytes, samples, and frames shouldn't result in rounding errors so long as frames * are used as the main unit when transferring audio data. Durations may result in rounding for certain sample rates; * e.g., 44.1 KHz. The ``frames_to_milliseconds_with_remainder`` function should be used for accuracy, as it takes * into account the remainder rather than just ignoring any rounding. @@ -76,7 +76,7 @@ class AudioStreamInfo { /// @brief Computes the duration, in microseconds, the given amount of frames represents. /// @param frames Number of audio frames - /// @return Duration in microseconds `frames` respresents. May be slightly inaccurate due to integer divison rounding + /// @return Duration in microseconds `frames` represents. May be slightly inaccurate due to integer division rounding /// for certain sample rates. uint32_t frames_to_microseconds(uint32_t frames) const; diff --git a/esphome/components/audio/audio_decoder.cpp b/esphome/components/audio/audio_decoder.cpp index 90ba1aec1e..d1ad571a52 100644 --- a/esphome/components/audio/audio_decoder.cpp +++ b/esphome/components/audio/audio_decoder.cpp @@ -229,18 +229,18 @@ FileDecoderState AudioDecoder::decode_flac_() { auto result = this->flac_decoder_->read_header(this->input_transfer_buffer_->get_buffer_start(), this->input_transfer_buffer_->available()); - if (result == esp_audio_libs::flac::FLAC_DECODER_HEADER_OUT_OF_DATA) { - return FileDecoderState::POTENTIALLY_FAILED; - } - - if (result != esp_audio_libs::flac::FLAC_DECODER_SUCCESS) { - // Couldn't read FLAC header + if (result > esp_audio_libs::flac::FLAC_DECODER_HEADER_OUT_OF_DATA) { + // Serrious error reading FLAC header, there is no recovery return FileDecoderState::FAILED; } size_t bytes_consumed = this->flac_decoder_->get_bytes_index(); this->input_transfer_buffer_->decrease_buffer_length(bytes_consumed); + if (result == esp_audio_libs::flac::FLAC_DECODER_HEADER_OUT_OF_DATA) { + return FileDecoderState::MORE_TO_PROCESS; + } + // Reallocate the output transfer buffer to the smallest necessary size this->free_buffer_required_ = flac_decoder_->get_output_buffer_size_bytes(); if (!this->output_transfer_buffer_->reallocate(this->free_buffer_required_)) { @@ -256,9 +256,9 @@ FileDecoderState AudioDecoder::decode_flac_() { } uint32_t output_samples = 0; - auto result = this->flac_decoder_->decode_frame( - this->input_transfer_buffer_->get_buffer_start(), this->input_transfer_buffer_->available(), - reinterpret_cast(this->output_transfer_buffer_->get_buffer_end()), &output_samples); + auto result = this->flac_decoder_->decode_frame(this->input_transfer_buffer_->get_buffer_start(), + this->input_transfer_buffer_->available(), + this->output_transfer_buffer_->get_buffer_end(), &output_samples); if (result == esp_audio_libs::flac::FLAC_DECODER_ERROR_OUT_OF_DATA) { // Not an issue, just needs more data that we'll get next time. diff --git a/esphome/components/audio_adc/__init__.py b/esphome/components/audio_adc/__init__.py index dd3c958821..2f95a039f5 100644 --- a/esphome/components/audio_adc/__init__.py +++ b/esphome/components/audio_adc/__init__.py @@ -2,7 +2,7 @@ from esphome import automation import esphome.codegen as cg import esphome.config_validation as cv from esphome.const import CONF_ID, CONF_MIC_GAIN -from esphome.core import coroutine_with_priority +from esphome.core import CoroPriority, coroutine_with_priority CODEOWNERS = ["@kbx81"] IS_PLATFORM_COMPONENT = True @@ -35,7 +35,7 @@ async def audio_adc_set_mic_gain_to_code(config, action_id, template_arg, args): return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): cg.add_define("USE_AUDIO_ADC") cg.add_global(audio_adc_ns.using) diff --git a/esphome/components/audio_dac/__init__.py b/esphome/components/audio_dac/__init__.py index 978ed195bd..92e6cb18fa 100644 --- a/esphome/components/audio_dac/__init__.py +++ b/esphome/components/audio_dac/__init__.py @@ -3,7 +3,7 @@ from esphome.automation import maybe_simple_id import esphome.codegen as cg import esphome.config_validation as cv from esphome.const import CONF_ID, CONF_VOLUME -from esphome.core import coroutine_with_priority +from esphome.core import CoroPriority, coroutine_with_priority CODEOWNERS = ["@kbx81"] IS_PLATFORM_COMPONENT = True @@ -51,7 +51,7 @@ async def audio_dac_set_volume_to_code(config, action_id, template_arg, args): return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): cg.add_define("USE_AUDIO_DAC") cg.add_global(audio_dac_ns.using) diff --git a/esphome/components/axs15231/touchscreen/axs15231_touchscreen.cpp b/esphome/components/axs15231/touchscreen/axs15231_touchscreen.cpp index e6e049e332..ab3f1dad4f 100644 --- a/esphome/components/axs15231/touchscreen/axs15231_touchscreen.cpp +++ b/esphome/components/axs15231/touchscreen/axs15231_touchscreen.cpp @@ -12,12 +12,11 @@ constexpr static const uint8_t AXS_READ_TOUCHPAD[11] = {0xb5, 0xab, 0xa5, 0x5a, #define ERROR_CHECK(err) \ if ((err) != i2c::ERROR_OK) { \ - this->status_set_warning("Failed to communicate"); \ + this->status_set_warning(LOG_STR("Failed to communicate")); \ return; \ } void AXS15231Touchscreen::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (this->reset_pin_ != nullptr) { this->reset_pin_->setup(); this->reset_pin_->digital_write(false); @@ -36,14 +35,13 @@ void AXS15231Touchscreen::setup() { if (this->y_raw_max_ == 0) { this->y_raw_max_ = this->display_->get_native_height(); } - ESP_LOGCONFIG(TAG, "AXS15231 Touchscreen setup complete"); } void AXS15231Touchscreen::update_touches() { i2c::ErrorCode err; uint8_t data[8]{}; - err = this->write(AXS_READ_TOUCHPAD, sizeof(AXS_READ_TOUCHPAD), false); + err = this->write(AXS_READ_TOUCHPAD, sizeof(AXS_READ_TOUCHPAD)); ERROR_CHECK(err); err = this->read(data, sizeof(data)); ERROR_CHECK(err); diff --git a/esphome/components/bedjet/bedjet_hub.cpp b/esphome/components/bedjet/bedjet_hub.cpp index 007ca1ca7d..38fcf29b3b 100644 --- a/esphome/components/bedjet/bedjet_hub.cpp +++ b/esphome/components/bedjet/bedjet_hub.cpp @@ -493,7 +493,7 @@ void BedJetHub::dump_config() { " ble_client.app_id: %d\n" " ble_client.conn_id: %d", this->get_name().c_str(), this->parent()->app_id, this->parent()->get_conn_id()); - LOG_UPDATE_INTERVAL(this) + LOG_UPDATE_INTERVAL(this); ESP_LOGCONFIG(TAG, " Child components (%d):", this->children_.size()); for (auto *child : this->children_) { ESP_LOGCONFIG(TAG, " - %s", child->describe().c_str()); diff --git a/esphome/components/beken_spi_led_strip/led_strip.cpp b/esphome/components/beken_spi_led_strip/led_strip.cpp index 17b2dd1808..67b8472257 100644 --- a/esphome/components/beken_spi_led_strip/led_strip.cpp +++ b/esphome/components/beken_spi_led_strip/led_strip.cpp @@ -121,8 +121,6 @@ void spi_dma_tx_finish_callback(unsigned int param) { } void BekenSPILEDStripLightOutput::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - size_t buffer_size = this->get_buffer_size_(); size_t dma_buffer_size = (buffer_size * 8) + (2 * 64); diff --git a/esphome/components/bh1750/bh1750.cpp b/esphome/components/bh1750/bh1750.cpp index 267a728fdd..2fc476c17d 100644 --- a/esphome/components/bh1750/bh1750.cpp +++ b/esphome/components/bh1750/bh1750.cpp @@ -38,7 +38,6 @@ MTreg: */ void BH1750Sensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); uint8_t turn_on = BH1750_COMMAND_POWER_ON; if (this->write(&turn_on, 1) != i2c::ERROR_OK) { this->mark_failed(); diff --git a/esphome/components/bh1900nux/__init__.py b/esphome/components/bh1900nux/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/esphome/components/bh1900nux/bh1900nux.cpp b/esphome/components/bh1900nux/bh1900nux.cpp new file mode 100644 index 0000000000..96a06adaa0 --- /dev/null +++ b/esphome/components/bh1900nux/bh1900nux.cpp @@ -0,0 +1,54 @@ +#include "esphome/core/log.h" +#include "bh1900nux.h" + +namespace esphome { +namespace bh1900nux { + +static const char *const TAG = "bh1900nux.sensor"; + +// I2C Registers +static const uint8_t TEMPERATURE_REG = 0x00; +static const uint8_t CONFIG_REG = 0x01; // Not used and supported yet +static const uint8_t TEMPERATURE_LOW_REG = 0x02; // Not used and supported yet +static const uint8_t TEMPERATURE_HIGH_REG = 0x03; // Not used and supported yet +static const uint8_t SOFT_RESET_REG = 0x04; + +// I2C Command payloads +static const uint8_t SOFT_RESET_PAYLOAD = 0x01; // Soft Reset value + +static const float SENSOR_RESOLUTION = 0.0625f; // Sensor resolution per bit in degrees celsius + +void BH1900NUXSensor::setup() { + // Initialize I2C device + i2c::ErrorCode result_code = + this->write_register(SOFT_RESET_REG, &SOFT_RESET_PAYLOAD, 1); // Software Reset to check communication + if (result_code != i2c::ERROR_OK) { + this->mark_failed(ESP_LOG_MSG_COMM_FAIL); + return; + } +} + +void BH1900NUXSensor::update() { + uint8_t temperature_raw[2]; + if (this->read_register(TEMPERATURE_REG, temperature_raw, 2) != i2c::ERROR_OK) { + ESP_LOGE(TAG, ESP_LOG_MSG_COMM_FAIL); + return; + } + + // Combined raw value, unsigned and unaligned 16 bit + // Temperature is represented in just 12 bits, shift needed + int16_t raw_temperature_register_value = encode_uint16(temperature_raw[0], temperature_raw[1]); + raw_temperature_register_value >>= 4; + float temperature_value = raw_temperature_register_value * SENSOR_RESOLUTION; // Apply sensor resolution + + this->publish_state(temperature_value); +} + +void BH1900NUXSensor::dump_config() { + LOG_SENSOR("", "BH1900NUX", this); + LOG_I2C_DEVICE(this); + LOG_UPDATE_INTERVAL(this); +} + +} // namespace bh1900nux +} // namespace esphome diff --git a/esphome/components/bh1900nux/bh1900nux.h b/esphome/components/bh1900nux/bh1900nux.h new file mode 100644 index 0000000000..fd7f8848d6 --- /dev/null +++ b/esphome/components/bh1900nux/bh1900nux.h @@ -0,0 +1,18 @@ +#pragma once + +#include "esphome/core/component.h" +#include "esphome/components/sensor/sensor.h" +#include "esphome/components/i2c/i2c.h" + +namespace esphome { +namespace bh1900nux { + +class BH1900NUXSensor : public sensor::Sensor, public PollingComponent, public i2c::I2CDevice { + public: + void setup() override; + void update() override; + void dump_config() override; +}; + +} // namespace bh1900nux +} // namespace esphome diff --git a/esphome/components/bh1900nux/sensor.py b/esphome/components/bh1900nux/sensor.py new file mode 100644 index 0000000000..5e1c0395af --- /dev/null +++ b/esphome/components/bh1900nux/sensor.py @@ -0,0 +1,34 @@ +import esphome.codegen as cg +from esphome.components import i2c, sensor +import esphome.config_validation as cv +from esphome.const import ( + DEVICE_CLASS_TEMPERATURE, + STATE_CLASS_MEASUREMENT, + UNIT_CELSIUS, +) + +DEPENDENCIES = ["i2c"] +CODEOWNERS = ["@B48D81EFCC"] + +sensor_ns = cg.esphome_ns.namespace("bh1900nux") +BH1900NUXSensor = sensor_ns.class_( + "BH1900NUXSensor", cg.PollingComponent, i2c.I2CDevice +) + +CONFIG_SCHEMA = ( + sensor.sensor_schema( + BH1900NUXSensor, + accuracy_decimals=1, + unit_of_measurement=UNIT_CELSIUS, + device_class=DEVICE_CLASS_TEMPERATURE, + state_class=STATE_CLASS_MEASUREMENT, + ) + .extend(cv.polling_component_schema("60s")) + .extend(i2c.i2c_device_schema(0x48)) +) + + +async def to_code(config): + var = await sensor.new_sensor(config) + await cg.register_component(var, config) + await i2c.register_i2c_device(var, config) diff --git a/esphome/components/binary_sensor/__init__.py b/esphome/components/binary_sensor/__init__.py index c97de6d5e5..6aa97d6e05 100644 --- a/esphome/components/binary_sensor/__init__.py +++ b/esphome/components/binary_sensor/__init__.py @@ -59,7 +59,7 @@ from esphome.const import ( DEVICE_CLASS_VIBRATION, DEVICE_CLASS_WINDOW, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass from esphome.util import Registry @@ -266,8 +266,10 @@ async def delayed_off_filter_to_code(config, filter_id): async def autorepeat_filter_to_code(config, filter_id): timings = [] if len(config) > 0: - for conf in config: - timings.append((conf[CONF_DELAY], conf[CONF_TIME_OFF], conf[CONF_TIME_ON])) + timings.extend( + (conf[CONF_DELAY], conf[CONF_TIME_OFF], conf[CONF_TIME_ON]) + for conf in config + ) else: timings.append( ( @@ -514,6 +516,7 @@ def binary_sensor_schema( icon: str = cv.UNDEFINED, entity_category: str = cv.UNDEFINED, device_class: str = cv.UNDEFINED, + filters: list = cv.UNDEFINED, ) -> cv.Schema: schema = {} @@ -525,6 +528,7 @@ def binary_sensor_schema( (CONF_ICON, icon, cv.icon), (CONF_ENTITY_CATEGORY, entity_category, cv.entity_category), (CONF_DEVICE_CLASS, device_class, validate_device_class), + (CONF_FILTERS, filters, validate_filters), ]: if default is not cv.UNDEFINED: schema[cv.Optional(key, default=default)] = validator @@ -573,16 +577,15 @@ async def setup_binary_sensor_core_(var, config): await automation.build_automation(trigger, [], conf) for conf in config.get(CONF_ON_MULTI_CLICK, []): - timings = [] - for tim in conf[CONF_TIMING]: - timings.append( - cg.StructInitializer( - MultiClickTriggerEvent, - ("state", tim[CONF_STATE]), - ("min_length", tim[CONF_MIN_LENGTH]), - ("max_length", tim.get(CONF_MAX_LENGTH, 4294967294)), - ) + timings = [ + cg.StructInitializer( + MultiClickTriggerEvent, + ("state", tim[CONF_STATE]), + ("min_length", tim[CONF_MIN_LENGTH]), + ("max_length", tim.get(CONF_MAX_LENGTH, 4294967294)), ) + for tim in conf[CONF_TIMING] + ] trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var, timings) if CONF_INVALID_COOLDOWN in conf: cg.add(trigger.set_invalid_cooldown(conf[CONF_INVALID_COOLDOWN])) @@ -649,9 +652,8 @@ async def binary_sensor_is_off_to_code(config, condition_id, template_arg, args) return cg.new_Pvariable(condition_id, template_arg, paren, False) -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_BINARY_SENSOR") cg.add_global(binary_sensor_ns.using) diff --git a/esphome/components/binary_sensor/binary_sensor.cpp b/esphome/components/binary_sensor/binary_sensor.cpp index 02b83af552..39319d3c1c 100644 --- a/esphome/components/binary_sensor/binary_sensor.cpp +++ b/esphome/components/binary_sensor/binary_sensor.cpp @@ -7,6 +7,19 @@ namespace binary_sensor { static const char *const TAG = "binary_sensor"; +// Function implementation of LOG_BINARY_SENSOR macro to reduce code size +void log_binary_sensor(const char *tag, const char *prefix, const char *type, BinarySensor *obj) { + if (obj == nullptr) { + return; + } + + ESP_LOGCONFIG(tag, "%s%s '%s'", prefix, type, obj->get_name().c_str()); + + if (!obj->get_device_class_ref().empty()) { + ESP_LOGCONFIG(tag, "%s Device Class: '%s'", prefix, obj->get_device_class_ref().c_str()); + } +} + void BinarySensor::publish_state(bool new_state) { if (this->filter_list_ == nullptr) { this->send_state_internal(new_state); diff --git a/esphome/components/binary_sensor/binary_sensor.h b/esphome/components/binary_sensor/binary_sensor.h index d61be7a49b..2bd17d97c9 100644 --- a/esphome/components/binary_sensor/binary_sensor.h +++ b/esphome/components/binary_sensor/binary_sensor.h @@ -10,13 +10,10 @@ namespace esphome { namespace binary_sensor { -#define LOG_BINARY_SENSOR(prefix, type, obj) \ - if ((obj) != nullptr) { \ - ESP_LOGCONFIG(TAG, "%s%s '%s'", prefix, LOG_STR_LITERAL(type), (obj)->get_name().c_str()); \ - if (!(obj)->get_device_class().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Device Class: '%s'", prefix, (obj)->get_device_class().c_str()); \ - } \ - } +class BinarySensor; +void log_binary_sensor(const char *tag, const char *prefix, const char *type, BinarySensor *obj); + +#define LOG_BINARY_SENSOR(prefix, type, obj) log_binary_sensor(TAG, prefix, LOG_STR_LITERAL(type), obj) #define SUB_BINARY_SENSOR(name) \ protected: \ diff --git a/esphome/components/bl0906/bl0906.cpp b/esphome/components/bl0906/bl0906.cpp index e48715010c..c1cd48a1ac 100644 --- a/esphome/components/bl0906/bl0906.cpp +++ b/esphome/components/bl0906/bl0906.cpp @@ -97,10 +97,10 @@ void BL0906::handle_actions_() { return; } ActionCallbackFuncPtr ptr_func = nullptr; - for (int i = 0; i < this->action_queue_.size(); i++) { + for (size_t i = 0; i < this->action_queue_.size(); i++) { ptr_func = this->action_queue_[i]; if (ptr_func) { - ESP_LOGI(TAG, "HandleActionCallback[%d]", i); + ESP_LOGI(TAG, "HandleActionCallback[%zu]", i); (this->*ptr_func)(); } } diff --git a/esphome/components/bl0940/__init__.py b/esphome/components/bl0940/__init__.py index 087626a4e7..066c2818b6 100644 --- a/esphome/components/bl0940/__init__.py +++ b/esphome/components/bl0940/__init__.py @@ -1 +1,6 @@ -CODEOWNERS = ["@tobias-"] +import esphome.codegen as cg + +CODEOWNERS = ["@tobias-", "@dan-s-github"] + +CONF_BL0940_ID = "bl0940_id" +bl0940_ns = cg.esphome_ns.namespace("bl0940") diff --git a/esphome/components/bl0940/bl0940.cpp b/esphome/components/bl0940/bl0940.cpp index 24990d5482..42e20eb69b 100644 --- a/esphome/components/bl0940/bl0940.cpp +++ b/esphome/components/bl0940/bl0940.cpp @@ -7,28 +7,26 @@ namespace bl0940 { static const char *const TAG = "bl0940"; -static const uint8_t BL0940_READ_COMMAND = 0x50; // 0x58 according to documentation static const uint8_t BL0940_FULL_PACKET = 0xAA; -static const uint8_t BL0940_PACKET_HEADER = 0x55; // 0x58 according to documentation +static const uint8_t BL0940_PACKET_HEADER = 0x55; // 0x58 according to en doc but 0x55 in cn doc -static const uint8_t BL0940_WRITE_COMMAND = 0xA0; // 0xA8 according to documentation static const uint8_t BL0940_REG_I_FAST_RMS_CTRL = 0x10; static const uint8_t BL0940_REG_MODE = 0x18; static const uint8_t BL0940_REG_SOFT_RESET = 0x19; static const uint8_t BL0940_REG_USR_WRPROT = 0x1A; static const uint8_t BL0940_REG_TPS_CTRL = 0x1B; -const uint8_t BL0940_INIT[5][6] = { +static const uint8_t BL0940_INIT[5][5] = { // Reset to default - {BL0940_WRITE_COMMAND, BL0940_REG_SOFT_RESET, 0x5A, 0x5A, 0x5A, 0x38}, + {BL0940_REG_SOFT_RESET, 0x5A, 0x5A, 0x5A, 0x38}, // Enable User Operation Write - {BL0940_WRITE_COMMAND, BL0940_REG_USR_WRPROT, 0x55, 0x00, 0x00, 0xF0}, + {BL0940_REG_USR_WRPROT, 0x55, 0x00, 0x00, 0xF0}, // 0x0100 = CF_UNABLE energy pulse, AC_FREQ_SEL 50Hz, RMS_UPDATE_SEL 800mS - {BL0940_WRITE_COMMAND, BL0940_REG_MODE, 0x00, 0x10, 0x00, 0x37}, + {BL0940_REG_MODE, 0x00, 0x10, 0x00, 0x37}, // 0x47FF = Over-current and leakage alarm on, Automatic temperature measurement, Interval 100mS - {BL0940_WRITE_COMMAND, BL0940_REG_TPS_CTRL, 0xFF, 0x47, 0x00, 0xFE}, + {BL0940_REG_TPS_CTRL, 0xFF, 0x47, 0x00, 0xFE}, // 0x181C = Half cycle, Fast RMS threshold 6172 - {BL0940_WRITE_COMMAND, BL0940_REG_I_FAST_RMS_CTRL, 0x1C, 0x18, 0x00, 0x1B}}; + {BL0940_REG_I_FAST_RMS_CTRL, 0x1C, 0x18, 0x00, 0x1B}}; void BL0940::loop() { DataPacket buffer; @@ -36,8 +34,8 @@ void BL0940::loop() { return; } if (read_array((uint8_t *) &buffer, sizeof(buffer))) { - if (validate_checksum(&buffer)) { - received_package_(&buffer); + if (this->validate_checksum_(&buffer)) { + this->received_package_(&buffer); } } else { ESP_LOGW(TAG, "Junk on wire. Throwing away partial message"); @@ -46,35 +44,151 @@ void BL0940::loop() { } } -bool BL0940::validate_checksum(const DataPacket *data) { - uint8_t checksum = BL0940_READ_COMMAND; +bool BL0940::validate_checksum_(DataPacket *data) { + uint8_t checksum = this->read_command_; // Whole package but checksum - for (uint32_t i = 0; i < sizeof(data->raw) - 1; i++) { - checksum += data->raw[i]; + uint8_t *raw = (uint8_t *) data; + for (uint32_t i = 0; i < sizeof(*data) - 1; i++) { + checksum += raw[i]; } checksum ^= 0xFF; if (checksum != data->checksum) { - ESP_LOGW(TAG, "BL0940 invalid checksum! 0x%02X != 0x%02X", checksum, data->checksum); + ESP_LOGW(TAG, "Invalid checksum! 0x%02X != 0x%02X", checksum, data->checksum); } return checksum == data->checksum; } void BL0940::update() { this->flush(); - this->write_byte(BL0940_READ_COMMAND); + this->write_byte(this->read_command_); this->write_byte(BL0940_FULL_PACKET); } void BL0940::setup() { +#ifdef USE_NUMBER + // add calibration callbacks + if (this->voltage_calibration_number_ != nullptr) { + this->voltage_calibration_number_->add_on_state_callback( + [this](float state) { this->voltage_calibration_callback_(state); }); + if (this->voltage_calibration_number_->has_state()) { + this->voltage_calibration_callback_(this->voltage_calibration_number_->state); + } + } + + if (this->current_calibration_number_ != nullptr) { + this->current_calibration_number_->add_on_state_callback( + [this](float state) { this->current_calibration_callback_(state); }); + if (this->current_calibration_number_->has_state()) { + this->current_calibration_callback_(this->current_calibration_number_->state); + } + } + + if (this->power_calibration_number_ != nullptr) { + this->power_calibration_number_->add_on_state_callback( + [this](float state) { this->power_calibration_callback_(state); }); + if (this->power_calibration_number_->has_state()) { + this->power_calibration_callback_(this->power_calibration_number_->state); + } + } + + if (this->energy_calibration_number_ != nullptr) { + this->energy_calibration_number_->add_on_state_callback( + [this](float state) { this->energy_calibration_callback_(state); }); + if (this->energy_calibration_number_->has_state()) { + this->energy_calibration_callback_(this->energy_calibration_number_->state); + } + } +#endif + + // calculate calibrated reference values + this->voltage_reference_cal_ = this->voltage_reference_ / this->voltage_cal_; + this->current_reference_cal_ = this->current_reference_ / this->current_cal_; + this->power_reference_cal_ = this->power_reference_ / this->power_cal_; + this->energy_reference_cal_ = this->energy_reference_ / this->energy_cal_; + for (auto *i : BL0940_INIT) { - this->write_array(i, 6); + this->write_byte(this->write_command_), this->write_array(i, 5); delay(1); } this->flush(); } -float BL0940::update_temp_(sensor::Sensor *sensor, ube16_t temperature) const { - auto tb = (float) (temperature.h << 8 | temperature.l); +float BL0940::calculate_power_reference_() { + // calculate power reference based on voltage and current reference + return this->voltage_reference_cal_ * this->current_reference_cal_ * 4046 / 324004 / 79931; +} + +float BL0940::calculate_energy_reference_() { + // formula: 3600000 * 4046 * RL * R1 * 1000 / (1638.4 * 256) / Vref² / (R1 + R2) + // or: power_reference_ * 3600000 / (1638.4 * 256) + return this->power_reference_cal_ * 3600000 / (1638.4 * 256); +} + +float BL0940::calculate_calibration_value_(float state) { return (100 + state) / 100; } + +void BL0940::reset_calibration() { +#ifdef USE_NUMBER + if (this->current_calibration_number_ != nullptr && this->current_cal_ != 1) { + this->current_calibration_number_->make_call().set_value(0).perform(); + } + if (this->voltage_calibration_number_ != nullptr && this->voltage_cal_ != 1) { + this->voltage_calibration_number_->make_call().set_value(0).perform(); + } + if (this->power_calibration_number_ != nullptr && this->power_cal_ != 1) { + this->power_calibration_number_->make_call().set_value(0).perform(); + } + if (this->energy_calibration_number_ != nullptr && this->energy_cal_ != 1) { + this->energy_calibration_number_->make_call().set_value(0).perform(); + } +#endif + ESP_LOGD(TAG, "external calibration values restored to initial state"); +} + +void BL0940::current_calibration_callback_(float state) { + this->current_cal_ = this->calculate_calibration_value_(state); + ESP_LOGV(TAG, "update current calibration state: %f", this->current_cal_); + this->recalibrate_(); +} +void BL0940::voltage_calibration_callback_(float state) { + this->voltage_cal_ = this->calculate_calibration_value_(state); + ESP_LOGV(TAG, "update voltage calibration state: %f", this->voltage_cal_); + this->recalibrate_(); +} +void BL0940::power_calibration_callback_(float state) { + this->power_cal_ = this->calculate_calibration_value_(state); + ESP_LOGV(TAG, "update power calibration state: %f", this->power_cal_); + this->recalibrate_(); +} +void BL0940::energy_calibration_callback_(float state) { + this->energy_cal_ = this->calculate_calibration_value_(state); + ESP_LOGV(TAG, "update energy calibration state: %f", this->energy_cal_); + this->recalibrate_(); +} + +void BL0940::recalibrate_() { + ESP_LOGV(TAG, "Recalibrating reference values"); + this->voltage_reference_cal_ = this->voltage_reference_ / this->voltage_cal_; + this->current_reference_cal_ = this->current_reference_ / this->current_cal_; + + if (this->voltage_cal_ != 1 || this->current_cal_ != 1) { + this->power_reference_ = this->calculate_power_reference_(); + } + this->power_reference_cal_ = this->power_reference_ / this->power_cal_; + + if (this->voltage_cal_ != 1 || this->current_cal_ != 1 || this->power_cal_ != 1) { + this->energy_reference_ = this->calculate_energy_reference_(); + } + this->energy_reference_cal_ = this->energy_reference_ / this->energy_cal_; + + ESP_LOGD(TAG, + "Recalibrated reference values:\n" + "Voltage: %f\n, Current: %f\n, Power: %f\n, Energy: %f\n", + this->voltage_reference_cal_, this->current_reference_cal_, this->power_reference_cal_, + this->energy_reference_cal_); +} + +float BL0940::update_temp_(sensor::Sensor *sensor, uint16_le_t temperature) const { + auto tb = (float) temperature; float converted_temp = ((float) 170 / 448) * (tb / 2 - 32) - 45; if (sensor != nullptr) { if (sensor->has_state() && std::abs(converted_temp - sensor->get_state()) > max_temperature_diff_) { @@ -87,33 +201,40 @@ float BL0940::update_temp_(sensor::Sensor *sensor, ube16_t temperature) const { return converted_temp; } -void BL0940::received_package_(const DataPacket *data) const { +void BL0940::received_package_(DataPacket *data) { // Bad header if (data->frame_header != BL0940_PACKET_HEADER) { ESP_LOGI(TAG, "Invalid data. Header mismatch: %d", data->frame_header); return; } - float v_rms = (float) to_uint32_t(data->v_rms) / voltage_reference_; - float i_rms = (float) to_uint32_t(data->i_rms) / current_reference_; - float watt = (float) to_int32_t(data->watt) / power_reference_; - uint32_t cf_cnt = to_uint32_t(data->cf_cnt); - float total_energy_consumption = (float) cf_cnt / energy_reference_; + // cf_cnt is only 24 bits, so track overflows + uint32_t cf_cnt = (uint24_t) data->cf_cnt; + cf_cnt |= this->prev_cf_cnt_ & 0xff000000; + if (cf_cnt < this->prev_cf_cnt_) { + cf_cnt += 0x1000000; + } + this->prev_cf_cnt_ = cf_cnt; - float tps1 = update_temp_(internal_temperature_sensor_, data->tps1); - float tps2 = update_temp_(external_temperature_sensor_, data->tps2); + float v_rms = (uint24_t) data->v_rms / this->voltage_reference_cal_; + float i_rms = (uint24_t) data->i_rms / this->current_reference_cal_; + float watt = (int24_t) data->watt / this->power_reference_cal_; + float total_energy_consumption = cf_cnt / this->energy_reference_cal_; - if (voltage_sensor_ != nullptr) { - voltage_sensor_->publish_state(v_rms); + float tps1 = update_temp_(this->internal_temperature_sensor_, data->tps1); + float tps2 = update_temp_(this->external_temperature_sensor_, data->tps2); + + if (this->voltage_sensor_ != nullptr) { + this->voltage_sensor_->publish_state(v_rms); } - if (current_sensor_ != nullptr) { - current_sensor_->publish_state(i_rms); + if (this->current_sensor_ != nullptr) { + this->current_sensor_->publish_state(i_rms); } - if (power_sensor_ != nullptr) { - power_sensor_->publish_state(watt); + if (this->power_sensor_ != nullptr) { + this->power_sensor_->publish_state(watt); } - if (energy_sensor_ != nullptr) { - energy_sensor_->publish_state(total_energy_consumption); + if (this->energy_sensor_ != nullptr) { + this->energy_sensor_->publish_state(total_energy_consumption); } ESP_LOGV(TAG, "BL0940: U %fV, I %fA, P %fW, Cnt %" PRId32 ", ∫P %fkWh, T1 %f°C, T2 %f°C", v_rms, i_rms, watt, cf_cnt, @@ -121,7 +242,27 @@ void BL0940::received_package_(const DataPacket *data) const { } void BL0940::dump_config() { // NOLINT(readability-function-cognitive-complexity) - ESP_LOGCONFIG(TAG, "BL0940:"); + ESP_LOGCONFIG(TAG, + "BL0940:\n" + " LEGACY MODE: %s\n" + " READ CMD: 0x%02X\n" + " WRITE CMD: 0x%02X\n" + " ------------------\n" + " Current reference: %f\n" + " Energy reference: %f\n" + " Power reference: %f\n" + " Voltage reference: %f\n", + TRUEFALSE(this->legacy_mode_enabled_), this->read_command_, this->write_command_, + this->current_reference_, this->energy_reference_, this->power_reference_, this->voltage_reference_); +#ifdef USE_NUMBER + ESP_LOGCONFIG(TAG, + "BL0940:\n" + " Current calibration: %f\n" + " Energy calibration: %f\n" + " Power calibration: %f\n" + " Voltage calibration: %f\n", + this->current_cal_, this->energy_cal_, this->power_cal_, this->voltage_cal_); +#endif LOG_SENSOR("", "Voltage", this->voltage_sensor_); LOG_SENSOR("", "Current", this->current_sensor_); LOG_SENSOR("", "Power", this->power_sensor_); @@ -130,9 +271,5 @@ void BL0940::dump_config() { // NOLINT(readability-function-cognitive-complexit LOG_SENSOR("", "External temperature", this->external_temperature_sensor_); } -uint32_t BL0940::to_uint32_t(ube24_t input) { return input.h << 16 | input.m << 8 | input.l; } - -int32_t BL0940::to_int32_t(sbe24_t input) { return input.h << 16 | input.m << 8 | input.l; } - } // namespace bl0940 } // namespace esphome diff --git a/esphome/components/bl0940/bl0940.h b/esphome/components/bl0940/bl0940.h index 2d4e7ccaac..93d54003f5 100644 --- a/esphome/components/bl0940/bl0940.h +++ b/esphome/components/bl0940/bl0940.h @@ -1,66 +1,48 @@ #pragma once #include "esphome/core/component.h" -#include "esphome/components/uart/uart.h" +#include "esphome/core/datatypes.h" +#include "esphome/core/defines.h" +#ifdef USE_BUTTON +#include "esphome/components/button/button.h" +#endif +#ifdef USE_NUMBER +#include "esphome/components/number/number.h" +#endif #include "esphome/components/sensor/sensor.h" +#include "esphome/components/uart/uart.h" namespace esphome { namespace bl0940 { -static const float BL0940_PREF = 1430; -static const float BL0940_UREF = 33000; -static const float BL0940_IREF = 275000; // 2750 from tasmota. Seems to generate values 100 times too high - -// Measured to 297J per click according to power consumption of 5 minutes -// Converted to kWh (3.6MJ per kwH). Used to be 256 * 1638.4 -static const float BL0940_EREF = 3.6e6 / 297; - -struct ube24_t { // NOLINT(readability-identifier-naming,altera-struct-pack-align) - uint8_t l; - uint8_t m; - uint8_t h; -} __attribute__((packed)); - -struct ube16_t { // NOLINT(readability-identifier-naming,altera-struct-pack-align) - uint8_t l; - uint8_t h; -} __attribute__((packed)); - -struct sbe24_t { // NOLINT(readability-identifier-naming,altera-struct-pack-align) - uint8_t l; - uint8_t m; - int8_t h; -} __attribute__((packed)); - // Caveat: All these values are big endian (low - middle - high) - -union DataPacket { // NOLINT(altera-struct-pack-align) - uint8_t raw[35]; - struct { - uint8_t frame_header; // value of 0x58 according to docs. 0x55 according to Tasmota real world tests. Reality wins. - ube24_t i_fast_rms; // 0x00 - ube24_t i_rms; // 0x04 - ube24_t RESERVED0; // reserved - ube24_t v_rms; // 0x06 - ube24_t RESERVED1; // reserved - sbe24_t watt; // 0x08 - ube24_t RESERVED2; // reserved - ube24_t cf_cnt; // 0x0A - ube24_t RESERVED3; // reserved - ube16_t tps1; // 0x0c - uint8_t RESERVED4; // value of 0x00 - ube16_t tps2; // 0x0c - uint8_t RESERVED5; // value of 0x00 - uint8_t checksum; // checksum - }; +struct DataPacket { + uint8_t frame_header; // Packet header (0x58 in EN docs, 0x55 in CN docs and Tasmota tests) + uint24_le_t i_fast_rms; // Fast RMS current + uint24_le_t i_rms; // RMS current + uint24_t RESERVED0; // Reserved + uint24_le_t v_rms; // RMS voltage + uint24_t RESERVED1; // Reserved + int24_le_t watt; // Active power (can be negative for bidirectional measurement) + uint24_t RESERVED2; // Reserved + uint24_le_t cf_cnt; // Energy pulse count + uint24_t RESERVED3; // Reserved + uint16_le_t tps1; // Internal temperature sensor 1 + uint8_t RESERVED4; // Reserved (should be 0x00) + uint16_le_t tps2; // Internal temperature sensor 2 + uint8_t RESERVED5; // Reserved (should be 0x00) + uint8_t checksum; // Packet checksum } __attribute__((packed)); class BL0940 : public PollingComponent, public uart::UARTDevice { public: + // Sensor setters void set_voltage_sensor(sensor::Sensor *voltage_sensor) { voltage_sensor_ = voltage_sensor; } void set_current_sensor(sensor::Sensor *current_sensor) { current_sensor_ = current_sensor; } void set_power_sensor(sensor::Sensor *power_sensor) { power_sensor_ = power_sensor; } void set_energy_sensor(sensor::Sensor *energy_sensor) { energy_sensor_ = energy_sensor; } + + // Temperature sensor setters void set_internal_temperature_sensor(sensor::Sensor *internal_temperature_sensor) { internal_temperature_sensor_ = internal_temperature_sensor; } @@ -68,42 +50,105 @@ class BL0940 : public PollingComponent, public uart::UARTDevice { external_temperature_sensor_ = external_temperature_sensor; } - void loop() override; + // Configuration setters + void set_legacy_mode(bool enable) { this->legacy_mode_enabled_ = enable; } + void set_read_command(uint8_t read_command) { this->read_command_ = read_command; } + void set_write_command(uint8_t write_command) { this->write_command_ = write_command; } + // Reference value setters (used for calibration and conversion) + void set_current_reference(float current_ref) { this->current_reference_ = current_ref; } + void set_energy_reference(float energy_ref) { this->energy_reference_ = energy_ref; } + void set_power_reference(float power_ref) { this->power_reference_ = power_ref; } + void set_voltage_reference(float voltage_ref) { this->voltage_reference_ = voltage_ref; } + +#ifdef USE_NUMBER + // Calibration number setters (for Home Assistant number entities) + void set_current_calibration_number(number::Number *num) { this->current_calibration_number_ = num; } + void set_voltage_calibration_number(number::Number *num) { this->voltage_calibration_number_ = num; } + void set_power_calibration_number(number::Number *num) { this->power_calibration_number_ = num; } + void set_energy_calibration_number(number::Number *num) { this->energy_calibration_number_ = num; } +#endif + +#ifdef USE_BUTTON + // Resets all calibration values to defaults (can be triggered by a button) + void reset_calibration(); +#endif + + // Core component methods + void loop() override; void update() override; void setup() override; void dump_config() override; protected: - sensor::Sensor *voltage_sensor_{nullptr}; - sensor::Sensor *current_sensor_{nullptr}; - // NB This may be negative as the circuits is seemingly able to measure - // power in both directions - sensor::Sensor *power_sensor_{nullptr}; - sensor::Sensor *energy_sensor_{nullptr}; - sensor::Sensor *internal_temperature_sensor_{nullptr}; - sensor::Sensor *external_temperature_sensor_{nullptr}; + // --- Sensor pointers --- + sensor::Sensor *voltage_sensor_{nullptr}; // Voltage sensor + sensor::Sensor *current_sensor_{nullptr}; // Current sensor + sensor::Sensor *power_sensor_{nullptr}; // Power sensor (can be negative for bidirectional) + sensor::Sensor *energy_sensor_{nullptr}; // Energy sensor + sensor::Sensor *internal_temperature_sensor_{nullptr}; // Internal temperature sensor + sensor::Sensor *external_temperature_sensor_{nullptr}; // External temperature sensor - // Max difference between two measurements of the temperature. Used to avoid noise. - float max_temperature_diff_{0}; - // Divide by this to turn into Watt - float power_reference_ = BL0940_PREF; - // Divide by this to turn into Volt - float voltage_reference_ = BL0940_UREF; - // Divide by this to turn into Ampere - float current_reference_ = BL0940_IREF; - // Divide by this to turn into kWh - float energy_reference_ = BL0940_EREF; +#ifdef USE_NUMBER + // --- Calibration number entities (for dynamic calibration via HA UI) --- + number::Number *voltage_calibration_number_{nullptr}; + number::Number *current_calibration_number_{nullptr}; + number::Number *power_calibration_number_{nullptr}; + number::Number *energy_calibration_number_{nullptr}; +#endif - float update_temp_(sensor::Sensor *sensor, ube16_t packed_temperature) const; + // --- Internal state --- + uint32_t prev_cf_cnt_ = 0; // Previous energy pulse count (for energy calculation) + float max_temperature_diff_{0}; // Max allowed temperature difference between two measurements (noise filter) - static uint32_t to_uint32_t(ube24_t input); + // --- Reference values for conversion --- + float power_reference_; // Divider for raw power to get Watts + float power_reference_cal_; // Calibrated power reference + float voltage_reference_; // Divider for raw voltage to get Volts + float voltage_reference_cal_; // Calibrated voltage reference + float current_reference_; // Divider for raw current to get Amperes + float current_reference_cal_; // Calibrated current reference + float energy_reference_; // Divider for raw energy to get kWh + float energy_reference_cal_; // Calibrated energy reference - static int32_t to_int32_t(sbe24_t input); + // --- Home Assistant calibration values (multipliers, default 1) --- + float current_cal_{1}; + float voltage_cal_{1}; + float power_cal_{1}; + float energy_cal_{1}; - static bool validate_checksum(const DataPacket *data); + // --- Protocol commands --- + uint8_t read_command_; + uint8_t write_command_; - void received_package_(const DataPacket *data) const; + // --- Mode flags --- + bool legacy_mode_enabled_ = true; + + // --- Methods --- + // Converts packed temperature value to float and updates the sensor + float update_temp_(sensor::Sensor *sensor, uint16_le_t packed_temperature) const; + + // Validates the checksum of a received data packet + bool validate_checksum_(DataPacket *data); + + // Handles a received data packet + void received_package_(DataPacket *data); + + // Calculates reference values for calibration and conversion + float calculate_energy_reference_(); + float calculate_power_reference_(); + float calculate_calibration_value_(float state); + + // Calibration update callbacks (used with number entities) + void current_calibration_callback_(float state); + void voltage_calibration_callback_(float state); + void power_calibration_callback_(float state); + void energy_calibration_callback_(float state); + void reset_calibration_callback_(); + + // Recalculates all reference values after calibration changes + void recalibrate_(); }; + } // namespace bl0940 } // namespace esphome diff --git a/esphome/components/bl0940/button/__init__.py b/esphome/components/bl0940/button/__init__.py new file mode 100644 index 0000000000..04d11e6e30 --- /dev/null +++ b/esphome/components/bl0940/button/__init__.py @@ -0,0 +1,27 @@ +import esphome.codegen as cg +from esphome.components import button +import esphome.config_validation as cv +from esphome.const import ENTITY_CATEGORY_CONFIG, ICON_RESTART + +from .. import CONF_BL0940_ID, bl0940_ns +from ..sensor import BL0940 + +CalibrationResetButton = bl0940_ns.class_( + "CalibrationResetButton", button.Button, cg.Component +) + +CONFIG_SCHEMA = cv.All( + button.button_schema( + CalibrationResetButton, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_RESTART, + ) + .extend({cv.GenerateID(CONF_BL0940_ID): cv.use_id(BL0940)}) + .extend(cv.COMPONENT_SCHEMA) +) + + +async def to_code(config): + var = await button.new_button(config) + await cg.register_component(var, config) + await cg.register_parented(var, config[CONF_BL0940_ID]) diff --git a/esphome/components/bl0940/button/calibration_reset_button.cpp b/esphome/components/bl0940/button/calibration_reset_button.cpp new file mode 100644 index 0000000000..79a6b872d8 --- /dev/null +++ b/esphome/components/bl0940/button/calibration_reset_button.cpp @@ -0,0 +1,20 @@ +#include "calibration_reset_button.h" +#include "../bl0940.h" +#include "esphome/core/hal.h" +#include "esphome/core/log.h" +#include "esphome/core/application.h" + +namespace esphome { +namespace bl0940 { + +static const char *const TAG = "bl0940.button.calibration_reset"; + +void CalibrationResetButton::dump_config() { LOG_BUTTON("", "Calibration Reset Button", this); } + +void CalibrationResetButton::press_action() { + ESP_LOGI(TAG, "Resetting calibration defaults..."); + this->parent_->reset_calibration(); +} + +} // namespace bl0940 +} // namespace esphome diff --git a/esphome/components/bl0940/button/calibration_reset_button.h b/esphome/components/bl0940/button/calibration_reset_button.h new file mode 100644 index 0000000000..6ea3b35cb4 --- /dev/null +++ b/esphome/components/bl0940/button/calibration_reset_button.h @@ -0,0 +1,19 @@ +#pragma once + +#include "esphome/core/component.h" +#include "esphome/components/button/button.h" + +namespace esphome { +namespace bl0940 { + +class BL0940; // Forward declaration of BL0940 class + +class CalibrationResetButton : public button::Button, public Component, public Parented { + public: + void dump_config() override; + + void press_action() override; +}; + +} // namespace bl0940 +} // namespace esphome diff --git a/esphome/components/bl0940/number/__init__.py b/esphome/components/bl0940/number/__init__.py new file mode 100644 index 0000000000..a640c2ae08 --- /dev/null +++ b/esphome/components/bl0940/number/__init__.py @@ -0,0 +1,94 @@ +import esphome.codegen as cg +from esphome.components import number +import esphome.config_validation as cv +from esphome.const import ( + CONF_MAX_VALUE, + CONF_MIN_VALUE, + CONF_MODE, + CONF_RESTORE_VALUE, + CONF_STEP, + ENTITY_CATEGORY_CONFIG, + UNIT_PERCENT, +) + +from .. import CONF_BL0940_ID, bl0940_ns +from ..sensor import BL0940 + +# Define calibration types +CONF_CURRENT_CALIBRATION = "current_calibration" +CONF_VOLTAGE_CALIBRATION = "voltage_calibration" +CONF_POWER_CALIBRATION = "power_calibration" +CONF_ENERGY_CALIBRATION = "energy_calibration" + +BL0940Number = bl0940_ns.class_("BL0940Number") + +CalibrationNumber = bl0940_ns.class_( + "CalibrationNumber", number.Number, cg.PollingComponent +) + + +def validate_min_max(config): + if config[CONF_MAX_VALUE] <= config[CONF_MIN_VALUE]: + raise cv.Invalid("max_value must be greater than min_value") + return config + + +CALIBRATION_SCHEMA = cv.All( + number.number_schema( + CalibrationNumber, + entity_category=ENTITY_CATEGORY_CONFIG, + unit_of_measurement=UNIT_PERCENT, + ) + .extend( + { + cv.Optional(CONF_MODE, default="BOX"): cv.enum(number.NUMBER_MODES), + cv.Optional(CONF_MAX_VALUE, default=10): cv.All( + cv.float_, cv.Range(max=50) + ), + cv.Optional(CONF_MIN_VALUE, default=-10): cv.All( + cv.float_, cv.Range(min=-50) + ), + cv.Optional(CONF_STEP, default=0.1): cv.positive_float, + cv.Optional(CONF_RESTORE_VALUE): cv.boolean, + } + ) + .extend(cv.COMPONENT_SCHEMA), + validate_min_max, +) + +# Configuration schema for BL0940 numbers +CONFIG_SCHEMA = cv.Schema( + { + cv.GenerateID(): cv.declare_id(BL0940Number), + cv.GenerateID(CONF_BL0940_ID): cv.use_id(BL0940), + cv.Optional(CONF_CURRENT_CALIBRATION): CALIBRATION_SCHEMA, + cv.Optional(CONF_VOLTAGE_CALIBRATION): CALIBRATION_SCHEMA, + cv.Optional(CONF_POWER_CALIBRATION): CALIBRATION_SCHEMA, + cv.Optional(CONF_ENERGY_CALIBRATION): CALIBRATION_SCHEMA, + } +) + + +async def to_code(config): + # Get the BL0940 component instance + bl0940 = await cg.get_variable(config[CONF_BL0940_ID]) + + # Process all calibration types + for cal_type, setter_method in [ + (CONF_CURRENT_CALIBRATION, "set_current_calibration_number"), + (CONF_VOLTAGE_CALIBRATION, "set_voltage_calibration_number"), + (CONF_POWER_CALIBRATION, "set_power_calibration_number"), + (CONF_ENERGY_CALIBRATION, "set_energy_calibration_number"), + ]: + if conf := config.get(cal_type): + var = await number.new_number( + conf, + min_value=conf.get(CONF_MIN_VALUE), + max_value=conf.get(CONF_MAX_VALUE), + step=conf.get(CONF_STEP), + ) + await cg.register_component(var, conf) + + if restore_value := config.get(CONF_RESTORE_VALUE): + cg.add(var.set_restore_value(restore_value)) + cg.add(getattr(bl0940, setter_method)(var)) diff --git a/esphome/components/bl0940/number/calibration_number.cpp b/esphome/components/bl0940/number/calibration_number.cpp new file mode 100644 index 0000000000..cdb26cd298 --- /dev/null +++ b/esphome/components/bl0940/number/calibration_number.cpp @@ -0,0 +1,29 @@ +#include "calibration_number.h" +#include "esphome/core/log.h" + +namespace esphome { +namespace bl0940 { + +static const char *const TAG = "bl0940.number"; + +void CalibrationNumber::setup() { + float value = 0.0f; + if (this->restore_value_) { + this->pref_ = global_preferences->make_preference(this->get_object_id_hash()); + if (!this->pref_.load(&value)) { + value = 0.0f; + } + } + this->publish_state(value); +} + +void CalibrationNumber::control(float value) { + this->publish_state(value); + if (this->restore_value_) + this->pref_.save(&value); +} + +void CalibrationNumber::dump_config() { LOG_NUMBER("", "Calibration Number", this); } + +} // namespace bl0940 +} // namespace esphome diff --git a/esphome/components/bl0940/number/calibration_number.h b/esphome/components/bl0940/number/calibration_number.h new file mode 100644 index 0000000000..3a19e36dc9 --- /dev/null +++ b/esphome/components/bl0940/number/calibration_number.h @@ -0,0 +1,26 @@ +#pragma once + +#include "esphome/components/number/number.h" +#include "esphome/core/component.h" +#include "esphome/core/preferences.h" + +namespace esphome { +namespace bl0940 { + +class CalibrationNumber : public number::Number, public Component { + public: + void setup() override; + void dump_config() override; + float get_setup_priority() const override { return setup_priority::HARDWARE; } + + void set_restore_value(bool restore_value) { this->restore_value_ = restore_value; } + + protected: + void control(float value) override; + bool restore_value_{true}; + + ESPPreferenceObject pref_; +}; + +} // namespace bl0940 +} // namespace esphome diff --git a/esphome/components/bl0940/sensor.py b/esphome/components/bl0940/sensor.py index f49e961f0a..d2e0ea435d 100644 --- a/esphome/components/bl0940/sensor.py +++ b/esphome/components/bl0940/sensor.py @@ -8,6 +8,7 @@ from esphome.const import ( CONF_ID, CONF_INTERNAL_TEMPERATURE, CONF_POWER, + CONF_REFERENCE_VOLTAGE, CONF_VOLTAGE, DEVICE_CLASS_CURRENT, DEVICE_CLASS_ENERGY, @@ -23,12 +24,133 @@ from esphome.const import ( UNIT_WATT, ) +from . import bl0940_ns + DEPENDENCIES = ["uart"] - -bl0940_ns = cg.esphome_ns.namespace("bl0940") BL0940 = bl0940_ns.class_("BL0940", cg.PollingComponent, uart.UARTDevice) +CONF_LEGACY_MODE = "legacy_mode" + +CONF_READ_COMMAND = "read_command" +CONF_WRITE_COMMAND = "write_command" + +CONF_RESISTOR_SHUNT = "resistor_shunt" +CONF_RESISTOR_ONE = "resistor_one" +CONF_RESISTOR_TWO = "resistor_two" + +CONF_CURRENT_REFERENCE = "current_reference" +CONF_ENERGY_REFERENCE = "energy_reference" +CONF_POWER_REFERENCE = "power_reference" +CONF_VOLTAGE_REFERENCE = "voltage_reference" + +DEFAULT_BL0940_READ_COMMAND = 0x58 +DEFAULT_BL0940_WRITE_COMMAND = 0xA1 + +# Values according to BL0940 application note: +# https://www.belling.com.cn/media/file_object/bel_product/BL0940/guide/BL0940_APPNote_TSSOP14_V1.04_EN.pdf +DEFAULT_BL0940_VREF = 1.218 # Vref = 1.218 +DEFAULT_BL0940_RL = 1 # RL = 1 mΩ +DEFAULT_BL0940_R1 = 0.51 # R1 = 0.51 kΩ +DEFAULT_BL0940_R2 = 1950 # R2 = 5 x 390 kΩ -> 1950 kΩ + +# ---------------------------------------------------- +# values from initial implementation +DEFAULT_BL0940_LEGACY_READ_COMMAND = 0x50 +DEFAULT_BL0940_LEGACY_WRITE_COMMAND = 0xA0 + +DEFAULT_BL0940_LEGACY_UREF = 33000 +DEFAULT_BL0940_LEGACY_IREF = 275000 +DEFAULT_BL0940_LEGACY_PREF = 1430 +# Measured to 297J per click according to power consumption of 5 minutes +# Converted to kWh (3.6MJ per kwH). Used to be 256 * 1638.4 +DEFAULT_BL0940_LEGACY_EREF = 3.6e6 / 297 +# ---------------------------------------------------- + + +# methods to calculate voltage and current reference values +def calculate_voltage_reference(vref, r_one, r_two): + # formula: 79931 / Vref * (R1 * 1000) / (R1 + R2) + return 79931 / vref * (r_one * 1000) / (r_one + r_two) + + +def calculate_current_reference(vref, r_shunt): + # formula: 324004 * RL / Vref + return 324004 * r_shunt / vref + + +def calculate_power_reference(voltage_reference, current_reference): + # calculate power reference based on voltage and current reference + return voltage_reference * current_reference * 4046 / 324004 / 79931 + + +def calculate_energy_reference(power_reference): + # formula: power_reference * 3600000 / (1638.4 * 256) + return power_reference * 3600000 / (1638.4 * 256) + + +def validate_legacy_mode(config): + # Only allow schematic calibration options if legacy_mode is False + if config.get(CONF_LEGACY_MODE, True): + forbidden = [ + CONF_REFERENCE_VOLTAGE, + CONF_RESISTOR_SHUNT, + CONF_RESISTOR_ONE, + CONF_RESISTOR_TWO, + ] + for key in forbidden: + if key in config: + raise cv.Invalid( + f"Option '{key}' is only allowed when legacy_mode: false" + ) + return config + + +def set_command_defaults(config): + # Set defaults for read_command and write_command based on legacy_mode + legacy = config.get(CONF_LEGACY_MODE, True) + if legacy: + config.setdefault(CONF_READ_COMMAND, DEFAULT_BL0940_LEGACY_READ_COMMAND) + config.setdefault(CONF_WRITE_COMMAND, DEFAULT_BL0940_LEGACY_WRITE_COMMAND) + else: + config.setdefault(CONF_READ_COMMAND, DEFAULT_BL0940_READ_COMMAND) + config.setdefault(CONF_WRITE_COMMAND, DEFAULT_BL0940_WRITE_COMMAND) + return config + + +def set_reference_values(config): + # Set default reference values based on legacy_mode + if config.get(CONF_LEGACY_MODE, True): + config.setdefault(CONF_VOLTAGE_REFERENCE, DEFAULT_BL0940_LEGACY_UREF) + config.setdefault(CONF_CURRENT_REFERENCE, DEFAULT_BL0940_LEGACY_IREF) + config.setdefault(CONF_POWER_REFERENCE, DEFAULT_BL0940_LEGACY_PREF) + config.setdefault(CONF_ENERGY_REFERENCE, DEFAULT_BL0940_LEGACY_PREF) + else: + vref = config.get(CONF_VOLTAGE_REFERENCE, DEFAULT_BL0940_VREF) + r_one = config.get(CONF_RESISTOR_ONE, DEFAULT_BL0940_R1) + r_two = config.get(CONF_RESISTOR_TWO, DEFAULT_BL0940_R2) + r_shunt = config.get(CONF_RESISTOR_SHUNT, DEFAULT_BL0940_RL) + + config.setdefault( + CONF_VOLTAGE_REFERENCE, calculate_voltage_reference(vref, r_one, r_two) + ) + config.setdefault( + CONF_CURRENT_REFERENCE, calculate_current_reference(vref, r_shunt) + ) + config.setdefault( + CONF_POWER_REFERENCE, + calculate_power_reference( + config.get(CONF_VOLTAGE_REFERENCE), config.get(CONF_CURRENT_REFERENCE) + ), + ) + config.setdefault( + CONF_ENERGY_REFERENCE, + calculate_energy_reference(config.get(CONF_POWER_REFERENCE)), + ) + + return config + + CONFIG_SCHEMA = ( cv.Schema( { @@ -69,10 +191,24 @@ CONFIG_SCHEMA = ( device_class=DEVICE_CLASS_TEMPERATURE, state_class=STATE_CLASS_MEASUREMENT, ), + cv.Optional(CONF_LEGACY_MODE, default=True): cv.boolean, + cv.Optional(CONF_READ_COMMAND): cv.hex_uint8_t, + cv.Optional(CONF_WRITE_COMMAND): cv.hex_uint8_t, + cv.Optional(CONF_REFERENCE_VOLTAGE): cv.float_, + cv.Optional(CONF_RESISTOR_SHUNT): cv.float_, + cv.Optional(CONF_RESISTOR_ONE): cv.float_, + cv.Optional(CONF_RESISTOR_TWO): cv.float_, + cv.Optional(CONF_CURRENT_REFERENCE): cv.float_, + cv.Optional(CONF_ENERGY_REFERENCE): cv.float_, + cv.Optional(CONF_POWER_REFERENCE): cv.float_, + cv.Optional(CONF_VOLTAGE_REFERENCE): cv.float_, } ) .extend(cv.polling_component_schema("60s")) .extend(uart.UART_DEVICE_SCHEMA) + .add_extra(validate_legacy_mode) + .add_extra(set_command_defaults) + .add_extra(set_reference_values) ) @@ -99,3 +235,16 @@ async def to_code(config): if external_temperature_config := config.get(CONF_EXTERNAL_TEMPERATURE): sens = await sensor.new_sensor(external_temperature_config) cg.add(var.set_external_temperature_sensor(sens)) + + # enable legacy mode + cg.add(var.set_legacy_mode(config.get(CONF_LEGACY_MODE))) + + # Set bl0940 commands after validator has determined which defaults to use if not set + cg.add(var.set_read_command(config.get(CONF_READ_COMMAND))) + cg.add(var.set_write_command(config.get(CONF_WRITE_COMMAND))) + + # Set reference values after validator has set the values either from defaults or calculated + cg.add(var.set_current_reference(config.get(CONF_CURRENT_REFERENCE))) + cg.add(var.set_voltage_reference(config.get(CONF_VOLTAGE_REFERENCE))) + cg.add(var.set_power_reference(config.get(CONF_POWER_REFERENCE))) + cg.add(var.set_energy_reference(config.get(CONF_ENERGY_REFERENCE))) diff --git a/esphome/components/bl0942/bl0942.cpp b/esphome/components/bl0942/bl0942.cpp index 86eff57147..95dd689b07 100644 --- a/esphome/components/bl0942/bl0942.cpp +++ b/esphome/components/bl0942/bl0942.cpp @@ -51,7 +51,7 @@ void BL0942::loop() { if (!avail) { return; } - if (avail < sizeof(buffer)) { + if (static_cast(avail) < sizeof(buffer)) { if (!this->rx_start_) { this->rx_start_ = millis(); } else if (millis() > this->rx_start_ + PKT_TIMEOUT_MS) { @@ -148,8 +148,8 @@ void BL0942::setup() { this->write_reg_(BL0942_REG_USR_WRPROT, 0); - if (this->read_reg_(BL0942_REG_MODE) != mode) - this->status_set_warning("BL0942 setup failed!"); + if (static_cast(this->read_reg_(BL0942_REG_MODE)) != mode) + this->status_set_warning(LOG_STR("BL0942 setup failed!")); this->flush(); } diff --git a/esphome/components/ble_client/__init__.py b/esphome/components/ble_client/__init__.py index a88172ca87..768a345213 100644 --- a/esphome/components/ble_client/__init__.py +++ b/esphome/components/ble_client/__init__.py @@ -116,7 +116,7 @@ CONFIG_SCHEMA = cv.All( ) .extend(cv.COMPONENT_SCHEMA) .extend(esp32_ble_tracker.ESP_BLE_DEVICE_SCHEMA), - esp32_ble_tracker.consume_connection_slots(1, "ble_client"), + esp32_ble.consume_connection_slots(1, "ble_client"), ) CONF_BLE_CLIENT_ID = "ble_client_id" @@ -175,8 +175,7 @@ BLE_REMOVE_BOND_ACTION_SCHEMA = cv.Schema( ) async def ble_disconnect_to_code(config, action_id, template_arg, args): parent = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, parent) - return var + return cg.new_Pvariable(action_id, template_arg, parent) @automation.register_action( @@ -184,8 +183,7 @@ async def ble_disconnect_to_code(config, action_id, template_arg, args): ) async def ble_connect_to_code(config, action_id, template_arg, args): parent = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, parent) - return var + return cg.new_Pvariable(action_id, template_arg, parent) @automation.register_action( @@ -282,14 +280,13 @@ async def passkey_reply_to_code(config, action_id, template_arg, args): ) async def remove_bond_to_code(config, action_id, template_arg, args): parent = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, parent) - - return var + return cg.new_Pvariable(action_id, template_arg, parent) async def to_code(config): # Register the loggers this component needs esp32_ble.register_bt_logger(BTLoggers.GATT, BTLoggers.SMP) + cg.add_define("USE_ESP32_BLE_UUID") var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) diff --git a/esphome/components/ble_client/output/__init__.py b/esphome/components/ble_client/output/__init__.py index 729885eb8b..22a6b29442 100644 --- a/esphome/components/ble_client/output/__init__.py +++ b/esphome/components/ble_client/output/__init__.py @@ -27,7 +27,7 @@ CONFIG_SCHEMA = cv.All( ) -def to_code(config): +async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) if len(config[CONF_SERVICE_UUID]) == len(esp32_ble_tracker.bt_uuid16_format): cg.add( @@ -63,6 +63,6 @@ def to_code(config): ) cg.add(var.set_char_uuid128(uuid128)) cg.add(var.set_require_response(config[CONF_REQUIRE_RESPONSE])) - yield output.register_output(var, config) - yield ble_client.register_ble_node(var, config) - yield cg.register_component(var, config) + await output.register_output(var, config) + await ble_client.register_ble_node(var, config) + await cg.register_component(var, config) diff --git a/esphome/components/ble_nus/__init__.py b/esphome/components/ble_nus/__init__.py new file mode 100644 index 0000000000..9570005902 --- /dev/null +++ b/esphome/components/ble_nus/__init__.py @@ -0,0 +1,29 @@ +import esphome.codegen as cg +from esphome.components.zephyr import zephyr_add_prj_conf +import esphome.config_validation as cv +from esphome.const import CONF_ID, CONF_LOGS, CONF_TYPE + +AUTO_LOAD = ["zephyr_ble_server"] +CODEOWNERS = ["@tomaszduda23"] + +ble_nus_ns = cg.esphome_ns.namespace("ble_nus") +BLENUS = ble_nus_ns.class_("BLENUS", cg.Component) + +CONFIG_SCHEMA = cv.All( + cv.Schema( + { + cv.GenerateID(): cv.declare_id(BLENUS), + cv.Optional(CONF_TYPE, default=CONF_LOGS): cv.one_of( + *[CONF_LOGS], lower=True + ), + } + ).extend(cv.COMPONENT_SCHEMA), + cv.only_with_framework("zephyr"), +) + + +async def to_code(config): + var = cg.new_Pvariable(config[CONF_ID]) + zephyr_add_prj_conf("BT_NUS", True) + cg.add(var.set_expose_log(config[CONF_TYPE] == CONF_LOGS)) + await cg.register_component(var, config) diff --git a/esphome/components/ble_nus/ble_nus.cpp b/esphome/components/ble_nus/ble_nus.cpp new file mode 100644 index 0000000000..9c4d0a3938 --- /dev/null +++ b/esphome/components/ble_nus/ble_nus.cpp @@ -0,0 +1,157 @@ +#ifdef USE_ZEPHYR +#include "ble_nus.h" +#include +#include +#include "esphome/core/log.h" +#ifdef USE_LOGGER +#include "esphome/components/logger/logger.h" +#include "esphome/core/application.h" +#endif +#include + +namespace esphome::ble_nus { + +constexpr size_t BLE_TX_BUF_SIZE = 2048; + +// NOLINTBEGIN(cppcoreguidelines-avoid-non-const-global-variables) +BLENUS *global_ble_nus; +RING_BUF_DECLARE(global_ble_tx_ring_buf, BLE_TX_BUF_SIZE); +// NOLINTEND(cppcoreguidelines-avoid-non-const-global-variables) + +static const char *const TAG = "ble_nus"; + +size_t BLENUS::write_array(const uint8_t *data, size_t len) { + if (atomic_get(&this->tx_status_) == TX_DISABLED) { + return 0; + } + return ring_buf_put(&global_ble_tx_ring_buf, data, len); +} + +void BLENUS::connected(bt_conn *conn, uint8_t err) { + if (err == 0) { + global_ble_nus->conn_.store(bt_conn_ref(conn)); + } +} + +void BLENUS::disconnected(bt_conn *conn, uint8_t reason) { + if (global_ble_nus->conn_) { + bt_conn_unref(global_ble_nus->conn_.load()); + // Connection array is global static. + // Reference can be kept even if disconnected. + } +} + +void BLENUS::tx_callback(bt_conn *conn) { + atomic_cas(&global_ble_nus->tx_status_, TX_BUSY, TX_ENABLED); + ESP_LOGVV(TAG, "Sent operation completed"); +} + +void BLENUS::send_enabled_callback(bt_nus_send_status status) { + switch (status) { + case BT_NUS_SEND_STATUS_ENABLED: + atomic_set(&global_ble_nus->tx_status_, TX_ENABLED); +#ifdef USE_LOGGER + if (global_ble_nus->expose_log_) { + App.schedule_dump_config(); + } +#endif + ESP_LOGD(TAG, "NUS notification has been enabled"); + break; + case BT_NUS_SEND_STATUS_DISABLED: + atomic_set(&global_ble_nus->tx_status_, TX_DISABLED); + ESP_LOGD(TAG, "NUS notification has been disabled"); + break; + } +} + +void BLENUS::rx_callback(bt_conn *conn, const uint8_t *const data, uint16_t len) { + ESP_LOGD(TAG, "Received %d bytes.", len); +} + +void BLENUS::setup() { + bt_nus_cb callbacks = { + .received = rx_callback, + .sent = tx_callback, + .send_enabled = send_enabled_callback, + }; + + bt_nus_init(&callbacks); + + static bt_conn_cb conn_callbacks = { + .connected = BLENUS::connected, + .disconnected = BLENUS::disconnected, + }; + + bt_conn_cb_register(&conn_callbacks); + + global_ble_nus = this; +#ifdef USE_LOGGER + if (logger::global_logger != nullptr && this->expose_log_) { + logger::global_logger->add_on_log_callback( + [this](int level, const char *tag, const char *message, size_t message_len) { + this->write_array(reinterpret_cast(message), message_len); + const char c = '\n'; + this->write_array(reinterpret_cast(&c), 1); + }); + } + +#endif +} + +void BLENUS::dump_config() { + ESP_LOGCONFIG(TAG, "ble nus:"); + ESP_LOGCONFIG(TAG, " log: %s", YESNO(this->expose_log_)); + uint32_t mtu = 0; + bt_conn *conn = this->conn_.load(); + if (conn) { + mtu = bt_nus_get_mtu(conn); + } + ESP_LOGCONFIG(TAG, " MTU: %u", mtu); +} + +void BLENUS::loop() { + if (ring_buf_is_empty(&global_ble_tx_ring_buf)) { + return; + } + + if (!atomic_cas(&this->tx_status_, TX_ENABLED, TX_BUSY)) { + if (atomic_get(&this->tx_status_) == TX_DISABLED) { + ring_buf_reset(&global_ble_tx_ring_buf); + } + return; + } + + bt_conn *conn = this->conn_.load(); + if (conn) { + conn = bt_conn_ref(conn); + } + + if (nullptr == conn) { + atomic_cas(&this->tx_status_, TX_BUSY, TX_ENABLED); + return; + } + + uint32_t req_len = bt_nus_get_mtu(conn); + + uint8_t *buf; + uint32_t size = ring_buf_get_claim(&global_ble_tx_ring_buf, &buf, req_len); + + int err, err2; + + err = bt_nus_send(conn, buf, size); + err2 = ring_buf_get_finish(&global_ble_tx_ring_buf, size); + if (err2) { + // It should no happen. + ESP_LOGE(TAG, "Size %u exceeds valid bytes in the ring buffer (%d error)", size, err2); + } + if (err == 0) { + ESP_LOGVV(TAG, "Sent %d bytes", size); + } else { + ESP_LOGE(TAG, "Failed to send %d bytes (%d error)", size, err); + atomic_cas(&this->tx_status_, TX_BUSY, TX_ENABLED); + } + bt_conn_unref(conn); +} + +} // namespace esphome::ble_nus +#endif diff --git a/esphome/components/ble_nus/ble_nus.h b/esphome/components/ble_nus/ble_nus.h new file mode 100644 index 0000000000..e8cba32b4c --- /dev/null +++ b/esphome/components/ble_nus/ble_nus.h @@ -0,0 +1,37 @@ +#pragma once +#ifdef USE_ZEPHYR +#include "esphome/core/defines.h" +#include "esphome/core/component.h" +#include +#include + +namespace esphome::ble_nus { + +class BLENUS : public Component { + enum TxStatus { + TX_DISABLED, + TX_ENABLED, + TX_BUSY, + }; + + public: + void setup() override; + void dump_config() override; + void loop() override; + size_t write_array(const uint8_t *data, size_t len); + void set_expose_log(bool expose_log) { this->expose_log_ = expose_log; } + + protected: + static void send_enabled_callback(bt_nus_send_status status); + static void tx_callback(bt_conn *conn); + static void rx_callback(bt_conn *conn, const uint8_t *data, uint16_t len); + static void connected(bt_conn *conn, uint8_t err); + static void disconnected(bt_conn *conn, uint8_t reason); + + std::atomic conn_ = nullptr; + bool expose_log_ = false; + atomic_t tx_status_ = ATOMIC_INIT(TX_DISABLED); +}; + +} // namespace esphome::ble_nus +#endif diff --git a/esphome/components/bluetooth_proxy/__init__.py b/esphome/components/bluetooth_proxy/__init__.py index 5c144cadcc..ad7528c156 100644 --- a/esphome/components/bluetooth_proxy/__init__.py +++ b/esphome/components/bluetooth_proxy/__init__.py @@ -1,3 +1,5 @@ +import logging + import esphome.codegen as cg from esphome.components import esp32_ble, esp32_ble_client, esp32_ble_tracker from esphome.components.esp32 import add_idf_sdkconfig_option @@ -7,7 +9,9 @@ from esphome.const import CONF_ACTIVE, CONF_ID AUTO_LOAD = ["esp32_ble_client", "esp32_ble_tracker"] DEPENDENCIES = ["api", "esp32"] -CODEOWNERS = ["@jesserockz"] +CODEOWNERS = ["@jesserockz", "@bdraco"] + +_LOGGER = logging.getLogger(__name__) CONF_CONNECTION_SLOTS = "connection_slots" CONF_CACHE_SERVICES = "cache_services" @@ -38,9 +42,8 @@ def validate_connections(config): ) elif config[CONF_ACTIVE]: connection_slots: int = config[CONF_CONNECTION_SLOTS] - esp32_ble_tracker.consume_connection_slots(connection_slots, "bluetooth_proxy")( - config - ) + esp32_ble.consume_connection_slots(connection_slots, "bluetooth_proxy")(config) + return { **config, CONF_CONNECTIONS: [CONNECTION_SCHEMA({}) for _ in range(connection_slots)], @@ -53,20 +56,18 @@ CONFIG_SCHEMA = cv.All( cv.Schema( { cv.GenerateID(): cv.declare_id(BluetoothProxy), - cv.Optional(CONF_ACTIVE, default=False): cv.boolean, - cv.SplitDefault(CONF_CACHE_SERVICES, esp32_idf=True): cv.All( - cv.only_with_esp_idf, cv.boolean - ), + cv.Optional(CONF_ACTIVE, default=True): cv.boolean, + cv.Optional(CONF_CACHE_SERVICES, default=True): cv.boolean, cv.Optional( CONF_CONNECTION_SLOTS, default=DEFAULT_CONNECTION_SLOTS, ): cv.All( cv.positive_int, - cv.Range(min=1, max=esp32_ble_tracker.max_connections()), + cv.Range(min=1, max=esp32_ble.IDF_MAX_CONNECTIONS), ), cv.Optional(CONF_CONNECTIONS): cv.All( cv.ensure_list(CONNECTION_SCHEMA), - cv.Length(min=1, max=esp32_ble_tracker.max_connections()), + cv.Length(min=1, max=esp32_ble.IDF_MAX_CONNECTIONS), ), } ) @@ -85,13 +86,23 @@ async def to_code(config): await cg.register_component(var, config) cg.add(var.set_active(config[CONF_ACTIVE])) - await esp32_ble_tracker.register_ble_device(var, config) + await esp32_ble_tracker.register_raw_ble_device(var, config) + + # Define max connections for protobuf fixed array + connection_count = len(config.get(CONF_CONNECTIONS, [])) + cg.add_define("BLUETOOTH_PROXY_MAX_CONNECTIONS", connection_count) + + # Define batch size for BLE advertisements + # Each advertisement is up to 80 bytes when packaged (including protocol overhead) + # 16 advertisements × 80 bytes (worst case) = 1280 bytes out of ~1320 bytes usable payload + # This achieves ~97% WiFi MTU utilization while staying under the limit + cg.add_define("BLUETOOTH_PROXY_ADVERTISEMENT_BATCH_SIZE", 16) for connection_conf in config.get(CONF_CONNECTIONS, []): connection_var = cg.new_Pvariable(connection_conf[CONF_ID]) await cg.register_component(connection_var, connection_conf) cg.add(var.register_connection(connection_var)) - await esp32_ble_tracker.register_client(connection_var, connection_conf) + await esp32_ble_tracker.register_raw_client(connection_var, connection_conf) if config.get(CONF_CACHE_SERVICES): add_idf_sdkconfig_option("CONFIG_BT_GATTC_CACHE_NVS_FLASH", True) diff --git a/esphome/components/bluetooth_proxy/bluetooth_connection.cpp b/esphome/components/bluetooth_proxy/bluetooth_connection.cpp index 44d434802c..fcc344dda9 100644 --- a/esphome/components/bluetooth_proxy/bluetooth_connection.cpp +++ b/esphome/components/bluetooth_proxy/bluetooth_connection.cpp @@ -8,16 +8,361 @@ #include "bluetooth_proxy.h" -namespace esphome { -namespace bluetooth_proxy { +namespace esphome::bluetooth_proxy { static const char *const TAG = "bluetooth_proxy.connection"; +// This function is allocation-free and directly packs UUIDs into the output array +// using precalculated constants for the Bluetooth base UUID +static void fill_128bit_uuid_array(std::array &out, esp_bt_uuid_t uuid_source) { + // Bluetooth base UUID: 00000000-0000-1000-8000-00805F9B34FB + // out[0] = bytes 8-15 (big-endian) + // - For 128-bit UUIDs: use bytes 8-15 as-is + // - For 16/32-bit UUIDs: insert into bytes 12-15, use 0x00001000 for bytes 8-11 + out[0] = uuid_source.len == ESP_UUID_LEN_128 + ? (((uint64_t) uuid_source.uuid.uuid128[15] << 56) | ((uint64_t) uuid_source.uuid.uuid128[14] << 48) | + ((uint64_t) uuid_source.uuid.uuid128[13] << 40) | ((uint64_t) uuid_source.uuid.uuid128[12] << 32) | + ((uint64_t) uuid_source.uuid.uuid128[11] << 24) | ((uint64_t) uuid_source.uuid.uuid128[10] << 16) | + ((uint64_t) uuid_source.uuid.uuid128[9] << 8) | ((uint64_t) uuid_source.uuid.uuid128[8])) + : (((uint64_t) (uuid_source.len == ESP_UUID_LEN_16 ? uuid_source.uuid.uuid16 : uuid_source.uuid.uuid32) + << 32) | + 0x00001000ULL); // Base UUID bytes 8-11 + // out[1] = bytes 0-7 (big-endian) + // - For 128-bit UUIDs: use bytes 0-7 as-is + // - For 16/32-bit UUIDs: use precalculated base UUID constant + out[1] = uuid_source.len == ESP_UUID_LEN_128 + ? ((uint64_t) uuid_source.uuid.uuid128[7] << 56) | ((uint64_t) uuid_source.uuid.uuid128[6] << 48) | + ((uint64_t) uuid_source.uuid.uuid128[5] << 40) | ((uint64_t) uuid_source.uuid.uuid128[4] << 32) | + ((uint64_t) uuid_source.uuid.uuid128[3] << 24) | ((uint64_t) uuid_source.uuid.uuid128[2] << 16) | + ((uint64_t) uuid_source.uuid.uuid128[1] << 8) | ((uint64_t) uuid_source.uuid.uuid128[0]) + : 0x800000805F9B34FBULL; // Base UUID bytes 0-7: 80-00-00-80-5F-9B-34-FB +} + +// Helper to fill UUID in the appropriate format based on client support and UUID type +static void fill_gatt_uuid(std::array &uuid_128, uint32_t &short_uuid, const esp_bt_uuid_t &uuid, + bool use_efficient_uuids) { + if (!use_efficient_uuids || uuid.len == ESP_UUID_LEN_128) { + // Use 128-bit format for old clients or when UUID is already 128-bit + fill_128bit_uuid_array(uuid_128, uuid); + } else if (uuid.len == ESP_UUID_LEN_16) { + short_uuid = uuid.uuid.uuid16; + } else if (uuid.len == ESP_UUID_LEN_32) { + short_uuid = uuid.uuid.uuid32; + } +} + +// Constants for size estimation +static constexpr uint8_t SERVICE_OVERHEAD_LEGACY = 25; // UUID(20) + handle(4) + overhead(1) +static constexpr uint8_t SERVICE_OVERHEAD_EFFICIENT = 10; // UUID(6) + handle(4) +static constexpr uint8_t CHAR_SIZE_128BIT = 35; // UUID(20) + handle(4) + props(4) + overhead(7) +static constexpr uint8_t DESC_SIZE_128BIT = 25; // UUID(20) + handle(4) + overhead(1) +static constexpr uint8_t DESC_SIZE_16BIT = 10; // UUID(6) + handle(4) +static constexpr uint8_t DESC_PER_CHAR = 1; // Assume 1 descriptor per characteristic + +// Helper to estimate service size before fetching all data +/** + * Estimate the size of a Bluetooth service based on the number of characteristics and UUID format. + * + * @param char_count The number of characteristics in the service. + * @param use_efficient_uuids Whether to use efficient UUIDs (16-bit or 32-bit) for newer APIVersions. + * @return The estimated size of the service in bytes. + * + * This function calculates the size of a Bluetooth service by considering: + * - A service overhead, which depends on whether efficient UUIDs are used. + * - The size of each characteristic, assuming 128-bit UUIDs for safety. + * - The size of descriptors, assuming one 128-bit descriptor per characteristic. + */ +static size_t estimate_service_size(uint16_t char_count, bool use_efficient_uuids) { + size_t service_overhead = use_efficient_uuids ? SERVICE_OVERHEAD_EFFICIENT : SERVICE_OVERHEAD_LEGACY; + // Always assume 128-bit UUIDs for characteristics to be safe + size_t char_size = CHAR_SIZE_128BIT; + // Assume one 128-bit descriptor per characteristic + size_t desc_size = DESC_SIZE_128BIT * DESC_PER_CHAR; + + return service_overhead + (char_size + desc_size) * char_count; +} + +bool BluetoothConnection::supports_efficient_uuids_() const { + auto *api_conn = this->proxy_->get_api_connection(); + return api_conn && api_conn->client_supports_api_version(1, 12); +} + void BluetoothConnection::dump_config() { ESP_LOGCONFIG(TAG, "BLE Connection:"); BLEClientBase::dump_config(); } +void BluetoothConnection::update_allocated_slot_(uint64_t find_value, uint64_t set_value) { + auto &allocated = this->proxy_->connections_free_response_.allocated; + for (auto &slot : allocated) { + if (slot == find_value) { + slot = set_value; + return; + } + } +} + +void BluetoothConnection::set_address(uint64_t address) { + // If we're clearing an address (disconnecting), update the pre-allocated message + if (address == 0 && this->address_ != 0) { + this->proxy_->connections_free_response_.free++; + this->update_allocated_slot_(this->address_, 0); + } + // If we're setting a new address (connecting), update the pre-allocated message + else if (address != 0 && this->address_ == 0) { + this->proxy_->connections_free_response_.free--; + this->update_allocated_slot_(0, address); + } + + // Call parent implementation to actually set the address + BLEClientBase::set_address(address); +} + +void BluetoothConnection::loop() { + BLEClientBase::loop(); + + // Early return if no active connection + if (this->address_ == 0) { + return; + } + + // Handle service discovery if in valid range + if (this->send_service_ >= 0 && this->send_service_ <= this->service_count_) { + this->send_service_for_discovery_(); + } + + // Check if we should disable the loop + // - For V3_WITH_CACHE: Services are never sent, disable after INIT state + // - For V3_WITHOUT_CACHE: Disable only after service discovery is complete + // (send_service_ == DONE_SENDING_SERVICES, which is only set after services are sent) + if (this->state_ != espbt::ClientState::INIT && (this->connection_type_ == espbt::ConnectionType::V3_WITH_CACHE || + this->send_service_ == DONE_SENDING_SERVICES)) { + this->disable_loop(); + } +} + +void BluetoothConnection::reset_connection_(esp_err_t reason) { + // Send disconnection notification + this->proxy_->send_device_connection(this->address_, false, 0, reason); + + // Important: If we were in the middle of sending services, we do NOT send + // send_gatt_services_done() here. This ensures the client knows that + // the service discovery was interrupted and can retry. The client + // (aioesphomeapi) implements a 30-second timeout (DEFAULT_BLE_TIMEOUT) + // to detect incomplete service discovery rather than relying on us to + // tell them about a partial list. + this->set_address(0); + this->send_service_ = INIT_SENDING_SERVICES; + this->proxy_->send_connections_free(); +} + +void BluetoothConnection::send_service_for_discovery_() { + if (this->send_service_ >= this->service_count_) { + this->send_service_ = DONE_SENDING_SERVICES; + this->proxy_->send_gatt_services_done(this->address_); + this->release_services(); + return; + } + + // Early return if no API connection + auto *api_conn = this->proxy_->get_api_connection(); + if (api_conn == nullptr) { + this->send_service_ = DONE_SENDING_SERVICES; + return; + } + + // Check if client supports efficient UUIDs + bool use_efficient_uuids = this->supports_efficient_uuids_(); + + // Prepare response + api::BluetoothGATTGetServicesResponse resp; + resp.address = this->address_; + + // Dynamic batching based on actual size + // Conservative MTU limit for API messages (accounts for WPA3 overhead) + static constexpr size_t MAX_PACKET_SIZE = 1360; + + // Keep running total of actual message size + size_t current_size = 0; + api::ProtoSize size; + resp.calculate_size(size); + current_size = size.get_size(); + + while (this->send_service_ < this->service_count_) { + esp_gattc_service_elem_t service_result; + uint16_t service_count = 1; + esp_gatt_status_t service_status = esp_ble_gattc_get_service(this->gattc_if_, this->conn_id_, nullptr, + &service_result, &service_count, this->send_service_); + + if (service_status != ESP_GATT_OK || service_count == 0) { + ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_get_service %s, status=%d, service_count=%d, offset=%d", + this->connection_index_, this->address_str().c_str(), + service_status != ESP_GATT_OK ? "error" : "missing", service_status, service_count, this->send_service_); + this->send_service_ = DONE_SENDING_SERVICES; + return; + } + + // Get the number of characteristics BEFORE adding to response + uint16_t total_char_count = 0; + esp_gatt_status_t char_count_status = + esp_ble_gattc_get_attr_count(this->gattc_if_, this->conn_id_, ESP_GATT_DB_CHARACTERISTIC, + service_result.start_handle, service_result.end_handle, 0, &total_char_count); + + if (char_count_status != ESP_GATT_OK) { + this->log_connection_error_("esp_ble_gattc_get_attr_count", char_count_status); + this->send_service_ = DONE_SENDING_SERVICES; + return; + } + + // If this service likely won't fit, send current batch (unless it's the first) + size_t estimated_size = estimate_service_size(total_char_count, use_efficient_uuids); + if (!resp.services.empty() && (current_size + estimated_size > MAX_PACKET_SIZE)) { + // This service likely won't fit, send current batch + break; + } + + // Now add the service since we know it will likely fit + resp.services.emplace_back(); + auto &service_resp = resp.services.back(); + + fill_gatt_uuid(service_resp.uuid, service_resp.short_uuid, service_result.uuid, use_efficient_uuids); + + service_resp.handle = service_result.start_handle; + + if (total_char_count > 0) { + // Initialize FixedVector with exact count and process characteristics + service_resp.characteristics.init(total_char_count); + uint16_t char_offset = 0; + esp_gattc_char_elem_t char_result; + while (true) { // characteristics + uint16_t char_count = 1; + esp_gatt_status_t char_status = + esp_ble_gattc_get_all_char(this->gattc_if_, this->conn_id_, service_result.start_handle, + service_result.end_handle, &char_result, &char_count, char_offset); + if (char_status == ESP_GATT_INVALID_OFFSET || char_status == ESP_GATT_NOT_FOUND) { + break; + } + if (char_status != ESP_GATT_OK) { + this->log_connection_error_("esp_ble_gattc_get_all_char", char_status); + this->send_service_ = DONE_SENDING_SERVICES; + return; + } + if (char_count == 0) { + break; + } + + service_resp.characteristics.emplace_back(); + auto &characteristic_resp = service_resp.characteristics.back(); + fill_gatt_uuid(characteristic_resp.uuid, characteristic_resp.short_uuid, char_result.uuid, use_efficient_uuids); + characteristic_resp.handle = char_result.char_handle; + characteristic_resp.properties = char_result.properties; + char_offset++; + + // Get the number of descriptors directly with one call + uint16_t total_desc_count = 0; + esp_gatt_status_t desc_count_status = esp_ble_gattc_get_attr_count( + this->gattc_if_, this->conn_id_, ESP_GATT_DB_DESCRIPTOR, 0, 0, char_result.char_handle, &total_desc_count); + + if (desc_count_status != ESP_GATT_OK) { + this->log_connection_error_("esp_ble_gattc_get_attr_count", desc_count_status); + this->send_service_ = DONE_SENDING_SERVICES; + return; + } + if (total_desc_count == 0) { + continue; + } + + // Initialize FixedVector with exact count and process descriptors + characteristic_resp.descriptors.init(total_desc_count); + uint16_t desc_offset = 0; + esp_gattc_descr_elem_t desc_result; + while (true) { // descriptors + uint16_t desc_count = 1; + esp_gatt_status_t desc_status = esp_ble_gattc_get_all_descr( + this->gattc_if_, this->conn_id_, char_result.char_handle, &desc_result, &desc_count, desc_offset); + if (desc_status == ESP_GATT_INVALID_OFFSET || desc_status == ESP_GATT_NOT_FOUND) { + break; + } + if (desc_status != ESP_GATT_OK) { + this->log_connection_error_("esp_ble_gattc_get_all_descr", desc_status); + this->send_service_ = DONE_SENDING_SERVICES; + return; + } + if (desc_count == 0) { + break; // No more descriptors + } + + characteristic_resp.descriptors.emplace_back(); + auto &descriptor_resp = characteristic_resp.descriptors.back(); + fill_gatt_uuid(descriptor_resp.uuid, descriptor_resp.short_uuid, desc_result.uuid, use_efficient_uuids); + descriptor_resp.handle = desc_result.handle; + desc_offset++; + } + } + } // end if (total_char_count > 0) + + // Calculate the actual size of just this service + api::ProtoSize service_sizer; + service_resp.calculate_size(service_sizer); + size_t service_size = service_sizer.get_size() + 1; // +1 for field tag + + // Check if adding this service would exceed the limit + if (current_size + service_size > MAX_PACKET_SIZE) { + // We would go over - pop the last service if we have more than one + if (resp.services.size() > 1) { + resp.services.pop_back(); + ESP_LOGD(TAG, "[%d] [%s] Service %d would exceed limit (current: %d + service: %d > %d), sending current batch", + this->connection_index_, this->address_str().c_str(), this->send_service_, current_size, service_size, + MAX_PACKET_SIZE); + // Don't increment send_service_ - we'll retry this service in next batch + } else { + // This single service is too large, but we have to send it anyway + ESP_LOGV(TAG, "[%d] [%s] Service %d is too large (%d bytes) but sending anyway", this->connection_index_, + this->address_str().c_str(), this->send_service_, service_size); + // Increment so we don't get stuck + this->send_service_++; + } + // Send what we have + break; + } + + // Now we know we're keeping this service, add its size + current_size += service_size; + // Successfully added this service, increment counter + this->send_service_++; + } + + // Send the message with dynamically batched services + api_conn->send_message(resp, api::BluetoothGATTGetServicesResponse::MESSAGE_TYPE); +} + +void BluetoothConnection::log_connection_error_(const char *operation, esp_gatt_status_t status) { + ESP_LOGE(TAG, "[%d] [%s] %s error, status=%d", this->connection_index_, this->address_str().c_str(), operation, + status); +} + +void BluetoothConnection::log_connection_warning_(const char *operation, esp_err_t err) { + ESP_LOGW(TAG, "[%d] [%s] %s failed, err=%d", this->connection_index_, this->address_str().c_str(), operation, err); +} + +void BluetoothConnection::log_gatt_not_connected_(const char *action, const char *type) { + ESP_LOGW(TAG, "[%d] [%s] Cannot %s GATT %s, not connected.", this->connection_index_, this->address_str().c_str(), + action, type); +} + +void BluetoothConnection::log_gatt_operation_error_(const char *operation, uint16_t handle, esp_gatt_status_t status) { + ESP_LOGW(TAG, "[%d] [%s] Error %s for handle 0x%2X, status=%d", this->connection_index_, this->address_str().c_str(), + operation, handle, status); +} + +esp_err_t BluetoothConnection::check_and_log_error_(const char *operation, esp_err_t err) { + if (err != ESP_OK) { + this->log_connection_warning_(operation, err); + return err; + } + return ESP_OK; +} + bool BluetoothConnection::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gattc_cb_param_t *param) { if (!BLEClientBase::gattc_event_handler(event, gattc_if, param)) @@ -25,22 +370,25 @@ bool BluetoothConnection::gattc_event_handler(esp_gattc_cb_event_t event, esp_ga switch (event) { case ESP_GATTC_DISCONNECT_EVT: { + // Don't reset connection yet - wait for CLOSE_EVT to ensure controller has freed resources + // This prevents race condition where we mark slot as free before controller cleanup is complete + ESP_LOGD(TAG, "[%d] [%s] Disconnect, reason=0x%02x", this->connection_index_, this->address_str_.c_str(), + param->disconnect.reason); + // Send disconnection notification but don't free the slot yet this->proxy_->send_device_connection(this->address_, false, 0, param->disconnect.reason); - this->set_address(0); - this->proxy_->send_connections_free(); break; } case ESP_GATTC_CLOSE_EVT: { - this->proxy_->send_device_connection(this->address_, false, 0, param->close.reason); - this->set_address(0); - this->proxy_->send_connections_free(); + ESP_LOGD(TAG, "[%d] [%s] Close, reason=0x%02x, freeing slot", this->connection_index_, this->address_str_.c_str(), + param->close.reason); + // Now the GATT connection is fully closed and controller resources are freed + // Safe to mark the connection slot as available + this->reset_connection_(param->close.reason); break; } case ESP_GATTC_OPEN_EVT: { if (param->open.status != ESP_GATT_OK && param->open.status != ESP_GATT_ALREADY_OPEN) { - this->proxy_->send_device_connection(this->address_, false, 0, param->open.status); - this->set_address(0); - this->proxy_->send_connections_free(); + this->reset_connection_(param->open.status); } else if (this->connection_type_ == espbt::ConnectionType::V3_WITH_CACHE) { this->proxy_->send_device_connection(this->address_, true, this->mtu_); this->proxy_->send_connections_free(); @@ -64,59 +412,54 @@ bool BluetoothConnection::gattc_event_handler(esp_gattc_cb_event_t event, esp_ga case ESP_GATTC_READ_DESCR_EVT: case ESP_GATTC_READ_CHAR_EVT: { if (param->read.status != ESP_GATT_OK) { - ESP_LOGW(TAG, "[%d] [%s] Error reading char/descriptor at handle 0x%2X, status=%d", this->connection_index_, - this->address_str_.c_str(), param->read.handle, param->read.status); + this->log_gatt_operation_error_("reading char/descriptor", param->read.handle, param->read.status); this->proxy_->send_gatt_error(this->address_, param->read.handle, param->read.status); break; } api::BluetoothGATTReadResponse resp; resp.address = this->address_; resp.handle = param->read.handle; - resp.data.reserve(param->read.value_len); - // Use bulk insert instead of individual push_backs - resp.data.insert(resp.data.end(), param->read.value, param->read.value + param->read.value_len); - this->proxy_->get_api_connection()->send_message(resp); + resp.set_data(param->read.value, param->read.value_len); + this->proxy_->get_api_connection()->send_message(resp, api::BluetoothGATTReadResponse::MESSAGE_TYPE); break; } case ESP_GATTC_WRITE_CHAR_EVT: case ESP_GATTC_WRITE_DESCR_EVT: { if (param->write.status != ESP_GATT_OK) { - ESP_LOGW(TAG, "[%d] [%s] Error writing char/descriptor at handle 0x%2X, status=%d", this->connection_index_, - this->address_str_.c_str(), param->write.handle, param->write.status); + this->log_gatt_operation_error_("writing char/descriptor", param->write.handle, param->write.status); this->proxy_->send_gatt_error(this->address_, param->write.handle, param->write.status); break; } api::BluetoothGATTWriteResponse resp; resp.address = this->address_; resp.handle = param->write.handle; - this->proxy_->get_api_connection()->send_message(resp); + this->proxy_->get_api_connection()->send_message(resp, api::BluetoothGATTWriteResponse::MESSAGE_TYPE); break; } case ESP_GATTC_UNREG_FOR_NOTIFY_EVT: { if (param->unreg_for_notify.status != ESP_GATT_OK) { - ESP_LOGW(TAG, "[%d] [%s] Error unregistering notifications for handle 0x%2X, status=%d", - this->connection_index_, this->address_str_.c_str(), param->unreg_for_notify.handle, - param->unreg_for_notify.status); + this->log_gatt_operation_error_("unregistering notifications", param->unreg_for_notify.handle, + param->unreg_for_notify.status); this->proxy_->send_gatt_error(this->address_, param->unreg_for_notify.handle, param->unreg_for_notify.status); break; } api::BluetoothGATTNotifyResponse resp; resp.address = this->address_; resp.handle = param->unreg_for_notify.handle; - this->proxy_->get_api_connection()->send_message(resp); + this->proxy_->get_api_connection()->send_message(resp, api::BluetoothGATTNotifyResponse::MESSAGE_TYPE); break; } case ESP_GATTC_REG_FOR_NOTIFY_EVT: { if (param->reg_for_notify.status != ESP_GATT_OK) { - ESP_LOGW(TAG, "[%d] [%s] Error registering notifications for handle 0x%2X, status=%d", this->connection_index_, - this->address_str_.c_str(), param->reg_for_notify.handle, param->reg_for_notify.status); + this->log_gatt_operation_error_("registering notifications", param->reg_for_notify.handle, + param->reg_for_notify.status); this->proxy_->send_gatt_error(this->address_, param->reg_for_notify.handle, param->reg_for_notify.status); break; } api::BluetoothGATTNotifyResponse resp; resp.address = this->address_; resp.handle = param->reg_for_notify.handle; - this->proxy_->get_api_connection()->send_message(resp); + this->proxy_->get_api_connection()->send_message(resp, api::BluetoothGATTNotifyResponse::MESSAGE_TYPE); break; } case ESP_GATTC_NOTIFY_EVT: { @@ -125,10 +468,8 @@ bool BluetoothConnection::gattc_event_handler(esp_gattc_cb_event_t event, esp_ga api::BluetoothGATTNotifyDataResponse resp; resp.address = this->address_; resp.handle = param->notify.handle; - resp.data.reserve(param->notify.value_len); - // Use bulk insert instead of individual push_backs - resp.data.insert(resp.data.end(), param->notify.value, param->notify.value + param->notify.value_len); - this->proxy_->get_api_connection()->send_message(resp); + resp.set_data(param->notify.value, param->notify.value_len); + this->proxy_->get_api_connection()->send_message(resp, api::BluetoothGATTNotifyDataResponse::MESSAGE_TYPE); break; } default: @@ -157,8 +498,7 @@ void BluetoothConnection::gap_event_handler(esp_gap_ble_cb_event_t event, esp_bl esp_err_t BluetoothConnection::read_characteristic(uint16_t handle) { if (!this->connected()) { - ESP_LOGW(TAG, "[%d] [%s] Cannot read GATT characteristic, not connected.", this->connection_index_, - this->address_str_.c_str()); + this->log_gatt_not_connected_("read", "characteristic"); return ESP_GATT_NOT_CONNECTED; } @@ -166,76 +506,59 @@ esp_err_t BluetoothConnection::read_characteristic(uint16_t handle) { handle); esp_err_t err = esp_ble_gattc_read_char(this->gattc_if_, this->conn_id_, handle, ESP_GATT_AUTH_REQ_NONE); - if (err != ERR_OK) { - ESP_LOGW(TAG, "[%d] [%s] esp_ble_gattc_read_char error, err=%d", this->connection_index_, - this->address_str_.c_str(), err); - return err; - } - return ESP_OK; + return this->check_and_log_error_("esp_ble_gattc_read_char", err); } -esp_err_t BluetoothConnection::write_characteristic(uint16_t handle, const std::string &data, bool response) { +esp_err_t BluetoothConnection::write_characteristic(uint16_t handle, const uint8_t *data, size_t length, + bool response) { if (!this->connected()) { - ESP_LOGW(TAG, "[%d] [%s] Cannot write GATT characteristic, not connected.", this->connection_index_, - this->address_str_.c_str()); + this->log_gatt_not_connected_("write", "characteristic"); return ESP_GATT_NOT_CONNECTED; } ESP_LOGV(TAG, "[%d] [%s] Writing GATT characteristic handle %d", this->connection_index_, this->address_str_.c_str(), handle); + // ESP-IDF's API requires a non-const uint8_t* but it doesn't modify the data + // The BTC layer immediately copies the data to its own buffer (see btc_gattc.c) + // const_cast is safe here and was previously hidden by a C-style cast esp_err_t err = - esp_ble_gattc_write_char(this->gattc_if_, this->conn_id_, handle, data.size(), (uint8_t *) data.data(), + esp_ble_gattc_write_char(this->gattc_if_, this->conn_id_, handle, length, const_cast(data), response ? ESP_GATT_WRITE_TYPE_RSP : ESP_GATT_WRITE_TYPE_NO_RSP, ESP_GATT_AUTH_REQ_NONE); - if (err != ERR_OK) { - ESP_LOGW(TAG, "[%d] [%s] esp_ble_gattc_write_char error, err=%d", this->connection_index_, - this->address_str_.c_str(), err); - return err; - } - return ESP_OK; + return this->check_and_log_error_("esp_ble_gattc_write_char", err); } esp_err_t BluetoothConnection::read_descriptor(uint16_t handle) { if (!this->connected()) { - ESP_LOGW(TAG, "[%d] [%s] Cannot read GATT descriptor, not connected.", this->connection_index_, - this->address_str_.c_str()); + this->log_gatt_not_connected_("read", "descriptor"); return ESP_GATT_NOT_CONNECTED; } ESP_LOGV(TAG, "[%d] [%s] Reading GATT descriptor handle %d", this->connection_index_, this->address_str_.c_str(), handle); esp_err_t err = esp_ble_gattc_read_char_descr(this->gattc_if_, this->conn_id_, handle, ESP_GATT_AUTH_REQ_NONE); - if (err != ERR_OK) { - ESP_LOGW(TAG, "[%d] [%s] esp_ble_gattc_read_char_descr error, err=%d", this->connection_index_, - this->address_str_.c_str(), err); - return err; - } - return ESP_OK; + return this->check_and_log_error_("esp_ble_gattc_read_char_descr", err); } -esp_err_t BluetoothConnection::write_descriptor(uint16_t handle, const std::string &data, bool response) { +esp_err_t BluetoothConnection::write_descriptor(uint16_t handle, const uint8_t *data, size_t length, bool response) { if (!this->connected()) { - ESP_LOGW(TAG, "[%d] [%s] Cannot write GATT descriptor, not connected.", this->connection_index_, - this->address_str_.c_str()); + this->log_gatt_not_connected_("write", "descriptor"); return ESP_GATT_NOT_CONNECTED; } ESP_LOGV(TAG, "[%d] [%s] Writing GATT descriptor handle %d", this->connection_index_, this->address_str_.c_str(), handle); + // ESP-IDF's API requires a non-const uint8_t* but it doesn't modify the data + // The BTC layer immediately copies the data to its own buffer (see btc_gattc.c) + // const_cast is safe here and was previously hidden by a C-style cast esp_err_t err = esp_ble_gattc_write_char_descr( - this->gattc_if_, this->conn_id_, handle, data.size(), (uint8_t *) data.data(), + this->gattc_if_, this->conn_id_, handle, length, const_cast(data), response ? ESP_GATT_WRITE_TYPE_RSP : ESP_GATT_WRITE_TYPE_NO_RSP, ESP_GATT_AUTH_REQ_NONE); - if (err != ERR_OK) { - ESP_LOGW(TAG, "[%d] [%s] esp_ble_gattc_write_char_descr error, err=%d", this->connection_index_, - this->address_str_.c_str(), err); - return err; - } - return ESP_OK; + return this->check_and_log_error_("esp_ble_gattc_write_char_descr", err); } esp_err_t BluetoothConnection::notify_characteristic(uint16_t handle, bool enable) { if (!this->connected()) { - ESP_LOGW(TAG, "[%d] [%s] Cannot notify GATT characteristic, not connected.", this->connection_index_, - this->address_str_.c_str()); + this->log_gatt_not_connected_("notify", "characteristic"); return ESP_GATT_NOT_CONNECTED; } @@ -243,29 +566,19 @@ esp_err_t BluetoothConnection::notify_characteristic(uint16_t handle, bool enabl ESP_LOGV(TAG, "[%d] [%s] Registering for GATT characteristic notifications handle %d", this->connection_index_, this->address_str_.c_str(), handle); esp_err_t err = esp_ble_gattc_register_for_notify(this->gattc_if_, this->remote_bda_, handle); - if (err != ESP_OK) { - ESP_LOGW(TAG, "[%d] [%s] esp_ble_gattc_register_for_notify failed, err=%d", this->connection_index_, - this->address_str_.c_str(), err); - return err; - } - } else { - ESP_LOGV(TAG, "[%d] [%s] Unregistering for GATT characteristic notifications handle %d", this->connection_index_, - this->address_str_.c_str(), handle); - esp_err_t err = esp_ble_gattc_unregister_for_notify(this->gattc_if_, this->remote_bda_, handle); - if (err != ESP_OK) { - ESP_LOGW(TAG, "[%d] [%s] esp_ble_gattc_unregister_for_notify failed, err=%d", this->connection_index_, - this->address_str_.c_str(), err); - return err; - } + return this->check_and_log_error_("esp_ble_gattc_register_for_notify", err); } - return ESP_OK; + + ESP_LOGV(TAG, "[%d] [%s] Unregistering for GATT characteristic notifications handle %d", this->connection_index_, + this->address_str_.c_str(), handle); + esp_err_t err = esp_ble_gattc_unregister_for_notify(this->gattc_if_, this->remote_bda_, handle); + return this->check_and_log_error_("esp_ble_gattc_unregister_for_notify", err); } esp32_ble_tracker::AdvertisementParserType BluetoothConnection::get_advertisement_parser_type() { return this->proxy_->get_advertisement_parser_type(); } -} // namespace bluetooth_proxy -} // namespace esphome +} // namespace esphome::bluetooth_proxy #endif // USE_ESP32 diff --git a/esphome/components/bluetooth_proxy/bluetooth_connection.h b/esphome/components/bluetooth_proxy/bluetooth_connection.h index 73c034d93b..60bbc93e8b 100644 --- a/esphome/components/bluetooth_proxy/bluetooth_connection.h +++ b/esphome/components/bluetooth_proxy/bluetooth_connection.h @@ -4,42 +4,53 @@ #include "esphome/components/esp32_ble_client/ble_client_base.h" -namespace esphome { -namespace bluetooth_proxy { +namespace esphome::bluetooth_proxy { class BluetoothProxy; -class BluetoothConnection : public esp32_ble_client::BLEClientBase { +class BluetoothConnection final : public esp32_ble_client::BLEClientBase { public: void dump_config() override; + void loop() override; bool gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gattc_cb_param_t *param) override; void gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param) override; esp32_ble_tracker::AdvertisementParserType get_advertisement_parser_type() override; esp_err_t read_characteristic(uint16_t handle); - esp_err_t write_characteristic(uint16_t handle, const std::string &data, bool response); + esp_err_t write_characteristic(uint16_t handle, const uint8_t *data, size_t length, bool response); esp_err_t read_descriptor(uint16_t handle); - esp_err_t write_descriptor(uint16_t handle, const std::string &data, bool response); + esp_err_t write_descriptor(uint16_t handle, const uint8_t *data, size_t length, bool response); esp_err_t notify_characteristic(uint16_t handle, bool enable); + void set_address(uint64_t address) override; + protected: friend class BluetoothProxy; + bool supports_efficient_uuids_() const; + void send_service_for_discovery_(); + void reset_connection_(esp_err_t reason); + void update_allocated_slot_(uint64_t find_value, uint64_t set_value); + void log_connection_error_(const char *operation, esp_gatt_status_t status); + void log_connection_warning_(const char *operation, esp_err_t err); + void log_gatt_not_connected_(const char *action, const char *type); + void log_gatt_operation_error_(const char *operation, uint16_t handle, esp_gatt_status_t status); + esp_err_t check_and_log_error_(const char *operation, esp_err_t err); + // Memory optimized layout for 32-bit systems // Group 1: Pointers (4 bytes each, naturally aligned) BluetoothProxy *proxy_; // Group 2: 2-byte types - int16_t send_service_{-2}; // Needs to handle negative values and service count + int16_t send_service_{-3}; // -3 = INIT_SENDING_SERVICES, -2 = DONE_SENDING_SERVICES, >=0 = service index // Group 3: 1-byte types bool seen_mtu_or_services_{false}; // 1 byte used, 1 byte padding }; -} // namespace bluetooth_proxy -} // namespace esphome +} // namespace esphome::bluetooth_proxy #endif // USE_ESP32 diff --git a/esphome/components/bluetooth_proxy/bluetooth_proxy.cpp b/esphome/components/bluetooth_proxy/bluetooth_proxy.cpp index a5e8ec0860..cd7261d5e5 100644 --- a/esphome/components/bluetooth_proxy/bluetooth_proxy.cpp +++ b/esphome/components/bluetooth_proxy/bluetooth_proxy.cpp @@ -3,30 +3,30 @@ #include "esphome/core/log.h" #include "esphome/core/macros.h" #include "esphome/core/application.h" +#include #ifdef USE_ESP32 -namespace esphome { -namespace bluetooth_proxy { +namespace esphome::bluetooth_proxy { static const char *const TAG = "bluetooth_proxy"; -static const int DONE_SENDING_SERVICES = -2; -std::vector get_128bit_uuid_vec(esp_bt_uuid_t uuid_source) { - esp_bt_uuid_t uuid = espbt::ESPBTUUID::from_uuid(uuid_source).as_128bit().get_uuid(); - return std::vector{((uint64_t) uuid.uuid.uuid128[15] << 56) | ((uint64_t) uuid.uuid.uuid128[14] << 48) | - ((uint64_t) uuid.uuid.uuid128[13] << 40) | ((uint64_t) uuid.uuid.uuid128[12] << 32) | - ((uint64_t) uuid.uuid.uuid128[11] << 24) | ((uint64_t) uuid.uuid.uuid128[10] << 16) | - ((uint64_t) uuid.uuid.uuid128[9] << 8) | ((uint64_t) uuid.uuid.uuid128[8]), - ((uint64_t) uuid.uuid.uuid128[7] << 56) | ((uint64_t) uuid.uuid.uuid128[6] << 48) | - ((uint64_t) uuid.uuid.uuid128[5] << 40) | ((uint64_t) uuid.uuid.uuid128[4] << 32) | - ((uint64_t) uuid.uuid.uuid128[3] << 24) | ((uint64_t) uuid.uuid.uuid128[2] << 16) | - ((uint64_t) uuid.uuid.uuid128[1] << 8) | ((uint64_t) uuid.uuid.uuid128[0])}; -} +// BLUETOOTH_PROXY_ADVERTISEMENT_BATCH_SIZE is defined during code generation +// It sets the batch size for BLE advertisements to maximize WiFi efficiency + +// Verify BLE advertisement data array size matches the BLE specification (31 bytes adv + 31 bytes scan response) +static_assert(sizeof(((api::BluetoothLERawAdvertisement *) nullptr)->data) == 62, + "BLE advertisement data array size mismatch"); BluetoothProxy::BluetoothProxy() { global_bluetooth_proxy = this; } void BluetoothProxy::setup() { + this->connections_free_response_.limit = BLUETOOTH_PROXY_MAX_CONNECTIONS; + this->connections_free_response_.free = BLUETOOTH_PROXY_MAX_CONNECTIONS; + + // Capture the configured scan mode from YAML before any API changes + this->configured_scan_active_ = this->parent_->get_scan_active(); + this->parent_->add_scanner_state_callback([this](esp32_ble_tracker::ScannerState state) { if (this->api_connection_ != nullptr) { this->send_bluetooth_scanner_state_(state); @@ -39,147 +39,98 @@ void BluetoothProxy::send_bluetooth_scanner_state_(esp32_ble_tracker::ScannerSta resp.state = static_cast(state); resp.mode = this->parent_->get_scan_active() ? api::enums::BluetoothScannerMode::BLUETOOTH_SCANNER_MODE_ACTIVE : api::enums::BluetoothScannerMode::BLUETOOTH_SCANNER_MODE_PASSIVE; - this->api_connection_->send_message(resp); + resp.configured_mode = this->configured_scan_active_ + ? api::enums::BluetoothScannerMode::BLUETOOTH_SCANNER_MODE_ACTIVE + : api::enums::BluetoothScannerMode::BLUETOOTH_SCANNER_MODE_PASSIVE; + this->api_connection_->send_message(resp, api::BluetoothScannerStateResponse::MESSAGE_TYPE); } +void BluetoothProxy::log_connection_request_ignored_(BluetoothConnection *connection, espbt::ClientState state) { + ESP_LOGW(TAG, "[%d] [%s] Connection request ignored, state: %s", connection->get_connection_index(), + connection->address_str().c_str(), espbt::client_state_to_string(state)); +} + +void BluetoothProxy::log_connection_info_(BluetoothConnection *connection, const char *message) { + ESP_LOGI(TAG, "[%d] [%s] Connecting %s", connection->get_connection_index(), connection->address_str().c_str(), + message); +} + +void BluetoothProxy::log_not_connected_gatt_(const char *action, const char *type) { + ESP_LOGW(TAG, "Cannot %s GATT %s, not connected", action, type); +} + +void BluetoothProxy::handle_gatt_not_connected_(uint64_t address, uint16_t handle, const char *action, + const char *type) { + this->log_not_connected_gatt_(action, type); + this->send_gatt_error(address, handle, ESP_GATT_NOT_CONNECTED); +} + +#ifdef USE_ESP32_BLE_DEVICE bool BluetoothProxy::parse_device(const esp32_ble_tracker::ESPBTDevice &device) { - if (!api::global_api_server->is_connected() || this->api_connection_ == nullptr || this->raw_advertisements_) - return false; - - ESP_LOGV(TAG, "Proxying packet from %s - %s. RSSI: %d dB", device.get_name().c_str(), device.address_str().c_str(), - device.get_rssi()); - this->send_api_packet_(device); - return true; + // This method should never be called since bluetooth_proxy always uses raw advertisements + // but we need to provide an implementation to satisfy the virtual method requirement + return false; } - -// Batch size for BLE advertisements to maximize WiFi efficiency -// Each advertisement is up to 80 bytes when packaged (including protocol overhead) -// Most advertisements are 20-30 bytes, allowing even more to fit per packet -// 16 advertisements × 80 bytes (worst case) = 1280 bytes out of ~1320 bytes usable payload -// This achieves ~97% WiFi MTU utilization while staying under the limit -static constexpr size_t FLUSH_BATCH_SIZE = 16; - -namespace { -// Batch buffer in anonymous namespace to avoid guard variable (saves 8 bytes) -// This is initialized at program startup before any threads -// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables) -std::vector batch_buffer; -} // namespace - -static std::vector &get_batch_buffer() { return batch_buffer; } +#endif bool BluetoothProxy::parse_devices(const esp32_ble::BLEScanResult *scan_results, size_t count) { - if (!api::global_api_server->is_connected() || this->api_connection_ == nullptr || !this->raw_advertisements_) + if (!api::global_api_server->is_connected() || this->api_connection_ == nullptr) return false; - // Get the batch buffer reference - auto &batch_buffer = get_batch_buffer(); + auto &advertisements = this->response_.advertisements; - // Reserve additional capacity if needed - size_t new_size = batch_buffer.size() + count; - if (batch_buffer.capacity() < new_size) { - batch_buffer.reserve(new_size); - } - - // Add new advertisements to the batch buffer for (size_t i = 0; i < count; i++) { auto &result = scan_results[i]; uint8_t length = result.adv_data_len + result.scan_rsp_len; - batch_buffer.emplace_back(); - auto &adv = batch_buffer.back(); + // Fill in the data directly at current position + auto &adv = advertisements[this->response_.advertisements_len]; adv.address = esp32_ble::ble_addr_to_uint64(result.bda); adv.rssi = result.rssi; adv.address_type = result.ble_addr_type; - adv.data.assign(&result.ble_adv[0], &result.ble_adv[length]); + adv.data_len = length; + std::memcpy(adv.data, result.ble_adv, length); + + this->response_.advertisements_len++; ESP_LOGV(TAG, "Queuing raw packet from %02X:%02X:%02X:%02X:%02X:%02X, length %d. RSSI: %d dB", result.bda[0], result.bda[1], result.bda[2], result.bda[3], result.bda[4], result.bda[5], length, result.rssi); - } - // Only send if we've accumulated a good batch size to maximize batching efficiency - // https://github.com/esphome/backlog/issues/21 - if (batch_buffer.size() >= FLUSH_BATCH_SIZE) { - this->flush_pending_advertisements(); + // Flush if we have reached BLUETOOTH_PROXY_ADVERTISEMENT_BATCH_SIZE + if (this->response_.advertisements_len >= BLUETOOTH_PROXY_ADVERTISEMENT_BATCH_SIZE) { + this->flush_pending_advertisements(); + } } return true; } void BluetoothProxy::flush_pending_advertisements() { - auto &batch_buffer = get_batch_buffer(); - if (batch_buffer.empty() || !api::global_api_server->is_connected() || this->api_connection_ == nullptr) + if (this->response_.advertisements_len == 0 || !api::global_api_server->is_connected() || + this->api_connection_ == nullptr) return; - api::BluetoothLERawAdvertisementsResponse resp; - resp.advertisements.swap(batch_buffer); - this->api_connection_->send_message(resp); -} + // Send the message + this->api_connection_->send_message(this->response_, api::BluetoothLERawAdvertisementsResponse::MESSAGE_TYPE); -void BluetoothProxy::send_api_packet_(const esp32_ble_tracker::ESPBTDevice &device) { - api::BluetoothLEAdvertisementResponse resp; - resp.address = device.address_uint64(); - resp.address_type = device.get_address_type(); - if (!device.get_name().empty()) - resp.name = device.get_name(); - resp.rssi = device.get_rssi(); + ESP_LOGV(TAG, "Sent batch of %u BLE advertisements", this->response_.advertisements_len); - // Pre-allocate vectors based on known sizes - auto service_uuids = device.get_service_uuids(); - resp.service_uuids.reserve(service_uuids.size()); - for (auto &uuid : service_uuids) { - resp.service_uuids.emplace_back(uuid.to_string()); - } - - // Pre-allocate service data vector - auto service_datas = device.get_service_datas(); - resp.service_data.reserve(service_datas.size()); - for (auto &data : service_datas) { - resp.service_data.emplace_back(); - auto &service_data = resp.service_data.back(); - service_data.uuid = data.uuid.to_string(); - service_data.data.assign(data.data.begin(), data.data.end()); - } - - // Pre-allocate manufacturer data vector - auto manufacturer_datas = device.get_manufacturer_datas(); - resp.manufacturer_data.reserve(manufacturer_datas.size()); - for (auto &data : manufacturer_datas) { - resp.manufacturer_data.emplace_back(); - auto &manufacturer_data = resp.manufacturer_data.back(); - manufacturer_data.uuid = data.uuid.to_string(); - manufacturer_data.data.assign(data.data.begin(), data.data.end()); - } - - this->api_connection_->send_message(resp); + // Reset the length for the next batch + this->response_.advertisements_len = 0; } void BluetoothProxy::dump_config() { - ESP_LOGCONFIG(TAG, "Bluetooth Proxy:"); ESP_LOGCONFIG(TAG, + "Bluetooth Proxy:\n" " Active: %s\n" - " Connections: %d\n" - " Raw advertisements: %s", - YESNO(this->active_), this->connections_.size(), YESNO(this->raw_advertisements_)); -} - -int BluetoothProxy::get_bluetooth_connections_free() { - int free = 0; - for (auto *connection : this->connections_) { - if (connection->address_ == 0) { - free++; - ESP_LOGV(TAG, "[%d] Free connection", connection->get_connection_index()); - } else { - ESP_LOGV(TAG, "[%d] Used connection by [%s]", connection->get_connection_index(), - connection->address_str().c_str()); - } - } - return free; + " Connections: %d", + YESNO(this->active_), this->connection_count_); } void BluetoothProxy::loop() { if (!api::global_api_server->is_connected() || this->api_connection_ == nullptr) { - for (auto *connection : this->connections_) { + for (uint8_t i = 0; i < this->connection_count_; i++) { + auto *connection = this->connections_[i]; if (connection->get_address() != 0 && !connection->disconnect_pending()) { connection->disconnect(); } @@ -188,143 +139,22 @@ void BluetoothProxy::loop() { } // Flush any pending BLE advertisements that have been accumulated but not yet sent - if (this->raw_advertisements_) { - static uint32_t last_flush_time = 0; - uint32_t now = App.get_loop_component_start_time(); + uint32_t now = App.get_loop_component_start_time(); - // Flush accumulated advertisements every 100ms - if (now - last_flush_time >= 100) { - this->flush_pending_advertisements(); - last_flush_time = now; - } - } - for (auto *connection : this->connections_) { - if (connection->send_service_ == connection->service_count_) { - connection->send_service_ = DONE_SENDING_SERVICES; - this->send_gatt_services_done(connection->get_address()); - if (connection->connection_type_ == espbt::ConnectionType::V3_WITH_CACHE || - connection->connection_type_ == espbt::ConnectionType::V3_WITHOUT_CACHE) { - connection->release_services(); - } - } else if (connection->send_service_ >= 0) { - esp_gattc_service_elem_t service_result; - uint16_t service_count = 1; - esp_gatt_status_t service_status = - esp_ble_gattc_get_service(connection->get_gattc_if(), connection->get_conn_id(), nullptr, &service_result, - &service_count, connection->send_service_); - connection->send_service_++; - if (service_status != ESP_GATT_OK) { - ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_get_service error at offset=%d, status=%d", - connection->get_connection_index(), connection->address_str().c_str(), connection->send_service_ - 1, - service_status); - continue; - } - if (service_count == 0) { - ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_get_service missing, service_count=%d", - connection->get_connection_index(), connection->address_str().c_str(), service_count); - continue; - } - api::BluetoothGATTGetServicesResponse resp; - resp.address = connection->get_address(); - resp.services.reserve(1); // Always one service per response in this implementation - api::BluetoothGATTService service_resp; - service_resp.uuid = get_128bit_uuid_vec(service_result.uuid); - service_resp.handle = service_result.start_handle; - uint16_t char_offset = 0; - esp_gattc_char_elem_t char_result; - // Get the number of characteristics directly with one call - uint16_t total_char_count = 0; - esp_gatt_status_t char_count_status = esp_ble_gattc_get_attr_count( - connection->get_gattc_if(), connection->get_conn_id(), ESP_GATT_DB_CHARACTERISTIC, - service_result.start_handle, service_result.end_handle, 0, &total_char_count); - - if (char_count_status == ESP_GATT_OK && total_char_count > 0) { - // Only reserve if we successfully got a count - service_resp.characteristics.reserve(total_char_count); - } else if (char_count_status != ESP_GATT_OK) { - ESP_LOGW(TAG, "[%d] [%s] Error getting characteristic count, status=%d", connection->get_connection_index(), - connection->address_str().c_str(), char_count_status); - } - - // Now process characteristics - while (true) { // characteristics - uint16_t char_count = 1; - esp_gatt_status_t char_status = esp_ble_gattc_get_all_char( - connection->get_gattc_if(), connection->get_conn_id(), service_result.start_handle, - service_result.end_handle, &char_result, &char_count, char_offset); - if (char_status == ESP_GATT_INVALID_OFFSET || char_status == ESP_GATT_NOT_FOUND) { - break; - } - if (char_status != ESP_GATT_OK) { - ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_get_all_char error, status=%d", connection->get_connection_index(), - connection->address_str().c_str(), char_status); - break; - } - if (char_count == 0) { - break; - } - api::BluetoothGATTCharacteristic characteristic_resp; - characteristic_resp.uuid = get_128bit_uuid_vec(char_result.uuid); - characteristic_resp.handle = char_result.char_handle; - characteristic_resp.properties = char_result.properties; - char_offset++; - - // Get the number of descriptors directly with one call - uint16_t total_desc_count = 0; - esp_gatt_status_t desc_count_status = - esp_ble_gattc_get_attr_count(connection->get_gattc_if(), connection->get_conn_id(), ESP_GATT_DB_DESCRIPTOR, - char_result.char_handle, service_result.end_handle, 0, &total_desc_count); - - if (desc_count_status == ESP_GATT_OK && total_desc_count > 0) { - // Only reserve if we successfully got a count - characteristic_resp.descriptors.reserve(total_desc_count); - } else if (desc_count_status != ESP_GATT_OK) { - ESP_LOGW(TAG, "[%d] [%s] Error getting descriptor count for char handle %d, status=%d", - connection->get_connection_index(), connection->address_str().c_str(), char_result.char_handle, - desc_count_status); - } - - // Now process descriptors - uint16_t desc_offset = 0; - esp_gattc_descr_elem_t desc_result; - while (true) { // descriptors - uint16_t desc_count = 1; - esp_gatt_status_t desc_status = - esp_ble_gattc_get_all_descr(connection->get_gattc_if(), connection->get_conn_id(), - char_result.char_handle, &desc_result, &desc_count, desc_offset); - if (desc_status == ESP_GATT_INVALID_OFFSET || desc_status == ESP_GATT_NOT_FOUND) { - break; - } - if (desc_status != ESP_GATT_OK) { - ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_get_all_descr error, status=%d", connection->get_connection_index(), - connection->address_str().c_str(), desc_status); - break; - } - if (desc_count == 0) { - break; - } - api::BluetoothGATTDescriptor descriptor_resp; - descriptor_resp.uuid = get_128bit_uuid_vec(desc_result.uuid); - descriptor_resp.handle = desc_result.handle; - characteristic_resp.descriptors.push_back(std::move(descriptor_resp)); - desc_offset++; - } - service_resp.characteristics.push_back(std::move(characteristic_resp)); - } - resp.services.push_back(std::move(service_resp)); - this->api_connection_->send_message(resp); - } + // Flush accumulated advertisements every 100ms + if (now - this->last_advertisement_flush_time_ >= 100) { + this->flush_pending_advertisements(); + this->last_advertisement_flush_time_ = now; } } esp32_ble_tracker::AdvertisementParserType BluetoothProxy::get_advertisement_parser_type() { - if (this->raw_advertisements_) - return esp32_ble_tracker::AdvertisementParserType::RAW_ADVERTISEMENTS; - return esp32_ble_tracker::AdvertisementParserType::PARSED_ADVERTISEMENTS; + return esp32_ble_tracker::AdvertisementParserType::RAW_ADVERTISEMENTS; } BluetoothConnection *BluetoothProxy::get_connection_(uint64_t address, bool reserve) { - for (auto *connection : this->connections_) { + for (uint8_t i = 0; i < this->connection_count_; i++) { + auto *connection = this->connections_[i]; if (connection->get_address() == address) return connection; } @@ -332,9 +162,10 @@ BluetoothConnection *BluetoothProxy::get_connection_(uint64_t address, bool rese if (!reserve) return nullptr; - for (auto *connection : this->connections_) { + for (uint8_t i = 0; i < this->connection_count_; i++) { + auto *connection = this->connections_[i]; if (connection->get_address() == 0) { - connection->send_service_ = DONE_SENDING_SERVICES; + connection->send_service_ = INIT_SENDING_SERVICES; connection->set_address(address); // All connections must start at INIT // We only set the state if we allocate the connection @@ -351,33 +182,25 @@ BluetoothConnection *BluetoothProxy::get_connection_(uint64_t address, bool rese void BluetoothProxy::bluetooth_device_request(const api::BluetoothDeviceRequest &msg) { switch (msg.request_type) { case api::enums::BLUETOOTH_DEVICE_REQUEST_TYPE_CONNECT_V3_WITH_CACHE: - case api::enums::BLUETOOTH_DEVICE_REQUEST_TYPE_CONNECT_V3_WITHOUT_CACHE: - case api::enums::BLUETOOTH_DEVICE_REQUEST_TYPE_CONNECT: { + case api::enums::BLUETOOTH_DEVICE_REQUEST_TYPE_CONNECT_V3_WITHOUT_CACHE: { auto *connection = this->get_connection_(msg.address, true); if (connection == nullptr) { ESP_LOGW(TAG, "No free connections available"); this->send_device_connection(msg.address, false); return; } + if (!msg.has_address_type) { + ESP_LOGE(TAG, "[%d] [%s] Missing address type in connect request", connection->get_connection_index(), + connection->address_str().c_str()); + this->send_device_connection(msg.address, false); + return; + } if (connection->state() == espbt::ClientState::CONNECTED || connection->state() == espbt::ClientState::ESTABLISHED) { - ESP_LOGW(TAG, "[%d] [%s] Connection already established", connection->get_connection_index(), - connection->address_str().c_str()); + this->log_connection_request_ignored_(connection, connection->state()); this->send_device_connection(msg.address, true); this->send_connections_free(); return; - } else if (connection->state() == espbt::ClientState::SEARCHING) { - ESP_LOGW(TAG, "[%d] [%s] Connection request ignored, already searching for device", - connection->get_connection_index(), connection->address_str().c_str()); - return; - } else if (connection->state() == espbt::ClientState::DISCOVERED) { - ESP_LOGW(TAG, "[%d] [%s] Connection request ignored, device already discovered", - connection->get_connection_index(), connection->address_str().c_str()); - return; - } else if (connection->state() == espbt::ClientState::READY_TO_CONNECT) { - ESP_LOGW(TAG, "[%d] [%s] Connection request ignored, waiting in line to connect", - connection->get_connection_index(), connection->address_str().c_str()); - return; } else if (connection->state() == espbt::ClientState::CONNECTING) { if (connection->disconnect_pending()) { ESP_LOGW(TAG, "[%d] [%s] Connection request while pending disconnect, cancelling pending disconnect", @@ -385,37 +208,22 @@ void BluetoothProxy::bluetooth_device_request(const api::BluetoothDeviceRequest connection->cancel_pending_disconnect(); return; } - ESP_LOGW(TAG, "[%d] [%s] Connection request ignored, already connecting", connection->get_connection_index(), - connection->address_str().c_str()); - return; - } else if (connection->state() == espbt::ClientState::DISCONNECTING) { - ESP_LOGW(TAG, "[%d] [%s] Connection request ignored, device is disconnecting", - connection->get_connection_index(), connection->address_str().c_str()); + this->log_connection_request_ignored_(connection, connection->state()); return; } else if (connection->state() != espbt::ClientState::INIT) { - ESP_LOGW(TAG, "[%d] [%s] Connection already in progress", connection->get_connection_index(), - connection->address_str().c_str()); + this->log_connection_request_ignored_(connection, connection->state()); return; } if (msg.request_type == api::enums::BLUETOOTH_DEVICE_REQUEST_TYPE_CONNECT_V3_WITH_CACHE) { connection->set_connection_type(espbt::ConnectionType::V3_WITH_CACHE); - ESP_LOGI(TAG, "[%d] [%s] Connecting v3 with cache", connection->get_connection_index(), - connection->address_str().c_str()); - } else if (msg.request_type == api::enums::BLUETOOTH_DEVICE_REQUEST_TYPE_CONNECT_V3_WITHOUT_CACHE) { + this->log_connection_info_(connection, "v3 with cache"); + } else { // BLUETOOTH_DEVICE_REQUEST_TYPE_CONNECT_V3_WITHOUT_CACHE connection->set_connection_type(espbt::ConnectionType::V3_WITHOUT_CACHE); - ESP_LOGI(TAG, "[%d] [%s] Connecting v3 without cache", connection->get_connection_index(), - connection->address_str().c_str()); - } else { - connection->set_connection_type(espbt::ConnectionType::V1); - ESP_LOGI(TAG, "[%d] [%s] Connecting v1", connection->get_connection_index(), connection->address_str().c_str()); - } - if (msg.has_address_type) { - uint64_to_bd_addr(msg.address, connection->remote_bda_); - connection->set_remote_addr_type(static_cast(msg.address_type)); - connection->set_state(espbt::ClientState::DISCOVERED); - } else { - connection->set_state(espbt::ClientState::SEARCHING); + this->log_connection_info_(connection, "v3 without cache"); } + uint64_to_bd_addr(msg.address, connection->remote_bda_); + connection->set_remote_addr_type(static_cast(msg.address_type)); + connection->set_state(espbt::ClientState::DISCOVERED); this->send_connections_free(); break; } @@ -465,18 +273,22 @@ void BluetoothProxy::bluetooth_device_request(const api::BluetoothDeviceRequest call.success = ret == ESP_OK; call.error = ret; - this->api_connection_->send_message(call); + this->api_connection_->send_message(call, api::BluetoothDeviceClearCacheResponse::MESSAGE_TYPE); break; } + case api::enums::BLUETOOTH_DEVICE_REQUEST_TYPE_CONNECT: { + ESP_LOGE(TAG, "V1 connections removed"); + this->send_device_connection(msg.address, false); + break; + } } } void BluetoothProxy::bluetooth_gatt_read(const api::BluetoothGATTReadRequest &msg) { auto *connection = this->get_connection_(msg.address, false); if (connection == nullptr) { - ESP_LOGW(TAG, "Cannot read GATT characteristic, not connected"); - this->send_gatt_error(msg.address, msg.handle, ESP_GATT_NOT_CONNECTED); + this->handle_gatt_not_connected_(msg.address, msg.handle, "read", "characteristic"); return; } @@ -489,12 +301,11 @@ void BluetoothProxy::bluetooth_gatt_read(const api::BluetoothGATTReadRequest &ms void BluetoothProxy::bluetooth_gatt_write(const api::BluetoothGATTWriteRequest &msg) { auto *connection = this->get_connection_(msg.address, false); if (connection == nullptr) { - ESP_LOGW(TAG, "Cannot write GATT characteristic, not connected"); - this->send_gatt_error(msg.address, msg.handle, ESP_GATT_NOT_CONNECTED); + this->handle_gatt_not_connected_(msg.address, msg.handle, "write", "characteristic"); return; } - auto err = connection->write_characteristic(msg.handle, msg.data, msg.response); + auto err = connection->write_characteristic(msg.handle, msg.data, msg.data_len, msg.response); if (err != ESP_OK) { this->send_gatt_error(msg.address, msg.handle, err); } @@ -503,8 +314,7 @@ void BluetoothProxy::bluetooth_gatt_write(const api::BluetoothGATTWriteRequest & void BluetoothProxy::bluetooth_gatt_read_descriptor(const api::BluetoothGATTReadDescriptorRequest &msg) { auto *connection = this->get_connection_(msg.address, false); if (connection == nullptr) { - ESP_LOGW(TAG, "Cannot read GATT descriptor, not connected"); - this->send_gatt_error(msg.address, msg.handle, ESP_GATT_NOT_CONNECTED); + this->handle_gatt_not_connected_(msg.address, msg.handle, "read", "descriptor"); return; } @@ -517,12 +327,11 @@ void BluetoothProxy::bluetooth_gatt_read_descriptor(const api::BluetoothGATTRead void BluetoothProxy::bluetooth_gatt_write_descriptor(const api::BluetoothGATTWriteDescriptorRequest &msg) { auto *connection = this->get_connection_(msg.address, false); if (connection == nullptr) { - ESP_LOGW(TAG, "Cannot write GATT descriptor, not connected"); - this->send_gatt_error(msg.address, msg.handle, ESP_GATT_NOT_CONNECTED); + this->handle_gatt_not_connected_(msg.address, msg.handle, "write", "descriptor"); return; } - auto err = connection->write_descriptor(msg.handle, msg.data, true); + auto err = connection->write_descriptor(msg.handle, msg.data, msg.data_len, true); if (err != ESP_OK) { this->send_gatt_error(msg.address, msg.handle, err); } @@ -531,8 +340,7 @@ void BluetoothProxy::bluetooth_gatt_write_descriptor(const api::BluetoothGATTWri void BluetoothProxy::bluetooth_gatt_send_services(const api::BluetoothGATTGetServicesRequest &msg) { auto *connection = this->get_connection_(msg.address, false); if (connection == nullptr || !connection->connected()) { - ESP_LOGW(TAG, "Cannot get GATT services, not connected"); - this->send_gatt_error(msg.address, 0, ESP_GATT_NOT_CONNECTED); + this->handle_gatt_not_connected_(msg.address, 0, "get", "services"); return; } if (!connection->service_count_) { @@ -540,16 +348,14 @@ void BluetoothProxy::bluetooth_gatt_send_services(const api::BluetoothGATTGetSer this->send_gatt_services_done(msg.address); return; } - if (connection->send_service_ == - DONE_SENDING_SERVICES) // Only start sending services if we're not already sending them + if (connection->send_service_ == INIT_SENDING_SERVICES) // Start sending services if not started yet connection->send_service_ = 0; } void BluetoothProxy::bluetooth_gatt_notify(const api::BluetoothGATTNotifyRequest &msg) { auto *connection = this->get_connection_(msg.address, false); if (connection == nullptr) { - ESP_LOGW(TAG, "Cannot notify GATT characteristic, not connected"); - this->send_gatt_error(msg.address, msg.handle, ESP_GATT_NOT_CONNECTED); + this->handle_gatt_not_connected_(msg.address, msg.handle, "notify", "characteristic"); return; } @@ -565,7 +371,6 @@ void BluetoothProxy::subscribe_api_connection(api::APIConnection *api_connection return; } this->api_connection_ = api_connection; - this->raw_advertisements_ = flags & BluetoothProxySubscriptionFlag::SUBSCRIPTION_RAW_ADVERTISEMENTS; this->parent_->recalculate_advertisement_parser_types(); this->send_bluetooth_scanner_state_(this->parent_->get_scanner_state()); @@ -577,7 +382,6 @@ void BluetoothProxy::unsubscribe_api_connection(api::APIConnection *api_connecti return; } this->api_connection_ = nullptr; - this->raw_advertisements_ = false; this->parent_->recalculate_advertisement_parser_types(); } @@ -589,20 +393,16 @@ void BluetoothProxy::send_device_connection(uint64_t address, bool connected, ui call.connected = connected; call.mtu = mtu; call.error = error; - this->api_connection_->send_message(call); + this->api_connection_->send_message(call, api::BluetoothDeviceConnectionResponse::MESSAGE_TYPE); } void BluetoothProxy::send_connections_free() { - if (this->api_connection_ == nullptr) - return; - api::BluetoothConnectionsFreeResponse call; - call.free = this->get_bluetooth_connections_free(); - call.limit = this->get_bluetooth_connections_limit(); - for (auto *connection : this->connections_) { - if (connection->address_ != 0) { - call.allocated.push_back(connection->address_); - } + if (this->api_connection_ != nullptr) { + this->send_connections_free(this->api_connection_); } - this->api_connection_->send_message(call); +} + +void BluetoothProxy::send_connections_free(api::APIConnection *api_connection) { + api_connection->send_message(this->connections_free_response_, api::BluetoothConnectionsFreeResponse::MESSAGE_TYPE); } void BluetoothProxy::send_gatt_services_done(uint64_t address) { @@ -610,7 +410,7 @@ void BluetoothProxy::send_gatt_services_done(uint64_t address) { return; api::BluetoothGATTGetServicesDoneResponse call; call.address = address; - this->api_connection_->send_message(call); + this->api_connection_->send_message(call, api::BluetoothGATTGetServicesDoneResponse::MESSAGE_TYPE); } void BluetoothProxy::send_gatt_error(uint64_t address, uint16_t handle, esp_err_t error) { @@ -620,7 +420,7 @@ void BluetoothProxy::send_gatt_error(uint64_t address, uint16_t handle, esp_err_ call.address = address; call.handle = handle; call.error = error; - this->api_connection_->send_message(call); + this->api_connection_->send_message(call, api::BluetoothGATTWriteResponse::MESSAGE_TYPE); } void BluetoothProxy::send_device_pairing(uint64_t address, bool paired, esp_err_t error) { @@ -629,7 +429,7 @@ void BluetoothProxy::send_device_pairing(uint64_t address, bool paired, esp_err_ call.paired = paired; call.error = error; - this->api_connection_->send_message(call); + this->api_connection_->send_message(call, api::BluetoothDevicePairingResponse::MESSAGE_TYPE); } void BluetoothProxy::send_device_unpairing(uint64_t address, bool success, esp_err_t error) { @@ -638,7 +438,7 @@ void BluetoothProxy::send_device_unpairing(uint64_t address, bool success, esp_e call.success = success; call.error = error; - this->api_connection_->send_message(call); + this->api_connection_->send_message(call, api::BluetoothDeviceUnpairingResponse::MESSAGE_TYPE); } void BluetoothProxy::bluetooth_scanner_set_mode(bool active) { @@ -654,7 +454,6 @@ void BluetoothProxy::bluetooth_scanner_set_mode(bool active) { BluetoothProxy *global_bluetooth_proxy = nullptr; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) -} // namespace bluetooth_proxy -} // namespace esphome +} // namespace esphome::bluetooth_proxy #endif // USE_ESP32 diff --git a/esphome/components/bluetooth_proxy/bluetooth_proxy.h b/esphome/components/bluetooth_proxy/bluetooth_proxy.h index f0632350e0..a5f0fbe32f 100644 --- a/esphome/components/bluetooth_proxy/bluetooth_proxy.h +++ b/esphome/components/bluetooth_proxy/bluetooth_proxy.h @@ -2,6 +2,7 @@ #ifdef USE_ESP32 +#include #include #include @@ -15,13 +16,16 @@ #include "bluetooth_connection.h" +#ifndef CONFIG_ESP_HOSTED_ENABLE_BT_BLUEDROID #include +#endif #include -namespace esphome { -namespace bluetooth_proxy { +namespace esphome::bluetooth_proxy { static const esp_err_t ESP_GATT_NOT_CONNECTED = -1; +static const int DONE_SENDING_SERVICES = -2; +static const int INIT_SENDING_SERVICES = -3; using namespace esp32_ble_client; @@ -48,10 +52,13 @@ enum BluetoothProxySubscriptionFlag : uint32_t { SUBSCRIPTION_RAW_ADVERTISEMENTS = 1 << 0, }; -class BluetoothProxy : public esp32_ble_tracker::ESPBTDeviceListener, public Component { +class BluetoothProxy final : public esp32_ble_tracker::ESPBTDeviceListener, public Component { + friend class BluetoothConnection; // Allow connection to update connections_free_response_ public: BluetoothProxy(); +#ifdef USE_ESP32_BLE_DEVICE bool parse_device(const esp32_ble_tracker::ESPBTDevice &device) override; +#endif bool parse_devices(const esp32_ble::BLEScanResult *scan_results, size_t count) override; void dump_config() override; void setup() override; @@ -60,8 +67,10 @@ class BluetoothProxy : public esp32_ble_tracker::ESPBTDeviceListener, public Com esp32_ble_tracker::AdvertisementParserType get_advertisement_parser_type() override; void register_connection(BluetoothConnection *connection) { - this->connections_.push_back(connection); - connection->proxy_ = this; + if (this->connection_count_ < BLUETOOTH_PROXY_MAX_CONNECTIONS) { + this->connections_[this->connection_count_++] = connection; + connection->proxy_ = this; + } } void bluetooth_device_request(const api::BluetoothDeviceRequest &msg); @@ -72,15 +81,13 @@ class BluetoothProxy : public esp32_ble_tracker::ESPBTDeviceListener, public Com void bluetooth_gatt_send_services(const api::BluetoothGATTGetServicesRequest &msg); void bluetooth_gatt_notify(const api::BluetoothGATTNotifyRequest &msg); - int get_bluetooth_connections_free(); - int get_bluetooth_connections_limit() { return this->connections_.size(); } - void subscribe_api_connection(api::APIConnection *api_connection, uint32_t flags); void unsubscribe_api_connection(api::APIConnection *api_connection); api::APIConnection *get_api_connection() { return this->api_connection_; } void send_device_connection(uint64_t address, bool connected, uint16_t mtu = 0, esp_err_t error = ESP_OK); void send_connections_free(); + void send_connections_free(api::APIConnection *api_connection); void send_gatt_services_done(uint64_t address); void send_gatt_error(uint64_t address, uint16_t handle, esp_err_t error); void send_device_pairing(uint64_t address, bool paired, esp_err_t error = ESP_OK); @@ -125,31 +132,45 @@ class BluetoothProxy : public esp32_ble_tracker::ESPBTDeviceListener, public Com std::string get_bluetooth_mac_address_pretty() { const uint8_t *mac = esp_bt_dev_get_address(); - return str_snprintf("%02X:%02X:%02X:%02X:%02X:%02X", 17, mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]); + char buf[18]; + format_mac_addr_upper(mac, buf); + return std::string(buf); } protected: - void send_api_packet_(const esp32_ble_tracker::ESPBTDevice &device); void send_bluetooth_scanner_state_(esp32_ble_tracker::ScannerState state); BluetoothConnection *get_connection_(uint64_t address, bool reserve); + void log_connection_request_ignored_(BluetoothConnection *connection, espbt::ClientState state); + void log_connection_info_(BluetoothConnection *connection, const char *message); + void log_not_connected_gatt_(const char *action, const char *type); + void handle_gatt_not_connected_(uint64_t address, uint16_t handle, const char *action, const char *type); // Memory optimized layout for 32-bit systems // Group 1: Pointers (4 bytes each, naturally aligned) api::APIConnection *api_connection_{nullptr}; - // Group 2: Container types (typically 12 bytes on 32-bit) - std::vector connections_{}; + // Group 2: Fixed-size array of connection pointers + std::array connections_{}; - // Group 3: 1-byte types grouped together + // BLE advertisement batching + api::BluetoothLERawAdvertisementsResponse response_; + + // Group 3: 4-byte types + uint32_t last_advertisement_flush_time_{0}; + + // Pre-allocated response message - always ready to send + api::BluetoothConnectionsFreeResponse connections_free_response_; + + // Group 4: 1-byte types grouped together bool active_; - bool raw_advertisements_{false}; - // 2 bytes used, 2 bytes padding + uint8_t connection_count_{0}; + bool configured_scan_active_{false}; // Configured scan mode from YAML + // 3 bytes used, 1 byte padding }; extern BluetoothProxy *global_bluetooth_proxy; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) -} // namespace bluetooth_proxy -} // namespace esphome +} // namespace esphome::bluetooth_proxy #endif // USE_ESP32 diff --git a/esphome/components/bme280_base/bme280_base.cpp b/esphome/components/bme280_base/bme280_base.cpp index d2524e5aac..86b65d361d 100644 --- a/esphome/components/bme280_base/bme280_base.cpp +++ b/esphome/components/bme280_base/bme280_base.cpp @@ -7,6 +7,8 @@ #include #include +#define BME280_ERROR_WRONG_CHIP_ID "Wrong chip ID" + namespace esphome { namespace bme280_base { @@ -88,7 +90,6 @@ const char *oversampling_to_str(BME280Oversampling oversampling) { // NOLINT } void BME280Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t chip_id = 0; // Mark as not failed before initializing. Some devices will turn off sensors to save on batteries @@ -99,18 +100,18 @@ void BME280Component::setup() { if (!this->read_byte(BME280_REGISTER_CHIPID, &chip_id)) { this->error_code_ = COMMUNICATION_FAILED; - this->mark_failed(); + this->mark_failed(ESP_LOG_MSG_COMM_FAIL); return; } if (chip_id != 0x60) { this->error_code_ = WRONG_CHIP_ID; - this->mark_failed(); + this->mark_failed(BME280_ERROR_WRONG_CHIP_ID); return; } // Send a soft reset. if (!this->write_byte(BME280_REGISTER_RESET, BME280_SOFT_RESET)) { - this->mark_failed(); + this->mark_failed("Reset failed"); return; } // Wait until the NVM data has finished loading. @@ -119,14 +120,12 @@ void BME280Component::setup() { do { // NOLINT delay(2); if (!this->read_byte(BME280_REGISTER_STATUS, &status)) { - ESP_LOGW(TAG, "Error reading status register."); - this->mark_failed(); + this->mark_failed("Error reading status register"); return; } } while ((status & BME280_STATUS_IM_UPDATE) && (--retry)); if (status & BME280_STATUS_IM_UPDATE) { - ESP_LOGW(TAG, "Timeout loading NVM."); - this->mark_failed(); + this->mark_failed("Timeout loading NVM"); return; } @@ -154,26 +153,26 @@ void BME280Component::setup() { uint8_t humid_control_val = 0; if (!this->read_byte(BME280_REGISTER_CONTROLHUMID, &humid_control_val)) { - this->mark_failed(); + this->mark_failed("Read humidity control"); return; } humid_control_val &= ~0b00000111; humid_control_val |= this->humidity_oversampling_ & 0b111; if (!this->write_byte(BME280_REGISTER_CONTROLHUMID, humid_control_val)) { - this->mark_failed(); + this->mark_failed("Write humidity control"); return; } uint8_t config_register = 0; if (!this->read_byte(BME280_REGISTER_CONFIG, &config_register)) { - this->mark_failed(); + this->mark_failed("Read config"); return; } config_register &= ~0b11111100; config_register |= 0b101 << 5; // 1000 ms standby time config_register |= (this->iir_filter_ & 0b111) << 2; if (!this->write_byte(BME280_REGISTER_CONFIG, config_register)) { - this->mark_failed(); + this->mark_failed("Write config"); return; } } @@ -184,7 +183,7 @@ void BME280Component::dump_config() { ESP_LOGE(TAG, ESP_LOG_MSG_COMM_FAIL); break; case WRONG_CHIP_ID: - ESP_LOGE(TAG, "BME280 has wrong chip ID! Is it a BME280?"); + ESP_LOGE(TAG, BME280_ERROR_WRONG_CHIP_ID); break; case NONE: default: @@ -224,21 +223,21 @@ void BME280Component::update() { this->set_timeout("data", uint32_t(ceilf(meas_time)), [this]() { uint8_t data[8]; if (!this->read_bytes(BME280_REGISTER_MEASUREMENTS, data, 8)) { - ESP_LOGW(TAG, "Error reading registers."); + ESP_LOGW(TAG, "Error reading registers"); this->status_set_warning(); return; } int32_t t_fine = 0; float const temperature = this->read_temperature_(data, &t_fine); if (std::isnan(temperature)) { - ESP_LOGW(TAG, "Invalid temperature, cannot read pressure & humidity values."); + ESP_LOGW(TAG, "Invalid temperature"); this->status_set_warning(); return; } float const pressure = this->read_pressure_(data, t_fine); float const humidity = this->read_humidity_(data, t_fine); - ESP_LOGV(TAG, "Got temperature=%.1f°C pressure=%.1fhPa humidity=%.1f%%", temperature, pressure, humidity); + ESP_LOGV(TAG, "Temperature=%.1f°C Pressure=%.1fhPa Humidity=%.1f%%", temperature, pressure, humidity); if (this->temperature_sensor_ != nullptr) this->temperature_sensor_->publish_state(temperature); if (this->pressure_sensor_ != nullptr) diff --git a/esphome/components/bme680/bme680.cpp b/esphome/components/bme680/bme680.cpp index 7e8f2f5a32..16435ccfee 100644 --- a/esphome/components/bme680/bme680.cpp +++ b/esphome/components/bme680/bme680.cpp @@ -28,7 +28,7 @@ const float BME680_GAS_LOOKUP_TABLE_1[16] PROGMEM = {0.0, 0.0, 0.0, 0.0, 0.0, const float BME680_GAS_LOOKUP_TABLE_2[16] PROGMEM = {0.0, 0.0, 0.0, 0.0, 0.1, 0.7, 0.0, -0.8, -0.1, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0}; -static const char *oversampling_to_str(BME680Oversampling oversampling) { +[[maybe_unused]] static const char *oversampling_to_str(BME680Oversampling oversampling) { switch (oversampling) { case BME680_OVERSAMPLING_NONE: return "None"; @@ -47,7 +47,7 @@ static const char *oversampling_to_str(BME680Oversampling oversampling) { } } -static const char *iir_filter_to_str(BME680IIRFilter filter) { +[[maybe_unused]] static const char *iir_filter_to_str(BME680IIRFilter filter) { switch (filter) { case BME680_IIR_FILTER_OFF: return "OFF"; @@ -71,7 +71,6 @@ static const char *iir_filter_to_str(BME680IIRFilter filter) { } void BME680Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t chip_id; if (!this->read_byte(BME680_REGISTER_CHIPID, &chip_id) || chip_id != 0x61) { this->mark_failed(); diff --git a/esphome/components/bme680_bsec/__init__.py b/esphome/components/bme680_bsec/__init__.py index 8ee463a59a..8a8d74b5f3 100644 --- a/esphome/components/bme680_bsec/__init__.py +++ b/esphome/components/bme680_bsec/__init__.py @@ -1,7 +1,7 @@ import esphome.codegen as cg from esphome.components import esp32, i2c import esphome.config_validation as cv -from esphome.const import CONF_ID, CONF_SAMPLE_RATE, CONF_TEMPERATURE_OFFSET +from esphome.const import CONF_ID, CONF_SAMPLE_RATE, CONF_TEMPERATURE_OFFSET, Framework CODEOWNERS = ["@trvrnrth"] DEPENDENCIES = ["i2c"] @@ -41,7 +41,7 @@ CONFIG_SCHEMA = cv.All( cv.Schema( { cv.GenerateID(): cv.declare_id(BME680BSECComponent), - cv.Optional(CONF_TEMPERATURE_OFFSET, default=0): cv.temperature, + cv.Optional(CONF_TEMPERATURE_OFFSET, default=0): cv.temperature_delta, cv.Optional(CONF_IAQ_MODE, default="STATIC"): cv.enum( IAQ_MODE_OPTIONS, upper=True ), @@ -56,7 +56,15 @@ CONFIG_SCHEMA = cv.All( ): cv.positive_time_period_minutes, } ).extend(i2c.i2c_device_schema(0x76)), - cv.only_with_arduino, + cv.only_with_framework( + frameworks=Framework.ARDUINO, + suggestions={ + Framework.ESP_IDF: ( + "bme68x_bsec2_i2c", + "sensor/bme68x_bsec2", + ) + }, + ), cv.Any( cv.only_on_esp8266, cv.All( diff --git a/esphome/components/bme680_bsec/bme680_bsec.cpp b/esphome/components/bme680_bsec/bme680_bsec.cpp index 562d39e7b5..d969c8fd98 100644 --- a/esphome/components/bme680_bsec/bme680_bsec.cpp +++ b/esphome/components/bme680_bsec/bme680_bsec.cpp @@ -15,8 +15,6 @@ std::vector uint8_t BME680BSECComponent::work_buffer_[BSEC_MAX_WORKBUFFER_SIZE] = {0}; void BME680BSECComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->device_id_.c_str()); - uint8_t new_idx = BME680BSECComponent::instances.size(); BME680BSECComponent::instances.push_back(this); diff --git a/esphome/components/bme68x_bsec2/__init__.py b/esphome/components/bme68x_bsec2/__init__.py index f4235b31b4..e421efb2d6 100644 --- a/esphome/components/bme68x_bsec2/__init__.py +++ b/esphome/components/bme68x_bsec2/__init__.py @@ -139,7 +139,7 @@ CONFIG_SCHEMA_BASE = ( cv.Optional(CONF_SUPPLY_VOLTAGE, default="3.3V"): cv.enum( VOLTAGE_OPTIONS, upper=True ), - cv.Optional(CONF_TEMPERATURE_OFFSET, default=0): cv.temperature, + cv.Optional(CONF_TEMPERATURE_OFFSET, default=0): cv.temperature_delta, cv.Optional( CONF_STATE_SAVE_INTERVAL, default="6hours" ): cv.positive_time_period_minutes, diff --git a/esphome/components/bme68x_bsec2/bme68x_bsec2.cpp b/esphome/components/bme68x_bsec2/bme68x_bsec2.cpp index a23711c4ca..f5dcfd65a1 100644 --- a/esphome/components/bme68x_bsec2/bme68x_bsec2.cpp +++ b/esphome/components/bme68x_bsec2/bme68x_bsec2.cpp @@ -21,8 +21,6 @@ static const char *const TAG = "bme68x_bsec2.sensor"; static const std::string IAQ_ACCURACY_STATES[4] = {"Stabilizing", "Uncertain", "Calibrating", "Calibrated"}; void BME68xBSEC2Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->bsec_status_ = bsec_init_m(&this->bsec_instance_); if (this->bsec_status_ != BSEC_OK) { this->mark_failed(); diff --git a/esphome/components/bmi160/bmi160.cpp b/esphome/components/bmi160/bmi160.cpp index aca42f1b52..4fcc3edb82 100644 --- a/esphome/components/bmi160/bmi160.cpp +++ b/esphome/components/bmi160/bmi160.cpp @@ -119,7 +119,6 @@ const float GRAVITY_EARTH = 9.80665f; void BMI160Component::internal_setup_(int stage) { switch (stage) { case 0: - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t chipid; if (!this->read_byte(BMI160_REGISTER_CHIPID, &chipid) || (chipid != 0b11010001)) { this->mark_failed(); @@ -204,7 +203,7 @@ void BMI160Component::dump_config() { i2c::ErrorCode BMI160Component::read_le_int16_(uint8_t reg, int16_t *value, uint8_t len) { uint8_t raw_data[len * 2]; // read using read_register because we have little-endian data, and read_bytes_16 will swap it - i2c::ErrorCode err = this->read_register(reg, raw_data, len * 2, true); + i2c::ErrorCode err = this->read_register(reg, raw_data, len * 2); if (err != i2c::ERROR_OK) { return err; } diff --git a/esphome/components/bmp085/bmp085.cpp b/esphome/components/bmp085/bmp085.cpp index 94dc61891b..657da34f9b 100644 --- a/esphome/components/bmp085/bmp085.cpp +++ b/esphome/components/bmp085/bmp085.cpp @@ -20,7 +20,6 @@ void BMP085Component::update() { this->set_timeout("temperature", 5, [this]() { this->read_temperature_(); }); } void BMP085Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t data[22]; if (!this->read_bytes(BMP085_REGISTER_AC1_H, data, 22)) { this->mark_failed(); diff --git a/esphome/components/bmp280_base/bmp280_base.cpp b/esphome/components/bmp280_base/bmp280_base.cpp index 94b8bd6540..39654f5875 100644 --- a/esphome/components/bmp280_base/bmp280_base.cpp +++ b/esphome/components/bmp280_base/bmp280_base.cpp @@ -2,6 +2,8 @@ #include "esphome/core/hal.h" #include "esphome/core/log.h" +#define BMP280_ERROR_WRONG_CHIP_ID "Wrong chip ID" + namespace esphome { namespace bmp280_base { @@ -57,30 +59,29 @@ static const char *iir_filter_to_str(BMP280IIRFilter filter) { } void BMP280Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t chip_id = 0; // Read the chip id twice, to work around a bug where the first read is 0. // https://community.st.com/t5/stm32-mcus-products/issue-with-reading-bmp280-chip-id-using-spi/td-p/691855 - if (!this->read_byte(0xD0, &chip_id)) { + if (!this->bmp_read_byte(0xD0, &chip_id)) { this->error_code_ = COMMUNICATION_FAILED; - this->mark_failed(); + this->mark_failed(ESP_LOG_MSG_COMM_FAIL); return; } - if (!this->read_byte(0xD0, &chip_id)) { + if (!this->bmp_read_byte(0xD0, &chip_id)) { this->error_code_ = COMMUNICATION_FAILED; - this->mark_failed(); + this->mark_failed(ESP_LOG_MSG_COMM_FAIL); return; } if (chip_id != 0x58) { this->error_code_ = WRONG_CHIP_ID; - this->mark_failed(); + this->mark_failed(BMP280_ERROR_WRONG_CHIP_ID); return; } // Send a soft reset. - if (!this->write_byte(BMP280_REGISTER_RESET, BMP280_SOFT_RESET)) { - this->mark_failed(); + if (!this->bmp_write_byte(BMP280_REGISTER_RESET, BMP280_SOFT_RESET)) { + this->mark_failed("Reset failed"); return; } // Wait until the NVM data has finished loading. @@ -88,15 +89,13 @@ void BMP280Component::setup() { uint8_t retry = 5; do { delay(2); - if (!this->read_byte(BMP280_REGISTER_STATUS, &status)) { - ESP_LOGW(TAG, "Error reading status register."); - this->mark_failed(); + if (!this->bmp_read_byte(BMP280_REGISTER_STATUS, &status)) { + this->mark_failed("Error reading status register"); return; } } while ((status & BMP280_STATUS_IM_UPDATE) && (--retry)); if (status & BMP280_STATUS_IM_UPDATE) { - ESP_LOGW(TAG, "Timeout loading NVM."); - this->mark_failed(); + this->mark_failed("Timeout loading NVM"); return; } @@ -116,15 +115,15 @@ void BMP280Component::setup() { this->calibration_.p9 = this->read_s16_le_(0x9E); uint8_t config_register = 0; - if (!this->read_byte(BMP280_REGISTER_CONFIG, &config_register)) { - this->mark_failed(); + if (!this->bmp_read_byte(BMP280_REGISTER_CONFIG, &config_register)) { + this->mark_failed("Read config"); return; } config_register &= ~0b11111100; config_register |= 0b000 << 5; // 0.5 ms standby time config_register |= (this->iir_filter_ & 0b111) << 2; - if (!this->write_byte(BMP280_REGISTER_CONFIG, config_register)) { - this->mark_failed(); + if (!this->bmp_write_byte(BMP280_REGISTER_CONFIG, config_register)) { + this->mark_failed("Write config"); return; } } @@ -135,7 +134,7 @@ void BMP280Component::dump_config() { ESP_LOGE(TAG, ESP_LOG_MSG_COMM_FAIL); break; case WRONG_CHIP_ID: - ESP_LOGE(TAG, "BMP280 has wrong chip ID! Is it a BME280?"); + ESP_LOGE(TAG, BMP280_ERROR_WRONG_CHIP_ID); break; case NONE: default: @@ -160,7 +159,7 @@ void BMP280Component::update() { meas_value |= (this->temperature_oversampling_ & 0b111) << 5; meas_value |= (this->pressure_oversampling_ & 0b111) << 2; meas_value |= 0b01; // Forced mode - if (!this->write_byte(BMP280_REGISTER_CONTROL, meas_value)) { + if (!this->bmp_write_byte(BMP280_REGISTER_CONTROL, meas_value)) { this->status_set_warning(); return; } @@ -173,13 +172,13 @@ void BMP280Component::update() { int32_t t_fine = 0; float temperature = this->read_temperature_(&t_fine); if (std::isnan(temperature)) { - ESP_LOGW(TAG, "Invalid temperature, cannot read pressure values."); + ESP_LOGW(TAG, "Invalid temperature"); this->status_set_warning(); return; } float pressure = this->read_pressure_(t_fine); - ESP_LOGD(TAG, "Got temperature=%.1f°C pressure=%.1fhPa", temperature, pressure); + ESP_LOGV(TAG, "Temperature=%.1f°C Pressure=%.1fhPa", temperature, pressure); if (this->temperature_sensor_ != nullptr) this->temperature_sensor_->publish_state(temperature); if (this->pressure_sensor_ != nullptr) @@ -189,9 +188,10 @@ void BMP280Component::update() { } float BMP280Component::read_temperature_(int32_t *t_fine) { - uint8_t data[3]; - if (!this->read_bytes(BMP280_REGISTER_TEMPDATA, data, 3)) + uint8_t data[3]{}; + if (!this->bmp_read_bytes(BMP280_REGISTER_TEMPDATA, data, 3)) return NAN; + ESP_LOGV(TAG, "Read temperature data, raw: %02X %02X %02X", data[0], data[1], data[2]); int32_t adc = ((data[0] & 0xFF) << 16) | ((data[1] & 0xFF) << 8) | (data[2] & 0xFF); adc >>= 4; if (adc == 0x80000) { @@ -213,7 +213,7 @@ float BMP280Component::read_temperature_(int32_t *t_fine) { float BMP280Component::read_pressure_(int32_t t_fine) { uint8_t data[3]; - if (!this->read_bytes(BMP280_REGISTER_PRESSUREDATA, data, 3)) + if (!this->bmp_read_bytes(BMP280_REGISTER_PRESSUREDATA, data, 3)) return NAN; int32_t adc = ((data[0] & 0xFF) << 16) | ((data[1] & 0xFF) << 8) | (data[2] & 0xFF); adc >>= 4; @@ -259,12 +259,12 @@ void BMP280Component::set_pressure_oversampling(BMP280Oversampling pressure_over void BMP280Component::set_iir_filter(BMP280IIRFilter iir_filter) { this->iir_filter_ = iir_filter; } uint8_t BMP280Component::read_u8_(uint8_t a_register) { uint8_t data = 0; - this->read_byte(a_register, &data); + this->bmp_read_byte(a_register, &data); return data; } uint16_t BMP280Component::read_u16_le_(uint8_t a_register) { uint16_t data = 0; - this->read_byte_16(a_register, &data); + this->bmp_read_byte_16(a_register, &data); return (data >> 8) | (data << 8); } int16_t BMP280Component::read_s16_le_(uint8_t a_register) { return this->read_u16_le_(a_register); } diff --git a/esphome/components/bmp280_base/bmp280_base.h b/esphome/components/bmp280_base/bmp280_base.h index 4b22e98f13..a47a794e96 100644 --- a/esphome/components/bmp280_base/bmp280_base.h +++ b/esphome/components/bmp280_base/bmp280_base.h @@ -67,12 +67,12 @@ class BMP280Component : public PollingComponent { float get_setup_priority() const override; void update() override; - virtual bool read_byte(uint8_t a_register, uint8_t *data) = 0; - virtual bool write_byte(uint8_t a_register, uint8_t data) = 0; - virtual bool read_bytes(uint8_t a_register, uint8_t *data, size_t len) = 0; - virtual bool read_byte_16(uint8_t a_register, uint16_t *data) = 0; - protected: + virtual bool bmp_read_byte(uint8_t a_register, uint8_t *data) = 0; + virtual bool bmp_write_byte(uint8_t a_register, uint8_t data) = 0; + virtual bool bmp_read_bytes(uint8_t a_register, uint8_t *data, size_t len) = 0; + virtual bool bmp_read_byte_16(uint8_t a_register, uint16_t *data) = 0; + /// Read the temperature value and store the calculated ambient temperature in t_fine. float read_temperature_(int32_t *t_fine); /// Read the pressure value in hPa using the provided t_fine value. diff --git a/esphome/components/bmp280_i2c/bmp280_i2c.cpp b/esphome/components/bmp280_i2c/bmp280_i2c.cpp index 04b8bd8b10..75d899008d 100644 --- a/esphome/components/bmp280_i2c/bmp280_i2c.cpp +++ b/esphome/components/bmp280_i2c/bmp280_i2c.cpp @@ -5,19 +5,6 @@ namespace esphome { namespace bmp280_i2c { -bool BMP280I2CComponent::read_byte(uint8_t a_register, uint8_t *data) { - return I2CDevice::read_byte(a_register, data); -}; -bool BMP280I2CComponent::write_byte(uint8_t a_register, uint8_t data) { - return I2CDevice::write_byte(a_register, data); -}; -bool BMP280I2CComponent::read_bytes(uint8_t a_register, uint8_t *data, size_t len) { - return I2CDevice::read_bytes(a_register, data, len); -}; -bool BMP280I2CComponent::read_byte_16(uint8_t a_register, uint16_t *data) { - return I2CDevice::read_byte_16(a_register, data); -}; - void BMP280I2CComponent::dump_config() { LOG_I2C_DEVICE(this); BMP280Component::dump_config(); diff --git a/esphome/components/bmp280_i2c/bmp280_i2c.h b/esphome/components/bmp280_i2c/bmp280_i2c.h index 66d78d788b..0ac956202b 100644 --- a/esphome/components/bmp280_i2c/bmp280_i2c.h +++ b/esphome/components/bmp280_i2c/bmp280_i2c.h @@ -11,10 +11,12 @@ static const char *const TAG = "bmp280_i2c.sensor"; /// This class implements support for the BMP280 Temperature+Pressure i2c sensor. class BMP280I2CComponent : public esphome::bmp280_base::BMP280Component, public i2c::I2CDevice { public: - bool read_byte(uint8_t a_register, uint8_t *data) override; - bool write_byte(uint8_t a_register, uint8_t data) override; - bool read_bytes(uint8_t a_register, uint8_t *data, size_t len) override; - bool read_byte_16(uint8_t a_register, uint16_t *data) override; + bool bmp_read_byte(uint8_t a_register, uint8_t *data) override { return read_byte(a_register, data); } + bool bmp_write_byte(uint8_t a_register, uint8_t data) override { return write_byte(a_register, data); } + bool bmp_read_bytes(uint8_t a_register, uint8_t *data, size_t len) override { + return read_bytes(a_register, data, len); + } + bool bmp_read_byte_16(uint8_t a_register, uint16_t *data) override { return read_byte_16(a_register, data); } void dump_config() override; }; diff --git a/esphome/components/bmp280_spi/bmp280_spi.cpp b/esphome/components/bmp280_spi/bmp280_spi.cpp index a35e829432..88983e77c3 100644 --- a/esphome/components/bmp280_spi/bmp280_spi.cpp +++ b/esphome/components/bmp280_spi/bmp280_spi.cpp @@ -28,7 +28,7 @@ void BMP280SPIComponent::setup() { // 0x77 is transferred, for read access, the byte 0xF7 is transferred. // https://www.bosch-sensortec.com/media/boschsensortec/downloads/datasheets/bst-bmp280-ds001.pdf -bool BMP280SPIComponent::read_byte(uint8_t a_register, uint8_t *data) { +bool BMP280SPIComponent::bmp_read_byte(uint8_t a_register, uint8_t *data) { this->enable(); this->transfer_byte(set_bit(a_register, 7)); *data = this->transfer_byte(0); @@ -36,7 +36,7 @@ bool BMP280SPIComponent::read_byte(uint8_t a_register, uint8_t *data) { return true; } -bool BMP280SPIComponent::write_byte(uint8_t a_register, uint8_t data) { +bool BMP280SPIComponent::bmp_write_byte(uint8_t a_register, uint8_t data) { this->enable(); this->transfer_byte(clear_bit(a_register, 7)); this->transfer_byte(data); @@ -44,7 +44,7 @@ bool BMP280SPIComponent::write_byte(uint8_t a_register, uint8_t data) { return true; } -bool BMP280SPIComponent::read_bytes(uint8_t a_register, uint8_t *data, size_t len) { +bool BMP280SPIComponent::bmp_read_bytes(uint8_t a_register, uint8_t *data, size_t len) { this->enable(); this->transfer_byte(set_bit(a_register, 7)); this->read_array(data, len); @@ -52,7 +52,7 @@ bool BMP280SPIComponent::read_bytes(uint8_t a_register, uint8_t *data, size_t le return true; } -bool BMP280SPIComponent::read_byte_16(uint8_t a_register, uint16_t *data) { +bool BMP280SPIComponent::bmp_read_byte_16(uint8_t a_register, uint16_t *data) { this->enable(); this->transfer_byte(set_bit(a_register, 7)); ((uint8_t *) data)[1] = this->transfer_byte(0); diff --git a/esphome/components/bmp280_spi/bmp280_spi.h b/esphome/components/bmp280_spi/bmp280_spi.h index dd226502f6..1bb7678e55 100644 --- a/esphome/components/bmp280_spi/bmp280_spi.h +++ b/esphome/components/bmp280_spi/bmp280_spi.h @@ -10,10 +10,10 @@ class BMP280SPIComponent : public esphome::bmp280_base::BMP280Component, public spi::SPIDevice { void setup() override; - bool read_byte(uint8_t a_register, uint8_t *data) override; - bool write_byte(uint8_t a_register, uint8_t data) override; - bool read_bytes(uint8_t a_register, uint8_t *data, size_t len) override; - bool read_byte_16(uint8_t a_register, uint16_t *data) override; + bool bmp_read_byte(uint8_t a_register, uint8_t *data) override; + bool bmp_write_byte(uint8_t a_register, uint8_t data) override; + bool bmp_read_bytes(uint8_t a_register, uint8_t *data, size_t len) override; + bool bmp_read_byte_16(uint8_t a_register, uint16_t *data) override; }; } // namespace bmp280_spi diff --git a/esphome/components/bmp3xx_base/bmp3xx_base.cpp b/esphome/components/bmp3xx_base/bmp3xx_base.cpp index 979f354cb2..acc28d4e85 100644 --- a/esphome/components/bmp3xx_base/bmp3xx_base.cpp +++ b/esphome/components/bmp3xx_base/bmp3xx_base.cpp @@ -70,7 +70,6 @@ static const LogString *iir_filter_to_str(IIRFilter filter) { void BMP3XXComponent::setup() { this->error_code_ = NONE; - ESP_LOGCONFIG(TAG, "Running setup"); // Call the Device base class "initialise" function if (!reset()) { ESP_LOGE(TAG, "Failed to reset"); diff --git a/esphome/components/bmp581/bmp581.cpp b/esphome/components/bmp581/bmp581.cpp index 2204a6af2e..301fc31df0 100644 --- a/esphome/components/bmp581/bmp581.cpp +++ b/esphome/components/bmp581/bmp581.cpp @@ -128,8 +128,6 @@ void BMP581Component::setup() { */ this->error_code_ = NONE; - ESP_LOGCONFIG(TAG, "Running setup"); - //////////////////// // 1) Soft reboot // //////////////////// diff --git a/esphome/components/bp1658cj/bp1658cj.cpp b/esphome/components/bp1658cj/bp1658cj.cpp index b502a738cd..b8ad5dc3d2 100644 --- a/esphome/components/bp1658cj/bp1658cj.cpp +++ b/esphome/components/bp1658cj/bp1658cj.cpp @@ -15,7 +15,6 @@ static const uint8_t BP1658CJ_ADDR_START_5CH = 0x30; static const uint8_t BP1658CJ_DELAY = 2; void BP1658CJ::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->data_pin_->setup(); this->data_pin_->digital_write(false); this->clock_pin_->setup(); diff --git a/esphome/components/bp5758d/bp5758d.cpp b/esphome/components/bp5758d/bp5758d.cpp index 797ddd919e..4f330b9c77 100644 --- a/esphome/components/bp5758d/bp5758d.cpp +++ b/esphome/components/bp5758d/bp5758d.cpp @@ -20,7 +20,6 @@ static const uint8_t BP5758D_ALL_DATA_CHANNEL_ENABLEMENT = 0b00011111; static const uint8_t BP5758D_DELAY = 2; void BP5758D::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->data_pin_->setup(); this->data_pin_->digital_write(false); delayMicroseconds(BP5758D_DELAY); diff --git a/esphome/components/button/__init__.py b/esphome/components/button/__init__.py index ed2670a5c5..e1ac875cb0 100644 --- a/esphome/components/button/__init__.py +++ b/esphome/components/button/__init__.py @@ -17,7 +17,7 @@ from esphome.const import ( DEVICE_CLASS_RESTART, DEVICE_CLASS_UPDATE, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass @@ -134,7 +134,6 @@ async def button_press_to_code(config, action_id, template_arg, args): return cg.new_Pvariable(action_id, template_arg, paren) -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): cg.add_global(button_ns.using) - cg.add_define("USE_BUTTON") diff --git a/esphome/components/button/button.cpp b/esphome/components/button/button.cpp index 4c4cb7740c..c968d31088 100644 --- a/esphome/components/button/button.cpp +++ b/esphome/components/button/button.cpp @@ -6,6 +6,19 @@ namespace button { static const char *const TAG = "button"; +// Function implementation of LOG_BUTTON macro to reduce code size +void log_button(const char *tag, const char *prefix, const char *type, Button *obj) { + if (obj == nullptr) { + return; + } + + ESP_LOGCONFIG(tag, "%s%s '%s'", prefix, type, obj->get_name().c_str()); + + if (!obj->get_icon_ref().empty()) { + ESP_LOGCONFIG(tag, "%s Icon: '%s'", prefix, obj->get_icon_ref().c_str()); + } +} + void Button::press() { ESP_LOGD(TAG, "'%s' Pressed.", this->get_name().c_str()); this->press_action(); diff --git a/esphome/components/button/button.h b/esphome/components/button/button.h index 9488eca221..75b76f9dcf 100644 --- a/esphome/components/button/button.h +++ b/esphome/components/button/button.h @@ -7,13 +7,10 @@ namespace esphome { namespace button { -#define LOG_BUTTON(prefix, type, obj) \ - if ((obj) != nullptr) { \ - ESP_LOGCONFIG(TAG, "%s%s '%s'", prefix, LOG_STR_LITERAL(type), (obj)->get_name().c_str()); \ - if (!(obj)->get_icon().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon().c_str()); \ - } \ - } +class Button; +void log_button(const char *tag, const char *prefix, const char *type, Button *obj); + +#define LOG_BUTTON(prefix, type, obj) log_button(TAG, prefix, LOG_STR_LITERAL(type), obj) #define SUB_BUTTON(name) \ protected: \ diff --git a/esphome/components/camera/buffer.h b/esphome/components/camera/buffer.h new file mode 100644 index 0000000000..f860877b94 --- /dev/null +++ b/esphome/components/camera/buffer.h @@ -0,0 +1,18 @@ +#pragma once + +#include +#include + +namespace esphome::camera { + +/// Interface for a generic buffer that stores image data. +class Buffer { + public: + /// Returns a pointer to the buffer's data. + virtual uint8_t *get_data_buffer() = 0; + /// Returns the length of the buffer in bytes. + virtual size_t get_data_length() = 0; + virtual ~Buffer() = default; +}; + +} // namespace esphome::camera diff --git a/esphome/components/camera/buffer_impl.cpp b/esphome/components/camera/buffer_impl.cpp new file mode 100644 index 0000000000..d17a4e2707 --- /dev/null +++ b/esphome/components/camera/buffer_impl.cpp @@ -0,0 +1,20 @@ +#include "buffer_impl.h" + +namespace esphome::camera { + +BufferImpl::BufferImpl(size_t size) { + this->data_ = this->allocator_.allocate(size); + this->size_ = size; +} + +BufferImpl::BufferImpl(CameraImageSpec *spec) { + this->data_ = this->allocator_.allocate(spec->bytes_per_image()); + this->size_ = spec->bytes_per_image(); +} + +BufferImpl::~BufferImpl() { + if (this->data_ != nullptr) + this->allocator_.deallocate(this->data_, this->size_); +} + +} // namespace esphome::camera diff --git a/esphome/components/camera/buffer_impl.h b/esphome/components/camera/buffer_impl.h new file mode 100644 index 0000000000..46398295fa --- /dev/null +++ b/esphome/components/camera/buffer_impl.h @@ -0,0 +1,26 @@ +#pragma once + +#include "buffer.h" +#include "camera.h" + +namespace esphome::camera { + +/// Default implementation of Buffer Interface. +/// Uses a RAMAllocator for memory reservation. +class BufferImpl : public Buffer { + public: + explicit BufferImpl(size_t size); + explicit BufferImpl(CameraImageSpec *spec); + // -------- Buffer -------- + uint8_t *get_data_buffer() override { return data_; } + size_t get_data_length() override { return size_; } + // ------------------------ + ~BufferImpl() override; + + protected: + RAMAllocator allocator_; + size_t size_{}; + uint8_t *data_{}; +}; + +} // namespace esphome::camera diff --git a/esphome/components/camera/camera.h b/esphome/components/camera/camera.h index fb9da58cc1..c28a756a06 100644 --- a/esphome/components/camera/camera.h +++ b/esphome/components/camera/camera.h @@ -15,6 +15,26 @@ namespace camera { */ enum CameraRequester : uint8_t { IDLE, API_REQUESTER, WEB_REQUESTER }; +/// Enumeration of different pixel formats. +enum PixelFormat : uint8_t { + PIXEL_FORMAT_GRAYSCALE = 0, ///< 8-bit grayscale. + PIXEL_FORMAT_RGB565, ///< 16-bit RGB (5-6-5). + PIXEL_FORMAT_BGR888, ///< RGB pixel data in 8-bit format, stored as B, G, R (1 byte each). +}; + +/// Returns string name for a given PixelFormat. +inline const char *to_string(PixelFormat format) { + switch (format) { + case PIXEL_FORMAT_GRAYSCALE: + return "PIXEL_FORMAT_GRAYSCALE"; + case PIXEL_FORMAT_RGB565: + return "PIXEL_FORMAT_RGB565"; + case PIXEL_FORMAT_BGR888: + return "PIXEL_FORMAT_BGR888"; + } + return "PIXEL_FORMAT_UNKNOWN"; +} + /** Abstract camera image base class. * Encapsulates the JPEG encoded data and it is shared among * all connected clients. @@ -43,6 +63,29 @@ class CameraImageReader { virtual ~CameraImageReader() {} }; +/// Specification of a caputured camera image. +/// This struct defines the format and size details for images captured +/// or processed by a camera component. +struct CameraImageSpec { + uint16_t width; + uint16_t height; + PixelFormat format; + size_t bytes_per_pixel() { + switch (format) { + case PIXEL_FORMAT_GRAYSCALE: + return 1; + case PIXEL_FORMAT_RGB565: + return 2; + case PIXEL_FORMAT_BGR888: + return 3; + } + + return 1; + } + size_t bytes_per_row() { return bytes_per_pixel() * width; } + size_t bytes_per_image() { return bytes_per_pixel() * width * height; } +}; + /** Abstract camera base class. Collaborates with API. * 1) API server starts and installs callback (add_image_callback) * which is called by the camera when a new image is available. diff --git a/esphome/components/camera/encoder.h b/esphome/components/camera/encoder.h new file mode 100644 index 0000000000..17ce828d23 --- /dev/null +++ b/esphome/components/camera/encoder.h @@ -0,0 +1,69 @@ +#pragma once + +#include "buffer.h" +#include "camera.h" + +namespace esphome::camera { + +/// Result codes from the encoder used to control camera pipeline flow. +enum EncoderError : uint8_t { + ENCODER_ERROR_SUCCESS = 0, ///< Encoding succeeded, continue pipeline normally. + ENCODER_ERROR_SKIP_FRAME, ///< Skip current frame, try again on next frame. + ENCODER_ERROR_RETRY_FRAME, ///< Retry current frame, after buffer growth or for incremental encoding. + ENCODER_ERROR_CONFIGURATION ///< Fatal config error, shut down pipeline. +}; + +/// Converts EncoderError to string. +inline const char *to_string(EncoderError error) { + switch (error) { + case ENCODER_ERROR_SUCCESS: + return "ENCODER_ERROR_SUCCESS"; + case ENCODER_ERROR_SKIP_FRAME: + return "ENCODER_ERROR_SKIP_FRAME"; + case ENCODER_ERROR_RETRY_FRAME: + return "ENCODER_ERROR_RETRY_FRAME"; + case ENCODER_ERROR_CONFIGURATION: + return "ENCODER_ERROR_CONFIGURATION"; + } + return "ENCODER_ERROR_INVALID"; +} + +/// Interface for an encoder buffer supporting resizing and variable-length data. +class EncoderBuffer { + public: + /// Sets logical buffer size, reallocates if needed. + /// @param size Required size in bytes. + /// @return true on success, false on allocation failure. + virtual bool set_buffer_size(size_t size) = 0; + + /// Returns a pointer to the buffer data. + virtual uint8_t *get_data() const = 0; + + /// Returns number of bytes currently used. + virtual size_t get_size() const = 0; + + /// Returns total allocated buffer size. + virtual size_t get_max_size() const = 0; + + virtual ~EncoderBuffer() = default; +}; + +/// Interface for image encoders used in a camera pipeline. +class Encoder { + public: + /// Encodes pixel data from a previous camera pipeline stage. + /// @param spec Specification of the input pixel data. + /// @param pixels Image pixels in RGB or grayscale format, as specified in @p spec. + /// @return EncoderError Indicating the result of the encoding operation. + virtual EncoderError encode_pixels(CameraImageSpec *spec, Buffer *pixels) = 0; + + /// Returns the encoder's output buffer. + /// @return Pointer to an EncoderBuffer containing encoded data. + virtual EncoderBuffer *get_output_buffer() = 0; + + /// Prints the encoder's configuration to the log. + virtual void dump_config() = 0; + virtual ~Encoder() = default; +}; + +} // namespace esphome::camera diff --git a/esphome/components/camera_encoder/__init__.py b/esphome/components/camera_encoder/__init__.py new file mode 100644 index 0000000000..89181d27b4 --- /dev/null +++ b/esphome/components/camera_encoder/__init__.py @@ -0,0 +1,60 @@ +import esphome.codegen as cg +from esphome.components.esp32 import add_idf_component +import esphome.config_validation as cv +from esphome.const import CONF_BUFFER_SIZE, CONF_ID, CONF_TYPE +from esphome.types import ConfigType + +CODEOWNERS = ["@DT-art1"] + +AUTO_LOAD = ["camera"] + +CONF_BUFFER_EXPAND_SIZE = "buffer_expand_size" +CONF_ENCODER_BUFFER_ID = "encoder_buffer_id" +CONF_QUALITY = "quality" + +ESP32_CAMERA_ENCODER = "esp32_camera" + +camera_ns = cg.esphome_ns.namespace("camera") +camera_encoder_ns = cg.esphome_ns.namespace("camera_encoder") + +Encoder = camera_ns.class_("Encoder") +EncoderBufferImpl = camera_encoder_ns.class_("EncoderBufferImpl") + +ESP32CameraJPEGEncoder = camera_encoder_ns.class_("ESP32CameraJPEGEncoder", Encoder) + +MAX_JPEG_BUFFER_SIZE_2MB = 2 * 1024 * 1024 + +ESP32_CAMERA_ENCODER_SCHEMA = cv.Schema( + { + cv.GenerateID(): cv.declare_id(ESP32CameraJPEGEncoder), + cv.Optional(CONF_QUALITY, default=80): cv.int_range(1, 100), + cv.Optional(CONF_BUFFER_SIZE, default=4096): cv.int_range( + 1024, MAX_JPEG_BUFFER_SIZE_2MB + ), + cv.Optional(CONF_BUFFER_EXPAND_SIZE, default=1024): cv.int_range( + 0, MAX_JPEG_BUFFER_SIZE_2MB + ), + cv.GenerateID(CONF_ENCODER_BUFFER_ID): cv.declare_id(EncoderBufferImpl), + } +) + +CONFIG_SCHEMA = cv.typed_schema( + { + ESP32_CAMERA_ENCODER: ESP32_CAMERA_ENCODER_SCHEMA, + }, + default_type=ESP32_CAMERA_ENCODER, +) + + +async def to_code(config: ConfigType) -> None: + buffer = cg.new_Pvariable(config[CONF_ENCODER_BUFFER_ID]) + cg.add(buffer.set_buffer_size(config[CONF_BUFFER_SIZE])) + if config[CONF_TYPE] == ESP32_CAMERA_ENCODER: + add_idf_component(name="espressif/esp32-camera", ref="2.1.1") + cg.add_define("USE_ESP32_CAMERA_JPEG_ENCODER") + var = cg.new_Pvariable( + config[CONF_ID], + config[CONF_QUALITY], + buffer, + ) + cg.add(var.set_buffer_expand_size(config[CONF_BUFFER_EXPAND_SIZE])) diff --git a/esphome/components/camera_encoder/encoder_buffer_impl.cpp b/esphome/components/camera_encoder/encoder_buffer_impl.cpp new file mode 100644 index 0000000000..db84026496 --- /dev/null +++ b/esphome/components/camera_encoder/encoder_buffer_impl.cpp @@ -0,0 +1,23 @@ +#include "encoder_buffer_impl.h" + +namespace esphome::camera_encoder { + +bool EncoderBufferImpl::set_buffer_size(size_t size) { + if (size > this->capacity_) { + uint8_t *p = this->allocator_.reallocate(this->data_, size); + if (p == nullptr) + return false; + + this->data_ = p; + this->capacity_ = size; + } + this->size_ = size; + return true; +} + +EncoderBufferImpl::~EncoderBufferImpl() { + if (this->data_ != nullptr) + this->allocator_.deallocate(this->data_, this->capacity_); +} + +} // namespace esphome::camera_encoder diff --git a/esphome/components/camera_encoder/encoder_buffer_impl.h b/esphome/components/camera_encoder/encoder_buffer_impl.h new file mode 100644 index 0000000000..13eccb7d56 --- /dev/null +++ b/esphome/components/camera_encoder/encoder_buffer_impl.h @@ -0,0 +1,25 @@ +#pragma once + +#include "esphome/components/camera/encoder.h" +#include "esphome/core/helpers.h" + +namespace esphome::camera_encoder { + +class EncoderBufferImpl : public camera::EncoderBuffer { + public: + // --- EncoderBuffer --- + bool set_buffer_size(size_t size) override; + uint8_t *get_data() const override { return this->data_; } + size_t get_size() const override { return this->size_; } + size_t get_max_size() const override { return this->capacity_; } + // ---------------------- + ~EncoderBufferImpl() override; + + protected: + RAMAllocator allocator_; + size_t capacity_{}; + size_t size_{}; + uint8_t *data_{}; +}; + +} // namespace esphome::camera_encoder diff --git a/esphome/components/camera_encoder/esp32_camera_jpeg_encoder.cpp b/esphome/components/camera_encoder/esp32_camera_jpeg_encoder.cpp new file mode 100644 index 0000000000..55a3f0b96c --- /dev/null +++ b/esphome/components/camera_encoder/esp32_camera_jpeg_encoder.cpp @@ -0,0 +1,84 @@ +#include "esphome/core/defines.h" + +#ifdef USE_ESP32_CAMERA_JPEG_ENCODER + +#include "esp32_camera_jpeg_encoder.h" + +namespace esphome::camera_encoder { + +static const char *const TAG = "camera_encoder"; + +ESP32CameraJPEGEncoder::ESP32CameraJPEGEncoder(uint8_t quality, camera::EncoderBuffer *output) { + this->quality_ = quality; + this->output_ = output; +} + +camera::EncoderError ESP32CameraJPEGEncoder::encode_pixels(camera::CameraImageSpec *spec, camera::Buffer *pixels) { + this->bytes_written_ = 0; + this->out_of_output_memory_ = false; + bool success = fmt2jpg_cb(pixels->get_data_buffer(), pixels->get_data_length(), spec->width, spec->height, + to_internal_(spec->format), this->quality_, callback, this); + + if (!success) + return camera::ENCODER_ERROR_CONFIGURATION; + + if (this->out_of_output_memory_) { + if (this->buffer_expand_size_ <= 0) + return camera::ENCODER_ERROR_SKIP_FRAME; + + size_t current_size = this->output_->get_max_size(); + size_t new_size = this->output_->get_max_size() + this->buffer_expand_size_; + if (!this->output_->set_buffer_size(new_size)) { + ESP_LOGE(TAG, "Failed to expand output buffer."); + this->buffer_expand_size_ = 0; + return camera::ENCODER_ERROR_SKIP_FRAME; + } + + ESP_LOGD(TAG, "Output buffer expanded (%u -> %u).", current_size, this->output_->get_max_size()); + return camera::ENCODER_ERROR_RETRY_FRAME; + } + + this->output_->set_buffer_size(this->bytes_written_); + return camera::ENCODER_ERROR_SUCCESS; +} + +void ESP32CameraJPEGEncoder::dump_config() { + ESP_LOGCONFIG(TAG, + "ESP32 Camera JPEG Encoder:\n" + " Size: %zu\n" + " Quality: %d\n" + " Expand: %d\n", + this->output_->get_max_size(), this->quality_, this->buffer_expand_size_); +} + +size_t ESP32CameraJPEGEncoder::callback(void *arg, size_t index, const void *data, size_t len) { + ESP32CameraJPEGEncoder *that = reinterpret_cast(arg); + uint8_t *buffer = that->output_->get_data(); + size_t buffer_length = that->output_->get_max_size(); + if (index + len > buffer_length) { + that->out_of_output_memory_ = true; + return 0; + } + + std::memcpy(&buffer[index], data, len); + that->bytes_written_ += len; + return len; +} + +pixformat_t ESP32CameraJPEGEncoder::to_internal_(camera::PixelFormat format) { + switch (format) { + case camera::PIXEL_FORMAT_GRAYSCALE: + return PIXFORMAT_GRAYSCALE; + case camera::PIXEL_FORMAT_RGB565: + return PIXFORMAT_RGB565; + // Internal representation for RGB is in byte order: B, G, R + case camera::PIXEL_FORMAT_BGR888: + return PIXFORMAT_RGB888; + } + + return PIXFORMAT_GRAYSCALE; +} + +} // namespace esphome::camera_encoder + +#endif diff --git a/esphome/components/camera_encoder/esp32_camera_jpeg_encoder.h b/esphome/components/camera_encoder/esp32_camera_jpeg_encoder.h new file mode 100644 index 0000000000..0ede366e73 --- /dev/null +++ b/esphome/components/camera_encoder/esp32_camera_jpeg_encoder.h @@ -0,0 +1,41 @@ +#pragma once + +#include "esphome/core/defines.h" + +#ifdef USE_ESP32_CAMERA_JPEG_ENCODER + +#include + +#include "esphome/components/camera/encoder.h" + +namespace esphome::camera_encoder { + +/// Encoder that uses the software-based JPEG implementation from Espressif's esp32-camera component. +class ESP32CameraJPEGEncoder : public camera::Encoder { + public: + /// Constructs a ESP32CameraJPEGEncoder instance. + /// @param quality Sets the quality of the encoded image (1-100). + /// @param output Pointer to preallocated output buffer. + ESP32CameraJPEGEncoder(uint8_t quality, camera::EncoderBuffer *output); + /// Sets the number of bytes to expand the output buffer on underflow during encoding. + /// @param buffer_expand_size Number of bytes to expand the buffer. + void set_buffer_expand_size(size_t buffer_expand_size) { this->buffer_expand_size_ = buffer_expand_size; } + // -------- Encoder -------- + camera::EncoderError encode_pixels(camera::CameraImageSpec *spec, camera::Buffer *pixels) override; + camera::EncoderBuffer *get_output_buffer() override { return output_; } + void dump_config() override; + // ------------------------- + protected: + static size_t callback(void *arg, size_t index, const void *data, size_t len); + pixformat_t to_internal_(camera::PixelFormat format); + + camera::EncoderBuffer *output_{}; + size_t buffer_expand_size_{}; + size_t bytes_written_{}; + uint8_t quality_{}; + bool out_of_output_memory_{}; +}; + +} // namespace esphome::camera_encoder + +#endif diff --git a/esphome/components/canbus/__init__.py b/esphome/components/canbus/__init__.py index cdb57fd481..e1de1eb2f2 100644 --- a/esphome/components/canbus/__init__.py +++ b/esphome/components/canbus/__init__.py @@ -22,9 +22,8 @@ def validate_id(config): if CONF_CAN_ID in config: can_id = config[CONF_CAN_ID] id_ext = config[CONF_USE_EXTENDED_ID] - if not id_ext: - if can_id > 0x7FF: - raise cv.Invalid("Standard IDs must be 11 Bit (0x000-0x7ff / 0-2047)") + if not id_ext and can_id > 0x7FF: + raise cv.Invalid("Standard IDs must be 11 Bit (0x000-0x7ff / 0-2047)") return config diff --git a/esphome/components/canbus/canbus.cpp b/esphome/components/canbus/canbus.cpp index d08558037e..e208b0fd66 100644 --- a/esphome/components/canbus/canbus.cpp +++ b/esphome/components/canbus/canbus.cpp @@ -7,7 +7,6 @@ namespace canbus { static const char *const TAG = "canbus"; void Canbus::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (!this->setup_internal()) { ESP_LOGE(TAG, "setup error!"); this->mark_failed(); @@ -22,8 +21,8 @@ void Canbus::dump_config() { } } -void Canbus::send_data(uint32_t can_id, bool use_extended_id, bool remote_transmission_request, - const std::vector &data) { +canbus::Error Canbus::send_data(uint32_t can_id, bool use_extended_id, bool remote_transmission_request, + const std::vector &data) { struct CanFrame can_message; uint8_t size = static_cast(data.size()); @@ -46,13 +45,15 @@ void Canbus::send_data(uint32_t can_id, bool use_extended_id, bool remote_transm ESP_LOGVV(TAG, " data[%d]=%02x", i, can_message.data[i]); } - if (this->send_message(&can_message) != canbus::ERROR_OK) { + canbus::Error error = this->send_message(&can_message); + if (error != canbus::ERROR_OK) { if (use_extended_id) { - ESP_LOGW(TAG, "send to extended id=0x%08" PRIx32 " failed!", can_id); + ESP_LOGW(TAG, "send to extended id=0x%08" PRIx32 " failed with error %d!", can_id, error); } else { - ESP_LOGW(TAG, "send to standard id=0x%03" PRIx32 " failed!", can_id); + ESP_LOGW(TAG, "send to standard id=0x%03" PRIx32 " failed with error %d!", can_id, error); } } + return error; } void Canbus::add_trigger(CanbusTrigger *trigger) { diff --git a/esphome/components/canbus/canbus.h b/esphome/components/canbus/canbus.h index 7319bfb4ad..51d7c0830a 100644 --- a/esphome/components/canbus/canbus.h +++ b/esphome/components/canbus/canbus.h @@ -70,11 +70,11 @@ class Canbus : public Component { float get_setup_priority() const override { return setup_priority::HARDWARE; } void loop() override; - void send_data(uint32_t can_id, bool use_extended_id, bool remote_transmission_request, - const std::vector &data); - void send_data(uint32_t can_id, bool use_extended_id, const std::vector &data) { + canbus::Error send_data(uint32_t can_id, bool use_extended_id, bool remote_transmission_request, + const std::vector &data); + canbus::Error send_data(uint32_t can_id, bool use_extended_id, const std::vector &data) { // for backwards compatibility only - this->send_data(can_id, use_extended_id, false, data); + return this->send_data(can_id, use_extended_id, false, data); } void set_can_id(uint32_t can_id) { this->can_id_ = can_id; } void set_use_extended_id(bool use_extended_id) { this->use_extended_id_ = use_extended_id; } @@ -105,9 +105,9 @@ class Canbus : public Component { CallbackManager &data)> callback_manager_{}; - virtual bool setup_internal(); - virtual Error send_message(struct CanFrame *frame); - virtual Error read_message(struct CanFrame *frame); + virtual bool setup_internal() = 0; + virtual Error send_message(struct CanFrame *frame) = 0; + virtual Error read_message(struct CanFrame *frame) = 0; }; template class CanbusSendAction : public Action, public Parented { diff --git a/esphome/components/cap1188/cap1188.cpp b/esphome/components/cap1188/cap1188.cpp index af167deb99..683e5cf487 100644 --- a/esphome/components/cap1188/cap1188.cpp +++ b/esphome/components/cap1188/cap1188.cpp @@ -8,19 +8,30 @@ namespace cap1188 { static const char *const TAG = "cap1188"; void CAP1188Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); + this->disable_loop(); - // Reset device using the reset pin - if (this->reset_pin_ != nullptr) { - this->reset_pin_->setup(); - this->reset_pin_->digital_write(false); - delay(100); // NOLINT - this->reset_pin_->digital_write(true); - delay(100); // NOLINT - this->reset_pin_->digital_write(false); - delay(100); // NOLINT + // no reset pin + if (this->reset_pin_ == nullptr) { + this->finish_setup_(); + return; } + // reset pin configured so reset before finishing setup + this->reset_pin_->setup(); + this->reset_pin_->digital_write(false); + // delay after reset pin write + this->set_timeout(100, [this]() { + this->reset_pin_->digital_write(true); + // delay after reset pin write + this->set_timeout(100, [this]() { + this->reset_pin_->digital_write(false); + // delay after reset pin write + this->set_timeout(100, [this]() { this->finish_setup_(); }); + }); + }); +} + +void CAP1188Component::finish_setup_() { // Check if CAP1188 is actually connected this->read_byte(CAP1188_PRODUCT_ID, &this->cap1188_product_id_); this->read_byte(CAP1188_MANUFACTURE_ID, &this->cap1188_manufacture_id_); @@ -46,6 +57,9 @@ void CAP1188Component::setup() { // Speed up a bit this->write_byte(CAP1188_STAND_BY_CONFIGURATION, 0x30); + + // Setup successful, so enable loop + this->enable_loop(); } void CAP1188Component::dump_config() { diff --git a/esphome/components/cap1188/cap1188.h b/esphome/components/cap1188/cap1188.h index baefd1c48f..297c601b05 100644 --- a/esphome/components/cap1188/cap1188.h +++ b/esphome/components/cap1188/cap1188.h @@ -49,6 +49,8 @@ class CAP1188Component : public Component, public i2c::I2CDevice { void loop() override; protected: + void finish_setup_(); + std::vector channels_{}; uint8_t touch_threshold_{0x20}; uint8_t allow_multiple_touches_{0x80}; diff --git a/esphome/components/captive_portal/__init__.py b/esphome/components/captive_portal/__init__.py index 7e8afd8fab..99acb76bcf 100644 --- a/esphome/components/captive_portal/__init__.py +++ b/esphome/components/captive_portal/__init__.py @@ -1,6 +1,7 @@ import esphome.codegen as cg from esphome.components import web_server_base from esphome.components.web_server_base import CONF_WEB_SERVER_BASE_ID +from esphome.config_helpers import filter_source_files_from_platform import esphome.config_validation as cv from esphome.const import ( CONF_ID, @@ -9,12 +10,21 @@ from esphome.const import ( PLATFORM_ESP8266, PLATFORM_LN882X, PLATFORM_RTL87XX, + PlatformFramework, ) from esphome.core import CORE, coroutine_with_priority +from esphome.coroutine import CoroPriority + + +def AUTO_LOAD() -> list[str]: + auto_load = ["web_server_base", "ota.web_server"] + if CORE.using_esp_idf: + auto_load.append("socket") + return auto_load + -AUTO_LOAD = ["web_server_base", "ota.web_server"] DEPENDENCIES = ["wifi"] -CODEOWNERS = ["@OttoWinter"] +CODEOWNERS = ["@esphome/core"] captive_portal_ns = cg.esphome_ns.namespace("captive_portal") CaptivePortal = captive_portal_ns.class_("CaptivePortal", cg.Component) @@ -40,7 +50,7 @@ CONFIG_SCHEMA = cv.All( ) -@coroutine_with_priority(64.0) +@coroutine_with_priority(CoroPriority.CAPTIVE_PORTAL) async def to_code(config): paren = await cg.get_variable(config[CONF_WEB_SERVER_BASE_ID]) @@ -57,3 +67,11 @@ async def to_code(config): cg.add_library("DNSServer", None) if CORE.is_libretiny: cg.add_library("DNSServer", None) + + +# Only compile the ESP-IDF DNS server when using ESP-IDF framework +FILTER_SOURCE_FILES = filter_source_files_from_platform( + { + "dns_server_esp32_idf.cpp": {PlatformFramework.ESP32_IDF}, + } +) diff --git a/esphome/components/captive_portal/captive_index.h b/esphome/components/captive_portal/captive_index.h index 8835762fb3..3122f27558 100644 --- a/esphome/components/captive_portal/captive_index.h +++ b/esphome/components/captive_portal/captive_index.h @@ -7,103 +7,83 @@ namespace esphome { namespace captive_portal { const uint8_t INDEX_GZ[] PROGMEM = { - 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0xdd, 0x58, 0x6d, 0x6f, 0xdb, 0x38, 0x12, 0xfe, 0xde, - 0x5f, 0x31, 0xa7, 0x36, 0x6b, 0x6b, 0x1b, 0x51, 0x22, 0xe5, 0xb7, 0xd8, 0x92, 0x16, 0x69, 0xae, 0x8b, 0x5d, 0xa0, - 0xdd, 0x2d, 0x90, 0x6c, 0xef, 0x43, 0x51, 0x20, 0xb4, 0x34, 0xb2, 0xd8, 0x48, 0xa4, 0x4e, 0xa4, 0x5f, 0x52, 0xc3, - 0xf7, 0xdb, 0x0f, 0x94, 0x6c, 0xc7, 0xe9, 0x35, 0x87, 0xeb, 0xe2, 0x0e, 0x87, 0xdd, 0x18, 0x21, 0x86, 0xe4, 0xcc, - 0x70, 0xe6, 0xf1, 0x0c, 0x67, 0xcc, 0xe8, 0x2f, 0x99, 0x4a, 0xcd, 0x7d, 0x8d, 0x50, 0x98, 0xaa, 0x4c, 0x22, 0x3b, - 0x42, 0xc9, 0xe5, 0x22, 0x46, 0x99, 0x44, 0x05, 0xf2, 0x2c, 0x89, 0x2a, 0x34, 0x1c, 0xd2, 0x82, 0x37, 0x1a, 0x4d, - 0xfc, 0xdb, 0xcd, 0x8f, 0xde, 0x04, 0xfc, 0x24, 0x2a, 0x85, 0xbc, 0x83, 0x06, 0xcb, 0x58, 0xa4, 0x4a, 0x42, 0xd1, - 0x60, 0x1e, 0x67, 0xdc, 0xf0, 0xa9, 0xa8, 0xf8, 0x02, 0x2d, 0x43, 0x2b, 0x26, 0x79, 0x85, 0xf1, 0x4a, 0xe0, 0xba, - 0x56, 0x8d, 0x81, 0x54, 0x49, 0x83, 0xd2, 0xc4, 0xce, 0x5a, 0x64, 0xa6, 0x88, 0x33, 0x5c, 0x89, 0x14, 0xbd, 0x76, - 0x72, 0x2e, 0xa4, 0x30, 0x82, 0x97, 0x9e, 0x4e, 0x79, 0x89, 0x31, 0x3d, 0x5f, 0x6a, 0x6c, 0xda, 0x09, 0x9f, 0x97, - 0x18, 0x4b, 0xe5, 0xf8, 0x49, 0xa4, 0xd3, 0x46, 0xd4, 0x06, 0xac, 0xbd, 0x71, 0xa5, 0xb2, 0x65, 0x89, 0x89, 0xef, - 0x73, 0xad, 0xd1, 0x68, 0x5f, 0xc8, 0x0c, 0x37, 0x64, 0x14, 0x86, 0x29, 0xe3, 0xe3, 0x9c, 0x7c, 0xd2, 0xcf, 0x32, - 0x95, 0x2e, 0x2b, 0x94, 0x86, 0x94, 0x2a, 0xe5, 0x46, 0x28, 0x49, 0x34, 0xf2, 0x26, 0x2d, 0xe2, 0x38, 0x76, 0x7e, - 0xd0, 0x7c, 0x85, 0xce, 0x77, 0xdf, 0xf5, 0x8f, 0x4c, 0x0b, 0x34, 0xaf, 0x4b, 0xb4, 0xa4, 0x7e, 0x75, 0x7f, 0xc3, - 0x17, 0xbf, 0xf0, 0x0a, 0xfb, 0x0e, 0xd7, 0x22, 0x43, 0xc7, 0xfd, 0x10, 0x7c, 0x24, 0xda, 0xdc, 0x97, 0x48, 0x32, - 0xa1, 0xeb, 0x92, 0xdf, 0xc7, 0xce, 0xbc, 0x54, 0xe9, 0x9d, 0xe3, 0xce, 0xf2, 0xa5, 0x4c, 0xad, 0x72, 0xd0, 0x7d, - 0x74, 0xb7, 0x25, 0x1a, 0x30, 0xf1, 0x5b, 0x6e, 0x0a, 0x52, 0xf1, 0x4d, 0xbf, 0x23, 0x84, 0xec, 0xb3, 0xef, 0xfb, - 0xf8, 0x92, 0x06, 0x81, 0x7b, 0xde, 0x0e, 0x81, 0xeb, 0xd3, 0x20, 0x98, 0x35, 0x68, 0x96, 0x8d, 0x04, 0xde, 0xbf, - 0x8d, 0x6a, 0x6e, 0x0a, 0xc8, 0x62, 0xa7, 0xa2, 0x8c, 0x04, 0xc1, 0x04, 0xe8, 0x05, 0x61, 0x43, 0x8f, 0x52, 0x12, - 0x7a, 0x74, 0x98, 0x8e, 0xbd, 0x21, 0xd0, 0x81, 0x37, 0x04, 0xc6, 0xc8, 0x10, 0x82, 0xcf, 0x0e, 0xe4, 0xa2, 0x2c, - 0x63, 0x47, 0x2a, 0x89, 0x0e, 0x68, 0xd3, 0xa8, 0x3b, 0x8c, 0x9d, 0x74, 0xd9, 0x34, 0x28, 0xcd, 0x95, 0x2a, 0x55, - 0xe3, 0xf8, 0xc9, 0x33, 0x78, 0xf4, 0xf7, 0xcd, 0x47, 0x98, 0x86, 0x4b, 0x9d, 0xab, 0xa6, 0x8a, 0x9d, 0xf6, 0x4b, - 0xe9, 0xbf, 0xd8, 0x9a, 0x1d, 0xd8, 0xc1, 0x3d, 0xd9, 0xf4, 0x54, 0x23, 0x16, 0x42, 0xc6, 0x0e, 0x65, 0x40, 0x27, - 0x8e, 0x9f, 0xdc, 0xba, 0xbb, 0x23, 0x26, 0xdc, 0x62, 0xb2, 0xf7, 0x52, 0xf5, 0x3f, 0xdc, 0x46, 0x7a, 0xb5, 0x80, - 0x4d, 0x55, 0x4a, 0x1d, 0x3b, 0x85, 0x31, 0xf5, 0xd4, 0xf7, 0xd7, 0xeb, 0x35, 0x59, 0x87, 0x44, 0x35, 0x0b, 0x9f, - 0x05, 0x41, 0xe0, 0xeb, 0xd5, 0xc2, 0x81, 0x2e, 0x3e, 0x1c, 0x36, 0x70, 0xa0, 0x40, 0xb1, 0x28, 0x4c, 0x4b, 0x27, - 0x2f, 0xb6, 0xb8, 0x8b, 0x2c, 0x47, 0x72, 0xfb, 0xf1, 0xe4, 0x14, 0x71, 0x72, 0x0a, 0xfe, 0x70, 0x82, 0x66, 0xef, - 0xad, 0x35, 0x6a, 0xcc, 0x19, 0x30, 0x08, 0xda, 0x0f, 0xf3, 0x2c, 0xbd, 0x9f, 0x79, 0x5f, 0xcc, 0xe0, 0x64, 0x06, - 0x0c, 0x9e, 0x01, 0xb0, 0x6a, 0xe4, 0x5d, 0x1c, 0xc5, 0xa9, 0xdd, 0x5e, 0xd1, 0xe0, 0x61, 0xc1, 0xca, 0xfc, 0x34, - 0x3a, 0x9d, 0x7b, 0xec, 0xbd, 0x65, 0xb0, 0xd8, 0x1f, 0x85, 0x3c, 0x56, 0xd0, 0xf7, 0x23, 0x3e, 0x84, 0xe1, 0x7e, - 0x65, 0xe8, 0x59, 0xfa, 0x38, 0xb3, 0x27, 0xc1, 0x70, 0xc5, 0x0a, 0x5a, 0x79, 0x23, 0x6f, 0xc8, 0x43, 0x08, 0xf7, - 0x26, 0x85, 0x10, 0xae, 0x58, 0x31, 0x7a, 0x3f, 0x3a, 0x5d, 0xf3, 0xc2, 0xcf, 0x3d, 0x0b, 0xf3, 0xd4, 0x71, 0x1e, - 0x30, 0x50, 0xa7, 0x18, 0x90, 0x4f, 0x4a, 0xc8, 0xbe, 0xe3, 0xb8, 0xbb, 0x1c, 0x4d, 0x5a, 0xf4, 0x1d, 0x3f, 0x55, - 0x32, 0x17, 0x0b, 0xf2, 0x49, 0x2b, 0xe9, 0xb8, 0xc4, 0x14, 0x28, 0xfb, 0x07, 0x51, 0x2b, 0x88, 0xed, 0x4e, 0xff, - 0xcb, 0x1d, 0xe3, 0x6e, 0x8f, 0xf9, 0x61, 0x84, 0x29, 0x31, 0x36, 0xc4, 0x66, 0xf4, 0xf9, 0x71, 0x75, 0xae, 0xb2, - 0xfb, 0x27, 0x52, 0xa7, 0xa0, 0x5d, 0xde, 0x08, 0x29, 0xb1, 0xb9, 0xc1, 0x8d, 0x89, 0x9d, 0xb7, 0x97, 0x57, 0x70, - 0x99, 0x65, 0x0d, 0x6a, 0x3d, 0x05, 0xe7, 0xa5, 0x21, 0x15, 0x4f, 0xff, 0x73, 0x5d, 0xf4, 0x91, 0xae, 0xbf, 0x89, - 0x1f, 0x05, 0xfc, 0x82, 0x66, 0xad, 0x9a, 0xbb, 0xbd, 0x36, 0x6b, 0xda, 0xcc, 0x66, 0x60, 0x13, 0x1b, 0xc2, 0x6b, - 0x4d, 0x74, 0x29, 0x52, 0xec, 0x53, 0x97, 0x54, 0xbc, 0x7e, 0xf0, 0x4a, 0x1e, 0x80, 0xba, 0x8d, 0x32, 0xb1, 0x82, - 0xb4, 0xe4, 0x5a, 0xc7, 0x8e, 0xec, 0x54, 0x39, 0xb0, 0x4f, 0x1b, 0x25, 0xd3, 0x52, 0xa4, 0x77, 0xb1, 0xf3, 0x95, - 0x1b, 0xe2, 0xd5, 0xfd, 0xcf, 0x59, 0xbf, 0xa7, 0xb5, 0xc8, 0x7a, 0x2e, 0x59, 0xf1, 0x72, 0x89, 0x10, 0x83, 0x29, - 0x84, 0x7e, 0x30, 0x70, 0xf6, 0xa4, 0x58, 0xad, 0xef, 0x7a, 0x2e, 0xc9, 0x55, 0xba, 0xd4, 0x7d, 0xd7, 0x39, 0x64, - 0x69, 0xc4, 0xbb, 0x3b, 0xd4, 0x79, 0xee, 0x7c, 0x61, 0x91, 0x57, 0x62, 0x6e, 0x9c, 0x87, 0x6c, 0x7e, 0xb1, 0xd5, - 0x7d, 0x49, 0x1a, 0xad, 0x85, 0xbb, 0x3b, 0x2e, 0x46, 0xba, 0xe6, 0xf2, 0x4b, 0x41, 0x6b, 0xa0, 0x4d, 0x1a, 0x49, - 0x2c, 0x65, 0x33, 0xa7, 0xe6, 0xf2, 0x78, 0xa0, 0xcf, 0x0f, 0xe4, 0x8b, 0xad, 0xe8, 0x4b, 0x7b, 0x4b, 0xde, 0x1d, - 0x35, 0x46, 0x7e, 0x26, 0x56, 0xc9, 0xed, 0xce, 0x7d, 0xf0, 0xe3, 0xef, 0x4b, 0x6c, 0xee, 0xaf, 0xb1, 0xc4, 0xd4, - 0xa8, 0xa6, 0xef, 0x3c, 0x97, 0x68, 0x1c, 0xb7, 0x73, 0xf8, 0xa7, 0x9b, 0xb7, 0x6f, 0x62, 0xd5, 0x6f, 0xdc, 0xf3, - 0xa7, 0xb8, 0x6d, 0xb5, 0xf8, 0xd0, 0x60, 0xf9, 0x8f, 0xb8, 0x67, 0xeb, 0x45, 0xef, 0xa3, 0xe3, 0x92, 0xd6, 0xdf, - 0xdb, 0x87, 0xa2, 0x61, 0x13, 0xfb, 0xe5, 0xa6, 0x2a, 0xcf, 0xad, 0x87, 0xde, 0x68, 0xe8, 0xee, 0x6e, 0x77, 0xee, - 0xce, 0x9d, 0x45, 0x7e, 0x77, 0xef, 0x27, 0x51, 0x7b, 0x05, 0x27, 0xdf, 0x6f, 0xe7, 0x6a, 0xe3, 0x69, 0xf1, 0x59, - 0xc8, 0xc5, 0x54, 0xc8, 0x02, 0x1b, 0x61, 0x76, 0x99, 0x58, 0x9d, 0x0b, 0x59, 0x2f, 0xcd, 0xb6, 0xe6, 0x59, 0x66, - 0x77, 0x86, 0xf5, 0x66, 0x96, 0x2b, 0x69, 0x2c, 0x27, 0x4e, 0x29, 0x56, 0xbb, 0x6e, 0xbf, 0xbd, 0x5b, 0xa6, 0x17, - 0xc3, 0xb3, 0x9d, 0x0d, 0xb8, 0xad, 0xc1, 0x8d, 0xf1, 0x78, 0x29, 0x16, 0x72, 0x9a, 0xa2, 0x34, 0xd8, 0x74, 0x42, - 0x39, 0xaf, 0x44, 0x79, 0x3f, 0xd5, 0x5c, 0x6a, 0x4f, 0x63, 0x23, 0xf2, 0xdd, 0x7c, 0x69, 0x8c, 0x92, 0xdb, 0xb9, - 0x6a, 0x32, 0x6c, 0xa6, 0xc1, 0xac, 0x23, 0xbc, 0x86, 0x67, 0x62, 0xa9, 0xa7, 0x24, 0x6c, 0xb0, 0x9a, 0xcd, 0x79, - 0x7a, 0xb7, 0x68, 0xd4, 0x52, 0x66, 0x5e, 0x6a, 0x6f, 0xe1, 0xe9, 0x73, 0x9a, 0xf3, 0x10, 0xd3, 0xd9, 0x7e, 0x96, - 0xe7, 0xf9, 0xac, 0x14, 0x12, 0xbd, 0xee, 0x56, 0x9b, 0x32, 0x32, 0xb0, 0x62, 0x27, 0x66, 0x12, 0x66, 0x17, 0x3a, - 0x1b, 0x69, 0x10, 0x9c, 0xcd, 0x0e, 0xee, 0x04, 0xb3, 0x74, 0xd9, 0x68, 0xd5, 0x4c, 0x6b, 0x25, 0xac, 0x99, 0xbb, - 0x8a, 0x0b, 0x79, 0x6a, 0xbd, 0x0d, 0x93, 0xd9, 0xbe, 0x3c, 0x4d, 0x85, 0x6c, 0x8f, 0x69, 0x8b, 0xd4, 0xac, 0x12, - 0xb2, 0x2b, 0xb2, 0x53, 0x36, 0x0a, 0xea, 0xcd, 0x8e, 0xec, 0x03, 0x64, 0x7b, 0xe0, 0xce, 0x4b, 0xdc, 0xcc, 0x3e, - 0x2d, 0xb5, 0x11, 0xf9, 0xbd, 0xb7, 0x2f, 0xd2, 0x53, 0x5d, 0xf3, 0x14, 0xbd, 0x39, 0x9a, 0x35, 0xa2, 0x9c, 0xb5, - 0x67, 0x78, 0xc2, 0x60, 0xa5, 0xf7, 0x38, 0x1d, 0xd5, 0xb4, 0x01, 0xfa, 0x58, 0xd7, 0xbf, 0xe3, 0xb6, 0xb1, 0xb8, - 0xad, 0x78, 0xb3, 0x10, 0xd2, 0x9b, 0x2b, 0x63, 0x54, 0x35, 0xf5, 0xc6, 0xf5, 0x66, 0xb6, 0x5f, 0xb2, 0xca, 0xa6, - 0xd4, 0x9a, 0xd9, 0xd6, 0xde, 0x03, 0xde, 0xb4, 0xde, 0x80, 0x56, 0xa5, 0xc8, 0xf6, 0x7c, 0x2d, 0x0b, 0x04, 0x47, - 0x78, 0xe8, 0xb0, 0xde, 0x80, 0x5d, 0x3b, 0x40, 0x3d, 0xc8, 0x27, 0x9c, 0x06, 0x5f, 0xf9, 0x46, 0xb2, 0x3c, 0x67, - 0xf3, 0xfc, 0x88, 0x94, 0x2d, 0xa1, 0x3b, 0xb1, 0x8f, 0x0a, 0x36, 0xa8, 0x37, 0xb3, 0xc3, 0x77, 0x33, 0xa8, 0x37, - 0x3b, 0xd1, 0xa6, 0xc5, 0xf6, 0x44, 0x4b, 0x1b, 0xaa, 0xd3, 0x65, 0x53, 0xf6, 0x9d, 0xaf, 0x84, 0xee, 0x59, 0x78, - 0xf5, 0x50, 0xe2, 0x7a, 0x4f, 0x97, 0xb8, 0x1e, 0xd8, 0xa6, 0xe8, 0x95, 0xda, 0xc4, 0xbd, 0xb6, 0xd8, 0x0c, 0x80, - 0x0d, 0x7a, 0x67, 0xe1, 0xeb, 0xb3, 0xf0, 0xea, 0xbf, 0x52, 0xbb, 0x7e, 0x77, 0xe1, 0xfa, 0x86, 0xaa, 0xf5, 0x8d, - 0x15, 0xab, 0xf3, 0xce, 0x3a, 0x7f, 0x16, 0xbe, 0x76, 0xdc, 0x9d, 0x20, 0x5a, 0x2c, 0xe8, 0xff, 0x02, 0xda, 0x7f, - 0xc5, 0x31, 0xbc, 0xa4, 0x13, 0x72, 0x01, 0xed, 0xd0, 0x41, 0x44, 0xc2, 0x09, 0x8c, 0xaf, 0x06, 0x64, 0x40, 0xc1, - 0xb6, 0x43, 0x23, 0x18, 0x93, 0xc9, 0x05, 0xd0, 0x11, 0x09, 0xc7, 0x40, 0x19, 0x30, 0x4a, 0x86, 0x6f, 0x58, 0x48, - 0x46, 0x43, 0x18, 0x5f, 0xb1, 0x80, 0x84, 0x0c, 0x3a, 0xde, 0x11, 0x61, 0x0c, 0x42, 0xcb, 0x12, 0x56, 0x01, 0xb0, - 0x34, 0x24, 0xc1, 0x18, 0x02, 0x18, 0x91, 0xe0, 0x82, 0x4c, 0x46, 0x30, 0x21, 0x63, 0x0a, 0x8c, 0x0c, 0x86, 0xa5, - 0x37, 0x24, 0x14, 0x46, 0x24, 0x1c, 0xf1, 0x09, 0x19, 0x84, 0xd0, 0x0e, 0x1d, 0x1c, 0x63, 0xc2, 0x98, 0x47, 0x02, - 0xfa, 0x26, 0x24, 0x6c, 0x0c, 0x63, 0x32, 0x18, 0x5c, 0xd2, 0x11, 0xb9, 0x18, 0x40, 0x37, 0x76, 0xf0, 0x52, 0x06, - 0xc3, 0xa7, 0x40, 0x63, 0x7f, 0x5e, 0xd0, 0x42, 0xc2, 0x28, 0x84, 0xe4, 0x62, 0xc2, 0x6d, 0x5f, 0xca, 0xa0, 0x1b, - 0x3b, 0xdc, 0x28, 0x85, 0xe0, 0x77, 0x63, 0x16, 0xfe, 0x79, 0x31, 0xa3, 0x16, 0x01, 0x46, 0x06, 0xe1, 0x25, 0x0d, - 0xc9, 0x08, 0xda, 0xa1, 0x3b, 0x9b, 0x32, 0x98, 0x5c, 0x5d, 0xc0, 0x04, 0x46, 0x64, 0x34, 0x81, 0x0b, 0x18, 0x5a, - 0x74, 0x2f, 0xc8, 0x64, 0xd0, 0x09, 0x79, 0x8c, 0x7c, 0x2b, 0x8c, 0x83, 0x3f, 0x30, 0x8c, 0x4f, 0xf9, 0xf4, 0x07, - 0x76, 0xe9, 0xff, 0x71, 0x05, 0x45, 0x7e, 0xd7, 0x86, 0x45, 0x7e, 0xf7, 0x3c, 0x60, 0xbb, 0xa8, 0x24, 0xb2, 0xdd, - 0x48, 0x12, 0x15, 0x14, 0x44, 0x16, 0x57, 0x3c, 0x4d, 0x4e, 0x5a, 0xfd, 0xc8, 0x2f, 0xe8, 0x61, 0xab, 0xa0, 0xc9, - 0xa3, 0xc6, 0xbd, 0xdb, 0x6b, 0x2b, 0x7d, 0x72, 0x53, 0x20, 0xbc, 0xbe, 0x7e, 0x07, 0x6b, 0x51, 0x96, 0x20, 0xd5, - 0x1a, 0x4c, 0x73, 0x0f, 0x46, 0xd9, 0x57, 0x03, 0x89, 0xa9, 0xb1, 0xa4, 0x29, 0x10, 0xf6, 0x7d, 0x04, 0x21, 0x24, - 0x9a, 0x37, 0xc9, 0xbb, 0x12, 0xb9, 0x46, 0x58, 0x88, 0x15, 0x82, 0x30, 0xa0, 0x55, 0x85, 0x60, 0x84, 0x1d, 0x8e, - 0x82, 0x2d, 0x5f, 0xe4, 0x77, 0x87, 0x74, 0x8d, 0xb2, 0xc8, 0x62, 0x89, 0x26, 0xd9, 0x77, 0xc4, 0x51, 0x11, 0x76, - 0x56, 0x5d, 0xa3, 0x31, 0x42, 0x2e, 0xac, 0x55, 0x61, 0x12, 0xd9, 0x5f, 0xb7, 0xc0, 0xdb, 0xdf, 0x0c, 0xb1, 0xbf, - 0x16, 0xb9, 0xb0, 0x6f, 0x06, 0x49, 0xd4, 0x76, 0x91, 0x56, 0x83, 0x6d, 0x64, 0xba, 0x07, 0x8e, 0x96, 0x2a, 0x51, - 0x2e, 0x4c, 0x11, 0x87, 0x0c, 0xea, 0x92, 0xa7, 0x58, 0xa8, 0x32, 0xc3, 0x26, 0xbe, 0xbe, 0xfe, 0xf9, 0xaf, 0xf6, - 0x35, 0xc4, 0x9a, 0x70, 0x94, 0xac, 0xf5, 0x5d, 0x27, 0x68, 0x89, 0xbd, 0xdc, 0x68, 0xd0, 0xbd, 0x6b, 0xd4, 0x5c, - 0xeb, 0xb5, 0x6a, 0xb2, 0x47, 0x5a, 0xde, 0x1d, 0x16, 0xf7, 0x9a, 0xda, 0xff, 0xb6, 0x1f, 0xed, 0x84, 0xf4, 0x72, - 0x5e, 0x09, 0x93, 0x5c, 0xf3, 0x15, 0x46, 0x7e, 0xb7, 0x91, 0x44, 0xbe, 0x75, 0xa0, 0xe3, 0x2d, 0xf6, 0x32, 0x05, - 0x4d, 0x7e, 0xbd, 0xb9, 0x84, 0xdf, 0xea, 0x8c, 0x1b, 0xec, 0xb0, 0x6f, 0xbd, 0xac, 0xd0, 0x14, 0x2a, 0x8b, 0xdf, - 0xfd, 0x7a, 0x7d, 0x73, 0xf4, 0x78, 0xd9, 0x32, 0x01, 0xca, 0xb4, 0x7b, 0x6f, 0x59, 0x96, 0x46, 0xd4, 0xbc, 0x31, - 0xad, 0x5a, 0xcf, 0x66, 0xc7, 0xc1, 0xa3, 0x76, 0x3f, 0x17, 0x25, 0x76, 0x4e, 0xed, 0x05, 0xfd, 0x04, 0xbe, 0x66, - 0xe3, 0xe1, 0xec, 0x2f, 0xac, 0xf4, 0xbb, 0x00, 0xf2, 0xbb, 0x68, 0xf2, 0xdb, 0xd7, 0xa8, 0x7f, 0x02, 0x14, 0xee, - 0xbc, 0x64, 0x9d, 0x12, 0x00, 0x00}; + 0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x95, 0x16, 0x6b, 0x8f, 0xdb, 0x36, 0xf2, 0x7b, 0x7e, + 0x05, 0x8f, 0x49, 0xbb, 0x52, 0xb3, 0x7a, 0x7a, 0xed, 0x6c, 0x24, 0x51, 0x45, 0x9a, 0xbb, 0xa2, 0x05, 0x9a, 0x36, + 0xc0, 0x6e, 0x73, 0x1f, 0x82, 0x00, 0x4b, 0x53, 0x23, 0x8b, 0x31, 0x45, 0xea, 0x48, 0xca, 0x8f, 0x18, 0xbe, 0xdf, + 0x7e, 0xa0, 0x24, 0x7b, 0x9d, 0x45, 0x73, 0xb8, 0xb3, 0x60, 0x61, 0x38, 0xef, 0x19, 0xcd, 0x83, 0xc5, 0xdf, 0x2a, + 0xc5, 0xec, 0xbe, 0x03, 0xd4, 0xd8, 0x56, 0x94, 0x85, 0x7b, 0x23, 0x41, 0xe5, 0x8a, 0x80, 0x2c, 0x8b, 0x06, 0x68, + 0x55, 0x16, 0x2d, 0x58, 0x8a, 0x58, 0x43, 0xb5, 0x01, 0x4b, 0xfe, 0xbc, 0xff, 0x39, 0xb8, 0x2d, 0x0b, 0xc1, 0xe5, + 0x1a, 0x69, 0x10, 0x84, 0x33, 0x25, 0x51, 0xa3, 0xa1, 0x26, 0x15, 0xb5, 0x34, 0xe3, 0x2d, 0x5d, 0xc1, 0x24, 0x22, + 0x69, 0x0b, 0x64, 0xc3, 0x61, 0xdb, 0x29, 0x6d, 0x11, 0x53, 0xd2, 0x82, 0xb4, 0x04, 0x6f, 0x79, 0x65, 0x1b, 0x52, + 0xc1, 0x86, 0x33, 0x08, 0x86, 0xc3, 0x35, 0x97, 0xdc, 0x72, 0x2a, 0x02, 0xc3, 0xa8, 0x00, 0x92, 0x5c, 0xf7, 0x06, + 0xf4, 0x70, 0xa0, 0x4b, 0x01, 0x44, 0x2a, 0x5c, 0x16, 0x86, 0x69, 0xde, 0x59, 0xe4, 0x5c, 0x25, 0xad, 0xaa, 0x7a, + 0x01, 0x65, 0x14, 0x51, 0x63, 0xc0, 0x9a, 0x88, 0xcb, 0x0a, 0x76, 0xe1, 0x32, 0x66, 0x2c, 0x86, 0xdb, 0xdb, 0xf0, + 0xb3, 0x79, 0x56, 0x29, 0xd6, 0xb7, 0x20, 0x6d, 0x28, 0x14, 0xa3, 0x96, 0x2b, 0x19, 0x1a, 0xa0, 0x9a, 0x35, 0x84, + 0x10, 0xfc, 0xa3, 0xa1, 0x1b, 0xc0, 0xdf, 0x7f, 0xef, 0x9d, 0x99, 0x56, 0x60, 0xff, 0x21, 0xc0, 0x81, 0xe6, 0xa7, + 0xfd, 0x3d, 0x5d, 0xfd, 0x4e, 0x5b, 0xf0, 0x30, 0x35, 0xbc, 0x02, 0xec, 0x7f, 0x8c, 0x3f, 0x85, 0xc6, 0xee, 0x05, + 0x84, 0x15, 0x37, 0x9d, 0xa0, 0x7b, 0x82, 0x97, 0x42, 0xb1, 0x35, 0xf6, 0xf3, 0xba, 0x97, 0xcc, 0x29, 0x47, 0xc6, + 0x03, 0xff, 0x20, 0xc0, 0x22, 0x4b, 0xde, 0x51, 0xdb, 0x84, 0x2d, 0xdd, 0x79, 0x23, 0xc0, 0xa5, 0x97, 0xfe, 0xe0, + 0xc1, 0xcb, 0x24, 0x8e, 0xfd, 0xeb, 0xe1, 0x15, 0xfb, 0x51, 0x12, 0xc7, 0xb9, 0x06, 0xdb, 0x6b, 0x89, 0xa8, 0xf7, + 0x50, 0x74, 0xd4, 0x36, 0xa8, 0x22, 0xf8, 0x5d, 0x92, 0xa2, 0xe4, 0x75, 0x98, 0xce, 0x7f, 0x0b, 0x5f, 0xa1, 0x9b, + 0x30, 0x9d, 0xb3, 0x57, 0xc1, 0x1c, 0x25, 0x37, 0xc1, 0x1c, 0xa5, 0x69, 0x38, 0x47, 0xf1, 0x17, 0x8c, 0x6a, 0x2e, + 0x04, 0xc1, 0x52, 0x49, 0xc0, 0xc8, 0x58, 0xad, 0xd6, 0x40, 0x30, 0xeb, 0xb5, 0x06, 0x69, 0xdf, 0x2a, 0xa1, 0x34, + 0x8e, 0xca, 0x67, 0xff, 0x97, 0x42, 0xab, 0xa9, 0x34, 0xb5, 0xd2, 0x2d, 0xc1, 0x43, 0xf6, 0xbd, 0x17, 0x07, 0x7b, + 0x44, 0xee, 0xe5, 0x5f, 0x10, 0x03, 0xa5, 0xf9, 0x8a, 0x4b, 0x82, 0x9d, 0xc6, 0x5b, 0x1c, 0x95, 0x0f, 0xfe, 0xf1, + 0x1c, 0x3d, 0x75, 0xd1, 0x4f, 0xf1, 0x28, 0xef, 0xe3, 0x43, 0x61, 0x36, 0x2b, 0xb4, 0x6b, 0x85, 0x34, 0x04, 0x37, + 0xd6, 0x76, 0x59, 0x14, 0x6d, 0xb7, 0xdb, 0x70, 0x3b, 0x0b, 0x95, 0x5e, 0x45, 0x69, 0x1c, 0xc7, 0x91, 0xd9, 0xac, + 0x30, 0x1a, 0x0b, 0x01, 0xa7, 0x37, 0x18, 0x35, 0xc0, 0x57, 0x8d, 0x1d, 0xe0, 0xf2, 0xc5, 0x01, 0x8e, 0x85, 0xe3, + 0x28, 0x1f, 0x3e, 0x5d, 0x58, 0xe1, 0x17, 0x56, 0xe0, 0x47, 0xea, 0xe1, 0x53, 0x98, 0x57, 0x43, 0x98, 0xaf, 0x68, + 0x8a, 0x52, 0x14, 0x0f, 0x4f, 0x1a, 0x38, 0x78, 0x3a, 0x05, 0x4f, 0x4e, 0xe8, 0xe2, 0xe4, 0xa0, 0x76, 0x11, 0xbc, + 0x3e, 0xcb, 0x26, 0x0e, 0xb3, 0x49, 0xe2, 0x47, 0x84, 0x13, 0xf8, 0x65, 0x71, 0x79, 0x0e, 0xd2, 0x0f, 0x97, 0x0c, + 0xce, 0x5a, 0x93, 0x7c, 0x58, 0xd0, 0x39, 0x9a, 0x4f, 0x98, 0x79, 0xe0, 0xe0, 0xf3, 0x09, 0xcd, 0x37, 0x69, 0x93, + 0xb4, 0xc1, 0x22, 0x98, 0xd3, 0x19, 0x9a, 0x4d, 0x8e, 0xcc, 0xd0, 0x6c, 0x93, 0x36, 0x8b, 0x0f, 0x8b, 0x4b, 0x5c, + 0x30, 0xfb, 0x72, 0x15, 0x95, 0xd8, 0xcf, 0x30, 0x7e, 0x8c, 0x5c, 0x5d, 0x46, 0x1e, 0x7e, 0x56, 0x5c, 0x7a, 0x18, + 0xfb, 0xc7, 0x1a, 0x2c, 0x6b, 0x3c, 0x1c, 0x31, 0x25, 0x6b, 0xbe, 0x0a, 0x3f, 0x1b, 0x25, 0xb1, 0x1f, 0xda, 0x06, + 0xa4, 0x77, 0x12, 0x75, 0x82, 0x30, 0x50, 0xbc, 0xa7, 0x14, 0xeb, 0x1f, 0xce, 0xf5, 0x6f, 0xb9, 0x15, 0x40, 0x6c, + 0xe8, 0x1a, 0xf6, 0xfa, 0x8c, 0x5d, 0xaa, 0x6a, 0xff, 0x8d, 0xd6, 0x68, 0x92, 0xb1, 0x2f, 0xb8, 0x94, 0xa0, 0xef, + 0x61, 0x67, 0x09, 0x7e, 0xf7, 0xe6, 0x2d, 0x7a, 0x53, 0x55, 0x1a, 0x8c, 0xc9, 0x10, 0x7e, 0x69, 0xc3, 0x96, 0xb2, + 0xff, 0x5d, 0x57, 0xf2, 0x95, 0xae, 0x7f, 0xf2, 0x9f, 0x39, 0xfa, 0x1d, 0xec, 0x56, 0xe9, 0xf5, 0xa4, 0xcd, 0xb9, + 0x96, 0xbb, 0x0e, 0xd3, 0xc4, 0x86, 0xb4, 0x33, 0xa1, 0x11, 0x9c, 0x81, 0x97, 0xf8, 0x61, 0x4b, 0xbb, 0xc7, 0xa8, + 0xe4, 0x29, 0x51, 0x0f, 0x45, 0xc5, 0x37, 0x88, 0x09, 0x6a, 0x0c, 0xc1, 0x72, 0x54, 0x85, 0xd1, 0x33, 0x34, 0xfc, + 0x94, 0x64, 0x82, 0xb3, 0x35, 0xc1, 0x7f, 0x31, 0x01, 0x7e, 0xda, 0xff, 0x5a, 0x79, 0x57, 0xc6, 0xf0, 0xea, 0xca, + 0x0f, 0x37, 0x54, 0xf4, 0x80, 0x08, 0xb2, 0x0d, 0x37, 0x8f, 0x0e, 0xe6, 0xdf, 0x14, 0xeb, 0xcc, 0xfa, 0xca, 0x0f, + 0x6b, 0xc5, 0x7a, 0xe3, 0xf9, 0xb8, 0x9c, 0xcc, 0x15, 0x74, 0x1c, 0x90, 0xf8, 0x39, 0x7e, 0xe2, 0x51, 0x20, 0xa0, + 0xb6, 0x67, 0x3e, 0x84, 0x5e, 0x1c, 0x8c, 0x27, 0x43, 0x6d, 0x0c, 0xf7, 0x8f, 0x67, 0x64, 0x61, 0x3a, 0x2a, 0x9f, + 0x0a, 0x3a, 0x07, 0x5d, 0xab, 0xc8, 0xd0, 0x41, 0xae, 0x5f, 0x3a, 0x2a, 0xcf, 0x06, 0x23, 0x7a, 0x02, 0x5f, 0x1c, + 0xb8, 0x27, 0xdd, 0x14, 0x5c, 0x9f, 0x35, 0x16, 0x51, 0xc5, 0x37, 0xe5, 0xc3, 0xd1, 0x7f, 0x8c, 0xe3, 0x5f, 0x3d, + 0xe8, 0xfd, 0x1d, 0x08, 0x60, 0x56, 0x69, 0x0f, 0x3f, 0x97, 0x60, 0xb1, 0x3f, 0x06, 0xfc, 0xcb, 0xfd, 0xbb, 0xdf, + 0x88, 0xf2, 0xb4, 0x7f, 0xfd, 0x2d, 0x6e, 0xb7, 0x0a, 0x3e, 0x6a, 0x10, 0xff, 0x26, 0x57, 0x6e, 0x19, 0x5c, 0x7d, + 0xc2, 0x7e, 0x38, 0xc4, 0xfb, 0xf0, 0xb8, 0x11, 0x5c, 0x3b, 0xbf, 0xdc, 0xb5, 0xe2, 0xda, 0x45, 0x18, 0x2c, 0xe6, + 0xfe, 0xf1, 0xe1, 0xe8, 0x1f, 0xfd, 0xbc, 0x88, 0xc6, 0xb9, 0x5e, 0x16, 0xc3, 0x88, 0x2d, 0x7f, 0x38, 0x2c, 0xd5, + 0x2e, 0x30, 0xfc, 0x0b, 0x97, 0xab, 0x8c, 0xcb, 0x06, 0x34, 0xb7, 0xc7, 0x8a, 0x6f, 0xae, 0xb9, 0xec, 0x7a, 0x7b, + 0xe8, 0x68, 0x55, 0x39, 0xca, 0xbc, 0xdb, 0xe5, 0xb5, 0x92, 0xd6, 0x71, 0x42, 0x96, 0x40, 0x7b, 0x1c, 0xe9, 0xc3, + 0x44, 0xc9, 0x5e, 0xcf, 0xbf, 0x3b, 0xba, 0x82, 0x3b, 0x58, 0xd8, 0xd9, 0x80, 0x0a, 0xbe, 0x92, 0x19, 0x03, 0x69, + 0x41, 0x8f, 0x42, 0x35, 0x6d, 0xb9, 0xd8, 0x67, 0x86, 0x4a, 0x13, 0x18, 0xd0, 0xbc, 0x3e, 0x2e, 0x7b, 0x6b, 0x95, + 0x3c, 0x2c, 0x95, 0xae, 0x40, 0x67, 0x71, 0x3e, 0x02, 0x81, 0xa6, 0x15, 0xef, 0x4d, 0x16, 0xce, 0x34, 0xb4, 0xf9, + 0x92, 0xb2, 0xf5, 0x4a, 0xab, 0x5e, 0x56, 0x01, 0x73, 0x93, 0x36, 0x7b, 0x9e, 0xd4, 0x74, 0x06, 0x2c, 0x9f, 0x4e, + 0x75, 0x5d, 0xe7, 0x82, 0x4b, 0x08, 0xc6, 0x59, 0x96, 0xa5, 0xe1, 0x8d, 0x13, 0xbb, 0x70, 0x33, 0x4c, 0x1d, 0x62, + 0xf4, 0x31, 0x89, 0xe3, 0xef, 0xf2, 0x53, 0x38, 0x71, 0xce, 0x7a, 0x6d, 0x94, 0xce, 0x3a, 0xc5, 0x9d, 0x9b, 0xc7, + 0x96, 0x72, 0x79, 0xe9, 0xbd, 0x2b, 0x93, 0x7c, 0x5a, 0x3f, 0x19, 0x97, 0x83, 0x99, 0x61, 0x09, 0xe5, 0x2d, 0x97, + 0xe3, 0x0e, 0xcd, 0xd2, 0x45, 0xdc, 0xed, 0x8e, 0xe1, 0x54, 0x20, 0x87, 0x13, 0x77, 0x2d, 0x60, 0x97, 0x7f, 0xee, + 0x8d, 0xe5, 0xf5, 0x3e, 0x98, 0x76, 0x70, 0x66, 0x3a, 0xca, 0x20, 0x58, 0x82, 0xdd, 0x02, 0xc8, 0x7c, 0xb0, 0x11, + 0x70, 0x0b, 0xad, 0x99, 0xf2, 0x74, 0x56, 0x33, 0x14, 0xe8, 0xd7, 0xba, 0xfe, 0x1b, 0xb7, 0xab, 0xc5, 0x43, 0x4b, + 0xf5, 0x8a, 0xcb, 0x60, 0xa9, 0xac, 0x55, 0x6d, 0x16, 0xbc, 0xea, 0x76, 0xf9, 0x84, 0x72, 0xca, 0xb2, 0xc4, 0xb9, + 0x39, 0xec, 0xd6, 0x53, 0xbe, 0x93, 0x6e, 0x87, 0x8c, 0x12, 0xbc, 0x9a, 0xf8, 0x06, 0x16, 0x14, 0x9f, 0xd3, 0x93, + 0xcc, 0xbb, 0x1d, 0x72, 0xb8, 0x53, 0xaa, 0x6f, 0xea, 0x5b, 0x9a, 0xc4, 0x7f, 0xf1, 0x45, 0xaa, 0xba, 0x4e, 0x97, + 0xf5, 0x39, 0x53, 0x6e, 0x4d, 0xba, 0xd6, 0x18, 0x4a, 0xab, 0x88, 0xc6, 0xdb, 0x8c, 0xab, 0x8c, 0xb2, 0x70, 0x19, + 0x2e, 0x8b, 0x26, 0x41, 0xbc, 0x22, 0x2d, 0x65, 0xe5, 0xc5, 0xf8, 0x2a, 0xa2, 0x26, 0x39, 0x91, 0x9a, 0xa4, 0xfc, + 0x6a, 0x18, 0x8d, 0xb4, 0xc1, 0xfb, 0xf2, 0xad, 0x92, 0x12, 0x98, 0xe5, 0x72, 0x85, 0xac, 0x42, 0x53, 0x0a, 0xc2, + 0x30, 0x2c, 0x96, 0xba, 0x7c, 0x2f, 0x80, 0x1a, 0x40, 0x5b, 0xca, 0x6d, 0x58, 0x44, 0x23, 0xff, 0xd8, 0xc7, 0xbc, + 0x22, 0x12, 0x6c, 0x39, 0x35, 0x6c, 0xd1, 0xcc, 0x46, 0x03, 0x77, 0x60, 0x9d, 0x26, 0x67, 0x60, 0x56, 0x16, 0x6e, + 0xe5, 0x22, 0x3a, 0x8c, 0x34, 0x12, 0x6d, 0x79, 0xcd, 0xdd, 0x95, 0xa5, 0x2c, 0x86, 0x22, 0x77, 0x1a, 0x5c, 0x9e, + 0xc7, 0xeb, 0xd5, 0x00, 0x09, 0x90, 0x2b, 0xdb, 0x90, 0x59, 0x8a, 0x3a, 0x41, 0x19, 0x34, 0x4a, 0x54, 0xa0, 0xc9, + 0xdd, 0xdd, 0xaf, 0x7f, 0x2f, 0x9d, 0x33, 0x8f, 0x72, 0x9d, 0x59, 0x8f, 0x62, 0x0e, 0x98, 0xa4, 0x16, 0x37, 0xe3, + 0xa5, 0xaa, 0xa3, 0xc6, 0x6c, 0x95, 0xae, 0xbe, 0xd2, 0xf1, 0x7e, 0x42, 0x8e, 0x7a, 0x86, 0xff, 0xd0, 0x2a, 0xe5, + 0x1d, 0xdd, 0x40, 0x11, 0x4d, 0x87, 0x22, 0x72, 0x0e, 0x8f, 0xf4, 0x66, 0xe2, 0x6b, 0x92, 0xf2, 0x8f, 0xfb, 0x37, + 0xe8, 0xcf, 0xae, 0xa2, 0x16, 0xc6, 0xb4, 0x0d, 0x51, 0xb5, 0x60, 0x1b, 0x55, 0x91, 0xf7, 0x7f, 0xdc, 0xdd, 0x9f, + 0x23, 0xec, 0x07, 0x26, 0x04, 0x92, 0x8d, 0xd7, 0xbb, 0x5e, 0x58, 0xde, 0x51, 0x6d, 0x07, 0xb5, 0x81, 0x9b, 0x22, + 0xa7, 0x18, 0x06, 0x7a, 0xcd, 0x05, 0x8c, 0x61, 0x8c, 0x82, 0x25, 0x3a, 0x79, 0x75, 0xb2, 0xf6, 0xc4, 0xaf, 0x68, + 0xfc, 0xda, 0xd1, 0xf8, 0xe9, 0xa3, 0xe1, 0xa6, 0xfb, 0x1f, 0x53, 0x58, 0x46, 0xb2, 0xf9, 0x0a, 0x00, 0x00}; } // namespace captive_portal } // namespace esphome diff --git a/esphome/components/captive_portal/captive_portal.cpp b/esphome/components/captive_portal/captive_portal.cpp index 25179fdacc..30438747f2 100644 --- a/esphome/components/captive_portal/captive_portal.cpp +++ b/esphome/components/captive_portal/captive_portal.cpp @@ -11,37 +11,53 @@ namespace captive_portal { static const char *const TAG = "captive_portal"; void CaptivePortal::handle_config(AsyncWebServerRequest *request) { - AsyncResponseStream *stream = request->beginResponseStream("application/json"); - stream->addHeader("cache-control", "public, max-age=0, must-revalidate"); + AsyncResponseStream *stream = request->beginResponseStream(ESPHOME_F("application/json")); + stream->addHeader(ESPHOME_F("cache-control"), ESPHOME_F("public, max-age=0, must-revalidate")); +#ifdef USE_ESP8266 + stream->print(ESPHOME_F("{\"mac\":\"")); + stream->print(get_mac_address_pretty().c_str()); + stream->print(ESPHOME_F("\",\"name\":\"")); + stream->print(App.get_name().c_str()); + stream->print(ESPHOME_F("\",\"aps\":[{}")); +#else stream->printf(R"({"mac":"%s","name":"%s","aps":[{})", get_mac_address_pretty().c_str(), App.get_name().c_str()); +#endif for (auto &scan : wifi::global_wifi_component->get_scan_result()) { if (scan.get_is_hidden()) continue; - // Assumes no " in ssid, possible unicode isses? + // Assumes no " in ssid, possible unicode isses? +#ifdef USE_ESP8266 + stream->print(ESPHOME_F(",{\"ssid\":\"")); + stream->print(scan.get_ssid().c_str()); + stream->print(ESPHOME_F("\",\"rssi\":")); + stream->print(scan.get_rssi()); + stream->print(ESPHOME_F(",\"lock\":")); + stream->print(scan.get_with_auth()); + stream->print(ESPHOME_F("}")); +#else stream->printf(R"(,{"ssid":"%s","rssi":%d,"lock":%d})", scan.get_ssid().c_str(), scan.get_rssi(), scan.get_with_auth()); +#endif } - stream->print(F("]}")); + stream->print(ESPHOME_F("]}")); request->send(stream); } void CaptivePortal::handle_wifisave(AsyncWebServerRequest *request) { - std::string ssid = request->arg("ssid").c_str(); - std::string psk = request->arg("psk").c_str(); + std::string ssid = request->arg("ssid").c_str(); // NOLINT(readability-redundant-string-cstr) + std::string psk = request->arg("psk").c_str(); // NOLINT(readability-redundant-string-cstr) ESP_LOGI(TAG, "Requested WiFi Settings Change:"); ESP_LOGI(TAG, " SSID='%s'", ssid.c_str()); ESP_LOGI(TAG, " Password=" LOG_SECRET("'%s'"), psk.c_str()); wifi::global_wifi_component->save_wifi_sta(ssid, psk); wifi::global_wifi_component->start_scanning(); - request->redirect("/?save"); + request->redirect(ESPHOME_F("/?save")); } void CaptivePortal::setup() { -#ifndef USE_ARDUINO - // No DNS server needed for non-Arduino frameworks + // Disable loop by default - will be enabled when captive portal starts this->disable_loop(); -#endif } void CaptivePortal::start() { this->base_->init(); @@ -49,46 +65,47 @@ void CaptivePortal::start() { this->base_->add_handler(this); } + network::IPAddress ip = wifi::global_wifi_component->wifi_soft_ap_ip(); + +#ifdef USE_ESP_IDF + // Create DNS server instance for ESP-IDF + this->dns_server_ = make_unique(); + this->dns_server_->start(ip); +#endif #ifdef USE_ARDUINO this->dns_server_ = make_unique(); this->dns_server_->setErrorReplyCode(DNSReplyCode::NoError); - network::IPAddress ip = wifi::global_wifi_component->wifi_soft_ap_ip(); - this->dns_server_->start(53, "*", ip); - // Re-enable loop() when DNS server is started - this->enable_loop(); + this->dns_server_->start(53, ESPHOME_F("*"), ip); #endif - this->base_->get_server()->onNotFound([this](AsyncWebServerRequest *req) { - if (!this->active_ || req->host().c_str() == wifi::global_wifi_component->wifi_soft_ap_ip().str()) { - req->send(404, "text/html", "File not found"); - return; - } - - auto url = "http://" + wifi::global_wifi_component->wifi_soft_ap_ip().str(); - req->redirect(url.c_str()); - }); - this->initialized_ = true; this->active_ = true; + + // Enable loop() now that captive portal is active + this->enable_loop(); + + ESP_LOGV(TAG, "Captive portal started"); } void CaptivePortal::handleRequest(AsyncWebServerRequest *req) { - if (req->url() == "/") { -#ifndef USE_ESP8266 - auto *response = req->beginResponse(200, "text/html", INDEX_GZ, sizeof(INDEX_GZ)); -#else - auto *response = req->beginResponse_P(200, "text/html", INDEX_GZ, sizeof(INDEX_GZ)); -#endif - response->addHeader("Content-Encoding", "gzip"); - req->send(response); - return; - } else if (req->url() == "/config.json") { + if (req->url() == ESPHOME_F("/config.json")) { this->handle_config(req); return; - } else if (req->url() == "/wifisave") { + } else if (req->url() == ESPHOME_F("/wifisave")) { this->handle_wifisave(req); return; } + + // All other requests get the captive portal page + // This includes OS captive portal detection endpoints which will trigger + // the captive portal when they don't receive their expected responses +#ifndef USE_ESP8266 + auto *response = req->beginResponse(200, ESPHOME_F("text/html"), INDEX_GZ, sizeof(INDEX_GZ)); +#else + auto *response = req->beginResponse_P(200, ESPHOME_F("text/html"), INDEX_GZ, sizeof(INDEX_GZ)); +#endif + response->addHeader(ESPHOME_F("Content-Encoding"), ESPHOME_F("gzip")); + req->send(response); } CaptivePortal::CaptivePortal(web_server_base::WebServerBase *base) : base_(base) { global_captive_portal = this; } diff --git a/esphome/components/captive_portal/captive_portal.h b/esphome/components/captive_portal/captive_portal.h index c78fff824a..f48c286f0c 100644 --- a/esphome/components/captive_portal/captive_portal.h +++ b/esphome/components/captive_portal/captive_portal.h @@ -5,6 +5,9 @@ #ifdef USE_ARDUINO #include #endif +#ifdef USE_ESP_IDF +#include "dns_server_esp32_idf.h" +#endif #include "esphome/core/component.h" #include "esphome/core/helpers.h" #include "esphome/core/preferences.h" @@ -19,41 +22,36 @@ class CaptivePortal : public AsyncWebHandler, public Component { CaptivePortal(web_server_base::WebServerBase *base); void setup() override; void dump_config() override; -#ifdef USE_ARDUINO void loop() override { +#ifdef USE_ARDUINO if (this->dns_server_ != nullptr) { this->dns_server_->processNextRequest(); - } else { - this->disable_loop(); } - } #endif +#ifdef USE_ESP_IDF + if (this->dns_server_ != nullptr) { + this->dns_server_->process_next_request(); + } +#endif + } float get_setup_priority() const override; void start(); bool is_active() const { return this->active_; } void end() { this->active_ = false; + this->disable_loop(); // Stop processing DNS requests this->base_->deinit(); -#ifdef USE_ARDUINO - this->dns_server_->stop(); - this->dns_server_ = nullptr; -#endif + if (this->dns_server_ != nullptr) { + this->dns_server_->stop(); + this->dns_server_ = nullptr; + } } bool canHandle(AsyncWebServerRequest *request) const override { - if (!this->active_) - return false; - - if (request->method() == HTTP_GET) { - if (request->url() == "/") - return true; - if (request->url() == "/config.json") - return true; - if (request->url() == "/wifisave") - return true; - } - - return false; + // Handle all GET requests when captive portal is active + // This allows us to respond with the portal page for any URL, + // triggering OS captive portal detection + return this->active_ && request->method() == HTTP_GET; } void handle_config(AsyncWebServerRequest *request); @@ -66,7 +64,7 @@ class CaptivePortal : public AsyncWebHandler, public Component { web_server_base::WebServerBase *base_; bool initialized_{false}; bool active_{false}; -#ifdef USE_ARDUINO +#if defined(USE_ARDUINO) || defined(USE_ESP_IDF) std::unique_ptr dns_server_{nullptr}; #endif }; diff --git a/esphome/components/captive_portal/dns_server_esp32_idf.cpp b/esphome/components/captive_portal/dns_server_esp32_idf.cpp new file mode 100644 index 0000000000..740107400a --- /dev/null +++ b/esphome/components/captive_portal/dns_server_esp32_idf.cpp @@ -0,0 +1,205 @@ +#include "dns_server_esp32_idf.h" +#ifdef USE_ESP_IDF + +#include "esphome/core/log.h" +#include "esphome/core/hal.h" +#include "esphome/components/socket/socket.h" +#include +#include + +namespace esphome::captive_portal { + +static const char *const TAG = "captive_portal.dns"; + +// DNS constants +static constexpr uint16_t DNS_PORT = 53; +static constexpr uint16_t DNS_QR_FLAG = 1 << 15; +static constexpr uint16_t DNS_OPCODE_MASK = 0x7800; +static constexpr uint16_t DNS_QTYPE_A = 0x0001; +static constexpr uint16_t DNS_QCLASS_IN = 0x0001; +static constexpr uint16_t DNS_ANSWER_TTL = 300; + +// DNS Header structure +struct DNSHeader { + uint16_t id; + uint16_t flags; + uint16_t qd_count; + uint16_t an_count; + uint16_t ns_count; + uint16_t ar_count; +} __attribute__((packed)); + +// DNS Question structure +struct DNSQuestion { + uint16_t type; + uint16_t dns_class; +} __attribute__((packed)); + +// DNS Answer structure +struct DNSAnswer { + uint16_t ptr_offset; + uint16_t type; + uint16_t dns_class; + uint32_t ttl; + uint16_t addr_len; + uint32_t ip_addr; +} __attribute__((packed)); + +void DNSServer::start(const network::IPAddress &ip) { + this->server_ip_ = ip; + ESP_LOGV(TAG, "Starting DNS server on %s", ip.str().c_str()); + + // Create loop-monitored UDP socket + this->socket_ = socket::socket_ip_loop_monitored(SOCK_DGRAM, IPPROTO_UDP); + if (this->socket_ == nullptr) { + ESP_LOGE(TAG, "Socket create failed"); + return; + } + + // Set socket options + int enable = 1; + this->socket_->setsockopt(SOL_SOCKET, SO_REUSEADDR, &enable, sizeof(enable)); + + // Bind to port 53 + struct sockaddr_storage server_addr = {}; + socklen_t addr_len = socket::set_sockaddr_any((struct sockaddr *) &server_addr, sizeof(server_addr), DNS_PORT); + + int err = this->socket_->bind((struct sockaddr *) &server_addr, addr_len); + if (err != 0) { + ESP_LOGE(TAG, "Bind failed: %d", errno); + this->socket_ = nullptr; + return; + } + ESP_LOGV(TAG, "Bound to port %d", DNS_PORT); +} + +void DNSServer::stop() { + if (this->socket_ != nullptr) { + this->socket_->close(); + this->socket_ = nullptr; + } + ESP_LOGV(TAG, "Stopped"); +} + +void DNSServer::process_next_request() { + // Process one request if socket is valid and data is available + if (this->socket_ == nullptr || !this->socket_->ready()) { + return; + } + struct sockaddr_in client_addr; + socklen_t client_addr_len = sizeof(client_addr); + + // Receive DNS request using raw fd for recvfrom + int fd = this->socket_->get_fd(); + if (fd < 0) { + return; + } + + ssize_t len = recvfrom(fd, this->buffer_, sizeof(this->buffer_), MSG_DONTWAIT, (struct sockaddr *) &client_addr, + &client_addr_len); + + if (len < 0) { + if (errno != EAGAIN && errno != EWOULDBLOCK && errno != EINTR) { + ESP_LOGE(TAG, "recvfrom failed: %d", errno); + } + return; + } + + ESP_LOGVV(TAG, "Received %d bytes from %s:%d", len, inet_ntoa(client_addr.sin_addr), ntohs(client_addr.sin_port)); + + if (len < static_cast(sizeof(DNSHeader) + 1)) { + ESP_LOGV(TAG, "Request too short: %d", len); + return; + } + + // Parse DNS header + DNSHeader *header = (DNSHeader *) this->buffer_; + uint16_t flags = ntohs(header->flags); + uint16_t qd_count = ntohs(header->qd_count); + + // Check if it's a standard query + if ((flags & DNS_QR_FLAG) || (flags & DNS_OPCODE_MASK) || qd_count != 1) { + ESP_LOGV(TAG, "Not a standard query: flags=0x%04X, qd_count=%d", flags, qd_count); + return; // Not a standard query + } + + // Parse domain name (we don't actually care about it - redirect everything) + uint8_t *ptr = this->buffer_ + sizeof(DNSHeader); + uint8_t *end = this->buffer_ + len; + + while (ptr < end && *ptr != 0) { + uint8_t label_len = *ptr; + if (label_len > 63) { // Check for invalid label length + return; + } + // Check if we have room for this label plus the length byte + if (ptr + label_len + 1 > end) { + return; // Would overflow + } + ptr += label_len + 1; + } + + // Check if we reached a proper null terminator + if (ptr >= end || *ptr != 0) { + return; // Name not terminated or truncated + } + ptr++; // Skip the null terminator + + // Check we have room for the question + if (ptr + sizeof(DNSQuestion) > end) { + return; // Request truncated + } + + // Parse DNS question + DNSQuestion *question = (DNSQuestion *) ptr; + uint16_t qtype = ntohs(question->type); + uint16_t qclass = ntohs(question->dns_class); + + // We only handle A queries + if (qtype != DNS_QTYPE_A || qclass != DNS_QCLASS_IN) { + ESP_LOGV(TAG, "Not an A query: type=0x%04X, class=0x%04X", qtype, qclass); + return; // Not an A query + } + + // Build DNS response by modifying the request in-place + header->flags = htons(DNS_QR_FLAG | 0x8000); // Response + Authoritative + header->an_count = htons(1); // One answer + + // Add answer section after the question + size_t question_len = (ptr + sizeof(DNSQuestion)) - this->buffer_ - sizeof(DNSHeader); + size_t answer_offset = sizeof(DNSHeader) + question_len; + + // Check if we have room for the answer + if (answer_offset + sizeof(DNSAnswer) > sizeof(this->buffer_)) { + ESP_LOGW(TAG, "Response too large"); + return; + } + + DNSAnswer *answer = (DNSAnswer *) (this->buffer_ + answer_offset); + + // Pointer to name in question (offset from start of packet) + answer->ptr_offset = htons(0xC000 | sizeof(DNSHeader)); + answer->type = htons(DNS_QTYPE_A); + answer->dns_class = htons(DNS_QCLASS_IN); + answer->ttl = htonl(DNS_ANSWER_TTL); + answer->addr_len = htons(4); + + // Get the raw IP address + ip4_addr_t addr = this->server_ip_; + answer->ip_addr = addr.addr; + + size_t response_len = answer_offset + sizeof(DNSAnswer); + + // Send response + ssize_t sent = + this->socket_->sendto(this->buffer_, response_len, 0, (struct sockaddr *) &client_addr, client_addr_len); + if (sent < 0) { + ESP_LOGV(TAG, "Send failed: %d", errno); + } else { + ESP_LOGV(TAG, "Sent %d bytes", sent); + } +} + +} // namespace esphome::captive_portal + +#endif // USE_ESP_IDF diff --git a/esphome/components/captive_portal/dns_server_esp32_idf.h b/esphome/components/captive_portal/dns_server_esp32_idf.h new file mode 100644 index 0000000000..13d9def8e3 --- /dev/null +++ b/esphome/components/captive_portal/dns_server_esp32_idf.h @@ -0,0 +1,27 @@ +#pragma once +#ifdef USE_ESP_IDF + +#include +#include "esphome/core/helpers.h" +#include "esphome/components/network/ip_address.h" +#include "esphome/components/socket/socket.h" + +namespace esphome::captive_portal { + +class DNSServer { + public: + void start(const network::IPAddress &ip); + void stop(); + void process_next_request(); + + protected: + static constexpr size_t DNS_BUFFER_SIZE = 192; + + std::unique_ptr socket_{nullptr}; + network::IPAddress server_ip_; + uint8_t buffer_[DNS_BUFFER_SIZE]; +}; + +} // namespace esphome::captive_portal + +#endif // USE_ESP_IDF diff --git a/esphome/components/ccs811/ccs811.cpp b/esphome/components/ccs811/ccs811.cpp index cecb92b3df..84355f2793 100644 --- a/esphome/components/ccs811/ccs811.cpp +++ b/esphome/components/ccs811/ccs811.cpp @@ -152,10 +152,10 @@ void CCS811Component::send_env_data_() { void CCS811Component::dump_config() { ESP_LOGCONFIG(TAG, "CCS811"); LOG_I2C_DEVICE(this) - LOG_UPDATE_INTERVAL(this) - LOG_SENSOR(" ", "CO2 Sensor", this->co2_) - LOG_SENSOR(" ", "TVOC Sensor", this->tvoc_) - LOG_TEXT_SENSOR(" ", "Firmware Version Sensor", this->version_) + LOG_UPDATE_INTERVAL(this); + LOG_SENSOR(" ", "CO2 Sensor", this->co2_); + LOG_SENSOR(" ", "TVOC Sensor", this->tvoc_); + LOG_TEXT_SENSOR(" ", "Firmware Version Sensor", this->version_); if (this->baseline_) { ESP_LOGCONFIG(TAG, " Baseline: %04X", *this->baseline_); } else { diff --git a/esphome/components/cd74hc4067/cd74hc4067.cpp b/esphome/components/cd74hc4067/cd74hc4067.cpp index 3c7b9038d7..174dc676f9 100644 --- a/esphome/components/cd74hc4067/cd74hc4067.cpp +++ b/esphome/components/cd74hc4067/cd74hc4067.cpp @@ -10,8 +10,6 @@ static const char *const TAG = "cd74hc4067"; float CD74HC4067Component::get_setup_priority() const { return setup_priority::DATA; } void CD74HC4067Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->pin_s0_->setup(); this->pin_s1_->setup(); this->pin_s2_->setup(); diff --git a/esphome/components/ch422g/ch422g.cpp b/esphome/components/ch422g/ch422g.cpp index 325c56e470..9a4e342525 100644 --- a/esphome/components/ch422g/ch422g.cpp +++ b/esphome/components/ch422g/ch422g.cpp @@ -14,7 +14,6 @@ static const uint8_t CH422G_REG_OUT_UPPER = 0x23; // write reg for output bit static const char *const TAG = "ch422g"; void CH422GComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // set outputs before mode this->write_outputs_(); // Set mode and check for errors @@ -92,7 +91,7 @@ bool CH422GComponent::read_inputs_() { // Write a register. Can't use the standard write_byte() method because there is no single pre-configured i2c address. bool CH422GComponent::write_reg_(uint8_t reg, uint8_t value) { - auto err = this->bus_->write(reg, &value, 1); + auto err = this->bus_->write_readv(reg, &value, 1, nullptr, 0); if (err != i2c::ERROR_OK) { this->status_set_warning(str_sprintf("write failed for register 0x%X, error %d", reg, err).c_str()); return false; @@ -103,7 +102,7 @@ bool CH422GComponent::write_reg_(uint8_t reg, uint8_t value) { uint8_t CH422GComponent::read_reg_(uint8_t reg) { uint8_t value; - auto err = this->bus_->read(reg, &value, 1); + auto err = this->bus_->write_readv(reg, nullptr, 0, &value, 1); if (err != i2c::ERROR_OK) { this->status_set_warning(str_sprintf("read failed for register 0x%X, error %d", reg, err).c_str()); return 0; diff --git a/esphome/components/chsc6x/chsc6x_touchscreen.cpp b/esphome/components/chsc6x/chsc6x_touchscreen.cpp index 524fa1eb36..31c9466691 100644 --- a/esphome/components/chsc6x/chsc6x_touchscreen.cpp +++ b/esphome/components/chsc6x/chsc6x_touchscreen.cpp @@ -4,7 +4,6 @@ namespace esphome { namespace chsc6x { void CHSC6XTouchscreen::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (this->interrupt_pin_ != nullptr) { this->interrupt_pin_->setup(); this->attach_interrupt_(this->interrupt_pin_, gpio::INTERRUPT_FALLING_EDGE); @@ -15,8 +14,6 @@ void CHSC6XTouchscreen::setup() { if (this->y_raw_max_ == this->y_raw_min_) { this->y_raw_max_ = this->display_->get_native_height(); } - - ESP_LOGCONFIG(TAG, "CHSC6X Touchscreen setup complete"); } void CHSC6XTouchscreen::update_touches() { diff --git a/esphome/components/climate/__init__.py b/esphome/components/climate/__init__.py index 9530ecdcca..c0c33d7242 100644 --- a/esphome/components/climate/__init__.py +++ b/esphome/components/climate/__init__.py @@ -47,7 +47,7 @@ from esphome.const import ( CONF_VISUAL, CONF_WEB_SERVER, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass @@ -517,7 +517,6 @@ async def climate_control_to_code(config, action_id, template_arg, args): return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_CLIMATE") cg.add_global(climate_ns.using) diff --git a/esphome/components/climate/climate.cpp b/esphome/components/climate/climate.cpp index edebc0de69..f3c93ed44e 100644 --- a/esphome/components/climate/climate.cpp +++ b/esphome/components/climate/climate.cpp @@ -96,7 +96,8 @@ void ClimateCall::validate_() { } if (this->target_temperature_.has_value()) { auto target = *this->target_temperature_; - if (traits.get_supports_two_point_target_temperature()) { + if (traits.has_feature_flags(CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | + CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE)) { ESP_LOGW(TAG, " Cannot set target temperature for climate device " "with two-point target temperature!"); this->target_temperature_.reset(); @@ -106,7 +107,8 @@ void ClimateCall::validate_() { } } if (this->target_temperature_low_.has_value() || this->target_temperature_high_.has_value()) { - if (!traits.get_supports_two_point_target_temperature()) { + if (!traits.has_feature_flags(CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | + CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE)) { ESP_LOGW(TAG, " Cannot set low/high target temperature for this device!"); this->target_temperature_low_.reset(); this->target_temperature_high_.reset(); @@ -327,7 +329,7 @@ void Climate::add_on_control_callback(std::function &&callb static const uint32_t RESTORE_STATE_VERSION = 0x848EA6ADUL; optional Climate::restore_state_() { - this->rtc_ = global_preferences->make_preference(this->get_object_id_hash() ^ + this->rtc_ = global_preferences->make_preference(this->get_preference_hash() ^ RESTORE_STATE_VERSION); ClimateDeviceRestoreState recovered{}; if (!this->rtc_.load(&recovered)) @@ -350,13 +352,14 @@ void Climate::save_state_() { state.mode = this->mode; auto traits = this->get_traits(); - if (traits.get_supports_two_point_target_temperature()) { + if (traits.has_feature_flags(CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | + CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE)) { state.target_temperature_low = this->target_temperature_low; state.target_temperature_high = this->target_temperature_high; } else { state.target_temperature = this->target_temperature; } - if (traits.get_supports_target_humidity()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TARGET_HUMIDITY)) { state.target_humidity = this->target_humidity; } if (traits.get_supports_fan_modes() && fan_mode.has_value()) { @@ -367,9 +370,11 @@ void Climate::save_state_() { state.uses_custom_fan_mode = true; const auto &supported = traits.get_supported_custom_fan_modes(); std::vector vec{supported.begin(), supported.end()}; - auto it = std::find(vec.begin(), vec.end(), custom_fan_mode); - if (it != vec.end()) { - state.custom_fan_mode = std::distance(vec.begin(), it); + for (size_t i = 0; i < vec.size(); i++) { + if (vec[i] == custom_fan_mode) { + state.custom_fan_mode = i; + break; + } } } if (traits.get_supports_presets() && preset.has_value()) { @@ -380,10 +385,11 @@ void Climate::save_state_() { state.uses_custom_preset = true; const auto &supported = traits.get_supported_custom_presets(); std::vector vec{supported.begin(), supported.end()}; - auto it = std::find(vec.begin(), vec.end(), custom_preset); - // only set custom preset if value exists, otherwise leave it as is - if (it != vec.cend()) { - state.custom_preset = std::distance(vec.begin(), it); + for (size_t i = 0; i < vec.size(); i++) { + if (vec[i] == custom_preset) { + state.custom_preset = i; + break; + } } } if (traits.get_supports_swing_modes()) { @@ -397,7 +403,7 @@ void Climate::publish_state() { auto traits = this->get_traits(); ESP_LOGD(TAG, " Mode: %s", LOG_STR_ARG(climate_mode_to_string(this->mode))); - if (traits.get_supports_action()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_ACTION)) { ESP_LOGD(TAG, " Action: %s", LOG_STR_ARG(climate_action_to_string(this->action))); } if (traits.get_supports_fan_modes() && this->fan_mode.has_value()) { @@ -415,19 +421,20 @@ void Climate::publish_state() { if (traits.get_supports_swing_modes()) { ESP_LOGD(TAG, " Swing Mode: %s", LOG_STR_ARG(climate_swing_mode_to_string(this->swing_mode))); } - if (traits.get_supports_current_temperature()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_TEMPERATURE)) { ESP_LOGD(TAG, " Current Temperature: %.2f°C", this->current_temperature); } - if (traits.get_supports_two_point_target_temperature()) { + if (traits.has_feature_flags(CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | + CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE)) { ESP_LOGD(TAG, " Target Temperature: Low: %.2f°C High: %.2f°C", this->target_temperature_low, this->target_temperature_high); } else { ESP_LOGD(TAG, " Target Temperature: %.2f°C", this->target_temperature); } - if (traits.get_supports_current_humidity()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_HUMIDITY)) { ESP_LOGD(TAG, " Current Humidity: %.0f%%", this->current_humidity); } - if (traits.get_supports_target_humidity()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TARGET_HUMIDITY)) { ESP_LOGD(TAG, " Target Humidity: %.0f%%", this->target_humidity); } @@ -482,13 +489,14 @@ ClimateCall ClimateDeviceRestoreState::to_call(Climate *climate) { auto call = climate->make_call(); auto traits = climate->get_traits(); call.set_mode(this->mode); - if (traits.get_supports_two_point_target_temperature()) { + if (traits.has_feature_flags(CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | + CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE)) { call.set_target_temperature_low(this->target_temperature_low); call.set_target_temperature_high(this->target_temperature_high); } else { call.set_target_temperature(this->target_temperature); } - if (traits.get_supports_target_humidity()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TARGET_HUMIDITY)) { call.set_target_humidity(this->target_humidity); } if (traits.get_supports_fan_modes() || !traits.get_supported_custom_fan_modes().empty()) { @@ -505,13 +513,14 @@ ClimateCall ClimateDeviceRestoreState::to_call(Climate *climate) { void ClimateDeviceRestoreState::apply(Climate *climate) { auto traits = climate->get_traits(); climate->mode = this->mode; - if (traits.get_supports_two_point_target_temperature()) { + if (traits.has_feature_flags(CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | + CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE)) { climate->target_temperature_low = this->target_temperature_low; climate->target_temperature_high = this->target_temperature_high; } else { climate->target_temperature = this->target_temperature; } - if (traits.get_supports_target_humidity()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TARGET_HUMIDITY)) { climate->target_humidity = this->target_humidity; } if (traits.get_supports_fan_modes() && !this->uses_custom_fan_mode) { @@ -577,28 +586,30 @@ void Climate::dump_traits_(const char *tag) { " Target: %.1f", traits.get_visual_min_temperature(), traits.get_visual_max_temperature(), traits.get_visual_target_temperature_step()); - if (traits.get_supports_current_temperature()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_TEMPERATURE)) { ESP_LOGCONFIG(tag, " Current: %.1f", traits.get_visual_current_temperature_step()); } - if (traits.get_supports_target_humidity() || traits.get_supports_current_humidity()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TARGET_HUMIDITY | + climate::CLIMATE_SUPPORTS_CURRENT_HUMIDITY)) { ESP_LOGCONFIG(tag, " - Min humidity: %.0f\n" " - Max humidity: %.0f", traits.get_visual_min_humidity(), traits.get_visual_max_humidity()); } - if (traits.get_supports_two_point_target_temperature()) { + if (traits.has_feature_flags(CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | + CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE)) { ESP_LOGCONFIG(tag, " [x] Supports two-point target temperature"); } - if (traits.get_supports_current_temperature()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_TEMPERATURE)) { ESP_LOGCONFIG(tag, " [x] Supports current temperature"); } - if (traits.get_supports_target_humidity()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TARGET_HUMIDITY)) { ESP_LOGCONFIG(tag, " [x] Supports target humidity"); } - if (traits.get_supports_current_humidity()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_HUMIDITY)) { ESP_LOGCONFIG(tag, " [x] Supports current humidity"); } - if (traits.get_supports_action()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_ACTION)) { ESP_LOGCONFIG(tag, " [x] Supports action"); } if (!traits.get_supported_modes().empty()) { diff --git a/esphome/components/climate/climate_mode.h b/esphome/components/climate/climate_mode.h index 80efb4c048..faec5d2537 100644 --- a/esphome/components/climate/climate_mode.h +++ b/esphome/components/climate/climate_mode.h @@ -98,6 +98,21 @@ enum ClimatePreset : uint8_t { CLIMATE_PRESET_ACTIVITY = 7, }; +enum ClimateFeature : uint32_t { + // Reporting current temperature is supported + CLIMATE_SUPPORTS_CURRENT_TEMPERATURE = 1 << 0, + // Setting two target temperatures is supported (used in conjunction with CLIMATE_MODE_HEAT_COOL) + CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE = 1 << 1, + // Single-point mode is NOT supported (UI always displays two handles, setting 'target_temperature' is not supported) + CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE = 1 << 2, + // Reporting current humidity is supported + CLIMATE_SUPPORTS_CURRENT_HUMIDITY = 1 << 3, + // Setting a target humidity is supported + CLIMATE_SUPPORTS_TARGET_HUMIDITY = 1 << 4, + // Reporting current climate action is supported + CLIMATE_SUPPORTS_ACTION = 1 << 5, +}; + /// Convert the given ClimateMode to a human-readable string. const LogString *climate_mode_to_string(ClimateMode mode); diff --git a/esphome/components/climate/climate_traits.h b/esphome/components/climate/climate_traits.h index c3a0dfca8f..50c1e79ad2 100644 --- a/esphome/components/climate/climate_traits.h +++ b/esphome/components/climate/climate_traits.h @@ -5,6 +5,13 @@ #include namespace esphome { + +#ifdef USE_API +namespace api { +class APIConnection; +} // namespace api +#endif + namespace climate { /** This class contains all static data for climate devices. @@ -14,48 +21,92 @@ namespace climate { * - Target Temperature * * All other properties and modes are optional and the integration must mark - * each of them as supported by setting the appropriate flag here. + * each of them as supported by setting the appropriate flag(s) here. * - * - supports current temperature - if the climate device supports reporting a current temperature - * - supports two point target temperature - if the climate device's target temperature should be - * split in target_temperature_low and target_temperature_high instead of just the single target_temperature + * - feature flags: see ClimateFeatures enum in climate_mode.h * - supports modes: * - auto mode (automatic control) * - cool mode (lowers current temperature) * - heat mode (increases current temperature) * - dry mode (removes humidity from air) * - fan mode (only turns on fan) - * - supports action - if the climate device supports reporting the active - * current action of the device with the action property. * - supports fan modes - optionally, if it has a fan which can be configured in different ways: * - on, off, auto, high, medium, low, middle, focus, diffuse, quiet * - supports swing modes - optionally, if it has a swing which can be configured in different ways: * - off, both, vertical, horizontal * * This class also contains static data for the climate device display: - * - visual min/max temperature - tells the frontend what range of temperatures the climate device - * should display (gauge min/max values) + * - visual min/max temperature/humidity - tells the frontend what range of temperature/humidity the + * climate device should display (gauge min/max values) * - temperature step - the step with which to increase/decrease target temperature. * This also affects with how many decimal places the temperature is shown */ class ClimateTraits { public: - bool get_supports_current_temperature() const { return this->supports_current_temperature_; } + /// Get/set feature flags (see ClimateFeatures enum in climate_mode.h) + uint32_t get_feature_flags() const { return this->feature_flags_; } + void add_feature_flags(uint32_t feature_flags) { this->feature_flags_ |= feature_flags; } + void clear_feature_flags(uint32_t feature_flags) { this->feature_flags_ &= ~feature_flags; } + bool has_feature_flags(uint32_t feature_flags) const { return this->feature_flags_ & feature_flags; } + void set_feature_flags(uint32_t feature_flags) { this->feature_flags_ = feature_flags; } + + ESPDEPRECATED("This method is deprecated, use get_feature_flags() instead", "2025.11.0") + bool get_supports_current_temperature() const { + return this->has_feature_flags(CLIMATE_SUPPORTS_CURRENT_TEMPERATURE); + } + ESPDEPRECATED("This method is deprecated, use add_feature_flags() instead", "2025.11.0") void set_supports_current_temperature(bool supports_current_temperature) { - this->supports_current_temperature_ = supports_current_temperature; + if (supports_current_temperature) { + this->add_feature_flags(CLIMATE_SUPPORTS_CURRENT_TEMPERATURE); + } else { + this->clear_feature_flags(CLIMATE_SUPPORTS_CURRENT_TEMPERATURE); + } } - bool get_supports_current_humidity() const { return this->supports_current_humidity_; } + ESPDEPRECATED("This method is deprecated, use get_feature_flags() instead", "2025.11.0") + bool get_supports_current_humidity() const { return this->has_feature_flags(CLIMATE_SUPPORTS_CURRENT_HUMIDITY); } + ESPDEPRECATED("This method is deprecated, use add_feature_flags() instead", "2025.11.0") void set_supports_current_humidity(bool supports_current_humidity) { - this->supports_current_humidity_ = supports_current_humidity; + if (supports_current_humidity) { + this->add_feature_flags(CLIMATE_SUPPORTS_CURRENT_HUMIDITY); + } else { + this->clear_feature_flags(CLIMATE_SUPPORTS_CURRENT_HUMIDITY); + } } - bool get_supports_two_point_target_temperature() const { return this->supports_two_point_target_temperature_; } + ESPDEPRECATED("This method is deprecated, use get_feature_flags() instead", "2025.11.0") + bool get_supports_two_point_target_temperature() const { + return this->has_feature_flags(CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE); + } + ESPDEPRECATED("This method is deprecated, use add_feature_flags() instead", "2025.11.0") void set_supports_two_point_target_temperature(bool supports_two_point_target_temperature) { - this->supports_two_point_target_temperature_ = supports_two_point_target_temperature; + if (supports_two_point_target_temperature) + // Use CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE to mimic previous behavior + { + this->add_feature_flags(CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE); + } else { + this->clear_feature_flags(CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE); + } } - bool get_supports_target_humidity() const { return this->supports_target_humidity_; } + ESPDEPRECATED("This method is deprecated, use get_feature_flags() instead", "2025.11.0") + bool get_supports_target_humidity() const { return this->has_feature_flags(CLIMATE_SUPPORTS_TARGET_HUMIDITY); } + ESPDEPRECATED("This method is deprecated, use add_feature_flags() instead", "2025.11.0") void set_supports_target_humidity(bool supports_target_humidity) { - this->supports_target_humidity_ = supports_target_humidity; + if (supports_target_humidity) { + this->add_feature_flags(CLIMATE_SUPPORTS_TARGET_HUMIDITY); + } else { + this->clear_feature_flags(CLIMATE_SUPPORTS_TARGET_HUMIDITY); + } } + ESPDEPRECATED("This method is deprecated, use get_feature_flags() instead", "2025.11.0") + bool get_supports_action() const { return this->has_feature_flags(CLIMATE_SUPPORTS_ACTION); } + ESPDEPRECATED("This method is deprecated, use add_feature_flags() instead", "2025.11.0") + void set_supports_action(bool supports_action) { + if (supports_action) { + this->add_feature_flags(CLIMATE_SUPPORTS_ACTION); + } else { + this->clear_feature_flags(CLIMATE_SUPPORTS_ACTION); + } + } + void set_supported_modes(std::set modes) { this->supported_modes_ = std::move(modes); } void add_supported_mode(ClimateMode mode) { this->supported_modes_.insert(mode); } ESPDEPRECATED("This method is deprecated, use set_supported_modes() instead", "v1.20") @@ -75,9 +126,6 @@ class ClimateTraits { bool supports_mode(ClimateMode mode) const { return this->supported_modes_.count(mode); } const std::set &get_supported_modes() const { return this->supported_modes_; } - void set_supports_action(bool supports_action) { this->supports_action_ = supports_action; } - bool get_supports_action() const { return this->supports_action_; } - void set_supported_fan_modes(std::set modes) { this->supported_fan_modes_ = std::move(modes); } void add_supported_fan_mode(ClimateFanMode mode) { this->supported_fan_modes_.insert(mode); } void add_supported_custom_fan_mode(const std::string &mode) { this->supported_custom_fan_modes_.insert(mode); } @@ -173,6 +221,23 @@ class ClimateTraits { void set_visual_max_humidity(float visual_max_humidity) { this->visual_max_humidity_ = visual_max_humidity; } protected: +#ifdef USE_API + // The API connection is a friend class to access internal methods + friend class api::APIConnection; + // These methods return references to internal data structures. + // They are used by the API to avoid copying data when encoding messages. + // Warning: Do not use these methods outside of the API connection code. + // They return references to internal data that can be invalidated. + const std::set &get_supported_modes_for_api_() const { return this->supported_modes_; } + const std::set &get_supported_fan_modes_for_api_() const { return this->supported_fan_modes_; } + const std::set &get_supported_custom_fan_modes_for_api_() const { + return this->supported_custom_fan_modes_; + } + const std::set &get_supported_presets_for_api_() const { return this->supported_presets_; } + const std::set &get_supported_custom_presets_for_api_() const { return this->supported_custom_presets_; } + const std::set &get_supported_swing_modes_for_api_() const { return this->supported_swing_modes_; } +#endif + void set_mode_support_(climate::ClimateMode mode, bool supported) { if (supported) { this->supported_modes_.insert(mode); @@ -195,24 +260,20 @@ class ClimateTraits { } } - bool supports_current_temperature_{false}; - bool supports_current_humidity_{false}; - bool supports_two_point_target_temperature_{false}; - bool supports_target_humidity_{false}; - std::set supported_modes_ = {climate::CLIMATE_MODE_OFF}; - bool supports_action_{false}; - std::set supported_fan_modes_; - std::set supported_swing_modes_; - std::set supported_presets_; - std::set supported_custom_fan_modes_; - std::set supported_custom_presets_; - + uint32_t feature_flags_{0}; float visual_min_temperature_{10}; float visual_max_temperature_{30}; float visual_target_temperature_step_{0.1}; float visual_current_temperature_step_{0.1}; float visual_min_humidity_{30}; float visual_max_humidity_{99}; + + std::set supported_modes_ = {climate::CLIMATE_MODE_OFF}; + std::set supported_fan_modes_; + std::set supported_swing_modes_; + std::set supported_presets_; + std::set supported_custom_fan_modes_; + std::set supported_custom_presets_; }; } // namespace climate diff --git a/esphome/components/cm1106/cm1106.cpp b/esphome/components/cm1106/cm1106.cpp index 109524c04a..d88ea2e1da 100644 --- a/esphome/components/cm1106/cm1106.cpp +++ b/esphome/components/cm1106/cm1106.cpp @@ -13,14 +13,13 @@ static const uint8_t C_M1106_CMD_SET_CO2_CALIB_RESPONSE[4] = {0x16, 0x01, 0x03, uint8_t cm1106_checksum(const uint8_t *response, size_t len) { uint8_t crc = 0; - for (int i = 0; i < len - 1; i++) { + for (size_t i = 0; i < len - 1; i++) { crc -= response[i]; } return crc; } void CM1106Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t response[8] = {0}; if (!this->cm1106_write_command_(C_M1106_CMD_GET_CO2, sizeof(C_M1106_CMD_GET_CO2), response, sizeof(response))) { ESP_LOGE(TAG, ESP_LOG_MSG_COMM_FAIL); diff --git a/esphome/components/const/__init__.py b/esphome/components/const/__init__.py index b084622f4c..19924f0da7 100644 --- a/esphome/components/const/__init__.py +++ b/esphome/components/const/__init__.py @@ -3,6 +3,12 @@ CODEOWNERS = ["@esphome/core"] CONF_BYTE_ORDER = "byte_order" +BYTE_ORDER_LITTLE = "little_endian" +BYTE_ORDER_BIG = "big_endian" + +CONF_COLOR_DEPTH = "color_depth" CONF_DRAW_ROUNDING = "draw_rounding" +CONF_ON_RECEIVE = "on_receive" CONF_ON_STATE_CHANGE = "on_state_change" CONF_REQUEST_HEADERS = "request_headers" +CONF_USE_PSRAM = "use_psram" diff --git a/esphome/components/copy/lock/copy_lock.cpp b/esphome/components/copy/lock/copy_lock.cpp index 67a8acffec..25bd8c33ef 100644 --- a/esphome/components/copy/lock/copy_lock.cpp +++ b/esphome/components/copy/lock/copy_lock.cpp @@ -11,7 +11,7 @@ void CopyLock::setup() { traits.set_assumed_state(source_->traits.get_assumed_state()); traits.set_requires_code(source_->traits.get_requires_code()); - traits.set_supported_states(source_->traits.get_supported_states()); + traits.set_supported_states_mask(source_->traits.get_supported_states_mask()); traits.set_supports_open(source_->traits.get_supports_open()); this->publish_state(source_->state); diff --git a/esphome/components/cover/__init__.py b/esphome/components/cover/__init__.py index cd97a38ecc..bec6dcbdac 100644 --- a/esphome/components/cover/__init__.py +++ b/esphome/components/cover/__init__.py @@ -32,7 +32,7 @@ from esphome.const import ( DEVICE_CLASS_SHUTTER, DEVICE_CLASS_WINDOW, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass @@ -228,9 +228,9 @@ async def cover_stop_to_code(config, action_id, template_arg, args): @automation.register_action("cover.toggle", ToggleAction, COVER_ACTION_SCHEMA) -def cover_toggle_to_code(config, action_id, template_arg, args): - paren = yield cg.get_variable(config[CONF_ID]) - yield cg.new_Pvariable(action_id, template_arg, paren) +async def cover_toggle_to_code(config, action_id, template_arg, args): + paren = await cg.get_variable(config[CONF_ID]) + return cg.new_Pvariable(action_id, template_arg, paren) COVER_CONTROL_ACTION_SCHEMA = cv.Schema( @@ -263,7 +263,6 @@ async def cover_control_to_code(config, action_id, template_arg, args): return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_COVER") cg.add_global(cover_ns.using) diff --git a/esphome/components/cover/cover.cpp b/esphome/components/cover/cover.cpp index d139bab8ee..3378279371 100644 --- a/esphome/components/cover/cover.cpp +++ b/esphome/components/cover/cover.cpp @@ -1,5 +1,6 @@ #include "cover.h" #include "esphome/core/log.h" +#include namespace esphome { namespace cover { @@ -99,43 +100,39 @@ const optional &CoverCall::get_tilt() const { return this->tilt_; } const optional &CoverCall::get_toggle() const { return this->toggle_; } void CoverCall::validate_() { auto traits = this->parent_->get_traits(); + const char *name = this->parent_->get_name().c_str(); + if (this->position_.has_value()) { auto pos = *this->position_; if (!traits.get_supports_position() && pos != COVER_OPEN && pos != COVER_CLOSED) { - ESP_LOGW(TAG, "'%s' - This cover device does not support setting position!", this->parent_->get_name().c_str()); + ESP_LOGW(TAG, "'%s': position unsupported", name); this->position_.reset(); } else if (pos < 0.0f || pos > 1.0f) { - ESP_LOGW(TAG, "'%s' - Position %.2f is out of range [0.0 - 1.0]", this->parent_->get_name().c_str(), pos); + ESP_LOGW(TAG, "'%s': position %.2f out of range", name, pos); this->position_ = clamp(pos, 0.0f, 1.0f); } } if (this->tilt_.has_value()) { auto tilt = *this->tilt_; if (!traits.get_supports_tilt()) { - ESP_LOGW(TAG, "'%s' - This cover device does not support tilt!", this->parent_->get_name().c_str()); + ESP_LOGW(TAG, "'%s': tilt unsupported", name); this->tilt_.reset(); } else if (tilt < 0.0f || tilt > 1.0f) { - ESP_LOGW(TAG, "'%s' - Tilt %.2f is out of range [0.0 - 1.0]", this->parent_->get_name().c_str(), tilt); + ESP_LOGW(TAG, "'%s': tilt %.2f out of range", name, tilt); this->tilt_ = clamp(tilt, 0.0f, 1.0f); } } if (this->toggle_.has_value()) { if (!traits.get_supports_toggle()) { - ESP_LOGW(TAG, "'%s' - This cover device does not support toggle!", this->parent_->get_name().c_str()); + ESP_LOGW(TAG, "'%s': toggle unsupported", name); this->toggle_.reset(); } } if (this->stop_) { - if (this->position_.has_value()) { - ESP_LOGW(TAG, "Cannot set position when stopping a cover!"); + if (this->position_.has_value() || this->tilt_.has_value() || this->toggle_.has_value()) { + ESP_LOGW(TAG, "'%s': cannot position/tilt/toggle when stopping", name); this->position_.reset(); - } - if (this->tilt_.has_value()) { - ESP_LOGW(TAG, "Cannot set tilt when stopping a cover!"); this->tilt_.reset(); - } - if (this->toggle_.has_value()) { - ESP_LOGW(TAG, "Cannot set toggle when stopping a cover!"); this->toggle_.reset(); } } @@ -198,7 +195,7 @@ void Cover::publish_state(bool save) { } } optional Cover::restore_state_() { - this->rtc_ = global_preferences->make_preference(this->get_object_id_hash()); + this->rtc_ = global_preferences->make_preference(this->get_preference_hash()); CoverRestoreState recovered{}; if (!this->rtc_.load(&recovered)) return {}; diff --git a/esphome/components/cover/cover.h b/esphome/components/cover/cover.h index 8b6f5b8a72..ada5953d57 100644 --- a/esphome/components/cover/cover.h +++ b/esphome/components/cover/cover.h @@ -19,8 +19,8 @@ const extern float COVER_CLOSED; if (traits_.get_is_assumed_state()) { \ ESP_LOGCONFIG(TAG, "%s Assumed State: YES", prefix); \ } \ - if (!(obj)->get_device_class().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Device Class: '%s'", prefix, (obj)->get_device_class().c_str()); \ + if (!(obj)->get_device_class_ref().empty()) { \ + ESP_LOGCONFIG(TAG, "%s Device Class: '%s'", prefix, (obj)->get_device_class_ref().c_str()); \ } \ } diff --git a/esphome/components/cs5460a/cs5460a.cpp b/esphome/components/cs5460a/cs5460a.cpp index e3a5941d94..e026eccf80 100644 --- a/esphome/components/cs5460a/cs5460a.cpp +++ b/esphome/components/cs5460a/cs5460a.cpp @@ -52,8 +52,6 @@ bool CS5460AComponent::softreset_() { } void CS5460AComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - float current_full_scale = (pga_gain_ == CS5460A_PGA_GAIN_10X) ? 0.25 : 0.10; float voltage_full_scale = 0.25; current_multiplier_ = current_full_scale / (fabsf(current_gain_) * 0x1000000); diff --git a/esphome/components/cse7761/cse7761.cpp b/esphome/components/cse7761/cse7761.cpp index 6c3d457f26..482636dd81 100644 --- a/esphome/components/cse7761/cse7761.cpp +++ b/esphome/components/cse7761/cse7761.cpp @@ -42,7 +42,6 @@ static const uint8_t CSE7761_CMD_ENABLE_WRITE = 0xE5; // Enable write operation enum CSE7761 { RMS_IAC, RMS_IBC, RMS_UC, POWER_PAC, POWER_PBC, POWER_SC, ENERGY_AC, ENERGY_BC }; void CSE7761Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->write_(CSE7761_SPECIAL_COMMAND, CSE7761_CMD_RESET); uint16_t syscon = this->read_(0x00, 2); // Default 0x0A04 if ((0x0A04 == syscon) && this->chip_init_()) { diff --git a/esphome/components/cst226/touchscreen/cst226_touchscreen.cpp b/esphome/components/cst226/touchscreen/cst226_touchscreen.cpp index c444dd7485..e65997b7fc 100644 --- a/esphome/components/cst226/touchscreen/cst226_touchscreen.cpp +++ b/esphome/components/cst226/touchscreen/cst226_touchscreen.cpp @@ -6,7 +6,6 @@ namespace cst226 { static const char *const TAG = "cst226.touchscreen"; void CST226Touchscreen::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (this->reset_pin_ != nullptr) { this->reset_pin_->setup(); this->reset_pin_->digital_write(true); @@ -95,7 +94,6 @@ void CST226Touchscreen::continue_setup_() { } } this->setup_complete_ = true; - ESP_LOGCONFIG(TAG, "CST226 Touchscreen setup complete"); } void CST226Touchscreen::update_button_state_(bool state) { if (this->button_touched_ == state) diff --git a/esphome/components/cst816/touchscreen/cst816_touchscreen.cpp b/esphome/components/cst816/touchscreen/cst816_touchscreen.cpp index 0c5099d4f0..0ba2d9df94 100644 --- a/esphome/components/cst816/touchscreen/cst816_touchscreen.cpp +++ b/esphome/components/cst816/touchscreen/cst816_touchscreen.cpp @@ -35,11 +35,9 @@ void CST816Touchscreen::continue_setup_() { if (this->y_raw_max_ == this->y_raw_min_) { this->y_raw_max_ = this->display_->get_native_height(); } - ESP_LOGCONFIG(TAG, "CST816 Touchscreen setup complete"); } void CST816Touchscreen::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (this->reset_pin_ != nullptr) { this->reset_pin_->setup(); this->reset_pin_->digital_write(true); diff --git a/esphome/components/dac7678/dac7678_output.cpp b/esphome/components/dac7678/dac7678_output.cpp index 5c10bbc1bc..83f8722e7f 100644 --- a/esphome/components/dac7678/dac7678_output.cpp +++ b/esphome/components/dac7678/dac7678_output.cpp @@ -20,8 +20,6 @@ static const uint8_t DAC7678_REG_INTERNAL_REF_0 = 0x80; static const uint8_t DAC7678_REG_INTERNAL_REF_1 = 0x90; void DAC7678Output::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - ESP_LOGV(TAG, "Resetting device"); // Reset device diff --git a/esphome/components/daikin_arc/daikin_arc.cpp b/esphome/components/daikin_arc/daikin_arc.cpp index f806463d00..068819ecd1 100644 --- a/esphome/components/daikin_arc/daikin_arc.cpp +++ b/esphome/components/daikin_arc/daikin_arc.cpp @@ -26,7 +26,7 @@ void DaikinArcClimate::transmit_query_() { uint8_t remote_header[8] = {0x11, 0xDA, 0x27, 0x00, 0x84, 0x87, 0x20, 0x00}; // Calculate checksum - for (int i = 0; i < sizeof(remote_header) - 1; i++) { + for (size_t i = 0; i < sizeof(remote_header) - 1; i++) { remote_header[sizeof(remote_header) - 1] += remote_header[i]; } @@ -102,7 +102,7 @@ void DaikinArcClimate::transmit_state() { remote_state[9] = fan_speed & 0xff; // Calculate checksum - for (int i = 0; i < sizeof(remote_header) - 1; i++) { + for (size_t i = 0; i < sizeof(remote_header) - 1; i++) { remote_header[sizeof(remote_header) - 1] += remote_header[i]; } @@ -350,7 +350,7 @@ bool DaikinArcClimate::on_receive(remote_base::RemoteReceiveData data) { bool valid_daikin_frame = false; if (data.expect_item(DAIKIN_HEADER_MARK, DAIKIN_HEADER_SPACE)) { valid_daikin_frame = true; - int bytes_count = data.size() / 2 / 8; + size_t bytes_count = data.size() / 2 / 8; std::unique_ptr buf(new char[bytes_count * 3 + 1]); buf[0] = '\0'; for (size_t i = 0; i < bytes_count; i++) { @@ -370,7 +370,7 @@ bool DaikinArcClimate::on_receive(remote_base::RemoteReceiveData data) { if (!valid_daikin_frame) { char sbuf[16 * 10 + 1]; sbuf[0] = '\0'; - for (size_t j = 0; j < data.size(); j++) { + for (size_t j = 0; j < static_cast(data.size()); j++) { if ((j - 2) % 16 == 0) { if (j > 0) { ESP_LOGD(TAG, "DATA %04x: %s", (j - 16 > 0xffff ? 0 : j - 16), sbuf); @@ -380,19 +380,26 @@ bool DaikinArcClimate::on_receive(remote_base::RemoteReceiveData data) { char type_ch = ' '; // debug_tolerance = 25% - if (DAIKIN_DBG_LOWER(DAIKIN_ARC_PRE_MARK) <= data[j] && data[j] <= DAIKIN_DBG_UPPER(DAIKIN_ARC_PRE_MARK)) + if (static_cast(DAIKIN_DBG_LOWER(DAIKIN_ARC_PRE_MARK)) <= data[j] && + data[j] <= static_cast(DAIKIN_DBG_UPPER(DAIKIN_ARC_PRE_MARK))) type_ch = 'P'; - if (DAIKIN_DBG_LOWER(DAIKIN_ARC_PRE_SPACE) <= -data[j] && -data[j] <= DAIKIN_DBG_UPPER(DAIKIN_ARC_PRE_SPACE)) + if (static_cast(DAIKIN_DBG_LOWER(DAIKIN_ARC_PRE_SPACE)) <= -data[j] && + -data[j] <= static_cast(DAIKIN_DBG_UPPER(DAIKIN_ARC_PRE_SPACE))) type_ch = 'a'; - if (DAIKIN_DBG_LOWER(DAIKIN_HEADER_MARK) <= data[j] && data[j] <= DAIKIN_DBG_UPPER(DAIKIN_HEADER_MARK)) + if (static_cast(DAIKIN_DBG_LOWER(DAIKIN_HEADER_MARK)) <= data[j] && + data[j] <= static_cast(DAIKIN_DBG_UPPER(DAIKIN_HEADER_MARK))) type_ch = 'H'; - if (DAIKIN_DBG_LOWER(DAIKIN_HEADER_SPACE) <= -data[j] && -data[j] <= DAIKIN_DBG_UPPER(DAIKIN_HEADER_SPACE)) + if (static_cast(DAIKIN_DBG_LOWER(DAIKIN_HEADER_SPACE)) <= -data[j] && + -data[j] <= static_cast(DAIKIN_DBG_UPPER(DAIKIN_HEADER_SPACE))) type_ch = 'h'; - if (DAIKIN_DBG_LOWER(DAIKIN_BIT_MARK) <= data[j] && data[j] <= DAIKIN_DBG_UPPER(DAIKIN_BIT_MARK)) + if (static_cast(DAIKIN_DBG_LOWER(DAIKIN_BIT_MARK)) <= data[j] && + data[j] <= static_cast(DAIKIN_DBG_UPPER(DAIKIN_BIT_MARK))) type_ch = 'B'; - if (DAIKIN_DBG_LOWER(DAIKIN_ONE_SPACE) <= -data[j] && -data[j] <= DAIKIN_DBG_UPPER(DAIKIN_ONE_SPACE)) + if (static_cast(DAIKIN_DBG_LOWER(DAIKIN_ONE_SPACE)) <= -data[j] && + -data[j] <= static_cast(DAIKIN_DBG_UPPER(DAIKIN_ONE_SPACE))) type_ch = '1'; - if (DAIKIN_DBG_LOWER(DAIKIN_ZERO_SPACE) <= -data[j] && -data[j] <= DAIKIN_DBG_UPPER(DAIKIN_ZERO_SPACE)) + if (static_cast(DAIKIN_DBG_LOWER(DAIKIN_ZERO_SPACE)) <= -data[j] && + -data[j] <= static_cast(DAIKIN_DBG_UPPER(DAIKIN_ZERO_SPACE))) type_ch = '0'; if (abs(data[j]) > 100000) { @@ -400,7 +407,7 @@ bool DaikinArcClimate::on_receive(remote_base::RemoteReceiveData data) { } else { sprintf(sbuf, "%s%-5d[%c] ", sbuf, (int) (round(data[j] / 10.) * 10), type_ch); } - if (j == data.size() - 1) { + if (j + 1 == static_cast(data.size())) { ESP_LOGD(TAG, "DATA %04x: %s", (j - 8 > 0xffff ? 0 : j - 8), sbuf); } } diff --git a/esphome/components/dallas_temp/dallas_temp.cpp b/esphome/components/dallas_temp/dallas_temp.cpp index 3796a888fd..a518c96489 100644 --- a/esphome/components/dallas_temp/dallas_temp.cpp +++ b/esphome/components/dallas_temp/dallas_temp.cpp @@ -64,13 +64,12 @@ bool DallasTemperatureSensor::read_scratch_pad_() { } } else { ESP_LOGW(TAG, "'%s' - reading scratch pad failed bus reset", this->get_name().c_str()); - this->status_set_warning("bus reset failed"); + this->status_set_warning(LOG_STR("bus reset failed")); } return success; } void DallasTemperatureSensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (!this->check_address_()) return; if (!this->read_scratch_pad_()) @@ -125,7 +124,7 @@ bool DallasTemperatureSensor::check_scratch_pad_() { crc8(this->scratch_pad_, 8)); #endif if (!chksum_validity) { - this->status_set_warning("scratch pad checksum invalid"); + this->status_set_warning(LOG_STR("scratch pad checksum invalid")); ESP_LOGD(TAG, "Scratch pad: %02X.%02X.%02X.%02X.%02X.%02X.%02X.%02X.%02X (%02X)", this->scratch_pad_[0], this->scratch_pad_[1], this->scratch_pad_[2], this->scratch_pad_[3], this->scratch_pad_[4], this->scratch_pad_[5], this->scratch_pad_[6], this->scratch_pad_[7], this->scratch_pad_[8], diff --git a/esphome/components/dashboard_import/dashboard_import.cpp b/esphome/components/dashboard_import/dashboard_import.cpp index 6875fd61a5..c04696fd53 100644 --- a/esphome/components/dashboard_import/dashboard_import.cpp +++ b/esphome/components/dashboard_import/dashboard_import.cpp @@ -5,7 +5,7 @@ namespace dashboard_import { static std::string g_package_import_url; // NOLINT -std::string get_package_import_url() { return g_package_import_url; } +const std::string &get_package_import_url() { return g_package_import_url; } void set_package_import_url(std::string url) { g_package_import_url = std::move(url); } } // namespace dashboard_import diff --git a/esphome/components/dashboard_import/dashboard_import.h b/esphome/components/dashboard_import/dashboard_import.h index 0ca2994aab..edcda6b803 100644 --- a/esphome/components/dashboard_import/dashboard_import.h +++ b/esphome/components/dashboard_import/dashboard_import.h @@ -5,7 +5,7 @@ namespace esphome { namespace dashboard_import { -std::string get_package_import_url(); +const std::string &get_package_import_url(); void set_package_import_url(std::string url); } // namespace dashboard_import diff --git a/esphome/components/datetime/__init__.py b/esphome/components/datetime/__init__.py index 4788810965..602db3827a 100644 --- a/esphome/components/datetime/__init__.py +++ b/esphome/components/datetime/__init__.py @@ -21,7 +21,7 @@ from esphome.const import ( CONF_WEB_SERVER, CONF_YEAR, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass @@ -164,7 +164,6 @@ async def register_datetime(var, config): cg.add(getattr(cg.App, f"register_{entity_type}")(var)) CORE.register_platform_component(entity_type, var) await setup_datetime_core_(var, config) - cg.add_define(f"USE_DATETIME_{config[CONF_TYPE]}") async def new_datetime(config, *args): @@ -173,9 +172,8 @@ async def new_datetime(config, *args): return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_DATETIME") cg.add_global(datetime_ns.using) diff --git a/esphome/components/datetime/date_entity.h b/esphome/components/datetime/date_entity.h index ce43c5639d..fcbb46cf17 100644 --- a/esphome/components/datetime/date_entity.h +++ b/esphome/components/datetime/date_entity.h @@ -16,8 +16,8 @@ namespace datetime { #define LOG_DATETIME_DATE(prefix, type, obj) \ if ((obj) != nullptr) { \ ESP_LOGCONFIG(TAG, "%s%s '%s'", prefix, LOG_STR_LITERAL(type), (obj)->get_name().c_str()); \ - if (!(obj)->get_icon().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon().c_str()); \ + if (!(obj)->get_icon_ref().empty()) { \ + ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon_ref().c_str()); \ } \ } diff --git a/esphome/components/datetime/datetime_base.h b/esphome/components/datetime/datetime_base.h index b7645f5539..b5f54ac96f 100644 --- a/esphome/components/datetime/datetime_base.h +++ b/esphome/components/datetime/datetime_base.h @@ -30,14 +30,12 @@ class DateTimeBase : public EntityBase { #endif }; -#ifdef USE_TIME class DateTimeStateTrigger : public Trigger { public: explicit DateTimeStateTrigger(DateTimeBase *parent) { parent->add_on_state_callback([this, parent]() { this->trigger(parent->state_as_esptime()); }); } }; -#endif } // namespace datetime } // namespace esphome diff --git a/esphome/components/datetime/datetime_entity.h b/esphome/components/datetime/datetime_entity.h index 27db84cf7e..275eedfd3b 100644 --- a/esphome/components/datetime/datetime_entity.h +++ b/esphome/components/datetime/datetime_entity.h @@ -16,8 +16,8 @@ namespace datetime { #define LOG_DATETIME_DATETIME(prefix, type, obj) \ if ((obj) != nullptr) { \ ESP_LOGCONFIG(TAG, "%s%s '%s'", prefix, LOG_STR_LITERAL(type), (obj)->get_name().c_str()); \ - if (!(obj)->get_icon().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon().c_str()); \ + if (!(obj)->get_icon_ref().empty()) { \ + ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon_ref().c_str()); \ } \ } diff --git a/esphome/components/datetime/time_entity.h b/esphome/components/datetime/time_entity.h index f7e0a7ddd9..e79b8c225d 100644 --- a/esphome/components/datetime/time_entity.h +++ b/esphome/components/datetime/time_entity.h @@ -16,8 +16,8 @@ namespace datetime { #define LOG_DATETIME_TIME(prefix, type, obj) \ if ((obj) != nullptr) { \ ESP_LOGCONFIG(TAG, "%s%s '%s'", prefix, LOG_STR_LITERAL(type), (obj)->get_name().c_str()); \ - if (!(obj)->get_icon().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon().c_str()); \ + if (!(obj)->get_icon_ref().empty()) { \ + ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon_ref().c_str()); \ } \ } diff --git a/esphome/components/debug/__init__.py b/esphome/components/debug/__init__.py index 500dfac1fe..6b4205a545 100644 --- a/esphome/components/debug/__init__.py +++ b/esphome/components/debug/__init__.py @@ -1,4 +1,5 @@ import esphome.codegen as cg +from esphome.components.zephyr import zephyr_add_prj_conf from esphome.config_helpers import filter_source_files_from_platform import esphome.config_validation as cv from esphome.const import ( @@ -10,8 +11,9 @@ from esphome.const import ( CONF_LOOP_TIME, PlatformFramework, ) +from esphome.core import CORE -CODEOWNERS = ["@OttoWinter"] +CODEOWNERS = ["@esphome/core"] DEPENDENCIES = ["logger"] CONF_DEBUG_ID = "debug_id" @@ -44,6 +46,17 @@ CONFIG_SCHEMA = cv.All( async def to_code(config): + if CORE.using_zephyr: + zephyr_add_prj_conf("HWINFO", True) + # gdb thread support + zephyr_add_prj_conf("DEBUG_THREAD_INFO", True) + # RTT + zephyr_add_prj_conf("USE_SEGGER_RTT", True) + zephyr_add_prj_conf("RTT_CONSOLE", True) + zephyr_add_prj_conf("LOG", True) + zephyr_add_prj_conf("LOG_BLOCK_IN_THREAD", True) + zephyr_add_prj_conf("LOG_BUFFER_SIZE", 4096) + zephyr_add_prj_conf("SEGGER_RTT_MODE_BLOCK_IF_FIFO_FULL", True) var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) @@ -62,5 +75,6 @@ FILTER_SOURCE_FILES = filter_source_files_from_platform( PlatformFramework.RTL87XX_ARDUINO, PlatformFramework.LN882X_ARDUINO, }, + "debug_zephyr.cpp": {PlatformFramework.NRF52_ZEPHYR}, } ) diff --git a/esphome/components/debug/debug_esp32.cpp b/esphome/components/debug/debug_esp32.cpp index 37990aeec5..1c3dc3699b 100644 --- a/esphome/components/debug/debug_esp32.cpp +++ b/esphome/components/debug/debug_esp32.cpp @@ -11,8 +11,6 @@ #include #include -#include - #ifdef USE_ARDUINO #include #endif @@ -52,7 +50,7 @@ void DebugComponent::on_shutdown() { char buffer[REBOOT_MAX_LEN]{}; auto pref = global_preferences->make_preference(REBOOT_MAX_LEN, fnv1_hash(REBOOT_KEY + App.get_name())); if (component != nullptr) { - strncpy(buffer, component->get_component_source(), REBOOT_MAX_LEN - 1); + strncpy(buffer, LOG_STR_ARG(component->get_component_log_str()), REBOOT_MAX_LEN - 1); buffer[REBOOT_MAX_LEN - 1] = '\0'; } ESP_LOGD(TAG, "Storing reboot source: %s", buffer); @@ -125,7 +123,12 @@ void DebugComponent::log_partition_info_() { uint32_t DebugComponent::get_free_heap_() { return heap_caps_get_free_size(MALLOC_CAP_INTERNAL); } -static const std::map CHIP_FEATURES = { +struct ChipFeature { + int bit; + const char *name; +}; + +static constexpr ChipFeature CHIP_FEATURES[] = { {CHIP_FEATURE_BLE, "BLE"}, {CHIP_FEATURE_BT, "BT"}, {CHIP_FEATURE_EMB_FLASH, "EMB Flash"}, @@ -170,11 +173,13 @@ void DebugComponent::get_device_info_(std::string &device_info) { esp_chip_info(&info); const char *model = ESPHOME_VARIANT; std::string features; - for (auto feature : CHIP_FEATURES) { - if (info.features & feature.first) { - features += feature.second; + + // Check each known feature bit + for (const auto &feature : CHIP_FEATURES) { + if (info.features & feature.bit) { + features += feature.name; features += ", "; - info.features &= ~feature.first; + info.features &= ~feature.bit; } } if (info.features != 0) diff --git a/esphome/components/debug/debug_zephyr.cpp b/esphome/components/debug/debug_zephyr.cpp new file mode 100644 index 0000000000..231b39a711 --- /dev/null +++ b/esphome/components/debug/debug_zephyr.cpp @@ -0,0 +1,331 @@ +#include "debug_component.h" +#ifdef USE_ZEPHYR +#include +#include "esphome/core/log.h" +#include +#include +#include + +#define BOOTLOADER_VERSION_REGISTER NRF_TIMER2->CC[0] + +namespace esphome { +namespace debug { + +static const char *const TAG = "debug"; +constexpr std::uintptr_t MBR_PARAM_PAGE_ADDR = 0xFFC; +constexpr std::uintptr_t MBR_BOOTLOADER_ADDR = 0xFF8; + +static void show_reset_reason(std::string &reset_reason, bool set, const char *reason) { + if (!set) { + return; + } + if (!reset_reason.empty()) { + reset_reason += ", "; + } + reset_reason += reason; +} + +static inline uint32_t read_mem_u32(uintptr_t addr) { + return *reinterpret_cast(addr); // NOLINT(performance-no-int-to-ptr) +} + +static inline uint8_t read_mem_u8(uintptr_t addr) { + return *reinterpret_cast(addr); // NOLINT(performance-no-int-to-ptr) +} + +// defines from https://github.com/adafruit/Adafruit_nRF52_Bootloader which prints those information +constexpr uint32_t SD_MAGIC_NUMBER = 0x51B1E5DB; +constexpr uintptr_t MBR_SIZE = 0x1000; +constexpr uintptr_t SOFTDEVICE_INFO_STRUCT_OFFSET = 0x2000; +constexpr uintptr_t SD_ID_OFFSET = SOFTDEVICE_INFO_STRUCT_OFFSET + 0x10; +constexpr uintptr_t SD_VERSION_OFFSET = SOFTDEVICE_INFO_STRUCT_OFFSET + 0x14; + +static inline bool is_sd_present() { + return read_mem_u32(SOFTDEVICE_INFO_STRUCT_OFFSET + MBR_SIZE + 4) == SD_MAGIC_NUMBER; +} +static inline uint32_t sd_id_get() { + if (read_mem_u8(MBR_SIZE + SOFTDEVICE_INFO_STRUCT_OFFSET) > (SD_ID_OFFSET - SOFTDEVICE_INFO_STRUCT_OFFSET)) { + return read_mem_u32(MBR_SIZE + SD_ID_OFFSET); + } + return 0; +} +static inline uint32_t sd_version_get() { + if (read_mem_u8(MBR_SIZE + SOFTDEVICE_INFO_STRUCT_OFFSET) > (SD_VERSION_OFFSET - SOFTDEVICE_INFO_STRUCT_OFFSET)) { + return read_mem_u32(MBR_SIZE + SD_VERSION_OFFSET); + } + return 0; +} + +std::string DebugComponent::get_reset_reason_() { + uint32_t cause; + auto ret = hwinfo_get_reset_cause(&cause); + if (ret) { + ESP_LOGE(TAG, "Unable to get reset cause: %d", ret); + return ""; + } + std::string reset_reason; + + show_reset_reason(reset_reason, cause & RESET_PIN, "External pin"); + show_reset_reason(reset_reason, cause & RESET_SOFTWARE, "Software reset"); + show_reset_reason(reset_reason, cause & RESET_BROWNOUT, "Brownout (drop in voltage)"); + show_reset_reason(reset_reason, cause & RESET_POR, "Power-on reset (POR)"); + show_reset_reason(reset_reason, cause & RESET_WATCHDOG, "Watchdog timer expiration"); + show_reset_reason(reset_reason, cause & RESET_DEBUG, "Debug event"); + show_reset_reason(reset_reason, cause & RESET_SECURITY, "Security violation"); + show_reset_reason(reset_reason, cause & RESET_LOW_POWER_WAKE, "Waking up from low power mode"); + show_reset_reason(reset_reason, cause & RESET_CPU_LOCKUP, "CPU lock-up detected"); + show_reset_reason(reset_reason, cause & RESET_PARITY, "Parity error"); + show_reset_reason(reset_reason, cause & RESET_PLL, "PLL error"); + show_reset_reason(reset_reason, cause & RESET_CLOCK, "Clock error"); + show_reset_reason(reset_reason, cause & RESET_HARDWARE, "Hardware reset"); + show_reset_reason(reset_reason, cause & RESET_USER, "User reset"); + show_reset_reason(reset_reason, cause & RESET_TEMPERATURE, "Temperature reset"); + + ESP_LOGD(TAG, "Reset Reason: %s", reset_reason.c_str()); + return reset_reason; +} + +uint32_t DebugComponent::get_free_heap_() { return INT_MAX; } + +void DebugComponent::get_device_info_(std::string &device_info) { + std::string supply = "Main supply status: "; + if (nrf_power_mainregstatus_get(NRF_POWER) == NRF_POWER_MAINREGSTATUS_NORMAL) { + supply += "Normal voltage."; + } else { + supply += "High voltage."; + } + ESP_LOGD(TAG, "%s", supply.c_str()); + device_info += "|" + supply; + + std::string reg0 = "Regulator stage 0: "; + if (nrf_power_mainregstatus_get(NRF_POWER) == NRF_POWER_MAINREGSTATUS_HIGH) { + reg0 += nrf_power_dcdcen_vddh_get(NRF_POWER) ? "DC/DC" : "LDO"; + reg0 += ", "; + switch (NRF_UICR->REGOUT0 & UICR_REGOUT0_VOUT_Msk) { + case (UICR_REGOUT0_VOUT_DEFAULT << UICR_REGOUT0_VOUT_Pos): + reg0 += "1.8V (default)"; + break; + case (UICR_REGOUT0_VOUT_1V8 << UICR_REGOUT0_VOUT_Pos): + reg0 += "1.8V"; + break; + case (UICR_REGOUT0_VOUT_2V1 << UICR_REGOUT0_VOUT_Pos): + reg0 += "2.1V"; + break; + case (UICR_REGOUT0_VOUT_2V4 << UICR_REGOUT0_VOUT_Pos): + reg0 += "2.4V"; + break; + case (UICR_REGOUT0_VOUT_2V7 << UICR_REGOUT0_VOUT_Pos): + reg0 += "2.7V"; + break; + case (UICR_REGOUT0_VOUT_3V0 << UICR_REGOUT0_VOUT_Pos): + reg0 += "3.0V"; + break; + case (UICR_REGOUT0_VOUT_3V3 << UICR_REGOUT0_VOUT_Pos): + reg0 += "3.3V"; + break; + default: + reg0 += "???V"; + } + } else { + reg0 += "disabled"; + } + ESP_LOGD(TAG, "%s", reg0.c_str()); + device_info += "|" + reg0; + + std::string reg1 = "Regulator stage 1: "; + reg1 += nrf_power_dcdcen_get(NRF_POWER) ? "DC/DC" : "LDO"; + ESP_LOGD(TAG, "%s", reg1.c_str()); + device_info += "|" + reg1; + + std::string usb_power = "USB power state: "; + if (nrf_power_usbregstatus_vbusdet_get(NRF_POWER)) { + if (nrf_power_usbregstatus_outrdy_get(NRF_POWER)) { + /**< From the power viewpoint, USB is ready for working. */ + usb_power += "ready"; + } else { + /**< The USB power is detected, but USB power regulator is not ready. */ + usb_power += "connected (regulator is not ready)"; + } + } else { + /**< No power on USB lines detected. */ + usb_power += "disconected"; + } + ESP_LOGD(TAG, "%s", usb_power.c_str()); + device_info += "|" + usb_power; + + bool enabled; + nrf_power_pof_thr_t pof_thr; + + pof_thr = nrf_power_pofcon_get(NRF_POWER, &enabled); + std::string pof = "Power-fail comparator: "; + if (enabled) { + switch (pof_thr) { + case POWER_POFCON_THRESHOLD_V17: + pof += "1.7V"; + break; + case POWER_POFCON_THRESHOLD_V18: + pof += "1.8V"; + break; + case POWER_POFCON_THRESHOLD_V19: + pof += "1.9V"; + break; + case POWER_POFCON_THRESHOLD_V20: + pof += "2.0V"; + break; + case POWER_POFCON_THRESHOLD_V21: + pof += "2.1V"; + break; + case POWER_POFCON_THRESHOLD_V22: + pof += "2.2V"; + break; + case POWER_POFCON_THRESHOLD_V23: + pof += "2.3V"; + break; + case POWER_POFCON_THRESHOLD_V24: + pof += "2.4V"; + break; + case POWER_POFCON_THRESHOLD_V25: + pof += "2.5V"; + break; + case POWER_POFCON_THRESHOLD_V26: + pof += "2.6V"; + break; + case POWER_POFCON_THRESHOLD_V27: + pof += "2.7V"; + break; + case POWER_POFCON_THRESHOLD_V28: + pof += "2.8V"; + break; + } + + if (nrf_power_mainregstatus_get(NRF_POWER) == NRF_POWER_MAINREGSTATUS_HIGH) { + pof += ", VDDH: "; + switch (nrf_power_pofcon_vddh_get(NRF_POWER)) { + case NRF_POWER_POFTHRVDDH_V27: + pof += "2.7V"; + break; + case NRF_POWER_POFTHRVDDH_V28: + pof += "2.8V"; + break; + case NRF_POWER_POFTHRVDDH_V29: + pof += "2.9V"; + break; + case NRF_POWER_POFTHRVDDH_V30: + pof += "3.0V"; + break; + case NRF_POWER_POFTHRVDDH_V31: + pof += "3.1V"; + break; + case NRF_POWER_POFTHRVDDH_V32: + pof += "3.2V"; + break; + case NRF_POWER_POFTHRVDDH_V33: + pof += "3.3V"; + break; + case NRF_POWER_POFTHRVDDH_V34: + pof += "3.4V"; + break; + case NRF_POWER_POFTHRVDDH_V35: + pof += "3.5V"; + break; + case NRF_POWER_POFTHRVDDH_V36: + pof += "3.6V"; + break; + case NRF_POWER_POFTHRVDDH_V37: + pof += "3.7V"; + break; + case NRF_POWER_POFTHRVDDH_V38: + pof += "3.8V"; + break; + case NRF_POWER_POFTHRVDDH_V39: + pof += "3.9V"; + break; + case NRF_POWER_POFTHRVDDH_V40: + pof += "4.0V"; + break; + case NRF_POWER_POFTHRVDDH_V41: + pof += "4.1V"; + break; + case NRF_POWER_POFTHRVDDH_V42: + pof += "4.2V"; + break; + } + } + } else { + pof += "disabled"; + } + ESP_LOGD(TAG, "%s", pof.c_str()); + device_info += "|" + pof; + + auto package = [](uint32_t value) { + switch (value) { + case 0x2004: + return "QIxx - 7x7 73-pin aQFN"; + case 0x2000: + return "QFxx - 6x6 48-pin QFN"; + case 0x2005: + return "CKxx - 3.544 x 3.607 WLCSP"; + } + return "Unspecified"; + }; + + ESP_LOGD(TAG, "Code page size: %u, code size: %u, device id: 0x%08x%08x", NRF_FICR->CODEPAGESIZE, NRF_FICR->CODESIZE, + NRF_FICR->DEVICEID[1], NRF_FICR->DEVICEID[0]); + ESP_LOGD(TAG, "Encryption root: 0x%08x%08x%08x%08x, Identity Root: 0x%08x%08x%08x%08x", NRF_FICR->ER[0], + NRF_FICR->ER[1], NRF_FICR->ER[2], NRF_FICR->ER[3], NRF_FICR->IR[0], NRF_FICR->IR[1], NRF_FICR->IR[2], + NRF_FICR->IR[3]); + ESP_LOGD(TAG, "Device address type: %s, address: %s", (NRF_FICR->DEVICEADDRTYPE & 0x1 ? "Random" : "Public"), + get_mac_address_pretty().c_str()); + ESP_LOGD(TAG, "Part code: nRF%x, version: %c%c%c%c, package: %s", NRF_FICR->INFO.PART, + NRF_FICR->INFO.VARIANT >> 24 & 0xFF, NRF_FICR->INFO.VARIANT >> 16 & 0xFF, NRF_FICR->INFO.VARIANT >> 8 & 0xFF, + NRF_FICR->INFO.VARIANT & 0xFF, package(NRF_FICR->INFO.PACKAGE)); + ESP_LOGD(TAG, "RAM: %ukB, Flash: %ukB, production test: %sdone", NRF_FICR->INFO.RAM, NRF_FICR->INFO.FLASH, + (NRF_FICR->PRODTEST[0] == 0xBB42319F ? "" : "not ")); + ESP_LOGD( + TAG, "GPIO as NFC pins: %s, GPIO as nRESET pin: %s", + YESNO((NRF_UICR->NFCPINS & UICR_NFCPINS_PROTECT_Msk) == (UICR_NFCPINS_PROTECT_NFC << UICR_NFCPINS_PROTECT_Pos)), + YESNO(((NRF_UICR->PSELRESET[0] & UICR_PSELRESET_CONNECT_Msk) != + (UICR_PSELRESET_CONNECT_Connected << UICR_PSELRESET_CONNECT_Pos)) || + ((NRF_UICR->PSELRESET[1] & UICR_PSELRESET_CONNECT_Msk) != + (UICR_PSELRESET_CONNECT_Connected << UICR_PSELRESET_CONNECT_Pos)))); + +#ifdef USE_BOOTLOADER_MCUBOOT + ESP_LOGD(TAG, "bootloader: mcuboot"); +#else + ESP_LOGD(TAG, "bootloader: Adafruit, version %u.%u.%u", (BOOTLOADER_VERSION_REGISTER >> 16) & 0xFF, + (BOOTLOADER_VERSION_REGISTER >> 8) & 0xFF, BOOTLOADER_VERSION_REGISTER & 0xFF); + ESP_LOGD(TAG, "MBR bootloader addr 0x%08x, UICR bootloader addr 0x%08x", read_mem_u32(MBR_BOOTLOADER_ADDR), + NRF_UICR->NRFFW[0]); + ESP_LOGD(TAG, "MBR param page addr 0x%08x, UICR param page addr 0x%08x", read_mem_u32(MBR_PARAM_PAGE_ADDR), + NRF_UICR->NRFFW[1]); + if (is_sd_present()) { + uint32_t const sd_id = sd_id_get(); + uint32_t const sd_version = sd_version_get(); + + uint32_t ver[3]; + ver[0] = sd_version / 1000000; + ver[1] = (sd_version - ver[0] * 1000000) / 1000; + ver[2] = (sd_version - ver[0] * 1000000 - ver[1] * 1000); + + ESP_LOGD(TAG, "SoftDevice: S%u %u.%u.%u", sd_id, ver[0], ver[1], ver[2]); +#ifdef USE_SOFTDEVICE_ID +#ifdef USE_SOFTDEVICE_VERSION + if (USE_SOFTDEVICE_ID != sd_id || USE_SOFTDEVICE_VERSION != ver[0]) { + ESP_LOGE(TAG, "Built for SoftDevice S%u %u.x.y. It may crash due to mismatch of bootloader version.", + USE_SOFTDEVICE_ID, USE_SOFTDEVICE_VERSION); + } +#else + if (USE_SOFTDEVICE_ID != sd_id) { + ESP_LOGE(TAG, "Built for SoftDevice S%u. It may crash due to mismatch of bootloader version.", USE_SOFTDEVICE_ID); + } +#endif +#endif + } +#endif +} + +void DebugComponent::update_platform_() {} + +} // namespace debug +} // namespace esphome +#endif diff --git a/esphome/components/debug/sensor.py b/esphome/components/debug/sensor.py index 4669095d5d..4484f15935 100644 --- a/esphome/components/debug/sensor.py +++ b/esphome/components/debug/sensor.py @@ -1,6 +1,7 @@ import esphome.codegen as cg from esphome.components import sensor from esphome.components.esp32 import CONF_CPU_FREQUENCY +from esphome.components.psram import DOMAIN as PSRAM_DOMAIN import esphome.config_validation as cv from esphome.const import ( CONF_BLOCK, @@ -54,7 +55,7 @@ CONFIG_SCHEMA = { ), cv.Optional(CONF_PSRAM): cv.All( cv.only_on_esp32, - cv.requires_component("psram"), + cv.requires_component(PSRAM_DOMAIN), sensor.sensor_schema( unit_of_measurement=UNIT_BYTES, icon=ICON_COUNTER, diff --git a/esphome/components/deep_sleep/__init__.py b/esphome/components/deep_sleep/__init__.py index 05ae60239d..19fb726016 100644 --- a/esphome/components/deep_sleep/__init__.py +++ b/esphome/components/deep_sleep/__init__.py @@ -197,7 +197,8 @@ CONFIG_SCHEMA = cv.All( cv.Optional(CONF_ESP32_EXT1_WAKEUP): cv.All( cv.only_on_esp32, esp32.only_on_variant( - unsupported=[VARIANT_ESP32C3], msg_prefix="Wakeup from ext1" + unsupported=[VARIANT_ESP32C2, VARIANT_ESP32C3], + msg_prefix="Wakeup from ext1", ), cv.Schema( { @@ -214,7 +215,13 @@ CONFIG_SCHEMA = cv.All( cv.Optional(CONF_TOUCH_WAKEUP): cv.All( cv.only_on_esp32, esp32.only_on_variant( - unsupported=[VARIANT_ESP32C3], msg_prefix="Wakeup from touch" + unsupported=[ + VARIANT_ESP32C2, + VARIANT_ESP32C3, + VARIANT_ESP32C6, + VARIANT_ESP32H2, + ], + msg_prefix="Wakeup from touch", ), cv.boolean, ), diff --git a/esphome/components/deep_sleep/deep_sleep_component.cpp b/esphome/components/deep_sleep/deep_sleep_component.cpp index 84fc102b66..8066b411ff 100644 --- a/esphome/components/deep_sleep/deep_sleep_component.cpp +++ b/esphome/components/deep_sleep/deep_sleep_component.cpp @@ -12,7 +12,6 @@ static const uint32_t TEARDOWN_TIMEOUT_DEEP_SLEEP_MS = 5000; bool global_has_deep_sleep = false; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) void DeepSleepComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); global_has_deep_sleep = true; const optional run_duration = get_run_duration_(); diff --git a/esphome/components/deep_sleep/deep_sleep_component.h b/esphome/components/deep_sleep/deep_sleep_component.h index 7a640b9ea5..38744163c7 100644 --- a/esphome/components/deep_sleep/deep_sleep_component.h +++ b/esphome/components/deep_sleep/deep_sleep_component.h @@ -34,7 +34,7 @@ enum WakeupPinMode { WAKEUP_PIN_MODE_INVERT_WAKEUP, }; -#if defined(USE_ESP32) && !defined(USE_ESP32_VARIANT_ESP32C3) +#if defined(USE_ESP32) && !defined(USE_ESP32_VARIANT_ESP32C2) && !defined(USE_ESP32_VARIANT_ESP32C3) struct Ext1Wakeup { uint64_t mask; esp_sleep_ext1_wakeup_mode_t wakeup_mode; @@ -50,7 +50,7 @@ struct WakeupCauseToRunDuration { uint32_t gpio_cause; }; -#endif +#endif // USE_ESP32 template class EnterDeepSleepAction; @@ -73,20 +73,22 @@ class DeepSleepComponent : public Component { void set_wakeup_pin(InternalGPIOPin *pin) { this->wakeup_pin_ = pin; } void set_wakeup_pin_mode(WakeupPinMode wakeup_pin_mode); -#endif +#endif // USE_ESP32 #if defined(USE_ESP32) -#if !defined(USE_ESP32_VARIANT_ESP32C3) - +#if !defined(USE_ESP32_VARIANT_ESP32C2) && !defined(USE_ESP32_VARIANT_ESP32C3) void set_ext1_wakeup(Ext1Wakeup ext1_wakeup); - - void set_touch_wakeup(bool touch_wakeup); - #endif + +#if !defined(USE_ESP32_VARIANT_ESP32C2) && !defined(USE_ESP32_VARIANT_ESP32C3) && \ + !defined(USE_ESP32_VARIANT_ESP32C6) && !defined(USE_ESP32_VARIANT_ESP32H2) + void set_touch_wakeup(bool touch_wakeup); +#endif + // Set the duration in ms for how long the code should run before entering // deep sleep mode, according to the cause the ESP32 has woken. void set_run_duration(WakeupCauseToRunDuration wakeup_cause_to_run_duration); -#endif +#endif // USE_ESP32 /// Set a duration in ms for how long the code should run before entering deep sleep mode. void set_run_duration(uint32_t time_ms); @@ -117,13 +119,13 @@ class DeepSleepComponent : public Component { InternalGPIOPin *wakeup_pin_; WakeupPinMode wakeup_pin_mode_{WAKEUP_PIN_MODE_IGNORE}; -#if !defined(USE_ESP32_VARIANT_ESP32C3) +#if !defined(USE_ESP32_VARIANT_ESP32C2) && !defined(USE_ESP32_VARIANT_ESP32C3) optional ext1_wakeup_; #endif optional touch_wakeup_; optional wakeup_cause_to_run_duration_; -#endif +#endif // USE_ESP32 optional run_duration_; bool next_enter_deep_sleep_{false}; bool prevent_{false}; diff --git a/esphome/components/deep_sleep/deep_sleep_esp32.cpp b/esphome/components/deep_sleep/deep_sleep_esp32.cpp index 7965ab738a..b93d9ce601 100644 --- a/esphome/components/deep_sleep/deep_sleep_esp32.cpp +++ b/esphome/components/deep_sleep/deep_sleep_esp32.cpp @@ -1,10 +1,32 @@ #ifdef USE_ESP32 +#include "soc/soc_caps.h" +#include "driver/gpio.h" #include "deep_sleep_component.h" #include "esphome/core/log.h" namespace esphome { namespace deep_sleep { +// Deep Sleep feature support matrix for ESP32 variants: +// +// | Variant | ext0 | ext1 | Touch | GPIO wakeup | +// |-----------|------|------|-------|-------------| +// | ESP32 | ✓ | ✓ | ✓ | | +// | ESP32-S2 | ✓ | ✓ | ✓ | | +// | ESP32-S3 | ✓ | ✓ | ✓ | | +// | ESP32-C2 | | | | ✓ | +// | ESP32-C3 | | | | ✓ | +// | ESP32-C5 | | (✓) | | (✓) | +// | ESP32-C6 | | ✓ | | ✓ | +// | ESP32-H2 | | ✓ | | | +// +// Notes: +// - (✓) = Supported by hardware but not yet implemented in ESPHome +// - ext0: Single pin wakeup using RTC GPIO (esp_sleep_enable_ext0_wakeup) +// - ext1: Multiple pin wakeup (esp_sleep_enable_ext1_wakeup) +// - Touch: Touch pad wakeup (esp_sleep_enable_touchpad_wakeup) +// - GPIO wakeup: GPIO wakeup for non-RTC pins (esp_deep_sleep_enable_gpio_wakeup) + static const char *const TAG = "deep_sleep"; optional DeepSleepComponent::get_run_duration_() const { @@ -28,13 +50,13 @@ void DeepSleepComponent::set_wakeup_pin_mode(WakeupPinMode wakeup_pin_mode) { this->wakeup_pin_mode_ = wakeup_pin_mode; } -#if !defined(USE_ESP32_VARIANT_ESP32C3) && !defined(USE_ESP32_VARIANT_ESP32C6) +#if !defined(USE_ESP32_VARIANT_ESP32C2) && !defined(USE_ESP32_VARIANT_ESP32C3) void DeepSleepComponent::set_ext1_wakeup(Ext1Wakeup ext1_wakeup) { this->ext1_wakeup_ = ext1_wakeup; } - -#if !defined(USE_ESP32_VARIANT_ESP32H2) -void DeepSleepComponent::set_touch_wakeup(bool touch_wakeup) { this->touch_wakeup_ = touch_wakeup; } #endif +#if !defined(USE_ESP32_VARIANT_ESP32C2) && !defined(USE_ESP32_VARIANT_ESP32C3) && \ + !defined(USE_ESP32_VARIANT_ESP32C6) && !defined(USE_ESP32_VARIANT_ESP32H2) +void DeepSleepComponent::set_touch_wakeup(bool touch_wakeup) { this->touch_wakeup_ = touch_wakeup; } #endif void DeepSleepComponent::set_run_duration(WakeupCauseToRunDuration wakeup_cause_to_run_duration) { @@ -70,38 +92,51 @@ bool DeepSleepComponent::prepare_to_sleep_() { } void DeepSleepComponent::deep_sleep_() { -#if !defined(USE_ESP32_VARIANT_ESP32C3) && !defined(USE_ESP32_VARIANT_ESP32C6) && !defined(USE_ESP32_VARIANT_ESP32H2) + // Timer wakeup - all variants support this if (this->sleep_duration_.has_value()) esp_sleep_enable_timer_wakeup(*this->sleep_duration_); + + // Single pin wakeup (ext0) - ESP32, S2, S3 only +#if !defined(USE_ESP32_VARIANT_ESP32C2) && !defined(USE_ESP32_VARIANT_ESP32C3) && \ + !defined(USE_ESP32_VARIANT_ESP32C6) && !defined(USE_ESP32_VARIANT_ESP32H2) if (this->wakeup_pin_ != nullptr) { + const auto gpio_pin = gpio_num_t(this->wakeup_pin_->get_pin()); + if (this->wakeup_pin_->get_flags() & gpio::FLAG_PULLUP) { + gpio_sleep_set_pull_mode(gpio_pin, GPIO_PULLUP_ONLY); + } else if (this->wakeup_pin_->get_flags() & gpio::FLAG_PULLDOWN) { + gpio_sleep_set_pull_mode(gpio_pin, GPIO_PULLDOWN_ONLY); + } + gpio_sleep_set_direction(gpio_pin, GPIO_MODE_INPUT); + gpio_hold_en(gpio_pin); +#if !SOC_GPIO_SUPPORT_HOLD_SINGLE_IO_IN_DSLP + // Some ESP32 variants support holding a single GPIO during deep sleep without this function + // For those variants, gpio_hold_en() is sufficient to hold the pin state during deep sleep + gpio_deep_sleep_hold_en(); +#endif bool level = !this->wakeup_pin_->is_inverted(); if (this->wakeup_pin_mode_ == WAKEUP_PIN_MODE_INVERT_WAKEUP && this->wakeup_pin_->digital_read()) { level = !level; } - esp_sleep_enable_ext0_wakeup(gpio_num_t(this->wakeup_pin_->get_pin()), level); - } - if (this->ext1_wakeup_.has_value()) { - esp_sleep_enable_ext1_wakeup(this->ext1_wakeup_->mask, this->ext1_wakeup_->wakeup_mode); - } - - if (this->touch_wakeup_.has_value() && *(this->touch_wakeup_)) { - esp_sleep_enable_touchpad_wakeup(); - esp_sleep_pd_config(ESP_PD_DOMAIN_RTC_PERIPH, ESP_PD_OPTION_ON); + esp_sleep_enable_ext0_wakeup(gpio_pin, level); } #endif -#if defined(USE_ESP32_VARIANT_ESP32H2) - if (this->sleep_duration_.has_value()) - esp_sleep_enable_timer_wakeup(*this->sleep_duration_); - if (this->ext1_wakeup_.has_value()) { - esp_sleep_enable_ext1_wakeup(this->ext1_wakeup_->mask, this->ext1_wakeup_->wakeup_mode); - } -#endif - -#if defined(USE_ESP32_VARIANT_ESP32C3) || defined(USE_ESP32_VARIANT_ESP32C6) - if (this->sleep_duration_.has_value()) - esp_sleep_enable_timer_wakeup(*this->sleep_duration_); + // GPIO wakeup - C2, C3, C6 only +#if defined(USE_ESP32_VARIANT_ESP32C2) || defined(USE_ESP32_VARIANT_ESP32C3) || defined(USE_ESP32_VARIANT_ESP32C6) if (this->wakeup_pin_ != nullptr) { + const auto gpio_pin = gpio_num_t(this->wakeup_pin_->get_pin()); + if (this->wakeup_pin_->get_flags() & gpio::FLAG_PULLUP) { + gpio_sleep_set_pull_mode(gpio_pin, GPIO_PULLUP_ONLY); + } else if (this->wakeup_pin_->get_flags() & gpio::FLAG_PULLDOWN) { + gpio_sleep_set_pull_mode(gpio_pin, GPIO_PULLDOWN_ONLY); + } + gpio_sleep_set_direction(gpio_pin, GPIO_MODE_INPUT); + gpio_hold_en(gpio_pin); +#if !SOC_GPIO_SUPPORT_HOLD_SINGLE_IO_IN_DSLP + // Some ESP32 variants support holding a single GPIO during deep sleep without this function + // For those variants, gpio_hold_en() is sufficient to hold the pin state during deep sleep + gpio_deep_sleep_hold_en(); +#endif bool level = !this->wakeup_pin_->is_inverted(); if (this->wakeup_pin_mode_ == WAKEUP_PIN_MODE_INVERT_WAKEUP && this->wakeup_pin_->digital_read()) { level = !level; @@ -110,9 +145,26 @@ void DeepSleepComponent::deep_sleep_() { static_cast(level)); } #endif + + // Multiple pin wakeup (ext1) - All except C2, C3 +#if !defined(USE_ESP32_VARIANT_ESP32C2) && !defined(USE_ESP32_VARIANT_ESP32C3) + if (this->ext1_wakeup_.has_value()) { + esp_sleep_enable_ext1_wakeup(this->ext1_wakeup_->mask, this->ext1_wakeup_->wakeup_mode); + } +#endif + + // Touch wakeup - ESP32, S2, S3 only +#if !defined(USE_ESP32_VARIANT_ESP32C2) && !defined(USE_ESP32_VARIANT_ESP32C3) && \ + !defined(USE_ESP32_VARIANT_ESP32C6) && !defined(USE_ESP32_VARIANT_ESP32H2) + if (this->touch_wakeup_.has_value() && *(this->touch_wakeup_)) { + esp_sleep_enable_touchpad_wakeup(); + esp_sleep_pd_config(ESP_PD_DOMAIN_RTC_PERIPH, ESP_PD_OPTION_ON); + } +#endif + esp_deep_sleep_start(); } } // namespace deep_sleep } // namespace esphome -#endif +#endif // USE_ESP32 diff --git a/esphome/components/dfrobot_sen0395/__init__.py b/esphome/components/dfrobot_sen0395/__init__.py index 8c9438dccb..d54b147036 100644 --- a/esphome/components/dfrobot_sen0395/__init__.py +++ b/esphome/components/dfrobot_sen0395/__init__.py @@ -74,8 +74,7 @@ def range_segment_list(input): if isinstance(input, list): for list_item in input: if isinstance(list_item, list): - for item in list_item: - flat_list.append(item) + flat_list.extend(list_item) else: flat_list.append(list_item) else: diff --git a/esphome/components/dht/dht.cpp b/esphome/components/dht/dht.cpp index 7248ef624e..cc0bf55a80 100644 --- a/esphome/components/dht/dht.cpp +++ b/esphome/components/dht/dht.cpp @@ -8,7 +8,6 @@ namespace dht { static const char *const TAG = "dht"; void DHT::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->pin_->digital_write(true); this->pin_->setup(); this->pin_->digital_write(true); diff --git a/esphome/components/dht12/dht12.cpp b/esphome/components/dht12/dht12.cpp index 54a6688b0b..445d150be0 100644 --- a/esphome/components/dht12/dht12.cpp +++ b/esphome/components/dht12/dht12.cpp @@ -34,7 +34,6 @@ void DHT12Component::update() { this->status_clear_warning(); } void DHT12Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t data[5]; if (!this->read_data_(data)) { this->mark_failed(); diff --git a/esphome/components/display/__init__.py b/esphome/components/display/__init__.py index 12c63231e7..ccbeedcd2f 100644 --- a/esphome/components/display/__init__.py +++ b/esphome/components/display/__init__.py @@ -12,8 +12,10 @@ from esphome.const import ( CONF_ROTATION, CONF_TO, CONF_TRIGGER_ID, + CONF_UPDATE_INTERVAL, + SCHEDULER_DONT_RUN, ) -from esphome.core import coroutine_with_priority +from esphome.core import CoroPriority, coroutine_with_priority IS_PLATFORM_COMPONENT = True @@ -51,8 +53,7 @@ DISPLAY_ROTATIONS = { def validate_rotation(value): value = cv.string(value) - if value.endswith("°"): - value = value[:-1] + value = value.removesuffix("°") return cv.enum(DISPLAY_ROTATIONS, int=True)(value) @@ -68,6 +69,18 @@ BASIC_DISPLAY_SCHEMA = cv.Schema( } ).extend(cv.polling_component_schema("1s")) + +def _validate_test_card(config): + if ( + config.get(CONF_SHOW_TEST_CARD, False) + and config.get(CONF_UPDATE_INTERVAL, False) == SCHEDULER_DONT_RUN + ): + raise cv.Invalid( + f"`{CONF_SHOW_TEST_CARD}: True` cannot be used with `{CONF_UPDATE_INTERVAL}: never` because this combination will not show a test_card." + ) + return config + + FULL_DISPLAY_SCHEMA = BASIC_DISPLAY_SCHEMA.extend( { cv.Optional(CONF_ROTATION): validate_rotation, @@ -95,6 +108,7 @@ FULL_DISPLAY_SCHEMA = BASIC_DISPLAY_SCHEMA.extend( cv.Optional(CONF_SHOW_TEST_CARD): cv.boolean, } ) +FULL_DISPLAY_SCHEMA.add_extra(_validate_test_card) async def setup_display_core_(var, config): @@ -162,7 +176,7 @@ async def display_page_show_to_code(config, action_id, template_arg, args): DisplayPageShowNextAction, maybe_simple_id( { - cv.Required(CONF_ID): cv.templatable(cv.use_id(Display)), + cv.GenerateID(CONF_ID): cv.templatable(cv.use_id(Display)), } ), ) @@ -176,7 +190,7 @@ async def display_page_show_next_to_code(config, action_id, template_arg, args): DisplayPageShowPrevAction, maybe_simple_id( { - cv.Required(CONF_ID): cv.templatable(cv.use_id(Display)), + cv.GenerateID(CONF_ID): cv.templatable(cv.use_id(Display)), } ), ) @@ -201,11 +215,10 @@ async def display_is_displaying_page_to_code(config, condition_id, template_arg, page = await cg.get_variable(config[CONF_PAGE_ID]) var = cg.new_Pvariable(condition_id, template_arg, paren) cg.add(var.set_page(page)) - return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): cg.add_global(display_ns.using) cg.add_define("USE_DISPLAY") diff --git a/esphome/components/display/display.cpp b/esphome/components/display/display.cpp index c666eee298..1451d14e2e 100644 --- a/esphome/components/display/display.cpp +++ b/esphome/components/display/display.cpp @@ -775,7 +775,7 @@ void Display::test_card() { int shift_y = (h - image_h) / 2; int line_w = (image_w - 6) / 6; int image_c = image_w / 2; - for (auto i = 0; i <= image_h; i++) { + for (auto i = 0; i != image_h; i++) { int c = esp_scale(i, image_h); this->horizontal_line(shift_x + 0, shift_y + i, line_w, r.fade_to_white(c)); this->horizontal_line(shift_x + line_w, shift_y + i, line_w, r.fade_to_black(c)); // @@ -809,8 +809,11 @@ void Display::test_card() { } } } - this->rectangle(0, 0, w, h, Color(127, 0, 127)); this->filled_rectangle(0, 0, 10, 10, Color(255, 0, 255)); + this->filled_rectangle(w - 10, 0, 10, 10, Color(255, 0, 255)); + this->filled_rectangle(0, h - 10, 10, 10, Color(255, 0, 255)); + this->filled_rectangle(w - 10, h - 10, 10, 10, Color(255, 0, 255)); + this->rectangle(0, 0, w, h, Color(255, 255, 255)); this->stop_poller(); } diff --git a/esphome/components/display_menu_base/__init__.py b/esphome/components/display_menu_base/__init__.py index f9c0424104..658292ec7a 100644 --- a/esphome/components/display_menu_base/__init__.py +++ b/esphome/components/display_menu_base/__init__.py @@ -17,6 +17,7 @@ from esphome.const import ( CONF_MODE, CONF_NUMBER, CONF_ON_VALUE, + CONF_SWITCH, CONF_TEXT, CONF_TRIGGER_ID, CONF_TYPE, @@ -33,7 +34,6 @@ CONF_LABEL = "label" CONF_MENU = "menu" CONF_BACK = "back" CONF_SELECT = "select" -CONF_SWITCH = "switch" CONF_ON_TEXT = "on_text" CONF_OFF_TEXT = "off_text" CONF_VALUE_LAMBDA = "value_lambda" diff --git a/esphome/components/dps310/dps310.cpp b/esphome/components/dps310/dps310.cpp index a7fb7ecd5e..6b6f9622fa 100644 --- a/esphome/components/dps310/dps310.cpp +++ b/esphome/components/dps310/dps310.cpp @@ -11,8 +11,6 @@ void DPS310Component::setup() { uint8_t coef_data_raw[DPS310_NUM_COEF_REGS]; auto timer = DPS310_INIT_TIMEOUT; uint8_t reg = 0; - - ESP_LOGCONFIG(TAG, "Running setup"); // first, reset the sensor if (!this->write_byte(DPS310_REG_RESET, DPS310_CMD_RESET)) { this->mark_failed(); diff --git a/esphome/components/ds1307/ds1307.cpp b/esphome/components/ds1307/ds1307.cpp index db0180e6f1..077db497b1 100644 --- a/esphome/components/ds1307/ds1307.cpp +++ b/esphome/components/ds1307/ds1307.cpp @@ -10,7 +10,6 @@ namespace ds1307 { static const char *const TAG = "ds1307"; void DS1307Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (!this->read_rtc_()) { this->mark_failed(); } diff --git a/esphome/components/ds2484/ds2484.cpp b/esphome/components/ds2484/ds2484.cpp index c3df9786b6..7c890ff433 100644 --- a/esphome/components/ds2484/ds2484.cpp +++ b/esphome/components/ds2484/ds2484.cpp @@ -5,7 +5,6 @@ namespace ds2484 { static const char *const TAG = "ds2484.onewire"; void DS2484OneWireBus::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->reset_device(); this->search(); } diff --git a/esphome/components/duty_cycle/duty_cycle_sensor.cpp b/esphome/components/duty_cycle/duty_cycle_sensor.cpp index 8939de0ee9..40a728d025 100644 --- a/esphome/components/duty_cycle/duty_cycle_sensor.cpp +++ b/esphome/components/duty_cycle/duty_cycle_sensor.cpp @@ -8,7 +8,6 @@ namespace duty_cycle { static const char *const TAG = "duty_cycle"; void DutyCycleSensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->get_name().c_str()); this->pin_->setup(); this->store_.pin = this->pin_->to_isr(); this->store_.last_level = this->pin_->digital_read(); diff --git a/esphome/components/duty_time/duty_time_sensor.cpp b/esphome/components/duty_time/duty_time_sensor.cpp index c7319f7c33..f77f1fcf53 100644 --- a/esphome/components/duty_time/duty_time_sensor.cpp +++ b/esphome/components/duty_time/duty_time_sensor.cpp @@ -41,7 +41,7 @@ void DutyTimeSensor::setup() { uint32_t seconds = 0; if (this->restore_) { - this->pref_ = global_preferences->make_preference(this->get_object_id_hash()); + this->pref_ = global_preferences->make_preference(this->get_preference_hash()); this->pref_.load(&seconds); } diff --git a/esphome/components/e131/e131_packet.cpp b/esphome/components/e131/e131_packet.cpp index b8fa73b707..e663a3d0fc 100644 --- a/esphome/components/e131/e131_packet.cpp +++ b/esphome/components/e131/e131_packet.cpp @@ -4,6 +4,7 @@ #include "esphome/components/network/ip_address.h" #include "esphome/core/log.h" #include "esphome/core/util.h" +#include "esphome/core/helpers.h" #include #include @@ -71,7 +72,11 @@ bool E131Component::join_igmp_groups_() { ip4_addr_t multicast_addr = network::IPAddress(239, 255, ((universe.first >> 8) & 0xff), ((universe.first >> 0) & 0xff)); - auto err = igmp_joingroup(IP4_ADDR_ANY4, &multicast_addr); + err_t err; + { + LwIPLock lock; + err = igmp_joingroup(IP4_ADDR_ANY4, &multicast_addr); + } if (err) { ESP_LOGW(TAG, "IGMP join for %d universe of E1.31 failed. Multicast might not work.", universe.first); @@ -104,6 +109,7 @@ void E131Component::leave_(int universe) { if (listen_method_ == E131_MULTICAST) { ip4_addr_t multicast_addr = network::IPAddress(239, 255, ((universe >> 8) & 0xff), ((universe >> 0) & 0xff)); + LwIPLock lock; igmp_leavegroup(IP4_ADDR_ANY4, &multicast_addr); } diff --git a/esphome/components/ee895/ee895.cpp b/esphome/components/ee895/ee895.cpp index bdaa3f3200..c6eaf4e728 100644 --- a/esphome/components/ee895/ee895.cpp +++ b/esphome/components/ee895/ee895.cpp @@ -16,7 +16,6 @@ static const uint16_t PRESSURE_ADDRESS = 0x04B0; void EE895Component::setup() { uint16_t crc16_check = 0; - ESP_LOGCONFIG(TAG, "Running setup"); write_command_(SERIAL_NUMBER, 8); uint8_t serial_number[20]; this->read(serial_number, 20); @@ -84,7 +83,7 @@ void EE895Component::write_command_(uint16_t addr, uint16_t reg_cnt) { crc16 = calc_crc16_(address, 6); address[5] = crc16 & 0xFF; address[6] = (crc16 >> 8) & 0xFF; - this->write(address, 7, true); + this->write(address, 7); } float EE895Component::read_float_() { diff --git a/esphome/components/ektf2232/touchscreen/__init__.py b/esphome/components/ektf2232/touchscreen/__init__.py index 7d946fdcb9..123f03ca08 100644 --- a/esphome/components/ektf2232/touchscreen/__init__.py +++ b/esphome/components/ektf2232/touchscreen/__init__.py @@ -2,7 +2,7 @@ from esphome import pins import esphome.codegen as cg from esphome.components import i2c, touchscreen import esphome.config_validation as cv -from esphome.const import CONF_ID, CONF_INTERRUPT_PIN +from esphome.const import CONF_ID, CONF_INTERRUPT_PIN, CONF_RESET_PIN CODEOWNERS = ["@jesserockz"] DEPENDENCIES = ["i2c"] @@ -15,7 +15,7 @@ EKTF2232Touchscreen = ektf2232_ns.class_( ) CONF_EKTF2232_ID = "ektf2232_id" -CONF_RTS_PIN = "rts_pin" +CONF_RTS_PIN = "rts_pin" # To be removed before 2026.4.0 CONFIG_SCHEMA = touchscreen.TOUCHSCREEN_SCHEMA.extend( cv.Schema( @@ -24,7 +24,10 @@ CONFIG_SCHEMA = touchscreen.TOUCHSCREEN_SCHEMA.extend( cv.Required(CONF_INTERRUPT_PIN): cv.All( pins.internal_gpio_input_pin_schema ), - cv.Required(CONF_RTS_PIN): pins.gpio_output_pin_schema, + cv.Required(CONF_RESET_PIN): pins.gpio_output_pin_schema, + cv.Optional(CONF_RTS_PIN): cv.invalid( + f"{CONF_RTS_PIN} has been renamed to {CONF_RESET_PIN}" + ), } ).extend(i2c.i2c_device_schema(0x15)) ) @@ -37,5 +40,5 @@ async def to_code(config): interrupt_pin = await cg.gpio_pin_expression(config[CONF_INTERRUPT_PIN]) cg.add(var.set_interrupt_pin(interrupt_pin)) - rts_pin = await cg.gpio_pin_expression(config[CONF_RTS_PIN]) - cg.add(var.set_rts_pin(rts_pin)) + reset_pin = await cg.gpio_pin_expression(config[CONF_RESET_PIN]) + cg.add(var.set_reset_pin(reset_pin)) diff --git a/esphome/components/ektf2232/touchscreen/ektf2232.cpp b/esphome/components/ektf2232/touchscreen/ektf2232.cpp index 666e56e2a7..63ebb2166b 100644 --- a/esphome/components/ektf2232/touchscreen/ektf2232.cpp +++ b/esphome/components/ektf2232/touchscreen/ektf2232.cpp @@ -16,13 +16,12 @@ static const uint8_t GET_Y_RES[4] = {0x53, 0x63, 0x00, 0x00}; static const uint8_t GET_POWER_STATE_CMD[4] = {0x53, 0x50, 0x00, 0x01}; void EKTF2232Touchscreen::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->interrupt_pin_->pin_mode(gpio::FLAG_INPUT | gpio::FLAG_PULLUP); this->interrupt_pin_->setup(); this->attach_interrupt_(this->interrupt_pin_, gpio::INTERRUPT_FALLING_EDGE); - this->rts_pin_->setup(); + this->reset_pin_->setup(); this->hard_reset_(); if (!this->soft_reset_()) { @@ -99,9 +98,9 @@ bool EKTF2232Touchscreen::get_power_state() { } void EKTF2232Touchscreen::hard_reset_() { - this->rts_pin_->digital_write(false); + this->reset_pin_->digital_write(false); delay(15); - this->rts_pin_->digital_write(true); + this->reset_pin_->digital_write(true); delay(15); } @@ -128,7 +127,7 @@ void EKTF2232Touchscreen::dump_config() { ESP_LOGCONFIG(TAG, "EKT2232 Touchscreen:"); LOG_I2C_DEVICE(this); LOG_PIN(" Interrupt Pin: ", this->interrupt_pin_); - LOG_PIN(" RTS Pin: ", this->rts_pin_); + LOG_PIN(" Reset Pin: ", this->reset_pin_); } } // namespace ektf2232 diff --git a/esphome/components/ektf2232/touchscreen/ektf2232.h b/esphome/components/ektf2232/touchscreen/ektf2232.h index e9288d0a27..2ddc60851f 100644 --- a/esphome/components/ektf2232/touchscreen/ektf2232.h +++ b/esphome/components/ektf2232/touchscreen/ektf2232.h @@ -17,7 +17,7 @@ class EKTF2232Touchscreen : public Touchscreen, public i2c::I2CDevice { void dump_config() override; void set_interrupt_pin(InternalGPIOPin *pin) { this->interrupt_pin_ = pin; } - void set_rts_pin(GPIOPin *pin) { this->rts_pin_ = pin; } + void set_reset_pin(GPIOPin *pin) { this->reset_pin_ = pin; } void set_power_state(bool enable); bool get_power_state(); @@ -28,7 +28,7 @@ class EKTF2232Touchscreen : public Touchscreen, public i2c::I2CDevice { void update_touches() override; InternalGPIOPin *interrupt_pin_; - GPIOPin *rts_pin_; + GPIOPin *reset_pin_; }; } // namespace ektf2232 diff --git a/esphome/components/emc2101/emc2101.cpp b/esphome/components/emc2101/emc2101.cpp index 75d324c2bb..7d85cd31cf 100644 --- a/esphome/components/emc2101/emc2101.cpp +++ b/esphome/components/emc2101/emc2101.cpp @@ -57,8 +57,6 @@ static const uint8_t EMC2101_POLARITY_BIT = 1 << 4; float Emc2101Component::get_setup_priority() const { return setup_priority::HARDWARE; } void Emc2101Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // make sure we're talking to the right chip uint8_t chip_id = reg(EMC2101_REGISTER_WHOAMI).get(); if ((chip_id != EMC2101_CHIP_ID) && (chip_id != EMC2101_ALT_CHIP_ID)) { diff --git a/esphome/components/ens160_base/ens160_base.cpp b/esphome/components/ens160_base/ens160_base.cpp index 7e5b8528b7..6ffaac9588 100644 --- a/esphome/components/ens160_base/ens160_base.cpp +++ b/esphome/components/ens160_base/ens160_base.cpp @@ -49,8 +49,6 @@ static const uint8_t ENS160_DATA_STATUS_NEWGPR = 0x01; static const uint8_t ENS160_DATA_AQI = 0x07; void ENS160Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // check part_id uint16_t part_id; if (!this->read_bytes(ENS160_REG_PART_ID, reinterpret_cast(&part_id), 2)) { diff --git a/esphome/components/ens210/ens210.cpp b/esphome/components/ens210/ens210.cpp index b296e9dd42..98a300f5d7 100644 --- a/esphome/components/ens210/ens210.cpp +++ b/esphome/components/ens210/ens210.cpp @@ -87,7 +87,6 @@ static uint32_t crc7(uint32_t value) { } void ENS210Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t data[2]; uint16_t part_id = 0; // Reset diff --git a/esphome/components/epaper_spi/__init__.py b/esphome/components/epaper_spi/__init__.py new file mode 100644 index 0000000000..f70ffa9520 --- /dev/null +++ b/esphome/components/epaper_spi/__init__.py @@ -0,0 +1 @@ +CODEOWNERS = ["@esphome/core"] diff --git a/esphome/components/epaper_spi/display.py b/esphome/components/epaper_spi/display.py new file mode 100644 index 0000000000..20549f049d --- /dev/null +++ b/esphome/components/epaper_spi/display.py @@ -0,0 +1,80 @@ +from esphome import core, pins +import esphome.codegen as cg +from esphome.components import display, spi +import esphome.config_validation as cv +from esphome.const import ( + CONF_BUSY_PIN, + CONF_DC_PIN, + CONF_ID, + CONF_LAMBDA, + CONF_MODEL, + CONF_PAGES, + CONF_RESET_DURATION, + CONF_RESET_PIN, +) + +AUTO_LOAD = ["split_buffer"] +DEPENDENCIES = ["spi"] + +epaper_spi_ns = cg.esphome_ns.namespace("epaper_spi") +EPaperBase = epaper_spi_ns.class_( + "EPaperBase", cg.PollingComponent, spi.SPIDevice, display.DisplayBuffer +) + +EPaperSpectraE6 = epaper_spi_ns.class_("EPaperSpectraE6", EPaperBase) +EPaper7p3InSpectraE6 = epaper_spi_ns.class_("EPaper7p3InSpectraE6", EPaperSpectraE6) + +MODELS = { + "7.3in-spectra-e6": EPaper7p3InSpectraE6, +} + + +CONFIG_SCHEMA = cv.All( + display.FULL_DISPLAY_SCHEMA.extend( + { + cv.GenerateID(): cv.declare_id(EPaperBase), + cv.Required(CONF_DC_PIN): pins.gpio_output_pin_schema, + cv.Required(CONF_MODEL): cv.one_of(*MODELS, lower=True, space="-"), + cv.Optional(CONF_RESET_PIN): pins.gpio_output_pin_schema, + cv.Optional(CONF_BUSY_PIN): pins.gpio_input_pin_schema, + cv.Optional(CONF_RESET_DURATION): cv.All( + cv.positive_time_period_milliseconds, + cv.Range(max=core.TimePeriod(milliseconds=500)), + ), + } + ) + .extend(cv.polling_component_schema("60s")) + .extend(spi.spi_device_schema()), + cv.has_at_most_one_key(CONF_PAGES, CONF_LAMBDA), +) + +FINAL_VALIDATE_SCHEMA = spi.final_validate_device_schema( + "epaper_spi", require_miso=False, require_mosi=True +) + + +async def to_code(config): + model = MODELS[config[CONF_MODEL]] + + rhs = model.new() + var = cg.Pvariable(config[CONF_ID], rhs, model) + + await display.register_display(var, config) + await spi.register_spi_device(var, config) + + dc = await cg.gpio_pin_expression(config[CONF_DC_PIN]) + cg.add(var.set_dc_pin(dc)) + + if CONF_LAMBDA in config: + lambda_ = await cg.process_lambda( + config[CONF_LAMBDA], [(display.DisplayRef, "it")], return_type=cg.void + ) + cg.add(var.set_writer(lambda_)) + if CONF_RESET_PIN in config: + reset = await cg.gpio_pin_expression(config[CONF_RESET_PIN]) + cg.add(var.set_reset_pin(reset)) + if CONF_BUSY_PIN in config: + busy = await cg.gpio_pin_expression(config[CONF_BUSY_PIN]) + cg.add(var.set_busy_pin(busy)) + if CONF_RESET_DURATION in config: + cg.add(var.set_reset_duration(config[CONF_RESET_DURATION])) diff --git a/esphome/components/epaper_spi/epaper_spi.cpp b/esphome/components/epaper_spi/epaper_spi.cpp new file mode 100644 index 0000000000..21be4a2c05 --- /dev/null +++ b/esphome/components/epaper_spi/epaper_spi.cpp @@ -0,0 +1,227 @@ +#include "epaper_spi.h" +#include +#include "esphome/core/application.h" +#include "esphome/core/helpers.h" +#include "esphome/core/log.h" + +namespace esphome::epaper_spi { + +static const char *const TAG = "epaper_spi"; + +static const LogString *epaper_state_to_string(EPaperState state) { + switch (state) { + case EPaperState::IDLE: + return LOG_STR("IDLE"); + case EPaperState::UPDATE: + return LOG_STR("UPDATE"); + case EPaperState::RESET: + return LOG_STR("RESET"); + case EPaperState::INITIALISE: + return LOG_STR("INITIALISE"); + case EPaperState::TRANSFER_DATA: + return LOG_STR("TRANSFER_DATA"); + case EPaperState::POWER_ON: + return LOG_STR("POWER_ON"); + case EPaperState::REFRESH_SCREEN: + return LOG_STR("REFRESH_SCREEN"); + case EPaperState::POWER_OFF: + return LOG_STR("POWER_OFF"); + case EPaperState::DEEP_SLEEP: + return LOG_STR("DEEP_SLEEP"); + default: + return LOG_STR("UNKNOWN"); + } +} + +void EPaperBase::setup() { + if (!this->init_buffer_(this->get_buffer_length())) { + this->mark_failed("Failed to initialise buffer"); + return; + } + this->setup_pins_(); + this->spi_setup(); +} + +bool EPaperBase::init_buffer_(size_t buffer_length) { + if (!this->buffer_.init(buffer_length)) { + return false; + } + this->clear(); + return true; +} + +void EPaperBase::setup_pins_() { + this->dc_pin_->setup(); // OUTPUT + this->dc_pin_->digital_write(false); + + if (this->reset_pin_ != nullptr) { + this->reset_pin_->setup(); // OUTPUT + this->reset_pin_->digital_write(true); + } + + if (this->busy_pin_ != nullptr) { + this->busy_pin_->setup(); // INPUT + } +} + +float EPaperBase::get_setup_priority() const { return setup_priority::PROCESSOR; } + +void EPaperBase::command(uint8_t value) { + this->start_command_(); + this->write_byte(value); + this->end_command_(); +} + +void EPaperBase::data(uint8_t value) { + this->start_data_(); + this->write_byte(value); + this->end_data_(); +} + +// write a command followed by zero or more bytes of data. +// The command is the first byte, length is the length of data only in the second byte, followed by the data. +// [COMMAND, LENGTH, DATA...] +void EPaperBase::cmd_data(const uint8_t *data) { + const uint8_t command = data[0]; + const uint8_t length = data[1]; + const uint8_t *ptr = data + 2; + + ESP_LOGVV(TAG, "Command: 0x%02X, Length: %d, Data: %s", command, length, + format_hex_pretty(ptr, length, '.', false).c_str()); + + this->dc_pin_->digital_write(false); + this->enable(); + this->write_byte(command); + if (length > 0) { + this->dc_pin_->digital_write(true); + this->write_array(ptr, length); + } + this->disable(); +} + +bool EPaperBase::is_idle_() { + if (this->busy_pin_ == nullptr) { + return true; + } + return !this->busy_pin_->digital_read(); +} + +void EPaperBase::reset() { + if (this->reset_pin_ != nullptr) { + this->reset_pin_->digital_write(false); + this->disable_loop(); + this->set_timeout(this->reset_duration_, [this] { + this->reset_pin_->digital_write(true); + this->set_timeout(20, [this] { this->enable_loop(); }); + }); + } +} + +void EPaperBase::update() { + if (!this->state_queue_.empty()) { + ESP_LOGE(TAG, "Display update already in progress - %s", + LOG_STR_ARG(epaper_state_to_string(this->state_queue_.front()))); + return; + } + + this->state_queue_.push(EPaperState::UPDATE); + this->state_queue_.push(EPaperState::RESET); + this->state_queue_.push(EPaperState::INITIALISE); + this->state_queue_.push(EPaperState::TRANSFER_DATA); + this->state_queue_.push(EPaperState::POWER_ON); + this->state_queue_.push(EPaperState::REFRESH_SCREEN); + this->state_queue_.push(EPaperState::POWER_OFF); + this->state_queue_.push(EPaperState::DEEP_SLEEP); + this->state_queue_.push(EPaperState::IDLE); + + this->enable_loop(); +} + +void EPaperBase::loop() { + if (this->waiting_for_idle_) { + if (this->is_idle_()) { + this->waiting_for_idle_ = false; + } else { + if (App.get_loop_component_start_time() - this->waiting_for_idle_last_print_ >= 1000) { + ESP_LOGV(TAG, "Waiting for idle"); + this->waiting_for_idle_last_print_ = App.get_loop_component_start_time(); + } + return; + } + } + + auto state = this->state_queue_.front(); + + switch (state) { + case EPaperState::IDLE: + this->disable_loop(); + break; + case EPaperState::UPDATE: + this->do_update_(); // Calls ESPHome (current page) lambda + break; + case EPaperState::RESET: + this->reset(); + break; + case EPaperState::INITIALISE: + this->initialise_(); + break; + case EPaperState::TRANSFER_DATA: + if (!this->transfer_data()) { + return; // Not done yet, come back next loop + } + break; + case EPaperState::POWER_ON: + this->power_on(); + break; + case EPaperState::REFRESH_SCREEN: + this->refresh_screen(); + break; + case EPaperState::POWER_OFF: + this->power_off(); + break; + case EPaperState::DEEP_SLEEP: + this->deep_sleep(); + break; + } + this->state_queue_.pop(); +} + +void EPaperBase::start_command_() { + this->dc_pin_->digital_write(false); + this->enable(); +} + +void EPaperBase::end_command_() { this->disable(); } + +void EPaperBase::start_data_() { + this->dc_pin_->digital_write(true); + this->enable(); +} +void EPaperBase::end_data_() { this->disable(); } + +void EPaperBase::on_safe_shutdown() { this->deep_sleep(); } + +void EPaperBase::initialise_() { + size_t index = 0; + const auto &sequence = this->init_sequence_; + const size_t sequence_size = this->init_sequence_length_; + while (index != sequence_size) { + if (sequence_size - index < 2) { + this->mark_failed("Malformed init sequence"); + return; + } + const auto *ptr = sequence + index; + const uint8_t length = ptr[1]; + if (sequence_size - index < length + 2) { + this->mark_failed("Malformed init sequence"); + return; + } + + this->cmd_data(ptr); + index += length + 2; + } + + this->power_on(); +} + +} // namespace esphome::epaper_spi diff --git a/esphome/components/epaper_spi/epaper_spi.h b/esphome/components/epaper_spi/epaper_spi.h new file mode 100644 index 0000000000..f6b2d41c65 --- /dev/null +++ b/esphome/components/epaper_spi/epaper_spi.h @@ -0,0 +1,93 @@ +#pragma once + +#include "esphome/components/display/display_buffer.h" +#include "esphome/components/spi/spi.h" +#include "esphome/components/split_buffer/split_buffer.h" +#include "esphome/core/component.h" + +#include + +namespace esphome::epaper_spi { + +enum class EPaperState : uint8_t { + IDLE, + UPDATE, + RESET, + INITIALISE, + TRANSFER_DATA, + POWER_ON, + REFRESH_SCREEN, + POWER_OFF, + DEEP_SLEEP, +}; + +static const uint8_t MAX_TRANSFER_TIME = 10; // Transfer in 10ms blocks to allow the loop to run + +class EPaperBase : public display::DisplayBuffer, + public spi::SPIDevice { + public: + EPaperBase(const uint8_t *init_sequence, const size_t init_sequence_length) + : init_sequence_length_(init_sequence_length), init_sequence_(init_sequence) {} + void set_dc_pin(GPIOPin *dc_pin) { dc_pin_ = dc_pin; } + float get_setup_priority() const override; + void set_reset_pin(GPIOPin *reset) { this->reset_pin_ = reset; } + void set_busy_pin(GPIOPin *busy) { this->busy_pin_ = busy; } + void set_reset_duration(uint32_t reset_duration) { this->reset_duration_ = reset_duration; } + + void command(uint8_t value); + void data(uint8_t value); + void cmd_data(const uint8_t *data); + + void update() override; + void loop() override; + + void setup() override; + + void on_safe_shutdown() override; + + protected: + bool is_idle_(); + void setup_pins_(); + virtual void reset(); + void initialise_(); + bool init_buffer_(size_t buffer_length); + + virtual int get_width_controller() { return this->get_width_internal(); }; + virtual void deep_sleep() = 0; + /** + * Send data to the device via SPI + * @return true if done, false if should be called next loop + */ + virtual bool transfer_data() = 0; + virtual void refresh_screen() = 0; + + virtual void power_on() = 0; + virtual void power_off() = 0; + virtual uint32_t get_buffer_length() = 0; + + void start_command_(); + void end_command_(); + void start_data_(); + void end_data_(); + + const size_t init_sequence_length_{0}; + + size_t current_data_index_{0}; + uint32_t reset_duration_{200}; + uint32_t waiting_for_idle_last_print_{0}; + + GPIOPin *dc_pin_; + GPIOPin *busy_pin_{nullptr}; + GPIOPin *reset_pin_{nullptr}; + + const uint8_t *init_sequence_{nullptr}; + + bool waiting_for_idle_{false}; + + split_buffer::SplitBuffer buffer_; + + std::queue state_queue_{{EPaperState::IDLE}}; +}; + +} // namespace esphome::epaper_spi diff --git a/esphome/components/epaper_spi/epaper_spi_model_7p3in_spectra_e6.cpp b/esphome/components/epaper_spi/epaper_spi_model_7p3in_spectra_e6.cpp new file mode 100644 index 0000000000..f6273b392f --- /dev/null +++ b/esphome/components/epaper_spi/epaper_spi_model_7p3in_spectra_e6.cpp @@ -0,0 +1,42 @@ +#include "epaper_spi_model_7p3in_spectra_e6.h" + +namespace esphome::epaper_spi { + +static constexpr const char *const TAG = "epaper_spi.7.3in-spectra-e6"; + +void EPaper7p3InSpectraE6::power_on() { + ESP_LOGI(TAG, "Power on"); + this->command(0x04); + this->waiting_for_idle_ = true; +} + +void EPaper7p3InSpectraE6::power_off() { + ESP_LOGI(TAG, "Power off"); + this->command(0x02); + this->data(0x00); + this->waiting_for_idle_ = true; +} + +void EPaper7p3InSpectraE6::refresh_screen() { + ESP_LOGI(TAG, "Refresh"); + this->command(0x12); + this->data(0x00); + this->waiting_for_idle_ = true; +} + +void EPaper7p3InSpectraE6::deep_sleep() { + ESP_LOGI(TAG, "Deep sleep"); + this->command(0x07); + this->data(0xA5); +} + +void EPaper7p3InSpectraE6::dump_config() { + LOG_DISPLAY("", "E-Paper SPI", this); + ESP_LOGCONFIG(TAG, " Model: 7.3in Spectra E6"); + LOG_PIN(" Reset Pin: ", this->reset_pin_); + LOG_PIN(" DC Pin: ", this->dc_pin_); + LOG_PIN(" Busy Pin: ", this->busy_pin_); + LOG_UPDATE_INTERVAL(this); +} + +} // namespace esphome::epaper_spi diff --git a/esphome/components/epaper_spi/epaper_spi_model_7p3in_spectra_e6.h b/esphome/components/epaper_spi/epaper_spi_model_7p3in_spectra_e6.h new file mode 100644 index 0000000000..6e850085ac --- /dev/null +++ b/esphome/components/epaper_spi/epaper_spi_model_7p3in_spectra_e6.h @@ -0,0 +1,45 @@ +#pragma once + +#include "epaper_spi_spectra_e6.h" + +namespace esphome::epaper_spi { + +class EPaper7p3InSpectraE6 : public EPaperSpectraE6 { + static constexpr const uint16_t WIDTH = 800; + static constexpr const uint16_t HEIGHT = 480; + // clang-format off + + // Command, data length, data + static constexpr uint8_t INIT_SEQUENCE[] = { + 0xAA, 6, 0x49, 0x55, 0x20, 0x08, 0x09, 0x18, + 0x01, 1, 0x3F, + 0x00, 2, 0x5F, 0x69, + 0x03, 4, 0x00, 0x54, 0x00, 0x44, + 0x05, 4, 0x40, 0x1F, 0x1F, 0x2C, + 0x06, 4, 0x6F, 0x1F, 0x17, 0x49, + 0x08, 4, 0x6F, 0x1F, 0x1F, 0x22, + 0x30, 1, 0x03, + 0x50, 1, 0x3F, + 0x60, 2, 0x02, 0x00, + 0x61, 4, WIDTH / 256, WIDTH % 256, HEIGHT / 256, HEIGHT % 256, + 0x84, 1, 0x01, + 0xE3, 1, 0x2F, + }; + // clang-format on + + public: + EPaper7p3InSpectraE6() : EPaperSpectraE6(INIT_SEQUENCE, sizeof(INIT_SEQUENCE)) {} + + void dump_config() override; + + protected: + int get_width_internal() override { return WIDTH; }; + int get_height_internal() override { return HEIGHT; }; + + void refresh_screen() override; + void power_on() override; + void power_off() override; + void deep_sleep() override; +}; + +} // namespace esphome::epaper_spi diff --git a/esphome/components/epaper_spi/epaper_spi_spectra_e6.cpp b/esphome/components/epaper_spi/epaper_spi_spectra_e6.cpp new file mode 100644 index 0000000000..dccc691252 --- /dev/null +++ b/esphome/components/epaper_spi/epaper_spi_spectra_e6.cpp @@ -0,0 +1,135 @@ +#include "epaper_spi_spectra_e6.h" + +#include "esphome/core/log.h" + +namespace esphome::epaper_spi { + +static constexpr const char *const TAG = "epaper_spi.6c"; + +static inline uint8_t color_to_hex(Color color) { + if (color.red > 127) { + if (color.green > 170) { + if (color.blue > 127) { + return 0x1; // White + } else { + return 0x2; // Yellow + } + } else { + return 0x3; // Red (or Magenta) + } + } else { + if (color.green > 127) { + if (color.blue > 127) { + return 0x5; // Cyan -> Blue + } else { + return 0x6; // Green + } + } else { + if (color.blue > 127) { + return 0x5; // Blue + } else { + return 0x0; // Black + } + } + } +} + +void EPaperSpectraE6::fill(Color color) { + uint8_t pixel_color; + if (color.is_on()) { + pixel_color = color_to_hex(color); + } else { + pixel_color = 0x1; + } + + // We store 8 bitset<3> in 3 bytes + // | byte 1 | byte 2 | byte 3 | + // |aaabbbaa|abbbaaab|bbaaabbb| + uint8_t byte_1 = pixel_color << 5 | pixel_color << 2 | pixel_color >> 1; + uint8_t byte_2 = pixel_color << 7 | pixel_color << 4 | pixel_color << 1 | pixel_color >> 2; + uint8_t byte_3 = pixel_color << 6 | pixel_color << 3 | pixel_color << 0; + + const size_t buffer_length = this->get_buffer_length(); + for (size_t i = 0; i < buffer_length; i += 3) { + this->buffer_[i + 0] = byte_1; + this->buffer_[i + 1] = byte_2; + this->buffer_[i + 2] = byte_3; + } +} + +uint32_t EPaperSpectraE6::get_buffer_length() { + // 6 colors buffer, 1 pixel = 3 bits, we will store 8 pixels in 24 bits = 3 bytes + return this->get_width_controller() * this->get_height_internal() / 8u * 3u; +} + +void HOT EPaperSpectraE6::draw_absolute_pixel_internal(int x, int y, Color color) { + if (x >= this->get_width_internal() || y >= this->get_height_internal() || x < 0 || y < 0) + return; + + uint8_t pixel_bits = color_to_hex(color); + uint32_t pixel_position = x + y * this->get_width_controller(); + uint32_t first_bit_position = pixel_position * 3; + uint32_t byte_position = first_bit_position / 8u; + uint32_t byte_subposition = first_bit_position % 8u; + + if (byte_subposition <= 5) { + this->buffer_[byte_position] = (this->buffer_[byte_position] & (0xFF ^ (0b111 << (5 - byte_subposition)))) | + (pixel_bits << (5 - byte_subposition)); + } else { + this->buffer_[byte_position] = (this->buffer_[byte_position] & (0xFF ^ (0b111 >> (byte_subposition - 5)))) | + (pixel_bits >> (byte_subposition - 5)); + + this->buffer_[byte_position + 1] = + (this->buffer_[byte_position + 1] & (0xFF ^ (0xFF & (0b111 << (13 - byte_subposition))))) | + (pixel_bits << (13 - byte_subposition)); + } +} + +bool HOT EPaperSpectraE6::transfer_data() { + const uint32_t start_time = App.get_loop_component_start_time(); + if (this->current_data_index_ == 0) { + ESP_LOGV(TAG, "Sending data"); + this->command(0x10); + } + + uint8_t bytes_to_send[4]{0}; + const size_t buffer_length = this->get_buffer_length(); + for (size_t i = this->current_data_index_; i < buffer_length; i += 3) { + const uint32_t triplet = encode_uint24(this->buffer_[i + 0], this->buffer_[i + 1], this->buffer_[i + 2]); + // 8 pixels are stored in 3 bytes + // |aaabbbaa|abbbaaab|bbaaabbb| + // | byte 1 | byte 2 | byte 3 | + bytes_to_send[0] = ((triplet >> 17) & 0b01110000) | ((triplet >> 18) & 0b00000111); + bytes_to_send[1] = ((triplet >> 11) & 0b01110000) | ((triplet >> 12) & 0b00000111); + bytes_to_send[2] = ((triplet >> 5) & 0b01110000) | ((triplet >> 6) & 0b00000111); + bytes_to_send[3] = ((triplet << 1) & 0b01110000) | ((triplet << 0) & 0b00000111); + + this->start_data_(); + this->write_array(bytes_to_send, sizeof(bytes_to_send)); + this->end_data_(); + + if (millis() - start_time > MAX_TRANSFER_TIME) { + // Let the main loop run and come back next loop + this->current_data_index_ = i + 3; + return false; + } + } + // Finished the entire dataset + this->current_data_index_ = 0; + return true; +} + +void EPaperSpectraE6::reset() { + if (this->reset_pin_ != nullptr) { + this->disable_loop(); + this->reset_pin_->digital_write(true); + this->set_timeout(20, [this] { + this->reset_pin_->digital_write(false); + delay(2); + this->reset_pin_->digital_write(true); + this->set_timeout(20, [this] { this->enable_loop(); }); + }); + } +} + +} // namespace esphome::epaper_spi diff --git a/esphome/components/epaper_spi/epaper_spi_spectra_e6.h b/esphome/components/epaper_spi/epaper_spi_spectra_e6.h new file mode 100644 index 0000000000..9f0652f79d --- /dev/null +++ b/esphome/components/epaper_spi/epaper_spi_spectra_e6.h @@ -0,0 +1,23 @@ +#pragma once + +#include "epaper_spi.h" + +namespace esphome::epaper_spi { + +class EPaperSpectraE6 : public EPaperBase { + public: + EPaperSpectraE6(const uint8_t *init_sequence, const size_t init_sequence_length) + : EPaperBase(init_sequence, init_sequence_length) {} + + display::DisplayType get_display_type() override { return display::DisplayType::DISPLAY_TYPE_COLOR; } + void fill(Color color) override; + + protected: + void draw_absolute_pixel_internal(int x, int y, Color color) override; + uint32_t get_buffer_length() override; + + bool transfer_data() override; + void reset() override; +}; + +} // namespace esphome::epaper_spi diff --git a/esphome/components/es7210/es7210.cpp b/esphome/components/es7210/es7210.cpp index bcbaf3d270..1358121c1b 100644 --- a/esphome/components/es7210/es7210.cpp +++ b/esphome/components/es7210/es7210.cpp @@ -38,8 +38,6 @@ void ES7210::dump_config() { } void ES7210::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // Software reset ES7210_ERROR_FAILED(this->write_byte(ES7210_RESET_REG00, 0xff)); ES7210_ERROR_FAILED(this->write_byte(ES7210_RESET_REG00, 0x32)); @@ -99,12 +97,12 @@ bool ES7210::set_mic_gain(float mic_gain) { } bool ES7210::configure_sample_rate_() { - int mclk_fre = this->sample_rate_ * MCLK_DIV_FRE; + uint32_t mclk_fre = this->sample_rate_ * MCLK_DIV_FRE; int coeff = -1; - for (int i = 0; i < (sizeof(ES7210_COEFFICIENTS) / sizeof(ES7210_COEFFICIENTS[0])); ++i) { + for (size_t i = 0; i < (sizeof(ES7210_COEFFICIENTS) / sizeof(ES7210_COEFFICIENTS[0])); ++i) { if (ES7210_COEFFICIENTS[i].lrclk == this->sample_rate_ && ES7210_COEFFICIENTS[i].mclk == mclk_fre) - coeff = i; + coeff = static_cast(i); } if (coeff >= 0) { diff --git a/esphome/components/es7243e/es7243e.cpp b/esphome/components/es7243e/es7243e.cpp index d5115cb880..d45c1d5a8c 100644 --- a/esphome/components/es7243e/es7243e.cpp +++ b/esphome/components/es7243e/es7243e.cpp @@ -34,8 +34,6 @@ void ES7243E::dump_config() { } void ES7243E::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - ES7243E_ERROR_FAILED(this->write_byte(ES7243E_CLOCK_MGR_REG01, 0x3A)); ES7243E_ERROR_FAILED(this->write_byte(ES7243E_RESET_REG00, 0x80)); ES7243E_ERROR_FAILED(this->write_byte(ES7243E_TEST_MODE_REGF9, 0x00)); diff --git a/esphome/components/es8156/es8156.cpp b/esphome/components/es8156/es8156.cpp index c8330b4f84..e84252efe2 100644 --- a/esphome/components/es8156/es8156.cpp +++ b/esphome/components/es8156/es8156.cpp @@ -17,8 +17,6 @@ static const char *const TAG = "es8156"; } void ES8156::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - ES8156_ERROR_FAILED(this->write_byte(ES8156_REG02_SCLK_MODE, 0x04)); ES8156_ERROR_FAILED(this->write_byte(ES8156_REG20_ANALOG_SYS1, 0x2A)); ES8156_ERROR_FAILED(this->write_byte(ES8156_REG21_ANALOG_SYS2, 0x3C)); diff --git a/esphome/components/es8311/es8311.cpp b/esphome/components/es8311/es8311.cpp index 0e59ac12d5..cf864187f9 100644 --- a/esphome/components/es8311/es8311.cpp +++ b/esphome/components/es8311/es8311.cpp @@ -22,8 +22,6 @@ static const char *const TAG = "es8311"; } void ES8311::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // Reset ES8311_ERROR_FAILED(this->write_byte(ES8311_REG00_RESET, 0x1F)); ES8311_ERROR_FAILED(this->write_byte(ES8311_REG00_RESET, 0x00)); diff --git a/esphome/components/es8388/es8388.cpp b/esphome/components/es8388/es8388.cpp index 87cf9a47ee..69c16a9615 100644 --- a/esphome/components/es8388/es8388.cpp +++ b/esphome/components/es8388/es8388.cpp @@ -23,8 +23,6 @@ static const char *const TAG = "es8388"; } void ES8388::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // mute DAC this->set_mute_state_(true); diff --git a/esphome/components/esp32/__init__.py b/esphome/components/esp32/__init__.py index fdc469e419..b7dd25e0d8 100644 --- a/esphome/components/esp32/__init__.py +++ b/esphome/components/esp32/__init__.py @@ -15,6 +15,7 @@ from esphome.const import ( CONF_FRAMEWORK, CONF_IGNORE_EFUSE_CUSTOM_MAC, CONF_IGNORE_EFUSE_MAC_CRC, + CONF_LOG_LEVEL, CONF_NAME, CONF_PATH, CONF_PLATFORM_VERSION, @@ -31,15 +32,16 @@ from esphome.const import ( KEY_TARGET_FRAMEWORK, KEY_TARGET_PLATFORM, PLATFORM_ESP32, + ThreadModel, __version__, ) from esphome.core import CORE, HexInt, TimePeriod -from esphome.cpp_generator import RawExpression import esphome.final_validate as fv -from esphome.helpers import copy_file_if_changed, mkdir_p, write_file_if_changed +from esphome.helpers import copy_file_if_changed, write_file_if_changed from esphome.types import ConfigType +from esphome.writer import clean_cmake_cache -from .boards import BOARDS +from .boards import BOARDS, STANDARD_BOARDS from .const import ( # noqa KEY_BOARD, KEY_COMPONENTS, @@ -75,8 +77,18 @@ CONF_ASSERTION_LEVEL = "assertion_level" CONF_COMPILER_OPTIMIZATION = "compiler_optimization" CONF_ENABLE_IDF_EXPERIMENTAL_FEATURES = "enable_idf_experimental_features" CONF_ENABLE_LWIP_ASSERT = "enable_lwip_assert" +CONF_EXECUTE_FROM_PSRAM = "execute_from_psram" CONF_RELEASE = "release" +LOG_LEVELS_IDF = [ + "NONE", + "ERROR", + "WARN", + "INFO", + "DEBUG", + "VERBOSE", +] + ASSERTION_LEVELS = { "DISABLE": "CONFIG_COMPILER_OPTIMIZATION_ASSERTIONS_DISABLE", "ENABLE": "CONFIG_COMPILER_OPTIMIZATION_ASSERTIONS_ENABLE", @@ -144,8 +156,6 @@ def set_core_data(config): conf = config[CONF_FRAMEWORK] if conf[CONF_TYPE] == FRAMEWORK_ESP_IDF: CORE.data[KEY_CORE][KEY_TARGET_FRAMEWORK] = "esp-idf" - CORE.data[KEY_ESP32][KEY_SDKCONFIG_OPTIONS] = {} - CORE.data[KEY_ESP32][KEY_COMPONENTS] = {} elif conf[CONF_TYPE] == FRAMEWORK_ARDUINO: CORE.data[KEY_CORE][KEY_TARGET_FRAMEWORK] = "arduino" if variant not in ARDUINO_ALLOWED_VARIANTS: @@ -153,6 +163,8 @@ def set_core_data(config): f"ESPHome does not support using the Arduino framework for the {variant}. Please use the ESP-IDF framework instead.", path=[CONF_FRAMEWORK, CONF_TYPE], ) + CORE.data[KEY_ESP32][KEY_SDKCONFIG_OPTIONS] = {} + CORE.data[KEY_ESP32][KEY_COMPONENTS] = {} CORE.data[KEY_CORE][KEY_FRAMEWORK_VERSION] = cv.Version.parse( config[CONF_FRAMEWORK][CONF_VERSION] ) @@ -223,8 +235,6 @@ SdkconfigValueType = bool | int | HexInt | str | RawSdkconfigValue def add_idf_sdkconfig_option(name: str, value: SdkconfigValueType): """Set an esp-idf sdkconfig value.""" - if not CORE.using_esp_idf: - raise ValueError("Not an esp-idf project") CORE.data[KEY_ESP32][KEY_SDKCONFIG_OPTIONS][name] = value @@ -239,8 +249,6 @@ def add_idf_component( submodules: list[str] | None = None, ): """Add an esp-idf component to the project.""" - if not CORE.using_esp_idf: - raise ValueError("Not an esp-idf project") if not repo and not ref and not path: raise ValueError("Requires at least one of repo, ref or path") if refresh or submodules or components: @@ -264,14 +272,14 @@ def add_idf_component( } -def add_extra_script(stage: str, filename: str, path: str): +def add_extra_script(stage: str, filename: str, path: Path): """Add an extra script to the project.""" key = f"{stage}:{filename}" if add_extra_build_file(filename, path): cg.add_platformio_option("extra_scripts", [key]) -def add_extra_build_file(filename: str, path: str) -> bool: +def add_extra_build_file(filename: str, path: Path) -> bool: """Add an extra build file to the project.""" if filename not in CORE.data[KEY_ESP32][KEY_EXTRA_BUILD_FILES]: CORE.data[KEY_ESP32][KEY_EXTRA_BUILD_FILES][filename] = { @@ -288,19 +296,25 @@ def _format_framework_arduino_version(ver: cv.Version) -> str: return f"pioarduino/framework-arduinoespressif32@https://github.com/espressif/arduino-esp32/releases/download/{str(ver)}/esp32-{str(ver)}.zip" -def _format_framework_espidf_version( - ver: cv.Version, release: str, for_platformio: bool -) -> str: - # format the given arduino (https://github.com/espressif/esp-idf/releases) version to +def _format_framework_espidf_version(ver: cv.Version, release: str) -> str: + # format the given espidf (https://github.com/pioarduino/esp-idf/releases) version to # a PIO platformio/framework-espidf value - # List of package versions: https://api.registry.platformio.org/v3/packages/platformio/tool/framework-espidf - if for_platformio: - return f"platformio/framework-espidf@~3.{ver.major}{ver.minor:02d}{ver.patch:02d}.0" if release: return f"pioarduino/framework-espidf@https://github.com/pioarduino/esp-idf/releases/download/v{str(ver)}.{release}/esp-idf-v{str(ver)}.zip" return f"pioarduino/framework-espidf@https://github.com/pioarduino/esp-idf/releases/download/v{str(ver)}/esp-idf-v{str(ver)}.zip" +def _is_framework_url(source: str) -> str: + # platformio accepts many URL schemes for framework repositories and archives including http, https, git, file, and symlink + import urllib.parse + + try: + parsed = urllib.parse.urlparse(source) + except ValueError: + return False + return bool(parsed.scheme) + + # NOTE: Keep this in mind when updating the recommended version: # * New framework historically have had some regressions, especially for WiFi. # The new version needs to be thoroughly validated before changing the @@ -309,154 +323,115 @@ def _format_framework_espidf_version( # The default/recommended arduino framework version # - https://github.com/espressif/arduino-esp32/releases -RECOMMENDED_ARDUINO_FRAMEWORK_VERSION = cv.Version(3, 1, 3) -# The platform-espressif32 version to use for arduino frameworks -# - https://github.com/pioarduino/platform-espressif32/releases -ARDUINO_PLATFORM_VERSION = cv.Version(53, 3, 13) +ARDUINO_FRAMEWORK_VERSION_LOOKUP = { + "recommended": cv.Version(3, 3, 2), + "latest": cv.Version(3, 3, 2), + "dev": cv.Version(3, 3, 2), +} +ARDUINO_PLATFORM_VERSION_LOOKUP = { + cv.Version(3, 3, 2): cv.Version(55, 3, 31, "1"), + cv.Version(3, 3, 1): cv.Version(55, 3, 31, "1"), + cv.Version(3, 3, 0): cv.Version(55, 3, 30, "2"), + cv.Version(3, 2, 1): cv.Version(54, 3, 21, "2"), + cv.Version(3, 2, 0): cv.Version(54, 3, 20), + cv.Version(3, 1, 3): cv.Version(53, 3, 13), + cv.Version(3, 1, 2): cv.Version(53, 3, 12), + cv.Version(3, 1, 1): cv.Version(53, 3, 11), + cv.Version(3, 1, 0): cv.Version(53, 3, 10), +} # The default/recommended esp-idf framework version # - https://github.com/espressif/esp-idf/releases -# - https://api.registry.platformio.org/v3/packages/platformio/tool/framework-espidf -RECOMMENDED_ESP_IDF_FRAMEWORK_VERSION = cv.Version(5, 3, 2) -# The platformio/espressif32 version to use for esp-idf frameworks -# - https://github.com/platformio/platform-espressif32/releases -# - https://api.registry.platformio.org/v3/packages/platformio/platform/espressif32 -ESP_IDF_PLATFORM_VERSION = cv.Version(53, 3, 13) +ESP_IDF_FRAMEWORK_VERSION_LOOKUP = { + "recommended": cv.Version(5, 5, 1), + "latest": cv.Version(5, 5, 1), + "dev": cv.Version(5, 5, 1), +} +ESP_IDF_PLATFORM_VERSION_LOOKUP = { + cv.Version(5, 5, 1): cv.Version(55, 3, 31, "1"), + cv.Version(5, 5, 0): cv.Version(55, 3, 31, "1"), + cv.Version(5, 4, 2): cv.Version(54, 3, 21, "2"), + cv.Version(5, 4, 1): cv.Version(54, 3, 21, "2"), + cv.Version(5, 4, 0): cv.Version(54, 3, 21, "2"), + cv.Version(5, 3, 2): cv.Version(53, 3, 13), + cv.Version(5, 3, 1): cv.Version(53, 3, 13), + cv.Version(5, 3, 0): cv.Version(53, 3, 13), + cv.Version(5, 1, 6): cv.Version(51, 3, 7), + cv.Version(5, 1, 5): cv.Version(51, 3, 7), +} -# List based on https://registry.platformio.org/tools/platformio/framework-espidf/versions -SUPPORTED_PLATFORMIO_ESP_IDF_5X = [ - cv.Version(5, 3, 1), - cv.Version(5, 3, 0), - cv.Version(5, 2, 2), - cv.Version(5, 2, 1), - cv.Version(5, 1, 2), - cv.Version(5, 1, 1), - cv.Version(5, 1, 0), - cv.Version(5, 0, 2), - cv.Version(5, 0, 1), - cv.Version(5, 0, 0), -] - -# pioarduino versions that don't require a release number -# List based on https://github.com/pioarduino/esp-idf/releases -SUPPORTED_PIOARDUINO_ESP_IDF_5X = [ - cv.Version(5, 5, 0), - cv.Version(5, 4, 2), - cv.Version(5, 4, 1), - cv.Version(5, 4, 0), - cv.Version(5, 3, 3), - cv.Version(5, 3, 2), - cv.Version(5, 3, 1), - cv.Version(5, 3, 0), - cv.Version(5, 1, 5), - cv.Version(5, 1, 6), -] +# The platform-espressif32 version +# - https://github.com/pioarduino/platform-espressif32/releases +PLATFORM_VERSION_LOOKUP = { + "recommended": cv.Version(55, 3, 31, "1"), + "latest": cv.Version(55, 3, 31, "1"), + "dev": cv.Version(55, 3, 31, "1"), +} -def _arduino_check_versions(value): +def _check_versions(value): value = value.copy() - lookups = { - "dev": (cv.Version(3, 1, 3), "https://github.com/espressif/arduino-esp32.git"), - "latest": (cv.Version(3, 1, 3), None), - "recommended": (RECOMMENDED_ARDUINO_FRAMEWORK_VERSION, None), - } - if value[CONF_VERSION] in lookups: - if CONF_SOURCE in value: + if value[CONF_VERSION] in PLATFORM_VERSION_LOOKUP: + if CONF_SOURCE in value or CONF_PLATFORM_VERSION in value: raise cv.Invalid( - "Framework version needs to be explicitly specified when custom source is used." + "Version needs to be explicitly set when a custom source or platform_version is used." ) - version, source = lookups[value[CONF_VERSION]] + platform_lookup = PLATFORM_VERSION_LOOKUP[value[CONF_VERSION]] + value[CONF_PLATFORM_VERSION] = _parse_platform_version(str(platform_lookup)) + + if value[CONF_TYPE] == FRAMEWORK_ARDUINO: + version = ARDUINO_FRAMEWORK_VERSION_LOOKUP[value[CONF_VERSION]] + else: + version = ESP_IDF_FRAMEWORK_VERSION_LOOKUP[value[CONF_VERSION]] else: version = cv.Version.parse(cv.version_number(value[CONF_VERSION])) - source = value.get(CONF_SOURCE, None) value[CONF_VERSION] = str(version) - value[CONF_SOURCE] = source or _format_framework_arduino_version(version) - value[CONF_PLATFORM_VERSION] = value.get( - CONF_PLATFORM_VERSION, _parse_platform_version(str(ARDUINO_PLATFORM_VERSION)) - ) + if value[CONF_TYPE] == FRAMEWORK_ARDUINO: + if version < cv.Version(3, 0, 0): + raise cv.Invalid("Only Arduino 3.0+ is supported.") + recommended_version = ARDUINO_FRAMEWORK_VERSION_LOOKUP["recommended"] + platform_lookup = ARDUINO_PLATFORM_VERSION_LOOKUP.get(version) + value[CONF_SOURCE] = value.get( + CONF_SOURCE, _format_framework_arduino_version(version) + ) + if _is_framework_url(value[CONF_SOURCE]): + value[CONF_SOURCE] = ( + f"pioarduino/framework-arduinoespressif32@{value[CONF_SOURCE]}" + ) + else: + if version < cv.Version(5, 0, 0): + raise cv.Invalid("Only ESP-IDF 5.0+ is supported.") + recommended_version = ESP_IDF_FRAMEWORK_VERSION_LOOKUP["recommended"] + platform_lookup = ESP_IDF_PLATFORM_VERSION_LOOKUP.get(version) + value[CONF_SOURCE] = value.get( + CONF_SOURCE, + _format_framework_espidf_version(version, value.get(CONF_RELEASE, None)), + ) + if _is_framework_url(value[CONF_SOURCE]): + value[CONF_SOURCE] = f"pioarduino/framework-espidf@{value[CONF_SOURCE]}" - if value[CONF_SOURCE].startswith("http"): - # prefix is necessary or platformio will complain with a cryptic error - value[CONF_SOURCE] = f"framework-arduinoespressif32@{value[CONF_SOURCE]}" + if CONF_PLATFORM_VERSION not in value: + if platform_lookup is None: + raise cv.Invalid( + "Framework version not recognized; please specify platform_version" + ) + value[CONF_PLATFORM_VERSION] = _parse_platform_version(str(platform_lookup)) - if version != RECOMMENDED_ARDUINO_FRAMEWORK_VERSION: + if version != recommended_version: _LOGGER.warning( - "The selected Arduino framework version is not the recommended one. " + "The selected framework version is not the recommended one. " "If there are connectivity or build issues please remove the manual version." ) - return value - - -def _esp_idf_check_versions(value): - value = value.copy() - lookups = { - "dev": (cv.Version(5, 3, 2), "https://github.com/espressif/esp-idf.git"), - "latest": (cv.Version(5, 3, 2), None), - "recommended": (RECOMMENDED_ESP_IDF_FRAMEWORK_VERSION, None), - } - - if value[CONF_VERSION] in lookups: - if CONF_SOURCE in value: - raise cv.Invalid( - "Framework version needs to be explicitly specified when custom source is used." - ) - - version, source = lookups[value[CONF_VERSION]] - else: - version = cv.Version.parse(cv.version_number(value[CONF_VERSION])) - source = value.get(CONF_SOURCE, None) - - if version < cv.Version(5, 0, 0): - raise cv.Invalid("Only ESP-IDF 5.0+ is supported.") - - # flag this for later *before* we set value[CONF_PLATFORM_VERSION] below - has_platform_ver = CONF_PLATFORM_VERSION in value - - value[CONF_PLATFORM_VERSION] = value.get( - CONF_PLATFORM_VERSION, _parse_platform_version(str(ESP_IDF_PLATFORM_VERSION)) - ) - - if ( - is_platformio := _platform_is_platformio(value[CONF_PLATFORM_VERSION]) - ) and version not in SUPPORTED_PLATFORMIO_ESP_IDF_5X: - raise cv.Invalid( - f"ESP-IDF {str(version)} not supported by platformio/espressif32" - ) - - if ( - version in SUPPORTED_PLATFORMIO_ESP_IDF_5X - and version not in SUPPORTED_PIOARDUINO_ESP_IDF_5X - ) and not has_platform_ver: - raise cv.Invalid( - f"ESP-IDF {value[CONF_VERSION]} may be supported by platformio/espressif32; please specify '{CONF_PLATFORM_VERSION}'" - ) - - if ( - not is_platformio - and CONF_RELEASE not in value - and version not in SUPPORTED_PIOARDUINO_ESP_IDF_5X + if value[CONF_PLATFORM_VERSION] != _parse_platform_version( + str(PLATFORM_VERSION_LOOKUP["recommended"]) ): - raise cv.Invalid( - f"ESP-IDF {value[CONF_VERSION]} is not available with pioarduino; you may need to specify '{CONF_RELEASE}'" - ) - - value[CONF_VERSION] = str(version) - value[CONF_SOURCE] = source or _format_framework_espidf_version( - version, value.get(CONF_RELEASE, None), is_platformio - ) - - if value[CONF_SOURCE].startswith("http"): - # prefix is necessary or platformio will complain with a cryptic error - value[CONF_SOURCE] = f"framework-espidf@{value[CONF_SOURCE]}" - - if version != RECOMMENDED_ESP_IDF_FRAMEWORK_VERSION: _LOGGER.warning( - "The selected ESP-IDF framework version is not the recommended one. " + "The selected platform version is not the recommended one. " "If there are connectivity or build issues please remove the manual version." ) @@ -466,103 +441,110 @@ def _esp_idf_check_versions(value): def _parse_platform_version(value): try: ver = cv.Version.parse(cv.version_number(value)) - if ver.major >= 50: # a pioarduino version - if "-" in value: - # maybe a release candidate?...definitely not our default, just use it as-is... - return f"https://github.com/pioarduino/platform-espressif32/releases/download/{value}/platform-espressif32.zip" - return f"https://github.com/pioarduino/platform-espressif32/releases/download/{ver.major}.{ver.minor:02d}.{ver.patch:02d}/platform-espressif32.zip" - # if platform version is a valid version constraint, prefix the default package - cv.platformio_version_constraint(value) - return f"platformio/espressif32@{value}" + release = f"{ver.major}.{ver.minor:02d}.{ver.patch:02d}" + if ver.extra: + release += f"-{ver.extra}" + return f"https://github.com/pioarduino/platform-espressif32/releases/download/{release}/platform-espressif32.zip" except cv.Invalid: return value -def _platform_is_platformio(value): - try: - ver = cv.Version.parse(cv.version_number(value)) - return ver.major < 50 - except cv.Invalid: - return "platformio" in value - - def _detect_variant(value): - board = value[CONF_BOARD] - if board in BOARDS: - variant = BOARDS[board][KEY_VARIANT] - if CONF_VARIANT in value and variant != value[CONF_VARIANT]: + board = value.get(CONF_BOARD) + variant = value.get(CONF_VARIANT) + if variant and board is None: + # If variant is set, we can derive the board from it + # variant has already been validated against the known set + value = value.copy() + value[CONF_BOARD] = STANDARD_BOARDS[variant] + elif board in BOARDS: + variant = variant or BOARDS[board][KEY_VARIANT] + if variant != BOARDS[board][KEY_VARIANT]: raise cv.Invalid( f"Option '{CONF_VARIANT}' does not match selected board.", path=[CONF_VARIANT], ) value = value.copy() value[CONF_VARIANT] = variant + elif not variant: + raise cv.Invalid( + "This board is unknown, if you are sure you want to compile with this board selection, " + f"override with option '{CONF_VARIANT}'", + path=[CONF_BOARD], + ) else: - if CONF_VARIANT not in value: - raise cv.Invalid( - "This board is unknown, if you are sure you want to compile with this board selection, " - f"override with option '{CONF_VARIANT}'", - path=[CONF_BOARD], - ) _LOGGER.warning( - "This board is unknown. Make sure the chosen chip component is correct.", + "This board is unknown; the specified variant '%s' will be used but this may not work as expected.", + variant, ) return value def final_validate(config): - if not ( - pio_options := fv.full_config.get()[CONF_ESPHOME].get(CONF_PLATFORMIO_OPTIONS) - ): - # Not specified or empty - return config - - pio_flash_size_key = "board_upload.flash_size" - pio_partitions_key = "board_build.partitions" - if CONF_PARTITIONS in config and pio_partitions_key in pio_options: - raise cv.Invalid( - f"Do not specify '{pio_partitions_key}' in '{CONF_PLATFORMIO_OPTIONS}' with '{CONF_PARTITIONS}' in esp32" - ) - - if pio_flash_size_key in pio_options: - raise cv.Invalid( - f"Please specify {CONF_FLASH_SIZE} within esp32 configuration only" - ) + # Imported locally to avoid circular import issues + from esphome.components.psram import DOMAIN as PSRAM_DOMAIN + errs = [] + full_config = fv.full_config.get() + if pio_options := full_config[CONF_ESPHOME].get(CONF_PLATFORMIO_OPTIONS): + pio_flash_size_key = "board_upload.flash_size" + pio_partitions_key = "board_build.partitions" + if CONF_PARTITIONS in config and pio_partitions_key in pio_options: + errs.append( + cv.Invalid( + f"Do not specify '{pio_partitions_key}' in '{CONF_PLATFORMIO_OPTIONS}' with '{CONF_PARTITIONS}' in esp32" + ) + ) + if pio_flash_size_key in pio_options: + errs.append( + cv.Invalid( + f"Please specify {CONF_FLASH_SIZE} within esp32 configuration only" + ) + ) if ( config[CONF_VARIANT] != VARIANT_ESP32 and CONF_ADVANCED in (conf_fw := config[CONF_FRAMEWORK]) and CONF_IGNORE_EFUSE_MAC_CRC in conf_fw[CONF_ADVANCED] ): - raise cv.Invalid( - f"{CONF_IGNORE_EFUSE_MAC_CRC} is not supported on {config[CONF_VARIANT]}" + errs.append( + cv.Invalid( + f"'{CONF_IGNORE_EFUSE_MAC_CRC}' is not supported on {config[CONF_VARIANT]}", + path=[CONF_FRAMEWORK, CONF_ADVANCED, CONF_IGNORE_EFUSE_MAC_CRC], + ) ) + if ( + config.get(CONF_FRAMEWORK, {}) + .get(CONF_ADVANCED, {}) + .get(CONF_EXECUTE_FROM_PSRAM) + ): + if config[CONF_VARIANT] != VARIANT_ESP32S3: + errs.append( + cv.Invalid( + f"'{CONF_EXECUTE_FROM_PSRAM}' is only supported on {VARIANT_ESP32S3} variant", + path=[CONF_FRAMEWORK, CONF_ADVANCED, CONF_EXECUTE_FROM_PSRAM], + ) + ) + if PSRAM_DOMAIN not in full_config: + errs.append( + cv.Invalid( + f"'{CONF_EXECUTE_FROM_PSRAM}' requires PSRAM to be configured", + path=[CONF_FRAMEWORK, CONF_ADVANCED, CONF_EXECUTE_FROM_PSRAM], + ) + ) + + if errs: + raise cv.MultipleInvalid(errs) return config -ARDUINO_FRAMEWORK_SCHEMA = cv.All( - cv.Schema( - { - cv.Optional(CONF_VERSION, default="recommended"): cv.string_strict, - cv.Optional(CONF_SOURCE): cv.string_strict, - cv.Optional(CONF_PLATFORM_VERSION): _parse_platform_version, - cv.Optional(CONF_ADVANCED, default={}): cv.Schema( - { - cv.Optional( - CONF_IGNORE_EFUSE_CUSTOM_MAC, default=False - ): cv.boolean, - } - ), - } - ), - _arduino_check_versions, -) - CONF_SDKCONFIG_OPTIONS = "sdkconfig_options" CONF_ENABLE_LWIP_DHCP_SERVER = "enable_lwip_dhcp_server" CONF_ENABLE_LWIP_MDNS_QUERIES = "enable_lwip_mdns_queries" CONF_ENABLE_LWIP_BRIDGE_INTERFACE = "enable_lwip_bridge_interface" +CONF_ENABLE_LWIP_TCPIP_CORE_LOCKING = "enable_lwip_tcpip_core_locking" +CONF_ENABLE_LWIP_CHECK_THREAD_SAFETY = "enable_lwip_check_thread_safety" +CONF_DISABLE_LIBC_LOCKS_IN_IRAM = "disable_libc_locks_in_iram" def _validate_idf_component(config: ConfigType) -> ConfigType: @@ -575,9 +557,14 @@ def _validate_idf_component(config: ConfigType) -> ConfigType: return config -ESP_IDF_FRAMEWORK_SCHEMA = cv.All( +FRAMEWORK_ESP_IDF = "esp-idf" +FRAMEWORK_ARDUINO = "arduino" +FRAMEWORK_SCHEMA = cv.All( cv.Schema( { + cv.Optional(CONF_TYPE, default=FRAMEWORK_ARDUINO): cv.one_of( + FRAMEWORK_ESP_IDF, FRAMEWORK_ARDUINO + ), cv.Optional(CONF_VERSION, default="recommended"): cv.string_strict, cv.Optional(CONF_RELEASE): cv.string_strict, cv.Optional(CONF_SOURCE): cv.string_strict, @@ -585,6 +572,9 @@ ESP_IDF_FRAMEWORK_SCHEMA = cv.All( cv.Optional(CONF_SDKCONFIG_OPTIONS, default={}): { cv.string_strict: cv.string_strict }, + cv.Optional(CONF_LOG_LEVEL, default="ERROR"): cv.one_of( + *LOG_LEVELS_IDF, upper=True + ), cv.Optional(CONF_ADVANCED, default={}): cv.Schema( { cv.Optional(CONF_ASSERTION_LEVEL): cv.one_of( @@ -611,6 +601,16 @@ ESP_IDF_FRAMEWORK_SCHEMA = cv.All( cv.Optional( CONF_ENABLE_LWIP_BRIDGE_INTERFACE, default=False ): cv.boolean, + cv.Optional( + CONF_ENABLE_LWIP_TCPIP_CORE_LOCKING, default=True + ): cv.boolean, + cv.Optional( + CONF_ENABLE_LWIP_CHECK_THREAD_SAFETY, default=True + ): cv.boolean, + cv.Optional( + CONF_DISABLE_LIBC_LOCKS_IN_IRAM, default=True + ): cv.boolean, + cv.Optional(CONF_EXECUTE_FROM_PSRAM): cv.boolean, } ), cv.Optional(CONF_COMPONENTS, default=[]): cv.ensure_list( @@ -631,37 +631,85 @@ ESP_IDF_FRAMEWORK_SCHEMA = cv.All( ), } ), - _esp_idf_check_versions, + _check_versions, ) +class _FrameworkMigrationWarning: + shown = False + + +def _show_framework_migration_message(name: str, variant: str) -> None: + """Show a friendly message about framework migration when defaulting to Arduino.""" + if _FrameworkMigrationWarning.shown: + return + _FrameworkMigrationWarning.shown = True + + from esphome.log import AnsiFore, color + + message = ( + color( + AnsiFore.BOLD_CYAN, + f"💡 IMPORTANT: {name} doesn't have a framework specified!", + ) + + "\n\n" + + f"Currently, {variant} defaults to the Arduino framework.\n" + + color(AnsiFore.YELLOW, "This will change to ESP-IDF in ESPHome 2026.1.0.\n") + + "\n" + + "Note: Newer ESP32 variants (C6, H2, P4, etc.) already use ESP-IDF by default.\n" + + "\n" + + "Why change? ESP-IDF offers:\n" + + color(AnsiFore.GREEN, " ✨ Up to 40% smaller binaries\n") + + color(AnsiFore.GREEN, " 🚀 Better performance and optimization\n") + + color(AnsiFore.GREEN, " ⚡ 2-3x faster compile times\n") + + color(AnsiFore.GREEN, " 📦 Custom-built firmware for your exact needs\n") + + color( + AnsiFore.GREEN, + " 🔧 Active development and testing by ESPHome developers\n", + ) + + "\n" + + "Trade-offs:\n" + + color(AnsiFore.YELLOW, " 🔄 Some components need migration\n") + + "\n" + + "What should I do?\n" + + color(AnsiFore.CYAN, " Option 1") + + ": Migrate to ESP-IDF (recommended)\n" + + " Add this to your YAML under 'esp32:':\n" + + color(AnsiFore.WHITE, " framework:\n") + + color(AnsiFore.WHITE, " type: esp-idf\n") + + "\n" + + color(AnsiFore.CYAN, " Option 2") + + ": Keep using Arduino (still supported)\n" + + " Add this to your YAML under 'esp32:':\n" + + color(AnsiFore.WHITE, " framework:\n") + + color(AnsiFore.WHITE, " type: arduino\n") + + "\n" + + "Need help? Check out the migration guide:\n" + + color( + AnsiFore.BLUE, + "https://esphome.io/guides/esp32_arduino_to_idf.html", + ) + ) + _LOGGER.warning(message) + + def _set_default_framework(config): if CONF_FRAMEWORK not in config: config = config.copy() variant = config[CONF_VARIANT] + config[CONF_FRAMEWORK] = FRAMEWORK_SCHEMA({}) if variant in ARDUINO_ALLOWED_VARIANTS: - config[CONF_FRAMEWORK] = ARDUINO_FRAMEWORK_SCHEMA({}) config[CONF_FRAMEWORK][CONF_TYPE] = FRAMEWORK_ARDUINO + _show_framework_migration_message( + config.get(CONF_NAME, "This device"), variant + ) else: - config[CONF_FRAMEWORK] = ESP_IDF_FRAMEWORK_SCHEMA({}) config[CONF_FRAMEWORK][CONF_TYPE] = FRAMEWORK_ESP_IDF return config -FRAMEWORK_ESP_IDF = "esp-idf" -FRAMEWORK_ARDUINO = "arduino" -FRAMEWORK_SCHEMA = cv.typed_schema( - { - FRAMEWORK_ESP_IDF: ESP_IDF_FRAMEWORK_SCHEMA, - FRAMEWORK_ARDUINO: ARDUINO_FRAMEWORK_SCHEMA, - }, - lower=True, - space="-", -) - - FLASH_SIZES = [ "2MB", "4MB", @@ -676,7 +724,7 @@ CONF_PARTITIONS = "partitions" CONFIG_SCHEMA = cv.All( cv.Schema( { - cv.Required(CONF_BOARD): cv.string_strict, + cv.Optional(CONF_BOARD): cv.string_strict, cv.Optional(CONF_CPU_FREQUENCY): cv.one_of( *FULL_CPU_FREQUENCIES, upper=True ), @@ -691,6 +739,7 @@ CONFIG_SCHEMA = cv.All( _detect_variant, _set_default_framework, set_core_data, + cv.has_at_least_one_key(CONF_BOARD, CONF_VARIANT), ) @@ -703,8 +752,10 @@ async def to_code(config): cg.set_cpp_standard("gnu++20") cg.add_build_flag("-DUSE_ESP32") cg.add_define("ESPHOME_BOARD", config[CONF_BOARD]) - cg.add_build_flag(f"-DUSE_ESP32_VARIANT_{config[CONF_VARIANT]}") - cg.add_define("ESPHOME_VARIANT", VARIANT_FRIENDLY[config[CONF_VARIANT]]) + variant = config[CONF_VARIANT] + cg.add_build_flag(f"-DUSE_ESP32_VARIANT_{variant}") + cg.add_define("ESPHOME_VARIANT", VARIANT_FRIENDLY[variant]) + cg.add_define(ThreadModel.MULTI_ATOMICS) cg.add_platformio_option("lib_ldf_mode", "off") cg.add_platformio_option("lib_compat_mode", "strict") @@ -713,134 +764,167 @@ async def to_code(config): conf = config[CONF_FRAMEWORK] cg.add_platformio_option("platform", conf[CONF_PLATFORM_VERSION]) + if CONF_SOURCE in conf: + cg.add_platformio_option("platform_packages", [conf[CONF_SOURCE]]) if conf[CONF_ADVANCED][CONF_IGNORE_EFUSE_CUSTOM_MAC]: cg.add_define("USE_ESP32_IGNORE_EFUSE_CUSTOM_MAC") + for clean_var in ("IDF_PATH", "IDF_TOOLS_PATH"): + os.environ.pop(clean_var, None) + add_extra_script( "post", "post_build.py", - os.path.join(os.path.dirname(__file__), "post_build.py.script"), + Path(__file__).parent / "post_build.py.script", ) - freq = config[CONF_CPU_FREQUENCY][:-3] if conf[CONF_TYPE] == FRAMEWORK_ESP_IDF: cg.add_platformio_option("framework", "espidf") cg.add_build_flag("-DUSE_ESP_IDF") cg.add_build_flag("-DUSE_ESP32_FRAMEWORK_ESP_IDF") - cg.add_build_flag("-Wno-nonnull-compare") - - cg.add_platformio_option("platform_packages", [conf[CONF_SOURCE]]) - - # platformio/toolchain-esp32ulp does not support linux_aarch64 yet and has not been updated for over 2 years - # This is espressif's own published version which is more up to date. - cg.add_platformio_option( - "platform_packages", ["espressif/toolchain-esp32ulp@2.35.0-20220830"] - ) - add_idf_sdkconfig_option( - f"CONFIG_ESPTOOLPY_FLASHSIZE_{config[CONF_FLASH_SIZE]}", True - ) - add_idf_sdkconfig_option("CONFIG_PARTITION_TABLE_SINGLE_APP", False) - add_idf_sdkconfig_option("CONFIG_PARTITION_TABLE_CUSTOM", True) - add_idf_sdkconfig_option( - "CONFIG_PARTITION_TABLE_CUSTOM_FILENAME", "partitions.csv" - ) - - # Increase freertos tick speed from 100Hz to 1kHz so that delay() resolution is 1ms - add_idf_sdkconfig_option("CONFIG_FREERTOS_HZ", 1000) - - # Setup watchdog - add_idf_sdkconfig_option("CONFIG_ESP_TASK_WDT", True) - add_idf_sdkconfig_option("CONFIG_ESP_TASK_WDT_PANIC", True) - add_idf_sdkconfig_option("CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU0", False) - add_idf_sdkconfig_option("CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU1", False) - - # Disable dynamic log level control to save memory - add_idf_sdkconfig_option("CONFIG_LOG_DYNAMIC_LEVEL_CONTROL", False) - - # Set default CPU frequency - add_idf_sdkconfig_option(f"CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ_{freq}", True) - - # Apply LWIP optimization settings - advanced = conf[CONF_ADVANCED] - # DHCP server: only disable if explicitly set to false - # WiFi component handles its own optimization when AP mode is not used - if ( - CONF_ENABLE_LWIP_DHCP_SERVER in advanced - and not advanced[CONF_ENABLE_LWIP_DHCP_SERVER] - ): - add_idf_sdkconfig_option("CONFIG_LWIP_DHCPS", False) - if not advanced.get(CONF_ENABLE_LWIP_MDNS_QUERIES, True): - add_idf_sdkconfig_option("CONFIG_LWIP_DNS_SUPPORT_MDNS_QUERIES", False) - if not advanced.get(CONF_ENABLE_LWIP_BRIDGE_INTERFACE, False): - add_idf_sdkconfig_option("CONFIG_LWIP_BRIDGEIF_MAX_PORTS", 0) - - cg.add_platformio_option("board_build.partitions", "partitions.csv") - if CONF_PARTITIONS in config: - add_extra_build_file( - "partitions.csv", CORE.relative_config_path(config[CONF_PARTITIONS]) - ) - - if assertion_level := advanced.get(CONF_ASSERTION_LEVEL): - for key, flag in ASSERTION_LEVELS.items(): - add_idf_sdkconfig_option(flag, assertion_level == key) - - add_idf_sdkconfig_option("CONFIG_COMPILER_OPTIMIZATION_DEFAULT", False) - compiler_optimization = advanced.get(CONF_COMPILER_OPTIMIZATION) - for key, flag in COMPILER_OPTIMIZATIONS.items(): - add_idf_sdkconfig_option(flag, compiler_optimization == key) - - add_idf_sdkconfig_option( - "CONFIG_LWIP_ESP_LWIP_ASSERT", - conf[CONF_ADVANCED][CONF_ENABLE_LWIP_ASSERT], - ) - - if advanced.get(CONF_IGNORE_EFUSE_MAC_CRC): - add_idf_sdkconfig_option("CONFIG_ESP_MAC_IGNORE_MAC_CRC_ERROR", True) - add_idf_sdkconfig_option( - "CONFIG_ESP_PHY_CALIBRATION_AND_DATA_STORAGE", False - ) - if advanced.get(CONF_ENABLE_IDF_EXPERIMENTAL_FEATURES): - _LOGGER.warning( - "Using experimental features in ESP-IDF may result in unexpected failures." - ) - add_idf_sdkconfig_option("CONFIG_IDF_EXPERIMENTAL_FEATURES", True) - - cg.add_define( - "USE_ESP_IDF_VERSION_CODE", - cg.RawExpression( - f"VERSION_CODE({framework_ver.major}, {framework_ver.minor}, {framework_ver.patch})" - ), - ) - - for name, value in conf[CONF_SDKCONFIG_OPTIONS].items(): - add_idf_sdkconfig_option(name, RawSdkconfigValue(value)) - - for component in conf[CONF_COMPONENTS]: - add_idf_component( - name=component[CONF_NAME], - repo=component.get(CONF_SOURCE), - ref=component.get(CONF_REF), - path=component.get(CONF_PATH), - ) - elif conf[CONF_TYPE] == FRAMEWORK_ARDUINO: - cg.add_platformio_option("framework", "arduino") + else: + cg.add_platformio_option("framework", "arduino, espidf") cg.add_build_flag("-DUSE_ARDUINO") cg.add_build_flag("-DUSE_ESP32_FRAMEWORK_ARDUINO") - cg.add_platformio_option("platform_packages", [conf[CONF_SOURCE]]) - - if CONF_PARTITIONS in config: - cg.add_platformio_option("board_build.partitions", config[CONF_PARTITIONS]) - else: - cg.add_platformio_option("board_build.partitions", "partitions.csv") - + cg.add_platformio_option( + "board_build.embed_txtfiles", + [ + "managed_components/espressif__esp_insights/server_certs/https_server.crt", + "managed_components/espressif__esp_rainmaker/server_certs/rmaker_mqtt_server.crt", + "managed_components/espressif__esp_rainmaker/server_certs/rmaker_claim_service_server.crt", + "managed_components/espressif__esp_rainmaker/server_certs/rmaker_ota_server.crt", + ], + ) cg.add_define( "USE_ARDUINO_VERSION_CODE", cg.RawExpression( f"VERSION_CODE({framework_ver.major}, {framework_ver.minor}, {framework_ver.patch})" ), ) - cg.add(RawExpression(f"setCpuFrequencyMhz({freq})")) + add_idf_sdkconfig_option("CONFIG_AUTOSTART_ARDUINO", True) + add_idf_sdkconfig_option("CONFIG_MBEDTLS_PSK_MODES", True) + add_idf_sdkconfig_option("CONFIG_MBEDTLS_CERTIFICATE_BUNDLE", True) + + cg.add_build_flag("-Wno-nonnull-compare") + + add_idf_sdkconfig_option(f"CONFIG_IDF_TARGET_{variant}", True) + add_idf_sdkconfig_option( + f"CONFIG_ESPTOOLPY_FLASHSIZE_{config[CONF_FLASH_SIZE]}", True + ) + add_idf_sdkconfig_option("CONFIG_PARTITION_TABLE_SINGLE_APP", False) + add_idf_sdkconfig_option("CONFIG_PARTITION_TABLE_CUSTOM", True) + add_idf_sdkconfig_option("CONFIG_PARTITION_TABLE_CUSTOM_FILENAME", "partitions.csv") + + # Increase freertos tick speed from 100Hz to 1kHz so that delay() resolution is 1ms + add_idf_sdkconfig_option("CONFIG_FREERTOS_HZ", 1000) + + # Setup watchdog + add_idf_sdkconfig_option("CONFIG_ESP_TASK_WDT", True) + add_idf_sdkconfig_option("CONFIG_ESP_TASK_WDT_PANIC", True) + add_idf_sdkconfig_option("CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU0", False) + add_idf_sdkconfig_option("CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU1", False) + + # Disable dynamic log level control to save memory + add_idf_sdkconfig_option("CONFIG_LOG_DYNAMIC_LEVEL_CONTROL", False) + + # Reduce PHY TX power in the event of a brownout + add_idf_sdkconfig_option("CONFIG_ESP_PHY_REDUCE_TX_POWER", True) + + # Set default CPU frequency + add_idf_sdkconfig_option( + f"CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ_{config[CONF_CPU_FREQUENCY][:-3]}", True + ) + + # Apply LWIP optimization settings + advanced = conf[CONF_ADVANCED] + # DHCP server: only disable if explicitly set to false + # WiFi component handles its own optimization when AP mode is not used + # When using Arduino with Ethernet, DHCP server functions must be available + # for the Network library to compile, even if not actively used + if ( + CONF_ENABLE_LWIP_DHCP_SERVER in advanced + and not advanced[CONF_ENABLE_LWIP_DHCP_SERVER] + and not ( + conf[CONF_TYPE] == FRAMEWORK_ARDUINO + and "ethernet" in CORE.loaded_integrations + ) + ): + add_idf_sdkconfig_option("CONFIG_LWIP_DHCPS", False) + if not advanced.get(CONF_ENABLE_LWIP_MDNS_QUERIES, True): + add_idf_sdkconfig_option("CONFIG_LWIP_DNS_SUPPORT_MDNS_QUERIES", False) + if not advanced.get(CONF_ENABLE_LWIP_BRIDGE_INTERFACE, False): + add_idf_sdkconfig_option("CONFIG_LWIP_BRIDGEIF_MAX_PORTS", 0) + if advanced.get(CONF_EXECUTE_FROM_PSRAM, False): + add_idf_sdkconfig_option("CONFIG_SPIRAM_FETCH_INSTRUCTIONS", True) + add_idf_sdkconfig_option("CONFIG_SPIRAM_RODATA", True) + + # Apply LWIP core locking for better socket performance + # This is already enabled by default in Arduino framework, where it provides + # significant performance benefits. Our benchmarks show socket operations are + # 24-200% faster with core locking enabled: + # - select() on 4 sockets: ~190μs (Arduino/core locking) vs ~235μs (ESP-IDF default) + # - Up to 200% slower under load when all operations queue through tcpip_thread + # Enabling this makes ESP-IDF socket performance match Arduino framework. + if advanced.get(CONF_ENABLE_LWIP_TCPIP_CORE_LOCKING, True): + add_idf_sdkconfig_option("CONFIG_LWIP_TCPIP_CORE_LOCKING", True) + if advanced.get(CONF_ENABLE_LWIP_CHECK_THREAD_SAFETY, True): + add_idf_sdkconfig_option("CONFIG_LWIP_CHECK_THREAD_SAFETY", True) + + # Disable placing libc locks in IRAM to save RAM + # This is safe for ESPHome since no IRAM ISRs (interrupts that run while cache is disabled) + # use libc lock APIs. Saves approximately 1.3KB (1,356 bytes) of IRAM. + if advanced.get(CONF_DISABLE_LIBC_LOCKS_IN_IRAM, True): + add_idf_sdkconfig_option("CONFIG_LIBC_LOCKS_PLACE_IN_IRAM", False) + + cg.add_platformio_option("board_build.partitions", "partitions.csv") + if CONF_PARTITIONS in config: + add_extra_build_file( + "partitions.csv", CORE.relative_config_path(config[CONF_PARTITIONS]) + ) + + if assertion_level := advanced.get(CONF_ASSERTION_LEVEL): + for key, flag in ASSERTION_LEVELS.items(): + add_idf_sdkconfig_option(flag, assertion_level == key) + + add_idf_sdkconfig_option("CONFIG_COMPILER_OPTIMIZATION_DEFAULT", False) + compiler_optimization = advanced.get(CONF_COMPILER_OPTIMIZATION) + for key, flag in COMPILER_OPTIMIZATIONS.items(): + add_idf_sdkconfig_option(flag, compiler_optimization == key) + + add_idf_sdkconfig_option( + "CONFIG_LWIP_ESP_LWIP_ASSERT", + conf[CONF_ADVANCED][CONF_ENABLE_LWIP_ASSERT], + ) + + if advanced.get(CONF_IGNORE_EFUSE_MAC_CRC): + add_idf_sdkconfig_option("CONFIG_ESP_MAC_IGNORE_MAC_CRC_ERROR", True) + add_idf_sdkconfig_option("CONFIG_ESP_PHY_CALIBRATION_AND_DATA_STORAGE", False) + if advanced.get(CONF_ENABLE_IDF_EXPERIMENTAL_FEATURES): + _LOGGER.warning( + "Using experimental features in ESP-IDF may result in unexpected failures." + ) + add_idf_sdkconfig_option("CONFIG_IDF_EXPERIMENTAL_FEATURES", True) + + cg.add_define( + "USE_ESP_IDF_VERSION_CODE", + cg.RawExpression( + f"VERSION_CODE({framework_ver.major}, {framework_ver.minor}, {framework_ver.patch})" + ), + ) + + add_idf_sdkconfig_option(f"CONFIG_LOG_DEFAULT_LEVEL_{conf[CONF_LOG_LEVEL]}", True) + + for name, value in conf[CONF_SDKCONFIG_OPTIONS].items(): + add_idf_sdkconfig_option(name, RawSdkconfigValue(value)) + + for component in conf[CONF_COMPONENTS]: + add_idf_component( + name=component[CONF_NAME], + repo=component.get(CONF_SOURCE), + ref=component.get(CONF_REF), + path=component.get(CONF_PATH), + ) APP_PARTITION_SIZES = { @@ -862,7 +946,7 @@ def get_arduino_partition_csv(flash_size): eeprom_partition_start = app1_partition_start + app_partition_size spiffs_partition_start = eeprom_partition_start + eeprom_partition_size - partition_csv = f"""\ + return f"""\ nvs, data, nvs, 0x9000, 0x5000, otadata, data, ota, 0xE000, 0x2000, app0, app, ota_0, 0x{app0_partition_start:X}, 0x{app_partition_size:X}, @@ -870,20 +954,18 @@ app1, app, ota_1, 0x{app1_partition_start:X}, 0x{app_partition_size:X}, eeprom, data, 0x99, 0x{eeprom_partition_start:X}, 0x{eeprom_partition_size:X}, spiffs, data, spiffs, 0x{spiffs_partition_start:X}, 0x{spiffs_partition_size:X} """ - return partition_csv def get_idf_partition_csv(flash_size): app_partition_size = APP_PARTITION_SIZES[flash_size] - partition_csv = f"""\ + return f"""\ otadata, data, ota, , 0x2000, phy_init, data, phy, , 0x1000, app0, app, ota_0, , 0x{app_partition_size:X}, app1, app, ota_1, , 0x{app_partition_size:X}, nvs, data, nvs, , 0x6D000, """ - return partition_csv def _format_sdkconfig_val(value: SdkconfigValueType) -> str: @@ -916,13 +998,14 @@ def _write_sdkconfig(): ) + "\n" ) + if write_file_if_changed(internal_path, contents): # internal changed, update real one write_file_if_changed(sdk_path, contents) def _write_idf_component_yml(): - yml_path = Path(CORE.relative_build_path("src/idf_component.yml")) + yml_path = CORE.relative_build_path("src/idf_component.yml") if CORE.data[KEY_ESP32][KEY_COMPONENTS]: components: dict = CORE.data[KEY_ESP32][KEY_COMPONENTS] dependencies = {} @@ -938,47 +1021,50 @@ def _write_idf_component_yml(): contents = yaml_util.dump({"dependencies": dependencies}) else: contents = "" - write_file_if_changed(yml_path, contents) + if write_file_if_changed(yml_path, contents): + dependencies_lock = CORE.relative_build_path("dependencies.lock") + if dependencies_lock.is_file(): + dependencies_lock.unlink() + clean_cmake_cache() # Called by writer.py def copy_files(): - if CORE.using_arduino: - if "partitions.csv" not in CORE.data[KEY_ESP32][KEY_EXTRA_BUILD_FILES]: + _write_sdkconfig() + _write_idf_component_yml() + + if "partitions.csv" not in CORE.data[KEY_ESP32][KEY_EXTRA_BUILD_FILES]: + if CORE.using_arduino: write_file_if_changed( CORE.relative_build_path("partitions.csv"), get_arduino_partition_csv( CORE.platformio_options.get("board_upload.flash_size") ), ) - if CORE.using_esp_idf: - _write_sdkconfig() - _write_idf_component_yml() - if "partitions.csv" not in CORE.data[KEY_ESP32][KEY_EXTRA_BUILD_FILES]: + else: write_file_if_changed( CORE.relative_build_path("partitions.csv"), get_idf_partition_csv( CORE.platformio_options.get("board_upload.flash_size") ), ) - # IDF build scripts look for version string to put in the build. - # However, if the build path does not have an initialized git repo, - # and no version.txt file exists, the CMake script fails for some setups. - # Fix by manually pasting a version.txt file, containing the ESPHome version - write_file_if_changed( - CORE.relative_build_path("version.txt"), - __version__, - ) + # IDF build scripts look for version string to put in the build. + # However, if the build path does not have an initialized git repo, + # and no version.txt file exists, the CMake script fails for some setups. + # Fix by manually pasting a version.txt file, containing the ESPHome version + write_file_if_changed( + CORE.relative_build_path("version.txt"), + __version__, + ) - for _, file in CORE.data[KEY_ESP32][KEY_EXTRA_BUILD_FILES].items(): - if file[KEY_PATH].startswith("http"): + for file in CORE.data[KEY_ESP32][KEY_EXTRA_BUILD_FILES].values(): + name: str = file[KEY_NAME] + path: Path = file[KEY_PATH] + if str(path).startswith("http"): import requests - mkdir_p(CORE.relative_build_path(os.path.dirname(file[KEY_NAME]))) - with open(CORE.relative_build_path(file[KEY_NAME]), "wb") as f: - f.write(requests.get(file[KEY_PATH], timeout=30).content) + CORE.relative_build_path(name).parent.mkdir(parents=True, exist_ok=True) + content = requests.get(path, timeout=30).content + CORE.relative_build_path(name).write_bytes(content) else: - copy_file_if_changed( - file[KEY_PATH], - CORE.relative_build_path(file[KEY_NAME]), - ) + copy_file_if_changed(path, CORE.relative_build_path(name)) diff --git a/esphome/components/esp32/boards.py b/esphome/components/esp32/boards.py index 68fee48830..cbb314650a 100644 --- a/esphome/components/esp32/boards.py +++ b/esphome/components/esp32/boards.py @@ -2,13 +2,30 @@ from .const import ( VARIANT_ESP32, VARIANT_ESP32C2, VARIANT_ESP32C3, + VARIANT_ESP32C5, VARIANT_ESP32C6, VARIANT_ESP32H2, VARIANT_ESP32P4, VARIANT_ESP32S2, VARIANT_ESP32S3, + VARIANTS, ) +STANDARD_BOARDS = { + VARIANT_ESP32: "esp32dev", + VARIANT_ESP32C2: "esp32-c2-devkitm-1", + VARIANT_ESP32C3: "esp32-c3-devkitm-1", + VARIANT_ESP32C5: "esp32-c5-devkitc-1", + VARIANT_ESP32C6: "esp32-c6-devkitm-1", + VARIANT_ESP32H2: "esp32-h2-devkitm-1", + VARIANT_ESP32P4: "esp32-p4-evboard", + VARIANT_ESP32S2: "esp32-s2-kaluga-1", + VARIANT_ESP32S3: "esp32-s3-devkitc-1", +} + +# Make sure not missed here if a new variant added. +assert all(v in STANDARD_BOARDS for v in VARIANTS) + ESP32_BASE_PINS = { "TX": 1, "RX": 3, @@ -1487,6 +1504,10 @@ BOARDS = { "name": "BPI-Bit", "variant": VARIANT_ESP32, }, + "bpi-centi-s3": { + "name": "BPI-Centi-S3", + "variant": VARIANT_ESP32S3, + }, "bpi_leaf_s3": { "name": "BPI-Leaf-S3", "variant": VARIANT_ESP32S3, @@ -1543,6 +1564,10 @@ BOARDS = { "name": "DFRobot Beetle ESP32-C3", "variant": VARIANT_ESP32C3, }, + "dfrobot_firebeetle2_esp32c6": { + "name": "DFRobot FireBeetle 2 ESP32-C6", + "variant": VARIANT_ESP32C6, + }, "dfrobot_firebeetle2_esp32e": { "name": "DFRobot Firebeetle 2 ESP32-E", "variant": VARIANT_ESP32, @@ -1583,6 +1608,22 @@ BOARDS = { "name": "Ai-Thinker ESP-C3-M1-I-Kit", "variant": VARIANT_ESP32C3, }, + "esp32-c5-devkitc-1": { + "name": "Espressif ESP32-C5-DevKitC-1 4MB no PSRAM", + "variant": VARIANT_ESP32C5, + }, + "esp32-c5-devkitc1-n16r4": { + "name": "Espressif ESP32-C5-DevKitC-1 N16R4 (16 MB Flash Quad, 4 MB PSRAM Quad)", + "variant": VARIANT_ESP32C5, + }, + "esp32-c5-devkitc1-n4": { + "name": "Espressif ESP32-C5-DevKitC-1 N4 (4MB no PSRAM)", + "variant": VARIANT_ESP32C5, + }, + "esp32-c5-devkitc1-n8r4": { + "name": "Espressif ESP32-C5-DevKitC-1 N8R4 (8 MB Flash Quad, 4 MB PSRAM Quad)", + "variant": VARIANT_ESP32C5, + }, "esp32-c6-devkitc-1": { "name": "Espressif ESP32-C6-DevKitC-1", "variant": VARIANT_ESP32C6, @@ -1647,10 +1688,46 @@ BOARDS = { "name": "Espressif ESP32-S3-DevKitC-1-N8 (8 MB QD, No PSRAM)", "variant": VARIANT_ESP32S3, }, + "esp32-s3-devkitc-1-n32r8v": { + "name": "Espressif ESP32-S3-DevKitC-1-N32R8V (32 MB Flash Octal, 8 MB PSRAM Octal)", + "variant": VARIANT_ESP32S3, + }, + "esp32-s3-devkitc1-n16r16": { + "name": "Espressif ESP32-S3-DevKitC-1-N16R16V (16 MB Flash Quad, 16 MB PSRAM Octal)", + "variant": VARIANT_ESP32S3, + }, + "esp32-s3-devkitc1-n16r2": { + "name": "Espressif ESP32-S3-DevKitC-1-N16R2 (16 MB Flash Quad, 2 MB PSRAM Quad)", + "variant": VARIANT_ESP32S3, + }, + "esp32-s3-devkitc1-n16r8": { + "name": "Espressif ESP32-S3-DevKitC-1-N16R8V (16 MB Flash Quad, 8 MB PSRAM Octal)", + "variant": VARIANT_ESP32S3, + }, + "esp32-s3-devkitc1-n4r2": { + "name": "Espressif ESP32-S3-DevKitC-1-N4R2 (4 MB Flash Quad, 2 MB PSRAM Quad)", + "variant": VARIANT_ESP32S3, + }, + "esp32-s3-devkitc1-n4r8": { + "name": "Espressif ESP32-S3-DevKitC-1-N4R8 (4 MB Flash Quad, 8 MB PSRAM Octal)", + "variant": VARIANT_ESP32S3, + }, + "esp32-s3-devkitc1-n8r2": { + "name": "Espressif ESP32-S3-DevKitC-1-N8R2 (8 MB Flash Quad, 2 MB PSRAM quad)", + "variant": VARIANT_ESP32S3, + }, + "esp32-s3-devkitc1-n8r8": { + "name": "Espressif ESP32-S3-DevKitC-1-N8R8 (8 MB Flash Quad, 8 MB PSRAM Octal)", + "variant": VARIANT_ESP32S3, + }, "esp32-s3-devkitm-1": { "name": "Espressif ESP32-S3-DevKitM-1", "variant": VARIANT_ESP32S3, }, + "esp32-s3-fh4r2": { + "name": "Espressif ESP32-S3-FH4R2 (4 MB QD, 2MB PSRAM)", + "variant": VARIANT_ESP32S3, + }, "esp32-solo1": { "name": "Espressif Generic ESP32-solo1 4M Flash", "variant": VARIANT_ESP32, @@ -1747,6 +1824,10 @@ BOARDS = { "name": "Franzininho WiFi MSC", "variant": VARIANT_ESP32S2, }, + "freenove-esp32-s3-n8r8": { + "name": "Freenove ESP32-S3 WROOM N8R8 (8MB Flash / 8MB PSRAM)", + "variant": VARIANT_ESP32S3, + }, "freenove_esp32_s3_wroom": { "name": "Freenove ESP32-S3 WROOM N8R8 (8MB Flash / 8MB PSRAM)", "variant": VARIANT_ESP32S3, @@ -1947,6 +2028,10 @@ BOARDS = { "name": "M5Stack AtomS3", "variant": VARIANT_ESP32S3, }, + "m5stack-atoms3u": { + "name": "M5Stack AtomS3U", + "variant": VARIANT_ESP32S3, + }, "m5stack-core-esp32": { "name": "M5Stack Core ESP32", "variant": VARIANT_ESP32, @@ -1983,6 +2068,10 @@ BOARDS = { "name": "M5Stack Station", "variant": VARIANT_ESP32, }, + "m5stack-tab5-p4": { + "name": "M5STACK Tab5 esp32-p4 Board", + "variant": VARIANT_ESP32P4, + }, "m5stack-timer-cam": { "name": "M5Stack Timer CAM", "variant": VARIANT_ESP32, @@ -2067,6 +2156,10 @@ BOARDS = { "name": "Ai-Thinker NodeMCU-32S2 (ESP-12K)", "variant": VARIANT_ESP32S2, }, + "nologo_esp32c3_super_mini": { + "name": "Nologo ESP32C3 SuperMini", + "variant": VARIANT_ESP32C3, + }, "nscreen-32": { "name": "YeaCreate NSCREEN-32", "variant": VARIANT_ESP32, @@ -2175,6 +2268,10 @@ BOARDS = { "name": "SparkFun LoRa Gateway 1-Channel", "variant": VARIANT_ESP32, }, + "sparkfun_pro_micro_esp32c3": { + "name": "SparkFun Pro Micro ESP32-C3", + "variant": VARIANT_ESP32C3, + }, "sparkfun_qwiic_pocket_esp32c6": { "name": "SparkFun ESP32-C6 Qwiic Pocket", "variant": VARIANT_ESP32C6, @@ -2239,6 +2336,14 @@ BOARDS = { "name": "Turta IoT Node", "variant": VARIANT_ESP32, }, + "um_bling": { + "name": "Unexpected Maker BLING!", + "variant": VARIANT_ESP32S3, + }, + "um_edges3_d": { + "name": "Unexpected Maker EDGES3[D]", + "variant": VARIANT_ESP32S3, + }, "um_feathers2": { "name": "Unexpected Maker FeatherS2", "variant": VARIANT_ESP32S2, @@ -2251,10 +2356,18 @@ BOARDS = { "name": "Unexpected Maker FeatherS3", "variant": VARIANT_ESP32S3, }, + "um_feathers3_neo": { + "name": "Unexpected Maker FeatherS3 Neo", + "variant": VARIANT_ESP32S3, + }, "um_nanos3": { "name": "Unexpected Maker NanoS3", "variant": VARIANT_ESP32S3, }, + "um_omgs3": { + "name": "Unexpected Maker OMGS3", + "variant": VARIANT_ESP32S3, + }, "um_pros3": { "name": "Unexpected Maker PROS3", "variant": VARIANT_ESP32S3, @@ -2263,6 +2376,14 @@ BOARDS = { "name": "Unexpected Maker RMP", "variant": VARIANT_ESP32S2, }, + "um_squixl": { + "name": "Unexpected Maker SQUiXL", + "variant": VARIANT_ESP32S3, + }, + "um_tinyc6": { + "name": "Unexpected Maker TinyC6", + "variant": VARIANT_ESP32C6, + }, "um_tinys2": { "name": "Unexpected Maker TinyS2", "variant": VARIANT_ESP32S2, @@ -2379,8 +2500,13 @@ BOARDS = { "name": "YelloByte YB-ESP32-S3-AMP (Rev.3)", "variant": VARIANT_ESP32S3, }, + "yb_esp32s3_drv": { + "name": "YelloByte YB-ESP32-S3-DRV", + "variant": VARIANT_ESP32S3, + }, "yb_esp32s3_eth": { "name": "YelloByte YB-ESP32-S3-ETH", "variant": VARIANT_ESP32S3, }, } +# DO NOT ADD ANYTHING BELOW THIS LINE diff --git a/esphome/components/esp32/gpio.cpp b/esphome/components/esp32/gpio.cpp index 27572063ca..a98245b889 100644 --- a/esphome/components/esp32/gpio.cpp +++ b/esphome/components/esp32/gpio.cpp @@ -54,13 +54,13 @@ struct ISRPinArg { ISRInternalGPIOPin ESP32InternalGPIOPin::to_isr() const { auto *arg = new ISRPinArg{}; // NOLINT(cppcoreguidelines-owning-memory) - arg->pin = this->pin_; + arg->pin = this->get_pin_num(); arg->flags = gpio::FLAG_NONE; - arg->inverted = inverted_; + arg->inverted = this->pin_flags_.inverted; #if defined(USE_ESP32_VARIANT_ESP32) - arg->use_rtc = rtc_gpio_is_valid_gpio(this->pin_); + arg->use_rtc = rtc_gpio_is_valid_gpio(this->get_pin_num()); if (arg->use_rtc) - arg->rtc_pin = rtc_io_number_get(this->pin_); + arg->rtc_pin = rtc_io_number_get(this->get_pin_num()); #endif return ISRInternalGPIOPin((void *) arg); } @@ -69,23 +69,23 @@ void ESP32InternalGPIOPin::attach_interrupt(void (*func)(void *), void *arg, gpi gpio_int_type_t idf_type = GPIO_INTR_ANYEDGE; switch (type) { case gpio::INTERRUPT_RISING_EDGE: - idf_type = inverted_ ? GPIO_INTR_NEGEDGE : GPIO_INTR_POSEDGE; + idf_type = this->pin_flags_.inverted ? GPIO_INTR_NEGEDGE : GPIO_INTR_POSEDGE; break; case gpio::INTERRUPT_FALLING_EDGE: - idf_type = inverted_ ? GPIO_INTR_POSEDGE : GPIO_INTR_NEGEDGE; + idf_type = this->pin_flags_.inverted ? GPIO_INTR_POSEDGE : GPIO_INTR_NEGEDGE; break; case gpio::INTERRUPT_ANY_EDGE: idf_type = GPIO_INTR_ANYEDGE; break; case gpio::INTERRUPT_LOW_LEVEL: - idf_type = inverted_ ? GPIO_INTR_HIGH_LEVEL : GPIO_INTR_LOW_LEVEL; + idf_type = this->pin_flags_.inverted ? GPIO_INTR_HIGH_LEVEL : GPIO_INTR_LOW_LEVEL; break; case gpio::INTERRUPT_HIGH_LEVEL: - idf_type = inverted_ ? GPIO_INTR_LOW_LEVEL : GPIO_INTR_HIGH_LEVEL; + idf_type = this->pin_flags_.inverted ? GPIO_INTR_LOW_LEVEL : GPIO_INTR_HIGH_LEVEL; break; } - gpio_set_intr_type(pin_, idf_type); - gpio_intr_enable(pin_); + gpio_set_intr_type(this->get_pin_num(), idf_type); + gpio_intr_enable(this->get_pin_num()); if (!isr_service_installed) { auto res = gpio_install_isr_service(ESP_INTR_FLAG_LEVEL3); if (res != ESP_OK) { @@ -94,31 +94,31 @@ void ESP32InternalGPIOPin::attach_interrupt(void (*func)(void *), void *arg, gpi } isr_service_installed = true; } - gpio_isr_handler_add(pin_, func, arg); + gpio_isr_handler_add(this->get_pin_num(), func, arg); } std::string ESP32InternalGPIOPin::dump_summary() const { char buffer[32]; - snprintf(buffer, sizeof(buffer), "GPIO%" PRIu32, static_cast(pin_)); + snprintf(buffer, sizeof(buffer), "GPIO%" PRIu32, static_cast(this->pin_)); return buffer; } void ESP32InternalGPIOPin::setup() { gpio_config_t conf{}; - conf.pin_bit_mask = 1ULL << static_cast(pin_); - conf.mode = flags_to_mode(flags_); - conf.pull_up_en = flags_ & gpio::FLAG_PULLUP ? GPIO_PULLUP_ENABLE : GPIO_PULLUP_DISABLE; - conf.pull_down_en = flags_ & gpio::FLAG_PULLDOWN ? GPIO_PULLDOWN_ENABLE : GPIO_PULLDOWN_DISABLE; + conf.pin_bit_mask = 1ULL << static_cast(this->pin_); + conf.mode = flags_to_mode(this->flags_); + conf.pull_up_en = this->flags_ & gpio::FLAG_PULLUP ? GPIO_PULLUP_ENABLE : GPIO_PULLUP_DISABLE; + conf.pull_down_en = this->flags_ & gpio::FLAG_PULLDOWN ? GPIO_PULLDOWN_ENABLE : GPIO_PULLDOWN_DISABLE; conf.intr_type = GPIO_INTR_DISABLE; gpio_config(&conf); - if (flags_ & gpio::FLAG_OUTPUT) { - gpio_set_drive_capability(pin_, drive_strength_); + if (this->flags_ & gpio::FLAG_OUTPUT) { + gpio_set_drive_capability(this->get_pin_num(), this->get_drive_strength()); } } void ESP32InternalGPIOPin::pin_mode(gpio::Flags flags) { // can't call gpio_config here because that logs in esp-idf which may cause issues - gpio_set_direction(pin_, flags_to_mode(flags)); + gpio_set_direction(this->get_pin_num(), flags_to_mode(flags)); gpio_pull_mode_t pull_mode = GPIO_FLOATING; if ((flags & gpio::FLAG_PULLUP) && (flags & gpio::FLAG_PULLDOWN)) { pull_mode = GPIO_PULLUP_PULLDOWN; @@ -127,12 +127,16 @@ void ESP32InternalGPIOPin::pin_mode(gpio::Flags flags) { } else if (flags & gpio::FLAG_PULLDOWN) { pull_mode = GPIO_PULLDOWN_ONLY; } - gpio_set_pull_mode(pin_, pull_mode); + gpio_set_pull_mode(this->get_pin_num(), pull_mode); } -bool ESP32InternalGPIOPin::digital_read() { return bool(gpio_get_level(pin_)) != inverted_; } -void ESP32InternalGPIOPin::digital_write(bool value) { gpio_set_level(pin_, value != inverted_ ? 1 : 0); } -void ESP32InternalGPIOPin::detach_interrupt() const { gpio_intr_disable(pin_); } +bool ESP32InternalGPIOPin::digital_read() { + return bool(gpio_get_level(this->get_pin_num())) != this->pin_flags_.inverted; +} +void ESP32InternalGPIOPin::digital_write(bool value) { + gpio_set_level(this->get_pin_num(), value != this->pin_flags_.inverted ? 1 : 0); +} +void ESP32InternalGPIOPin::detach_interrupt() const { gpio_intr_disable(this->get_pin_num()); } } // namespace esp32 diff --git a/esphome/components/esp32/gpio.h b/esphome/components/esp32/gpio.h index 0fefc1c058..565e276ea8 100644 --- a/esphome/components/esp32/gpio.h +++ b/esphome/components/esp32/gpio.h @@ -7,12 +7,18 @@ namespace esphome { namespace esp32 { +// Static assertions to ensure our bit-packed fields can hold the enum values +static_assert(GPIO_NUM_MAX <= 256, "gpio_num_t has too many values for uint8_t"); +static_assert(GPIO_DRIVE_CAP_MAX <= 4, "gpio_drive_cap_t has too many values for 2-bit field"); + class ESP32InternalGPIOPin : public InternalGPIOPin { public: - void set_pin(gpio_num_t pin) { pin_ = pin; } - void set_inverted(bool inverted) { inverted_ = inverted; } - void set_drive_strength(gpio_drive_cap_t drive_strength) { drive_strength_ = drive_strength; } - void set_flags(gpio::Flags flags) { flags_ = flags; } + void set_pin(gpio_num_t pin) { this->pin_ = static_cast(pin); } + void set_inverted(bool inverted) { this->pin_flags_.inverted = inverted; } + void set_drive_strength(gpio_drive_cap_t drive_strength) { + this->pin_flags_.drive_strength = static_cast(drive_strength); + } + void set_flags(gpio::Flags flags) { this->flags_ = flags; } void setup() override; void pin_mode(gpio::Flags flags) override; @@ -21,17 +27,26 @@ class ESP32InternalGPIOPin : public InternalGPIOPin { std::string dump_summary() const override; void detach_interrupt() const override; ISRInternalGPIOPin to_isr() const override; - uint8_t get_pin() const override { return (uint8_t) pin_; } - gpio::Flags get_flags() const override { return flags_; } - bool is_inverted() const override { return inverted_; } + uint8_t get_pin() const override { return this->pin_; } + gpio::Flags get_flags() const override { return this->flags_; } + bool is_inverted() const override { return this->pin_flags_.inverted; } + gpio_num_t get_pin_num() const { return static_cast(this->pin_); } + gpio_drive_cap_t get_drive_strength() const { return static_cast(this->pin_flags_.drive_strength); } protected: void attach_interrupt(void (*func)(void *), void *arg, gpio::InterruptType type) const override; - gpio_num_t pin_; - gpio_drive_cap_t drive_strength_; - gpio::Flags flags_; - bool inverted_; + // Memory layout: 8 bytes total on 32-bit systems + // - 3 bytes for members below + // - 1 byte padding for alignment + // - 4 bytes for vtable pointer + uint8_t pin_; // GPIO pin number (0-255, actual max ~54 on ESP32) + gpio::Flags flags_; // GPIO flags (1 byte) + struct PinFlags { + uint8_t inverted : 1; // Invert pin logic (1 bit) + uint8_t drive_strength : 2; // Drive strength 0-3 (2 bits) + uint8_t reserved : 5; // Reserved for future use (5 bits) + } pin_flags_; // Total: 1 byte // NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables) static bool isr_service_installed; }; diff --git a/esphome/components/esp32/gpio.py b/esphome/components/esp32/gpio.py index c35e5c2215..513f463d57 100644 --- a/esphome/components/esp32/gpio.py +++ b/esphome/components/esp32/gpio.py @@ -187,8 +187,7 @@ def validate_supports(value): "Open-drain only works with output mode", [CONF_MODE, CONF_OPEN_DRAIN] ) - value = _esp32_validations[variant].usage_validation(value) - return value + return _esp32_validations[variant].usage_validation(value) # https://docs.espressif.com/projects/esp-idf/en/v3.3.5/api-reference/peripherals/gpio.html#_CPPv416gpio_drive_cap_t diff --git a/esphome/components/esp32/gpio_esp32_h2.py b/esphome/components/esp32/gpio_esp32_h2.py index 7c3a658b17..f37297764b 100644 --- a/esphome/components/esp32/gpio_esp32_h2.py +++ b/esphome/components/esp32/gpio_esp32_h2.py @@ -2,6 +2,7 @@ import logging import esphome.config_validation as cv from esphome.const import CONF_INPUT, CONF_MODE, CONF_NUMBER +from esphome.pins import check_strapping_pin _ESP32H2_SPI_FLASH_PINS = {6, 7, 15, 16, 17, 18, 19, 20, 21} @@ -15,13 +16,6 @@ _LOGGER = logging.getLogger(__name__) def esp32_h2_validate_gpio_pin(value): if value < 0 or value > 27: raise cv.Invalid(f"Invalid pin number: {value} (must be 0-27)") - if value in _ESP32H2_STRAPPING_PINS: - _LOGGER.warning( - "GPIO%d is a Strapping PIN and should be avoided.\n" - "Attaching external pullup/down resistors to strapping pins can cause unexpected failures.\n" - "See https://esphome.io/guides/faq.html#why-am-i-getting-a-warning-about-strapping-pins", - value, - ) if value in _ESP32H2_SPI_FLASH_PINS: _LOGGER.warning( "GPIO%d is reserved for SPI Flash communication on some ESP32-H2 chip variants.\n" @@ -49,4 +43,5 @@ def esp32_h2_validate_supports(value): if is_input: # All ESP32 pins support input mode pass + check_strapping_pin(value, _ESP32H2_STRAPPING_PINS, _LOGGER) return value diff --git a/esphome/components/esp32/gpio_esp32_p4.py b/esphome/components/esp32/gpio_esp32_p4.py index 650d06e108..34d1b3139d 100644 --- a/esphome/components/esp32/gpio_esp32_p4.py +++ b/esphome/components/esp32/gpio_esp32_p4.py @@ -2,6 +2,7 @@ import logging import esphome.config_validation as cv from esphome.const import CONF_INPUT, CONF_MODE, CONF_NUMBER +from esphome.pins import check_strapping_pin _ESP32P4_USB_JTAG_PINS = {24, 25} @@ -13,13 +14,6 @@ _LOGGER = logging.getLogger(__name__) def esp32_p4_validate_gpio_pin(value): if value < 0 or value > 54: raise cv.Invalid(f"Invalid pin number: {value} (must be 0-54)") - if value in _ESP32P4_STRAPPING_PINS: - _LOGGER.warning( - "GPIO%d is a Strapping PIN and should be avoided.\n" - "Attaching external pullup/down resistors to strapping pins can cause unexpected failures.\n" - "See https://esphome.io/guides/faq.html#why-am-i-getting-a-warning-about-strapping-pins", - value, - ) if value in _ESP32P4_USB_JTAG_PINS: _LOGGER.warning( "GPIO%d is reserved for the USB-Serial-JTAG interface.\n" @@ -40,4 +34,5 @@ def esp32_p4_validate_supports(value): if is_input: # All ESP32 pins support input mode pass + check_strapping_pin(value, _ESP32P4_STRAPPING_PINS, _LOGGER) return value diff --git a/esphome/components/esp32/helpers.cpp b/esphome/components/esp32/helpers.cpp index 310e7bd94a..051b7ce162 100644 --- a/esphome/components/esp32/helpers.cpp +++ b/esphome/components/esp32/helpers.cpp @@ -1,4 +1,5 @@ #include "esphome/core/helpers.h" +#include "esphome/core/defines.h" #ifdef USE_ESP32 @@ -30,6 +31,45 @@ void Mutex::unlock() { xSemaphoreGive(this->handle_); } IRAM_ATTR InterruptLock::InterruptLock() { portDISABLE_INTERRUPTS(); } IRAM_ATTR InterruptLock::~InterruptLock() { portENABLE_INTERRUPTS(); } +#ifdef CONFIG_LWIP_TCPIP_CORE_LOCKING +#include "lwip/priv/tcpip_priv.h" +#endif + +LwIPLock::LwIPLock() { +#ifdef CONFIG_LWIP_TCPIP_CORE_LOCKING + // When CONFIG_LWIP_TCPIP_CORE_LOCKING is enabled, lwIP uses a global mutex to protect + // its internal state. Any thread can take this lock to safely access lwIP APIs. + // + // sys_thread_tcpip(LWIP_CORE_LOCK_QUERY_HOLDER) returns true if the current thread + // already holds the lwIP core lock. This prevents recursive locking attempts and + // allows nested LwIPLock instances to work correctly. + // + // If we don't already hold the lock, acquire it. This will block until the lock + // is available if another thread currently holds it. + if (!sys_thread_tcpip(LWIP_CORE_LOCK_QUERY_HOLDER)) { + LOCK_TCPIP_CORE(); + } +#endif +} + +LwIPLock::~LwIPLock() { +#ifdef CONFIG_LWIP_TCPIP_CORE_LOCKING + // Only release the lwIP core lock if this thread currently holds it. + // + // sys_thread_tcpip(LWIP_CORE_LOCK_QUERY_HOLDER) queries lwIP's internal lock + // ownership tracking. It returns true only if the current thread is registered + // as the lock holder. + // + // This check is essential because: + // 1. We may not have acquired the lock in the constructor (if we already held it) + // 2. The lock might have been released by other means between constructor and destructor + // 3. Calling UNLOCK_TCPIP_CORE() without holding the lock causes undefined behavior + if (sys_thread_tcpip(LWIP_CORE_LOCK_QUERY_HOLDER)) { + UNLOCK_TCPIP_CORE(); + } +#endif +} + void get_mac_address_raw(uint8_t *mac) { // NOLINT(readability-non-const-parameter) #if defined(CONFIG_SOC_IEEE802154_SUPPORTED) // When CONFIG_SOC_IEEE802154_SUPPORTED is defined, esp_efuse_mac_get_default diff --git a/esphome/components/esp32/post_build.py.script b/esphome/components/esp32/post_build.py.script index c181cf30b1..c995214232 100644 --- a/esphome/components/esp32/post_build.py.script +++ b/esphome/components/esp32/post_build.py.script @@ -1,77 +1,128 @@ -# Source https://github.com/letscontrolit/ESPEasy/pull/3845#issuecomment-1005864664 +Import("env") # noqa: F821 -# pylint: disable=E0602 -Import("env") # noqa - -import os -import shutil - -if os.environ.get("ESPHOME_USE_SUBPROCESS") is None: - try: - import esptool - except ImportError: - env.Execute("$PYTHONEXE -m pip install esptool") -else: - import subprocess -from SCons.Script import ARGUMENTS - -# Copy over the default sdkconfig. -from os import path - -if path.exists("./sdkconfig.defaults"): - os.makedirs(".temp", exist_ok=True) - shutil.copy("./sdkconfig.defaults", "./.temp/sdkconfig-esp32-idf") +import itertools # noqa: E402 +import json # noqa: E402 +import os # noqa: E402 +import pathlib # noqa: E402 +import shutil # noqa: E402 -def esp32_create_combined_bin(source, target, env): - verbose = bool(int(ARGUMENTS.get("PIOVERBOSE", "0"))) - if verbose: - print("Generating combined binary for serial flashing") - app_offset = 0x10000 +def merge_factory_bin(source, target, env): + """ + Merges all flash sections into a single .factory.bin using esptool. + Attempts multiple methods to detect image layout: flasher_args.json, FLASH_EXTRA_IMAGES, fallback guesses. + """ + firmware_name = os.path.basename(env.subst("$PROGNAME")) + ".bin" + build_dir = pathlib.Path(env.subst("$BUILD_DIR")) + firmware_path = build_dir / firmware_name + flash_size = env.BoardConfig().get("upload.flash_size", "4MB") + chip = env.BoardConfig().get("build.mcu", "esp32") - new_file_name = env.subst("$BUILD_DIR/${PROGNAME}.factory.bin") - sections = env.subst(env.get("FLASH_EXTRA_IMAGES")) - firmware_name = env.subst("$BUILD_DIR/${PROGNAME}.bin") - chip = env.get("BOARD_MCU") - flash_size = env.BoardConfig().get("upload.flash_size") + sections = [] + flasher_args_path = build_dir / "flasher_args.json" + + # 1. Try flasher_args.json + if flasher_args_path.exists(): + try: + with flasher_args_path.open() as f: + flash_data = json.load(f) + for addr, fname in sorted( + flash_data["flash_files"].items(), key=lambda kv: int(kv[0], 16) + ): + file_path = pathlib.Path(fname) + if file_path.exists(): + sections.append((addr, str(file_path))) + else: + print(f"Info: {file_path.name} not found - skipping") + except Exception as e: + print(f"Warning: Failed to parse flasher_args.json - {e}") + + # 2. Try FLASH_EXTRA_IMAGES if flasher_args.json failed or was empty + if not sections: + flash_images = env.get("FLASH_EXTRA_IMAGES") + if flash_images: + print("Using FLASH_EXTRA_IMAGES from PlatformIO environment") + # flatten any nested lists + flat = list( + itertools.chain.from_iterable( + x if isinstance(x, (list, tuple)) else [x] for x in flash_images + ) + ) + entries = [env.subst(x) for x in flat] + for i in range(0, len(entries) - 1, 2): + addr, fname = entries[i], entries[i + 1] + if isinstance(fname, (list, tuple)): + print( + f"Warning: Skipping malformed FLASH_EXTRA_IMAGES entry: {fname}" + ) + continue + file_path = pathlib.Path(str(fname)) + if file_path.exists(): + sections.append((addr, file_path)) + else: + print(f"Info: {file_path.name} not found — skipping") + if sections: + # Append main firmware to sections + sections.append(("0x10000", firmware_path)) + + # 3. Final fallback: guess standard image locations + if not sections: + print("Fallback: guessing legacy image paths") + guesses = [ + ("0x0", build_dir / "bootloader" / "bootloader.bin"), + ("0x8000", build_dir / "partition_table" / "partition-table.bin"), + ("0xe000", build_dir / "ota_data_initial.bin"), + ("0x10000", firmware_path), + ] + for addr, file_path in guesses: + if file_path.exists(): + sections.append((addr, file_path)) + else: + print(f"Info: {file_path.name} not found — skipping") + + # If no valid sections found, skip merge + if not sections: + print("No valid flash sections found — skipping .factory.bin creation.") + return + + output_path = firmware_path.with_suffix(".factory.bin") + python_exe = f'"{env.subst("$PYTHONEXE")}"' cmd = [ + python_exe, + "-m", + "esptool", "--chip", chip, - "merge_bin", - "-o", - new_file_name, - "--flash_size", + "merge-bin", + "--flash-size", flash_size, + "--output", + str(output_path), ] - if verbose: - print(" Offset | File") - for section in sections: - sect_adr, sect_file = section.split(" ", 1) - if verbose: - print(f" - {sect_adr} | {sect_file}") - cmd += [sect_adr, sect_file] + for addr, file_path in sections: + cmd += [addr, str(file_path)] - cmd += [hex(app_offset), firmware_name] + print(f"Merging binaries into {output_path}") + result = env.Execute( + env.VerboseAction(" ".join(cmd), "Merging binaries with esptool") + ) - if verbose: - print(f" - {hex(app_offset)} | {firmware_name}") - print() - print(f"Using esptool.py arguments: {' '.join(cmd)}") - print() - - if os.environ.get("ESPHOME_USE_SUBPROCESS") is None: - esptool.main(cmd) + if result == 0: + print(f"Successfully created {output_path}") else: - subprocess.run(["esptool.py", *cmd]) + print(f"Error: esptool merge-bin failed with code {result}") def esp32_copy_ota_bin(source, target, env): + """ + Copy the main firmware to a .ota.bin file for compatibility with ESPHome OTA tools. + """ firmware_name = env.subst("$BUILD_DIR/${PROGNAME}.bin") new_file_name = env.subst("$BUILD_DIR/${PROGNAME}.ota.bin") - shutil.copyfile(firmware_name, new_file_name) + print(f"Copied firmware to {new_file_name}") -# pylint: disable=E0602 -env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_create_combined_bin) # noqa -env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_copy_ota_bin) # noqa +# Run merge first, then ota copy second +env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", merge_factory_bin) # noqa: F821 +env.AddPostAction("$BUILD_DIR/${PROGNAME}.bin", esp32_copy_ota_bin) # noqa: F821 diff --git a/esphome/components/esp32/preferences.cpp b/esphome/components/esp32/preferences.cpp index e53cdd90d3..7bdbb265ca 100644 --- a/esphome/components/esp32/preferences.cpp +++ b/esphome/components/esp32/preferences.cpp @@ -8,6 +8,7 @@ #include #include #include +#include namespace esphome { namespace esp32 { @@ -16,7 +17,14 @@ static const char *const TAG = "esp32.preferences"; struct NVSData { std::string key; - std::vector data; + std::unique_ptr data; + size_t len; + + void set_data(const uint8_t *src, size_t size) { + data = std::make_unique(size); + memcpy(data.get(), src, size); + len = size; + } }; static std::vector s_pending_save; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) @@ -29,26 +37,26 @@ class ESP32PreferenceBackend : public ESPPreferenceBackend { // try find in pending saves and update that for (auto &obj : s_pending_save) { if (obj.key == key) { - obj.data.assign(data, data + len); + obj.set_data(data, len); return true; } } NVSData save{}; save.key = key; - save.data.assign(data, data + len); - s_pending_save.emplace_back(save); - ESP_LOGVV(TAG, "s_pending_save: key: %s, len: %d", key.c_str(), len); + save.set_data(data, len); + s_pending_save.emplace_back(std::move(save)); + ESP_LOGVV(TAG, "s_pending_save: key: %s, len: %zu", key.c_str(), len); return true; } bool load(uint8_t *data, size_t len) override { // try find in pending saves and load from that for (auto &obj : s_pending_save) { if (obj.key == key) { - if (obj.data.size() != len) { + if (obj.len != len) { // size mismatch return false; } - memcpy(data, obj.data.data(), len); + memcpy(data, obj.data.get(), len); return true; } } @@ -60,7 +68,7 @@ class ESP32PreferenceBackend : public ESPPreferenceBackend { return false; } if (actual_len != len) { - ESP_LOGVV(TAG, "NVS length does not match (%u!=%u)", actual_len, len); + ESP_LOGVV(TAG, "NVS length does not match (%zu!=%zu)", actual_len, len); return false; } err = nvs_get_blob(nvs_handle, key.c_str(), data, &len); @@ -68,7 +76,7 @@ class ESP32PreferenceBackend : public ESPPreferenceBackend { ESP_LOGV(TAG, "nvs_get_blob('%s') failed: %s", key.c_str(), esp_err_to_name(err)); return false; } else { - ESP_LOGVV(TAG, "nvs_get_blob: key: %s, len: %d", key.c_str(), len); + ESP_LOGVV(TAG, "nvs_get_blob: key: %s, len: %zu", key.c_str(), len); } return true; } @@ -111,7 +119,7 @@ class ESP32Preferences : public ESPPreferences { if (s_pending_save.empty()) return true; - ESP_LOGV(TAG, "Saving %d items...", s_pending_save.size()); + ESP_LOGV(TAG, "Saving %zu items...", s_pending_save.size()); // goal try write all pending saves even if one fails int cached = 0, written = 0, failed = 0; esp_err_t last_err = ESP_OK; @@ -122,11 +130,10 @@ class ESP32Preferences : public ESPPreferences { const auto &save = s_pending_save[i]; ESP_LOGVV(TAG, "Checking if NVS data %s has changed", save.key.c_str()); if (is_changed(nvs_handle, save)) { - esp_err_t err = nvs_set_blob(nvs_handle, save.key.c_str(), save.data.data(), save.data.size()); - ESP_LOGV(TAG, "sync: key: %s, len: %d", save.key.c_str(), save.data.size()); + esp_err_t err = nvs_set_blob(nvs_handle, save.key.c_str(), save.data.get(), save.len); + ESP_LOGV(TAG, "sync: key: %s, len: %zu", save.key.c_str(), save.len); if (err != 0) { - ESP_LOGV(TAG, "nvs_set_blob('%s', len=%u) failed: %s", save.key.c_str(), save.data.size(), - esp_err_to_name(err)); + ESP_LOGV(TAG, "nvs_set_blob('%s', len=%zu) failed: %s", save.key.c_str(), save.len, esp_err_to_name(err)); failed++; last_err = err; last_key = save.key; @@ -134,7 +141,7 @@ class ESP32Preferences : public ESPPreferences { } written++; } else { - ESP_LOGV(TAG, "NVS data not changed skipping %s len=%u", save.key.c_str(), save.data.size()); + ESP_LOGV(TAG, "NVS data not changed skipping %s len=%zu", save.key.c_str(), save.len); cached++; } s_pending_save.erase(s_pending_save.begin() + i); @@ -156,20 +163,23 @@ class ESP32Preferences : public ESPPreferences { return failed == 0; } bool is_changed(const uint32_t nvs_handle, const NVSData &to_save) { - NVSData stored_data{}; size_t actual_len; esp_err_t err = nvs_get_blob(nvs_handle, to_save.key.c_str(), nullptr, &actual_len); if (err != 0) { ESP_LOGV(TAG, "nvs_get_blob('%s'): %s - the key might not be set yet", to_save.key.c_str(), esp_err_to_name(err)); return true; } - stored_data.data.resize(actual_len); - err = nvs_get_blob(nvs_handle, to_save.key.c_str(), stored_data.data.data(), &actual_len); + // Check size first before allocating memory + if (actual_len != to_save.len) { + return true; + } + auto stored_data = std::make_unique(actual_len); + err = nvs_get_blob(nvs_handle, to_save.key.c_str(), stored_data.get(), &actual_len); if (err != 0) { ESP_LOGV(TAG, "nvs_get_blob('%s') failed: %s", to_save.key.c_str(), esp_err_to_name(err)); return true; } - return to_save.data != stored_data.data; + return memcmp(to_save.data.get(), stored_data.get(), to_save.len) != 0; } bool reset() override { diff --git a/esphome/components/esp32_ble/__init__.py b/esphome/components/esp32_ble/__init__.py index 93bb643596..411c2add71 100644 --- a/esphome/components/esp32_ble/__init__.py +++ b/esphome/components/esp32_ble/__init__.py @@ -1,17 +1,30 @@ +from collections.abc import Callable, MutableMapping +from dataclasses import dataclass from enum import Enum +import logging import re +from typing import Any from esphome import automation import esphome.codegen as cg from esphome.components.esp32 import add_idf_sdkconfig_option, const, get_esp32_variant import esphome.config_validation as cv -from esphome.const import CONF_ENABLE_ON_BOOT, CONF_ESPHOME, CONF_ID, CONF_NAME -from esphome.core import CORE -from esphome.core.config import CONF_NAME_ADD_MAC_SUFFIX +from esphome.const import ( + CONF_ENABLE_ON_BOOT, + CONF_ESPHOME, + CONF_ID, + CONF_MAX_CONNECTIONS, + CONF_NAME, + CONF_NAME_ADD_MAC_SUFFIX, +) +from esphome.core import CORE, CoroPriority, TimePeriod, coroutine_with_priority import esphome.final_validate as fv DEPENDENCIES = ["esp32"] -CODEOWNERS = ["@jesserockz", "@Rapsssito"] +CODEOWNERS = ["@jesserockz", "@Rapsssito", "@bdraco"] +DOMAIN = "esp32_ble" + +_LOGGER = logging.getLogger(__name__) class BTLoggers(Enum): @@ -95,8 +108,65 @@ class BTLoggers(Enum): """ESP32 WiFi provisioning over Bluetooth""" -# Set to track which loggers are needed by components -_required_loggers: set[BTLoggers] = set() +# Key for storing required loggers in CORE.data +ESP32_BLE_REQUIRED_LOGGERS_KEY = "esp32_ble_required_loggers" + + +def _get_required_loggers() -> set[BTLoggers]: + """Get the set of required Bluetooth loggers from CORE.data.""" + return CORE.data.setdefault(ESP32_BLE_REQUIRED_LOGGERS_KEY, set()) + + +# Dataclass for handler registration counts +@dataclass +class HandlerCounts: + gap_event: int = 0 + gap_scan_event: int = 0 + gattc_event: int = 0 + gatts_event: int = 0 + ble_status_event: int = 0 + + +# Track handler registration counts for StaticVector sizing +_handler_counts = HandlerCounts() + + +def register_gap_event_handler(parent_var: cg.MockObj, handler_var: cg.MockObj) -> None: + """Register a GAP event handler and track the count.""" + _handler_counts.gap_event += 1 + cg.add(parent_var.register_gap_event_handler(handler_var)) + + +def register_gap_scan_event_handler( + parent_var: cg.MockObj, handler_var: cg.MockObj +) -> None: + """Register a GAP scan event handler and track the count.""" + _handler_counts.gap_scan_event += 1 + cg.add(parent_var.register_gap_scan_event_handler(handler_var)) + + +def register_gattc_event_handler( + parent_var: cg.MockObj, handler_var: cg.MockObj +) -> None: + """Register a GATTc event handler and track the count.""" + _handler_counts.gattc_event += 1 + cg.add(parent_var.register_gattc_event_handler(handler_var)) + + +def register_gatts_event_handler( + parent_var: cg.MockObj, handler_var: cg.MockObj +) -> None: + """Register a GATTs event handler and track the count.""" + _handler_counts.gatts_event += 1 + cg.add(parent_var.register_gatts_event_handler(handler_var)) + + +def register_ble_status_event_handler( + parent_var: cg.MockObj, handler_var: cg.MockObj +) -> None: + """Register a BLE status event handler and track the count.""" + _handler_counts.ble_status_event += 1 + cg.add(parent_var.register_ble_status_event_handler(handler_var)) def register_bt_logger(*loggers: BTLoggers) -> None: @@ -105,18 +175,44 @@ def register_bt_logger(*loggers: BTLoggers) -> None: Args: *loggers: One or more BTLoggers enum members """ + required_loggers = _get_required_loggers() for logger in loggers: if not isinstance(logger, BTLoggers): raise TypeError( f"Logger must be a BTLoggers enum member, got {type(logger)}" ) - _required_loggers.add(logger) + required_loggers.add(logger) CONF_BLE_ID = "ble_id" CONF_IO_CAPABILITY = "io_capability" +CONF_ADVERTISING = "advertising" CONF_ADVERTISING_CYCLE_TIME = "advertising_cycle_time" CONF_DISABLE_BT_LOGS = "disable_bt_logs" +CONF_CONNECTION_TIMEOUT = "connection_timeout" +CONF_MAX_NOTIFICATIONS = "max_notifications" + +# BLE connection limits +# ESP-IDF CONFIG_BT_ACL_CONNECTIONS has range 1-9, default 4 +# Total instances: 10 (ADV + SCAN + connections) +# - ADV only: up to 9 connections +# - SCAN only: up to 9 connections +# - ADV + SCAN: up to 8 connections +DEFAULT_MAX_CONNECTIONS = 3 +IDF_MAX_CONNECTIONS = 9 + +# Connection slot tracking keys +KEY_ESP32_BLE = "esp32_ble" +KEY_USED_CONNECTION_SLOTS = "used_connection_slots" + +# Export for use by other components (bluetooth_proxy, etc.) +__all__ = [ + "DEFAULT_MAX_CONNECTIONS", + "IDF_MAX_CONNECTIONS", + "KEY_ESP32_BLE", + "KEY_USED_CONNECTION_SLOTS", + "consume_connection_slots", +] NO_BLUETOOTH_VARIANTS = [const.VARIANT_ESP32S2] @@ -161,11 +257,21 @@ CONFIG_SCHEMA = cv.Schema( IO_CAPABILITY, lower=True ), cv.Optional(CONF_ENABLE_ON_BOOT, default=True): cv.boolean, + cv.Optional(CONF_ADVERTISING, default=False): cv.boolean, cv.Optional( CONF_ADVERTISING_CYCLE_TIME, default="10s" ): cv.positive_time_period_milliseconds, - cv.SplitDefault(CONF_DISABLE_BT_LOGS, esp32_idf=True): cv.All( - cv.only_with_esp_idf, cv.boolean + cv.Optional(CONF_DISABLE_BT_LOGS, default=True): cv.boolean, + cv.Optional(CONF_CONNECTION_TIMEOUT, default="20s"): cv.All( + cv.positive_time_period_seconds, + cv.Range(min=TimePeriod(seconds=10), max=TimePeriod(seconds=180)), + ), + cv.Optional(CONF_MAX_NOTIFICATIONS, default=12): cv.All( + cv.positive_int, + cv.Range(min=1, max=64), + ), + cv.Optional(CONF_MAX_CONNECTIONS, default=DEFAULT_MAX_CONNECTIONS): cv.All( + cv.positive_int, cv.Range(min=1, max=IDF_MAX_CONNECTIONS) ), } ).extend(cv.COMPONENT_SCHEMA) @@ -214,6 +320,60 @@ def validate_variant(_): raise cv.Invalid(f"{variant} does not support Bluetooth") +def consume_connection_slots( + value: int, consumer: str +) -> Callable[[MutableMapping], MutableMapping]: + """Reserve BLE connection slots for a component. + + Args: + value: Number of connection slots to reserve + consumer: Name of the component consuming the slots + + Returns: + A validator function that records the slot usage + """ + + def _consume_connection_slots(config: MutableMapping) -> MutableMapping: + data: dict[str, Any] = CORE.data.setdefault(KEY_ESP32_BLE, {}) + slots: list[str] = data.setdefault(KEY_USED_CONNECTION_SLOTS, []) + slots.extend([consumer] * value) + return config + + return _consume_connection_slots + + +def validate_connection_slots(max_connections: int) -> None: + """Validate that BLE connection slots don't exceed the configured maximum.""" + # Skip validation in testing mode to allow component grouping + if CORE.testing_mode: + return + + ble_data = CORE.data.get(KEY_ESP32_BLE, {}) + used_slots = ble_data.get(KEY_USED_CONNECTION_SLOTS, []) + num_used = len(used_slots) + + if num_used <= max_connections: + return + + slot_users = ", ".join(used_slots) + + if num_used > IDF_MAX_CONNECTIONS: + raise cv.Invalid( + f"BLE components require {num_used} connection slots but maximum is {IDF_MAX_CONNECTIONS}. " + f"Reduce the number of BLE clients. Components: {slot_users}" + ) + + _LOGGER.warning( + "BLE components require %d connection slot(s) but only %d configured. " + "Please set 'max_connections: %d' in the 'esp32_ble' component. " + "Components: %s", + num_used, + max_connections, + num_used, + slot_users, + ) + + def final_validation(config): validate_variant(config) if (name := config.get(CONF_NAME)) is not None: @@ -226,12 +386,92 @@ def final_validation(config): f"Name '{name}' is too long, maximum length is {max_length} characters" ) + # Set GATT Client/Server sdkconfig options based on which components are loaded + full_config = fv.full_config.get() + + # Validate connection slots usage + max_connections = config.get(CONF_MAX_CONNECTIONS, DEFAULT_MAX_CONNECTIONS) + validate_connection_slots(max_connections) + + # Check if hosted bluetooth is being used + if "esp32_hosted" in full_config: + add_idf_sdkconfig_option("CONFIG_BT_CLASSIC_ENABLED", False) + add_idf_sdkconfig_option("CONFIG_BT_BLE_ENABLED", True) + add_idf_sdkconfig_option("CONFIG_BT_BLUEDROID_ENABLED", True) + add_idf_sdkconfig_option("CONFIG_BT_CONTROLLER_DISABLED", True) + add_idf_sdkconfig_option("CONFIG_ESP_HOSTED_ENABLE_BT_BLUEDROID", True) + add_idf_sdkconfig_option("CONFIG_ESP_HOSTED_BLUEDROID_HCI_VHCI", True) + + # Check if BLE Server is needed + has_ble_server = "esp32_ble_server" in full_config + + # Check if BLE Client is needed (via esp32_ble_tracker or esp32_ble_client) + has_ble_client = ( + "esp32_ble_tracker" in full_config or "esp32_ble_client" in full_config + ) + + # ESP-IDF BLE stack requires GATT Server to be enabled when GATT Client is enabled + # This is an internal dependency in the Bluedroid stack (tested ESP-IDF 5.4.2-5.5.1) + # See: https://github.com/espressif/esp-idf/issues/17724 + add_idf_sdkconfig_option("CONFIG_BT_GATTS_ENABLE", has_ble_server or has_ble_client) + add_idf_sdkconfig_option("CONFIG_BT_GATTC_ENABLE", has_ble_client) + + # Handle max_connections: check for deprecated location in esp32_ble_tracker + max_connections = config.get(CONF_MAX_CONNECTIONS, DEFAULT_MAX_CONNECTIONS) + + # Use value from tracker if esp32_ble doesn't have it explicitly set (backward compat) + if "esp32_ble_tracker" in full_config: + tracker_config = full_config["esp32_ble_tracker"] + if "max_connections" in tracker_config and CONF_MAX_CONNECTIONS not in config: + max_connections = tracker_config["max_connections"] + + # Set CONFIG_BT_ACL_CONNECTIONS to the maximum connections needed + 1 for ADV/SCAN + # This is the Bluedroid host stack total instance limit (range 1-9, default 4) + # Total instances = ADV/SCAN (1) + connection slots (max_connections) + # Shared between client (tracker/ble_client) and server + add_idf_sdkconfig_option("CONFIG_BT_ACL_CONNECTIONS", max_connections + 1) + + # Set controller-specific max connections for ESP32 (classic) + # CONFIG_BTDM_CTRL_BLE_MAX_CONN is ESP32-specific controller limit (just connections, not ADV/SCAN) + # For newer chips (C3/S3/etc), different configs are used automatically + add_idf_sdkconfig_option("CONFIG_BTDM_CTRL_BLE_MAX_CONN", max_connections) + return config FINAL_VALIDATE_SCHEMA = final_validation +# This needs to be run as a job with CoroPriority.FINAL priority so that all components have +# a chance to register their handlers before the counts are added to defines. +@coroutine_with_priority(CoroPriority.FINAL) +async def _add_ble_handler_defines(): + # Add defines for StaticVector sizing based on handler registration counts + # Only define if count > 0 to avoid allocating unnecessary memory + if _handler_counts.gap_event > 0: + cg.add_define( + "ESPHOME_ESP32_BLE_GAP_EVENT_HANDLER_COUNT", _handler_counts.gap_event + ) + if _handler_counts.gap_scan_event > 0: + cg.add_define( + "ESPHOME_ESP32_BLE_GAP_SCAN_EVENT_HANDLER_COUNT", + _handler_counts.gap_scan_event, + ) + if _handler_counts.gattc_event > 0: + cg.add_define( + "ESPHOME_ESP32_BLE_GATTC_EVENT_HANDLER_COUNT", _handler_counts.gattc_event + ) + if _handler_counts.gatts_event > 0: + cg.add_define( + "ESPHOME_ESP32_BLE_GATTS_EVENT_HANDLER_COUNT", _handler_counts.gatts_event + ) + if _handler_counts.ble_status_event > 0: + cg.add_define( + "ESPHOME_ESP32_BLE_BLE_STATUS_EVENT_HANDLER_COUNT", + _handler_counts.ble_status_event, + ) + + async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) cg.add(var.set_enable_on_boot(config[CONF_ENABLE_ON_BOOT])) @@ -241,22 +481,55 @@ async def to_code(config): cg.add(var.set_name(name)) await cg.register_component(var, config) - if CORE.using_esp_idf: - add_idf_sdkconfig_option("CONFIG_BT_ENABLED", True) - add_idf_sdkconfig_option("CONFIG_BT_BLE_42_FEATURES_SUPPORTED", True) + # Define max connections for use in C++ code (e.g., ble_server.h) + max_connections = config.get(CONF_MAX_CONNECTIONS, DEFAULT_MAX_CONNECTIONS) + cg.add_define("USE_ESP32_BLE_MAX_CONNECTIONS", max_connections) - # Register the core BLE loggers that are always needed - register_bt_logger(BTLoggers.GAP, BTLoggers.BTM, BTLoggers.HCI) + add_idf_sdkconfig_option("CONFIG_BT_ENABLED", True) + add_idf_sdkconfig_option("CONFIG_BT_BLE_42_FEATURES_SUPPORTED", True) - # Apply logger settings if log disabling is enabled - if config.get(CONF_DISABLE_BT_LOGS, False): - # Disable all Bluetooth loggers that are not required - for logger in BTLoggers: - if logger not in _required_loggers: - add_idf_sdkconfig_option(f"{logger.value}_NONE", True) + # Register the core BLE loggers that are always needed + register_bt_logger(BTLoggers.GAP, BTLoggers.BTM, BTLoggers.HCI) + + # Apply logger settings if log disabling is enabled + if config.get(CONF_DISABLE_BT_LOGS, False): + # Disable all Bluetooth loggers that are not required + required_loggers = _get_required_loggers() + for logger in BTLoggers: + if logger not in required_loggers: + add_idf_sdkconfig_option(f"{logger.value}_NONE", True) + + # Set BLE connection establishment timeout to match aioesphomeapi/bleak-retry-connector + # Default is 20 seconds instead of ESP-IDF's 30 seconds. Because there is no way to + # cancel a BLE connection in progress, when aioesphomeapi times out at 20 seconds, + # the connection slot remains occupied for the remaining time, preventing new connection + # attempts and wasting valuable connection slots. + if CONF_CONNECTION_TIMEOUT in config: + timeout_seconds = int(config[CONF_CONNECTION_TIMEOUT].total_seconds) + add_idf_sdkconfig_option("CONFIG_BT_BLE_ESTAB_LINK_CONN_TOUT", timeout_seconds) + # Increase GATT client connection retry count for problematic devices + # Default in ESP-IDF is 3, we increase to 10 for better reliability with + # low-power/timing-sensitive devices + add_idf_sdkconfig_option("CONFIG_BT_GATTC_CONNECT_RETRY_COUNT", 10) + + # Set the maximum number of notification registrations + # This controls how many BLE characteristics can have notifications enabled + # across all connections for a single GATT client interface + # https://github.com/esphome/issues/issues/6808 + if CONF_MAX_NOTIFICATIONS in config: + add_idf_sdkconfig_option( + "CONFIG_BT_GATTC_NOTIF_REG_MAX", config[CONF_MAX_NOTIFICATIONS] + ) cg.add_define("USE_ESP32_BLE") + if config[CONF_ADVERTISING]: + cg.add_define("USE_ESP32_BLE_ADVERTISING") + cg.add_define("USE_ESP32_BLE_UUID") + + # Schedule the handler defines to be added after all components register + CORE.add_job(_add_ble_handler_defines) + @automation.register_condition("ble.enabled", BLEEnabledCondition, cv.Schema({})) async def ble_enabled_to_code(config, condition_id, template_arg, args): diff --git a/esphome/components/esp32_ble/ble.cpp b/esphome/components/esp32_ble/ble.cpp index 8b0cf4da98..5bbd5fe9ed 100644 --- a/esphome/components/esp32_ble/ble.cpp +++ b/esphome/components/esp32_ble/ble.cpp @@ -1,12 +1,20 @@ -#ifdef USE_ESP32 - #include "ble.h" +#ifdef USE_ESP32 + #include "esphome/core/application.h" #include "esphome/core/helpers.h" #include "esphome/core/log.h" +#ifndef CONFIG_ESP_HOSTED_ENABLE_BT_BLUEDROID #include +#else +extern "C" { +#include +#include +#include +} +#endif #include #include #include @@ -19,15 +27,12 @@ #include #endif -namespace esphome { -namespace esp32_ble { +namespace esphome::esp32_ble { static const char *const TAG = "esp32_ble"; void ESP32BLE::setup() { global_ble = this; - ESP_LOGCONFIG(TAG, "Running setup"); - if (!ble_pre_setup_()) { ESP_LOGE(TAG, "BLE could not be prepared for configuration"); this->mark_failed(); @@ -56,6 +61,7 @@ void ESP32BLE::disable() { bool ESP32BLE::is_active() { return this->state_ == BLE_COMPONENT_STATE_ACTIVE; } +#ifdef USE_ESP32_BLE_ADVERTISING void ESP32BLE::advertising_start() { this->advertising_init_(); if (!this->is_active()) @@ -75,6 +81,28 @@ void ESP32BLE::advertising_set_manufacturer_data(const std::vector &dat this->advertising_start(); } +void ESP32BLE::advertising_set_service_data_and_name(std::span data, bool include_name) { + // This method atomically updates both service data and device name inclusion in BLE advertising. + // When include_name is true, the device name is included in the advertising packet making it + // visible to passive BLE scanners. When false, the name is only visible in scan response + // (requires active scanning). This atomic operation ensures we only restart advertising once + // when changing both properties, avoiding the brief gap that would occur with separate calls. + + this->advertising_init_(); + + if (include_name) { + // When including name, clear service data first to avoid packet overflow + this->advertising_->set_service_data(std::span{}); + this->advertising_->set_include_name(true); + } else { + // When including service data, clear name first to avoid packet overflow + this->advertising_->set_include_name(false); + this->advertising_->set_service_data(data); + } + + this->advertising_start(); +} + void ESP32BLE::advertising_register_raw_advertisement_callback(std::function &&callback) { this->advertising_init_(); this->advertising_->register_raw_advertisement_callback(std::move(callback)); @@ -91,6 +119,7 @@ void ESP32BLE::advertising_remove_service_uuid(ESPBTUUID uuid) { this->advertising_->remove_service_uuid(uuid); this->advertising_start(); } +#endif bool ESP32BLE::ble_pre_setup_() { esp_err_t err = nvs_flash_init(); @@ -101,6 +130,7 @@ bool ESP32BLE::ble_pre_setup_() { return true; } +#ifdef USE_ESP32_BLE_ADVERTISING void ESP32BLE::advertising_init_() { if (this->advertising_ != nullptr) return; @@ -110,9 +140,11 @@ void ESP32BLE::advertising_init_() { this->advertising_->set_min_preferred_interval(0x06); this->advertising_->set_appearance(this->appearance_); } +#endif bool ESP32BLE::ble_setup_() { esp_err_t err; +#ifndef CONFIG_ESP_HOSTED_ENABLE_BT_BLUEDROID #ifdef USE_ARDUINO if (!btStart()) { ESP_LOGE(TAG, "btStart failed: %d", esp_bt_controller_get_status()); @@ -146,6 +178,28 @@ bool ESP32BLE::ble_setup_() { #endif esp_bt_controller_mem_release(ESP_BT_MODE_CLASSIC_BT); +#else + esp_hosted_connect_to_slave(); // NOLINT + + if (esp_hosted_bt_controller_init() != ESP_OK) { + ESP_LOGW(TAG, "esp_hosted_bt_controller_init failed"); + return false; + } + + if (esp_hosted_bt_controller_enable() != ESP_OK) { + ESP_LOGW(TAG, "esp_hosted_bt_controller_enable failed"); + return false; + } + + hosted_hci_bluedroid_open(); + + esp_bluedroid_hci_driver_operations_t operations = { + .send = hosted_hci_bluedroid_send, + .check_send_available = hosted_hci_bluedroid_check_send_available, + .register_host_callback = hosted_hci_bluedroid_register_host_callback, + }; + esp_bluedroid_attach_hci_driver(&operations); +#endif err = esp_bluedroid_init(); if (err != ESP_OK) { @@ -158,43 +212,48 @@ bool ESP32BLE::ble_setup_() { return false; } - if (!this->gap_event_handlers_.empty()) { - err = esp_ble_gap_register_callback(ESP32BLE::gap_event_handler); - if (err != ESP_OK) { - ESP_LOGE(TAG, "esp_ble_gap_register_callback failed: %d", err); - return false; - } +#ifdef ESPHOME_ESP32_BLE_GAP_EVENT_HANDLER_COUNT + err = esp_ble_gap_register_callback(ESP32BLE::gap_event_handler); + if (err != ESP_OK) { + ESP_LOGE(TAG, "esp_ble_gap_register_callback failed: %d", err); + return false; } +#endif - if (!this->gatts_event_handlers_.empty()) { - err = esp_ble_gatts_register_callback(ESP32BLE::gatts_event_handler); - if (err != ESP_OK) { - ESP_LOGE(TAG, "esp_ble_gatts_register_callback failed: %d", err); - return false; - } +#if defined(USE_ESP32_BLE_SERVER) && defined(ESPHOME_ESP32_BLE_GATTS_EVENT_HANDLER_COUNT) + err = esp_ble_gatts_register_callback(ESP32BLE::gatts_event_handler); + if (err != ESP_OK) { + ESP_LOGE(TAG, "esp_ble_gatts_register_callback failed: %d", err); + return false; } +#endif - if (!this->gattc_event_handlers_.empty()) { - err = esp_ble_gattc_register_callback(ESP32BLE::gattc_event_handler); - if (err != ESP_OK) { - ESP_LOGE(TAG, "esp_ble_gattc_register_callback failed: %d", err); - return false; - } +#if defined(USE_ESP32_BLE_CLIENT) && defined(ESPHOME_ESP32_BLE_GATTC_EVENT_HANDLER_COUNT) + err = esp_ble_gattc_register_callback(ESP32BLE::gattc_event_handler); + if (err != ESP_OK) { + ESP_LOGE(TAG, "esp_ble_gattc_register_callback failed: %d", err); + return false; } +#endif std::string name; if (this->name_.has_value()) { name = this->name_.value(); if (App.is_name_add_mac_suffix_enabled()) { - name += "-" + get_mac_address().substr(6); + // MAC address suffix length (last 6 characters of 12-char MAC address string) + constexpr size_t mac_address_suffix_len = 6; + const std::string mac_addr = get_mac_address(); + const char *mac_suffix_ptr = mac_addr.c_str() + mac_address_suffix_len; + name = make_name_with_suffix(name, '-', mac_suffix_ptr, mac_address_suffix_len); } } else { name = App.get_name(); if (name.length() > 20) { if (App.is_name_add_mac_suffix_enabled()) { - name.erase(name.begin() + 13, name.end() - 7); // Remove characters between 13 and the mac address + // Keep first 13 chars and last 7 chars (MAC suffix), remove middle + name.erase(13, name.length() - 20); } else { - name = name.substr(0, 20); + name.resize(20); } } } @@ -229,6 +288,7 @@ bool ESP32BLE::ble_dismantle_() { return false; } +#ifndef CONFIG_ESP_HOSTED_ENABLE_BT_BLUEDROID #ifdef USE_ARDUINO if (!btStop()) { ESP_LOGE(TAG, "btStop failed: %d", esp_bt_controller_get_status()); @@ -258,6 +318,19 @@ bool ESP32BLE::ble_dismantle_() { return false; } } +#endif +#else + if (esp_hosted_bt_controller_disable() != ESP_OK) { + ESP_LOGW(TAG, "esp_hosted_bt_controller_disable failed"); + return false; + } + + if (esp_hosted_bt_controller_deinit(false) != ESP_OK) { + ESP_LOGW(TAG, "esp_hosted_bt_controller_deinit failed"); + return false; + } + + hosted_hci_bluedroid_close(); #endif return true; } @@ -270,9 +343,11 @@ void ESP32BLE::loop() { case BLE_COMPONENT_STATE_DISABLE: { ESP_LOGD(TAG, "Disabling"); +#ifdef ESPHOME_ESP32_BLE_BLE_STATUS_EVENT_HANDLER_COUNT for (auto *ble_event_handler : this->ble_status_event_handlers_) { ble_event_handler->ble_before_disabled_event_handler(); } +#endif if (!ble_dismantle_()) { ESP_LOGE(TAG, "Could not be dismantled"); @@ -302,34 +377,40 @@ void ESP32BLE::loop() { BLEEvent *ble_event = this->ble_events_.pop(); while (ble_event != nullptr) { switch (ble_event->type_) { +#if defined(USE_ESP32_BLE_SERVER) && defined(ESPHOME_ESP32_BLE_GATTS_EVENT_HANDLER_COUNT) case BLEEvent::GATTS: { esp_gatts_cb_event_t event = ble_event->event_.gatts.gatts_event; esp_gatt_if_t gatts_if = ble_event->event_.gatts.gatts_if; - esp_ble_gatts_cb_param_t *param = ble_event->event_.gatts.gatts_param; + esp_ble_gatts_cb_param_t *param = &ble_event->event_.gatts.gatts_param; ESP_LOGV(TAG, "gatts_event [esp_gatt_if: %d] - %d", gatts_if, event); for (auto *gatts_handler : this->gatts_event_handlers_) { gatts_handler->gatts_event_handler(event, gatts_if, param); } break; } +#endif +#if defined(USE_ESP32_BLE_CLIENT) && defined(ESPHOME_ESP32_BLE_GATTC_EVENT_HANDLER_COUNT) case BLEEvent::GATTC: { esp_gattc_cb_event_t event = ble_event->event_.gattc.gattc_event; esp_gatt_if_t gattc_if = ble_event->event_.gattc.gattc_if; - esp_ble_gattc_cb_param_t *param = ble_event->event_.gattc.gattc_param; + esp_ble_gattc_cb_param_t *param = &ble_event->event_.gattc.gattc_param; ESP_LOGV(TAG, "gattc_event [esp_gatt_if: %d] - %d", gattc_if, event); for (auto *gattc_handler : this->gattc_event_handlers_) { gattc_handler->gattc_event_handler(event, gattc_if, param); } break; } +#endif case BLEEvent::GAP: { esp_gap_ble_cb_event_t gap_event = ble_event->event_.gap.gap_event; switch (gap_event) { case ESP_GAP_BLE_SCAN_RESULT_EVT: +#ifdef ESPHOME_ESP32_BLE_GAP_SCAN_EVENT_HANDLER_COUNT // Use the new scan event handler - no memcpy! for (auto *scan_handler : this->gap_scan_event_handlers_) { scan_handler->gap_scan_event_handler(ble_event->scan_result()); } +#endif break; // Scan complete events @@ -341,10 +422,12 @@ void ESP32BLE::loop() { // This is verified at compile-time by static_assert checks in ble_event.h // The struct already contains our copy of the status (copied in BLEEvent constructor) ESP_LOGV(TAG, "gap_event_handler - %d", gap_event); +#ifdef ESPHOME_ESP32_BLE_GAP_EVENT_HANDLER_COUNT for (auto *gap_handler : this->gap_event_handlers_) { gap_handler->gap_event_handler( gap_event, reinterpret_cast(&ble_event->event_.gap.scan_complete)); } +#endif break; // Advertising complete events @@ -355,19 +438,23 @@ void ESP32BLE::loop() { case ESP_GAP_BLE_ADV_STOP_COMPLETE_EVT: // All advertising complete events have the same structure with just status ESP_LOGV(TAG, "gap_event_handler - %d", gap_event); +#ifdef ESPHOME_ESP32_BLE_GAP_EVENT_HANDLER_COUNT for (auto *gap_handler : this->gap_event_handlers_) { gap_handler->gap_event_handler( gap_event, reinterpret_cast(&ble_event->event_.gap.adv_complete)); } +#endif break; // RSSI complete event case ESP_GAP_BLE_READ_RSSI_COMPLETE_EVT: ESP_LOGV(TAG, "gap_event_handler - %d", gap_event); +#ifdef ESPHOME_ESP32_BLE_GAP_EVENT_HANDLER_COUNT for (auto *gap_handler : this->gap_event_handlers_) { gap_handler->gap_event_handler( gap_event, reinterpret_cast(&ble_event->event_.gap.read_rssi_complete)); } +#endif break; // Security events @@ -377,10 +464,12 @@ void ESP32BLE::loop() { case ESP_GAP_BLE_PASSKEY_REQ_EVT: case ESP_GAP_BLE_NC_REQ_EVT: ESP_LOGV(TAG, "gap_event_handler - %d", gap_event); +#ifdef ESPHOME_ESP32_BLE_GAP_EVENT_HANDLER_COUNT for (auto *gap_handler : this->gap_event_handlers_) { gap_handler->gap_event_handler( gap_event, reinterpret_cast(&ble_event->event_.gap.security)); } +#endif break; default: @@ -397,9 +486,11 @@ void ESP32BLE::loop() { this->ble_event_pool_.release(ble_event); ble_event = this->ble_events_.pop(); } +#ifdef USE_ESP32_BLE_ADVERTISING if (this->advertising_ != nullptr) { this->advertising_->loop(); } +#endif // Log dropped events periodically uint16_t dropped = this->ble_events_.get_and_reset_dropped_count(); @@ -413,13 +504,17 @@ void load_ble_event(BLEEvent *event, esp_gap_ble_cb_event_t e, esp_ble_gap_cb_pa event->load_gap_event(e, p); } +#ifdef USE_ESP32_BLE_CLIENT void load_ble_event(BLEEvent *event, esp_gattc_cb_event_t e, esp_gatt_if_t i, esp_ble_gattc_cb_param_t *p) { event->load_gattc_event(e, i, p); } +#endif +#ifdef USE_ESP32_BLE_SERVER void load_ble_event(BLEEvent *event, esp_gatts_cb_event_t e, esp_gatt_if_t i, esp_ble_gatts_cb_param_t *p) { event->load_gatts_event(e, i, p); } +#endif template void enqueue_ble_event(Args... args) { // Allocate an event from the pool @@ -440,8 +535,12 @@ template void enqueue_ble_event(Args... args) { // Explicit template instantiations for the friend function template void enqueue_ble_event(esp_gap_ble_cb_event_t, esp_ble_gap_cb_param_t *); +#ifdef USE_ESP32_BLE_SERVER template void enqueue_ble_event(esp_gatts_cb_event_t, esp_gatt_if_t, esp_ble_gatts_cb_param_t *); +#endif +#ifdef USE_ESP32_BLE_CLIENT template void enqueue_ble_event(esp_gattc_cb_event_t, esp_gatt_if_t, esp_ble_gattc_cb_param_t *); +#endif void ESP32BLE::gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param) { switch (event) { @@ -471,6 +570,8 @@ void ESP32BLE::gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_pa // Ignore these GAP events as they are not relevant for our use case case ESP_GAP_BLE_UPDATE_CONN_PARAMS_EVT: case ESP_GAP_BLE_SET_PKT_LENGTH_COMPLETE_EVT: + case ESP_GAP_BLE_PHY_UPDATE_COMPLETE_EVT: // BLE 5.0 PHY update complete + case ESP_GAP_BLE_CHANNEL_SELECT_ALGORITHM_EVT: // BLE 5.0 channel selection algorithm return; default: @@ -479,15 +580,19 @@ void ESP32BLE::gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_pa ESP_LOGW(TAG, "Ignoring unexpected GAP event type: %d", event); } +#ifdef USE_ESP32_BLE_SERVER void ESP32BLE::gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t gatts_if, esp_ble_gatts_cb_param_t *param) { enqueue_ble_event(event, gatts_if, param); } +#endif +#ifdef USE_ESP32_BLE_CLIENT void ESP32BLE::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gattc_cb_param_t *param) { enqueue_ble_event(event, gattc_if, param); } +#endif float ESP32BLE::get_setup_priority() const { return setup_priority::BLUETOOTH; } @@ -538,7 +643,6 @@ uint64_t ble_addr_to_uint64(const esp_bd_addr_t address) { ESP32BLE *global_ble = nullptr; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) -} // namespace esp32_ble -} // namespace esphome +} // namespace esphome::esp32_ble #endif diff --git a/esphome/components/esp32_ble/ble.h b/esphome/components/esp32_ble/ble.h index 2c5697df82..dc973f0e82 100644 --- a/esphome/components/esp32_ble/ble.h +++ b/esphome/components/esp32_ble/ble.h @@ -1,14 +1,18 @@ #pragma once -#include "ble_advertising.h" +#include "esphome/core/defines.h" // Must be included before conditional includes + #include "ble_uuid.h" #include "ble_scan_result.h" +#ifdef USE_ESP32_BLE_ADVERTISING +#include "ble_advertising.h" +#endif #include +#include #include "esphome/core/automation.h" #include "esphome/core/component.h" -#include "esphome/core/defines.h" #include "esphome/core/helpers.h" #include "ble_event.h" @@ -21,24 +25,16 @@ #include #include -namespace esphome { -namespace esp32_ble { +namespace esphome::esp32_ble { -// Maximum number of BLE scan results to buffer -// Sized to handle bursts of advertisements while allowing for processing delays -// With 16 advertisements per batch and some safety margin: -// - Without PSRAM: 24 entries (1.5× batch size) -// - With PSRAM: 36 entries (2.25× batch size) -// The reduced structure size (~80 bytes vs ~400 bytes) allows for larger buffers +// Maximum size of the BLE event queue +// Increased to absorb the ring buffer capacity from esp32_ble_tracker #ifdef USE_PSRAM -static constexpr uint8_t SCAN_RESULT_BUFFER_SIZE = 36; +static constexpr uint8_t MAX_BLE_QUEUE_SIZE = 100; // 64 + 36 (ring buffer size with PSRAM) #else -static constexpr uint8_t SCAN_RESULT_BUFFER_SIZE = 24; +static constexpr uint8_t MAX_BLE_QUEUE_SIZE = 88; // 64 + 24 (ring buffer size without PSRAM) #endif -// Maximum size of the BLE event queue - must be power of 2 for lock-free queue -static constexpr size_t MAX_BLE_QUEUE_SIZE = 64; - uint64_t ble_addr_to_uint64(const esp_bd_addr_t address); // NOLINTNEXTLINE(modernize-use-using) @@ -79,17 +75,21 @@ class GAPScanEventHandler { virtual void gap_scan_event_handler(const BLEScanResult &scan_result) = 0; }; +#ifdef USE_ESP32_BLE_CLIENT class GATTcEventHandler { public: virtual void gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gattc_cb_param_t *param) = 0; }; +#endif +#ifdef USE_ESP32_BLE_SERVER class GATTsEventHandler { public: virtual void gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t gatts_if, esp_ble_gatts_cb_param_t *param) = 0; }; +#endif class BLEStatusEventHandler { public: @@ -114,44 +114,73 @@ class ESP32BLE : public Component { float get_setup_priority() const override; void set_name(const std::string &name) { this->name_ = name; } +#ifdef USE_ESP32_BLE_ADVERTISING void advertising_start(); void advertising_set_service_data(const std::vector &data); void advertising_set_manufacturer_data(const std::vector &data); void advertising_set_appearance(uint16_t appearance) { this->appearance_ = appearance; } + void advertising_set_service_data_and_name(std::span data, bool include_name); void advertising_add_service_uuid(ESPBTUUID uuid); void advertising_remove_service_uuid(ESPBTUUID uuid); void advertising_register_raw_advertisement_callback(std::function &&callback); +#endif +#ifdef ESPHOME_ESP32_BLE_GAP_EVENT_HANDLER_COUNT void register_gap_event_handler(GAPEventHandler *handler) { this->gap_event_handlers_.push_back(handler); } +#endif +#ifdef ESPHOME_ESP32_BLE_GAP_SCAN_EVENT_HANDLER_COUNT void register_gap_scan_event_handler(GAPScanEventHandler *handler) { this->gap_scan_event_handlers_.push_back(handler); } +#endif +#if defined(USE_ESP32_BLE_CLIENT) && defined(ESPHOME_ESP32_BLE_GATTC_EVENT_HANDLER_COUNT) void register_gattc_event_handler(GATTcEventHandler *handler) { this->gattc_event_handlers_.push_back(handler); } +#endif +#if defined(USE_ESP32_BLE_SERVER) && defined(ESPHOME_ESP32_BLE_GATTS_EVENT_HANDLER_COUNT) void register_gatts_event_handler(GATTsEventHandler *handler) { this->gatts_event_handlers_.push_back(handler); } +#endif +#ifdef ESPHOME_ESP32_BLE_BLE_STATUS_EVENT_HANDLER_COUNT void register_ble_status_event_handler(BLEStatusEventHandler *handler) { this->ble_status_event_handlers_.push_back(handler); } +#endif void set_enable_on_boot(bool enable_on_boot) { this->enable_on_boot_ = enable_on_boot; } protected: +#ifdef USE_ESP32_BLE_SERVER static void gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t gatts_if, esp_ble_gatts_cb_param_t *param); +#endif +#ifdef USE_ESP32_BLE_CLIENT static void gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gattc_cb_param_t *param); +#endif static void gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param); bool ble_setup_(); bool ble_dismantle_(); bool ble_pre_setup_(); +#ifdef USE_ESP32_BLE_ADVERTISING void advertising_init_(); +#endif private: template friend void enqueue_ble_event(Args... args); - // Vectors (12 bytes each on 32-bit, naturally aligned to 4 bytes) - std::vector gap_event_handlers_; - std::vector gap_scan_event_handlers_; - std::vector gattc_event_handlers_; - std::vector gatts_event_handlers_; - std::vector ble_status_event_handlers_; + // Handler vectors - use StaticVector when counts are known at compile time +#ifdef ESPHOME_ESP32_BLE_GAP_EVENT_HANDLER_COUNT + StaticVector gap_event_handlers_; +#endif +#ifdef ESPHOME_ESP32_BLE_GAP_SCAN_EVENT_HANDLER_COUNT + StaticVector gap_scan_event_handlers_; +#endif +#if defined(USE_ESP32_BLE_CLIENT) && defined(ESPHOME_ESP32_BLE_GATTC_EVENT_HANDLER_COUNT) + StaticVector gattc_event_handlers_; +#endif +#if defined(USE_ESP32_BLE_SERVER) && defined(ESPHOME_ESP32_BLE_GATTS_EVENT_HANDLER_COUNT) + StaticVector gatts_event_handlers_; +#endif +#ifdef ESPHOME_ESP32_BLE_BLE_STATUS_EVENT_HANDLER_COUNT + StaticVector ble_status_event_handlers_; +#endif // Large objects (size depends on template parameters, but typically aligned to 4 bytes) esphome::LockFreeQueue ble_events_; @@ -161,7 +190,9 @@ class ESP32BLE : public Component { optional name_; // 4-byte aligned members - BLEAdvertising *advertising_{}; // 4 bytes (pointer) +#ifdef USE_ESP32_BLE_ADVERTISING + BLEAdvertising *advertising_{}; // 4 bytes (pointer) +#endif esp_ble_io_cap_t io_cap_{ESP_IO_CAP_NONE}; // 4 bytes (enum) uint32_t advertising_cycle_time_{}; // 4 bytes @@ -191,7 +222,6 @@ template class BLEDisableAction : public Action { void play(Ts... x) override { global_ble->disable(); } }; -} // namespace esp32_ble -} // namespace esphome +} // namespace esphome::esp32_ble #endif diff --git a/esphome/components/esp32_ble/ble_advertising.cpp b/esphome/components/esp32_ble/ble_advertising.cpp index 8d43b5af33..68704e49e2 100644 --- a/esphome/components/esp32_ble/ble_advertising.cpp +++ b/esphome/components/esp32_ble/ble_advertising.cpp @@ -1,6 +1,7 @@ #include "ble_advertising.h" #ifdef USE_ESP32 +#ifdef USE_ESP32_BLE_ADVERTISING #include #include @@ -8,8 +9,7 @@ #include "esphome/core/log.h" #include "esphome/core/application.h" -namespace esphome { -namespace esp32_ble { +namespace esphome::esp32_ble { static const char *const TAG = "esp32_ble.advertising"; @@ -43,7 +43,7 @@ void BLEAdvertising::remove_service_uuid(ESPBTUUID uuid) { this->advertising_uuids_.end()); } -void BLEAdvertising::set_service_data(const std::vector &data) { +void BLEAdvertising::set_service_data(std::span data) { delete[] this->advertising_data_.p_service_data; this->advertising_data_.p_service_data = nullptr; this->advertising_data_.service_data_len = data.size(); @@ -54,6 +54,10 @@ void BLEAdvertising::set_service_data(const std::vector &data) { } } +void BLEAdvertising::set_service_data(const std::vector &data) { + this->set_service_data(std::span(data)); +} + void BLEAdvertising::set_manufacturer_data(const std::vector &data) { delete[] this->advertising_data_.p_manufacturer_data; this->advertising_data_.p_manufacturer_data = nullptr; @@ -84,7 +88,7 @@ esp_err_t BLEAdvertising::services_advertisement_() { esp_err_t err; this->advertising_data_.set_scan_rsp = false; - this->advertising_data_.include_name = !this->scan_response_; + this->advertising_data_.include_name = this->include_name_in_adv_ || !this->scan_response_; this->advertising_data_.include_txpower = !this->scan_response_; err = esp_ble_gap_config_adv_data(&this->advertising_data_); if (err != ESP_OK) { @@ -148,7 +152,7 @@ void BLEAdvertising::loop() { if (now - this->last_advertisement_time_ > this->advertising_cycle_time_) { this->stop(); this->current_adv_index_ += 1; - if (this->current_adv_index_ >= this->raw_advertisements_callbacks_.size()) { + if (static_cast(this->current_adv_index_) >= this->raw_advertisements_callbacks_.size()) { this->current_adv_index_ = -1; } this->start(); @@ -160,7 +164,7 @@ void BLEAdvertising::register_raw_advertisement_callback(std::functionraw_advertisements_callbacks_.push_back(std::move(callback)); } -} // namespace esp32_ble -} // namespace esphome +} // namespace esphome::esp32_ble -#endif +#endif // USE_ESP32_BLE_ADVERTISING +#endif // USE_ESP32 diff --git a/esphome/components/esp32_ble/ble_advertising.h b/esphome/components/esp32_ble/ble_advertising.h index 0b2142115d..d7f1eeac9d 100644 --- a/esphome/components/esp32_ble/ble_advertising.h +++ b/esphome/components/esp32_ble/ble_advertising.h @@ -1,17 +1,22 @@ #pragma once +#include "esphome/core/defines.h" + #include #include +#include #include #ifdef USE_ESP32 +#ifdef USE_ESP32_BLE_ADVERTISING +#ifndef CONFIG_ESP_HOSTED_ENABLE_BT_BLUEDROID #include +#endif #include #include -namespace esphome { -namespace esp32_ble { +namespace esphome::esp32_ble { using raw_adv_data_t = struct { uint8_t *data; @@ -34,6 +39,8 @@ class BLEAdvertising { void set_manufacturer_data(const std::vector &data); void set_appearance(uint16_t appearance) { this->advertising_data_.appearance = appearance; } void set_service_data(const std::vector &data); + void set_service_data(std::span data); + void set_include_name(bool include_name) { this->include_name_in_adv_ = include_name; } void register_raw_advertisement_callback(std::function &&callback); void start(); @@ -43,6 +50,7 @@ class BLEAdvertising { esp_err_t services_advertisement_(); bool scan_response_; + bool include_name_in_adv_{false}; esp_ble_adv_data_t advertising_data_; esp_ble_adv_data_t scan_response_data_; esp_ble_adv_params_t advertising_params_; @@ -55,7 +63,7 @@ class BLEAdvertising { int8_t current_adv_index_{-1}; // -1 means standard scan response }; -} // namespace esp32_ble -} // namespace esphome +} // namespace esphome::esp32_ble -#endif +#endif // USE_ESP32_BLE_ADVERTISING +#endif // USE_ESP32 diff --git a/esphome/components/esp32_ble/ble_event.h b/esphome/components/esp32_ble/ble_event.h index 9268c710f3..299fd7705f 100644 --- a/esphome/components/esp32_ble/ble_event.h +++ b/esphome/components/esp32_ble/ble_event.h @@ -3,16 +3,14 @@ #ifdef USE_ESP32 #include // for offsetof -#include - +#include // for memcpy #include #include #include #include "ble_scan_result.h" -namespace esphome { -namespace esp32_ble { +namespace esphome::esp32_ble { // Compile-time verification that ESP-IDF scan complete events only contain a status field // This ensures our reinterpret_cast in ble.cpp is safe @@ -63,10 +61,24 @@ static_assert(offsetof(esp_ble_gap_cb_param_t, read_rssi_cmpl.rssi) == sizeof(es static_assert(offsetof(esp_ble_gap_cb_param_t, read_rssi_cmpl.remote_addr) == sizeof(esp_bt_status_t) + sizeof(int8_t), "remote_addr must follow rssi in read_rssi_cmpl"); +// Param struct sizes on ESP32 +static constexpr size_t GATTC_PARAM_SIZE = 28; +static constexpr size_t GATTS_PARAM_SIZE = 32; + +// Maximum size for inline storage of data +// GATTC: 80 - 28 (param) - 8 (other fields) = 44 bytes for data +// GATTS: 80 - 32 (param) - 8 (other fields) = 40 bytes for data +static constexpr size_t GATTC_INLINE_DATA_SIZE = 44; +static constexpr size_t GATTS_INLINE_DATA_SIZE = 40; + +// Verify param struct sizes +static_assert(sizeof(esp_ble_gattc_cb_param_t) == GATTC_PARAM_SIZE, "GATTC param size unexpected"); +static_assert(sizeof(esp_ble_gatts_cb_param_t) == GATTS_PARAM_SIZE, "GATTS param size unexpected"); + // Received GAP, GATTC and GATTS events are only queued, and get processed in the main loop(). // This class stores each event with minimal memory usage. -// GAP events (99% of traffic) don't have the vector overhead. -// GATTC/GATTS events use heap allocation for their param and data. +// GAP events (99% of traffic) don't have the heap allocation overhead. +// GATTC/GATTS events use heap allocation for their param and inline storage for small data. // // Event flow: // 1. ESP-IDF BLE stack calls our static handlers in the BLE task context @@ -113,21 +125,21 @@ class BLEEvent { this->init_gap_data_(e, p); } - // Constructor for GATTC events - uses heap allocation - // IMPORTANT: The heap allocation is REQUIRED and must not be removed as an optimization. - // The param pointer from ESP-IDF is only valid during the callback execution. - // Since BLE events are processed asynchronously in the main loop, we must create - // our own copy to ensure the data remains valid until the event is processed. + // Constructor for GATTC events - param stored inline, data may use heap + // IMPORTANT: We MUST copy the param struct because the pointer from ESP-IDF + // is only valid during the callback execution. Since BLE events are processed + // asynchronously in the main loop, we store our own copy inline to ensure + // the data remains valid until the event is processed. BLEEvent(esp_gattc_cb_event_t e, esp_gatt_if_t i, esp_ble_gattc_cb_param_t *p) { this->type_ = GATTC; this->init_gattc_data_(e, i, p); } - // Constructor for GATTS events - uses heap allocation - // IMPORTANT: The heap allocation is REQUIRED and must not be removed as an optimization. - // The param pointer from ESP-IDF is only valid during the callback execution. - // Since BLE events are processed asynchronously in the main loop, we must create - // our own copy to ensure the data remains valid until the event is processed. + // Constructor for GATTS events - param stored inline, data may use heap + // IMPORTANT: We MUST copy the param struct because the pointer from ESP-IDF + // is only valid during the callback execution. Since BLE events are processed + // asynchronously in the main loop, we store our own copy inline to ensure + // the data remains valid until the event is processed. BLEEvent(esp_gatts_cb_event_t e, esp_gatt_if_t i, esp_ble_gatts_cb_param_t *p) { this->type_ = GATTS; this->init_gatts_data_(e, i, p); @@ -137,25 +149,32 @@ class BLEEvent { ~BLEEvent() { this->release(); } // Default constructor for pre-allocation in pool - BLEEvent() : type_(GAP) {} + BLEEvent() : event_{}, type_(GAP) {} // Invoked on return to EventPool - clean up any heap-allocated data void release() { - if (this->type_ == GAP) { - return; - } - if (this->type_ == GATTC) { - delete this->event_.gattc.gattc_param; - delete this->event_.gattc.data; - this->event_.gattc.gattc_param = nullptr; - this->event_.gattc.data = nullptr; - return; - } - if (this->type_ == GATTS) { - delete this->event_.gatts.gatts_param; - delete this->event_.gatts.data; - this->event_.gatts.gatts_param = nullptr; - this->event_.gatts.data = nullptr; + switch (this->type_) { + case GAP: + // GAP events don't have heap allocations + break; + case GATTC: + // Param is now stored inline, only delete heap data if it was heap-allocated + if (!this->event_.gattc.is_inline && this->event_.gattc.data.heap_data != nullptr) { + delete[] this->event_.gattc.data.heap_data; + } + // Clear critical fields to prevent issues if type changes + this->event_.gattc.is_inline = false; + this->event_.gattc.data.heap_data = nullptr; + break; + case GATTS: + // Param is now stored inline, only delete heap data if it was heap-allocated + if (!this->event_.gatts.is_inline && this->event_.gatts.data.heap_data != nullptr) { + delete[] this->event_.gatts.data.heap_data; + } + // Clear critical fields to prevent issues if type changes + this->event_.gatts.is_inline = false; + this->event_.gatts.data.heap_data = nullptr; + break; } } @@ -207,20 +226,30 @@ class BLEEvent { // NOLINTNEXTLINE(readability-identifier-naming) struct gattc_event { - esp_gattc_cb_event_t gattc_event; - esp_gatt_if_t gattc_if; - esp_ble_gattc_cb_param_t *gattc_param; // Heap-allocated - std::vector *data; // Heap-allocated - } gattc; // 16 bytes (pointers only) + esp_ble_gattc_cb_param_t gattc_param; // Stored inline (28 bytes) + esp_gattc_cb_event_t gattc_event; // 4 bytes + union { + uint8_t *heap_data; // 4 bytes when heap-allocated + uint8_t inline_data[GATTC_INLINE_DATA_SIZE]; // 44 bytes when stored inline + } data; // 44 bytes total + uint16_t data_len; // 2 bytes + esp_gatt_if_t gattc_if; // 1 byte + bool is_inline; // 1 byte - true when data is stored inline + } gattc; // Total: 80 bytes // NOLINTNEXTLINE(readability-identifier-naming) struct gatts_event { - esp_gatts_cb_event_t gatts_event; - esp_gatt_if_t gatts_if; - esp_ble_gatts_cb_param_t *gatts_param; // Heap-allocated - std::vector *data; // Heap-allocated - } gatts; // 16 bytes (pointers only) - } event_; // 80 bytes + esp_ble_gatts_cb_param_t gatts_param; // Stored inline (32 bytes) + esp_gatts_cb_event_t gatts_event; // 4 bytes + union { + uint8_t *heap_data; // 4 bytes when heap-allocated + uint8_t inline_data[GATTS_INLINE_DATA_SIZE]; // 40 bytes when stored inline + } data; // 40 bytes total + uint16_t data_len; // 2 bytes + esp_gatt_if_t gatts_if; // 1 byte + bool is_inline; // 1 byte - true when data is stored inline + } gatts; // Total: 80 bytes + } event_; // 80 bytes ble_event_t type_; @@ -234,6 +263,29 @@ class BLEEvent { const esp_ble_sec_t &security() const { return event_.gap.security; } private: + // Helper to copy data with inline storage optimization + template + void copy_data_with_inline_storage_(EventStruct &event, const uint8_t *src_data, uint16_t len, + uint8_t **param_value_ptr) { + event.data_len = len; + if (len > 0) { + if (len <= InlineSize) { + event.is_inline = true; + memcpy(event.data.inline_data, src_data, len); + *param_value_ptr = event.data.inline_data; + } else { + event.is_inline = false; + event.data.heap_data = new uint8_t[len]; + memcpy(event.data.heap_data, src_data, len); + *param_value_ptr = event.data.heap_data; + } + } else { + event.is_inline = false; + event.data.heap_data = nullptr; + *param_value_ptr = nullptr; + } + } + // Initialize GAP event data void init_gap_data_(esp_gap_ble_cb_event_t e, esp_ble_gap_cb_param_t *p) { this->event_.gap.gap_event = e; @@ -318,35 +370,38 @@ class BLEEvent { this->event_.gattc.gattc_if = i; if (p == nullptr) { - this->event_.gattc.gattc_param = nullptr; - this->event_.gattc.data = nullptr; + // Zero out the param struct when null + memset(&this->event_.gattc.gattc_param, 0, sizeof(this->event_.gattc.gattc_param)); + this->event_.gattc.is_inline = false; + this->event_.gattc.data.heap_data = nullptr; + this->event_.gattc.data_len = 0; return; // Invalid event, but we can't log in header file } - // Heap-allocate param and data - // Heap allocation is used because GATTC/GATTS events are rare (<1% of events) - // while GAP events (99%) are stored inline to minimize memory usage - // IMPORTANT: This heap allocation provides clear ownership semantics: - // - The BLEEvent owns the allocated memory for its lifetime - // - The data remains valid from the BLE callback context until processed in the main loop - // - Without this copy, we'd have use-after-free bugs as ESP-IDF reuses the callback memory - this->event_.gattc.gattc_param = new esp_ble_gattc_cb_param_t(*p); + // Copy param struct inline (no heap allocation!) + // GATTC/GATTS events are rare (<1% of events) but we can still store them inline + // along with small data payloads, eliminating all heap allocations for typical BLE operations + // CRITICAL: This copy is REQUIRED for memory safety - the ESP-IDF param pointer + // is only valid during the callback and will be reused/freed after we return + this->event_.gattc.gattc_param = *p; // Copy data for events that need it // The param struct contains pointers (e.g., notify.value) that point to temporary buffers. // We must copy this data to ensure it remains valid when the event is processed later. switch (e) { case ESP_GATTC_NOTIFY_EVT: - this->event_.gattc.data = new std::vector(p->notify.value, p->notify.value + p->notify.value_len); - this->event_.gattc.gattc_param->notify.value = this->event_.gattc.data->data(); + copy_data_with_inline_storage_event_.gattc), GATTC_INLINE_DATA_SIZE>( + this->event_.gattc, p->notify.value, p->notify.value_len, &this->event_.gattc.gattc_param.notify.value); break; case ESP_GATTC_READ_CHAR_EVT: case ESP_GATTC_READ_DESCR_EVT: - this->event_.gattc.data = new std::vector(p->read.value, p->read.value + p->read.value_len); - this->event_.gattc.gattc_param->read.value = this->event_.gattc.data->data(); + copy_data_with_inline_storage_event_.gattc), GATTC_INLINE_DATA_SIZE>( + this->event_.gattc, p->read.value, p->read.value_len, &this->event_.gattc.gattc_param.read.value); break; default: - this->event_.gattc.data = nullptr; + this->event_.gattc.is_inline = false; + this->event_.gattc.data.heap_data = nullptr; + this->event_.gattc.data_len = 0; break; } } @@ -357,30 +412,33 @@ class BLEEvent { this->event_.gatts.gatts_if = i; if (p == nullptr) { - this->event_.gatts.gatts_param = nullptr; - this->event_.gatts.data = nullptr; + // Zero out the param struct when null + memset(&this->event_.gatts.gatts_param, 0, sizeof(this->event_.gatts.gatts_param)); + this->event_.gatts.is_inline = false; + this->event_.gatts.data.heap_data = nullptr; + this->event_.gatts.data_len = 0; return; // Invalid event, but we can't log in header file } - // Heap-allocate param and data - // Heap allocation is used because GATTC/GATTS events are rare (<1% of events) - // while GAP events (99%) are stored inline to minimize memory usage - // IMPORTANT: This heap allocation provides clear ownership semantics: - // - The BLEEvent owns the allocated memory for its lifetime - // - The data remains valid from the BLE callback context until processed in the main loop - // - Without this copy, we'd have use-after-free bugs as ESP-IDF reuses the callback memory - this->event_.gatts.gatts_param = new esp_ble_gatts_cb_param_t(*p); + // Copy param struct inline (no heap allocation!) + // GATTC/GATTS events are rare (<1% of events) but we can still store them inline + // along with small data payloads, eliminating all heap allocations for typical BLE operations + // CRITICAL: This copy is REQUIRED for memory safety - the ESP-IDF param pointer + // is only valid during the callback and will be reused/freed after we return + this->event_.gatts.gatts_param = *p; // Copy data for events that need it // The param struct contains pointers (e.g., write.value) that point to temporary buffers. // We must copy this data to ensure it remains valid when the event is processed later. switch (e) { case ESP_GATTS_WRITE_EVT: - this->event_.gatts.data = new std::vector(p->write.value, p->write.value + p->write.len); - this->event_.gatts.gatts_param->write.value = this->event_.gatts.data->data(); + copy_data_with_inline_storage_event_.gatts), GATTS_INLINE_DATA_SIZE>( + this->event_.gatts, p->write.value, p->write.len, &this->event_.gatts.gatts_param.write.value); break; default: - this->event_.gatts.data = nullptr; + this->event_.gatts.is_inline = false; + this->event_.gatts.data.heap_data = nullptr; + this->event_.gatts.data_len = 0; break; } } @@ -390,12 +448,20 @@ class BLEEvent { // The gap member in the union should be 80 bytes (including the gap_event enum) static_assert(sizeof(decltype(((BLEEvent *) nullptr)->event_.gap)) <= 80, "gap_event struct has grown beyond 80 bytes"); +// Verify GATTC and GATTS structs don't exceed GAP struct size +// This ensures the union size is determined by GAP (the most common event type) +static_assert(sizeof(decltype(((BLEEvent *) nullptr)->event_.gattc)) <= + sizeof(decltype(((BLEEvent *) nullptr)->event_.gap)), + "gattc_event struct exceeds gap_event size - union size would increase"); +static_assert(sizeof(decltype(((BLEEvent *) nullptr)->event_.gatts)) <= + sizeof(decltype(((BLEEvent *) nullptr)->event_.gap)), + "gatts_event struct exceeds gap_event size - union size would increase"); + // Verify esp_ble_sec_t fits within our union static_assert(sizeof(esp_ble_sec_t) <= 73, "esp_ble_sec_t is larger than BLEScanResult"); // BLEEvent total size: 84 bytes (80 byte union + 1 byte type + 3 bytes padding) -} // namespace esp32_ble -} // namespace esphome +} // namespace esphome::esp32_ble #endif diff --git a/esphome/components/esp32_ble/ble_scan_result.h b/esphome/components/esp32_ble/ble_scan_result.h index 49b0d5523d..980b39b0b2 100644 --- a/esphome/components/esp32_ble/ble_scan_result.h +++ b/esphome/components/esp32_ble/ble_scan_result.h @@ -4,8 +4,7 @@ #include -namespace esphome { -namespace esp32_ble { +namespace esphome::esp32_ble { // Structure for BLE scan results - only fields we actually use struct __attribute__((packed)) BLEScanResult { @@ -18,7 +17,6 @@ struct __attribute__((packed)) BLEScanResult { uint8_t search_evt; }; // ~73 bytes vs ~400 bytes for full esp_ble_gap_cb_param_t -} // namespace esp32_ble -} // namespace esphome +} // namespace esphome::esp32_ble #endif diff --git a/esphome/components/esp32_ble/ble_uuid.cpp b/esphome/components/esp32_ble/ble_uuid.cpp index aa1edd96b2..dcbb285e07 100644 --- a/esphome/components/esp32_ble/ble_uuid.cpp +++ b/esphome/components/esp32_ble/ble_uuid.cpp @@ -1,14 +1,15 @@ #include "ble_uuid.h" #ifdef USE_ESP32 +#ifdef USE_ESP32_BLE_UUID #include #include #include #include "esphome/core/log.h" +#include "esphome/core/helpers.h" -namespace esphome { -namespace esp32_ble { +namespace esphome::esp32_ble { static const char *const TAG = "esp32_ble"; @@ -41,32 +42,18 @@ ESPBTUUID ESPBTUUID::from_raw_reversed(const uint8_t *data) { ESPBTUUID ESPBTUUID::from_raw(const std::string &data) { ESPBTUUID ret; if (data.length() == 4) { - ret.uuid_.len = ESP_UUID_LEN_16; - ret.uuid_.uuid.uuid16 = 0; - for (uint i = 0; i < data.length(); i += 2) { - uint8_t msb = data.c_str()[i]; - uint8_t lsb = data.c_str()[i + 1]; - uint8_t lsb_shift = i <= 2 ? (2 - i) * 4 : 0; - - if (msb > '9') - msb -= 7; - if (lsb > '9') - lsb -= 7; - ret.uuid_.uuid.uuid16 += (((msb & 0x0F) << 4) | (lsb & 0x0F)) << lsb_shift; + // 16-bit UUID as 4-character hex string + auto parsed = parse_hex(data); + if (parsed.has_value()) { + ret.uuid_.len = ESP_UUID_LEN_16; + ret.uuid_.uuid.uuid16 = parsed.value(); } } else if (data.length() == 8) { - ret.uuid_.len = ESP_UUID_LEN_32; - ret.uuid_.uuid.uuid32 = 0; - for (uint i = 0; i < data.length(); i += 2) { - uint8_t msb = data.c_str()[i]; - uint8_t lsb = data.c_str()[i + 1]; - uint8_t lsb_shift = i <= 6 ? (6 - i) * 4 : 0; - - if (msb > '9') - msb -= 7; - if (lsb > '9') - lsb -= 7; - ret.uuid_.uuid.uuid32 += (((msb & 0x0F) << 4) | (lsb & 0x0F)) << lsb_shift; + // 32-bit UUID as 8-character hex string + auto parsed = parse_hex(data); + if (parsed.has_value()) { + ret.uuid_.len = ESP_UUID_LEN_32; + ret.uuid_.uuid.uuid32 = parsed.value(); } } else if (data.length() == 16) { // how we can have 16 byte length string reprezenting 128 bit uuid??? needs to be // investigated (lack of time) @@ -144,52 +131,60 @@ bool ESPBTUUID::operator==(const ESPBTUUID &uuid) const { if (this->uuid_.len == uuid.uuid_.len) { switch (this->uuid_.len) { case ESP_UUID_LEN_16: - if (uuid.uuid_.uuid.uuid16 == this->uuid_.uuid.uuid16) { - return true; - } - break; + return this->uuid_.uuid.uuid16 == uuid.uuid_.uuid.uuid16; case ESP_UUID_LEN_32: - if (uuid.uuid_.uuid.uuid32 == this->uuid_.uuid.uuid32) { - return true; - } - break; + return this->uuid_.uuid.uuid32 == uuid.uuid_.uuid.uuid32; case ESP_UUID_LEN_128: - for (uint8_t i = 0; i < ESP_UUID_LEN_128; i++) { - if (uuid.uuid_.uuid.uuid128[i] != this->uuid_.uuid.uuid128[i]) { - return false; - } - } - return true; - break; + return memcmp(this->uuid_.uuid.uuid128, uuid.uuid_.uuid.uuid128, ESP_UUID_LEN_128) == 0; + default: + return false; } - } else { - return this->as_128bit() == uuid.as_128bit(); } - return false; + return this->as_128bit() == uuid.as_128bit(); } esp_bt_uuid_t ESPBTUUID::get_uuid() const { return this->uuid_; } std::string ESPBTUUID::to_string() const { + char buf[40]; // Enough for 128-bit UUID with dashes + char *pos = buf; + switch (this->uuid_.len) { case ESP_UUID_LEN_16: - return str_snprintf("0x%02X%02X", 6, this->uuid_.uuid.uuid16 >> 8, this->uuid_.uuid.uuid16 & 0xff); + *pos++ = '0'; + *pos++ = 'x'; + *pos++ = format_hex_pretty_char(this->uuid_.uuid.uuid16 >> 12); + *pos++ = format_hex_pretty_char((this->uuid_.uuid.uuid16 >> 8) & 0x0F); + *pos++ = format_hex_pretty_char((this->uuid_.uuid.uuid16 >> 4) & 0x0F); + *pos++ = format_hex_pretty_char(this->uuid_.uuid.uuid16 & 0x0F); + *pos = '\0'; + return std::string(buf); + case ESP_UUID_LEN_32: - return str_snprintf("0x%02" PRIX32 "%02" PRIX32 "%02" PRIX32 "%02" PRIX32, 10, (this->uuid_.uuid.uuid32 >> 24), - (this->uuid_.uuid.uuid32 >> 16 & 0xff), (this->uuid_.uuid.uuid32 >> 8 & 0xff), - this->uuid_.uuid.uuid32 & 0xff); + *pos++ = '0'; + *pos++ = 'x'; + for (int shift = 28; shift >= 0; shift -= 4) { + *pos++ = format_hex_pretty_char((this->uuid_.uuid.uuid32 >> shift) & 0x0F); + } + *pos = '\0'; + return std::string(buf); + default: case ESP_UUID_LEN_128: - std::string buf; + // Format: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX for (int8_t i = 15; i >= 0; i--) { - buf += str_snprintf("%02X", 2, this->uuid_.uuid.uuid128[i]); - if (i == 6 || i == 8 || i == 10 || i == 12) - buf += "-"; + uint8_t byte = this->uuid_.uuid.uuid128[i]; + *pos++ = format_hex_pretty_char(byte >> 4); + *pos++ = format_hex_pretty_char(byte & 0x0F); + if (i == 12 || i == 10 || i == 8 || i == 6) { + *pos++ = '-'; + } } - return buf; + *pos = '\0'; + return std::string(buf); } return ""; } -} // namespace esp32_ble -} // namespace esphome +} // namespace esphome::esp32_ble -#endif +#endif // USE_ESP32_BLE_UUID +#endif // USE_ESP32 diff --git a/esphome/components/esp32_ble/ble_uuid.h b/esphome/components/esp32_ble/ble_uuid.h index 06f84d4da7..4cf2d10abd 100644 --- a/esphome/components/esp32_ble/ble_uuid.h +++ b/esphome/components/esp32_ble/ble_uuid.h @@ -1,15 +1,16 @@ #pragma once +#include "esphome/core/defines.h" #include "esphome/core/hal.h" #include "esphome/core/helpers.h" #ifdef USE_ESP32 +#ifdef USE_ESP32_BLE_UUID #include #include -namespace esphome { -namespace esp32_ble { +namespace esphome::esp32_ble { class ESPBTUUID { public: @@ -41,7 +42,7 @@ class ESPBTUUID { esp_bt_uuid_t uuid_; }; -} // namespace esp32_ble -} // namespace esphome +} // namespace esphome::esp32_ble -#endif +#endif // USE_ESP32_BLE_UUID +#endif // USE_ESP32 diff --git a/esphome/components/esp32_ble_beacon/__init__.py b/esphome/components/esp32_ble_beacon/__init__.py index 6e0d103aa0..ba5ae4331c 100644 --- a/esphome/components/esp32_ble_beacon/__init__.py +++ b/esphome/components/esp32_ble_beacon/__init__.py @@ -4,7 +4,7 @@ from esphome.components.esp32 import add_idf_sdkconfig_option from esphome.components.esp32_ble import CONF_BLE_ID import esphome.config_validation as cv from esphome.const import CONF_ID, CONF_TX_POWER, CONF_TYPE, CONF_UUID -from esphome.core import CORE, TimePeriod +from esphome.core import TimePeriod AUTO_LOAD = ["esp32_ble"] DEPENDENCIES = ["esp32"] @@ -65,6 +65,8 @@ FINAL_VALIDATE_SCHEMA = esp32_ble.validate_variant async def to_code(config): + cg.add_define("USE_ESP32_BLE_UUID") + uuid = config[CONF_UUID].hex uuid_arr = [ cg.RawExpression(f"0x{uuid[i : i + 2]}") for i in range(0, len(uuid), 2) @@ -72,7 +74,7 @@ async def to_code(config): var = cg.new_Pvariable(config[CONF_ID], uuid_arr) parent = await cg.get_variable(config[esp32_ble.CONF_BLE_ID]) - cg.add(parent.register_gap_event_handler(var)) + esp32_ble.register_gap_event_handler(parent, var) await cg.register_component(var, config) cg.add(var.set_major(config[CONF_MAJOR])) @@ -82,6 +84,7 @@ async def to_code(config): cg.add(var.set_measured_power(config[CONF_MEASURED_POWER])) cg.add(var.set_tx_power(config[CONF_TX_POWER])) - if CORE.using_esp_idf: - add_idf_sdkconfig_option("CONFIG_BT_ENABLED", True) - add_idf_sdkconfig_option("CONFIG_BT_BLE_42_FEATURES_SUPPORTED", True) + cg.add_define("USE_ESP32_BLE_ADVERTISING") + + add_idf_sdkconfig_option("CONFIG_BT_ENABLED", True) + add_idf_sdkconfig_option("CONFIG_BT_BLE_42_FEATURES_SUPPORTED", True) diff --git a/esphome/components/esp32_ble_beacon/esp32_ble_beacon.cpp b/esphome/components/esp32_ble_beacon/esp32_ble_beacon.cpp index 423fe61592..f2aa7e762e 100644 --- a/esphome/components/esp32_ble_beacon/esp32_ble_beacon.cpp +++ b/esphome/components/esp32_ble_beacon/esp32_ble_beacon.cpp @@ -3,7 +3,9 @@ #ifdef USE_ESP32 +#ifndef CONFIG_ESP_HOSTED_ENABLE_BT_BLUEDROID #include +#endif #include #include #include @@ -14,10 +16,6 @@ #include "esphome/core/hal.h" #include "esphome/core/helpers.h" -#ifdef USE_ARDUINO -#include -#endif - namespace esphome { namespace esp32_ble_beacon { @@ -31,12 +29,13 @@ void ESP32BLEBeacon::dump_config() { char uuid[37]; char *bpos = uuid; for (int8_t ii = 0; ii < 16; ++ii) { - bpos += sprintf(bpos, "%02X", this->uuid_[ii]); + *bpos++ = format_hex_pretty_char(this->uuid_[ii] >> 4); + *bpos++ = format_hex_pretty_char(this->uuid_[ii] & 0x0F); if (ii == 3 || ii == 5 || ii == 7 || ii == 9) { - bpos += sprintf(bpos, "-"); + *bpos++ = '-'; } } - uuid[36] = '\0'; + *bpos = '\0'; ESP_LOGCONFIG(TAG, " UUID: %s, Major: %u, Minor: %u, Min Interval: %ums, Max Interval: %ums, Measured Power: %d" ", TX Power: %ddBm", diff --git a/esphome/components/esp32_ble_beacon/esp32_ble_beacon.h b/esphome/components/esp32_ble_beacon/esp32_ble_beacon.h index e37edf6cde..05afdc7379 100644 --- a/esphome/components/esp32_ble_beacon/esp32_ble_beacon.h +++ b/esphome/components/esp32_ble_beacon/esp32_ble_beacon.h @@ -5,7 +5,9 @@ #ifdef USE_ESP32 +#ifndef CONFIG_ESP_HOSTED_ENABLE_BT_BLUEDROID #include +#endif #include namespace esphome { diff --git a/esphome/components/esp32_ble_client/__init__.py b/esphome/components/esp32_ble_client/__init__.py index 25957ed0da..55619f1fc0 100644 --- a/esphome/components/esp32_ble_client/__init__.py +++ b/esphome/components/esp32_ble_client/__init__.py @@ -2,7 +2,7 @@ import esphome.codegen as cg from esphome.components import esp32_ble_tracker AUTO_LOAD = ["esp32_ble_tracker"] -CODEOWNERS = ["@jesserockz"] +CODEOWNERS = ["@jesserockz", "@bdraco"] DEPENDENCIES = ["esp32"] esp32_ble_client_ns = cg.esphome_ns.namespace("esp32_ble_client") diff --git a/esphome/components/esp32_ble_client/ble_characteristic.cpp b/esphome/components/esp32_ble_client/ble_characteristic.cpp index 2fd7fe9871..36229c23c3 100644 --- a/esphome/components/esp32_ble_client/ble_characteristic.cpp +++ b/esphome/components/esp32_ble_client/ble_characteristic.cpp @@ -5,9 +5,9 @@ #include "esphome/core/log.h" #ifdef USE_ESP32 +#ifdef USE_ESP32_BLE_DEVICE -namespace esphome { -namespace esp32_ble_client { +namespace esphome::esp32_ble_client { static const char *const TAG = "esp32_ble_client"; @@ -93,7 +93,7 @@ esp_err_t BLECharacteristic::write_value(uint8_t *new_val, int16_t new_val_size) return write_value(new_val, new_val_size, ESP_GATT_WRITE_TYPE_NO_RSP); } -} // namespace esp32_ble_client -} // namespace esphome +} // namespace esphome::esp32_ble_client +#endif // USE_ESP32_BLE_DEVICE #endif // USE_ESP32 diff --git a/esphome/components/esp32_ble_client/ble_characteristic.h b/esphome/components/esp32_ble_client/ble_characteristic.h index a014788e65..1428b42739 100644 --- a/esphome/components/esp32_ble_client/ble_characteristic.h +++ b/esphome/components/esp32_ble_client/ble_characteristic.h @@ -1,6 +1,9 @@ #pragma once +#include "esphome/core/defines.h" + #ifdef USE_ESP32 +#ifdef USE_ESP32_BLE_DEVICE #include "esphome/components/esp32_ble_tracker/esp32_ble_tracker.h" @@ -8,8 +11,7 @@ #include -namespace esphome { -namespace esp32_ble_client { +namespace esphome::esp32_ble_client { namespace espbt = esphome::esp32_ble_tracker; @@ -33,7 +35,7 @@ class BLECharacteristic { BLEService *service; }; -} // namespace esp32_ble_client -} // namespace esphome +} // namespace esphome::esp32_ble_client +#endif // USE_ESP32_BLE_DEVICE #endif // USE_ESP32 diff --git a/esphome/components/esp32_ble_client/ble_client_base.cpp b/esphome/components/esp32_ble_client/ble_client_base.cpp index 7d0a3bbfd5..18321ef91c 100644 --- a/esphome/components/esp32_ble_client/ble_client_base.cpp +++ b/esphome/components/esp32_ble_client/ble_client_base.cpp @@ -5,10 +5,28 @@ #ifdef USE_ESP32 -namespace esphome { -namespace esp32_ble_client { +#include +#include +#include + +namespace esphome::esp32_ble_client { static const char *const TAG = "esp32_ble_client"; + +// Intermediate connection parameters for standard operation +// ESP-IDF defaults (12.5-15ms) are too slow for stable connections through WiFi-based BLE proxies, +// causing disconnections. These medium parameters balance responsiveness with bandwidth usage. +static const uint16_t MEDIUM_MIN_CONN_INTERVAL = 0x07; // 7 * 1.25ms = 8.75ms +static const uint16_t MEDIUM_MAX_CONN_INTERVAL = 0x09; // 9 * 1.25ms = 11.25ms +// The timeout value was increased from 6s to 8s to address stability issues observed +// in certain BLE devices when operating through WiFi-based BLE proxies. The longer +// timeout reduces the likelihood of disconnections during periods of high latency. +static const uint16_t MEDIUM_CONN_TIMEOUT = 800; // 800 * 10ms = 8s + +// Fastest connection parameters for devices with short discovery timeouts +static const uint16_t FAST_MIN_CONN_INTERVAL = 0x06; // 6 * 1.25ms = 7.5ms (BLE minimum) +static const uint16_t FAST_MAX_CONN_INTERVAL = 0x06; // 6 * 1.25ms = 7.5ms +static const uint16_t FAST_CONN_TIMEOUT = 1000; // 1000 * 10ms = 10s static const esp_bt_uuid_t NOTIFY_DESC_UUID = { .len = ESP_UUID_LEN_16, .uuid = @@ -25,11 +43,6 @@ void BLEClientBase::setup() { void BLEClientBase::set_state(espbt::ClientState st) { ESP_LOGV(TAG, "[%d] [%s] Set state %d", this->connection_index_, this->address_str_.c_str(), (int) st); ESPBTClient::set_state(st); - - if (st == espbt::ClientState::READY_TO_CONNECT) { - // Enable loop when we need to connect - this->enable_loop(); - } } void BLEClientBase::loop() { @@ -45,13 +58,8 @@ void BLEClientBase::loop() { } this->set_state(espbt::ClientState::IDLE); } - // READY_TO_CONNECT means we have discovered the device - // and the scanner has been stopped by the tracker. - else if (this->state_ == espbt::ClientState::READY_TO_CONNECT) { - this->connect(); - } - // If its idle, we can disable the loop as set_state - // will enable it again when we need to connect. + // If idle, we can disable the loop as connect() + // will enable it again when a connection is needed. else if (this->state_ == espbt::ClientState::IDLE) { this->disable_loop(); } @@ -64,40 +72,7 @@ void BLEClientBase::dump_config() { " Address: %s\n" " Auto-Connect: %s", this->address_str().c_str(), TRUEFALSE(this->auto_connect_)); - std::string state_name; - switch (this->state()) { - case espbt::ClientState::INIT: - state_name = "INIT"; - break; - case espbt::ClientState::DISCONNECTING: - state_name = "DISCONNECTING"; - break; - case espbt::ClientState::IDLE: - state_name = "IDLE"; - break; - case espbt::ClientState::SEARCHING: - state_name = "SEARCHING"; - break; - case espbt::ClientState::DISCOVERED: - state_name = "DISCOVERED"; - break; - case espbt::ClientState::READY_TO_CONNECT: - state_name = "READY_TO_CONNECT"; - break; - case espbt::ClientState::CONNECTING: - state_name = "CONNECTING"; - break; - case espbt::ClientState::CONNECTED: - state_name = "CONNECTED"; - break; - case espbt::ClientState::ESTABLISHED: - state_name = "ESTABLISHED"; - break; - default: - state_name = "UNKNOWN_STATE"; - break; - } - ESP_LOGCONFIG(TAG, " State: %s", state_name.c_str()); + ESP_LOGCONFIG(TAG, " State: %s", espbt::client_state_to_string(this->state())); if (this->status_ == ESP_GATT_NO_RESOURCES) { ESP_LOGE(TAG, " Failed due to no resources. Try to reduce number of BLE clients in config."); } else if (this->status_ != ESP_GATT_OK) { @@ -105,12 +80,13 @@ void BLEClientBase::dump_config() { } } +#ifdef USE_ESP32_BLE_DEVICE bool BLEClientBase::parse_device(const espbt::ESPBTDevice &device) { if (!this->auto_connect_) return false; if (this->address_ == 0 || device.address_uint64() != this->address_) return false; - if (this->state_ != espbt::ClientState::IDLE && this->state_ != espbt::ClientState::SEARCHING) + if (this->state_ != espbt::ClientState::IDLE) return false; this->log_event_("Found device"); @@ -122,36 +98,49 @@ bool BLEClientBase::parse_device(const espbt::ESPBTDevice &device) { this->remote_addr_type_ = device.get_address_type(); return true; } +#endif void BLEClientBase::connect() { - ESP_LOGI(TAG, "[%d] [%s] 0x%02x Attempting BLE connection", this->connection_index_, this->address_str_.c_str(), + // Prevent duplicate connection attempts + if (this->state_ == espbt::ClientState::CONNECTING || this->state_ == espbt::ClientState::CONNECTED || + this->state_ == espbt::ClientState::ESTABLISHED) { + ESP_LOGW(TAG, "[%d] [%s] Connection already in progress, state=%s", this->connection_index_, + this->address_str_.c_str(), espbt::client_state_to_string(this->state_)); + return; + } + ESP_LOGI(TAG, "[%d] [%s] 0x%02x Connecting", this->connection_index_, this->address_str_.c_str(), this->remote_addr_type_); this->paired_ = false; - auto ret = esp_ble_gattc_open(this->gattc_if_, this->remote_bda_, this->remote_addr_type_, true); - if (ret) { - ESP_LOGW(TAG, "[%d] [%s] esp_ble_gattc_open error, status=%d", this->connection_index_, this->address_str_.c_str(), - ret); - this->set_state(espbt::ClientState::IDLE); - } else { - this->set_state(espbt::ClientState::CONNECTING); + // Enable loop for state processing + this->enable_loop(); + // Immediately transition to CONNECTING to prevent duplicate connection attempts + this->set_state(espbt::ClientState::CONNECTING); + + // Determine connection parameters based on connection type + if (this->connection_type_ == espbt::ConnectionType::V3_WITHOUT_CACHE) { + // V3 without cache needs fast params for service discovery + this->set_conn_params_(FAST_MIN_CONN_INTERVAL, FAST_MAX_CONN_INTERVAL, 0, FAST_CONN_TIMEOUT, "fast"); + } else if (this->connection_type_ == espbt::ConnectionType::V3_WITH_CACHE) { + // V3 with cache can use medium params + this->set_conn_params_(MEDIUM_MIN_CONN_INTERVAL, MEDIUM_MAX_CONN_INTERVAL, 0, MEDIUM_CONN_TIMEOUT, "medium"); } + // For V1/Legacy, don't set params - use ESP-IDF defaults + + // Open the connection + auto ret = esp_ble_gattc_open(this->gattc_if_, this->remote_bda_, this->remote_addr_type_, true); + this->handle_connection_result_(ret); } esp_err_t BLEClientBase::pair() { return esp_ble_set_encryption(this->remote_bda_, ESP_BLE_SEC_ENCRYPT); } void BLEClientBase::disconnect() { - if (this->state_ == espbt::ClientState::IDLE) { - ESP_LOGI(TAG, "[%d] [%s] Disconnect requested, but already idle.", this->connection_index_, - this->address_str_.c_str()); - return; - } - if (this->state_ == espbt::ClientState::DISCONNECTING) { - ESP_LOGI(TAG, "[%d] [%s] Disconnect requested, but already disconnecting.", this->connection_index_, - this->address_str_.c_str()); + if (this->state_ == espbt::ClientState::IDLE || this->state_ == espbt::ClientState::DISCONNECTING) { + ESP_LOGI(TAG, "[%d] [%s] Disconnect requested, but already %s", this->connection_index_, this->address_str_.c_str(), + espbt::client_state_to_string(this->state_)); return; } if (this->state_ == espbt::ClientState::CONNECTING || this->conn_id_ == UNSET_CONN_ID) { - ESP_LOGW(TAG, "[%d] [%s] Disconnecting before connected, disconnect scheduled.", this->connection_index_, + ESP_LOGD(TAG, "[%d] [%s] Disconnect before connected, disconnect scheduled", this->connection_index_, this->address_str_.c_str()); this->want_disconnect_ = true; return; @@ -164,13 +153,11 @@ void BLEClientBase::unconditional_disconnect() { ESP_LOGI(TAG, "[%d] [%s] Disconnecting (conn_id: %d).", this->connection_index_, this->address_str_.c_str(), this->conn_id_); if (this->state_ == espbt::ClientState::DISCONNECTING) { - ESP_LOGE(TAG, "[%d] [%s] Tried to disconnect while already disconnecting.", this->connection_index_, - this->address_str_.c_str()); + this->log_error_("Already disconnecting"); return; } if (this->conn_id_ == UNSET_CONN_ID) { - ESP_LOGE(TAG, "[%d] [%s] No connection ID set, cannot disconnect.", this->connection_index_, - this->address_str_.c_str()); + this->log_error_("conn id unset, cannot disconnect"); return; } auto err = esp_ble_gattc_close(this->gattc_if_, this->conn_id_); @@ -182,12 +169,10 @@ void BLEClientBase::unconditional_disconnect() { // In the future we might consider App.reboot() here since // the BLE stack is in an indeterminate state. // - ESP_LOGE(TAG, "[%d] [%s] esp_ble_gattc_close error, err=%d", this->connection_index_, this->address_str_.c_str(), - err); + this->log_gattc_warning_("esp_ble_gattc_close", err); } - if (this->state_ == espbt::ClientState::SEARCHING || this->state_ == espbt::ClientState::READY_TO_CONNECT || - this->state_ == espbt::ClientState::DISCOVERED) { + if (this->state_ == espbt::ClientState::DISCOVERED) { this->set_address(0); this->set_state(espbt::ClientState::IDLE); } else { @@ -196,9 +181,11 @@ void BLEClientBase::unconditional_disconnect() { } void BLEClientBase::release_services() { +#ifdef USE_ESP32_BLE_DEVICE for (auto &svc : this->services_) delete svc; // NOLINT(cppcoreguidelines-owning-memory) this->services_.clear(); +#endif #ifndef CONFIG_BT_GATTC_CACHE_NVS_FLASH esp_ble_gattc_cache_clean(this->remote_bda_); #endif @@ -208,6 +195,68 @@ void BLEClientBase::log_event_(const char *name) { ESP_LOGD(TAG, "[%d] [%s] %s", this->connection_index_, this->address_str_.c_str(), name); } +void BLEClientBase::log_gattc_event_(const char *name) { + ESP_LOGD(TAG, "[%d] [%s] ESP_GATTC_%s_EVT", this->connection_index_, this->address_str_.c_str(), name); +} + +void BLEClientBase::log_gattc_warning_(const char *operation, esp_gatt_status_t status) { + ESP_LOGW(TAG, "[%d] [%s] %s error, status=%d", this->connection_index_, this->address_str_.c_str(), operation, + status); +} + +void BLEClientBase::log_gattc_warning_(const char *operation, esp_err_t err) { + ESP_LOGW(TAG, "[%d] [%s] %s error, status=%d", this->connection_index_, this->address_str_.c_str(), operation, err); +} + +void BLEClientBase::log_connection_params_(const char *param_type) { + ESP_LOGD(TAG, "[%d] [%s] %s conn params", this->connection_index_, this->address_str_.c_str(), param_type); +} + +void BLEClientBase::handle_connection_result_(esp_err_t ret) { + if (ret) { + this->log_gattc_warning_("esp_ble_gattc_open", ret); + this->set_state(espbt::ClientState::IDLE); + } +} + +void BLEClientBase::log_error_(const char *message) { + ESP_LOGE(TAG, "[%d] [%s] %s", this->connection_index_, this->address_str_.c_str(), message); +} + +void BLEClientBase::log_error_(const char *message, int code) { + ESP_LOGE(TAG, "[%d] [%s] %s=%d", this->connection_index_, this->address_str_.c_str(), message, code); +} + +void BLEClientBase::log_warning_(const char *message) { + ESP_LOGW(TAG, "[%d] [%s] %s", this->connection_index_, this->address_str_.c_str(), message); +} + +void BLEClientBase::update_conn_params_(uint16_t min_interval, uint16_t max_interval, uint16_t latency, + uint16_t timeout, const char *param_type) { + esp_ble_conn_update_params_t conn_params = {{0}}; + memcpy(conn_params.bda, this->remote_bda_, sizeof(esp_bd_addr_t)); + conn_params.min_int = min_interval; + conn_params.max_int = max_interval; + conn_params.latency = latency; + conn_params.timeout = timeout; + this->log_connection_params_(param_type); + esp_err_t err = esp_ble_gap_update_conn_params(&conn_params); + if (err != ESP_OK) { + this->log_gattc_warning_("esp_ble_gap_update_conn_params", err); + } +} + +void BLEClientBase::set_conn_params_(uint16_t min_interval, uint16_t max_interval, uint16_t latency, uint16_t timeout, + const char *param_type) { + // Set preferred connection parameters before connecting + // These will be used when establishing the connection + this->log_connection_params_(param_type); + esp_err_t err = esp_ble_gap_set_prefer_conn_params(this->remote_bda_, min_interval, max_interval, latency, timeout); + if (err != ESP_OK) { + this->log_gattc_warning_("esp_ble_gap_set_prefer_conn_params", err); + } +} + bool BLEClientBase::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t esp_gattc_if, esp_ble_gattc_cb_param_t *param) { if (event == ESP_GATTC_REG_EVT && this->app_id != param->reg.app_id) @@ -225,8 +274,7 @@ bool BLEClientBase::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_ this->app_id); this->gattc_if_ = esp_gattc_if; } else { - ESP_LOGE(TAG, "[%d] [%s] gattc app registration failed id=%d code=%d", this->connection_index_, - this->address_str_.c_str(), param->reg.app_id, param->reg.status); + this->log_error_("gattc app registration failed status", param->reg.status); this->status_ = param->reg.status; this->mark_failed(); } @@ -235,30 +283,28 @@ bool BLEClientBase::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_ case ESP_GATTC_OPEN_EVT: { if (!this->check_addr(param->open.remote_bda)) return false; - this->log_event_("ESP_GATTC_OPEN_EVT"); - this->conn_id_ = param->open.conn_id; + this->log_gattc_event_("OPEN"); + // conn_id was already set in ESP_GATTC_CONNECT_EVT this->service_count_ = 0; + + // ESP-IDF's BLE stack may send ESP_GATTC_OPEN_EVT after esp_ble_gattc_open() returns an + // error, if the error occurred at the BTA/GATT layer. This can result in the event + // arriving after we've already transitioned to IDLE state. + if (this->state_ == espbt::ClientState::IDLE) { + ESP_LOGD(TAG, "[%d] [%s] ESP_GATTC_OPEN_EVT in IDLE state (status=%d), ignoring", this->connection_index_, + this->address_str_.c_str(), param->open.status); + break; + } + if (this->state_ != espbt::ClientState::CONNECTING) { // This should not happen but lets log it in case it does // because it means we have a bad assumption about how the // ESP BT stack works. - if (this->state_ == espbt::ClientState::CONNECTED) { - ESP_LOGE(TAG, "[%d] [%s] Got ESP_GATTC_OPEN_EVT while already connected, status=%d", this->connection_index_, - this->address_str_.c_str(), param->open.status); - } else if (this->state_ == espbt::ClientState::ESTABLISHED) { - ESP_LOGE(TAG, "[%d] [%s] Got ESP_GATTC_OPEN_EVT while already established, status=%d", - this->connection_index_, this->address_str_.c_str(), param->open.status); - } else if (this->state_ == espbt::ClientState::DISCONNECTING) { - ESP_LOGE(TAG, "[%d] [%s] Got ESP_GATTC_OPEN_EVT while disconnecting, status=%d", this->connection_index_, - this->address_str_.c_str(), param->open.status); - } else { - ESP_LOGE(TAG, "[%d] [%s] Got ESP_GATTC_OPEN_EVT while not in connecting state, status=%d", - this->connection_index_, this->address_str_.c_str(), param->open.status); - } + ESP_LOGE(TAG, "[%d] [%s] ESP_GATTC_OPEN_EVT in %s state (status=%d)", this->connection_index_, + this->address_str_.c_str(), espbt::client_state_to_string(this->state_), param->open.status); } if (param->open.status != ESP_GATT_OK && param->open.status != ESP_GATT_ALREADY_OPEN) { - ESP_LOGW(TAG, "[%d] [%s] Connection failed, status=%d", this->connection_index_, this->address_str_.c_str(), - param->open.status); + this->log_gattc_warning_("Connection open", param->open.status); this->set_state(espbt::ClientState::IDLE); break; } @@ -270,32 +316,47 @@ bool BLEClientBase::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_ this->conn_id_ = UNSET_CONN_ID; break; } - auto ret = esp_ble_gattc_send_mtu_req(this->gattc_if_, param->open.conn_id); - if (ret) { - ESP_LOGW(TAG, "[%d] [%s] esp_ble_gattc_send_mtu_req failed, status=%x", this->connection_index_, - this->address_str_.c_str(), ret); - } + // MTU negotiation already started in ESP_GATTC_CONNECT_EVT this->set_state(espbt::ClientState::CONNECTED); + ESP_LOGI(TAG, "[%d] [%s] Connection open", this->connection_index_, this->address_str_.c_str()); if (this->connection_type_ == espbt::ConnectionType::V3_WITH_CACHE) { - ESP_LOGI(TAG, "[%d] [%s] Connected", this->connection_index_, this->address_str_.c_str()); + // Cached connections already connected with medium parameters, no update needed // only set our state, subclients might have more stuff to do yet. this->state_ = espbt::ClientState::ESTABLISHED; break; } + // For V3_WITHOUT_CACHE, we already set fast params before connecting + // No need to update them again here + this->log_event_("Searching for services"); esp_ble_gattc_search_service(esp_gattc_if, param->cfg_mtu.conn_id, nullptr); break; } case ESP_GATTC_CONNECT_EVT: { if (!this->check_addr(param->connect.remote_bda)) return false; - this->log_event_("ESP_GATTC_CONNECT_EVT"); + this->log_gattc_event_("CONNECT"); + this->conn_id_ = param->connect.conn_id; + // Start MTU negotiation immediately as recommended by ESP-IDF examples + // (gatt_client, ble_throughput) which call esp_ble_gattc_send_mtu_req in + // ESP_GATTC_CONNECT_EVT instead of waiting for ESP_GATTC_OPEN_EVT. + // This saves ~3ms in the connection process. + auto ret = esp_ble_gattc_send_mtu_req(this->gattc_if_, param->connect.conn_id); + if (ret) { + this->log_gattc_warning_("esp_ble_gattc_send_mtu_req", ret); + } break; } case ESP_GATTC_DISCONNECT_EVT: { if (!this->check_addr(param->disconnect.remote_bda)) return false; - ESP_LOGD(TAG, "[%d] [%s] ESP_GATTC_DISCONNECT_EVT, reason %d", this->connection_index_, - this->address_str_.c_str(), param->disconnect.reason); + // Check if we were disconnected while waiting for service discovery + if (param->disconnect.reason == ESP_GATT_CONN_TERMINATE_PEER_USER && + this->state_ == espbt::ClientState::CONNECTED) { + this->log_warning_("Remote closed during discovery"); + } else { + ESP_LOGD(TAG, "[%d] [%s] ESP_GATTC_DISCONNECT_EVT, reason 0x%02x", this->connection_index_, + this->address_str_.c_str(), param->disconnect.reason); + } this->release_services(); this->set_state(espbt::ClientState::IDLE); break; @@ -318,7 +379,7 @@ bool BLEClientBase::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_ case ESP_GATTC_CLOSE_EVT: { if (this->conn_id_ != param->close.conn_id) return false; - this->log_event_("ESP_GATTC_CLOSE_EVT"); + this->log_gattc_event_("CLOSE"); this->release_services(); this->set_state(espbt::ClientState::IDLE); this->conn_id_ = UNSET_CONN_ID; @@ -330,63 +391,73 @@ bool BLEClientBase::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_ this->service_count_++; if (this->connection_type_ == espbt::ConnectionType::V3_WITHOUT_CACHE) { // V3 clients don't need services initialized since - // they only request by handle after receiving the services. + // as they use the ESP APIs to get services. break; } +#ifdef USE_ESP32_BLE_DEVICE BLEService *ble_service = new BLEService(); // NOLINT(cppcoreguidelines-owning-memory) ble_service->uuid = espbt::ESPBTUUID::from_uuid(param->search_res.srvc_id.uuid); ble_service->start_handle = param->search_res.start_handle; ble_service->end_handle = param->search_res.end_handle; ble_service->client = this; this->services_.push_back(ble_service); +#endif break; } case ESP_GATTC_SEARCH_CMPL_EVT: { if (this->conn_id_ != param->search_cmpl.conn_id) return false; - this->log_event_("ESP_GATTC_SEARCH_CMPL_EVT"); - for (auto &svc : this->services_) { - ESP_LOGV(TAG, "[%d] [%s] Service UUID: %s", this->connection_index_, this->address_str_.c_str(), - svc->uuid.to_string().c_str()); - ESP_LOGV(TAG, "[%d] [%s] start_handle: 0x%x end_handle: 0x%x", this->connection_index_, - this->address_str_.c_str(), svc->start_handle, svc->end_handle); + this->log_gattc_event_("SEARCH_CMPL"); + // For V3_WITHOUT_CACHE, switch back to medium connection parameters after service discovery + // This balances performance with bandwidth usage after the critical discovery phase + if (this->connection_type_ == espbt::ConnectionType::V3_WITHOUT_CACHE) { + this->update_conn_params_(MEDIUM_MIN_CONN_INTERVAL, MEDIUM_MAX_CONN_INTERVAL, 0, MEDIUM_CONN_TIMEOUT, "medium"); + } else if (this->connection_type_ != espbt::ConnectionType::V3_WITH_CACHE) { +#ifdef USE_ESP32_BLE_DEVICE + for (auto &svc : this->services_) { + ESP_LOGV(TAG, "[%d] [%s] Service UUID: %s", this->connection_index_, this->address_str_.c_str(), + svc->uuid.to_string().c_str()); + ESP_LOGV(TAG, "[%d] [%s] start_handle: 0x%x end_handle: 0x%x", this->connection_index_, + this->address_str_.c_str(), svc->start_handle, svc->end_handle); + } +#endif } - ESP_LOGI(TAG, "[%d] [%s] Connected", this->connection_index_, this->address_str_.c_str()); + ESP_LOGI(TAG, "[%d] [%s] Service discovery complete", this->connection_index_, this->address_str_.c_str()); this->state_ = espbt::ClientState::ESTABLISHED; break; } case ESP_GATTC_READ_DESCR_EVT: { if (this->conn_id_ != param->write.conn_id) return false; - this->log_event_("ESP_GATTC_READ_DESCR_EVT"); + this->log_gattc_event_("READ_DESCR"); break; } case ESP_GATTC_WRITE_DESCR_EVT: { if (this->conn_id_ != param->write.conn_id) return false; - this->log_event_("ESP_GATTC_WRITE_DESCR_EVT"); + this->log_gattc_event_("WRITE_DESCR"); break; } case ESP_GATTC_WRITE_CHAR_EVT: { if (this->conn_id_ != param->write.conn_id) return false; - this->log_event_("ESP_GATTC_WRITE_CHAR_EVT"); + this->log_gattc_event_("WRITE_CHAR"); break; } case ESP_GATTC_READ_CHAR_EVT: { if (this->conn_id_ != param->read.conn_id) return false; - this->log_event_("ESP_GATTC_READ_CHAR_EVT"); + this->log_gattc_event_("READ_CHAR"); break; } case ESP_GATTC_NOTIFY_EVT: { if (this->conn_id_ != param->notify.conn_id) return false; - this->log_event_("ESP_GATTC_NOTIFY_EVT"); + this->log_gattc_event_("NOTIFY"); break; } case ESP_GATTC_REG_FOR_NOTIFY_EVT: { - this->log_event_("ESP_GATTC_REG_FOR_NOTIFY_EVT"); + this->log_gattc_event_("REG_FOR_NOTIFY"); if (this->connection_type_ == espbt::ConnectionType::V3_WITH_CACHE || this->connection_type_ == espbt::ConnectionType::V3_WITHOUT_CACHE) { // Client is responsible for flipping the descriptor value @@ -398,8 +469,7 @@ bool BLEClientBase::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_ esp_gatt_status_t descr_status = esp_ble_gattc_get_descr_by_char_handle( this->gattc_if_, this->conn_id_, param->reg_for_notify.handle, NOTIFY_DESC_UUID, &desc_result, &count); if (descr_status != ESP_GATT_OK) { - ESP_LOGW(TAG, "[%d] [%s] esp_ble_gattc_get_descr_by_char_handle error, status=%d", this->connection_index_, - this->address_str_.c_str(), descr_status); + this->log_gattc_warning_("esp_ble_gattc_get_descr_by_char_handle", descr_status); break; } esp_gattc_char_elem_t char_result; @@ -407,8 +477,7 @@ bool BLEClientBase::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_ esp_ble_gattc_get_all_char(this->gattc_if_, this->conn_id_, param->reg_for_notify.handle, param->reg_for_notify.handle, &char_result, &count, 0); if (char_status != ESP_GATT_OK) { - ESP_LOGW(TAG, "[%d] [%s] esp_ble_gattc_get_all_char error, status=%d", this->connection_index_, - this->address_str_.c_str(), char_status); + this->log_gattc_warning_("esp_ble_gattc_get_all_char", char_status); break; } @@ -422,12 +491,16 @@ bool BLEClientBase::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_ (uint8_t *) ¬ify_en, ESP_GATT_WRITE_TYPE_RSP, ESP_GATT_AUTH_REQ_NONE); ESP_LOGD(TAG, "Wrote notify descriptor %d, properties=%d", notify_en, char_result.properties); if (status) { - ESP_LOGW(TAG, "[%d] [%s] esp_ble_gattc_write_char_descr error, status=%d", this->connection_index_, - this->address_str_.c_str(), status); + this->log_gattc_warning_("esp_ble_gattc_write_char_descr", status); } break; } + case ESP_GATTC_UNREG_FOR_NOTIFY_EVT: { + this->log_gattc_event_("UNREG_FOR_NOTIFY"); + break; + } + default: // ideally would check all other events for matching conn_id ESP_LOGD(TAG, "[%d] [%s] Event %d", this->connection_index_, this->address_str_.c_str(), event); @@ -456,16 +529,14 @@ void BLEClientBase::gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_ return; esp_bd_addr_t bd_addr; memcpy(bd_addr, param->ble_security.auth_cmpl.bd_addr, sizeof(esp_bd_addr_t)); - ESP_LOGI(TAG, "[%d] [%s] auth complete. remote BD_ADDR: %s", this->connection_index_, this->address_str_.c_str(), + ESP_LOGI(TAG, "[%d] [%s] auth complete addr: %s", this->connection_index_, this->address_str_.c_str(), format_hex(bd_addr, 6).c_str()); if (!param->ble_security.auth_cmpl.success) { - ESP_LOGE(TAG, "[%d] [%s] auth fail reason = 0x%x", this->connection_index_, this->address_str_.c_str(), - param->ble_security.auth_cmpl.fail_reason); + this->log_error_("auth fail reason", param->ble_security.auth_cmpl.fail_reason); } else { this->paired_ = true; - ESP_LOGD(TAG, "[%d] [%s] auth success. address type = %d auth mode = %d", this->connection_index_, - this->address_str_.c_str(), param->ble_security.auth_cmpl.addr_type, - param->ble_security.auth_cmpl.auth_mode); + ESP_LOGD(TAG, "[%d] [%s] auth success type = %d mode = %d", this->connection_index_, this->address_str_.c_str(), + param->ble_security.auth_cmpl.addr_type, param->ble_security.auth_cmpl.auth_mode); } break; @@ -531,6 +602,7 @@ float BLEClientBase::parse_char_value(uint8_t *value, uint16_t length) { return NAN; } +#ifdef USE_ESP32_BLE_DEVICE BLEService *BLEClientBase::get_service(espbt::ESPBTUUID uuid) { for (auto *svc : this->services_) { if (svc->uuid == uuid) @@ -607,8 +679,8 @@ BLEDescriptor *BLEClientBase::get_descriptor(uint16_t handle) { } return nullptr; } +#endif // USE_ESP32_BLE_DEVICE -} // namespace esp32_ble_client -} // namespace esphome +} // namespace esphome::esp32_ble_client #endif // USE_ESP32 diff --git a/esphome/components/esp32_ble_client/ble_client_base.h b/esphome/components/esp32_ble_client/ble_client_base.h index bf3b589b1b..f2edd6c2b3 100644 --- a/esphome/components/esp32_ble_client/ble_client_base.h +++ b/esphome/components/esp32_ble_client/ble_client_base.h @@ -5,7 +5,9 @@ #include "esphome/components/esp32_ble_tracker/esp32_ble_tracker.h" #include "esphome/core/component.h" +#ifdef USE_ESP32_BLE_DEVICE #include "ble_service.h" +#endif #include #include @@ -16,8 +18,7 @@ #include #include -namespace esphome { -namespace esp32_ble_client { +namespace esphome::esp32_ble_client { namespace espbt = esphome::esp32_ble_tracker; @@ -31,7 +32,9 @@ class BLEClientBase : public espbt::ESPBTClient, public Component { void dump_config() override; void run_later(std::function &&f); // NOLINT +#ifdef USE_ESP32_BLE_DEVICE bool parse_device(const espbt::ESPBTDevice &device) override; +#endif void on_scan_end() override {} bool gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gattc_cb_param_t *param) override; @@ -46,7 +49,7 @@ class BLEClientBase : public espbt::ESPBTClient, public Component { void set_auto_connect(bool auto_connect) { this->auto_connect_ = auto_connect; } - void set_address(uint64_t address) { + virtual void set_address(uint64_t address) { this->address_ = address; this->remote_bda_[0] = (address >> 40) & 0xFF; this->remote_bda_[1] = (address >> 32) & 0xFF; @@ -57,15 +60,19 @@ class BLEClientBase : public espbt::ESPBTClient, public Component { if (address == 0) { this->address_str_ = ""; } else { - this->address_str_ = - str_snprintf("%02X:%02X:%02X:%02X:%02X:%02X", 17, (uint8_t) (this->address_ >> 40) & 0xff, - (uint8_t) (this->address_ >> 32) & 0xff, (uint8_t) (this->address_ >> 24) & 0xff, - (uint8_t) (this->address_ >> 16) & 0xff, (uint8_t) (this->address_ >> 8) & 0xff, - (uint8_t) (this->address_ >> 0) & 0xff); + char buf[18]; + uint8_t mac[6] = { + (uint8_t) ((this->address_ >> 40) & 0xff), (uint8_t) ((this->address_ >> 32) & 0xff), + (uint8_t) ((this->address_ >> 24) & 0xff), (uint8_t) ((this->address_ >> 16) & 0xff), + (uint8_t) ((this->address_ >> 8) & 0xff), (uint8_t) ((this->address_ >> 0) & 0xff), + }; + format_mac_addr_upper(mac, buf); + this->address_str_ = buf; } } - std::string address_str() const { return this->address_str_; } + const std::string &address_str() const { return this->address_str_; } +#ifdef USE_ESP32_BLE_DEVICE BLEService *get_service(espbt::ESPBTUUID uuid); BLEService *get_service(uint16_t uuid); BLECharacteristic *get_characteristic(espbt::ESPBTUUID service, espbt::ESPBTUUID chr); @@ -76,6 +83,7 @@ class BLEClientBase : public espbt::ESPBTClient, public Component { BLEDescriptor *get_descriptor(uint16_t handle); // Get the configuration descriptor for the given characteristic handle. BLEDescriptor *get_config_descriptor(uint16_t handle); +#endif float parse_char_value(uint8_t *value, uint16_t length); @@ -102,7 +110,9 @@ class BLEClientBase : public espbt::ESPBTClient, public Component { // Group 2: Container types (grouped for memory optimization) std::string address_str_{}; +#ifdef USE_ESP32_BLE_DEVICE std::vector services_; +#endif // Group 3: 4-byte types int gattc_if_; @@ -125,9 +135,21 @@ class BLEClientBase : public espbt::ESPBTClient, public Component { // 6 bytes used, 2 bytes padding void log_event_(const char *name); + void log_gattc_event_(const char *name); + void update_conn_params_(uint16_t min_interval, uint16_t max_interval, uint16_t latency, uint16_t timeout, + const char *param_type); + void set_conn_params_(uint16_t min_interval, uint16_t max_interval, uint16_t latency, uint16_t timeout, + const char *param_type); + void log_gattc_warning_(const char *operation, esp_gatt_status_t status); + void log_gattc_warning_(const char *operation, esp_err_t err); + void log_connection_params_(const char *param_type); + void handle_connection_result_(esp_err_t ret); + // Compact error logging helpers to reduce flash usage + void log_error_(const char *message); + void log_error_(const char *message, int code); + void log_warning_(const char *message); }; -} // namespace esp32_ble_client -} // namespace esphome +} // namespace esphome::esp32_ble_client #endif // USE_ESP32 diff --git a/esphome/components/esp32_ble_client/ble_descriptor.h b/esphome/components/esp32_ble_client/ble_descriptor.h index c05430144f..fb2b78a7b1 100644 --- a/esphome/components/esp32_ble_client/ble_descriptor.h +++ b/esphome/components/esp32_ble_client/ble_descriptor.h @@ -1,11 +1,13 @@ #pragma once +#include "esphome/core/defines.h" + #ifdef USE_ESP32 +#ifdef USE_ESP32_BLE_DEVICE #include "esphome/components/esp32_ble_tracker/esp32_ble_tracker.h" -namespace esphome { -namespace esp32_ble_client { +namespace esphome::esp32_ble_client { namespace espbt = esphome::esp32_ble_tracker; @@ -19,7 +21,7 @@ class BLEDescriptor { BLECharacteristic *characteristic; }; -} // namespace esp32_ble_client -} // namespace esphome +} // namespace esphome::esp32_ble_client +#endif // USE_ESP32_BLE_DEVICE #endif // USE_ESP32 diff --git a/esphome/components/esp32_ble_client/ble_service.cpp b/esphome/components/esp32_ble_client/ble_service.cpp index b22d2a1788..accaad15e1 100644 --- a/esphome/components/esp32_ble_client/ble_service.cpp +++ b/esphome/components/esp32_ble_client/ble_service.cpp @@ -4,9 +4,9 @@ #include "esphome/core/log.h" #ifdef USE_ESP32 +#ifdef USE_ESP32_BLE_DEVICE -namespace esphome { -namespace esp32_ble_client { +namespace esphome::esp32_ble_client { static const char *const TAG = "esp32_ble_client"; @@ -71,7 +71,7 @@ void BLEService::parse_characteristics() { } } -} // namespace esp32_ble_client -} // namespace esphome +} // namespace esphome::esp32_ble_client +#endif // USE_ESP32_BLE_DEVICE #endif // USE_ESP32 diff --git a/esphome/components/esp32_ble_client/ble_service.h b/esphome/components/esp32_ble_client/ble_service.h index 41fc3e838b..00ecc777e7 100644 --- a/esphome/components/esp32_ble_client/ble_service.h +++ b/esphome/components/esp32_ble_client/ble_service.h @@ -1,6 +1,9 @@ #pragma once +#include "esphome/core/defines.h" + #ifdef USE_ESP32 +#ifdef USE_ESP32_BLE_DEVICE #include "esphome/components/esp32_ble_tracker/esp32_ble_tracker.h" @@ -8,8 +11,7 @@ #include -namespace esphome { -namespace esp32_ble_client { +namespace esphome::esp32_ble_client { namespace espbt = esphome::esp32_ble_tracker; @@ -30,7 +32,7 @@ class BLEService { BLECharacteristic *get_characteristic(uint16_t uuid); }; -} // namespace esp32_ble_client -} // namespace esphome +} // namespace esphome::esp32_ble_client +#endif // USE_ESP32_BLE_DEVICE #endif // USE_ESP32 diff --git a/esphome/components/esp32_ble_server/__init__.py b/esphome/components/esp32_ble_server/__init__.py index 773445a1a7..55310f3275 100644 --- a/esphome/components/esp32_ble_server/__init__.py +++ b/esphome/components/esp32_ble_server/__init__.py @@ -26,7 +26,7 @@ from esphome.const import ( from esphome.core import CORE from esphome.schema_extractors import SCHEMA_EXTRACT -AUTO_LOAD = ["esp32_ble", "bytebuffer", "event_emitter"] +AUTO_LOAD = ["esp32_ble", "bytebuffer"] CODEOWNERS = ["@jesserockz", "@clydebarrow", "@Rapsssito"] DEPENDENCIES = ["esp32"] DOMAIN = "esp32_ble_server" @@ -140,20 +140,22 @@ VALUE_TYPES = { def validate_char_on_write(char_config): - if CONF_ON_WRITE in char_config: - if not char_config[CONF_WRITE] and not char_config[CONF_WRITE_NO_RESPONSE]: - raise cv.Invalid( - f"{CONF_ON_WRITE} requires the {CONF_WRITE} or {CONF_WRITE_NO_RESPONSE} property to be set" - ) + if ( + CONF_ON_WRITE in char_config + and not char_config[CONF_WRITE] + and not char_config[CONF_WRITE_NO_RESPONSE] + ): + raise cv.Invalid( + f"{CONF_ON_WRITE} requires the {CONF_WRITE} or {CONF_WRITE_NO_RESPONSE} property to be set" + ) return char_config def validate_descriptor(desc_config): - if CONF_ON_WRITE in desc_config: - if not desc_config[CONF_WRITE]: - raise cv.Invalid( - f"{CONF_ON_WRITE} requires the {CONF_WRITE} property to be set" - ) + if CONF_ON_WRITE in desc_config and not desc_config[CONF_WRITE]: + raise cv.Invalid( + f"{CONF_ON_WRITE} requires the {CONF_WRITE} property to be set" + ) if CONF_MAX_LENGTH not in desc_config: value = desc_config[CONF_VALUE][CONF_DATA] if cg.is_template(value): @@ -486,6 +488,7 @@ async def to_code_descriptor(descriptor_conf, char_var): cg.add(desc_var.set_value(value)) if CONF_ON_WRITE in descriptor_conf: on_write_conf = descriptor_conf[CONF_ON_WRITE] + cg.add_define("USE_ESP32_BLE_SERVER_DESCRIPTOR_ON_WRITE") await automation.build_automation( BLETriggers_ns.create_descriptor_on_write_trigger(desc_var), [(cg.std_vector.template(cg.uint8), "x"), (cg.uint16, "id")], @@ -503,23 +506,32 @@ async def to_code_characteristic(service_var, char_conf): ) if CONF_ON_WRITE in char_conf: on_write_conf = char_conf[CONF_ON_WRITE] + cg.add_define("USE_ESP32_BLE_SERVER_CHARACTERISTIC_ON_WRITE") await automation.build_automation( BLETriggers_ns.create_characteristic_on_write_trigger(char_var), [(cg.std_vector.template(cg.uint8), "x"), (cg.uint16, "id")], on_write_conf, ) if CONF_VALUE in char_conf: - action_conf = { - CONF_ID: char_conf[CONF_ID], - CONF_VALUE: char_conf[CONF_VALUE], - } - value_action = await ble_server_characteristic_set_value( - action_conf, - char_conf[CONF_CHAR_VALUE_ACTION_ID_], - cg.TemplateArguments(), - {}, - ) - cg.add(value_action.play()) + # Check if the value is templated (Lambda) + value_data = char_conf[CONF_VALUE][CONF_DATA] + if isinstance(value_data, cv.Lambda): + # Templated value - need the full action infrastructure + action_conf = { + CONF_ID: char_conf[CONF_ID], + CONF_VALUE: char_conf[CONF_VALUE], + } + value_action = await ble_server_characteristic_set_value( + action_conf, + char_conf[CONF_CHAR_VALUE_ACTION_ID_], + cg.TemplateArguments(), + {}, + ) + cg.add(value_action.play()) + else: + # Static value - just set it directly without action infrastructure + value = await parse_value(char_conf[CONF_VALUE], {}) + cg.add(char_var.set_value(value)) for descriptor_conf in char_conf[CONF_DESCRIPTORS]: await to_code_descriptor(descriptor_conf, char_var) @@ -527,14 +539,15 @@ async def to_code_characteristic(service_var, char_conf): async def to_code(config): # Register the loggers this component needs esp32_ble.register_bt_logger(BTLoggers.GATT, BTLoggers.SMP) + cg.add_define("USE_ESP32_BLE_UUID") var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) parent = await cg.get_variable(config[esp32_ble.CONF_BLE_ID]) - cg.add(parent.register_gatts_event_handler(var)) - cg.add(parent.register_ble_status_event_handler(var)) + esp32_ble.register_gatts_event_handler(parent, var) + esp32_ble.register_ble_status_event_handler(parent, var) cg.add(var.set_parent(parent)) cg.add(parent.advertising_set_appearance(config[CONF_APPEARANCE])) if CONF_MANUFACTURER_DATA in config: @@ -557,20 +570,22 @@ async def to_code(config): else: cg.add(var.enqueue_start_service(service_var)) if CONF_ON_CONNECT in config: + cg.add_define("USE_ESP32_BLE_SERVER_ON_CONNECT") await automation.build_automation( BLETriggers_ns.create_server_on_connect_trigger(var), [(cg.uint16, "id")], config[CONF_ON_CONNECT], ) if CONF_ON_DISCONNECT in config: + cg.add_define("USE_ESP32_BLE_SERVER_ON_DISCONNECT") await automation.build_automation( BLETriggers_ns.create_server_on_disconnect_trigger(var), [(cg.uint16, "id")], config[CONF_ON_DISCONNECT], ) cg.add_define("USE_ESP32_BLE_SERVER") - if CORE.using_esp_idf: - add_idf_sdkconfig_option("CONFIG_BT_ENABLED", True) + cg.add_define("USE_ESP32_BLE_ADVERTISING") + add_idf_sdkconfig_option("CONFIG_BT_ENABLED", True) @automation.register_action( @@ -591,6 +606,7 @@ async def ble_server_characteristic_set_value(config, action_id, template_arg, a var = cg.new_Pvariable(action_id, template_arg, paren) value = await parse_value(config[CONF_VALUE], args) cg.add(var.set_buffer(value)) + cg.add_define("USE_ESP32_BLE_SERVER_SET_VALUE_ACTION") return var @@ -609,6 +625,7 @@ async def ble_server_descriptor_set_value(config, action_id, template_arg, args) var = cg.new_Pvariable(action_id, template_arg, paren) value = await parse_value(config[CONF_VALUE], args) cg.add(var.set_buffer(value)) + cg.add_define("USE_ESP32_BLE_SERVER_DESCRIPTOR_SET_VALUE_ACTION") return var @@ -626,5 +643,5 @@ async def ble_server_descriptor_set_value(config, action_id, template_arg, args) ) async def ble_server_characteristic_notify(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + cg.add_define("USE_ESP32_BLE_SERVER_NOTIFY_ACTION") + return cg.new_Pvariable(action_id, template_arg, paren) diff --git a/esphome/components/esp32_ble_server/ble_characteristic.cpp b/esphome/components/esp32_ble_server/ble_characteristic.cpp index 373d57436e..87f562a250 100644 --- a/esphome/components/esp32_ble_server/ble_characteristic.cpp +++ b/esphome/components/esp32_ble_server/ble_characteristic.cpp @@ -49,13 +49,17 @@ void BLECharacteristic::notify() { this->service_->get_server()->get_connected_client_count() == 0) return; - for (auto &client : this->service_->get_server()->get_clients()) { + const uint16_t *clients = this->service_->get_server()->get_clients(); + uint8_t client_count = this->service_->get_server()->get_client_count(); + + for (uint8_t i = 0; i < client_count; i++) { + uint16_t client = clients[i]; size_t length = this->value_.size(); - // If the client is not in the list of clients to notify, skip it - if (this->clients_to_notify_.count(client) == 0) + // Find the client in the list of clients to notify + auto *entry = this->find_client_in_notify_list_(client); + if (entry == nullptr) continue; - // If the client is in the list of clients to notify, check if it requires an ack (i.e. INDICATE) - bool require_ack = this->clients_to_notify_[client]; + bool require_ack = entry->indicate; // TODO: Remove this block when INDICATE acknowledgment is supported if (require_ack) { ESP_LOGW(TAG, "INDICATE acknowledgment is not yet supported (i.e. it works as a NOTIFY)"); @@ -73,16 +77,17 @@ void BLECharacteristic::notify() { void BLECharacteristic::add_descriptor(BLEDescriptor *descriptor) { // If the descriptor is the CCCD descriptor, listen to its write event to know if the client wants to be notified if (descriptor->get_uuid() == ESPBTUUID::from_uint16(ESP_GATT_UUID_CHAR_CLIENT_CONFIG)) { - descriptor->on(BLEDescriptorEvt::VectorEvt::ON_WRITE, [this](const std::vector &value, uint16_t conn_id) { + descriptor->on_write([this](std::span value, uint16_t conn_id) { if (value.size() != 2) return; uint16_t cccd = encode_uint16(value[1], value[0]); bool notify = (cccd & 1) != 0; bool indicate = (cccd & 2) != 0; + // Remove existing entry if present + this->remove_client_from_notify_list_(conn_id); + // Add new entry if needed if (notify || indicate) { - this->clients_to_notify_[conn_id] = indicate; - } else { - this->clients_to_notify_.erase(conn_id); + this->clients_to_notify_.push_back({conn_id, indicate}); } }); } @@ -120,69 +125,49 @@ bool BLECharacteristic::is_created() { if (this->state_ != CREATING_DEPENDENTS) return false; - bool created = true; for (auto *descriptor : this->descriptors_) { - created &= descriptor->is_created(); + if (!descriptor->is_created()) + return false; } - if (created) - this->state_ = CREATED; - return this->state_ == CREATED; + // All descriptors are created if we reach here + this->state_ = CREATED; + return true; } bool BLECharacteristic::is_failed() { if (this->state_ == FAILED) return true; - bool failed = false; for (auto *descriptor : this->descriptors_) { - failed |= descriptor->is_failed(); + if (descriptor->is_failed()) { + this->state_ = FAILED; + return true; + } + } + return false; +} + +void BLECharacteristic::set_property_bit_(esp_gatt_char_prop_t bit, bool value) { + if (value) { + this->properties_ = (esp_gatt_char_prop_t) (this->properties_ | bit); + } else { + this->properties_ = (esp_gatt_char_prop_t) (this->properties_ & ~bit); } - if (failed) - this->state_ = FAILED; - return this->state_ == FAILED; } void BLECharacteristic::set_broadcast_property(bool value) { - if (value) { - this->properties_ = (esp_gatt_char_prop_t) (this->properties_ | ESP_GATT_CHAR_PROP_BIT_BROADCAST); - } else { - this->properties_ = (esp_gatt_char_prop_t) (this->properties_ & ~ESP_GATT_CHAR_PROP_BIT_BROADCAST); - } + this->set_property_bit_(ESP_GATT_CHAR_PROP_BIT_BROADCAST, value); } void BLECharacteristic::set_indicate_property(bool value) { - if (value) { - this->properties_ = (esp_gatt_char_prop_t) (this->properties_ | ESP_GATT_CHAR_PROP_BIT_INDICATE); - } else { - this->properties_ = (esp_gatt_char_prop_t) (this->properties_ & ~ESP_GATT_CHAR_PROP_BIT_INDICATE); - } + this->set_property_bit_(ESP_GATT_CHAR_PROP_BIT_INDICATE, value); } void BLECharacteristic::set_notify_property(bool value) { - if (value) { - this->properties_ = (esp_gatt_char_prop_t) (this->properties_ | ESP_GATT_CHAR_PROP_BIT_NOTIFY); - } else { - this->properties_ = (esp_gatt_char_prop_t) (this->properties_ & ~ESP_GATT_CHAR_PROP_BIT_NOTIFY); - } -} -void BLECharacteristic::set_read_property(bool value) { - if (value) { - this->properties_ = (esp_gatt_char_prop_t) (this->properties_ | ESP_GATT_CHAR_PROP_BIT_READ); - } else { - this->properties_ = (esp_gatt_char_prop_t) (this->properties_ & ~ESP_GATT_CHAR_PROP_BIT_READ); - } -} -void BLECharacteristic::set_write_property(bool value) { - if (value) { - this->properties_ = (esp_gatt_char_prop_t) (this->properties_ | ESP_GATT_CHAR_PROP_BIT_WRITE); - } else { - this->properties_ = (esp_gatt_char_prop_t) (this->properties_ & ~ESP_GATT_CHAR_PROP_BIT_WRITE); - } + this->set_property_bit_(ESP_GATT_CHAR_PROP_BIT_NOTIFY, value); } +void BLECharacteristic::set_read_property(bool value) { this->set_property_bit_(ESP_GATT_CHAR_PROP_BIT_READ, value); } +void BLECharacteristic::set_write_property(bool value) { this->set_property_bit_(ESP_GATT_CHAR_PROP_BIT_WRITE, value); } void BLECharacteristic::set_write_no_response_property(bool value) { - if (value) { - this->properties_ = (esp_gatt_char_prop_t) (this->properties_ | ESP_GATT_CHAR_PROP_BIT_WRITE_NR); - } else { - this->properties_ = (esp_gatt_char_prop_t) (this->properties_ & ~ESP_GATT_CHAR_PROP_BIT_WRITE_NR); - } + this->set_property_bit_(ESP_GATT_CHAR_PROP_BIT_WRITE_NR, value); } void BLECharacteristic::gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t gatts_if, @@ -207,8 +192,9 @@ void BLECharacteristic::gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt if (!param->read.need_rsp) break; // For some reason you can request a read but not want a response - this->EventEmitter::emit_(BLECharacteristicEvt::EmptyEvt::ON_READ, - param->read.conn_id); + if (this->on_read_callback_) { + (*this->on_read_callback_)(param->read.conn_id); + } uint16_t max_offset = 22; @@ -276,8 +262,9 @@ void BLECharacteristic::gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt } if (!param->write.is_prep) { - this->EventEmitter, uint16_t>::emit_( - BLECharacteristicEvt::VectorEvt::ON_WRITE, this->value_, param->write.conn_id); + if (this->on_write_callback_) { + (*this->on_write_callback_)(this->value_, param->write.conn_id); + } } break; @@ -288,8 +275,9 @@ void BLECharacteristic::gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt break; this->write_event_ = false; if (param->exec_write.exec_write_flag == ESP_GATT_PREP_WRITE_EXEC) { - this->EventEmitter, uint16_t>::emit_( - BLECharacteristicEvt::VectorEvt::ON_WRITE, this->value_, param->exec_write.conn_id); + if (this->on_write_callback_) { + (*this->on_write_callback_)(this->value_, param->exec_write.conn_id); + } } esp_err_t err = esp_ble_gatts_send_response(gatts_if, param->write.conn_id, param->write.trans_id, ESP_GATT_OK, nullptr); @@ -307,6 +295,28 @@ void BLECharacteristic::gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt } } +void BLECharacteristic::remove_client_from_notify_list_(uint16_t conn_id) { + // Since we typically have very few clients (often just 1), we can optimize + // for the common case by swapping with the last element and popping + for (size_t i = 0; i < this->clients_to_notify_.size(); i++) { + if (this->clients_to_notify_[i].conn_id == conn_id) { + // Swap with last element and pop (safe even when i is the last element) + this->clients_to_notify_[i] = this->clients_to_notify_.back(); + this->clients_to_notify_.pop_back(); + return; + } + } +} + +BLECharacteristic::ClientNotificationEntry *BLECharacteristic::find_client_in_notify_list_(uint16_t conn_id) { + for (auto &entry : this->clients_to_notify_) { + if (entry.conn_id == conn_id) { + return &entry; + } + } + return nullptr; +} + } // namespace esp32_ble_server } // namespace esphome diff --git a/esphome/components/esp32_ble_server/ble_characteristic.h b/esphome/components/esp32_ble_server/ble_characteristic.h index 3698b8c4aa..7cceec0ef1 100644 --- a/esphome/components/esp32_ble_server/ble_characteristic.h +++ b/esphome/components/esp32_ble_server/ble_characteristic.h @@ -2,11 +2,12 @@ #include "ble_descriptor.h" #include "esphome/components/esp32_ble/ble_uuid.h" -#include "esphome/components/event_emitter/event_emitter.h" #include "esphome/components/bytebuffer/bytebuffer.h" #include -#include +#include +#include +#include #ifdef USE_ESP32 @@ -23,22 +24,10 @@ namespace esp32_ble_server { using namespace esp32_ble; using namespace bytebuffer; -using namespace event_emitter; class BLEService; -namespace BLECharacteristicEvt { -enum VectorEvt { - ON_WRITE, -}; - -enum EmptyEvt { - ON_READ, -}; -} // namespace BLECharacteristicEvt - -class BLECharacteristic : public EventEmitter, uint16_t>, - public EventEmitter { +class BLECharacteristic { public: BLECharacteristic(ESPBTUUID uuid, uint32_t properties); ~BLECharacteristic(); @@ -77,6 +66,15 @@ class BLECharacteristic : public EventEmitter, uint16_t)> &&callback) { + this->on_write_callback_ = + std::make_unique, uint16_t)>>(std::move(callback)); + } + void on_read(std::function &&callback) { + this->on_read_callback_ = std::make_unique>(std::move(callback)); + } + protected: bool write_event_{false}; BLEService *service_{}; @@ -89,7 +87,20 @@ class BLECharacteristic : public EventEmitter descriptors_; - std::unordered_map clients_to_notify_; + + struct ClientNotificationEntry { + uint16_t conn_id; + bool indicate; // true = indicate, false = notify + }; + std::vector clients_to_notify_; + + void remove_client_from_notify_list_(uint16_t conn_id); + ClientNotificationEntry *find_client_in_notify_list_(uint16_t conn_id); + + void set_property_bit_(esp_gatt_char_prop_t bit, bool value); + + std::unique_ptr, uint16_t)>> on_write_callback_; + std::unique_ptr> on_read_callback_; esp_gatt_perm_t permissions_ = ESP_GATT_PERM_READ | ESP_GATT_PERM_WRITE; diff --git a/esphome/components/esp32_ble_server/ble_descriptor.cpp b/esphome/components/esp32_ble_server/ble_descriptor.cpp index afbe579513..16941cca0f 100644 --- a/esphome/components/esp32_ble_server/ble_descriptor.cpp +++ b/esphome/components/esp32_ble_server/ble_descriptor.cpp @@ -74,9 +74,10 @@ void BLEDescriptor::gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_ break; this->value_.attr_len = param->write.len; memcpy(this->value_.attr_value, param->write.value, param->write.len); - this->emit_(BLEDescriptorEvt::VectorEvt::ON_WRITE, - std::vector(param->write.value, param->write.value + param->write.len), - param->write.conn_id); + if (this->on_write_callback_) { + (*this->on_write_callback_)(std::span(param->write.value, param->write.len), + param->write.conn_id); + } break; } default: diff --git a/esphome/components/esp32_ble_server/ble_descriptor.h b/esphome/components/esp32_ble_server/ble_descriptor.h index 8d3c22c5a1..425462a316 100644 --- a/esphome/components/esp32_ble_server/ble_descriptor.h +++ b/esphome/components/esp32_ble_server/ble_descriptor.h @@ -1,30 +1,26 @@ #pragma once #include "esphome/components/esp32_ble/ble_uuid.h" -#include "esphome/components/event_emitter/event_emitter.h" #include "esphome/components/bytebuffer/bytebuffer.h" #ifdef USE_ESP32 #include #include +#include +#include +#include namespace esphome { namespace esp32_ble_server { using namespace esp32_ble; using namespace bytebuffer; -using namespace event_emitter; class BLECharacteristic; -namespace BLEDescriptorEvt { -enum VectorEvt { - ON_WRITE, -}; -} // namespace BLEDescriptorEvt - -class BLEDescriptor : public EventEmitter, uint16_t> { +// Base class for BLE descriptors +class BLEDescriptor { public: BLEDescriptor(ESPBTUUID uuid, uint16_t max_len = 100, bool read = true, bool write = true); virtual ~BLEDescriptor(); @@ -39,6 +35,12 @@ class BLEDescriptor : public EventEmitterstate_ == CREATED; } bool is_failed() { return this->state_ == FAILED; } + // Direct callback registration - only allocates when callback is set + void on_write(std::function, uint16_t)> &&callback) { + this->on_write_callback_ = + std::make_unique, uint16_t)>>(std::move(callback)); + } + protected: BLECharacteristic *characteristic_{nullptr}; ESPBTUUID uuid_; @@ -46,6 +48,8 @@ class BLEDescriptor : public EventEmitter, uint16_t)>> on_write_callback_; + esp_gatt_perm_t permissions_{}; enum State : uint8_t { diff --git a/esphome/components/esp32_ble_server/ble_server.cpp b/esphome/components/esp32_ble_server/ble_server.cpp index 5339bf8aed..0e58224a5a 100644 --- a/esphome/components/esp32_ble_server/ble_server.cpp +++ b/esphome/components/esp32_ble_server/ble_server.cpp @@ -10,7 +10,9 @@ #include #include #include +#ifndef CONFIG_ESP_HOSTED_ENABLE_BT_BLUEDROID #include +#endif #include #include @@ -70,11 +72,11 @@ void BLEServer::loop() { // it is at the top of the GATT table this->device_information_service_->do_create(this); // Create all services previously created - for (auto &pair : this->services_) { - if (pair.second == this->device_information_service_) { + for (auto &entry : this->services_) { + if (entry.service == this->device_information_service_) { continue; } - pair.second->do_create(this); + entry.service->do_create(this); } this->state_ = STARTING_SERVICE; } @@ -118,7 +120,7 @@ BLEService *BLEServer::create_service(ESPBTUUID uuid, bool advertise, uint16_t n } BLEService *service = // NOLINT(cppcoreguidelines-owning-memory) new BLEService(uuid, num_handles, inst_id, advertise); - this->services_.emplace(BLEServer::get_service_key(uuid, inst_id), service); + this->services_.push_back({uuid, inst_id, service}); if (this->parent_->is_active() && this->registered_) { service->do_create(this); } @@ -127,26 +129,32 @@ BLEService *BLEServer::create_service(ESPBTUUID uuid, bool advertise, uint16_t n void BLEServer::remove_service(ESPBTUUID uuid, uint8_t inst_id) { ESP_LOGV(TAG, "Removing BLE service - %s %d", uuid.to_string().c_str(), inst_id); - BLEService *service = this->get_service(uuid, inst_id); - if (service == nullptr) { - ESP_LOGW(TAG, "BLE service %s %d does not exist", uuid.to_string().c_str(), inst_id); - return; + for (auto it = this->services_.begin(); it != this->services_.end(); ++it) { + if (it->uuid == uuid && it->inst_id == inst_id) { + it->service->do_delete(); + delete it->service; // NOLINT(cppcoreguidelines-owning-memory) + this->services_.erase(it); + return; + } } - service->do_delete(); - delete service; // NOLINT(cppcoreguidelines-owning-memory) - this->services_.erase(BLEServer::get_service_key(uuid, inst_id)); + ESP_LOGW(TAG, "BLE service %s %d does not exist", uuid.to_string().c_str(), inst_id); } BLEService *BLEServer::get_service(ESPBTUUID uuid, uint8_t inst_id) { - BLEService *service = nullptr; - if (this->services_.count(BLEServer::get_service_key(uuid, inst_id)) > 0) { - service = this->services_.at(BLEServer::get_service_key(uuid, inst_id)); + for (auto &entry : this->services_) { + if (entry.uuid == uuid && entry.inst_id == inst_id) { + return entry.service; + } } - return service; + return nullptr; } -std::string BLEServer::get_service_key(ESPBTUUID uuid, uint8_t inst_id) { - return uuid.to_string() + std::to_string(inst_id); +void BLEServer::dispatch_callbacks_(CallbackType type, uint16_t conn_id) { + for (auto &entry : this->callbacks_) { + if (entry.type == type) { + entry.callback(conn_id); + } + } } void BLEServer::gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t gatts_if, @@ -155,14 +163,14 @@ void BLEServer::gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t ga case ESP_GATTS_CONNECT_EVT: { ESP_LOGD(TAG, "BLE Client connected"); this->add_client_(param->connect.conn_id); - this->emit_(BLEServerEvt::EmptyEvt::ON_CONNECT, param->connect.conn_id); + this->dispatch_callbacks_(CallbackType::ON_CONNECT, param->connect.conn_id); break; } case ESP_GATTS_DISCONNECT_EVT: { ESP_LOGD(TAG, "BLE Client disconnected"); this->remove_client_(param->disconnect.conn_id); this->parent_->advertising_start(); - this->emit_(BLEServerEvt::EmptyEvt::ON_DISCONNECT, param->disconnect.conn_id); + this->dispatch_callbacks_(CallbackType::ON_DISCONNECT, param->disconnect.conn_id); break; } case ESP_GATTS_REG_EVT: { @@ -174,17 +182,46 @@ void BLEServer::gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t ga break; } - for (const auto &pair : this->services_) { - pair.second->gatts_event_handler(event, gatts_if, param); + for (auto &entry : this->services_) { + entry.service->gatts_event_handler(event, gatts_if, param); + } +} + +int8_t BLEServer::find_client_index_(uint16_t conn_id) const { + for (uint8_t i = 0; i < this->client_count_; i++) { + if (this->clients_[i] == conn_id) + return i; + } + return -1; +} + +void BLEServer::add_client_(uint16_t conn_id) { + // Check if already in list + if (this->find_client_index_(conn_id) >= 0) + return; + // Add if there's space + if (this->client_count_ < USE_ESP32_BLE_MAX_CONNECTIONS) { + this->clients_[this->client_count_++] = conn_id; + } else { + // This should never happen since max clients is known at compile time + ESP_LOGE(TAG, "Client array full"); + } +} + +void BLEServer::remove_client_(uint16_t conn_id) { + int8_t index = this->find_client_index_(conn_id); + if (index >= 0) { + // Replace with last element and decrement count (client order not preserved) + this->clients_[index] = this->clients_[--this->client_count_]; } } void BLEServer::ble_before_disabled_event_handler() { // Delete all clients - this->clients_.clear(); + this->client_count_ = 0; // Delete all services - for (auto &pair : this->services_) { - pair.second->do_delete(); + for (auto &entry : this->services_) { + entry.service->do_delete(); } this->registered_ = false; this->state_ = INIT; diff --git a/esphome/components/esp32_ble_server/ble_server.h b/esphome/components/esp32_ble_server/ble_server.h index 531b52d6b9..6fa86dd67f 100644 --- a/esphome/components/esp32_ble_server/ble_server.h +++ b/esphome/components/esp32_ble_server/ble_server.h @@ -12,7 +12,7 @@ #include #include #include -#include +#include #ifdef USE_ESP32 @@ -24,18 +24,7 @@ namespace esp32_ble_server { using namespace esp32_ble; using namespace bytebuffer; -namespace BLEServerEvt { -enum EmptyEvt { - ON_CONNECT, - ON_DISCONNECT, -}; -} // namespace BLEServerEvt - -class BLEServer : public Component, - public GATTsEventHandler, - public BLEStatusEventHandler, - public Parented, - public EventEmitter { +class BLEServer : public Component, public GATTsEventHandler, public BLEStatusEventHandler, public Parented { public: void setup() override; void loop() override; @@ -57,27 +46,56 @@ class BLEServer : public Component, void set_device_information_service(BLEService *service) { this->device_information_service_ = service; } esp_gatt_if_t get_gatts_if() { return this->gatts_if_; } - uint32_t get_connected_client_count() { return this->clients_.size(); } - const std::unordered_set &get_clients() { return this->clients_; } + uint32_t get_connected_client_count() { return this->client_count_; } + const uint16_t *get_clients() const { return this->clients_; } + uint8_t get_client_count() const { return this->client_count_; } void gatts_event_handler(esp_gatts_cb_event_t event, esp_gatt_if_t gatts_if, esp_ble_gatts_cb_param_t *param) override; void ble_before_disabled_event_handler() override; + // Direct callback registration - supports multiple callbacks + void on_connect(std::function &&callback) { + this->callbacks_.push_back({CallbackType::ON_CONNECT, std::move(callback)}); + } + void on_disconnect(std::function &&callback) { + this->callbacks_.push_back({CallbackType::ON_DISCONNECT, std::move(callback)}); + } + protected: - static std::string get_service_key(ESPBTUUID uuid, uint8_t inst_id); + enum class CallbackType : uint8_t { + ON_CONNECT, + ON_DISCONNECT, + }; + + struct CallbackEntry { + CallbackType type; + std::function callback; + }; + + struct ServiceEntry { + ESPBTUUID uuid; + uint8_t inst_id; + BLEService *service; + }; + void restart_advertising_(); - void add_client_(uint16_t conn_id) { this->clients_.insert(conn_id); } - void remove_client_(uint16_t conn_id) { this->clients_.erase(conn_id); } + int8_t find_client_index_(uint16_t conn_id) const; + void add_client_(uint16_t conn_id); + void remove_client_(uint16_t conn_id); + void dispatch_callbacks_(CallbackType type, uint16_t conn_id); + + std::vector callbacks_; std::vector manufacturer_data_{}; esp_gatt_if_t gatts_if_{0}; bool registered_{false}; - std::unordered_set clients_; - std::unordered_map services_{}; + uint16_t clients_[USE_ESP32_BLE_MAX_CONNECTIONS]{}; + uint8_t client_count_{0}; + std::vector services_{}; std::vector services_to_start_{}; BLEService *device_information_service_{}; diff --git a/esphome/components/esp32_ble_server/ble_server_automations.cpp b/esphome/components/esp32_ble_server/ble_server_automations.cpp index 41ef2b8bfe..0761de994a 100644 --- a/esphome/components/esp32_ble_server/ble_server_automations.cpp +++ b/esphome/components/esp32_ble_server/ble_server_automations.cpp @@ -9,67 +9,83 @@ namespace esp32_ble_server_automations { using namespace esp32_ble; +#ifdef USE_ESP32_BLE_SERVER_CHARACTERISTIC_ON_WRITE Trigger, uint16_t> *BLETriggers::create_characteristic_on_write_trigger( BLECharacteristic *characteristic) { Trigger, uint16_t> *on_write_trigger = // NOLINT(cppcoreguidelines-owning-memory) new Trigger, uint16_t>(); - characteristic->EventEmitter, uint16_t>::on( - BLECharacteristicEvt::VectorEvt::ON_WRITE, - [on_write_trigger](const std::vector &data, uint16_t id) { on_write_trigger->trigger(data, id); }); + characteristic->on_write([on_write_trigger](std::span data, uint16_t id) { + // Convert span to vector for trigger + on_write_trigger->trigger(std::vector(data.begin(), data.end()), id); + }); return on_write_trigger; } +#endif +#ifdef USE_ESP32_BLE_SERVER_DESCRIPTOR_ON_WRITE Trigger, uint16_t> *BLETriggers::create_descriptor_on_write_trigger(BLEDescriptor *descriptor) { Trigger, uint16_t> *on_write_trigger = // NOLINT(cppcoreguidelines-owning-memory) new Trigger, uint16_t>(); - descriptor->on( - BLEDescriptorEvt::VectorEvt::ON_WRITE, - [on_write_trigger](const std::vector &data, uint16_t id) { on_write_trigger->trigger(data, id); }); + descriptor->on_write([on_write_trigger](std::span data, uint16_t id) { + // Convert span to vector for trigger + on_write_trigger->trigger(std::vector(data.begin(), data.end()), id); + }); return on_write_trigger; } +#endif +#ifdef USE_ESP32_BLE_SERVER_ON_CONNECT Trigger *BLETriggers::create_server_on_connect_trigger(BLEServer *server) { Trigger *on_connect_trigger = new Trigger(); // NOLINT(cppcoreguidelines-owning-memory) - server->on(BLEServerEvt::EmptyEvt::ON_CONNECT, - [on_connect_trigger](uint16_t conn_id) { on_connect_trigger->trigger(conn_id); }); + server->on_connect([on_connect_trigger](uint16_t conn_id) { on_connect_trigger->trigger(conn_id); }); return on_connect_trigger; } +#endif +#ifdef USE_ESP32_BLE_SERVER_ON_DISCONNECT Trigger *BLETriggers::create_server_on_disconnect_trigger(BLEServer *server) { Trigger *on_disconnect_trigger = new Trigger(); // NOLINT(cppcoreguidelines-owning-memory) - server->on(BLEServerEvt::EmptyEvt::ON_DISCONNECT, - [on_disconnect_trigger](uint16_t conn_id) { on_disconnect_trigger->trigger(conn_id); }); + server->on_disconnect([on_disconnect_trigger](uint16_t conn_id) { on_disconnect_trigger->trigger(conn_id); }); return on_disconnect_trigger; } +#endif +#ifdef USE_ESP32_BLE_SERVER_SET_VALUE_ACTION void BLECharacteristicSetValueActionManager::set_listener(BLECharacteristic *characteristic, - EventEmitterListenerID listener_id, const std::function &pre_notify_listener) { - // Check if there is already a listener for this characteristic - if (this->listeners_.count(characteristic) > 0) { - // Unpack the pair listener_id, pre_notify_listener_id - auto listener_pairs = this->listeners_[characteristic]; - EventEmitterListenerID old_listener_id = listener_pairs.first; - EventEmitterListenerID old_pre_notify_listener_id = listener_pairs.second; - // Remove the previous listener - characteristic->EventEmitter::off(BLECharacteristicEvt::EmptyEvt::ON_READ, - old_listener_id); - // Remove the pre-notify listener - this->off(BLECharacteristicSetValueActionEvt::PRE_NOTIFY, old_pre_notify_listener_id); + // Find and remove existing listener for this characteristic + auto *existing = this->find_listener_(characteristic); + if (existing != nullptr) { + // Remove from vector + this->remove_listener_(characteristic); } - // Create a new listener for the pre-notify event - EventEmitterListenerID pre_notify_listener_id = - this->on(BLECharacteristicSetValueActionEvt::PRE_NOTIFY, - [pre_notify_listener, characteristic](const BLECharacteristic *evt_characteristic) { - // Only call the pre-notify listener if the characteristic is the one we are interested in - if (characteristic == evt_characteristic) { - pre_notify_listener(); - } - }); - // Save the pair listener_id, pre_notify_listener_id to the map - this->listeners_[characteristic] = std::make_pair(listener_id, pre_notify_listener_id); + // Save the entry to the vector + this->listeners_.push_back({characteristic, pre_notify_listener}); } +BLECharacteristicSetValueActionManager::ListenerEntry *BLECharacteristicSetValueActionManager::find_listener_( + BLECharacteristic *characteristic) { + for (auto &entry : this->listeners_) { + if (entry.characteristic == characteristic) { + return &entry; + } + } + return nullptr; +} + +void BLECharacteristicSetValueActionManager::remove_listener_(BLECharacteristic *characteristic) { + // Since we typically have very few listeners, optimize by swapping with back and popping + for (size_t i = 0; i < this->listeners_.size(); i++) { + if (this->listeners_[i].characteristic == characteristic) { + // Swap with last element and pop (safe even when i is the last element) + this->listeners_[i] = this->listeners_.back(); + this->listeners_.pop_back(); + return; + } + } +} +#endif + } // namespace esp32_ble_server_automations } // namespace esp32_ble_server } // namespace esphome diff --git a/esphome/components/esp32_ble_server/ble_server_automations.h b/esphome/components/esp32_ble_server/ble_server_automations.h index eab6b05f05..543b1153fc 100644 --- a/esphome/components/esp32_ble_server/ble_server_automations.h +++ b/esphome/components/esp32_ble_server/ble_server_automations.h @@ -4,11 +4,9 @@ #include "ble_characteristic.h" #include "ble_descriptor.h" -#include "esphome/components/event_emitter/event_emitter.h" #include "esphome/core/automation.h" #include -#include #include #ifdef USE_ESP32 @@ -19,41 +17,53 @@ namespace esp32_ble_server { namespace esp32_ble_server_automations { using namespace esp32_ble; -using namespace event_emitter; class BLETriggers { public: +#ifdef USE_ESP32_BLE_SERVER_CHARACTERISTIC_ON_WRITE static Trigger, uint16_t> *create_characteristic_on_write_trigger( BLECharacteristic *characteristic); +#endif +#ifdef USE_ESP32_BLE_SERVER_DESCRIPTOR_ON_WRITE static Trigger, uint16_t> *create_descriptor_on_write_trigger(BLEDescriptor *descriptor); +#endif +#ifdef USE_ESP32_BLE_SERVER_ON_CONNECT static Trigger *create_server_on_connect_trigger(BLEServer *server); +#endif +#ifdef USE_ESP32_BLE_SERVER_ON_DISCONNECT static Trigger *create_server_on_disconnect_trigger(BLEServer *server); +#endif }; -enum BLECharacteristicSetValueActionEvt { - PRE_NOTIFY, -}; - +#ifdef USE_ESP32_BLE_SERVER_SET_VALUE_ACTION // Class to make sure only one BLECharacteristicSetValueAction is active at a time for each characteristic -class BLECharacteristicSetValueActionManager - : public EventEmitter { +class BLECharacteristicSetValueActionManager { public: // Singleton pattern static BLECharacteristicSetValueActionManager *get_instance() { static BLECharacteristicSetValueActionManager instance; return &instance; } - void set_listener(BLECharacteristic *characteristic, EventEmitterListenerID listener_id, - const std::function &pre_notify_listener); - EventEmitterListenerID get_listener(BLECharacteristic *characteristic) { - return this->listeners_[characteristic].first; - } + void set_listener(BLECharacteristic *characteristic, const std::function &pre_notify_listener); + bool has_listener(BLECharacteristic *characteristic) { return this->find_listener_(characteristic) != nullptr; } void emit_pre_notify(BLECharacteristic *characteristic) { - this->emit_(BLECharacteristicSetValueActionEvt::PRE_NOTIFY, characteristic); + for (const auto &entry : this->listeners_) { + if (entry.characteristic == characteristic) { + entry.pre_notify_listener(); + break; + } + } } private: - std::unordered_map> listeners_; + struct ListenerEntry { + BLECharacteristic *characteristic; + std::function pre_notify_listener; + }; + std::vector listeners_; + + ListenerEntry *find_listener_(BLECharacteristic *characteristic); + void remove_listener_(BLECharacteristic *characteristic); }; template class BLECharacteristicSetValueAction : public Action { @@ -63,32 +73,34 @@ template class BLECharacteristicSetValueAction : public Actionset_buffer(buffer.get_data()); } void play(Ts... x) override { // If the listener is already set, do nothing - if (BLECharacteristicSetValueActionManager::get_instance()->get_listener(this->parent_) == this->listener_id_) + if (BLECharacteristicSetValueActionManager::get_instance()->has_listener(this->parent_)) return; // Set initial value this->parent_->set_value(this->buffer_.value(x...)); // Set the listener for read events - this->listener_id_ = this->parent_->EventEmitter::on( - BLECharacteristicEvt::EmptyEvt::ON_READ, [this, x...](uint16_t id) { - // Set the value of the characteristic every time it is read - this->parent_->set_value(this->buffer_.value(x...)); - }); + this->parent_->on_read([this, x...](uint16_t id) { + // Set the value of the characteristic every time it is read + this->parent_->set_value(this->buffer_.value(x...)); + }); // Set the listener in the global manager so only one BLECharacteristicSetValueAction is set for each characteristic BLECharacteristicSetValueActionManager::get_instance()->set_listener( - this->parent_, this->listener_id_, [this, x...]() { this->parent_->set_value(this->buffer_.value(x...)); }); + this->parent_, [this, x...]() { this->parent_->set_value(this->buffer_.value(x...)); }); } protected: BLECharacteristic *parent_; - EventEmitterListenerID listener_id_; }; +#endif // USE_ESP32_BLE_SERVER_SET_VALUE_ACTION +#ifdef USE_ESP32_BLE_SERVER_NOTIFY_ACTION template class BLECharacteristicNotifyAction : public Action { public: BLECharacteristicNotifyAction(BLECharacteristic *characteristic) : parent_(characteristic) {} void play(Ts... x) override { +#ifdef USE_ESP32_BLE_SERVER_SET_VALUE_ACTION // Call the pre-notify event BLECharacteristicSetValueActionManager::get_instance()->emit_pre_notify(this->parent_); +#endif // Notify the characteristic this->parent_->notify(); } @@ -96,7 +108,9 @@ template class BLECharacteristicNotifyAction : public Action class BLEDescriptorSetValueAction : public Action { public: BLEDescriptorSetValueAction(BLEDescriptor *descriptor) : parent_(descriptor) {} @@ -107,6 +121,7 @@ template class BLEDescriptorSetValueAction : public Action set[BLEFeatures]: + """Get the set of required BLE features from CORE.data.""" + return CORE.data.setdefault(ESP32_BLE_TRACKER_REQUIRED_FEATURES_KEY, set()) + + +def _get_registration_counts() -> RegistrationCounts: + """Get the registration counts from CORE.data.""" + return CORE.data.setdefault( + ESP32_BLE_TRACKER_REGISTRATION_COUNTS_KEY, RegistrationCounts() + ) + + +def register_ble_features(features: set[BLEFeatures]) -> None: + """Register BLE features that a component needs. + + Args: + features: Set of BLEFeatures enum members + """ + _get_required_features().update(features) + + esp32_ble_tracker_ns = cg.esphome_ns.namespace("esp32_ble_tracker") ESP32BLETracker = esp32_ble_tracker_ns.class_( "ESP32BLETracker", @@ -106,6 +142,15 @@ def validate_scan_parameters(config): return config +def validate_max_connections_deprecated(config: ConfigType) -> ConfigType: + if CONF_MAX_CONNECTIONS in config: + _LOGGER.warning( + "The 'max_connections' option in 'esp32_ble_tracker' is deprecated. " + "Please move it to the 'esp32_ble' component instead." + ) + return config + + def as_hex(value): return cg.RawExpression(f"0x{value}ULL") @@ -128,29 +173,13 @@ def as_reversed_hex_array(value): ) -def max_connections() -> int: - return IDF_MAX_CONNECTIONS if CORE.using_esp_idf else DEFAULT_MAX_CONNECTIONS - - -def consume_connection_slots( - value: int, consumer: str -) -> Callable[[MutableMapping], MutableMapping]: - def _consume_connection_slots(config: MutableMapping) -> MutableMapping: - data: dict[str, Any] = CORE.data.setdefault(KEY_ESP32_BLE_TRACKER, {}) - slots: list[str] = data.setdefault(KEY_USED_CONNECTION_SLOTS, []) - slots.extend([consumer] * value) - return config - - return _consume_connection_slots - - CONFIG_SCHEMA = cv.All( cv.Schema( { cv.GenerateID(): cv.declare_id(ESP32BLETracker), cv.GenerateID(esp32_ble.CONF_BLE_ID): cv.use_id(esp32_ble.ESP32BLE), - cv.Optional(CONF_MAX_CONNECTIONS, default=DEFAULT_MAX_CONNECTIONS): cv.All( - cv.positive_int, cv.Range(min=0, max=max_connections()) + cv.Optional(CONF_MAX_CONNECTIONS): cv.All( + cv.positive_int, cv.Range(min=0, max=IDF_MAX_CONNECTIONS) ), cv.Optional(CONF_SCAN_PARAMETERS, default={}): cv.All( cv.Schema( @@ -206,49 +235,11 @@ CONFIG_SCHEMA = cv.All( cv.OnlyWith(CONF_SOFTWARE_COEXISTENCE, "wifi", default=True): bool, } ).extend(cv.COMPONENT_SCHEMA), + validate_max_connections_deprecated, ) -def validate_remaining_connections(config): - data: dict[str, Any] = CORE.data.get(KEY_ESP32_BLE_TRACKER, {}) - slots: list[str] = data.get(KEY_USED_CONNECTION_SLOTS, []) - used_slots = len(slots) - if used_slots <= config[CONF_MAX_CONNECTIONS]: - return config - slot_users = ", ".join(slots) - hard_limit = max_connections() - - if used_slots < hard_limit: - _LOGGER.warning( - "esp32_ble_tracker exceeded `%s`: components attempted to consume %d " - "connection slot(s) out of available configured maximum %d connection " - "slot(s); The system automatically increased `%s` to %d to match the " - "number of used connection slot(s) by components: %s.", - CONF_MAX_CONNECTIONS, - used_slots, - config[CONF_MAX_CONNECTIONS], - CONF_MAX_CONNECTIONS, - used_slots, - slot_users, - ) - config[CONF_MAX_CONNECTIONS] = used_slots - return config - - msg = ( - f"esp32_ble_tracker exceeded `{CONF_MAX_CONNECTIONS}`: " - f"components attempted to consume {used_slots} connection slot(s) " - f"out of available configured maximum {config[CONF_MAX_CONNECTIONS]} " - f"connection slot(s); Decrease the number of BLE clients ({slot_users})" - ) - if config[CONF_MAX_CONNECTIONS] < hard_limit: - msg += f" or increase {CONF_MAX_CONNECTIONS}` to {used_slots}" - msg += f" to stay under the {hard_limit} connection slot(s) limit." - raise cv.Invalid(msg) - - -FINAL_VALIDATE_SCHEMA = cv.All( - validate_remaining_connections, esp32_ble.validate_variant -) +FINAL_VALIDATE_SCHEMA = esp32_ble.validate_variant ESP_BLE_DEVICE_SCHEMA = cv.Schema( { @@ -265,10 +256,10 @@ async def to_code(config): await cg.register_component(var, config) parent = await cg.get_variable(config[esp32_ble.CONF_BLE_ID]) - cg.add(parent.register_gap_event_handler(var)) - cg.add(parent.register_gap_scan_event_handler(var)) - cg.add(parent.register_gattc_event_handler(var)) - cg.add(parent.register_ble_status_event_handler(var)) + esp32_ble.register_gap_event_handler(parent, var) + esp32_ble.register_gap_scan_event_handler(parent, var) + esp32_ble.register_gattc_event_handler(parent, var) + esp32_ble.register_ble_status_event_handler(parent, var) cg.add(var.set_parent(parent)) params = config[CONF_SCAN_PARAMETERS] @@ -277,15 +268,26 @@ async def to_code(config): cg.add(var.set_scan_window(int(params[CONF_WINDOW].total_milliseconds / 0.625))) cg.add(var.set_scan_active(params[CONF_ACTIVE])) cg.add(var.set_scan_continuous(params[CONF_CONTINUOUS])) + + # Register ESP_BT_DEVICE feature if any of the automation triggers are used + if ( + config.get(CONF_ON_BLE_ADVERTISE) + or config.get(CONF_ON_BLE_SERVICE_DATA_ADVERTISE) + or config.get(CONF_ON_BLE_MANUFACTURER_DATA_ADVERTISE) + ): + register_ble_features({BLEFeatures.ESP_BT_DEVICE}) + + registration_counts = _get_registration_counts() + for conf in config.get(CONF_ON_BLE_ADVERTISE, []): + registration_counts.listeners += 1 trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var) if CONF_MAC_ADDRESS in conf: - addr_list = [] - for it in conf[CONF_MAC_ADDRESS]: - addr_list.append(it.as_hex) + addr_list = [it.as_hex for it in conf[CONF_MAC_ADDRESS]] cg.add(trigger.set_addresses(addr_list)) await automation.build_automation(trigger, [(ESPBTDeviceConstRef, "x")], conf) for conf in config.get(CONF_ON_BLE_SERVICE_DATA_ADVERTISE, []): + registration_counts.listeners += 1 trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var) if len(conf[CONF_SERVICE_UUID]) == len(bt_uuid16_format): cg.add(trigger.set_service_uuid16(as_hex(conf[CONF_SERVICE_UUID]))) @@ -298,6 +300,7 @@ async def to_code(config): cg.add(trigger.set_address(conf[CONF_MAC_ADDRESS].as_hex)) await automation.build_automation(trigger, [(adv_data_t_const_ref, "x")], conf) for conf in config.get(CONF_ON_BLE_MANUFACTURER_DATA_ADVERTISE, []): + registration_counts.listeners += 1 trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var) if len(conf[CONF_MANUFACTURER_ID]) == len(bt_uuid16_format): cg.add(trigger.set_manufacturer_uuid16(as_hex(conf[CONF_MANUFACTURER_ID]))) @@ -310,34 +313,54 @@ async def to_code(config): cg.add(trigger.set_address(conf[CONF_MAC_ADDRESS].as_hex)) await automation.build_automation(trigger, [(adv_data_t_const_ref, "x")], conf) for conf in config.get(CONF_ON_SCAN_END, []): + registration_counts.listeners += 1 trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var) await automation.build_automation(trigger, [], conf) - if CORE.using_esp_idf: - add_idf_sdkconfig_option("CONFIG_BT_ENABLED", True) - if config.get(CONF_SOFTWARE_COEXISTENCE): - add_idf_sdkconfig_option("CONFIG_SW_COEXIST_ENABLE", True) - # https://github.com/espressif/esp-idf/issues/4101 - # https://github.com/espressif/esp-idf/issues/2503 - # Match arduino CONFIG_BTU_TASK_STACK_SIZE - # https://github.com/espressif/arduino-esp32/blob/fd72cf46ad6fc1a6de99c1d83ba8eba17d80a4ee/tools/sdk/esp32/sdkconfig#L1866 - add_idf_sdkconfig_option("CONFIG_BT_BTU_TASK_STACK_SIZE", 8192) - add_idf_sdkconfig_option("CONFIG_BT_ACL_CONNECTIONS", 9) - add_idf_sdkconfig_option( - "CONFIG_BTDM_CTRL_BLE_MAX_CONN", config[CONF_MAX_CONNECTIONS] - ) - # CONFIG_BT_GATTC_NOTIF_REG_MAX controls the number of - # max notifications in 5.x, setting CONFIG_BT_ACL_CONNECTIONS - # is enough in 4.x - # https://github.com/esphome/issues/issues/6808 - add_idf_sdkconfig_option("CONFIG_BT_GATTC_NOTIF_REG_MAX", 9) + add_idf_sdkconfig_option("CONFIG_BT_ENABLED", True) + if config.get(CONF_SOFTWARE_COEXISTENCE): + add_idf_sdkconfig_option("CONFIG_SW_COEXIST_ENABLE", True) + # https://github.com/espressif/esp-idf/issues/4101 + # https://github.com/espressif/esp-idf/issues/2503 + # Match arduino CONFIG_BTU_TASK_STACK_SIZE + # https://github.com/espressif/arduino-esp32/blob/fd72cf46ad6fc1a6de99c1d83ba8eba17d80a4ee/tools/sdk/esp32/sdkconfig#L1866 + add_idf_sdkconfig_option("CONFIG_BT_BTU_TASK_STACK_SIZE", 8192) + # Note: CONFIG_BT_ACL_CONNECTIONS and CONFIG_BTDM_CTRL_BLE_MAX_CONN are now + # configured in esp32_ble component based on max_connections setting cg.add_define("USE_OTA_STATE_CALLBACK") # To be notified when an OTA update starts cg.add_define("USE_ESP32_BLE_CLIENT") + + CORE.add_job(_add_ble_features) + if config.get(CONF_SOFTWARE_COEXISTENCE): cg.add_define("USE_ESP32_BLE_SOFTWARE_COEXISTENCE") +# This needs to be run as a job with very low priority so that all components have +# chance to call register_ble_tracker and register_client before the list is checked +# and added to the global defines list. +@coroutine_with_priority(CoroPriority.FINAL) +async def _add_ble_features(): + # Add feature-specific defines based on what's needed + required_features = _get_required_features() + if BLEFeatures.ESP_BT_DEVICE in required_features: + cg.add_define("USE_ESP32_BLE_DEVICE") + cg.add_define("USE_ESP32_BLE_UUID") + + # Add defines for StaticVector sizing based on registration counts + # Only define if count > 0 to avoid allocating unnecessary memory + registration_counts = _get_registration_counts() + if registration_counts.listeners > 0: + cg.add_define( + "ESPHOME_ESP32_BLE_TRACKER_LISTENER_COUNT", registration_counts.listeners + ) + if registration_counts.clients > 0: + cg.add_define( + "ESPHOME_ESP32_BLE_TRACKER_CLIENT_COUNT", registration_counts.clients + ) + + ESP32_BLE_START_SCAN_ACTION_SCHEMA = cv.Schema( { cv.GenerateID(): cv.use_id(ESP32BLETracker), @@ -382,13 +405,47 @@ async def esp32_ble_tracker_stop_scan_action_to_code( return var -async def register_ble_device(var, config): +async def register_ble_device( + var: cg.SafeExpType, config: ConfigType +) -> cg.SafeExpType: + register_ble_features({BLEFeatures.ESP_BT_DEVICE}) + _get_registration_counts().listeners += 1 paren = await cg.get_variable(config[CONF_ESP32_BLE_ID]) cg.add(paren.register_listener(var)) return var -async def register_client(var, config): +async def register_client(var: cg.SafeExpType, config: ConfigType) -> cg.SafeExpType: + register_ble_features({BLEFeatures.ESP_BT_DEVICE}) + _get_registration_counts().clients += 1 + paren = await cg.get_variable(config[CONF_ESP32_BLE_ID]) + cg.add(paren.register_client(var)) + return var + + +async def register_raw_ble_device( + var: cg.SafeExpType, config: ConfigType +) -> cg.SafeExpType: + """Register a BLE device listener that only needs raw advertisement data. + + This does NOT register the ESP_BT_DEVICE feature, meaning ESPBTDevice + will not be compiled in if this is the only registration method used. + """ + _get_registration_counts().listeners += 1 + paren = await cg.get_variable(config[CONF_ESP32_BLE_ID]) + cg.add(paren.register_listener(var)) + return var + + +async def register_raw_client( + var: cg.SafeExpType, config: ConfigType +) -> cg.SafeExpType: + """Register a BLE client that only needs raw advertisement data. + + This does NOT register the ESP_BT_DEVICE feature, meaning ESPBTDevice + will not be compiled in if this is the only registration method used. + """ + _get_registration_counts().clients += 1 paren = await cg.get_variable(config[CONF_ESP32_BLE_ID]) cg.add(paren.register_client(var)) return var diff --git a/esphome/components/esp32_ble_tracker/automation.h b/esphome/components/esp32_ble_tracker/automation.h index 6bef9edcb3..784f2eaaa2 100644 --- a/esphome/components/esp32_ble_tracker/automation.h +++ b/esphome/components/esp32_ble_tracker/automation.h @@ -5,8 +5,8 @@ #ifdef USE_ESP32 -namespace esphome { -namespace esp32_ble_tracker { +namespace esphome::esp32_ble_tracker { +#ifdef USE_ESP32_BLE_DEVICE class ESPBTAdvertiseTrigger : public Trigger, public ESPBTDeviceListener { public: explicit ESPBTAdvertiseTrigger(ESP32BLETracker *parent) { parent->register_listener(this); } @@ -80,11 +80,15 @@ class BLEManufacturerDataAdvertiseTrigger : public Trigger, ESPBTUUID uuid_; }; +#endif // USE_ESP32_BLE_DEVICE + class BLEEndOfScanTrigger : public Trigger<>, public ESPBTDeviceListener { public: explicit BLEEndOfScanTrigger(ESP32BLETracker *parent) { parent->register_listener(this); } +#ifdef USE_ESP32_BLE_DEVICE bool parse_device(const ESPBTDevice &device) override { return false; } +#endif void on_scan_end() override { this->trigger(); } }; @@ -106,7 +110,6 @@ template class ESP32BLEStopScanAction : public Action, pu void play(Ts... x) override { this->parent_->stop_scan(); } }; -} // namespace esp32_ble_tracker -} // namespace esphome +} // namespace esphome::esp32_ble_tracker #endif diff --git a/esphome/components/esp32_ble_tracker/esp32_ble_tracker.cpp b/esphome/components/esp32_ble_tracker/esp32_ble_tracker.cpp index d950ccb5f1..8577f12a92 100644 --- a/esphome/components/esp32_ble_tracker/esp32_ble_tracker.cpp +++ b/esphome/components/esp32_ble_tracker/esp32_ble_tracker.cpp @@ -7,7 +7,9 @@ #include "esphome/core/helpers.h" #include "esphome/core/log.h" +#ifndef CONFIG_ESP_HOSTED_ENABLE_BT_BLUEDROID #include +#endif #include #include #include @@ -25,23 +27,39 @@ #include #endif -#ifdef USE_ARDUINO -#include -#endif - #define MBEDTLS_AES_ALT #include // bt_trace.h #undef TAG -namespace esphome { -namespace esp32_ble_tracker { +namespace esphome::esp32_ble_tracker { static const char *const TAG = "esp32_ble_tracker"; ESP32BLETracker *global_esp32_ble_tracker = nullptr; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) +const char *client_state_to_string(ClientState state) { + switch (state) { + case ClientState::INIT: + return "INIT"; + case ClientState::DISCONNECTING: + return "DISCONNECTING"; + case ClientState::IDLE: + return "IDLE"; + case ClientState::DISCOVERED: + return "DISCOVERED"; + case ClientState::CONNECTING: + return "CONNECTING"; + case ClientState::CONNECTED: + return "CONNECTED"; + case ClientState::ESTABLISHED: + return "ESTABLISHED"; + default: + return "UNKNOWN"; + } +} + float ESP32BLETracker::get_setup_priority() const { return setup_priority::AFTER_BLUETOOTH; } void ESP32BLETracker::setup() { @@ -50,13 +68,6 @@ void ESP32BLETracker::setup() { ESP_LOGE(TAG, "BLE Tracker was marked failed by ESP32BLE"); return; } - RAMAllocator allocator; - this->scan_ring_buffer_ = allocator.allocate(SCAN_RESULT_BUFFER_SIZE); - - if (this->scan_ring_buffer_ == nullptr) { - ESP_LOGE(TAG, "Could not allocate ring buffer for BLE Tracker!"); - this->mark_failed(); - } global_esp32_ble_tracker = this; @@ -65,9 +76,11 @@ void ESP32BLETracker::setup() { [this](ota::OTAState state, float progress, uint8_t error, ota::OTAComponent *comp) { if (state == ota::OTA_STARTED) { this->stop_scan(); +#ifdef ESPHOME_ESP32_BLE_TRACKER_CLIENT_COUNT for (auto *client : this->clients_) { client->disconnect(); } +#endif } }); #endif @@ -84,118 +97,48 @@ void ESP32BLETracker::loop() { this->start_scan(); } } - int connecting = 0; - int discovered = 0; - int searching = 0; - int disconnecting = 0; - for (auto *client : this->clients_) { - switch (client->state()) { - case ClientState::DISCONNECTING: - disconnecting++; - break; - case ClientState::DISCOVERED: - discovered++; - break; - case ClientState::SEARCHING: - searching++; - break; - case ClientState::CONNECTING: - case ClientState::READY_TO_CONNECT: - connecting++; - break; - default: - break; - } - } - if (connecting != connecting_ || discovered != discovered_ || searching != searching_ || - disconnecting != disconnecting_) { - connecting_ = connecting; - discovered_ = discovered; - searching_ = searching; - disconnecting_ = disconnecting; - ESP_LOGD(TAG, "connecting: %d, discovered: %d, searching: %d, disconnecting: %d", connecting_, discovered_, - searching_, disconnecting_); - } - bool promote_to_connecting = discovered && !searching && !connecting; - // Process scan results from lock-free SPSC ring buffer - // Consumer side: This runs in the main loop thread + // Check for scan timeout - moved here from scheduler to avoid false reboots + // when the loop is blocked if (this->scanner_state_ == ScannerState::RUNNING) { - // Load our own index with relaxed ordering (we're the only writer) - uint8_t read_idx = this->ring_read_index_.load(std::memory_order_relaxed); - - // Load producer's index with acquire to see their latest writes - uint8_t write_idx = this->ring_write_index_.load(std::memory_order_acquire); - - while (read_idx != write_idx) { - // Process one result at a time directly from ring buffer - BLEScanResult &scan_result = this->scan_ring_buffer_[read_idx]; - - if (this->raw_advertisements_) { - for (auto *listener : this->listeners_) { - listener->parse_devices(&scan_result, 1); - } - for (auto *client : this->clients_) { - client->parse_devices(&scan_result, 1); + switch (this->scan_timeout_state_) { + case ScanTimeoutState::MONITORING: { + uint32_t now = App.get_loop_component_start_time(); + uint32_t timeout_ms = this->scan_duration_ * 2000; + // Robust time comparison that handles rollover correctly + // This works because unsigned arithmetic wraps around predictably + if ((now - this->scan_start_time_) > timeout_ms) { + // First time we've seen the timeout exceeded - wait one more loop iteration + // This ensures all components have had a chance to process pending events + // This is because esp32_ble may not have run yet and called + // gap_scan_event_handler yet when the loop unblocks + ESP_LOGW(TAG, "Scan timeout exceeded"); + this->scan_timeout_state_ = ScanTimeoutState::EXCEEDED_WAIT; } + break; } + case ScanTimeoutState::EXCEEDED_WAIT: + // We've waited at least one full loop iteration, and scan is still running + ESP_LOGE(TAG, "Scan never terminated, rebooting"); + App.reboot(); + break; - if (this->parse_advertisements_) { - ESPBTDevice device; - device.parse_scan_rst(scan_result); - - bool found = false; - for (auto *listener : this->listeners_) { - if (listener->parse_device(device)) - found = true; - } - - for (auto *client : this->clients_) { - if (client->parse_device(device)) { - found = true; - if (!connecting && client->state() == ClientState::DISCOVERED) { - promote_to_connecting = true; - } - } - } - - if (!found && !this->scan_continuous_) { - this->print_bt_device_info(device); - } - } - - // Move to next entry in ring buffer - read_idx = (read_idx + 1) % SCAN_RESULT_BUFFER_SIZE; - - // Store with release to ensure reads complete before index update - this->ring_read_index_.store(read_idx, std::memory_order_release); - } - - // Log dropped results periodically - size_t dropped = this->scan_results_dropped_.exchange(0, std::memory_order_relaxed); - if (dropped > 0) { - ESP_LOGW(TAG, "Dropped %zu BLE scan results due to buffer overflow", dropped); + case ScanTimeoutState::INACTIVE: + // This case should be unreachable - scanner and timeout states are always synchronized + break; } } - if (this->scanner_state_ == ScannerState::STOPPED) { - this->end_of_scan_(); // Change state to IDLE + + ClientStateCounts counts = this->count_client_states_(); + if (counts != this->client_state_counts_) { + this->client_state_counts_ = counts; + ESP_LOGD(TAG, "connecting: %d, discovered: %d, disconnecting: %d", this->client_state_counts_.connecting, + this->client_state_counts_.discovered, this->client_state_counts_.disconnecting); } + if (this->scanner_state_ == ScannerState::FAILED || (this->scan_set_param_failed_ && this->scanner_state_ == ScannerState::RUNNING)) { - this->stop_scan_(); - if (this->scan_start_fail_count_ == std::numeric_limits::max()) { - ESP_LOGE(TAG, "Scan could not restart after %d attempts, rebooting to restore stack (IDF)", - std::numeric_limits::max()); - App.reboot(); - } - if (this->scan_start_failed_) { - ESP_LOGE(TAG, "Scan start failed: %d", this->scan_start_failed_); - this->scan_start_failed_ = ESP_BT_STATUS_SUCCESS; - } - if (this->scan_set_param_failed_) { - ESP_LOGE(TAG, "Scan set param failed: %d", this->scan_set_param_failed_); - this->scan_set_param_failed_ = ESP_BT_STATUS_SUCCESS; - } + this->handle_scanner_failure_(); } /* @@ -210,45 +153,23 @@ void ESP32BLETracker::loop() { https://github.com/espressif/esp-idf/issues/6688 */ - if (this->scanner_state_ == ScannerState::IDLE && !connecting && !disconnecting && !promote_to_connecting) { + + if (this->scanner_state_ == ScannerState::IDLE && !counts.connecting && !counts.disconnecting && !counts.discovered) { #ifdef USE_ESP32_BLE_SOFTWARE_COEXISTENCE - if (this->coex_prefer_ble_) { - this->coex_prefer_ble_ = false; - ESP_LOGD(TAG, "Setting coexistence preference to balanced."); - esp_coex_preference_set(ESP_COEX_PREFER_BALANCE); // Reset to default - } + this->update_coex_preference_(false); #endif if (this->scan_continuous_) { this->start_scan_(false); // first = false } } // If there is a discovered client and no connecting - // clients and no clients using the scanner to search for - // devices, then stop scanning and promote the discovered - // client to ready to connect. - if (promote_to_connecting && + // clients, then promote the discovered client to ready to connect. + // We check both RUNNING and IDLE states because: + // - RUNNING: gap_scan_event_handler initiates stop_scan_() but promotion can happen immediately + // - IDLE: Scanner has already stopped (naturally or by gap_scan_event_handler) + if (counts.discovered && !counts.connecting && (this->scanner_state_ == ScannerState::RUNNING || this->scanner_state_ == ScannerState::IDLE)) { - for (auto *client : this->clients_) { - if (client->state() == ClientState::DISCOVERED) { - if (this->scanner_state_ == ScannerState::RUNNING) { - ESP_LOGD(TAG, "Stopping scan to make connection"); - this->stop_scan_(); - } else if (this->scanner_state_ == ScannerState::IDLE) { - ESP_LOGD(TAG, "Promoting client to connect"); - // We only want to promote one client at a time. - // once the scanner is fully stopped. -#ifdef USE_ESP32_BLE_SOFTWARE_COEXISTENCE - ESP_LOGD(TAG, "Setting coexistence to Bluetooth to make connection."); - if (!this->coex_prefer_ble_) { - this->coex_prefer_ble_ = true; - esp_coex_preference_set(ESP_COEX_PREFER_BT); // Prioritize Bluetooth - } -#endif - client->set_state(ClientState::READY_TO_CONNECT); - } - break; - } - } + this->try_promote_discovered_clients_(); } } @@ -264,18 +185,11 @@ void ESP32BLETracker::ble_before_disabled_event_handler() { this->stop_scan_(); void ESP32BLETracker::stop_scan_() { if (this->scanner_state_ != ScannerState::RUNNING && this->scanner_state_ != ScannerState::FAILED) { - if (this->scanner_state_ == ScannerState::IDLE) { - ESP_LOGE(TAG, "Scan is already stopped while trying to stop."); - } else if (this->scanner_state_ == ScannerState::STARTING) { - ESP_LOGE(TAG, "Scan is starting while trying to stop."); - } else if (this->scanner_state_ == ScannerState::STOPPING) { - ESP_LOGE(TAG, "Scan is already stopping while trying to stop."); - } else if (this->scanner_state_ == ScannerState::STOPPED) { - ESP_LOGE(TAG, "Scan is already stopped while trying to stop."); - } + ESP_LOGE(TAG, "Cannot stop scan: %s", this->scanner_state_to_string_(this->scanner_state_)); return; } - this->cancel_timeout("scan"); + // Reset timeout state machine when stopping scan + this->scan_timeout_state_ = ScanTimeoutState::INACTIVE; this->set_scanner_state_(ScannerState::STOPPING); esp_err_t err = esp_ble_gap_stop_scanning(); if (err != ESP_OK) { @@ -290,37 +204,30 @@ void ESP32BLETracker::start_scan_(bool first) { return; } if (this->scanner_state_ != ScannerState::IDLE) { - if (this->scanner_state_ == ScannerState::STARTING) { - ESP_LOGE(TAG, "Cannot start scan while already starting."); - } else if (this->scanner_state_ == ScannerState::RUNNING) { - ESP_LOGE(TAG, "Cannot start scan while already running."); - } else if (this->scanner_state_ == ScannerState::STOPPING) { - ESP_LOGE(TAG, "Cannot start scan while already stopping."); - } else if (this->scanner_state_ == ScannerState::FAILED) { - ESP_LOGE(TAG, "Cannot start scan while already failed."); - } else if (this->scanner_state_ == ScannerState::STOPPED) { - ESP_LOGE(TAG, "Cannot start scan while already stopped."); - } + this->log_unexpected_state_("start scan", ScannerState::IDLE); return; } this->set_scanner_state_(ScannerState::STARTING); ESP_LOGD(TAG, "Starting scan, set scanner state to STARTING."); if (!first) { +#ifdef ESPHOME_ESP32_BLE_TRACKER_LISTENER_COUNT for (auto *listener : this->listeners_) listener->on_scan_end(); +#endif } +#ifdef USE_ESP32_BLE_DEVICE this->already_discovered_.clear(); +#endif this->scan_params_.scan_type = this->scan_active_ ? BLE_SCAN_TYPE_ACTIVE : BLE_SCAN_TYPE_PASSIVE; this->scan_params_.own_addr_type = BLE_ADDR_TYPE_PUBLIC; this->scan_params_.scan_filter_policy = BLE_SCAN_FILTER_ALLOW_ALL; this->scan_params_.scan_interval = this->scan_interval_; this->scan_params_.scan_window = this->scan_window_; - // Start timeout before scan is started. Otherwise scan never starts if any error. - this->set_timeout("scan", this->scan_duration_ * 2000, []() { - ESP_LOGE(TAG, "Scan never terminated, rebooting to restore stack (IDF)"); - App.reboot(); - }); + // Start timeout monitoring in loop() instead of using scheduler + // This prevents false reboots when the loop is blocked + this->scan_start_time_ = App.get_loop_component_start_time(); + this->scan_timeout_state_ = ScanTimeoutState::MONITORING; esp_err_t err = esp_ble_gap_set_scan_params(&this->scan_params_); if (err != ESP_OK) { @@ -334,36 +241,26 @@ void ESP32BLETracker::start_scan_(bool first) { } } -void ESP32BLETracker::end_of_scan_() { - // The lock must be held when calling this function. - if (this->scanner_state_ != ScannerState::STOPPED) { - ESP_LOGE(TAG, "end_of_scan_ called while scanner is not stopped."); - return; - } - ESP_LOGD(TAG, "End of scan, set scanner state to IDLE."); - this->already_discovered_.clear(); - this->cancel_timeout("scan"); - - for (auto *listener : this->listeners_) - listener->on_scan_end(); - this->set_scanner_state_(ScannerState::IDLE); -} - void ESP32BLETracker::register_client(ESPBTClient *client) { +#ifdef ESPHOME_ESP32_BLE_TRACKER_CLIENT_COUNT client->app_id = ++this->app_id_; this->clients_.push_back(client); this->recalculate_advertisement_parser_types(); +#endif } void ESP32BLETracker::register_listener(ESPBTDeviceListener *listener) { +#ifdef ESPHOME_ESP32_BLE_TRACKER_LISTENER_COUNT listener->set_parent(this); this->listeners_.push_back(listener); this->recalculate_advertisement_parser_types(); +#endif } void ESP32BLETracker::recalculate_advertisement_parser_types() { this->raw_advertisements_ = false; this->parse_advertisements_ = false; +#ifdef ESPHOME_ESP32_BLE_TRACKER_LISTENER_COUNT for (auto *listener : this->listeners_) { if (listener->get_advertisement_parser_type() == AdvertisementParserType::PARSED_ADVERTISEMENTS) { this->parse_advertisements_ = true; @@ -371,6 +268,8 @@ void ESP32BLETracker::recalculate_advertisement_parser_types() { this->raw_advertisements_ = true; } } +#endif +#ifdef ESPHOME_ESP32_BLE_TRACKER_CLIENT_COUNT for (auto *client : this->clients_) { if (client->get_advertisement_parser_type() == AdvertisementParserType::PARSED_ADVERTISEMENTS) { this->parse_advertisements_ = true; @@ -378,9 +277,12 @@ void ESP32BLETracker::recalculate_advertisement_parser_types() { this->raw_advertisements_ = true; } } +#endif } void ESP32BLETracker::gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param) { + // Note: This handler is called from the main loop context, not directly from the BT task. + // The esp32_ble component queues events via enqueue_ble_event() and processes them in loop(). switch (event) { case ESP_GAP_BLE_SCAN_PARAM_SET_COMPLETE_EVT: this->gap_scan_set_param_complete_(param->scan_param_cmpl); @@ -394,58 +296,34 @@ void ESP32BLETracker::gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_ga default: break; } - // Forward all events to clients (scan results are handled separately via gap_scan_event_handler) + // Forward all events to clients (scan results are handled separately via gap_scan_event_handler) +#ifdef ESPHOME_ESP32_BLE_TRACKER_CLIENT_COUNT for (auto *client : this->clients_) { client->gap_event_handler(event, param); } +#endif } void ESP32BLETracker::gap_scan_event_handler(const BLEScanResult &scan_result) { - ESP_LOGV(TAG, "gap_scan_result - event %d", scan_result.search_evt); + // Note: This handler is called from the main loop context via esp32_ble's event queue. + // We process advertisements immediately instead of buffering them. + ESP_LOGVV(TAG, "gap_scan_result - event %d", scan_result.search_evt); if (scan_result.search_evt == ESP_GAP_SEARCH_INQ_RES_EVT) { - // Lock-free SPSC ring buffer write (Producer side) - // This runs in the ESP-IDF Bluetooth stack callback thread - // IMPORTANT: Only this thread writes to ring_write_index_ - - // Load our own index with relaxed ordering (we're the only writer) - uint8_t write_idx = this->ring_write_index_.load(std::memory_order_relaxed); - uint8_t next_write_idx = (write_idx + 1) % SCAN_RESULT_BUFFER_SIZE; - - // Load consumer's index with acquire to see their latest updates - uint8_t read_idx = this->ring_read_index_.load(std::memory_order_acquire); - - // Check if buffer is full - if (next_write_idx != read_idx) { - // Write to ring buffer - this->scan_ring_buffer_[write_idx] = scan_result; - - // Store with release to ensure the write is visible before index update - this->ring_write_index_.store(next_write_idx, std::memory_order_release); - } else { - // Buffer full, track dropped results - this->scan_results_dropped_.fetch_add(1, std::memory_order_relaxed); - } + // Process the scan result immediately + this->process_scan_result_(scan_result); } else if (scan_result.search_evt == ESP_GAP_SEARCH_INQ_CMPL_EVT) { // Scan finished on its own if (this->scanner_state_ != ScannerState::RUNNING) { - if (this->scanner_state_ == ScannerState::STOPPING) { - ESP_LOGE(TAG, "Scan was not running when scan completed."); - } else if (this->scanner_state_ == ScannerState::STARTING) { - ESP_LOGE(TAG, "Scan was not started when scan completed."); - } else if (this->scanner_state_ == ScannerState::FAILED) { - ESP_LOGE(TAG, "Scan was in failed state when scan completed."); - } else if (this->scanner_state_ == ScannerState::IDLE) { - ESP_LOGE(TAG, "Scan was idle when scan completed."); - } else if (this->scanner_state_ == ScannerState::STOPPED) { - ESP_LOGE(TAG, "Scan was stopped when scan completed."); - } + this->log_unexpected_state_("scan complete", ScannerState::RUNNING); } - this->set_scanner_state_(ScannerState::STOPPED); + // Scan completed naturally, perform cleanup and transition to IDLE + this->cleanup_scan_state_(false); } } void ESP32BLETracker::gap_scan_set_param_complete_(const esp_ble_gap_cb_param_t::ble_scan_param_cmpl_evt_param ¶m) { + // Called from main loop context via gap_event_handler after being queued from BT task ESP_LOGV(TAG, "gap_scan_set_param_complete - status %d", param.status); if (param.status == ESP_BT_STATUS_DONE) { this->scan_set_param_failed_ = ESP_BT_STATUS_SUCCESS; @@ -455,20 +333,11 @@ void ESP32BLETracker::gap_scan_set_param_complete_(const esp_ble_gap_cb_param_t: } void ESP32BLETracker::gap_scan_start_complete_(const esp_ble_gap_cb_param_t::ble_scan_start_cmpl_evt_param ¶m) { + // Called from main loop context via gap_event_handler after being queued from BT task ESP_LOGV(TAG, "gap_scan_start_complete - status %d", param.status); this->scan_start_failed_ = param.status; if (this->scanner_state_ != ScannerState::STARTING) { - if (this->scanner_state_ == ScannerState::RUNNING) { - ESP_LOGE(TAG, "Scan was already running when start complete."); - } else if (this->scanner_state_ == ScannerState::STOPPING) { - ESP_LOGE(TAG, "Scan was stopping when start complete."); - } else if (this->scanner_state_ == ScannerState::FAILED) { - ESP_LOGE(TAG, "Scan was in failed state when start complete."); - } else if (this->scanner_state_ == ScannerState::IDLE) { - ESP_LOGE(TAG, "Scan was idle when start complete."); - } else if (this->scanner_state_ == ScannerState::STOPPED) { - ESP_LOGE(TAG, "Scan was stopped when start complete."); - } + this->log_unexpected_state_("start complete", ScannerState::STARTING); } if (param.status == ESP_BT_STATUS_SUCCESS) { this->scan_start_fail_count_ = 0; @@ -482,28 +351,24 @@ void ESP32BLETracker::gap_scan_start_complete_(const esp_ble_gap_cb_param_t::ble } void ESP32BLETracker::gap_scan_stop_complete_(const esp_ble_gap_cb_param_t::ble_scan_stop_cmpl_evt_param ¶m) { + // Called from main loop context via gap_event_handler after being queued from BT task + // This allows us to safely transition to IDLE state and perform cleanup without race conditions ESP_LOGV(TAG, "gap_scan_stop_complete - status %d", param.status); if (this->scanner_state_ != ScannerState::STOPPING) { - if (this->scanner_state_ == ScannerState::RUNNING) { - ESP_LOGE(TAG, "Scan was not running when stop complete."); - } else if (this->scanner_state_ == ScannerState::STARTING) { - ESP_LOGE(TAG, "Scan was not started when stop complete."); - } else if (this->scanner_state_ == ScannerState::FAILED) { - ESP_LOGE(TAG, "Scan was in failed state when stop complete."); - } else if (this->scanner_state_ == ScannerState::IDLE) { - ESP_LOGE(TAG, "Scan was idle when stop complete."); - } else if (this->scanner_state_ == ScannerState::STOPPED) { - ESP_LOGE(TAG, "Scan was stopped when stop complete."); - } + this->log_unexpected_state_("stop complete", ScannerState::STOPPING); } - this->set_scanner_state_(ScannerState::STOPPED); + + // Perform cleanup and transition to IDLE + this->cleanup_scan_state_(true); } void ESP32BLETracker::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gattc_cb_param_t *param) { +#ifdef ESPHOME_ESP32_BLE_TRACKER_CLIENT_COUNT for (auto *client : this->clients_) { client->gattc_event_handler(event, gattc_if, param); } +#endif } void ESP32BLETracker::set_scanner_state_(ScannerState state) { @@ -511,6 +376,7 @@ void ESP32BLETracker::set_scanner_state_(ScannerState state) { this->scanner_state_callbacks_.call(state); } +#ifdef USE_ESP32_BLE_DEVICE ESPBLEiBeacon::ESPBLEiBeacon(const uint8_t *data) { memcpy(&this->beacon_data_, data, sizeof(beacon_data_)); } optional ESPBLEiBeacon::from_manufacturer_data(const ServiceData &data) { if (!data.uuid.contains(0x4C, 0x00)) @@ -751,13 +617,15 @@ void ESPBTDevice::parse_adv_(const uint8_t *payload, uint8_t len) { } } } + std::string ESPBTDevice::address_str() const { - char mac[24]; - snprintf(mac, sizeof(mac), "%02X:%02X:%02X:%02X:%02X:%02X", this->address_[0], this->address_[1], this->address_[2], - this->address_[3], this->address_[4], this->address_[5]); + char mac[18]; + format_mac_addr_upper(this->address_, mac); return mac; } + uint64_t ESPBTDevice::address_uint64() const { return esp32_ble::ble_addr_to_uint64(this->address_); } +#endif // USE_ESP32_BLE_DEVICE void ESP32BLETracker::dump_config() { ESP_LOGCONFIG(TAG, "BLE Tracker:"); @@ -769,33 +637,15 @@ void ESP32BLETracker::dump_config() { " Continuous Scanning: %s", this->scan_duration_, this->scan_interval_ * 0.625f, this->scan_window_ * 0.625f, this->scan_active_ ? "ACTIVE" : "PASSIVE", YESNO(this->scan_continuous_)); - switch (this->scanner_state_) { - case ScannerState::IDLE: - ESP_LOGCONFIG(TAG, " Scanner State: IDLE"); - break; - case ScannerState::STARTING: - ESP_LOGCONFIG(TAG, " Scanner State: STARTING"); - break; - case ScannerState::RUNNING: - ESP_LOGCONFIG(TAG, " Scanner State: RUNNING"); - break; - case ScannerState::STOPPING: - ESP_LOGCONFIG(TAG, " Scanner State: STOPPING"); - break; - case ScannerState::STOPPED: - ESP_LOGCONFIG(TAG, " Scanner State: STOPPED"); - break; - case ScannerState::FAILED: - ESP_LOGCONFIG(TAG, " Scanner State: FAILED"); - break; - } - ESP_LOGCONFIG(TAG, " Connecting: %d, discovered: %d, searching: %d, disconnecting: %d", connecting_, discovered_, - searching_, disconnecting_); + ESP_LOGCONFIG(TAG, " Scanner State: %s", this->scanner_state_to_string_(this->scanner_state_)); + ESP_LOGCONFIG(TAG, " Connecting: %d, discovered: %d, disconnecting: %d", this->client_state_counts_.connecting, + this->client_state_counts_.discovered, this->client_state_counts_.disconnecting); if (this->scan_start_fail_count_) { ESP_LOGCONFIG(TAG, " Scan Start Fail Count: %d", this->scan_start_fail_count_); } } +#ifdef USE_ESP32_BLE_DEVICE void ESP32BLETracker::print_bt_device_info(const ESPBTDevice &device) { const uint64_t address = device.address_uint64(); for (auto &disc : this->already_discovered_) { @@ -867,7 +717,150 @@ bool ESPBTDevice::resolve_irk(const uint8_t *irk) const { ecb_ciphertext[13] == ((addr64 >> 16) & 0xff); } -} // namespace esp32_ble_tracker -} // namespace esphome +#endif // USE_ESP32_BLE_DEVICE +void ESP32BLETracker::process_scan_result_(const BLEScanResult &scan_result) { + // Process raw advertisements + if (this->raw_advertisements_) { +#ifdef ESPHOME_ESP32_BLE_TRACKER_LISTENER_COUNT + for (auto *listener : this->listeners_) { + listener->parse_devices(&scan_result, 1); + } #endif +#ifdef ESPHOME_ESP32_BLE_TRACKER_CLIENT_COUNT + for (auto *client : this->clients_) { + client->parse_devices(&scan_result, 1); + } +#endif + } + + // Process parsed advertisements + if (this->parse_advertisements_) { +#ifdef USE_ESP32_BLE_DEVICE + ESPBTDevice device; + device.parse_scan_rst(scan_result); + + bool found = false; +#ifdef ESPHOME_ESP32_BLE_TRACKER_LISTENER_COUNT + for (auto *listener : this->listeners_) { + if (listener->parse_device(device)) + found = true; + } +#endif + +#ifdef ESPHOME_ESP32_BLE_TRACKER_CLIENT_COUNT + for (auto *client : this->clients_) { + if (client->parse_device(device)) { + found = true; + } + } +#endif + + if (!found && !this->scan_continuous_) { + this->print_bt_device_info(device); + } +#endif // USE_ESP32_BLE_DEVICE + } +} + +void ESP32BLETracker::cleanup_scan_state_(bool is_stop_complete) { + ESP_LOGD(TAG, "Scan %scomplete, set scanner state to IDLE.", is_stop_complete ? "stop " : ""); +#ifdef USE_ESP32_BLE_DEVICE + this->already_discovered_.clear(); +#endif + // Reset timeout state machine instead of cancelling scheduler timeout + this->scan_timeout_state_ = ScanTimeoutState::INACTIVE; + +#ifdef ESPHOME_ESP32_BLE_TRACKER_LISTENER_COUNT + for (auto *listener : this->listeners_) + listener->on_scan_end(); +#endif + + this->set_scanner_state_(ScannerState::IDLE); +} + +void ESP32BLETracker::handle_scanner_failure_() { + this->stop_scan_(); + if (this->scan_start_fail_count_ == std::numeric_limits::max()) { + ESP_LOGE(TAG, "Scan could not restart after %d attempts, rebooting to restore stack (IDF)", + std::numeric_limits::max()); + App.reboot(); + } + if (this->scan_start_failed_) { + ESP_LOGE(TAG, "Scan start failed: %d", this->scan_start_failed_); + this->scan_start_failed_ = ESP_BT_STATUS_SUCCESS; + } + if (this->scan_set_param_failed_) { + ESP_LOGE(TAG, "Scan set param failed: %d", this->scan_set_param_failed_); + this->scan_set_param_failed_ = ESP_BT_STATUS_SUCCESS; + } +} + +void ESP32BLETracker::try_promote_discovered_clients_() { + // Only promote the first discovered client to avoid multiple simultaneous connections +#ifdef ESPHOME_ESP32_BLE_TRACKER_CLIENT_COUNT + for (auto *client : this->clients_) { + if (client->state() != ClientState::DISCOVERED) { + continue; + } + + if (this->scanner_state_ == ScannerState::RUNNING) { + ESP_LOGD(TAG, "Stopping scan to make connection"); + this->stop_scan_(); + // Don't wait for scan stop complete - promote immediately. + // This is safe because ESP-IDF processes BLE commands sequentially through its internal mailbox queue. + // This guarantees that the stop scan command will be fully processed before any subsequent connect command, + // preventing race conditions or overlapping operations. + } + + ESP_LOGD(TAG, "Promoting client to connect"); +#ifdef USE_ESP32_BLE_SOFTWARE_COEXISTENCE + this->update_coex_preference_(true); +#endif + client->connect(); + break; + } +#endif +} + +const char *ESP32BLETracker::scanner_state_to_string_(ScannerState state) const { + switch (state) { + case ScannerState::IDLE: + return "IDLE"; + case ScannerState::STARTING: + return "STARTING"; + case ScannerState::RUNNING: + return "RUNNING"; + case ScannerState::STOPPING: + return "STOPPING"; + case ScannerState::FAILED: + return "FAILED"; + default: + return "UNKNOWN"; + } +} + +void ESP32BLETracker::log_unexpected_state_(const char *operation, ScannerState expected_state) const { + ESP_LOGE(TAG, "Unexpected state: %s on %s, expected: %s", this->scanner_state_to_string_(this->scanner_state_), + operation, this->scanner_state_to_string_(expected_state)); +} + +#ifdef USE_ESP32_BLE_SOFTWARE_COEXISTENCE +void ESP32BLETracker::update_coex_preference_(bool force_ble) { +#ifndef CONFIG_ESP_HOSTED_ENABLE_BT_BLUEDROID + if (force_ble && !this->coex_prefer_ble_) { + ESP_LOGD(TAG, "Setting coexistence to Bluetooth to make connection."); + this->coex_prefer_ble_ = true; + esp_coex_preference_set(ESP_COEX_PREFER_BT); // Prioritize Bluetooth + } else if (!force_ble && this->coex_prefer_ble_) { + ESP_LOGD(TAG, "Setting coexistence preference to balanced."); + this->coex_prefer_ble_ = false; + esp_coex_preference_set(ESP_COEX_PREFER_BALANCE); // Reset to default + } +#endif // CONFIG_ESP_HOSTED_ENABLE_BT_BLUEDROID +} +#endif + +} // namespace esphome::esp32_ble_tracker + +#endif // USE_ESP32 diff --git a/esphome/components/esp32_ble_tracker/esp32_ble_tracker.h b/esphome/components/esp32_ble_tracker/esp32_ble_tracker.h index f5ed75a93e..f80f3e2670 100644 --- a/esphome/components/esp32_ble_tracker/esp32_ble_tracker.h +++ b/esphome/components/esp32_ble_tracker/esp32_ble_tracker.h @@ -6,7 +6,6 @@ #include "esphome/core/helpers.h" #include -#include #include #include @@ -21,9 +20,9 @@ #include "esphome/components/esp32_ble/ble.h" #include "esphome/components/esp32_ble/ble_uuid.h" +#include "esphome/components/esp32_ble/ble_scan_result.h" -namespace esphome { -namespace esp32_ble_tracker { +namespace esphome::esp32_ble_tracker { using namespace esp32_ble; @@ -34,11 +33,14 @@ enum AdvertisementParserType { RAW_ADVERTISEMENTS, }; +#ifdef USE_ESP32_BLE_UUID struct ServiceData { ESPBTUUID uuid; adv_data_t data; }; +#endif +#ifdef USE_ESP32_BLE_DEVICE class ESPBLEiBeacon { public: ESPBLEiBeacon() { memset(&this->beacon_data_, 0, sizeof(this->beacon_data_)); } @@ -116,13 +118,16 @@ class ESPBTDevice { std::vector service_datas_{}; const BLEScanResult *scan_result_{nullptr}; }; +#endif // USE_ESP32_BLE_DEVICE class ESP32BLETracker; class ESPBTDeviceListener { public: virtual void on_scan_end() {} +#ifdef USE_ESP32_BLE_DEVICE virtual bool parse_device(const ESPBTDevice &device) = 0; +#endif virtual bool parse_devices(const BLEScanResult *scan_results, size_t count) { return false; }; virtual AdvertisementParserType get_advertisement_parser_type() { return AdvertisementParserType::PARSED_ADVERTISEMENTS; @@ -133,6 +138,18 @@ class ESPBTDeviceListener { ESP32BLETracker *parent_{nullptr}; }; +struct ClientStateCounts { + uint8_t connecting = 0; + uint8_t discovered = 0; + uint8_t disconnecting = 0; + + bool operator==(const ClientStateCounts &other) const { + return connecting == other.connecting && discovered == other.discovered && disconnecting == other.disconnecting; + } + + bool operator!=(const ClientStateCounts &other) const { return !(*this == other); } +}; + enum class ClientState : uint8_t { // Connection is allocated INIT, @@ -140,12 +157,8 @@ enum class ClientState : uint8_t { DISCONNECTING, // Connection is idle, no device detected. IDLE, - // Searching for device. - SEARCHING, // Device advertisement found. DISCOVERED, - // Device is discovered and the scanner is stopped - READY_TO_CONNECT, // Connection in progress. CONNECTING, // Initial connection established. @@ -155,20 +168,21 @@ enum class ClientState : uint8_t { }; enum class ScannerState { - // Scanner is idle, init state, set from the main loop when processing STOPPED + // Scanner is idle, init state IDLE, - // Scanner is starting, set from the main loop only + // Scanner is starting STARTING, - // Scanner is running, set from the ESP callback only + // Scanner is running RUNNING, - // Scanner failed to start, set from the ESP callback only + // Scanner failed to start FAILED, - // Scanner is stopping, set from the main loop only + // Scanner is stopping STOPPING, - // Scanner is stopped, set from the ESP callback only - STOPPED, }; +// Helper function to convert ClientState to string +const char *client_state_to_string(ClientState state); + enum class ConnectionType : uint8_t { // The default connection type, we hold all the services in ram // for the duration of the connection. @@ -237,7 +251,9 @@ class ESP32BLETracker : public Component, void register_client(ESPBTClient *client); void recalculate_advertisement_parser_types(); +#ifdef USE_ESP32_BLE_DEVICE void print_bt_device_info(const ESPBTDevice &device); +#endif void start_scan(); void stop_scan(); @@ -257,8 +273,6 @@ class ESP32BLETracker : public Component, void stop_scan_(); /// Start a single scan by setting up the parameters and doing some esp-idf calls. void start_scan_(bool first); - /// Called when a scan ends - void end_of_scan_(); /// Called when a `ESP_GAP_BLE_SCAN_RESULT_EVT` event is received. void gap_scan_result_(const esp_ble_gap_cb_param_t::ble_scan_result_evt_param ¶m); /// Called when a `ESP_GAP_BLE_SCAN_PARAM_SET_COMPLETE_EVT` event is received. @@ -269,53 +283,96 @@ class ESP32BLETracker : public Component, void gap_scan_stop_complete_(const esp_ble_gap_cb_param_t::ble_scan_stop_cmpl_evt_param ¶m); /// Called to set the scanner state. Will also call callbacks to let listeners know when state is changed. void set_scanner_state_(ScannerState state); + /// Common cleanup logic when transitioning scanner to IDLE state + void cleanup_scan_state_(bool is_stop_complete); + /// Process a single scan result immediately + void process_scan_result_(const BLEScanResult &scan_result); + /// Handle scanner failure states + void handle_scanner_failure_(); + /// Try to promote discovered clients to ready to connect + void try_promote_discovered_clients_(); + /// Convert scanner state enum to string for logging + const char *scanner_state_to_string_(ScannerState state) const; + /// Log an unexpected scanner state + void log_unexpected_state_(const char *operation, ScannerState expected_state) const; +#ifdef USE_ESP32_BLE_SOFTWARE_COEXISTENCE + /// Update BLE coexistence preference + void update_coex_preference_(bool force_ble); +#endif + /// Count clients in each state + ClientStateCounts count_client_states_() const { + ClientStateCounts counts; +#ifdef ESPHOME_ESP32_BLE_TRACKER_CLIENT_COUNT + for (auto *client : this->clients_) { + switch (client->state()) { + case ClientState::DISCONNECTING: + counts.disconnecting++; + break; + case ClientState::DISCOVERED: + counts.discovered++; + break; + case ClientState::CONNECTING: + counts.connecting++; + break; + default: + break; + } + } +#endif + return counts; + } - uint8_t app_id_{0}; - + // Group 1: Large objects (12+ bytes) - vectors and callback manager +#ifdef ESPHOME_ESP32_BLE_TRACKER_LISTENER_COUNT + StaticVector listeners_; +#endif +#ifdef ESPHOME_ESP32_BLE_TRACKER_CLIENT_COUNT + StaticVector clients_; +#endif + CallbackManager scanner_state_callbacks_; +#ifdef USE_ESP32_BLE_DEVICE /// Vector of addresses that have already been printed in print_bt_device_info std::vector already_discovered_; - std::vector listeners_; - /// Client parameters. - std::vector clients_; +#endif + + // Group 2: Structs (aligned to 4 bytes) /// A structure holding the ESP BLE scan parameters. esp_ble_scan_params_t scan_params_; + ClientStateCounts client_state_counts_; + + // Group 3: 4-byte types /// The interval in seconds to perform scans. uint32_t scan_duration_; uint32_t scan_interval_; uint32_t scan_window_; + esp_bt_status_t scan_start_failed_{ESP_BT_STATUS_SUCCESS}; + esp_bt_status_t scan_set_param_failed_{ESP_BT_STATUS_SUCCESS}; + + // Group 4: 1-byte types (enums, uint8_t, bool) + uint8_t app_id_{0}; uint8_t scan_start_fail_count_{0}; + ScannerState scanner_state_{ScannerState::IDLE}; bool scan_continuous_; bool scan_active_; - ScannerState scanner_state_{ScannerState::IDLE}; - CallbackManager scanner_state_callbacks_; bool ble_was_disabled_{true}; bool raw_advertisements_{false}; bool parse_advertisements_{false}; - - // Lock-free Single-Producer Single-Consumer (SPSC) ring buffer for scan results - // Producer: ESP-IDF Bluetooth stack callback (gap_scan_event_handler) - // Consumer: ESPHome main loop (loop() method) - // This design ensures zero blocking in the BT callback and prevents scan result loss - BLEScanResult *scan_ring_buffer_; - std::atomic ring_write_index_{0}; // Written only by BT callback (producer) - std::atomic ring_read_index_{0}; // Written only by main loop (consumer) - std::atomic scan_results_dropped_{0}; // Tracks buffer overflow events - - esp_bt_status_t scan_start_failed_{ESP_BT_STATUS_SUCCESS}; - esp_bt_status_t scan_set_param_failed_{ESP_BT_STATUS_SUCCESS}; - int connecting_{0}; - int discovered_{0}; - int searching_{0}; - int disconnecting_{0}; #ifdef USE_ESP32_BLE_SOFTWARE_COEXISTENCE bool coex_prefer_ble_{false}; #endif + // Scan timeout state machine + enum class ScanTimeoutState : uint8_t { + INACTIVE, // No timeout monitoring + MONITORING, // Actively monitoring for timeout + EXCEEDED_WAIT, // Timeout exceeded, waiting one loop before reboot + }; + uint32_t scan_start_time_{0}; + ScanTimeoutState scan_timeout_state_{ScanTimeoutState::INACTIVE}; }; // NOLINTNEXTLINE extern ESP32BLETracker *global_esp32_ble_tracker; -} // namespace esp32_ble_tracker -} // namespace esphome +} // namespace esphome::esp32_ble_tracker #endif diff --git a/esphome/components/esp32_camera/__init__.py b/esphome/components/esp32_camera/__init__.py index 6e36f7d5a7..d8ba098645 100644 --- a/esphome/components/esp32_camera/__init__.py +++ b/esphome/components/esp32_camera/__init__.py @@ -1,3 +1,5 @@ +import logging + from esphome import automation, pins import esphome.codegen as cg from esphome.components import i2c @@ -8,6 +10,7 @@ from esphome.const import ( CONF_CONTRAST, CONF_DATA_PINS, CONF_FREQUENCY, + CONF_I2C, CONF_I2C_ID, CONF_ID, CONF_PIN, @@ -18,8 +21,10 @@ from esphome.const import ( CONF_TRIGGER_ID, CONF_VSYNC_PIN, ) -from esphome.core import CORE from esphome.core.entity_helpers import setup_entity +import esphome.final_validate as fv + +_LOGGER = logging.getLogger(__name__) DEPENDENCIES = ["esp32"] @@ -113,6 +118,12 @@ ENUM_SPECIAL_EFFECT = { "SEPIA": ESP32SpecialEffect.ESP32_SPECIAL_EFFECT_SEPIA, } +camera_fb_location_t = cg.global_ns.enum("camera_fb_location_t") +ENUM_FB_LOCATION = { + "PSRAM": cg.global_ns.CAMERA_FB_IN_PSRAM, + "DRAM": cg.global_ns.CAMERA_FB_IN_DRAM, +} + # pin assignment CONF_HREF_PIN = "href_pin" CONF_PIXEL_CLOCK_PIN = "pixel_clock_pin" @@ -143,6 +154,7 @@ CONF_MAX_FRAMERATE = "max_framerate" CONF_IDLE_FRAMERATE = "idle_framerate" # frame buffer CONF_FRAME_BUFFER_COUNT = "frame_buffer_count" +CONF_FRAME_BUFFER_LOCATION = "frame_buffer_location" # stream trigger CONF_ON_STREAM_START = "on_stream_start" @@ -224,6 +236,9 @@ CONFIG_SCHEMA = cv.All( cv.framerate, cv.Range(min=0, max=1) ), cv.Optional(CONF_FRAME_BUFFER_COUNT, default=1): cv.int_range(min=1, max=2), + cv.Optional(CONF_FRAME_BUFFER_LOCATION, default="PSRAM"): cv.enum( + ENUM_FB_LOCATION, upper=True + ), cv.Optional(CONF_ON_STREAM_START): automation.validate_automation( { cv.GenerateID(CONF_TRIGGER_ID): cv.declare_id( @@ -250,6 +265,22 @@ CONFIG_SCHEMA = cv.All( cv.has_exactly_one_key(CONF_I2C_PINS, CONF_I2C_ID), ) + +def _final_validate(config): + if CONF_I2C_PINS not in config: + return + fconf = fv.full_config.get() + if fconf.get(CONF_I2C): + raise cv.Invalid( + "The `i2c_pins:` config option is incompatible with an dedicated `i2c:` block, use `i2c_id` instead" + ) + _LOGGER.warning( + "The `i2c_pins:` config option is deprecated. Use `i2c_id:` with a dedicated `i2c:` definition instead." + ) + + +FINAL_VALIDATE_SCHEMA = _final_validate + SETTERS = { # pin assignment CONF_DATA_PINS: "set_data_pins", @@ -279,6 +310,7 @@ SETTERS = { CONF_WB_MODE: "set_wb_mode", # test pattern CONF_TEST_PATTERN: "set_test_pattern", + CONF_FRAME_BUFFER_LOCATION: "set_frame_buffer_location", } @@ -306,12 +338,12 @@ async def to_code(config): else: cg.add(var.set_idle_update_interval(1000 / config[CONF_IDLE_FRAMERATE])) cg.add(var.set_frame_buffer_count(config[CONF_FRAME_BUFFER_COUNT])) + cg.add(var.set_frame_buffer_location(config[CONF_FRAME_BUFFER_LOCATION])) cg.add(var.set_frame_size(config[CONF_RESOLUTION])) cg.add_define("USE_CAMERA") - if CORE.using_esp_idf: - add_idf_component(name="espressif/esp32-camera", ref="2.0.15") + add_idf_component(name="espressif/esp32-camera", ref="2.1.1") for conf in config.get(CONF_ON_STREAM_START, []): trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var) diff --git a/esphome/components/esp32_camera/esp32_camera.cpp b/esphome/components/esp32_camera/esp32_camera.cpp index eadb8a4408..38bd8d5822 100644 --- a/esphome/components/esp32_camera/esp32_camera.cpp +++ b/esphome/components/esp32_camera/esp32_camera.cpp @@ -133,6 +133,7 @@ void ESP32Camera::dump_config() { ESP_LOGCONFIG(TAG, " JPEG Quality: %u\n" " Framebuffer Count: %u\n" + " Framebuffer Location: %s\n" " Contrast: %d\n" " Brightness: %d\n" " Saturation: %d\n" @@ -140,8 +141,9 @@ void ESP32Camera::dump_config() { " Horizontal Mirror: %s\n" " Special Effect: %u\n" " White Balance Mode: %u", - st.quality, conf.fb_count, st.contrast, st.brightness, st.saturation, ONOFF(st.vflip), - ONOFF(st.hmirror), st.special_effect, st.wb_mode); + st.quality, conf.fb_count, this->config_.fb_location == CAMERA_FB_IN_PSRAM ? "PSRAM" : "DRAM", + st.contrast, st.brightness, st.saturation, ONOFF(st.vflip), ONOFF(st.hmirror), st.special_effect, + st.wb_mode); // ESP_LOGCONFIG(TAG, " Auto White Balance: %u", st.awb); // ESP_LOGCONFIG(TAG, " Auto White Balance Gain: %u", st.awb_gain); ESP_LOGCONFIG(TAG, @@ -350,6 +352,9 @@ void ESP32Camera::set_frame_buffer_count(uint8_t fb_count) { this->config_.fb_count = fb_count; this->set_frame_buffer_mode(fb_count > 1 ? CAMERA_GRAB_LATEST : CAMERA_GRAB_WHEN_EMPTY); } +void ESP32Camera::set_frame_buffer_location(camera_fb_location_t fb_location) { + this->config_.fb_location = fb_location; +} /* ---------------- public API (specific) ---------------- */ void ESP32Camera::add_image_callback(std::function)> &&callback) { diff --git a/esphome/components/esp32_camera/esp32_camera.h b/esphome/components/esp32_camera/esp32_camera.h index 8ce3faf039..0e7f7c0ea6 100644 --- a/esphome/components/esp32_camera/esp32_camera.h +++ b/esphome/components/esp32_camera/esp32_camera.h @@ -152,6 +152,7 @@ class ESP32Camera : public camera::Camera { /* -- frame buffer */ void set_frame_buffer_mode(camera_grab_mode_t mode); void set_frame_buffer_count(uint8_t fb_count); + void set_frame_buffer_location(camera_fb_location_t fb_location); /* public API (derivated) */ void setup() override; diff --git a/esphome/components/esp32_can/esp32_can.cpp b/esphome/components/esp32_can/esp32_can.cpp index b5e72497ce..cdef7b1930 100644 --- a/esphome/components/esp32_can/esp32_can.cpp +++ b/esphome/components/esp32_can/esp32_can.cpp @@ -67,8 +67,16 @@ static bool get_bitrate(canbus::CanSpeed bitrate, twai_timing_config_t *t_config } bool ESP32Can::setup_internal() { + static int next_twai_ctrl_num = 0; + if (static_cast(next_twai_ctrl_num) >= SOC_TWAI_CONTROLLER_NUM) { + ESP_LOGW(TAG, "Maximum number of esp32_can components created already"); + this->mark_failed(); + return false; + } + twai_general_config_t g_config = TWAI_GENERAL_CONFIG_DEFAULT((gpio_num_t) this->tx_, (gpio_num_t) this->rx_, TWAI_MODE_NORMAL); + g_config.controller_id = next_twai_ctrl_num++; if (this->tx_queue_len_.has_value()) { g_config.tx_queue_len = this->tx_queue_len_.value(); } @@ -86,14 +94,14 @@ bool ESP32Can::setup_internal() { } // Install TWAI driver - if (twai_driver_install(&g_config, &t_config, &f_config) != ESP_OK) { + if (twai_driver_install_v2(&g_config, &t_config, &f_config, &(this->twai_handle_)) != ESP_OK) { // Failed to install driver this->mark_failed(); return false; } // Start TWAI driver - if (twai_start() != ESP_OK) { + if (twai_start_v2(this->twai_handle_) != ESP_OK) { // Failed to start driver this->mark_failed(); return false; @@ -102,6 +110,11 @@ bool ESP32Can::setup_internal() { } canbus::Error ESP32Can::send_message(struct canbus::CanFrame *frame) { + if (this->twai_handle_ == nullptr) { + // not setup yet or setup failed + return canbus::ERROR_FAIL; + } + if (frame->can_data_length_code > canbus::CAN_MAX_DATA_LENGTH) { return canbus::ERROR_FAILTX; } @@ -124,7 +137,7 @@ canbus::Error ESP32Can::send_message(struct canbus::CanFrame *frame) { memcpy(message.data, frame->data, frame->can_data_length_code); } - if (twai_transmit(&message, this->tx_enqueue_timeout_ticks_) == ESP_OK) { + if (twai_transmit_v2(this->twai_handle_, &message, this->tx_enqueue_timeout_ticks_) == ESP_OK) { return canbus::ERROR_OK; } else { return canbus::ERROR_ALLTXBUSY; @@ -132,9 +145,14 @@ canbus::Error ESP32Can::send_message(struct canbus::CanFrame *frame) { } canbus::Error ESP32Can::read_message(struct canbus::CanFrame *frame) { + if (this->twai_handle_ == nullptr) { + // not setup yet or setup failed + return canbus::ERROR_FAIL; + } + twai_message_t message; - if (twai_receive(&message, 0) != ESP_OK) { + if (twai_receive_v2(this->twai_handle_, &message, 0) != ESP_OK) { return canbus::ERROR_NOMSG; } diff --git a/esphome/components/esp32_can/esp32_can.h b/esphome/components/esp32_can/esp32_can.h index 416f037083..dc44aceb36 100644 --- a/esphome/components/esp32_can/esp32_can.h +++ b/esphome/components/esp32_can/esp32_can.h @@ -5,6 +5,8 @@ #include "esphome/components/canbus/canbus.h" #include "esphome/core/component.h" +#include + namespace esphome { namespace esp32_can { @@ -29,6 +31,7 @@ class ESP32Can : public canbus::Canbus { TickType_t tx_enqueue_timeout_ticks_{}; optional tx_queue_len_{}; optional rx_queue_len_{}; + twai_handle_t twai_handle_{nullptr}; }; } // namespace esp32_can diff --git a/esphome/components/esp32_dac/esp32_dac.cpp b/esphome/components/esp32_dac/esp32_dac.cpp index 01bf0e04c3..8f226a5cc2 100644 --- a/esphome/components/esp32_dac/esp32_dac.cpp +++ b/esphome/components/esp32_dac/esp32_dac.cpp @@ -2,11 +2,7 @@ #include "esphome/core/helpers.h" #include "esphome/core/log.h" -#ifdef USE_ESP32 - -#ifdef USE_ARDUINO -#include -#endif +#if defined(USE_ESP32_VARIANT_ESP32) || defined(USE_ESP32_VARIANT_ESP32S2) namespace esphome { namespace esp32_dac { @@ -20,22 +16,15 @@ static constexpr uint8_t DAC0_PIN = 25; static const char *const TAG = "esp32_dac"; void ESP32DAC::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->pin_->setup(); this->turn_off(); -#ifdef USE_ESP_IDF const dac_channel_t channel = this->pin_->get_pin() == DAC0_PIN ? DAC_CHAN_0 : DAC_CHAN_1; const dac_oneshot_config_t oneshot_cfg{channel}; dac_oneshot_new_channel(&oneshot_cfg, &this->dac_handle_); -#endif } -void ESP32DAC::on_safe_shutdown() { -#ifdef USE_ESP_IDF - dac_oneshot_del_channel(this->dac_handle_); -#endif -} +void ESP32DAC::on_safe_shutdown() { dac_oneshot_del_channel(this->dac_handle_); } void ESP32DAC::dump_config() { ESP_LOGCONFIG(TAG, "ESP32 DAC:"); @@ -49,15 +38,10 @@ void ESP32DAC::write_state(float state) { state = state * 255; -#ifdef USE_ESP_IDF dac_oneshot_output_voltage(this->dac_handle_, state); -#endif -#ifdef USE_ARDUINO - dacWrite(this->pin_->get_pin(), state); -#endif } } // namespace esp32_dac } // namespace esphome -#endif +#endif // USE_ESP32_VARIANT_ESP32 || USE_ESP32_VARIANT_ESP32S2 diff --git a/esphome/components/esp32_dac/esp32_dac.h b/esphome/components/esp32_dac/esp32_dac.h index 63d0c914a1..95c687d307 100644 --- a/esphome/components/esp32_dac/esp32_dac.h +++ b/esphome/components/esp32_dac/esp32_dac.h @@ -1,15 +1,13 @@ #pragma once +#include "esphome/components/output/float_output.h" +#include "esphome/core/automation.h" #include "esphome/core/component.h" #include "esphome/core/hal.h" -#include "esphome/core/automation.h" -#include "esphome/components/output/float_output.h" -#ifdef USE_ESP32 +#if defined(USE_ESP32_VARIANT_ESP32) || defined(USE_ESP32_VARIANT_ESP32S2) -#ifdef USE_ESP_IDF #include -#endif namespace esphome { namespace esp32_dac { @@ -29,12 +27,10 @@ class ESP32DAC : public output::FloatOutput, public Component { void write_state(float state) override; InternalGPIOPin *pin_; -#ifdef USE_ESP_IDF dac_oneshot_handle_t dac_handle_; -#endif }; } // namespace esp32_dac } // namespace esphome -#endif +#endif // USE_ESP32_VARIANT_ESP32 || USE_ESP32_VARIANT_ESP32S2 diff --git a/esphome/components/esp32_hosted/__init__.py b/esphome/components/esp32_hosted/__init__.py index 330800df12..7e9f1b05b5 100644 --- a/esphome/components/esp32_hosted/__init__.py +++ b/esphome/components/esp32_hosted/__init__.py @@ -1,4 +1,5 @@ import os +from pathlib import Path from esphome import pins from esphome.components import esp32 @@ -91,11 +92,16 @@ async def to_code(config): framework_ver: cv.Version = CORE.data[KEY_CORE][KEY_FRAMEWORK_VERSION] os.environ["ESP_IDF_VERSION"] = f"{framework_ver.major}.{framework_ver.minor}" - esp32.add_idf_component(name="espressif/esp_wifi_remote", ref="0.10.2") - esp32.add_idf_component(name="espressif/eppp_link", ref="0.2.0") - esp32.add_idf_component(name="espressif/esp_hosted", ref="2.0.11") + if framework_ver >= cv.Version(5, 5, 0): + esp32.add_idf_component(name="espressif/esp_wifi_remote", ref="1.1.5") + esp32.add_idf_component(name="espressif/eppp_link", ref="1.1.3") + esp32.add_idf_component(name="espressif/esp_hosted", ref="2.5.11") + else: + esp32.add_idf_component(name="espressif/esp_wifi_remote", ref="0.13.0") + esp32.add_idf_component(name="espressif/eppp_link", ref="0.2.0") + esp32.add_idf_component(name="espressif/esp_hosted", ref="2.0.11") esp32.add_extra_script( "post", "esp32_hosted.py", - os.path.join(os.path.dirname(__file__), "esp32_hosted.py.script"), + Path(__file__).parent / "esp32_hosted.py.script", ) diff --git a/esphome/components/esp32_improv/esp32_improv_component.cpp b/esphome/components/esp32_improv/esp32_improv_component.cpp index d41094fda1..d83caf931b 100644 --- a/esphome/components/esp32_improv/esp32_improv_component.cpp +++ b/esphome/components/esp32_improv/esp32_improv_component.cpp @@ -15,6 +15,15 @@ using namespace bytebuffer; static const char *const TAG = "esp32_improv.component"; static const char *const ESPHOME_MY_LINK = "https://my.home-assistant.io/redirect/config_flow_start?domain=esphome"; +static constexpr uint16_t STOP_ADVERTISING_DELAY = + 10000; // Delay (ms) before stopping service to allow BLE clients to read the final state +static constexpr uint16_t NAME_ADVERTISING_INTERVAL = 60000; // Advertise name every 60 seconds +static constexpr uint16_t NAME_ADVERTISING_DURATION = 1000; // Advertise name for 1 second + +// Improv service data constants +static constexpr uint8_t IMPROV_SERVICE_DATA_SIZE = 8; +static constexpr uint8_t IMPROV_PROTOCOL_ID_1 = 0x77; // 'P' << 1 | 'R' >> 7 +static constexpr uint8_t IMPROV_PROTOCOL_ID_2 = 0x46; // 'I' << 1 | 'M' >> 7 ESP32ImprovComponent::ESP32ImprovComponent() { global_improv_component = this; } @@ -29,8 +38,10 @@ void ESP32ImprovComponent::setup() { }); } #endif - global_ble_server->on(BLEServerEvt::EmptyEvt::ON_DISCONNECT, - [this](uint16_t conn_id) { this->set_error_(improv::ERROR_NONE); }); + global_ble_server->on_disconnect([this](uint16_t conn_id) { this->set_error_(improv::ERROR_NONE); }); + + // Start with loop disabled - will be enabled by start() when needed + this->disable_loop(); } void ESP32ImprovComponent::setup_characteristics() { @@ -45,12 +56,11 @@ void ESP32ImprovComponent::setup_characteristics() { this->error_->add_descriptor(error_descriptor); this->rpc_ = this->service_->create_characteristic(improv::RPC_COMMAND_UUID, BLECharacteristic::PROPERTY_WRITE); - this->rpc_->EventEmitter, uint16_t>::on( - BLECharacteristicEvt::VectorEvt::ON_WRITE, [this](const std::vector &data, uint16_t id) { - if (!data.empty()) { - this->incoming_data_.insert(this->incoming_data_.end(), data.begin(), data.end()); - } - }); + this->rpc_->on_write([this](std::span data, uint16_t id) { + if (!data.empty()) { + this->incoming_data_.insert(this->incoming_data_.end(), data.begin(), data.end()); + } + }); BLEDescriptor *rpc_descriptor = new BLE2902(); this->rpc_->add_descriptor(rpc_descriptor); @@ -94,6 +104,11 @@ void ESP32ImprovComponent::loop() { this->process_incoming_data_(); uint32_t now = App.get_loop_component_start_time(); + // Check if we need to update advertising type + if (this->state_ != improv::STATE_STOPPED && this->state_ != improv::STATE_PROVISIONED) { + this->update_advertising_type_(); + } + switch (this->state_) { case improv::STATE_STOPPED: this->set_status_indicator_state_(false); @@ -102,9 +117,15 @@ void ESP32ImprovComponent::loop() { if (this->service_->is_created()) { this->service_->start(); } else if (this->service_->is_running()) { + // Start by advertising the device name first BEFORE setting any state + ESP_LOGV(TAG, "Starting with device name advertising"); + this->advertising_device_name_ = true; + this->last_name_adv_time_ = App.get_loop_component_start_time(); + esp32_ble::global_ble->advertising_set_service_data_and_name(std::span{}, true); esp32_ble::global_ble->advertising_start(); - this->set_state_(improv::STATE_AWAITING_AUTHORIZATION); + // Set initial state based on whether we have an authorizer + this->set_state_(this->get_initial_state_(), false); this->set_error_(improv::ERROR_NONE); ESP_LOGD(TAG, "Service started!"); } @@ -115,54 +136,33 @@ void ESP32ImprovComponent::loop() { if (this->authorizer_ == nullptr || (this->authorized_start_ != 0 && ((now - this->authorized_start_) < this->authorized_duration_))) { this->set_state_(improv::STATE_AUTHORIZED); - } else -#else - { this->set_state_(improv::STATE_AUTHORIZED); } -#endif - { + } else { if (!this->check_identify_()) this->set_status_indicator_state_(true); } +#else + this->set_state_(improv::STATE_AUTHORIZED); +#endif + this->check_wifi_connection_(); break; } case improv::STATE_AUTHORIZED: { #ifdef USE_BINARY_SENSOR - if (this->authorizer_ != nullptr) { - if (now - this->authorized_start_ > this->authorized_duration_) { - ESP_LOGD(TAG, "Authorization timeout"); - this->set_state_(improv::STATE_AWAITING_AUTHORIZATION); - return; - } + if (this->authorizer_ != nullptr && now - this->authorized_start_ > this->authorized_duration_) { + ESP_LOGD(TAG, "Authorization timeout"); + this->set_state_(improv::STATE_AWAITING_AUTHORIZATION); + return; } #endif if (!this->check_identify_()) { this->set_status_indicator_state_((now % 1000) < 500); } + this->check_wifi_connection_(); break; } case improv::STATE_PROVISIONING: { this->set_status_indicator_state_((now % 200) < 100); - if (wifi::global_wifi_component->is_connected()) { - wifi::global_wifi_component->save_wifi_sta(this->connecting_sta_.get_ssid(), - this->connecting_sta_.get_password()); - this->connecting_sta_ = {}; - this->cancel_timeout("wifi-connect-timeout"); - this->set_state_(improv::STATE_PROVISIONED); - - std::vector urls = {ESPHOME_MY_LINK}; -#ifdef USE_WEBSERVER - for (auto &ip : wifi::global_wifi_component->wifi_sta_ip_addresses()) { - if (ip.is_ip4()) { - std::string webserver_url = "http://" + ip.str() + ":" + to_string(USE_WEBSERVER_PORT); - urls.push_back(webserver_url); - break; - } - } -#endif - std::vector data = improv::build_rpc_response(improv::WIFI_SETTINGS, urls); - this->send_response_(data); - this->stop(); - } + this->check_wifi_connection_(); break; } case improv::STATE_PROVISIONED: { @@ -190,6 +190,25 @@ void ESP32ImprovComponent::set_status_indicator_state_(bool state) { #endif } +#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_DEBUG +const char *ESP32ImprovComponent::state_to_string_(improv::State state) { + switch (state) { + case improv::STATE_STOPPED: + return "STOPPED"; + case improv::STATE_AWAITING_AUTHORIZATION: + return "AWAITING_AUTHORIZATION"; + case improv::STATE_AUTHORIZED: + return "AUTHORIZED"; + case improv::STATE_PROVISIONING: + return "PROVISIONING"; + case improv::STATE_PROVISIONED: + return "PROVISIONED"; + default: + return "UNKNOWN"; + } +} +#endif + bool ESP32ImprovComponent::check_identify_() { uint32_t now = millis(); @@ -202,32 +221,34 @@ bool ESP32ImprovComponent::check_identify_() { return identify; } -void ESP32ImprovComponent::set_state_(improv::State state) { - ESP_LOGV(TAG, "Setting state: %d", state); +void ESP32ImprovComponent::set_state_(improv::State state, bool update_advertising) { + // Skip if state hasn't changed + if (this->state_ == state) { + return; + } + +#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_DEBUG + ESP_LOGD(TAG, "State transition: %s (0x%02X) -> %s (0x%02X)", this->state_to_string_(this->state_), this->state_, + this->state_to_string_(state), state); +#endif this->state_ = state; - if (this->status_->get_value().empty() || this->status_->get_value()[0] != state) { + if (this->status_ != nullptr && (this->status_->get_value().empty() || this->status_->get_value()[0] != state)) { this->status_->set_value(ByteBuffer::wrap(static_cast(state))); if (state != improv::STATE_STOPPED) this->status_->notify(); } - std::vector service_data(8, 0); - service_data[0] = 0x77; // PR - service_data[1] = 0x46; // IM - service_data[2] = static_cast(state); - - uint8_t capabilities = 0x00; -#ifdef USE_OUTPUT - if (this->status_indicator_ != nullptr) - capabilities |= improv::CAPABILITY_IDENTIFY; -#endif - - service_data[3] = capabilities; - service_data[4] = 0x00; // Reserved - service_data[5] = 0x00; // Reserved - service_data[6] = 0x00; // Reserved - service_data[7] = 0x00; // Reserved - - esp32_ble::global_ble->advertising_set_service_data(service_data); + // Only advertise valid Improv states (0x01-0x04). + // STATE_STOPPED (0x00) is internal only and not part of the Improv spec. + // Advertising 0x00 causes undefined behavior in some clients and makes them + // repeatedly connect trying to determine the actual state. + if (state != improv::STATE_STOPPED && update_advertising) { + // State change always overrides name advertising and resets the timer + this->advertising_device_name_ = false; + // Reset the timer so we wait another 60 seconds before advertising name + this->last_name_adv_time_ = App.get_loop_component_start_time(); + // Advertise the new state via service data + this->advertise_service_data_(); + } #ifdef USE_ESP32_IMPROV_STATE_CALLBACK this->state_callback_.call(this->state_, this->error_state_); #endif @@ -237,7 +258,12 @@ void ESP32ImprovComponent::set_error_(improv::Error error) { if (error != improv::ERROR_NONE) { ESP_LOGE(TAG, "Error: %d", error); } - if (this->error_->get_value().empty() || this->error_->get_value()[0] != error) { + // The error_ characteristic is initialized in setup_characteristics() which is called + // from the loop, while the BLE disconnect callback is registered in setup(). + // error_ can be nullptr if: + // 1. A client connects/disconnects before setup_characteristics() is called + // 2. The device is already provisioned so the service never starts (should_start_ is false) + if (this->error_ != nullptr && (this->error_->get_value().empty() || this->error_->get_value()[0] != error)) { this->error_->set_value(ByteBuffer::wrap(static_cast(error))); if (this->state_ != improv::STATE_STOPPED) this->error_->notify(); @@ -261,7 +287,10 @@ void ESP32ImprovComponent::start() { void ESP32ImprovComponent::stop() { this->should_start_ = false; - this->set_timeout("end-service", 1000, [this] { + // Wait before stopping the service to ensure all BLE clients see the state change. + // This prevents clients from repeatedly reconnecting and wasting resources by allowing + // them to observe that the device is provisioned before the service disappears. + this->set_timeout("end-service", STOP_ADVERTISING_DELAY, [this] { if (this->state_ == improv::STATE_STOPPED || this->service_ == nullptr) return; this->service_->stop(); @@ -345,6 +374,90 @@ void ESP32ImprovComponent::on_wifi_connect_timeout_() { wifi::global_wifi_component->clear_sta(); } +void ESP32ImprovComponent::check_wifi_connection_() { + if (!wifi::global_wifi_component->is_connected()) { + return; + } + + if (this->state_ == improv::STATE_PROVISIONING) { + wifi::global_wifi_component->save_wifi_sta(this->connecting_sta_.get_ssid(), this->connecting_sta_.get_password()); + this->connecting_sta_ = {}; + this->cancel_timeout("wifi-connect-timeout"); + + std::vector urls = {ESPHOME_MY_LINK}; +#ifdef USE_WEBSERVER + for (auto &ip : wifi::global_wifi_component->wifi_sta_ip_addresses()) { + if (ip.is_ip4()) { + std::string webserver_url = "http://" + ip.str() + ":" + to_string(USE_WEBSERVER_PORT); + urls.push_back(webserver_url); + break; + } + } +#endif + std::vector data = improv::build_rpc_response(improv::WIFI_SETTINGS, urls); + this->send_response_(data); + } else if (this->is_active() && this->state_ != improv::STATE_PROVISIONED) { + ESP_LOGD(TAG, "WiFi provisioned externally"); + } + + this->set_state_(improv::STATE_PROVISIONED); + this->stop(); +} + +void ESP32ImprovComponent::advertise_service_data_() { + uint8_t service_data[IMPROV_SERVICE_DATA_SIZE] = {}; + service_data[0] = IMPROV_PROTOCOL_ID_1; // PR + service_data[1] = IMPROV_PROTOCOL_ID_2; // IM + service_data[2] = static_cast(this->state_); + + uint8_t capabilities = 0x00; +#ifdef USE_OUTPUT + if (this->status_indicator_ != nullptr) + capabilities |= improv::CAPABILITY_IDENTIFY; +#endif + + service_data[3] = capabilities; + // service_data[4-7] are already 0 (Reserved) + + // Atomically set service data and disable name in advertising + esp32_ble::global_ble->advertising_set_service_data_and_name(std::span(service_data), false); +} + +void ESP32ImprovComponent::update_advertising_type_() { + uint32_t now = App.get_loop_component_start_time(); + + // If we're advertising the device name and it's been more than NAME_ADVERTISING_DURATION, switch back to service data + if (this->advertising_device_name_) { + if (now - this->last_name_adv_time_ >= NAME_ADVERTISING_DURATION) { + ESP_LOGV(TAG, "Switching back to service data advertising"); + this->advertising_device_name_ = false; + // Restore service data advertising + this->advertise_service_data_(); + } + return; + } + + // Check if it's time to advertise the device name (every NAME_ADVERTISING_INTERVAL) + if (now - this->last_name_adv_time_ >= NAME_ADVERTISING_INTERVAL) { + ESP_LOGV(TAG, "Switching to device name advertising"); + this->advertising_device_name_ = true; + this->last_name_adv_time_ = now; + + // Atomically clear service data and enable name in advertising data + esp32_ble::global_ble->advertising_set_service_data_and_name(std::span{}, true); + } +} + +improv::State ESP32ImprovComponent::get_initial_state_() const { +#ifdef USE_BINARY_SENSOR + // If we have an authorizer, start in awaiting authorization state + return this->authorizer_ == nullptr ? improv::STATE_AUTHORIZED : improv::STATE_AWAITING_AUTHORIZATION; +#else + // No binary_sensor support = no authorizer possible, start as authorized + return improv::STATE_AUTHORIZED; +#endif +} + ESP32ImprovComponent *global_improv_component = nullptr; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) } // namespace esp32_improv diff --git a/esphome/components/esp32_improv/esp32_improv_component.h b/esphome/components/esp32_improv/esp32_improv_component.h index 87cec23876..6782430ffe 100644 --- a/esphome/components/esp32_improv/esp32_improv_component.h +++ b/esphome/components/esp32_improv/esp32_improv_component.h @@ -79,12 +79,12 @@ class ESP32ImprovComponent : public Component { std::vector incoming_data_; wifi::WiFiAP connecting_sta_; - BLEService *service_ = nullptr; - BLECharacteristic *status_; - BLECharacteristic *error_; - BLECharacteristic *rpc_; - BLECharacteristic *rpc_response_; - BLECharacteristic *capabilities_; + BLEService *service_{nullptr}; + BLECharacteristic *status_{nullptr}; + BLECharacteristic *error_{nullptr}; + BLECharacteristic *rpc_{nullptr}; + BLECharacteristic *rpc_response_{nullptr}; + BLECharacteristic *capabilities_{nullptr}; #ifdef USE_BINARY_SENSOR binary_sensor::BinarySensor *authorizer_{nullptr}; @@ -100,14 +100,23 @@ class ESP32ImprovComponent : public Component { #endif bool status_indicator_state_{false}; + uint32_t last_name_adv_time_{0}; + bool advertising_device_name_{false}; void set_status_indicator_state_(bool state); + void update_advertising_type_(); - void set_state_(improv::State state); + void set_state_(improv::State state, bool update_advertising = true); void set_error_(improv::Error error); + improv::State get_initial_state_() const; void send_response_(std::vector &response); void process_incoming_data_(); void on_wifi_connect_timeout_(); + void check_wifi_connection_(); bool check_identify_(); + void advertise_service_data_(); +#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_DEBUG + const char *state_to_string_(improv::State state); +#endif }; // NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables) diff --git a/esphome/components/esp32_rmt_led_strip/led_strip.cpp b/esphome/components/esp32_rmt_led_strip/led_strip.cpp index 389c32882b..2c7963b366 100644 --- a/esphome/components/esp32_rmt_led_strip/led_strip.cpp +++ b/esphome/components/esp32_rmt_led_strip/led_strip.cpp @@ -35,16 +35,18 @@ static size_t IRAM_ATTR HOT encoder_callback(const void *data, size_t size, size if (symbols_free < RMT_SYMBOLS_PER_BYTE) { return 0; } - for (int32_t i = 0; i < RMT_SYMBOLS_PER_BYTE; i++) { + for (size_t i = 0; i < RMT_SYMBOLS_PER_BYTE; i++) { if (bytes[index] & (1 << (7 - i))) { symbols[i] = params->bit1; } else { symbols[i] = params->bit0; } } +#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 5, 1) if ((index + 1) >= size && params->reset.duration0 == 0 && params->reset.duration1 == 0) { *done = true; } +#endif return RMT_SYMBOLS_PER_BYTE; } @@ -59,8 +61,6 @@ static size_t IRAM_ATTR HOT encoder_callback(const void *data, size_t size, size #endif void ESP32RMTLEDStripLightOutput::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - size_t buffer_size = this->get_buffer_size_(); RAMAllocator allocator(this->use_psram_ ? 0 : RAMAllocator::ALLOC_INTERNAL); @@ -112,7 +112,7 @@ void ESP32RMTLEDStripLightOutput::setup() { memset(&encoder, 0, sizeof(encoder)); encoder.callback = encoder_callback; encoder.arg = &this->params_; - encoder.min_chunk_size = 8; + encoder.min_chunk_size = RMT_SYMBOLS_PER_BYTE; if (rmt_new_simple_encoder(&encoder, &this->encoder_) != ESP_OK) { ESP_LOGE(TAG, "Encoder creation failed"); this->mark_failed(); diff --git a/esphome/components/esp32_rmt_led_strip/light.py b/esphome/components/esp32_rmt_led_strip/light.py index 33ae44e435..ac4f0b2e92 100644 --- a/esphome/components/esp32_rmt_led_strip/light.py +++ b/esphome/components/esp32_rmt_led_strip/light.py @@ -4,6 +4,7 @@ import logging from esphome import pins import esphome.codegen as cg from esphome.components import esp32, light +from esphome.components.const import CONF_USE_PSRAM import esphome.config_validation as cv from esphome.const import ( CONF_CHIPSET, @@ -57,7 +58,6 @@ CHIPSETS = { "SM16703": LEDStripTimings(300, 900, 900, 300, 0, 0), } -CONF_USE_PSRAM = "use_psram" CONF_IS_WRGB = "is_wrgb" CONF_BIT0_HIGH = "bit0_high" CONF_BIT0_LOW = "bit0_low" diff --git a/esphome/components/esp32_touch/__init__.py b/esphome/components/esp32_touch/__init__.py index 224e301683..b6cb19ebb1 100644 --- a/esphome/components/esp32_touch/__init__.py +++ b/esphome/components/esp32_touch/__init__.py @@ -294,9 +294,8 @@ async def to_code(config): ) ) - if get_esp32_variant() == VARIANT_ESP32: - if CONF_IIR_FILTER in config: - cg.add(touch.set_iir_filter(config[CONF_IIR_FILTER])) + if get_esp32_variant() == VARIANT_ESP32 and CONF_IIR_FILTER in config: + cg.add(touch.set_iir_filter(config[CONF_IIR_FILTER])) if get_esp32_variant() == VARIANT_ESP32S2 or get_esp32_variant() == VARIANT_ESP32S3: if CONF_FILTER_MODE in config: diff --git a/esphome/components/esp32_touch/esp32_touch.h b/esphome/components/esp32_touch/esp32_touch.h index 5a91b1c750..fb1973e26f 100644 --- a/esphome/components/esp32_touch/esp32_touch.h +++ b/esphome/components/esp32_touch/esp32_touch.h @@ -171,8 +171,8 @@ class ESP32TouchComponent : public Component { // based on the filter configuration uint32_t read_touch_value(touch_pad_t pad) const; - // Helper to update touch state with a known state - void update_touch_state_(ESP32TouchBinarySensor *child, bool is_touched); + // Helper to update touch state with a known state and value + void update_touch_state_(ESP32TouchBinarySensor *child, bool is_touched, uint32_t value); // Helper to read touch value and update state for a given child bool check_and_update_touch_state_(ESP32TouchBinarySensor *child); @@ -234,9 +234,13 @@ class ESP32TouchBinarySensor : public binary_sensor::BinarySensor { touch_pad_t get_touch_pad() const { return this->touch_pad_; } uint32_t get_threshold() const { return this->threshold_; } void set_threshold(uint32_t threshold) { this->threshold_ = threshold; } -#ifdef USE_ESP32_VARIANT_ESP32 + + /// Get the raw touch measurement value. + /// @note Although this method may appear unused within the component, it is a public API + /// used by lambdas in user configurations for custom touch value processing. + /// @return The current raw touch sensor reading uint32_t get_value() const { return this->value_; } -#endif + uint32_t get_wakeup_threshold() const { return this->wakeup_threshold_; } protected: @@ -245,9 +249,8 @@ class ESP32TouchBinarySensor : public binary_sensor::BinarySensor { touch_pad_t touch_pad_{TOUCH_PAD_MAX}; uint32_t threshold_{0}; uint32_t benchmark_{}; -#ifdef USE_ESP32_VARIANT_ESP32 + /// Stores the last raw touch measurement value. uint32_t value_{0}; -#endif bool last_state_{false}; const uint32_t wakeup_threshold_{0}; diff --git a/esphome/components/esp32_touch/esp32_touch_common.cpp b/esphome/components/esp32_touch/esp32_touch_common.cpp index 2d93de077e..a0b1df38c1 100644 --- a/esphome/components/esp32_touch/esp32_touch_common.cpp +++ b/esphome/components/esp32_touch/esp32_touch_common.cpp @@ -100,6 +100,8 @@ void ESP32TouchComponent::process_setup_mode_logging_(uint32_t now) { #else // Read the value being used for touch detection uint32_t value = this->read_touch_value(child->get_touch_pad()); + // Store the value for get_value() access in lambdas + child->value_ = value; ESP_LOGD(TAG, "Touch Pad '%s' (T%d): %d", child->get_name().c_str(), child->get_touch_pad(), value); #endif } diff --git a/esphome/components/esp32_touch/esp32_touch_v1.cpp b/esphome/components/esp32_touch/esp32_touch_v1.cpp index c3d43c6bbf..ffb805e008 100644 --- a/esphome/components/esp32_touch/esp32_touch_v1.cpp +++ b/esphome/components/esp32_touch/esp32_touch_v1.cpp @@ -16,6 +16,8 @@ namespace esp32_touch { static const char *const TAG = "esp32_touch"; +static const uint32_t SETUP_MODE_THRESHOLD = 0xFFFF; + void ESP32TouchComponent::setup() { // Create queue for touch events // Queue size calculation: children * 4 allows for burst scenarios where ISR @@ -44,7 +46,11 @@ void ESP32TouchComponent::setup() { // Configure each touch pad for (auto *child : this->children_) { - touch_pad_config(child->get_touch_pad(), child->get_threshold()); + if (this->setup_mode_) { + touch_pad_config(child->get_touch_pad(), SETUP_MODE_THRESHOLD); + } else { + touch_pad_config(child->get_touch_pad(), child->get_threshold()); + } } // Register ISR handler @@ -114,8 +120,8 @@ void ESP32TouchComponent::loop() { child->publish_state(new_state); // Original ESP32: ISR only fires when touched, release is detected by timeout // Note: ESP32 v1 uses inverted logic - touched when value < threshold - ESP_LOGV(TAG, "Touch Pad '%s' state: ON (value: %" PRIu32 " < threshold: %" PRIu32 ")", - child->get_name().c_str(), event.value, child->get_threshold()); + ESP_LOGV(TAG, "Touch Pad '%s' state: %s (value: %" PRIu32 " < threshold: %" PRIu32 ")", + child->get_name().c_str(), ONOFF(new_state), event.value, child->get_threshold()); } break; // Exit inner loop after processing matching pad } @@ -188,11 +194,6 @@ void IRAM_ATTR ESP32TouchComponent::touch_isr_handler(void *arg) { // as any pad remains touched. This allows us to detect both new touches and // continued touches, but releases must be detected by timeout in the main loop. - // IMPORTANT: ESP32 v1 touch detection logic - INVERTED compared to v2! - // ESP32 v1: Touch is detected when capacitance INCREASES, causing the measured value to DECREASE - // Therefore: touched = (value < threshold) - // This is opposite to ESP32-S2/S3 v2 where touched = (value > threshold) - // Process all configured pads to check their current state // Note: ESP32 v1 doesn't tell us which specific pad triggered the interrupt, // so we must scan all configured pads to find which ones were touched @@ -200,22 +201,25 @@ void IRAM_ATTR ESP32TouchComponent::touch_isr_handler(void *arg) { touch_pad_t pad = child->get_touch_pad(); // Read current value using ISR-safe API - uint32_t value; - if (component->iir_filter_enabled_()) { - uint16_t temp_value = 0; - touch_pad_read_filtered(pad, &temp_value); - value = temp_value; - } else { - // Use low-level HAL function when filter is not enabled - value = touch_ll_read_raw_data(pad); - } + // IMPORTANT: ESP-IDF v5.4 regression - touch_pad_read_filtered() is no longer ISR-safe + // In ESP-IDF v5.3 and earlier it was ISR-safe, but ESP-IDF v5.4 added mutex protection that causes: + // "assert failed: xQueueSemaphoreTake queue.c:1718" + // We must use raw values even when filter is enabled as a workaround. + // Users should adjust thresholds to compensate for the lack of IIR filtering. + // See: https://github.com/espressif/esp-idf/issues/17045 + uint32_t value = touch_ll_read_raw_data(pad); // Skip pads that aren’t in the trigger mask - bool is_touched = (mask >> pad) & 1; - if (!is_touched) { + if (((mask >> pad) & 1) == 0) { continue; } + // IMPORTANT: ESP32 v1 touch detection logic - INVERTED compared to v2! + // ESP32 v1: Touch is detected when capacitance INCREASES, causing the measured value to DECREASE + // Therefore: touched = (value < threshold) + // This is opposite to ESP32-S2/S3 v2 where touched = (value > threshold) + bool is_touched = value < child->get_threshold(); + // Always send the current state - the main loop will filter for changes // We send both touched and untouched states because the ISR doesn't // track previous state (to keep ISR fast and simple) diff --git a/esphome/components/esp32_touch/esp32_touch_v2.cpp b/esphome/components/esp32_touch/esp32_touch_v2.cpp index afd2655fd7..9662b009f6 100644 --- a/esphome/components/esp32_touch/esp32_touch_v2.cpp +++ b/esphome/components/esp32_touch/esp32_touch_v2.cpp @@ -10,8 +10,11 @@ namespace esp32_touch { static const char *const TAG = "esp32_touch"; -// Helper to update touch state with a known state -void ESP32TouchComponent::update_touch_state_(ESP32TouchBinarySensor *child, bool is_touched) { +// Helper to update touch state with a known state and value +void ESP32TouchComponent::update_touch_state_(ESP32TouchBinarySensor *child, bool is_touched, uint32_t value) { + // Store the value for get_value() access in lambdas + child->value_ = value; + // Always update timer when touched if (is_touched) { child->last_touch_time_ = App.get_loop_component_start_time(); @@ -21,9 +24,8 @@ void ESP32TouchComponent::update_touch_state_(ESP32TouchBinarySensor *child, boo child->last_state_ = is_touched; child->publish_state(is_touched); if (is_touched) { - // ESP32-S2/S3 v2: touched when value > threshold ESP_LOGV(TAG, "Touch Pad '%s' state: ON (value: %" PRIu32 " > threshold: %" PRIu32 ")", child->get_name().c_str(), - this->read_touch_value(child->touch_pad_), child->threshold_ + child->benchmark_); + value, child->threshold_ + child->benchmark_); } else { ESP_LOGV(TAG, "Touch Pad '%s' state: OFF", child->get_name().c_str()); } @@ -41,7 +43,7 @@ bool ESP32TouchComponent::check_and_update_touch_state_(ESP32TouchBinarySensor * child->get_name().c_str(), child->touch_pad_, value, child->threshold_, child->benchmark_); bool is_touched = value > child->benchmark_ + child->threshold_; - this->update_touch_state_(child, is_touched); + this->update_touch_state_(child, is_touched, value); return is_touched; } @@ -296,7 +298,9 @@ void ESP32TouchComponent::loop() { this->check_and_update_touch_state_(child); } else if (event.intr_mask & TOUCH_PAD_INTR_MASK_ACTIVE) { // We only get ACTIVE interrupts now, releases are detected by timeout - this->update_touch_state_(child, true); // Always touched for ACTIVE interrupts + // Read the current value + uint32_t value = this->read_touch_value(child->touch_pad_); + this->update_touch_state_(child, true, value); // Always touched for ACTIVE interrupts } break; } diff --git a/esphome/components/esp8266/__init__.py b/esphome/components/esp8266/__init__.py index 01b20bdcb1..9d8e6b7d1e 100644 --- a/esphome/components/esp8266/__init__.py +++ b/esphome/components/esp8266/__init__.py @@ -1,5 +1,5 @@ import logging -import os +from pathlib import Path import esphome.codegen as cg import esphome.config_validation as cv @@ -15,8 +15,9 @@ from esphome.const import ( KEY_TARGET_FRAMEWORK, KEY_TARGET_PLATFORM, PLATFORM_ESP8266, + ThreadModel, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.helpers import copy_file_if_changed from .boards import BOARDS, ESP8266_LD_SCRIPTS @@ -175,7 +176,7 @@ CONFIG_SCHEMA = cv.All( ) -@coroutine_with_priority(1000) +@coroutine_with_priority(CoroPriority.PLATFORM) async def to_code(config): cg.add(esp8266_ns.setup_preferences()) @@ -187,8 +188,9 @@ async def to_code(config): cg.set_cpp_standard("gnu++20") cg.add_define("ESPHOME_BOARD", config[CONF_BOARD]) cg.add_define("ESPHOME_VARIANT", "ESP8266") + cg.add_define(ThreadModel.SINGLE) - cg.add_platformio_option("extra_scripts", ["post:post_build.py"]) + cg.add_platformio_option("extra_scripts", ["pre:iram_fix.py", "post:post_build.py"]) conf = config[CONF_FRAMEWORK] cg.add_platformio_option("framework", "arduino") @@ -228,6 +230,12 @@ async def to_code(config): # For cases where nullptrs can be handled, use nothrow: `new (std::nothrow) T;` cg.add_build_flag("-DNEW_OOM_ABORT") + # In testing mode, fake a larger IRAM to allow linking grouped component tests + # Real ESP8266 hardware only has 32KB IRAM, but for CI testing we pretend it has 2MB + # This is done via a pre-build script that generates a custom linker script + if CORE.testing_mode: + cg.add_build_flag("-DESPHOME_TESTING_MODE") + cg.add_platformio_option("board_build.flash_mode", config[CONF_BOARD_FLASH_MODE]) ver: cv.Version = CORE.data[KEY_CORE][KEY_FRAMEWORK_VERSION] @@ -243,7 +251,7 @@ async def to_code(config): if ver <= cv.Version(2, 3, 0): # No ld script support ld_script = None - if ver <= cv.Version(2, 4, 2): + elif ver <= cv.Version(2, 4, 2): # Old ld script path ld_script = ld_scripts[0] else: @@ -257,9 +265,14 @@ async def to_code(config): # Called by writer.py def copy_files(): - dir = os.path.dirname(__file__) - post_build_file = os.path.join(dir, "post_build.py.script") + dir = Path(__file__).parent + post_build_file = dir / "post_build.py.script" copy_file_if_changed( post_build_file, CORE.relative_build_path("post_build.py"), ) + iram_fix_file = dir / "iram_fix.py.script" + copy_file_if_changed( + iram_fix_file, + CORE.relative_build_path("iram_fix.py"), + ) diff --git a/esphome/components/esp8266/core.cpp b/esphome/components/esp8266/core.cpp index 2d3959b031..200ca567c2 100644 --- a/esphome/components/esp8266/core.cpp +++ b/esphome/components/esp8266/core.cpp @@ -58,8 +58,8 @@ extern "C" void resetPins() { // NOLINT #ifdef USE_ESP8266_EARLY_PIN_INIT for (int i = 0; i < 16; i++) { - uint8_t mode = ESPHOME_ESP8266_GPIO_INITIAL_MODE[i]; - uint8_t level = ESPHOME_ESP8266_GPIO_INITIAL_LEVEL[i]; + uint8_t mode = progmem_read_byte(&ESPHOME_ESP8266_GPIO_INITIAL_MODE[i]); + uint8_t level = progmem_read_byte(&ESPHOME_ESP8266_GPIO_INITIAL_LEVEL[i]); if (mode != 255) pinMode(i, mode); // NOLINT if (level != 255) diff --git a/esphome/components/esp8266/gpio.py b/esphome/components/esp8266/gpio.py index 050efaacae..e7492fc505 100644 --- a/esphome/components/esp8266/gpio.py +++ b/esphome/components/esp8266/gpio.py @@ -17,7 +17,7 @@ from esphome.const import ( CONF_PULLUP, PLATFORM_ESP8266, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from . import boards from .const import KEY_BOARD, KEY_ESP8266, KEY_PIN_INITIAL_STATES, esp8266_ns @@ -188,7 +188,7 @@ async def esp8266_pin_to_code(config): return var -@coroutine_with_priority(-999.0) +@coroutine_with_priority(CoroPriority.WORKAROUNDS) async def add_pin_initial_states_array(): # Add includes at the very end, so that they override everything initial_states: list[PinInitialState] = CORE.data[KEY_ESP8266][ @@ -199,11 +199,11 @@ async def add_pin_initial_states_array(): cg.add_global( cg.RawExpression( - f"const uint8_t ESPHOME_ESP8266_GPIO_INITIAL_MODE[16] = {{{initial_modes_s}}}" + f"const uint8_t ESPHOME_ESP8266_GPIO_INITIAL_MODE[16] PROGMEM = {{{initial_modes_s}}}" ) ) cg.add_global( cg.RawExpression( - f"const uint8_t ESPHOME_ESP8266_GPIO_INITIAL_LEVEL[16] = {{{initial_levels_s}}}" + f"const uint8_t ESPHOME_ESP8266_GPIO_INITIAL_LEVEL[16] PROGMEM = {{{initial_levels_s}}}" ) ) diff --git a/esphome/components/esp8266/helpers.cpp b/esphome/components/esp8266/helpers.cpp index 993de710c6..036594fa17 100644 --- a/esphome/components/esp8266/helpers.cpp +++ b/esphome/components/esp8266/helpers.cpp @@ -22,6 +22,10 @@ void Mutex::unlock() {} IRAM_ATTR InterruptLock::InterruptLock() { state_ = xt_rsil(15); } IRAM_ATTR InterruptLock::~InterruptLock() { xt_wsr_ps(state_); } +// ESP8266 doesn't support lwIP core locking, so this is a no-op +LwIPLock::LwIPLock() {} +LwIPLock::~LwIPLock() {} + void get_mac_address_raw(uint8_t *mac) { // NOLINT(readability-non-const-parameter) wifi_get_macaddr(STATION_IF, mac); } diff --git a/esphome/components/esp8266/iram_fix.py.script b/esphome/components/esp8266/iram_fix.py.script new file mode 100644 index 0000000000..96bddc2ced --- /dev/null +++ b/esphome/components/esp8266/iram_fix.py.script @@ -0,0 +1,44 @@ +import os +import re + +# pylint: disable=E0602 +Import("env") # noqa + + +def patch_linker_script_after_preprocess(source, target, env): + """Patch the local linker script after PlatformIO preprocesses it.""" + # Check if we're in testing mode by looking for the define + build_flags = env.get("BUILD_FLAGS", []) + testing_mode = any("-DESPHOME_TESTING_MODE" in flag for flag in build_flags) + + if not testing_mode: + return + + # Get the local linker script path + build_dir = env.subst("$BUILD_DIR") + local_ld = os.path.join(build_dir, "ld", "local.eagle.app.v6.common.ld") + + if not os.path.exists(local_ld): + return + + # Read the linker script + with open(local_ld, "r") as f: + content = f.read() + + # Replace IRAM size from 0x8000 (32KB) to 0x200000 (2MB) + # The line looks like: iram1_0_seg : org = 0x40100000, len = 0x8000 + updated = re.sub( + r"(iram1_0_seg\s*:\s*org\s*=\s*0x40100000\s*,\s*len\s*=\s*)0x8000", + r"\g<1>0x200000", + content, + ) + + if updated != content: + with open(local_ld, "w") as f: + f.write(updated) + print("ESPHome: Patched IRAM size to 2MB for testing mode") + + +# Hook into the build process right before linking +# This runs after PlatformIO has already preprocessed the linker scripts +env.AddPreAction("$BUILD_DIR/${PROGNAME}.elf", patch_linker_script_after_preprocess) diff --git a/esphome/components/esp8266/preferences.cpp b/esphome/components/esp8266/preferences.cpp index efd226e8f8..a26e9cc498 100644 --- a/esphome/components/esp8266/preferences.cpp +++ b/esphome/components/esp8266/preferences.cpp @@ -1,6 +1,7 @@ #ifdef USE_ESP8266 #include +#include extern "C" { #include "spi_flash.h" } @@ -12,7 +13,7 @@ extern "C" { #include "preferences.h" #include -#include +#include namespace esphome { namespace esp8266 { @@ -67,6 +68,8 @@ static uint32_t get_esp8266_flash_sector() { } static uint32_t get_esp8266_flash_address() { return get_esp8266_flash_sector() * SPI_FLASH_SEC_SIZE; } +static inline size_t bytes_to_words(size_t bytes) { return (bytes + 3) / 4; } + template uint32_t calculate_crc(It first, It last, uint32_t type) { uint32_t crc = type; while (first != last) { @@ -117,47 +120,42 @@ static bool load_from_rtc(size_t offset, uint32_t *data, size_t len) { class ESP8266PreferenceBackend : public ESPPreferenceBackend { public: - size_t offset = 0; uint32_t type = 0; + uint16_t offset = 0; + uint8_t length_words = 0; // Max 255 words (1020 bytes of data) bool in_flash = false; - size_t length_words = 0; bool save(const uint8_t *data, size_t len) override { - if ((len + 3) / 4 != length_words) { + if (bytes_to_words(len) != length_words) { return false; } - std::vector buffer; - buffer.resize(length_words + 1); - memcpy(buffer.data(), data, len); - buffer[buffer.size() - 1] = calculate_crc(buffer.begin(), buffer.end() - 1, type); + size_t buffer_size = static_cast(length_words) + 1; + std::unique_ptr buffer(new uint32_t[buffer_size]()); // Note the () for zero-initialization + memcpy(buffer.get(), data, len); + buffer[length_words] = calculate_crc(buffer.get(), buffer.get() + length_words, type); if (in_flash) { - return save_to_flash(offset, buffer.data(), buffer.size()); - } else { - return save_to_rtc(offset, buffer.data(), buffer.size()); + return save_to_flash(offset, buffer.get(), buffer_size); } + return save_to_rtc(offset, buffer.get(), buffer_size); } bool load(uint8_t *data, size_t len) override { - if ((len + 3) / 4 != length_words) { + if (bytes_to_words(len) != length_words) { return false; } - std::vector buffer; - buffer.resize(length_words + 1); - bool ret; - if (in_flash) { - ret = load_from_flash(offset, buffer.data(), buffer.size()); - } else { - ret = load_from_rtc(offset, buffer.data(), buffer.size()); - } + size_t buffer_size = static_cast(length_words) + 1; + std::unique_ptr buffer(new uint32_t[buffer_size]()); + bool ret = in_flash ? load_from_flash(offset, buffer.get(), buffer_size) + : load_from_rtc(offset, buffer.get(), buffer_size); if (!ret) return false; - uint32_t crc = calculate_crc(buffer.begin(), buffer.end() - 1, type); - if (buffer[buffer.size() - 1] != crc) { + uint32_t crc = calculate_crc(buffer.get(), buffer.get() + length_words, type); + if (buffer[length_words] != crc) { return false; } - memcpy(data, buffer.data(), len); + memcpy(data, buffer.get(), len); return true; } }; @@ -178,16 +176,20 @@ class ESP8266Preferences : public ESPPreferences { } ESPPreferenceObject make_preference(size_t length, uint32_t type, bool in_flash) override { - uint32_t length_words = (length + 3) / 4; + uint32_t length_words = bytes_to_words(length); + if (length_words > 255) { + ESP_LOGE(TAG, "Preference too large: %" PRIu32 " words > 255", length_words); + return {}; + } if (in_flash) { uint32_t start = current_flash_offset; uint32_t end = start + length_words + 1; if (end > ESP8266_FLASH_STORAGE_SIZE) return {}; auto *pref = new ESP8266PreferenceBackend(); // NOLINT(cppcoreguidelines-owning-memory) - pref->offset = start; + pref->offset = static_cast(start); pref->type = type; - pref->length_words = length_words; + pref->length_words = static_cast(length_words); pref->in_flash = true; current_flash_offset = end; return {pref}; @@ -213,9 +215,9 @@ class ESP8266Preferences : public ESPPreferences { uint32_t rtc_offset = in_normal ? start + 32 : start - 96; auto *pref = new ESP8266PreferenceBackend(); // NOLINT(cppcoreguidelines-owning-memory) - pref->offset = rtc_offset; + pref->offset = static_cast(rtc_offset); pref->type = type; - pref->length_words = length_words; + pref->length_words = static_cast(length_words); pref->in_flash = false; current_offset += length_words + 1; return pref; diff --git a/esphome/components/esp8266_pwm/esp8266_pwm.cpp b/esphome/components/esp8266_pwm/esp8266_pwm.cpp index 03fa3c683e..0aaef597d3 100644 --- a/esphome/components/esp8266_pwm/esp8266_pwm.cpp +++ b/esphome/components/esp8266_pwm/esp8266_pwm.cpp @@ -14,7 +14,6 @@ namespace esp8266_pwm { static const char *const TAG = "esp8266_pwm"; void ESP8266PWM::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->pin_->setup(); this->turn_off(); } diff --git a/esphome/components/esphome/ota/__init__.py b/esphome/components/esphome/ota/__init__.py index 901657ec82..69a50a2de9 100644 --- a/esphome/components/esphome/ota/__init__.py +++ b/esphome/components/esphome/ota/__init__.py @@ -16,16 +16,31 @@ from esphome.const import ( CONF_SAFE_MODE, CONF_VERSION, ) -from esphome.core import coroutine_with_priority +from esphome.core import CORE, coroutine_with_priority +from esphome.coroutine import CoroPriority import esphome.final_validate as fv +from esphome.types import ConfigType _LOGGER = logging.getLogger(__name__) CODEOWNERS = ["@esphome/core"] -AUTO_LOAD = ["md5", "socket"] DEPENDENCIES = ["network"] + +def supports_sha256() -> bool: + """Check if the current platform supports SHA256 for OTA authentication.""" + return bool(CORE.is_esp32 or CORE.is_esp8266 or CORE.is_rp2040 or CORE.is_libretiny) + + +def AUTO_LOAD() -> list[str]: + """Conditionally auto-load sha256 only on platforms that support it.""" + base_components = ["md5", "socket"] + if supports_sha256(): + return base_components + ["sha256"] + return base_components + + esphome = cg.esphome_ns.namespace("esphome") ESPHomeOTAComponent = esphome.class_("ESPHomeOTAComponent", OTAComponent) @@ -73,8 +88,7 @@ def ota_esphome_final_validate(config): else: new_ota_conf.append(ota_conf) - for port_conf in merged_ota_esphome_configs_by_port.values(): - new_ota_conf.append(port_conf) + new_ota_conf.extend(merged_ota_esphome_configs_by_port.values()) full_conf[CONF_OTA] = new_ota_conf fv.full_config.set(full_conf) @@ -122,13 +136,20 @@ CONFIG_SCHEMA = ( FINAL_VALIDATE_SCHEMA = ota_esphome_final_validate -@coroutine_with_priority(52.0) -async def to_code(config): +@coroutine_with_priority(CoroPriority.OTA_UPDATES) +async def to_code(config: ConfigType) -> None: var = cg.new_Pvariable(config[CONF_ID]) cg.add(var.set_port(config[CONF_PORT])) - if CONF_PASSWORD in config: + + # Password could be set to an empty string and we can assume that means no password + if config.get(CONF_PASSWORD): cg.add(var.set_auth_password(config[CONF_PASSWORD])) cg.add_define("USE_OTA_PASSWORD") + # Only include hash algorithms when password is configured + cg.add_define("USE_OTA_MD5") + # Only include SHA256 support on platforms that have it + if supports_sha256(): + cg.add_define("USE_OTA_SHA256") cg.add_define("USE_OTA_VERSION", config[CONF_VERSION]) await cg.register_component(var, config) diff --git a/esphome/components/esphome/ota/ota_esphome.cpp b/esphome/components/esphome/ota/ota_esphome.cpp index 4cc82b9094..569268ea15 100644 --- a/esphome/components/esphome/ota/ota_esphome.cpp +++ b/esphome/components/esphome/ota/ota_esphome.cpp @@ -1,6 +1,13 @@ #include "ota_esphome.h" #ifdef USE_OTA +#ifdef USE_OTA_PASSWORD +#ifdef USE_OTA_MD5 #include "esphome/components/md5/md5.h" +#endif +#ifdef USE_OTA_SHA256 +#include "esphome/components/sha256/sha256.h" +#endif +#endif #include "esphome/components/network/util.h" #include "esphome/components/ota/ota_backend.h" #include "esphome/components/ota/ota_backend_arduino_esp32.h" @@ -10,6 +17,7 @@ #include "esphome/components/ota/ota_backend_esp_idf.h" #include "esphome/core/application.h" #include "esphome/core/hal.h" +#include "esphome/core/helpers.h" #include "esphome/core/log.h" #include "esphome/core/util.h" @@ -19,7 +27,19 @@ namespace esphome { static const char *const TAG = "esphome.ota"; -static constexpr u_int16_t OTA_BLOCK_SIZE = 8192; +static constexpr uint16_t OTA_BLOCK_SIZE = 8192; +static constexpr size_t OTA_BUFFER_SIZE = 1024; // buffer size for OTA data transfer +static constexpr uint32_t OTA_SOCKET_TIMEOUT_HANDSHAKE = 20000; // milliseconds for initial handshake +static constexpr uint32_t OTA_SOCKET_TIMEOUT_DATA = 90000; // milliseconds for data transfer + +#ifdef USE_OTA_PASSWORD +#ifdef USE_OTA_MD5 +static constexpr size_t MD5_HEX_SIZE = 32; // MD5 hash as hex string (16 bytes * 2) +#endif +#ifdef USE_OTA_SHA256 +static constexpr size_t SHA256_HEX_SIZE = 64; // SHA256 hash as hex string (32 bytes * 2) +#endif +#endif // USE_OTA_PASSWORD void ESPHomeOTAComponent::setup() { #ifdef USE_OTA_STATE_CALLBACK @@ -28,19 +48,19 @@ void ESPHomeOTAComponent::setup() { this->server_ = socket::socket_ip_loop_monitored(SOCK_STREAM, 0); // monitored for incoming connections if (this->server_ == nullptr) { - ESP_LOGW(TAG, "Could not create socket"); + this->log_socket_error_(LOG_STR("creation")); this->mark_failed(); return; } int enable = 1; int err = this->server_->setsockopt(SOL_SOCKET, SO_REUSEADDR, &enable, sizeof(int)); if (err != 0) { - ESP_LOGW(TAG, "Socket unable to set reuseaddr: errno %d", err); + this->log_socket_error_(LOG_STR("reuseaddr")); // we can still continue } err = this->server_->setblocking(false); if (err != 0) { - ESP_LOGW(TAG, "Socket unable to set nonblocking mode: errno %d", err); + this->log_socket_error_(LOG_STR("non-blocking")); this->mark_failed(); return; } @@ -49,21 +69,21 @@ void ESPHomeOTAComponent::setup() { socklen_t sl = socket::set_sockaddr_any((struct sockaddr *) &server, sizeof(server), this->port_); if (sl == 0) { - ESP_LOGW(TAG, "Socket unable to set sockaddr: errno %d", errno); + this->log_socket_error_(LOG_STR("set sockaddr")); this->mark_failed(); return; } err = this->server_->bind((struct sockaddr *) &server, sizeof(server)); if (err != 0) { - ESP_LOGW(TAG, "Socket unable to bind: errno %d", errno); + this->log_socket_error_(LOG_STR("bind")); this->mark_failed(); return; } - err = this->server_->listen(4); + err = this->server_->listen(1); // Only one client at a time if (err != 0) { - ESP_LOGW(TAG, "Socket unable to listen: errno %d", errno); + this->log_socket_error_(LOG_STR("listen")); this->mark_failed(); return; } @@ -83,148 +103,182 @@ void ESPHomeOTAComponent::dump_config() { } void ESPHomeOTAComponent::loop() { - // Skip handle_() call if no client connected and no incoming connections + // Skip handle_handshake_() call if no client connected and no incoming connections // This optimization reduces idle loop overhead when OTA is not active - // Note: No need to check server_ for null as the component is marked failed in setup() if server_ creation fails + // Note: No need to check server_ for null as the component is marked failed in setup() + // if server_ creation fails if (this->client_ != nullptr || this->server_->ready()) { - this->handle_(); + this->handle_handshake_(); } } static const uint8_t FEATURE_SUPPORTS_COMPRESSION = 0x01; - -void ESPHomeOTAComponent::handle_() { - ota::OTAResponseTypes error_code = ota::OTA_RESPONSE_ERROR_UNKNOWN; - bool update_started = false; - size_t total = 0; - uint32_t last_progress = 0; - uint8_t buf[1024]; - char *sbuf = reinterpret_cast(buf); - size_t ota_size; - uint8_t ota_features; - std::unique_ptr backend; - (void) ota_features; -#if USE_OTA_VERSION == 2 - size_t size_acknowledged = 0; +#ifdef USE_OTA_SHA256 +static const uint8_t FEATURE_SUPPORTS_SHA256_AUTH = 0x02; #endif +// Temporary flag to allow MD5 downgrade for ~3 versions (until 2026.1.0) +// This allows users to downgrade via OTA if they encounter issues after updating. +// Without this, users would need to do a serial flash to downgrade. +// TODO: Remove this flag and all associated code in 2026.1.0 +#define ALLOW_OTA_DOWNGRADE_MD5 + +void ESPHomeOTAComponent::handle_handshake_() { + /// Handle the OTA handshake and authentication. + /// + /// This method is non-blocking and will return immediately if no data is available. + /// It manages the state machine through connection, magic bytes validation, feature + /// negotiation, and authentication before entering the blocking data transfer phase. + if (this->client_ == nullptr) { // We already checked server_->ready() in loop(), so we can accept directly struct sockaddr_storage source_addr; socklen_t addr_len = sizeof(source_addr); - this->client_ = this->server_->accept((struct sockaddr *) &source_addr, &addr_len); + int enable = 1; + + this->client_ = this->server_->accept_loop_monitored((struct sockaddr *) &source_addr, &addr_len); if (this->client_ == nullptr) return; + int err = this->client_->setsockopt(IPPROTO_TCP, TCP_NODELAY, &enable, sizeof(int)); + if (err != 0) { + this->log_socket_error_(LOG_STR("nodelay")); + this->cleanup_connection_(); + return; + } + err = this->client_->setblocking(false); + if (err != 0) { + this->log_socket_error_(LOG_STR("non-blocking")); + this->cleanup_connection_(); + return; + } + this->log_start_(LOG_STR("handshake")); + this->client_connect_time_ = App.get_loop_component_start_time(); + this->handshake_buf_pos_ = 0; // Reset handshake buffer position + this->ota_state_ = OTAState::MAGIC_READ; } - int enable = 1; - int err = this->client_->setsockopt(IPPROTO_TCP, TCP_NODELAY, &enable, sizeof(int)); - if (err != 0) { - ESP_LOGW(TAG, "Socket could not enable TCP nodelay, errno %d", errno); - this->client_->close(); - this->client_ = nullptr; + // Check for handshake timeout + uint32_t now = App.get_loop_component_start_time(); + if (now - this->client_connect_time_ > OTA_SOCKET_TIMEOUT_HANDSHAKE) { + ESP_LOGW(TAG, "Handshake timeout"); + this->cleanup_connection_(); return; } - ESP_LOGD(TAG, "Starting update from %s", this->client_->getpeername().c_str()); - this->status_set_warning(); -#ifdef USE_OTA_STATE_CALLBACK - this->state_callback_.call(ota::OTA_STARTED, 0.0f, 0); + switch (this->ota_state_) { + case OTAState::MAGIC_READ: { + // Try to read remaining magic bytes (5 total) + if (!this->try_read_(5, LOG_STR("read magic"))) { + return; + } + + // Validate magic bytes + static const uint8_t MAGIC_BYTES[5] = {0x6C, 0x26, 0xF7, 0x5C, 0x45}; + if (memcmp(this->handshake_buf_, MAGIC_BYTES, 5) != 0) { + ESP_LOGW(TAG, "Magic bytes mismatch! 0x%02X-0x%02X-0x%02X-0x%02X-0x%02X", this->handshake_buf_[0], + this->handshake_buf_[1], this->handshake_buf_[2], this->handshake_buf_[3], this->handshake_buf_[4]); + this->send_error_and_cleanup_(ota::OTA_RESPONSE_ERROR_MAGIC); + return; + } + + // Magic bytes valid, move to next state + this->transition_ota_state_(OTAState::MAGIC_ACK); + this->handshake_buf_[0] = ota::OTA_RESPONSE_OK; + this->handshake_buf_[1] = USE_OTA_VERSION; + [[fallthrough]]; + } + + case OTAState::MAGIC_ACK: { + // Send OK and version - 2 bytes + if (!this->try_write_(2, LOG_STR("ack magic"))) { + return; + } + // All bytes sent, create backend and move to next state + this->backend_ = ota::make_ota_backend(); + this->transition_ota_state_(OTAState::FEATURE_READ); + [[fallthrough]]; + } + + case OTAState::FEATURE_READ: { + // Read features - 1 byte + if (!this->try_read_(1, LOG_STR("read feature"))) { + return; + } + this->ota_features_ = this->handshake_buf_[0]; + ESP_LOGV(TAG, "Features: 0x%02X", this->ota_features_); + this->transition_ota_state_(OTAState::FEATURE_ACK); + this->handshake_buf_[0] = + ((this->ota_features_ & FEATURE_SUPPORTS_COMPRESSION) != 0 && this->backend_->supports_compression()) + ? ota::OTA_RESPONSE_SUPPORTS_COMPRESSION + : ota::OTA_RESPONSE_HEADER_OK; + [[fallthrough]]; + } + + case OTAState::FEATURE_ACK: { + // Acknowledge header - 1 byte + if (!this->try_write_(1, LOG_STR("ack feature"))) { + return; + } +#ifdef USE_OTA_PASSWORD + // If password is set, move to auth phase + if (!this->password_.empty()) { + this->transition_ota_state_(OTAState::AUTH_SEND); + } else #endif - - if (!this->readall_(buf, 5)) { - ESP_LOGW(TAG, "Reading magic bytes failed"); - goto error; // NOLINT(cppcoreguidelines-avoid-goto) - } - // 0x6C, 0x26, 0xF7, 0x5C, 0x45 - if (buf[0] != 0x6C || buf[1] != 0x26 || buf[2] != 0xF7 || buf[3] != 0x5C || buf[4] != 0x45) { - ESP_LOGW(TAG, "Magic bytes do not match! 0x%02X-0x%02X-0x%02X-0x%02X-0x%02X", buf[0], buf[1], buf[2], buf[3], - buf[4]); - error_code = ota::OTA_RESPONSE_ERROR_MAGIC; - goto error; // NOLINT(cppcoreguidelines-avoid-goto) - } - - // Send OK and version - 2 bytes - buf[0] = ota::OTA_RESPONSE_OK; - buf[1] = USE_OTA_VERSION; - this->writeall_(buf, 2); - - backend = ota::make_ota_backend(); - - // Read features - 1 byte - if (!this->readall_(buf, 1)) { - ESP_LOGW(TAG, "Reading features failed"); - goto error; // NOLINT(cppcoreguidelines-avoid-goto) - } - ota_features = buf[0]; // NOLINT - ESP_LOGV(TAG, "Features: 0x%02X", ota_features); - - // Acknowledge header - 1 byte - buf[0] = ota::OTA_RESPONSE_HEADER_OK; - if ((ota_features & FEATURE_SUPPORTS_COMPRESSION) != 0 && backend->supports_compression()) { - buf[0] = ota::OTA_RESPONSE_SUPPORTS_COMPRESSION; - } - - this->writeall_(buf, 1); + { + // No password, move directly to data phase + this->transition_ota_state_(OTAState::DATA); + } + [[fallthrough]]; + } #ifdef USE_OTA_PASSWORD - if (!this->password_.empty()) { - buf[0] = ota::OTA_RESPONSE_REQUEST_AUTH; - this->writeall_(buf, 1); - md5::MD5Digest md5{}; - md5.init(); - sprintf(sbuf, "%08" PRIx32, random_uint32()); - md5.add(sbuf, 8); - md5.calculate(); - md5.get_hex(sbuf); - ESP_LOGV(TAG, "Auth: Nonce is %s", sbuf); - - // Send nonce, 32 bytes hex MD5 - if (!this->writeall_(reinterpret_cast(sbuf), 32)) { - ESP_LOGW(TAG, "Auth: Writing nonce failed"); - goto error; // NOLINT(cppcoreguidelines-avoid-goto) + case OTAState::AUTH_SEND: { + // Non-blocking authentication send + if (!this->handle_auth_send_()) { + return; + } + this->transition_ota_state_(OTAState::AUTH_READ); + [[fallthrough]]; } - // prepare challenge - md5.init(); - md5.add(this->password_.c_str(), this->password_.length()); - // add nonce - md5.add(sbuf, 32); - - // Receive cnonce, 32 bytes hex MD5 - if (!this->readall_(buf, 32)) { - ESP_LOGW(TAG, "Auth: Reading cnonce failed"); - goto error; // NOLINT(cppcoreguidelines-avoid-goto) + case OTAState::AUTH_READ: { + // Non-blocking authentication read & verify + if (!this->handle_auth_read_()) { + return; + } + this->transition_ota_state_(OTAState::DATA); + [[fallthrough]]; } - sbuf[32] = '\0'; - ESP_LOGV(TAG, "Auth: CNonce is %s", sbuf); - // add cnonce - md5.add(sbuf, 32); +#endif - // calculate result - md5.calculate(); - md5.get_hex(sbuf); - ESP_LOGV(TAG, "Auth: Result is %s", sbuf); + case OTAState::DATA: + this->handle_data_(); + return; - // Receive result, 32 bytes hex MD5 - if (!this->readall_(buf + 64, 32)) { - ESP_LOGW(TAG, "Auth: Reading response failed"); - goto error; // NOLINT(cppcoreguidelines-avoid-goto) - } - sbuf[64 + 32] = '\0'; - ESP_LOGV(TAG, "Auth: Response is %s", sbuf + 64); - - bool matches = true; - for (uint8_t i = 0; i < 32; i++) - matches = matches && buf[i] == buf[64 + i]; - - if (!matches) { - ESP_LOGW(TAG, "Auth failed! Passwords do not match"); - error_code = ota::OTA_RESPONSE_ERROR_AUTH_INVALID; - goto error; // NOLINT(cppcoreguidelines-avoid-goto) - } + default: + break; } -#endif // USE_OTA_PASSWORD +} + +void ESPHomeOTAComponent::handle_data_() { + /// Handle the OTA data transfer and update process. + /// + /// This method is blocking and will not return until the OTA update completes, + /// fails, or times out. It receives the firmware data, writes it to flash, + /// and reboots on success. + /// + /// Authentication has already been handled in the non-blocking states AUTH_SEND/AUTH_READ. + ota::OTAResponseTypes error_code = ota::OTA_RESPONSE_ERROR_UNKNOWN; + bool update_started = false; + size_t total = 0; + uint32_t last_progress = 0; + uint8_t buf[OTA_BUFFER_SIZE]; + char *sbuf = reinterpret_cast(buf); + size_t ota_size; +#if USE_OTA_VERSION == 2 + size_t size_acknowledged = 0; +#endif // Acknowledge auth OK - 1 byte buf[0] = ota::OTA_RESPONSE_AUTH_OK; @@ -232,7 +286,7 @@ void ESPHomeOTAComponent::handle_() { // Read size, 4 bytes MSB first if (!this->readall_(buf, 4)) { - ESP_LOGW(TAG, "Reading size failed"); + this->log_read_error_(LOG_STR("size")); goto error; // NOLINT(cppcoreguidelines-avoid-goto) } ota_size = 0; @@ -242,7 +296,18 @@ void ESPHomeOTAComponent::handle_() { } ESP_LOGV(TAG, "Size is %u bytes", ota_size); - error_code = backend->begin(ota_size); + // Now that we've passed authentication and are actually + // starting the update, set the warning status and notify + // listeners. This ensures that port scanners do not + // accidentally trigger the update process. + this->log_start_(LOG_STR("update")); + this->status_set_warning(); +#ifdef USE_OTA_STATE_CALLBACK + this->state_callback_.call(ota::OTA_STARTED, 0.0f, 0); +#endif + + // This will block for a few seconds as it locks flash + error_code = this->backend_->begin(ota_size); if (error_code != ota::OTA_RESPONSE_OK) goto error; // NOLINT(cppcoreguidelines-avoid-goto) update_started = true; @@ -253,12 +318,12 @@ void ESPHomeOTAComponent::handle_() { // Read binary MD5, 32 bytes if (!this->readall_(buf, 32)) { - ESP_LOGW(TAG, "Reading binary MD5 checksum failed"); + this->log_read_error_(LOG_STR("MD5 checksum")); goto error; // NOLINT(cppcoreguidelines-avoid-goto) } sbuf[32] = '\0'; ESP_LOGV(TAG, "Update: Binary MD5 is %s", sbuf); - backend->set_update_md5(sbuf); + this->backend_->set_update_md5(sbuf); // Acknowledge MD5 OK - 1 byte buf[0] = ota::OTA_RESPONSE_BIN_MD5_OK; @@ -266,27 +331,24 @@ void ESPHomeOTAComponent::handle_() { while (total < ota_size) { // TODO: timeout check - size_t requested = std::min(sizeof(buf), ota_size - total); + size_t remaining = ota_size - total; + size_t requested = remaining < OTA_BUFFER_SIZE ? remaining : OTA_BUFFER_SIZE; ssize_t read = this->client_->read(buf, requested); if (read == -1) { - if (errno == EAGAIN || errno == EWOULDBLOCK) { - App.feed_wdt(); - delay(1); + if (this->would_block_(errno)) { + this->yield_and_feed_watchdog_(); continue; } - ESP_LOGW(TAG, "Error receiving data for update, errno %d", errno); + ESP_LOGW(TAG, "Read err %d", errno); goto error; // NOLINT(cppcoreguidelines-avoid-goto) } else if (read == 0) { - // $ man recv - // "When a stream socket peer has performed an orderly shutdown, the return value will - // be 0 (the traditional "end-of-file" return)." - ESP_LOGW(TAG, "Remote end closed connection"); + ESP_LOGW(TAG, "Remote closed"); goto error; // NOLINT(cppcoreguidelines-avoid-goto) } - error_code = backend->write(buf, read); + error_code = this->backend_->write(buf, read); if (error_code != ota::OTA_RESPONSE_OK) { - ESP_LOGW(TAG, "Error writing binary data to flash!, error_code: %d", error_code); + ESP_LOGW(TAG, "Flash write err %d", error_code); goto error; // NOLINT(cppcoreguidelines-avoid-goto) } total += read; @@ -307,8 +369,7 @@ void ESPHomeOTAComponent::handle_() { this->state_callback_.call(ota::OTA_IN_PROGRESS, percentage, 0); #endif // feed watchdog and give other tasks a chance to run - App.feed_wdt(); - yield(); + this->yield_and_feed_watchdog_(); } } @@ -316,9 +377,9 @@ void ESPHomeOTAComponent::handle_() { buf[0] = ota::OTA_RESPONSE_RECEIVE_OK; this->writeall_(buf, 1); - error_code = backend->end(); + error_code = this->backend_->end(); if (error_code != ota::OTA_RESPONSE_OK) { - ESP_LOGW(TAG, "Error ending update! error_code: %d", error_code); + ESP_LOGW(TAG, "End update err %d", error_code); goto error; // NOLINT(cppcoreguidelines-avoid-goto) } @@ -328,12 +389,11 @@ void ESPHomeOTAComponent::handle_() { // Read ACK if (!this->readall_(buf, 1) || buf[0] != ota::OTA_RESPONSE_OK) { - ESP_LOGW(TAG, "Reading back acknowledgement failed"); + this->log_read_error_(LOG_STR("ack")); // do not go to error, this is not fatal } - this->client_->close(); - this->client_ = nullptr; + this->cleanup_connection_(); delay(10); ESP_LOGI(TAG, "Update complete"); this->status_clear_warning(); @@ -346,11 +406,10 @@ void ESPHomeOTAComponent::handle_() { error: buf[0] = static_cast(error_code); this->writeall_(buf, 1); - this->client_->close(); - this->client_ = nullptr; + this->cleanup_connection_(); - if (backend != nullptr && update_started) { - backend->abort(); + if (this->backend_ != nullptr && update_started) { + this->backend_->abort(); } this->status_momentary_error("onerror", 5000); @@ -364,28 +423,24 @@ bool ESPHomeOTAComponent::readall_(uint8_t *buf, size_t len) { uint32_t at = 0; while (len - at > 0) { uint32_t now = millis(); - if (now - start > 1000) { - ESP_LOGW(TAG, "Timed out reading %d bytes of data", len); + if (now - start > OTA_SOCKET_TIMEOUT_DATA) { + ESP_LOGW(TAG, "Timeout reading %d bytes", len); return false; } ssize_t read = this->client_->read(buf + at, len - at); if (read == -1) { - if (errno == EAGAIN || errno == EWOULDBLOCK) { - App.feed_wdt(); - delay(1); - continue; + if (!this->would_block_(errno)) { + ESP_LOGW(TAG, "Read err %d bytes, errno %d", len, errno); + return false; } - ESP_LOGW(TAG, "Failed to read %d bytes of data, errno %d", len, errno); - return false; } else if (read == 0) { - ESP_LOGW(TAG, "Remote closed connection"); + ESP_LOGW(TAG, "Remote closed"); return false; } else { at += read; } - App.feed_wdt(); - delay(1); + this->yield_and_feed_watchdog_(); } return true; @@ -395,25 +450,21 @@ bool ESPHomeOTAComponent::writeall_(const uint8_t *buf, size_t len) { uint32_t at = 0; while (len - at > 0) { uint32_t now = millis(); - if (now - start > 1000) { - ESP_LOGW(TAG, "Timed out writing %d bytes of data", len); + if (now - start > OTA_SOCKET_TIMEOUT_DATA) { + ESP_LOGW(TAG, "Timeout writing %d bytes", len); return false; } ssize_t written = this->client_->write(buf + at, len - at); if (written == -1) { - if (errno == EAGAIN || errno == EWOULDBLOCK) { - App.feed_wdt(); - delay(1); - continue; + if (!this->would_block_(errno)) { + ESP_LOGW(TAG, "Write err %d bytes, errno %d", len, errno); + return false; } - ESP_LOGW(TAG, "Failed to write %d bytes of data, errno %d", len, errno); - return false; } else { at += written; } - App.feed_wdt(); - delay(1); + this->yield_and_feed_watchdog_(); } return true; } @@ -421,5 +472,342 @@ bool ESPHomeOTAComponent::writeall_(const uint8_t *buf, size_t len) { float ESPHomeOTAComponent::get_setup_priority() const { return setup_priority::AFTER_WIFI; } uint16_t ESPHomeOTAComponent::get_port() const { return this->port_; } void ESPHomeOTAComponent::set_port(uint16_t port) { this->port_ = port; } + +void ESPHomeOTAComponent::log_socket_error_(const LogString *msg) { + ESP_LOGW(TAG, "Socket %s: errno %d", LOG_STR_ARG(msg), errno); +} + +void ESPHomeOTAComponent::log_read_error_(const LogString *what) { ESP_LOGW(TAG, "Read %s failed", LOG_STR_ARG(what)); } + +void ESPHomeOTAComponent::log_start_(const LogString *phase) { + ESP_LOGD(TAG, "Starting %s from %s", LOG_STR_ARG(phase), this->client_->getpeername().c_str()); +} + +void ESPHomeOTAComponent::log_remote_closed_(const LogString *during) { + ESP_LOGW(TAG, "Remote closed at %s", LOG_STR_ARG(during)); +} + +bool ESPHomeOTAComponent::handle_read_error_(ssize_t read, const LogString *desc) { + if (read == -1 && this->would_block_(errno)) { + return false; // No data yet, try again next loop + } + + if (read <= 0) { + read == 0 ? this->log_remote_closed_(desc) : this->log_socket_error_(desc); + this->cleanup_connection_(); + return false; + } + return true; +} + +bool ESPHomeOTAComponent::handle_write_error_(ssize_t written, const LogString *desc) { + if (written == -1) { + if (this->would_block_(errno)) { + return false; // Try again next loop + } + this->log_socket_error_(desc); + this->cleanup_connection_(); + return false; + } + return true; +} + +bool ESPHomeOTAComponent::try_read_(size_t to_read, const LogString *desc) { + // Read bytes into handshake buffer, starting at handshake_buf_pos_ + size_t bytes_to_read = to_read - this->handshake_buf_pos_; + ssize_t read = this->client_->read(this->handshake_buf_ + this->handshake_buf_pos_, bytes_to_read); + + if (!this->handle_read_error_(read, desc)) { + return false; + } + + this->handshake_buf_pos_ += read; + // Return true only if we have all the requested bytes + return this->handshake_buf_pos_ >= to_read; +} + +bool ESPHomeOTAComponent::try_write_(size_t to_write, const LogString *desc) { + // Write bytes from handshake buffer, starting at handshake_buf_pos_ + size_t bytes_to_write = to_write - this->handshake_buf_pos_; + ssize_t written = this->client_->write(this->handshake_buf_ + this->handshake_buf_pos_, bytes_to_write); + + if (!this->handle_write_error_(written, desc)) { + return false; + } + + this->handshake_buf_pos_ += written; + // Return true only if we have written all the requested bytes + return this->handshake_buf_pos_ >= to_write; +} + +void ESPHomeOTAComponent::cleanup_connection_() { + this->client_->close(); + this->client_ = nullptr; + this->client_connect_time_ = 0; + this->handshake_buf_pos_ = 0; + this->ota_state_ = OTAState::IDLE; + this->ota_features_ = 0; + this->backend_ = nullptr; +#ifdef USE_OTA_PASSWORD + this->cleanup_auth_(); +#endif +} + +void ESPHomeOTAComponent::yield_and_feed_watchdog_() { + App.feed_wdt(); + delay(1); +} + +#ifdef USE_OTA_PASSWORD +void ESPHomeOTAComponent::log_auth_warning_(const LogString *msg) { ESP_LOGW(TAG, "Auth: %s", LOG_STR_ARG(msg)); } + +bool ESPHomeOTAComponent::select_auth_type_() { +#ifdef USE_OTA_SHA256 + bool client_supports_sha256 = (this->ota_features_ & FEATURE_SUPPORTS_SHA256_AUTH) != 0; + +#ifdef ALLOW_OTA_DOWNGRADE_MD5 + // Allow fallback to MD5 if client doesn't support SHA256 + if (client_supports_sha256) { + this->auth_type_ = ota::OTA_RESPONSE_REQUEST_SHA256_AUTH; + return true; + } +#ifdef USE_OTA_MD5 + this->log_auth_warning_(LOG_STR("Using deprecated MD5")); + this->auth_type_ = ota::OTA_RESPONSE_REQUEST_AUTH; + return true; +#else + this->log_auth_warning_(LOG_STR("SHA256 required")); + this->send_error_and_cleanup_(ota::OTA_RESPONSE_ERROR_AUTH_INVALID); + return false; +#endif // USE_OTA_MD5 + +#else // !ALLOW_OTA_DOWNGRADE_MD5 + // Require SHA256 + if (!client_supports_sha256) { + this->log_auth_warning_(LOG_STR("SHA256 required")); + this->send_error_and_cleanup_(ota::OTA_RESPONSE_ERROR_AUTH_INVALID); + return false; + } + this->auth_type_ = ota::OTA_RESPONSE_REQUEST_SHA256_AUTH; + return true; +#endif // ALLOW_OTA_DOWNGRADE_MD5 + +#else // !USE_OTA_SHA256 +#ifdef USE_OTA_MD5 + // Only MD5 available + this->auth_type_ = ota::OTA_RESPONSE_REQUEST_AUTH; + return true; +#else + // No auth methods available + this->log_auth_warning_(LOG_STR("No auth methods available")); + this->send_error_and_cleanup_(ota::OTA_RESPONSE_ERROR_AUTH_INVALID); + return false; +#endif // USE_OTA_MD5 +#endif // USE_OTA_SHA256 +} + +bool ESPHomeOTAComponent::handle_auth_send_() { + // Initialize auth buffer if not already done + if (!this->auth_buf_) { + // Select auth type based on client capabilities and configuration + if (!this->select_auth_type_()) { + return false; + } + + // Generate nonce - hasher must be created and used in same stack frame + // CRITICAL ESP32-S3 HARDWARE SHA ACCELERATION REQUIREMENTS: + // 1. Hash objects must NEVER be passed to another function (different stack frame) + // 2. NO Variable Length Arrays (VLAs) - they corrupt the stack with hardware DMA + // 3. All hash operations (init/add/calculate) must happen in the SAME function where object is created + // Violating these causes truncated hash output (20 bytes instead of 32) or memory corruption. + // + // Buffer layout after AUTH_READ completes: + // [0]: auth_type (1 byte) + // [1...hex_size]: nonce (hex_size bytes) - our random nonce sent in AUTH_SEND + // [1+hex_size...1+2*hex_size-1]: cnonce (hex_size bytes) - client's nonce + // [1+2*hex_size...1+3*hex_size-1]: response (hex_size bytes) - client's hash + + // Declare both hash objects in same stack frame, use pointer to select. + // NOTE: Both objects are declared here even though only one is used. This is REQUIRED for ESP32-S3 + // hardware SHA acceleration - the object must exist in this stack frame for all operations. + // Do NOT try to "optimize" by creating the object inside the if block, as it would go out of scope. +#ifdef USE_OTA_SHA256 + sha256::SHA256 sha_hasher; +#endif +#ifdef USE_OTA_MD5 + md5::MD5Digest md5_hasher; +#endif + HashBase *hasher = nullptr; + +#ifdef USE_OTA_SHA256 + if (this->auth_type_ == ota::OTA_RESPONSE_REQUEST_SHA256_AUTH) { + hasher = &sha_hasher; + } +#endif +#ifdef USE_OTA_MD5 + if (this->auth_type_ == ota::OTA_RESPONSE_REQUEST_AUTH) { + hasher = &md5_hasher; + } +#endif + + const size_t hex_size = hasher->get_size() * 2; + const size_t nonce_len = hasher->get_size() / 4; + const size_t auth_buf_size = 1 + 3 * hex_size; + this->auth_buf_ = std::make_unique(auth_buf_size); + this->auth_buf_pos_ = 0; + + char *buf = reinterpret_cast(this->auth_buf_.get() + 1); + if (!random_bytes(reinterpret_cast(buf), nonce_len)) { + this->log_auth_warning_(LOG_STR("Random failed")); + this->send_error_and_cleanup_(ota::OTA_RESPONSE_ERROR_UNKNOWN); + return false; + } + + hasher->init(); + hasher->add(buf, nonce_len); + hasher->calculate(); + this->auth_buf_[0] = this->auth_type_; + hasher->get_hex(buf); + +#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE + char log_buf[65]; // Fixed size for SHA256 hex (64) + null, works for MD5 (32) too + memcpy(log_buf, buf, hex_size); + log_buf[hex_size] = '\0'; + ESP_LOGV(TAG, "Auth: Nonce is %s", log_buf); +#endif + } + + // Try to write auth_type + nonce + size_t hex_size = this->get_auth_hex_size_(); + const size_t to_write = 1 + hex_size; + size_t remaining = to_write - this->auth_buf_pos_; + + ssize_t written = this->client_->write(this->auth_buf_.get() + this->auth_buf_pos_, remaining); + if (!this->handle_write_error_(written, LOG_STR("ack auth"))) { + return false; + } + + this->auth_buf_pos_ += written; + + // Check if we still have more to write + if (this->auth_buf_pos_ < to_write) { + return false; // More to write, try again next loop + } + + // All written, prepare for reading phase + this->auth_buf_pos_ = 0; + return true; +} + +bool ESPHomeOTAComponent::handle_auth_read_() { + size_t hex_size = this->get_auth_hex_size_(); + const size_t to_read = hex_size * 2; // CNonce + Response + + // Try to read remaining bytes (CNonce + Response) + // We read cnonce+response starting at offset 1+hex_size (after auth_type and our nonce) + size_t cnonce_offset = 1 + hex_size; // Offset where cnonce should be stored in buffer + size_t remaining = to_read - this->auth_buf_pos_; + ssize_t read = this->client_->read(this->auth_buf_.get() + cnonce_offset + this->auth_buf_pos_, remaining); + + if (!this->handle_read_error_(read, LOG_STR("read auth"))) { + return false; + } + + this->auth_buf_pos_ += read; + + // Check if we still need more data + if (this->auth_buf_pos_ < to_read) { + return false; // More to read, try again next loop + } + + // We have all the data, verify it + const char *nonce = reinterpret_cast(this->auth_buf_.get() + 1); + const char *cnonce = nonce + hex_size; + const char *response = cnonce + hex_size; + + // CRITICAL ESP32-S3: Hash objects must stay in same stack frame (no passing to other functions). + // Declare both hash objects in same stack frame, use pointer to select. + // NOTE: Both objects are declared here even though only one is used. This is REQUIRED for ESP32-S3 + // hardware SHA acceleration - the object must exist in this stack frame for all operations. + // Do NOT try to "optimize" by creating the object inside the if block, as it would go out of scope. +#ifdef USE_OTA_SHA256 + sha256::SHA256 sha_hasher; +#endif +#ifdef USE_OTA_MD5 + md5::MD5Digest md5_hasher; +#endif + HashBase *hasher = nullptr; + +#ifdef USE_OTA_SHA256 + if (this->auth_type_ == ota::OTA_RESPONSE_REQUEST_SHA256_AUTH) { + hasher = &sha_hasher; + } +#endif +#ifdef USE_OTA_MD5 + if (this->auth_type_ == ota::OTA_RESPONSE_REQUEST_AUTH) { + hasher = &md5_hasher; + } +#endif + + hasher->init(); + hasher->add(this->password_.c_str(), this->password_.length()); + hasher->add(nonce, hex_size * 2); // Add both nonce and cnonce (contiguous in buffer) + hasher->calculate(); + +#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE + char log_buf[65]; // Fixed size for SHA256 hex (64) + null, works for MD5 (32) too + // Log CNonce + memcpy(log_buf, cnonce, hex_size); + log_buf[hex_size] = '\0'; + ESP_LOGV(TAG, "Auth: CNonce is %s", log_buf); + + // Log computed hash + hasher->get_hex(log_buf); + log_buf[hex_size] = '\0'; + ESP_LOGV(TAG, "Auth: Result is %s", log_buf); + + // Log received response + memcpy(log_buf, response, hex_size); + log_buf[hex_size] = '\0'; + ESP_LOGV(TAG, "Auth: Response is %s", log_buf); +#endif + + // Compare response + bool matches = hasher->equals_hex(response); + + if (!matches) { + this->log_auth_warning_(LOG_STR("Password mismatch")); + this->send_error_and_cleanup_(ota::OTA_RESPONSE_ERROR_AUTH_INVALID); + return false; + } + + // Authentication successful - clean up auth state + this->cleanup_auth_(); + + return true; +} + +size_t ESPHomeOTAComponent::get_auth_hex_size_() const { +#ifdef USE_OTA_SHA256 + if (this->auth_type_ == ota::OTA_RESPONSE_REQUEST_SHA256_AUTH) { + return SHA256_HEX_SIZE; + } +#endif +#ifdef USE_OTA_MD5 + return MD5_HEX_SIZE; +#else +#ifndef USE_OTA_SHA256 +#error "Either USE_OTA_MD5 or USE_OTA_SHA256 must be defined when USE_OTA_PASSWORD is enabled" +#endif +#endif +} + +void ESPHomeOTAComponent::cleanup_auth_() { + this->auth_buf_ = nullptr; + this->auth_buf_pos_ = 0; + this->auth_type_ = 0; +} +#endif // USE_OTA_PASSWORD + } // namespace esphome #endif diff --git a/esphome/components/esphome/ota/ota_esphome.h b/esphome/components/esphome/ota/ota_esphome.h index e0d09ff37e..d4a8410d35 100644 --- a/esphome/components/esphome/ota/ota_esphome.h +++ b/esphome/components/esphome/ota/ota_esphome.h @@ -2,16 +2,30 @@ #include "esphome/core/defines.h" #ifdef USE_OTA -#include "esphome/core/helpers.h" -#include "esphome/core/preferences.h" #include "esphome/components/ota/ota_backend.h" #include "esphome/components/socket/socket.h" +#include "esphome/core/helpers.h" +#include "esphome/core/log.h" +#include "esphome/core/preferences.h" +#include "esphome/core/hash_base.h" namespace esphome { /// ESPHomeOTAComponent provides a simple way to integrate Over-the-Air updates into your app using ArduinoOTA. class ESPHomeOTAComponent : public ota::OTAComponent { public: + enum class OTAState : uint8_t { + IDLE, + MAGIC_READ, // Reading magic bytes + MAGIC_ACK, // Sending OK and version after magic bytes + FEATURE_READ, // Reading feature flags from client + FEATURE_ACK, // Sending feature acknowledgment +#ifdef USE_OTA_PASSWORD + AUTH_SEND, // Sending authentication request + AUTH_READ, // Reading authentication data +#endif // USE_OTA_PASSWORD + DATA, // BLOCKING! Processing OTA data (update, etc.) + }; #ifdef USE_OTA_PASSWORD void set_auth_password(const std::string &password) { password_ = password; } #endif // USE_OTA_PASSWORD @@ -27,18 +41,61 @@ class ESPHomeOTAComponent : public ota::OTAComponent { uint16_t get_port() const; protected: - void handle_(); + void handle_handshake_(); + void handle_data_(); +#ifdef USE_OTA_PASSWORD + bool handle_auth_send_(); + bool handle_auth_read_(); + bool select_auth_type_(); + size_t get_auth_hex_size_() const; + void cleanup_auth_(); + void log_auth_warning_(const LogString *msg); +#endif // USE_OTA_PASSWORD bool readall_(uint8_t *buf, size_t len); bool writeall_(const uint8_t *buf, size_t len); + bool try_read_(size_t to_read, const LogString *desc); + bool try_write_(size_t to_write, const LogString *desc); + + inline bool would_block_(int error_code) const { return error_code == EAGAIN || error_code == EWOULDBLOCK; } + bool handle_read_error_(ssize_t read, const LogString *desc); + bool handle_write_error_(ssize_t written, const LogString *desc); + inline void transition_ota_state_(OTAState next_state) { + this->ota_state_ = next_state; + this->handshake_buf_pos_ = 0; // Reset buffer position for next state + } + + void log_socket_error_(const LogString *msg); + void log_read_error_(const LogString *what); + void log_start_(const LogString *phase); + void log_remote_closed_(const LogString *during); + void cleanup_connection_(); + inline void send_error_and_cleanup_(ota::OTAResponseTypes error) { + uint8_t error_byte = static_cast(error); + this->client_->write(&error_byte, 1); // Best effort, non-blocking + this->cleanup_connection_(); + } + void yield_and_feed_watchdog_(); + #ifdef USE_OTA_PASSWORD std::string password_; #endif // USE_OTA_PASSWORD - uint16_t port_; - std::unique_ptr server_; std::unique_ptr client_; + std::unique_ptr backend_; + + uint32_t client_connect_time_{0}; + uint16_t port_; + uint8_t handshake_buf_[5]; + OTAState ota_state_{OTAState::IDLE}; + uint8_t handshake_buf_pos_{0}; + uint8_t ota_features_{0}; +#ifdef USE_OTA_PASSWORD + std::unique_ptr auth_buf_; + uint8_t auth_buf_pos_{0}; + uint8_t auth_type_{0}; // Store auth type to know which hasher to use +#endif // USE_OTA_PASSWORD }; } // namespace esphome diff --git a/esphome/components/espnow/__init__.py b/esphome/components/espnow/__init__.py new file mode 100644 index 0000000000..9d2f17440c --- /dev/null +++ b/esphome/components/espnow/__init__.py @@ -0,0 +1,309 @@ +from esphome import automation, core +import esphome.codegen as cg +from esphome.components import wifi +from esphome.components.udp import CONF_ON_RECEIVE +import esphome.config_validation as cv +from esphome.const import ( + CONF_ADDRESS, + CONF_CHANNEL, + CONF_DATA, + CONF_ENABLE_ON_BOOT, + CONF_ID, + CONF_ON_ERROR, + CONF_TRIGGER_ID, + CONF_WIFI, +) +from esphome.core import CORE, HexInt +from esphome.types import ConfigType + +CODEOWNERS = ["@jesserockz"] + +byte_vector = cg.std_vector.template(cg.uint8) +peer_address_t = cg.std_ns.class_("array").template(cg.uint8, 6) + +espnow_ns = cg.esphome_ns.namespace("espnow") +ESPNowComponent = espnow_ns.class_("ESPNowComponent", cg.Component) + +# Handler interfaces that other components can use to register callbacks +ESPNowReceivedPacketHandler = espnow_ns.class_("ESPNowReceivedPacketHandler") +ESPNowUnknownPeerHandler = espnow_ns.class_("ESPNowUnknownPeerHandler") +ESPNowBroadcastedHandler = espnow_ns.class_("ESPNowBroadcastedHandler") + +ESPNowRecvInfo = espnow_ns.class_("ESPNowRecvInfo") +ESPNowRecvInfoConstRef = ESPNowRecvInfo.operator("const").operator("ref") + +SendAction = espnow_ns.class_("SendAction", automation.Action) +SetChannelAction = espnow_ns.class_("SetChannelAction", automation.Action) +AddPeerAction = espnow_ns.class_("AddPeerAction", automation.Action) +DeletePeerAction = espnow_ns.class_("DeletePeerAction", automation.Action) + +ESPNowHandlerTrigger = automation.Trigger.template( + ESPNowRecvInfoConstRef, + cg.uint8.operator("const").operator("ptr"), + cg.uint8, +) + +OnUnknownPeerTrigger = espnow_ns.class_( + "OnUnknownPeerTrigger", ESPNowHandlerTrigger, ESPNowUnknownPeerHandler +) +OnReceiveTrigger = espnow_ns.class_( + "OnReceiveTrigger", ESPNowHandlerTrigger, ESPNowReceivedPacketHandler +) +OnBroadcastedTrigger = espnow_ns.class_( + "OnBroadcastedTrigger", ESPNowHandlerTrigger, ESPNowBroadcastedHandler +) + + +CONF_AUTO_ADD_PEER = "auto_add_peer" +CONF_PEERS = "peers" +CONF_ON_SENT = "on_sent" +CONF_ON_UNKNOWN_PEER = "on_unknown_peer" +CONF_ON_BROADCAST = "on_broadcast" +CONF_CONTINUE_ON_ERROR = "continue_on_error" +CONF_WAIT_FOR_SENT = "wait_for_sent" + +MAX_ESPNOW_PACKET_SIZE = 250 # Maximum size of the payload in bytes + + +CONFIG_SCHEMA = cv.All( + cv.Schema( + { + cv.GenerateID(): cv.declare_id(ESPNowComponent), + cv.OnlyWithout(CONF_CHANNEL, CONF_WIFI): wifi.validate_channel, + cv.Optional(CONF_ENABLE_ON_BOOT, default=True): cv.boolean, + cv.Optional(CONF_AUTO_ADD_PEER, default=False): cv.boolean, + cv.Optional(CONF_PEERS): cv.ensure_list(cv.mac_address), + cv.Optional(CONF_ON_UNKNOWN_PEER): automation.validate_automation( + { + cv.GenerateID(CONF_TRIGGER_ID): cv.declare_id(OnUnknownPeerTrigger), + }, + single=True, + ), + cv.Optional(CONF_ON_RECEIVE): automation.validate_automation( + { + cv.GenerateID(CONF_TRIGGER_ID): cv.declare_id(OnReceiveTrigger), + cv.Optional(CONF_ADDRESS): cv.mac_address, + } + ), + cv.Optional(CONF_ON_BROADCAST): automation.validate_automation( + { + cv.GenerateID(CONF_TRIGGER_ID): cv.declare_id(OnBroadcastedTrigger), + cv.Optional(CONF_ADDRESS): cv.mac_address, + } + ), + }, + ).extend(cv.COMPONENT_SCHEMA), + cv.only_on_esp32, +) + + +async def _trigger_to_code(config): + if address := config.get(CONF_ADDRESS): + address = address.parts + trigger = cg.new_Pvariable(config[CONF_TRIGGER_ID], address) + await automation.build_automation( + trigger, + [ + (ESPNowRecvInfoConstRef, "info"), + (cg.uint8.operator("const").operator("ptr"), "data"), + (cg.uint8, "size"), + ], + config, + ) + return trigger + + +async def to_code(config): + var = cg.new_Pvariable(config[CONF_ID]) + await cg.register_component(var, config) + + if CORE.using_arduino: + cg.add_library("WiFi", None) + + cg.add_define("USE_ESPNOW") + if wifi_channel := config.get(CONF_CHANNEL): + cg.add(var.set_wifi_channel(wifi_channel)) + + cg.add(var.set_auto_add_peer(config[CONF_AUTO_ADD_PEER])) + + for peer in config.get(CONF_PEERS, []): + cg.add(var.add_peer(peer.parts)) + + if on_receive := config.get(CONF_ON_UNKNOWN_PEER): + trigger = await _trigger_to_code(on_receive) + cg.add(var.register_unknown_peer_handler(trigger)) + + for on_receive in config.get(CONF_ON_RECEIVE, []): + trigger = await _trigger_to_code(on_receive) + cg.add(var.register_received_handler(trigger)) + + for on_receive in config.get(CONF_ON_BROADCAST, []): + trigger = await _trigger_to_code(on_receive) + cg.add(var.register_broadcasted_handler(trigger)) + + +# ========================================== A C T I O N S ================================================ + + +def validate_peer(value): + if isinstance(value, cv.Lambda): + return cv.returning_lambda(value) + return cv.mac_address(value) + + +def _validate_raw_data(value): + if isinstance(value, str): + if len(value) >= MAX_ESPNOW_PACKET_SIZE: + raise cv.Invalid( + f"'{CONF_DATA}' must be less than {MAX_ESPNOW_PACKET_SIZE} characters long, got {len(value)}" + ) + return value + if isinstance(value, list): + if len(value) > MAX_ESPNOW_PACKET_SIZE: + raise cv.Invalid( + f"'{CONF_DATA}' must be less than {MAX_ESPNOW_PACKET_SIZE} bytes long, got {len(value)}" + ) + return cv.Schema([cv.hex_uint8_t])(value) + raise cv.Invalid( + f"'{CONF_DATA}' must either be a string wrapped in quotes or a list of bytes" + ) + + +async def register_peer(var, config, args): + peer = config[CONF_ADDRESS] + if isinstance(peer, core.MACAddress): + peer = [HexInt(p) for p in peer.parts] + + template_ = await cg.templatable(peer, args, peer_address_t, peer_address_t) + cg.add(var.set_address(template_)) + + +PEER_SCHEMA = cv.Schema( + { + cv.GenerateID(): cv.use_id(ESPNowComponent), + cv.Required(CONF_ADDRESS): cv.templatable(cv.mac_address), + } +) + +SEND_SCHEMA = PEER_SCHEMA.extend( + { + cv.Required(CONF_DATA): cv.templatable(_validate_raw_data), + cv.Optional(CONF_ON_SENT): automation.validate_action_list, + cv.Optional(CONF_ON_ERROR): automation.validate_action_list, + cv.Optional(CONF_WAIT_FOR_SENT, default=True): cv.boolean, + cv.Optional(CONF_CONTINUE_ON_ERROR, default=True): cv.boolean, + } +) + + +def _validate_send_action(config): + if not config[CONF_WAIT_FOR_SENT] and not config[CONF_CONTINUE_ON_ERROR]: + raise cv.Invalid( + f"'{CONF_CONTINUE_ON_ERROR}' cannot be false if '{CONF_WAIT_FOR_SENT}' is false as the automation will not wait for the failed result.", + path=[CONF_CONTINUE_ON_ERROR], + ) + return config + + +SEND_SCHEMA.add_extra(_validate_send_action) + + +@automation.register_action( + "espnow.send", + SendAction, + SEND_SCHEMA, +) +@automation.register_action( + "espnow.broadcast", + SendAction, + cv.maybe_simple_value( + SEND_SCHEMA.extend( + { + cv.Optional(CONF_ADDRESS, default="FF:FF:FF:FF:FF:FF"): cv.mac_address, + } + ), + key=CONF_DATA, + ), +) +async def send_action( + config: ConfigType, + action_id: core.ID, + template_arg: cg.TemplateArguments, + args: list[tuple], +): + var = cg.new_Pvariable(action_id, template_arg) + await cg.register_parented(var, config[CONF_ID]) + + await register_peer(var, config, args) + + data = config.get(CONF_DATA, []) + if isinstance(data, str): + data = [cg.RawExpression(f"'{c}'") for c in data] + templ = await cg.templatable(data, args, byte_vector, byte_vector) + cg.add(var.set_data(templ)) + + cg.add(var.set_wait_for_sent(config[CONF_WAIT_FOR_SENT])) + cg.add(var.set_continue_on_error(config[CONF_CONTINUE_ON_ERROR])) + + if on_sent_config := config.get(CONF_ON_SENT): + actions = await automation.build_action_list(on_sent_config, template_arg, args) + cg.add(var.add_on_sent(actions)) + if on_error_config := config.get(CONF_ON_ERROR): + actions = await automation.build_action_list( + on_error_config, template_arg, args + ) + cg.add(var.add_on_error(actions)) + return var + + +@automation.register_action( + "espnow.peer.add", + AddPeerAction, + cv.maybe_simple_value( + PEER_SCHEMA, + key=CONF_ADDRESS, + ), +) +@automation.register_action( + "espnow.peer.delete", + DeletePeerAction, + cv.maybe_simple_value( + PEER_SCHEMA, + key=CONF_ADDRESS, + ), +) +async def peer_action( + config: ConfigType, + action_id: core.ID, + template_arg: cg.TemplateArguments, + args: list[tuple], +): + var = cg.new_Pvariable(action_id, template_arg) + await cg.register_parented(var, config[CONF_ID]) + await register_peer(var, config, args) + + return var + + +@automation.register_action( + "espnow.set_channel", + SetChannelAction, + cv.maybe_simple_value( + { + cv.GenerateID(): cv.use_id(ESPNowComponent), + cv.Required(CONF_CHANNEL): cv.templatable(wifi.validate_channel), + }, + key=CONF_CHANNEL, + ), +) +async def channel_action( + config: ConfigType, + action_id: core.ID, + template_arg: cg.TemplateArguments, + args: list[tuple], +): + var = cg.new_Pvariable(action_id, template_arg) + await cg.register_parented(var, config[CONF_ID]) + template_ = await cg.templatable(config[CONF_CHANNEL], args, cg.uint8) + cg.add(var.set_channel(template_)) + return var diff --git a/esphome/components/espnow/automation.h b/esphome/components/espnow/automation.h new file mode 100644 index 0000000000..2416377859 --- /dev/null +++ b/esphome/components/espnow/automation.h @@ -0,0 +1,175 @@ +#pragma once + +#ifdef USE_ESP32 + +#include "espnow_component.h" + +#include "esphome/core/automation.h" +#include "esphome/core/base_automation.h" + +namespace esphome::espnow { + +template class SendAction : public Action, public Parented { + TEMPLATABLE_VALUE(peer_address_t, address); + TEMPLATABLE_VALUE(std::vector, data); + + public: + void add_on_sent(const std::vector *> &actions) { + this->sent_.add_actions(actions); + if (this->flags_.wait_for_sent) { + this->sent_.add_action(new LambdaAction([this](Ts... x) { this->play_next_(x...); })); + } + } + void add_on_error(const std::vector *> &actions) { + this->error_.add_actions(actions); + if (this->flags_.wait_for_sent) { + this->error_.add_action(new LambdaAction([this](Ts... x) { + if (this->flags_.continue_on_error) { + this->play_next_(x...); + } else { + this->stop_complex(); + } + })); + } + } + + void set_wait_for_sent(bool wait_for_sent) { this->flags_.wait_for_sent = wait_for_sent; } + void set_continue_on_error(bool continue_on_error) { this->flags_.continue_on_error = continue_on_error; } + + void play_complex(Ts... x) override { + this->num_running_++; + send_callback_t send_callback = [this, x...](esp_err_t status) { + if (status == ESP_OK) { + if (!this->sent_.empty()) { + this->sent_.play(x...); + } else if (this->flags_.wait_for_sent) { + this->play_next_(x...); + } + } else { + if (!this->error_.empty()) { + this->error_.play(x...); + } else if (this->flags_.wait_for_sent) { + if (this->flags_.continue_on_error) { + this->play_next_(x...); + } else { + this->stop_complex(); + } + } + } + }; + peer_address_t address = this->address_.value(x...); + std::vector data = this->data_.value(x...); + esp_err_t err = this->parent_->send(address.data(), data, send_callback); + if (err != ESP_OK) { + send_callback(err); + } else if (!this->flags_.wait_for_sent) { + this->play_next_(x...); + } + } + + void play(Ts... x) override { /* ignore - see play_complex */ + } + + void stop() override { + this->sent_.stop(); + this->error_.stop(); + } + + protected: + ActionList sent_; + ActionList error_; + + struct { + uint8_t wait_for_sent : 1; // Wait for the send operation to complete before continuing automation + uint8_t continue_on_error : 1; // Continue automation even if the send operation fails + uint8_t reserved : 6; // Reserved for future use + } flags_{0}; +}; + +template class AddPeerAction : public Action, public Parented { + TEMPLATABLE_VALUE(peer_address_t, address); + + public: + void play(Ts... x) override { + peer_address_t address = this->address_.value(x...); + this->parent_->add_peer(address.data()); + } +}; + +template class DeletePeerAction : public Action, public Parented { + TEMPLATABLE_VALUE(peer_address_t, address); + + public: + void play(Ts... x) override { + peer_address_t address = this->address_.value(x...); + this->parent_->del_peer(address.data()); + } +}; + +template class SetChannelAction : public Action, public Parented { + public: + TEMPLATABLE_VALUE(uint8_t, channel) + void play(Ts... x) override { + if (this->parent_->is_wifi_enabled()) { + return; + } + this->parent_->set_wifi_channel(this->channel_.value(x...)); + this->parent_->apply_wifi_channel(); + } +}; + +class OnReceiveTrigger : public Trigger, + public ESPNowReceivedPacketHandler { + public: + explicit OnReceiveTrigger(std::array address) : has_address_(true) { + memcpy(this->address_, address.data(), ESP_NOW_ETH_ALEN); + } + + explicit OnReceiveTrigger() : has_address_(false) {} + + bool on_received(const ESPNowRecvInfo &info, const uint8_t *data, uint8_t size) override { + bool match = !this->has_address_ || (memcmp(this->address_, info.src_addr, ESP_NOW_ETH_ALEN) == 0); + if (!match) + return false; + + this->trigger(info, data, size); + return false; // Return false to continue processing other internal handlers + } + + protected: + bool has_address_{false}; + const uint8_t *address_[ESP_NOW_ETH_ALEN]; +}; +class OnUnknownPeerTrigger : public Trigger, + public ESPNowUnknownPeerHandler { + public: + bool on_unknown_peer(const ESPNowRecvInfo &info, const uint8_t *data, uint8_t size) override { + this->trigger(info, data, size); + return false; // Return false to continue processing other internal handlers + } +}; +class OnBroadcastedTrigger : public Trigger, + public ESPNowBroadcastedHandler { + public: + explicit OnBroadcastedTrigger(std::array address) : has_address_(true) { + memcpy(this->address_, address.data(), ESP_NOW_ETH_ALEN); + } + explicit OnBroadcastedTrigger() : has_address_(false) {} + + bool on_broadcasted(const ESPNowRecvInfo &info, const uint8_t *data, uint8_t size) override { + bool match = !this->has_address_ || (memcmp(this->address_, info.src_addr, ESP_NOW_ETH_ALEN) == 0); + if (!match) + return false; + + this->trigger(info, data, size); + return false; // Return false to continue processing other internal handlers + } + + protected: + bool has_address_{false}; + const uint8_t *address_[ESP_NOW_ETH_ALEN]; +}; + +} // namespace esphome::espnow + +#endif // USE_ESP32 diff --git a/esphome/components/espnow/espnow_component.cpp b/esphome/components/espnow/espnow_component.cpp new file mode 100644 index 0000000000..b0d5938dba --- /dev/null +++ b/esphome/components/espnow/espnow_component.cpp @@ -0,0 +1,468 @@ +#include "espnow_component.h" + +#ifdef USE_ESP32 + +#include "espnow_err.h" + +#include "esphome/core/defines.h" +#include "esphome/core/log.h" + +#include +#include +#include +#include +#include +#include +#include + +#ifdef USE_WIFI +#include "esphome/components/wifi/wifi_component.h" +#endif + +namespace esphome::espnow { + +static constexpr const char *TAG = "espnow"; + +static const esp_err_t CONFIG_ESPNOW_WAKE_WINDOW = 50; +static const esp_err_t CONFIG_ESPNOW_WAKE_INTERVAL = 100; + +ESPNowComponent *global_esp_now = nullptr; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) + +static const LogString *espnow_error_to_str(esp_err_t error) { + switch (error) { + case ESP_ERR_ESPNOW_FAILED: + return LOG_STR("ESPNow is in fail mode"); + case ESP_ERR_ESPNOW_OWN_ADDRESS: + return LOG_STR("Message to your self"); + case ESP_ERR_ESPNOW_DATA_SIZE: + return LOG_STR("Data size to large"); + case ESP_ERR_ESPNOW_PEER_NOT_SET: + return LOG_STR("Peer address not set"); + case ESP_ERR_ESPNOW_PEER_NOT_PAIRED: + return LOG_STR("Peer address not paired"); + case ESP_ERR_ESPNOW_NOT_INIT: + return LOG_STR("Not init"); + case ESP_ERR_ESPNOW_ARG: + return LOG_STR("Invalid argument"); + case ESP_ERR_ESPNOW_INTERNAL: + return LOG_STR("Internal Error"); + case ESP_ERR_ESPNOW_NO_MEM: + return LOG_STR("Our of memory"); + case ESP_ERR_ESPNOW_NOT_FOUND: + return LOG_STR("Peer not found"); + case ESP_ERR_ESPNOW_IF: + return LOG_STR("Interface does not match"); + case ESP_OK: + return LOG_STR("OK"); + case ESP_NOW_SEND_FAIL: + return LOG_STR("Failed"); + default: + return LOG_STR("Unknown Error"); + } +} + +std::string peer_str(uint8_t *peer) { + if (peer == nullptr || peer[0] == 0) { + return "[Not Set]"; + } else if (memcmp(peer, ESPNOW_BROADCAST_ADDR, ESP_NOW_ETH_ALEN) == 0) { + return "[Broadcast]"; + } else if (memcmp(peer, ESPNOW_MULTICAST_ADDR, ESP_NOW_ETH_ALEN) == 0) { + return "[Multicast]"; + } else { + return format_mac_address_pretty(peer); + } +} + +#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 5, 0) +void on_send_report(const esp_now_send_info_t *info, esp_now_send_status_t status) +#else +void on_send_report(const uint8_t *mac_addr, esp_now_send_status_t status) +#endif +{ + // Allocate an event from the pool + ESPNowPacket *packet = global_esp_now->receive_packet_pool_.allocate(); + if (packet == nullptr) { + // No events available - queue is full or we're out of memory + global_esp_now->receive_packet_queue_.increment_dropped_count(); + return; + } + +// Load new packet data (replaces previous packet) +#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 5, 0) + packet->load_sent_data(info->des_addr, status); +#else + packet->load_sent_data(mac_addr, status); +#endif + + // Push the packet to the queue + global_esp_now->receive_packet_queue_.push(packet); + // Push always because we're the only producer and the pool ensures we never exceed queue size +} + +void on_data_received(const esp_now_recv_info_t *info, const uint8_t *data, int size) { + // Allocate an event from the pool + ESPNowPacket *packet = global_esp_now->receive_packet_pool_.allocate(); + if (packet == nullptr) { + // No events available - queue is full or we're out of memory + global_esp_now->receive_packet_queue_.increment_dropped_count(); + return; + } + + // Load new packet data (replaces previous packet) + packet->load_received_data(info, data, size); + + // Push the packet to the queue + global_esp_now->receive_packet_queue_.push(packet); + // Push always because we're the only producer and the pool ensures we never exceed queue size +} + +ESPNowComponent::ESPNowComponent() { global_esp_now = this; } + +void ESPNowComponent::dump_config() { + uint32_t version = 0; + esp_now_get_version(&version); + + ESP_LOGCONFIG(TAG, "espnow:"); + if (this->is_disabled()) { + ESP_LOGCONFIG(TAG, " Disabled"); + return; + } + ESP_LOGCONFIG(TAG, + " Own address: %s\n" + " Version: v%" PRIu32 "\n" + " Wi-Fi channel: %d", + format_mac_address_pretty(this->own_address_).c_str(), version, this->wifi_channel_); +#ifdef USE_WIFI + ESP_LOGCONFIG(TAG, " Wi-Fi enabled: %s", YESNO(this->is_wifi_enabled())); +#endif +} + +bool ESPNowComponent::is_wifi_enabled() { +#ifdef USE_WIFI + return wifi::global_wifi_component != nullptr && !wifi::global_wifi_component->is_disabled(); +#else + return false; +#endif +} + +void ESPNowComponent::setup() { + if (this->enable_on_boot_) { + this->enable_(); + } else { + this->state_ = ESPNOW_STATE_DISABLED; + } +} + +void ESPNowComponent::enable() { + if (this->state_ == ESPNOW_STATE_ENABLED) + return; + + ESP_LOGD(TAG, "Enabling"); + this->state_ = ESPNOW_STATE_OFF; + + this->enable_(); +} + +void ESPNowComponent::enable_() { + if (!this->is_wifi_enabled()) { + esp_event_loop_create_default(); + + wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT(); + + ESP_ERROR_CHECK(esp_wifi_init(&cfg)); + ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA)); + ESP_ERROR_CHECK(esp_wifi_set_storage(WIFI_STORAGE_RAM)); + ESP_ERROR_CHECK(esp_wifi_set_ps(WIFI_PS_NONE)); + ESP_ERROR_CHECK(esp_wifi_start()); + ESP_ERROR_CHECK(esp_wifi_disconnect()); + + this->apply_wifi_channel(); + } + this->get_wifi_channel(); + + esp_err_t err = esp_now_init(); + if (err != ESP_OK) { + ESP_LOGE(TAG, "esp_now_init failed: %s", esp_err_to_name(err)); + this->mark_failed(); + return; + } + + err = esp_now_register_recv_cb(on_data_received); + if (err != ESP_OK) { + ESP_LOGE(TAG, "esp_now_register_recv_cb failed: %s", esp_err_to_name(err)); + this->mark_failed(); + return; + } + + err = esp_now_register_send_cb(on_send_report); + if (err != ESP_OK) { + ESP_LOGE(TAG, "esp_now_register_recv_cb failed: %s", esp_err_to_name(err)); + this->mark_failed(); + return; + } + + esp_wifi_get_mac(WIFI_IF_STA, this->own_address_); + +#ifdef USE_DEEP_SLEEP + esp_now_set_wake_window(CONFIG_ESPNOW_WAKE_WINDOW); + esp_wifi_connectionless_module_set_wake_interval(CONFIG_ESPNOW_WAKE_INTERVAL); +#endif + + this->state_ = ESPNOW_STATE_ENABLED; + + for (auto peer : this->peers_) { + this->add_peer(peer.address); + } +} + +void ESPNowComponent::disable() { + if (this->state_ == ESPNOW_STATE_DISABLED) + return; + + ESP_LOGD(TAG, "Disabling"); + this->state_ = ESPNOW_STATE_DISABLED; + + esp_now_unregister_recv_cb(); + esp_now_unregister_send_cb(); + + esp_err_t err = esp_now_deinit(); + if (err != ESP_OK) { + ESP_LOGE(TAG, "esp_now_deinit failed! 0x%x", err); + } +} + +void ESPNowComponent::apply_wifi_channel() { + if (this->state_ == ESPNOW_STATE_DISABLED) { + ESP_LOGE(TAG, "Cannot set channel when ESPNOW disabled"); + this->mark_failed(); + return; + } + + if (this->is_wifi_enabled()) { + ESP_LOGE(TAG, "Cannot set channel when Wi-Fi enabled"); + this->mark_failed(); + return; + } + + ESP_LOGI(TAG, "Channel set to %d.", this->wifi_channel_); + esp_wifi_set_promiscuous(true); + esp_wifi_set_channel(this->wifi_channel_, WIFI_SECOND_CHAN_NONE); + esp_wifi_set_promiscuous(false); +} + +void ESPNowComponent::loop() { +#ifdef USE_WIFI + if (wifi::global_wifi_component != nullptr && wifi::global_wifi_component->is_connected()) { + int32_t new_channel = wifi::global_wifi_component->get_wifi_channel(); + if (new_channel != this->wifi_channel_) { + ESP_LOGI(TAG, "Wifi Channel is changed from %d to %d.", this->wifi_channel_, new_channel); + this->wifi_channel_ = new_channel; + } + } +#endif + // Process received packets + ESPNowPacket *packet = this->receive_packet_queue_.pop(); + while (packet != nullptr) { + switch (packet->type_) { + case ESPNowPacket::RECEIVED: { + const ESPNowRecvInfo info = packet->get_receive_info(); + if (!esp_now_is_peer_exist(info.src_addr)) { + bool handled = false; + for (auto *handler : this->unknown_peer_handlers_) { + if (handler->on_unknown_peer(info, packet->packet_.receive.data, packet->packet_.receive.size)) { + handled = true; + break; // If a handler returns true, stop processing further handlers + } + } + if (!handled && this->auto_add_peer_) { + this->add_peer(info.src_addr); + } + } + // Intentionally left as if instead of else in case the peer is added above + if (esp_now_is_peer_exist(info.src_addr)) { +#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE + ESP_LOGV(TAG, "<<< [%s -> %s] %s", format_mac_address_pretty(info.src_addr).c_str(), + format_mac_address_pretty(info.des_addr).c_str(), + format_hex_pretty(packet->packet_.receive.data, packet->packet_.receive.size).c_str()); +#endif + if (memcmp(info.des_addr, ESPNOW_BROADCAST_ADDR, ESP_NOW_ETH_ALEN) == 0) { + for (auto *handler : this->broadcasted_handlers_) { + if (handler->on_broadcasted(info, packet->packet_.receive.data, packet->packet_.receive.size)) + break; // If a handler returns true, stop processing further handlers + } + } else { + for (auto *handler : this->received_handlers_) { + if (handler->on_received(info, packet->packet_.receive.data, packet->packet_.receive.size)) + break; // If a handler returns true, stop processing further handlers + } + } + } + break; + } + case ESPNowPacket::SENT: { +#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE + ESP_LOGV(TAG, ">>> [%s] %s", format_mac_address_pretty(packet->packet_.sent.address).c_str(), + LOG_STR_ARG(espnow_error_to_str(packet->packet_.sent.status))); +#endif + if (this->current_send_packet_ != nullptr) { + this->current_send_packet_->callback_(packet->packet_.sent.status); + this->send_packet_pool_.release(this->current_send_packet_); + this->current_send_packet_ = nullptr; // Reset current packet after sending + } + break; + } + default: + break; + } + // Return the packet to the pool + this->receive_packet_pool_.release(packet); + packet = this->receive_packet_queue_.pop(); + } + + // Process sending packet queue + if (this->current_send_packet_ == nullptr) { + this->send_(); + } + + // Log dropped received packets periodically + uint16_t received_dropped = this->receive_packet_queue_.get_and_reset_dropped_count(); + if (received_dropped > 0) { + ESP_LOGW(TAG, "Dropped %u received packets due to buffer overflow", received_dropped); + } + + // Log dropped send packets periodically + uint16_t send_dropped = this->send_packet_queue_.get_and_reset_dropped_count(); + if (send_dropped > 0) { + ESP_LOGW(TAG, "Dropped %u send packets due to buffer overflow", send_dropped); + } +} + +uint8_t ESPNowComponent::get_wifi_channel() { + wifi_second_chan_t dummy; + esp_wifi_get_channel(&this->wifi_channel_, &dummy); + return this->wifi_channel_; +} + +esp_err_t ESPNowComponent::send(const uint8_t *peer_address, const uint8_t *payload, size_t size, + const send_callback_t &callback) { + if (this->state_ != ESPNOW_STATE_ENABLED) { + return ESP_ERR_ESPNOW_NOT_INIT; + } else if (this->is_failed()) { + return ESP_ERR_ESPNOW_FAILED; + } else if (peer_address == 0ULL) { + return ESP_ERR_ESPNOW_PEER_NOT_SET; + } else if (memcmp(peer_address, this->own_address_, ESP_NOW_ETH_ALEN) == 0) { + return ESP_ERR_ESPNOW_OWN_ADDRESS; + } else if (size > ESP_NOW_MAX_DATA_LEN) { + return ESP_ERR_ESPNOW_DATA_SIZE; + } else if (!esp_now_is_peer_exist(peer_address)) { + if (memcmp(peer_address, ESPNOW_BROADCAST_ADDR, ESP_NOW_ETH_ALEN) == 0 || this->auto_add_peer_) { + esp_err_t err = this->add_peer(peer_address); + if (err != ESP_OK) { + return err; + } + } else { + return ESP_ERR_ESPNOW_PEER_NOT_PAIRED; + } + } + // Allocate a packet from the pool + ESPNowSendPacket *packet = this->send_packet_pool_.allocate(); + if (packet == nullptr) { + this->send_packet_queue_.increment_dropped_count(); + ESP_LOGE(TAG, "Failed to allocate send packet from pool"); + this->status_momentary_warning("send-packet-pool-full"); + return ESP_ERR_ESPNOW_NO_MEM; + } + // Load the packet data + packet->load_data(peer_address, payload, size, callback); + // Push the packet to the send queue + this->send_packet_queue_.push(packet); + return ESP_OK; +} + +void ESPNowComponent::send_() { + ESPNowSendPacket *packet = this->send_packet_queue_.pop(); + if (packet == nullptr) { + return; // No packets to send + } + + this->current_send_packet_ = packet; + esp_err_t err = esp_now_send(packet->address_, packet->data_, packet->size_); + if (err != ESP_OK) { + ESP_LOGE(TAG, "Failed to send packet to %s - %s", format_mac_address_pretty(packet->address_).c_str(), + LOG_STR_ARG(espnow_error_to_str(err))); + if (packet->callback_ != nullptr) { + packet->callback_(err); + } + this->status_momentary_warning("send-failed"); + this->send_packet_pool_.release(packet); + this->current_send_packet_ = nullptr; // Reset current packet + return; + } +} + +esp_err_t ESPNowComponent::add_peer(const uint8_t *peer) { + if (this->state_ != ESPNOW_STATE_ENABLED || this->is_failed()) { + return ESP_ERR_ESPNOW_NOT_INIT; + } + + if (memcmp(peer, this->own_address_, ESP_NOW_ETH_ALEN) == 0) { + this->status_momentary_warning("peer-add-failed"); + return ESP_ERR_INVALID_MAC; + } + + if (!esp_now_is_peer_exist(peer)) { + esp_now_peer_info_t peer_info = {}; + memset(&peer_info, 0, sizeof(esp_now_peer_info_t)); + peer_info.ifidx = WIFI_IF_STA; + memcpy(peer_info.peer_addr, peer, ESP_NOW_ETH_ALEN); + esp_err_t err = esp_now_add_peer(&peer_info); + + if (err != ESP_OK) { + ESP_LOGE(TAG, "Failed to add peer %s - %s", format_mac_address_pretty(peer).c_str(), + LOG_STR_ARG(espnow_error_to_str(err))); + this->status_momentary_warning("peer-add-failed"); + return err; + } + } + bool found = false; + for (auto &it : this->peers_) { + if (it == peer) { + found = true; + break; + } + } + if (!found) { + ESPNowPeer new_peer; + memcpy(new_peer.address, peer, ESP_NOW_ETH_ALEN); + this->peers_.push_back(new_peer); + } + + return ESP_OK; +} + +esp_err_t ESPNowComponent::del_peer(const uint8_t *peer) { + if (this->state_ != ESPNOW_STATE_ENABLED || this->is_failed()) { + return ESP_ERR_ESPNOW_NOT_INIT; + } + if (esp_now_is_peer_exist(peer)) { + esp_err_t err = esp_now_del_peer(peer); + if (err != ESP_OK) { + ESP_LOGE(TAG, "Failed to delete peer %s - %s", format_mac_address_pretty(peer).c_str(), + LOG_STR_ARG(espnow_error_to_str(err))); + this->status_momentary_warning("peer-del-failed"); + return err; + } + } + for (auto it = this->peers_.begin(); it != this->peers_.end(); ++it) { + if (*it == peer) { + this->peers_.erase(it); + break; + } + } + return ESP_OK; +} + +} // namespace esphome::espnow + +#endif // USE_ESP32 diff --git a/esphome/components/espnow/espnow_component.h b/esphome/components/espnow/espnow_component.h new file mode 100644 index 0000000000..9941e97227 --- /dev/null +++ b/esphome/components/espnow/espnow_component.h @@ -0,0 +1,183 @@ +#pragma once + +#include "esphome/core/automation.h" +#include "esphome/core/component.h" + +#ifdef USE_ESP32 + +#include "esphome/core/event_pool.h" +#include "esphome/core/lock_free_queue.h" +#include "espnow_packet.h" + +#include + +#include +#include + +#include +#include +#include +#include +#include + +namespace esphome::espnow { + +// Maximum size of the ESPNow event queue - must be power of 2 for lock-free queue +static constexpr size_t MAX_ESP_NOW_SEND_QUEUE_SIZE = 16; +static constexpr size_t MAX_ESP_NOW_RECEIVE_QUEUE_SIZE = 16; + +using peer_address_t = std::array; + +enum class ESPNowTriggers : uint8_t { + TRIGGER_NONE = 0, + ON_NEW_PEER = 1, + ON_RECEIVED = 2, + ON_BROADCASTED = 3, + ON_SUCCEED = 10, + ON_FAILED = 11, +}; + +enum ESPNowState : uint8_t { + /** Nothing has been initialized yet. */ + ESPNOW_STATE_OFF = 0, + /** ESPNOW is disabled. */ + ESPNOW_STATE_DISABLED, + /** ESPNOW is enabled. */ + ESPNOW_STATE_ENABLED, +}; + +struct ESPNowPeer { + uint8_t address[ESP_NOW_ETH_ALEN]; // MAC address of the peer + + bool operator==(const ESPNowPeer &other) const { return memcmp(this->address, other.address, ESP_NOW_ETH_ALEN) == 0; } + bool operator==(const uint8_t *other) const { return memcmp(this->address, other, ESP_NOW_ETH_ALEN) == 0; } +}; + +/// Handler interface for receiving ESPNow packets from unknown peers +/// Components should inherit from this class to handle incoming ESPNow data +class ESPNowUnknownPeerHandler { + public: + /// Called when an ESPNow packet is received from an unknown peer + /// @param info Information about the received packet (sender MAC, etc.) + /// @param data Pointer to the received data payload + /// @param size Size of the received data in bytes + /// @return true if the packet was handled, false otherwise + virtual bool on_unknown_peer(const ESPNowRecvInfo &info, const uint8_t *data, uint8_t size) = 0; +}; + +/// Handler interface for receiving ESPNow packets +/// Components should inherit from this class to handle incoming ESPNow data +class ESPNowReceivedPacketHandler { + public: + /// Called when an ESPNow packet is received + /// @param info Information about the received packet (sender MAC, etc.) + /// @param data Pointer to the received data payload + /// @param size Size of the received data in bytes + /// @return true if the packet was handled, false otherwise + virtual bool on_received(const ESPNowRecvInfo &info, const uint8_t *data, uint8_t size) = 0; +}; +/// Handler interface for receiving broadcasted ESPNow packets +/// Components should inherit from this class to handle incoming ESPNow data +class ESPNowBroadcastedHandler { + public: + /// Called when a broadcasted ESPNow packet is received + /// @param info Information about the received packet (sender MAC, etc.) + /// @param data Pointer to the received data payload + /// @param size Size of the received data in bytes + /// @return true if the packet was handled, false otherwise + virtual bool on_broadcasted(const ESPNowRecvInfo &info, const uint8_t *data, uint8_t size) = 0; +}; + +class ESPNowComponent : public Component { + public: + ESPNowComponent(); + void setup() override; + void loop() override; + void dump_config() override; + + float get_setup_priority() const override { return setup_priority::LATE; } + + // Add a peer to the internal list of peers + void add_peer(peer_address_t address) { + ESPNowPeer peer; + memcpy(peer.address, address.data(), ESP_NOW_ETH_ALEN); + this->peers_.push_back(peer); + } + // Add a peer with the esp_now api and add to the internal list if doesnt exist already + esp_err_t add_peer(const uint8_t *peer); + // Remove a peer with the esp_now api and remove from the internal list if exists + esp_err_t del_peer(const uint8_t *peer); + + void set_wifi_channel(uint8_t channel) { this->wifi_channel_ = channel; } + void apply_wifi_channel(); + uint8_t get_wifi_channel(); + + void set_auto_add_peer(bool value) { this->auto_add_peer_ = value; } + + void enable(); + void disable(); + bool is_disabled() const { return this->state_ == ESPNOW_STATE_DISABLED; }; + void set_enable_on_boot(bool enable_on_boot) { this->enable_on_boot_ = enable_on_boot; } + bool is_wifi_enabled(); + + /// @brief Queue a packet to be sent to a specific peer address. + /// This method will add the packet to the internal queue and + /// call the callback when the packet is sent. + /// Only one packet will be sent at any given time and the next one will not be sent until + /// the previous one has been acknowledged or failed. + /// @param peer_address MAC address of the peer to send the packet to + /// @param payload Data payload to send + /// @param callback Callback to call when the send operation is complete + /// @return ESP_OK on success, or an error code on failure + esp_err_t send(const uint8_t *peer_address, const std::vector &payload, + const send_callback_t &callback = nullptr) { + return this->send(peer_address, payload.data(), payload.size(), callback); + } + esp_err_t send(const uint8_t *peer_address, const uint8_t *payload, size_t size, + const send_callback_t &callback = nullptr); + + void register_received_handler(ESPNowReceivedPacketHandler *handler) { this->received_handlers_.push_back(handler); } + void register_unknown_peer_handler(ESPNowUnknownPeerHandler *handler) { + this->unknown_peer_handlers_.push_back(handler); + } + void register_broadcasted_handler(ESPNowBroadcastedHandler *handler) { + this->broadcasted_handlers_.push_back(handler); + } + + protected: + friend void on_data_received(const esp_now_recv_info_t *info, const uint8_t *data, int size); +#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 5, 0) + friend void on_send_report(const esp_now_send_info_t *info, esp_now_send_status_t status); +#else + friend void on_send_report(const uint8_t *mac_addr, esp_now_send_status_t status); +#endif + + void enable_(); + void send_(); + + std::vector unknown_peer_handlers_; + std::vector received_handlers_; + std::vector broadcasted_handlers_; + + std::vector peers_{}; + + uint8_t own_address_[ESP_NOW_ETH_ALEN]{0}; + LockFreeQueue receive_packet_queue_{}; + EventPool receive_packet_pool_{}; + + LockFreeQueue send_packet_queue_{}; + EventPool send_packet_pool_{}; + ESPNowSendPacket *current_send_packet_{nullptr}; // Currently sending packet, nullptr if none + + uint8_t wifi_channel_{0}; + ESPNowState state_{ESPNOW_STATE_OFF}; + + bool auto_add_peer_{false}; + bool enable_on_boot_{true}; +}; + +extern ESPNowComponent *global_esp_now; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) + +} // namespace esphome::espnow + +#endif // USE_ESP32 diff --git a/esphome/components/espnow/espnow_err.h b/esphome/components/espnow/espnow_err.h new file mode 100644 index 0000000000..ceda1b7683 --- /dev/null +++ b/esphome/components/espnow/espnow_err.h @@ -0,0 +1,19 @@ +#pragma once + +#ifdef USE_ESP32 + +#include +#include + +namespace esphome::espnow { + +static const esp_err_t ESP_ERR_ESPNOW_CMP_BASE = (ESP_ERR_ESPNOW_BASE + 20); +static const esp_err_t ESP_ERR_ESPNOW_FAILED = (ESP_ERR_ESPNOW_CMP_BASE + 1); +static const esp_err_t ESP_ERR_ESPNOW_OWN_ADDRESS = (ESP_ERR_ESPNOW_CMP_BASE + 2); +static const esp_err_t ESP_ERR_ESPNOW_DATA_SIZE = (ESP_ERR_ESPNOW_CMP_BASE + 3); +static const esp_err_t ESP_ERR_ESPNOW_PEER_NOT_SET = (ESP_ERR_ESPNOW_CMP_BASE + 4); +static const esp_err_t ESP_ERR_ESPNOW_PEER_NOT_PAIRED = (ESP_ERR_ESPNOW_CMP_BASE + 5); + +} // namespace esphome::espnow + +#endif // USE_ESP32 diff --git a/esphome/components/espnow/espnow_packet.h b/esphome/components/espnow/espnow_packet.h new file mode 100644 index 0000000000..b6192a0d41 --- /dev/null +++ b/esphome/components/espnow/espnow_packet.h @@ -0,0 +1,166 @@ +#pragma once + +#ifdef USE_ESP32 + +#include "espnow_err.h" + +#include +#include +#include +#include +#include + +#include +#include +#include + +namespace esphome::espnow { + +static const uint8_t ESPNOW_BROADCAST_ADDR[ESP_NOW_ETH_ALEN] = {0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF}; +static const uint8_t ESPNOW_MULTICAST_ADDR[ESP_NOW_ETH_ALEN] = {0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFE}; + +struct WifiPacketRxControl { + int8_t rssi; // Received Signal Strength Indicator (RSSI) of packet, unit: dBm + uint32_t timestamp; // Timestamp in microseconds when the packet was received, precise only if modem sleep or + // light sleep is not enabled +}; + +struct ESPNowRecvInfo { + uint8_t src_addr[ESP_NOW_ETH_ALEN]; /**< Source address of ESPNOW packet */ + uint8_t des_addr[ESP_NOW_ETH_ALEN]; /**< Destination address of ESPNOW packet */ + wifi_pkt_rx_ctrl_t *rx_ctrl; /**< Rx control info of ESPNOW packet */ +}; + +using send_callback_t = std::function; + +class ESPNowPacket { + public: + // NOLINTNEXTLINE(readability-identifier-naming) + enum esp_now_packet_type_t : uint8_t { + RECEIVED, + SENT, + }; + + // Constructor for received data + ESPNowPacket(const esp_now_recv_info_t *info, const uint8_t *data, int size) { + this->init_received_data_(info, data, size); + }; + +#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 5, 0) + // Constructor for sent data + ESPNowPacket(const esp_now_send_info_t *info, esp_now_send_status_t status) { + this->init_sent_data_(info->src_addr, status); + } +#else + // Constructor for sent data + ESPNowPacket(const uint8_t *mac_addr, esp_now_send_status_t status) { this->init_sent_data_(mac_addr, status); } +#endif + + // Default constructor for pre-allocation in pool + ESPNowPacket() {} + + void release() {} + + void load_received_data(const esp_now_recv_info_t *info, const uint8_t *data, int size) { + this->type_ = RECEIVED; + this->init_received_data_(info, data, size); + } + + void load_sent_data(const uint8_t *mac_addr, esp_now_send_status_t status) { + this->type_ = SENT; + this->init_sent_data_(mac_addr, status); + } + + // Disable copy to prevent double-delete + ESPNowPacket(const ESPNowPacket &) = delete; + ESPNowPacket &operator=(const ESPNowPacket &) = delete; + + union { + // NOLINTNEXTLINE(readability-identifier-naming) + struct received_data { + ESPNowRecvInfo info; // Information about the received packet + uint8_t data[ESP_NOW_MAX_DATA_LEN]; // Data received in the packet + uint8_t size; // Size of the received data + WifiPacketRxControl rx_ctrl; // Status of the received packet + } receive; + + // NOLINTNEXTLINE(readability-identifier-naming) + struct sent_data { + uint8_t address[ESP_NOW_ETH_ALEN]; + esp_now_send_status_t status; + } sent; + } packet_; + + esp_now_packet_type_t type_; + + esp_now_packet_type_t type() const { return this->type_; } + const ESPNowRecvInfo &get_receive_info() const { return this->packet_.receive.info; } + + private: + void init_received_data_(const esp_now_recv_info_t *info, const uint8_t *data, int size) { + memcpy(this->packet_.receive.info.src_addr, info->src_addr, ESP_NOW_ETH_ALEN); + memcpy(this->packet_.receive.info.des_addr, info->des_addr, ESP_NOW_ETH_ALEN); + memcpy(this->packet_.receive.data, data, size); + this->packet_.receive.size = size; + + this->packet_.receive.rx_ctrl.rssi = info->rx_ctrl->rssi; + this->packet_.receive.rx_ctrl.timestamp = info->rx_ctrl->timestamp; + + this->packet_.receive.info.rx_ctrl = reinterpret_cast(&this->packet_.receive.rx_ctrl); + } + + void init_sent_data_(const uint8_t *mac_addr, esp_now_send_status_t status) { + memcpy(this->packet_.sent.address, mac_addr, ESP_NOW_ETH_ALEN); + this->packet_.sent.status = status; + } +}; + +class ESPNowSendPacket { + public: + ESPNowSendPacket(const uint8_t *peer_address, const uint8_t *payload, size_t size, const send_callback_t &&callback) + : callback_(callback) { + this->init_data_(peer_address, payload, size); + } + ESPNowSendPacket(const uint8_t *peer_address, const uint8_t *payload, size_t size) { + this->init_data_(peer_address, payload, size); + } + + // Default constructor for pre-allocation in pool + ESPNowSendPacket() {} + + void release() {} + + // Disable copy to prevent double-delete + ESPNowSendPacket(const ESPNowSendPacket &) = delete; + ESPNowSendPacket &operator=(const ESPNowSendPacket &) = delete; + + void load_data(const uint8_t *peer_address, const uint8_t *payload, size_t size, const send_callback_t &callback) { + this->init_data_(peer_address, payload, size); + this->callback_ = callback; + } + + void load_data(const uint8_t *peer_address, const uint8_t *payload, size_t size) { + this->init_data_(peer_address, payload, size); + this->callback_ = nullptr; // Reset callback + } + + uint8_t address_[ESP_NOW_ETH_ALEN]{0}; // MAC address of the peer to send the packet to + uint8_t data_[ESP_NOW_MAX_DATA_LEN]{0}; // Data to send + uint8_t size_{0}; // Size of the data to send, must be <= ESP_NOW_MAX_DATA_LEN + send_callback_t callback_{nullptr}; // Callback to call when the send operation is complete + + private: + void init_data_(const uint8_t *peer_address, const uint8_t *payload, size_t size) { + memcpy(this->address_, peer_address, ESP_NOW_ETH_ALEN); + if (size > ESP_NOW_MAX_DATA_LEN) { + this->size_ = 0; + return; + } + this->size_ = size; + memcpy(this->data_, payload, this->size_); + } +}; + +} // namespace esphome::espnow + +#endif // USE_ESP32 diff --git a/esphome/components/ethernet/__init__.py b/esphome/components/ethernet/__init__.py index 619346b914..7384bb26d3 100644 --- a/esphome/components/ethernet/__init__.py +++ b/esphome/components/ethernet/__init__.py @@ -2,9 +2,15 @@ import logging from esphome import pins import esphome.codegen as cg -from esphome.components.esp32 import add_idf_sdkconfig_option, get_esp32_variant +from esphome.components.esp32 import ( + add_idf_component, + add_idf_sdkconfig_option, + get_esp32_variant, +) from esphome.components.esp32.const import ( + VARIANT_ESP32, VARIANT_ESP32C3, + VARIANT_ESP32P4, VARIANT_ESP32S2, VARIANT_ESP32S3, ) @@ -21,6 +27,7 @@ from esphome.const import ( CONF_GATEWAY, CONF_ID, CONF_INTERRUPT_PIN, + CONF_MAC_ADDRESS, CONF_MANUAL_IP, CONF_MISO_PIN, CONF_MODE, @@ -38,7 +45,12 @@ from esphome.const import ( KEY_CORE, KEY_FRAMEWORK_VERSION, ) -from esphome.core import CORE, TimePeriodMilliseconds, coroutine_with_priority +from esphome.core import ( + CORE, + CoroPriority, + TimePeriodMilliseconds, + coroutine_with_priority, +) import esphome.final_validate as fv CONFLICTS_WITH = ["wifi"] @@ -70,6 +82,15 @@ ETHERNET_TYPES = { "W5500": EthernetType.ETHERNET_TYPE_W5500, "OPENETH": EthernetType.ETHERNET_TYPE_OPENETH, "DM9051": EthernetType.ETHERNET_TYPE_DM9051, + "LAN8670": EthernetType.ETHERNET_TYPE_LAN8670, +} + +# PHY types that need compile-time defines for conditional compilation +_PHY_TYPE_TO_DEFINE = { + "KSZ8081": "USE_ETHERNET_KSZ8081", + "KSZ8081RNA": "USE_ETHERNET_KSZ8081", + "LAN8670": "USE_ETHERNET_LAN8670", + # Add other PHY types here only if they need conditional compilation } SPI_ETHERNET_TYPES = ["W5500", "DM9051"] @@ -105,19 +126,15 @@ ManualIP = ethernet_ns.struct("ManualIP") def _is_framework_spi_polling_mode_supported(): # SPI Ethernet without IRQ feature is added in - # esp-idf >= (5.3+ ,5.2.1+, 5.1.4) and arduino-esp32 >= 3.0.0 + # esp-idf >= (5.3+ ,5.2.1+, 5.1.4) + # Note: Arduino now uses ESP-IDF as a component, so we only check IDF version framework_version = CORE.data[KEY_CORE][KEY_FRAMEWORK_VERSION] - if CORE.using_esp_idf: - if framework_version >= cv.Version(5, 3, 0): - return True - if cv.Version(5, 3, 0) > framework_version >= cv.Version(5, 2, 1): - return True - if cv.Version(5, 2, 0) > framework_version >= cv.Version(5, 1, 4): - return True - return False - if CORE.using_arduino: - return framework_version >= cv.Version(3, 0, 0) - # fail safe: Unknown framework + if framework_version >= cv.Version(5, 3, 0): + return True + if cv.Version(5, 3, 0) > framework_version >= cv.Version(5, 2, 1): + return True + if cv.Version(5, 2, 0) > framework_version >= cv.Version(5, 1, 4): # noqa: SIM103 + return True return False @@ -128,6 +145,7 @@ def _validate(config): else: use_address = CORE.name + config[CONF_DOMAIN] config[CONF_USE_ADDRESS] = use_address + if config[CONF_TYPE] in SPI_ETHERNET_TYPES: if _is_framework_spi_polling_mode_supported(): if CONF_POLLING_INTERVAL in config and CONF_INTERRUPT_PIN in config: @@ -160,6 +178,12 @@ def _validate(config): del config[CONF_CLK_MODE] elif CONF_CLK not in config: raise cv.Invalid("'clk' is a required option for [ethernet].") + variant = get_esp32_variant() + if variant not in (VARIANT_ESP32, VARIANT_ESP32P4): + raise cv.Invalid( + f"{config[CONF_TYPE]} PHY requires RMII interface and is only supported " + f"on ESP32 classic and ESP32-P4, not {variant}" + ) return config @@ -174,6 +198,7 @@ BASE_SCHEMA = cv.Schema( "This option has been removed. Please use the [disabled] option under the " "new mdns component instead." ), + cv.Optional(CONF_MAC_ADDRESS): cv.mac_address, } ).extend(cv.COMPONENT_SCHEMA) @@ -240,6 +265,7 @@ CONFIG_SCHEMA = cv.All( "W5500": SPI_SCHEMA, "OPENETH": BASE_SCHEMA, "DM9051": SPI_SCHEMA, + "LAN8670": RMII_SCHEMA, }, upper=True, ), @@ -289,7 +315,7 @@ def phy_register(address: int, value: int, page: int): ) -@coroutine_with_priority(60.0) +@coroutine_with_priority(CoroPriority.COMMUNICATION) async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) @@ -310,11 +336,8 @@ async def to_code(config): cg.add(var.set_clock_speed(config[CONF_CLOCK_SPEED])) cg.add_define("USE_ETHERNET_SPI") - if CORE.using_esp_idf: - add_idf_sdkconfig_option("CONFIG_ETH_USE_SPI_ETHERNET", True) - add_idf_sdkconfig_option( - f"CONFIG_ETH_SPI_ETHERNET_{config[CONF_TYPE]}", True - ) + add_idf_sdkconfig_option("CONFIG_ETH_USE_SPI_ETHERNET", True) + add_idf_sdkconfig_option(f"CONFIG_ETH_SPI_ETHERNET_{config[CONF_TYPE]}", True) elif config[CONF_TYPE] == "OPENETH": cg.add_define("USE_ETHERNET_OPENETH") add_idf_sdkconfig_option("CONFIG_ETH_USE_OPENETH", True) @@ -340,13 +363,23 @@ async def to_code(config): if CONF_MANUAL_IP in config: cg.add(var.set_manual_ip(manual_ip(config[CONF_MANUAL_IP]))) + # Add compile-time define for PHY types with specific code + if phy_define := _PHY_TYPE_TO_DEFINE.get(config[CONF_TYPE]): + cg.add_define(phy_define) + + if mac_address := config.get(CONF_MAC_ADDRESS): + cg.add(var.set_fixed_mac(mac_address.parts)) + cg.add_define("USE_ETHERNET") # Disable WiFi when using Ethernet to save memory - if CORE.using_esp_idf: - add_idf_sdkconfig_option("CONFIG_ESP_WIFI_ENABLED", False) - # Also disable WiFi/BT coexistence since WiFi is disabled - add_idf_sdkconfig_option("CONFIG_SW_COEXIST_ENABLE", False) + add_idf_sdkconfig_option("CONFIG_ESP_WIFI_ENABLED", False) + # Also disable WiFi/BT coexistence since WiFi is disabled + add_idf_sdkconfig_option("CONFIG_SW_COEXIST_ENABLE", False) + + if config[CONF_TYPE] == "LAN8670": + # Add LAN867x 10BASE-T1S PHY support component + add_idf_component(name="espressif/lan867x", ref="2.0.0") if CORE.using_arduino: cg.add_library("WiFi", None) diff --git a/esphome/components/ethernet/esp_eth_phy_jl1101.c b/esphome/components/ethernet/esp_eth_phy_jl1101.c index 4f31e0a9fd..5e73e99101 100644 --- a/esphome/components/ethernet/esp_eth_phy_jl1101.c +++ b/esphome/components/ethernet/esp_eth_phy_jl1101.c @@ -25,6 +25,9 @@ #include "driver/gpio.h" #include "esp_rom_gpio.h" #include "esp_rom_sys.h" +#include "esp_idf_version.h" + +#if defined(USE_ARDUINO) || ESP_IDF_VERSION < ESP_IDF_VERSION_VAL(5, 4, 2) static const char *TAG = "jl1101"; #define PHY_CHECK(a, str, goto_tag, ...) \ @@ -336,4 +339,6 @@ esp_eth_phy_t *esp_eth_phy_new_jl1101(const eth_phy_config_t *config) { err: return NULL; } + +#endif /* USE_ARDUINO */ #endif /* USE_ESP32 */ diff --git a/esphome/components/ethernet/ethernet_component.cpp b/esphome/components/ethernet/ethernet_component.cpp index f8c2f3a72e..24b6e8154b 100644 --- a/esphome/components/ethernet/ethernet_component.cpp +++ b/esphome/components/ethernet/ethernet_component.cpp @@ -9,6 +9,10 @@ #include #include "esp_event.h" +#ifdef USE_ETHERNET_LAN8670 +#include "esp_eth_phy_lan867x.h" +#endif + #ifdef USE_ETHERNET_SPI #include #include @@ -37,24 +41,26 @@ static const char *const TAG = "ethernet"; EthernetComponent *global_eth_component; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) +void EthernetComponent::log_error_and_mark_failed_(esp_err_t err, const char *message) { + ESP_LOGE(TAG, "%s: (%d) %s", message, err, esp_err_to_name(err)); + this->mark_failed(); +} + #define ESPHL_ERROR_CHECK(err, message) \ if ((err) != ESP_OK) { \ - ESP_LOGE(TAG, message ": (%d) %s", err, esp_err_to_name(err)); \ - this->mark_failed(); \ + this->log_error_and_mark_failed_(err, message); \ return; \ } #define ESPHL_ERROR_CHECK_RET(err, message, ret) \ if ((err) != ESP_OK) { \ - ESP_LOGE(TAG, message ": (%d) %s", err, esp_err_to_name(err)); \ - this->mark_failed(); \ + this->log_error_and_mark_failed_(err, message); \ return ret; \ } EthernetComponent::EthernetComponent() { global_eth_component = this; } void EthernetComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (esp_reset_reason() != ESP_RST_DEEPSLEEP) { // Delay here to allow power to stabilise before Ethernet is initialized. delay(300); // NOLINT @@ -201,6 +207,12 @@ void EthernetComponent::setup() { this->phy_ = esp_eth_phy_new_ksz80xx(&phy_config); break; } +#ifdef USE_ETHERNET_LAN8670 + case ETHERNET_TYPE_LAN8670: { + this->phy_ = esp_eth_phy_new_lan867x(&phy_config); + break; + } +#endif #endif #ifdef USE_ETHERNET_SPI #if CONFIG_ETH_SPI_ETHERNET_W5500 @@ -230,10 +242,12 @@ void EthernetComponent::setup() { ESPHL_ERROR_CHECK(err, "ETH driver install error"); #ifndef USE_ETHERNET_SPI +#ifdef USE_ETHERNET_KSZ8081 if (this->type_ == ETHERNET_TYPE_KSZ8081RNA && this->clk_mode_ == EMAC_CLK_OUT) { // KSZ8081RNA default is incorrect. It expects a 25MHz clock instead of the 50MHz we provide. this->ksz8081_set_clock_reference_(mac); } +#endif // USE_ETHERNET_KSZ8081 for (const auto &phy_register : this->phy_registers_) { this->write_phy_register_(mac, phy_register); @@ -242,7 +256,11 @@ void EthernetComponent::setup() { // use ESP internal eth mac uint8_t mac_addr[6]; - esp_read_mac(mac_addr, ESP_MAC_ETH); + if (this->fixed_mac_.has_value()) { + memcpy(mac_addr, this->fixed_mac_->data(), 6); + } else { + esp_read_mac(mac_addr, ESP_MAC_ETH); + } err = esp_eth_ioctl(this->eth_handle_, ETH_CMD_S_MAC_ADDR, mac_addr); ESPHL_ERROR_CHECK(err, "set mac address error"); @@ -301,6 +319,7 @@ void EthernetComponent::loop() { this->state_ = EthernetComponentState::CONNECTING; this->start_connect_(); } else { + this->finish_connect_(); // When connected and stable, disable the loop to save CPU cycles this->disable_loop(); } @@ -351,6 +370,12 @@ void EthernetComponent::dump_config() { eth_type = "DM9051"; break; +#ifdef USE_ETHERNET_LAN8670 + case ETHERNET_TYPE_LAN8670: + eth_type = "LAN8670"; + break; +#endif + default: eth_type = "Unknown"; break; @@ -420,6 +445,7 @@ network::IPAddresses EthernetComponent::get_ip_addresses() { } network::IPAddress EthernetComponent::get_dns_address(uint8_t num) { + LwIPLock lock; const ip_addr_t *dns_ip = dns_getserver(num); return dns_ip; } @@ -486,13 +512,38 @@ void EthernetComponent::got_ip6_event_handler(void *arg, esp_event_base_t event_ } #endif /* USE_NETWORK_IPV6 */ +void EthernetComponent::finish_connect_() { +#if USE_NETWORK_IPV6 + // Retry IPv6 link-local setup if it failed during initial connect + // This handles the case where min_ipv6_addr_count is NOT set (or is 0), + // allowing us to reach CONNECTED state with just IPv4. + // If IPv6 setup failed in start_connect_() because the interface wasn't ready: + // - Bootup timing issues (#10281) + // - Cable unplugged/network interruption (#10705) + // We can now retry since we're in CONNECTED state and the interface is definitely up. + if (!this->ipv6_setup_done_) { + esp_err_t err = esp_netif_create_ip6_linklocal(this->eth_netif_); + if (err == ESP_OK) { + ESP_LOGD(TAG, "IPv6 link-local address created (retry succeeded)"); + } + // Always set the flag to prevent continuous retries + // If IPv6 setup fails here with the interface up and stable, it's + // likely a persistent issue (IPv6 disabled at router, hardware + // limitation, etc.) that won't be resolved by further retries. + // The device continues to work with IPv4. + this->ipv6_setup_done_ = true; + } +#endif /* USE_NETWORK_IPV6 */ +} + void EthernetComponent::start_connect_() { global_eth_component->got_ipv4_address_ = false; #if USE_NETWORK_IPV6 global_eth_component->ipv6_count_ = 0; + this->ipv6_setup_done_ = false; #endif /* USE_NETWORK_IPV6 */ this->connect_begin_ = millis(); - this->status_set_warning("waiting for IP configuration"); + this->status_set_warning(LOG_STR("waiting for IP configuration")); esp_err_t err; err = esp_netif_set_hostname(this->eth_netif_, App.get_name().c_str()); @@ -527,6 +578,7 @@ void EthernetComponent::start_connect_() { ESPHL_ERROR_CHECK(err, "DHCPC set IP info error"); if (this->manual_ip_.has_value()) { + LwIPLock lock; if (this->manual_ip_->dns1.is_set()) { ip_addr_t d; d = this->manual_ip_->dns1; @@ -544,9 +596,27 @@ void EthernetComponent::start_connect_() { } } #if USE_NETWORK_IPV6 + // Attempt to create IPv6 link-local address + // We MUST attempt this here, not just in finish_connect_(), because with + // min_ipv6_addr_count set, the component won't reach CONNECTED state without IPv6. + // However, this may fail with ESP_FAIL if the interface is not up yet: + // - At bootup when link isn't ready (#10281) + // - After disconnection/cable unplugged (#10705) + // We'll retry in finish_connect_() if it fails here. err = esp_netif_create_ip6_linklocal(this->eth_netif_); if (err != ESP_OK) { - ESPHL_ERROR_CHECK(err, "Enable IPv6 link local failed"); + if (err == ESP_ERR_ESP_NETIF_INVALID_PARAMS) { + // This is a programming error, not a transient failure + ESPHL_ERROR_CHECK(err, "esp_netif_create_ip6_linklocal invalid parameters"); + } else { + // ESP_FAIL means the interface isn't up yet + // This is expected and non-fatal, happens in multiple scenarios: + // - During reconnection after network interruptions (#10705) + // - At bootup when the link isn't ready yet (#10281) + // We'll retry once we reach CONNECTED state and the interface is up + ESP_LOGW(TAG, "esp_netif_create_ip6_linklocal failed: %s", esp_err_to_name(err)); + // Don't mark component as failed - this is a transient error + } } #endif /* USE_NETWORK_IPV6 */ @@ -559,8 +629,13 @@ bool EthernetComponent::is_connected() { return this->state_ == EthernetComponen void EthernetComponent::dump_connect_params_() { esp_netif_ip_info_t ip; esp_netif_get_ip_info(this->eth_netif_, &ip); - const ip_addr_t *dns_ip1 = dns_getserver(0); - const ip_addr_t *dns_ip2 = dns_getserver(1); + const ip_addr_t *dns_ip1; + const ip_addr_t *dns_ip2; + { + LwIPLock lock; + dns_ip1 = dns_getserver(0); + dns_ip2 = dns_getserver(1); + } ESP_LOGCONFIG(TAG, " IP Address: %s\n" @@ -614,12 +689,9 @@ void EthernetComponent::add_phy_register(PHYRegister register_value) { this->phy void EthernetComponent::set_type(EthernetType type) { this->type_ = type; } void EthernetComponent::set_manual_ip(const ManualIP &manual_ip) { this->manual_ip_ = manual_ip; } -std::string EthernetComponent::get_use_address() const { - if (this->use_address_.empty()) { - return App.get_name() + ".local"; - } - return this->use_address_; -} +// set_use_address() is guaranteed to be called during component setup by Python code generation, +// so use_address_ will always be valid when get_use_address() is called - no fallback needed. +const std::string &EthernetComponent::get_use_address() const { return this->use_address_; } void EthernetComponent::set_use_address(const std::string &use_address) { this->use_address_ = use_address; } @@ -632,7 +704,9 @@ void EthernetComponent::get_eth_mac_address_raw(uint8_t *mac) { std::string EthernetComponent::get_eth_mac_address_pretty() { uint8_t mac[6]; get_eth_mac_address_raw(mac); - return str_snprintf("%02X:%02X:%02X:%02X:%02X:%02X", 17, mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]); + char buf[18]; + format_mac_addr_upper(mac, buf); + return std::string(buf); } eth_duplex_t EthernetComponent::get_duplex_mode() { @@ -669,6 +743,7 @@ bool EthernetComponent::powerdown() { #ifndef USE_ETHERNET_SPI +#ifdef USE_ETHERNET_KSZ8081 constexpr uint8_t KSZ80XX_PC2R_REG_ADDR = 0x1F; void EthernetComponent::ksz8081_set_clock_reference_(esp_eth_mac_t *mac) { @@ -697,6 +772,7 @@ void EthernetComponent::ksz8081_set_clock_reference_(esp_eth_mac_t *mac) { ESP_LOGVV(TAG, "KSZ8081 PHY Control 2: %s", format_hex_pretty((u_int8_t *) &phy_control_2, 2).c_str()); } } +#endif // USE_ETHERNET_KSZ8081 void EthernetComponent::write_phy_register_(esp_eth_mac_t *mac, PHYRegister register_data) { esp_err_t err; diff --git a/esphome/components/ethernet/ethernet_component.h b/esphome/components/ethernet/ethernet_component.h index 1b347946f5..d5dda3e3ae 100644 --- a/esphome/components/ethernet/ethernet_component.h +++ b/esphome/components/ethernet/ethernet_component.h @@ -11,6 +11,7 @@ #include "esp_eth_mac.h" #include "esp_netif.h" #include "esp_mac.h" +#include "esp_idf_version.h" namespace esphome { namespace ethernet { @@ -27,6 +28,7 @@ enum EthernetType : uint8_t { ETHERNET_TYPE_W5500, ETHERNET_TYPE_OPENETH, ETHERNET_TYPE_DM9051, + ETHERNET_TYPE_LAN8670, }; struct ManualIP { @@ -82,10 +84,11 @@ class EthernetComponent : public Component { #endif void set_type(EthernetType type); void set_manual_ip(const ManualIP &manual_ip); + void set_fixed_mac(const std::array &mac) { this->fixed_mac_ = mac; } network::IPAddresses get_ip_addresses(); network::IPAddress get_dns_address(uint8_t num); - std::string get_use_address() const; + const std::string &get_use_address() const; void set_use_address(const std::string &use_address); void get_eth_mac_address_raw(uint8_t *mac); std::string get_eth_mac_address_pretty(); @@ -101,9 +104,13 @@ class EthernetComponent : public Component { #endif /* LWIP_IPV6 */ void start_connect_(); + void finish_connect_(); void dump_connect_params_(); + void log_error_and_mark_failed_(esp_err_t err, const char *message); +#ifdef USE_ETHERNET_KSZ8081 /// @brief Set `RMII Reference Clock Select` bit for KSZ8081. void ksz8081_set_clock_reference_(esp_eth_mac_t *mac); +#endif /// @brief Set arbitratry PHY registers from config. void write_phy_register_(esp_eth_mac_t *mac, PHYRegister register_data); @@ -143,17 +150,22 @@ class EthernetComponent : public Component { bool got_ipv4_address_{false}; #if LWIP_IPV6 uint8_t ipv6_count_{0}; + bool ipv6_setup_done_{false}; #endif /* LWIP_IPV6 */ // Pointers at the end (naturally aligned) esp_netif_t *eth_netif_{nullptr}; esp_eth_handle_t eth_handle_; esp_eth_phy_t *phy_{nullptr}; + optional> fixed_mac_; }; // NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables) extern EthernetComponent *global_eth_component; + +#if ESP_IDF_VERSION < ESP_IDF_VERSION_VAL(5, 4, 2) extern "C" esp_eth_phy_t *esp_eth_phy_new_jl1101(const eth_phy_config_t *config); +#endif } // namespace ethernet } // namespace esphome diff --git a/esphome/components/ethernet_info/ethernet_info_text_sensor.h b/esphome/components/ethernet_info/ethernet_info_text_sensor.h index 2e67694bbd..2adc08e31e 100644 --- a/esphome/components/ethernet_info/ethernet_info_text_sensor.h +++ b/esphome/components/ethernet_info/ethernet_info_text_sensor.h @@ -29,7 +29,6 @@ class IPAddressEthernetInfo : public PollingComponent, public text_sensor::TextS } float get_setup_priority() const override { return setup_priority::ETHERNET; } - std::string unique_id() override { return get_mac_address() + "-ethernetinfo"; } void dump_config() override; void add_ip_sensors(uint8_t index, text_sensor::TextSensor *s) { this->ip_sensors_[index] = s; } @@ -52,7 +51,6 @@ class DNSAddressEthernetInfo : public PollingComponent, public text_sensor::Text } } float get_setup_priority() const override { return setup_priority::ETHERNET; } - std::string unique_id() override { return get_mac_address() + "-ethernetinfo-dns"; } void dump_config() override; protected: @@ -63,7 +61,6 @@ class MACAddressEthernetInfo : public Component, public text_sensor::TextSensor public: void setup() override { this->publish_state(ethernet::global_eth_component->get_eth_mac_address_pretty()); } float get_setup_priority() const override { return setup_priority::ETHERNET; } - std::string unique_id() override { return get_mac_address() + "-ethernetinfo-mac"; } void dump_config() override; }; diff --git a/esphome/components/event/__init__.py b/esphome/components/event/__init__.py index 3aff96a48e..449cc48625 100644 --- a/esphome/components/event/__init__.py +++ b/esphome/components/event/__init__.py @@ -17,7 +17,7 @@ from esphome.const import ( DEVICE_CLASS_EMPTY, DEVICE_CLASS_MOTION, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass @@ -143,7 +143,6 @@ async def event_fire_to_code(config, action_id, template_arg, args): return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_EVENT") cg.add_global(event_ns.using) diff --git a/esphome/components/event/event.h b/esphome/components/event/event.h index 03c3c8d95a..a90c8ebe05 100644 --- a/esphome/components/event/event.h +++ b/esphome/components/event/event.h @@ -13,11 +13,11 @@ namespace event { #define LOG_EVENT(prefix, type, obj) \ if ((obj) != nullptr) { \ ESP_LOGCONFIG(TAG, "%s%s '%s'", prefix, LOG_STR_LITERAL(type), (obj)->get_name().c_str()); \ - if (!(obj)->get_icon().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon().c_str()); \ + if (!(obj)->get_icon_ref().empty()) { \ + ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon_ref().c_str()); \ } \ - if (!(obj)->get_device_class().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Device Class: '%s'", prefix, (obj)->get_device_class().c_str()); \ + if (!(obj)->get_device_class_ref().empty()) { \ + ESP_LOGCONFIG(TAG, "%s Device Class: '%s'", prefix, (obj)->get_device_class_ref().c_str()); \ } \ } diff --git a/esphome/components/event_emitter/__init__.py b/esphome/components/event_emitter/__init__.py deleted file mode 100644 index fcbbf26f02..0000000000 --- a/esphome/components/event_emitter/__init__.py +++ /dev/null @@ -1,5 +0,0 @@ -CODEOWNERS = ["@Rapsssito"] - -# Allows event_emitter to be configured in yaml, to allow use of the C++ api. - -CONFIG_SCHEMA = {} diff --git a/esphome/components/event_emitter/event_emitter.cpp b/esphome/components/event_emitter/event_emitter.cpp deleted file mode 100644 index 8487e19c2f..0000000000 --- a/esphome/components/event_emitter/event_emitter.cpp +++ /dev/null @@ -1,14 +0,0 @@ -#include "event_emitter.h" - -namespace esphome { -namespace event_emitter { - -static const char *const TAG = "event_emitter"; - -void raise_event_emitter_full_error() { - ESP_LOGE(TAG, "EventEmitter has reached the maximum number of listeners for event"); - ESP_LOGW(TAG, "Removing listener to make space for new listener"); -} - -} // namespace event_emitter -} // namespace esphome diff --git a/esphome/components/event_emitter/event_emitter.h b/esphome/components/event_emitter/event_emitter.h deleted file mode 100644 index 3876a2cc14..0000000000 --- a/esphome/components/event_emitter/event_emitter.h +++ /dev/null @@ -1,63 +0,0 @@ -#pragma once -#include -#include -#include -#include - -#include "esphome/core/log.h" - -namespace esphome { -namespace event_emitter { - -using EventEmitterListenerID = uint32_t; -void raise_event_emitter_full_error(); - -// EventEmitter class that can emit events with a specific name (it is highly recommended to use an enum class for this) -// and a list of arguments. Supports multiple listeners for each event. -template class EventEmitter { - public: - EventEmitterListenerID on(EvtType event, std::function listener) { - EventEmitterListenerID listener_id = get_next_id_(event); - listeners_[event][listener_id] = listener; - return listener_id; - } - - void off(EvtType event, EventEmitterListenerID id) { - if (listeners_.count(event) == 0) - return; - listeners_[event].erase(id); - } - - protected: - void emit_(EvtType event, Args... args) { - if (listeners_.count(event) == 0) - return; - for (const auto &listener : listeners_[event]) { - listener.second(args...); - } - } - - EventEmitterListenerID get_next_id_(EvtType event) { - // Check if the map is full - if (listeners_[event].size() == std::numeric_limits::max()) { - // Raise an error if the map is full - raise_event_emitter_full_error(); - off(event, 0); - return 0; - } - // Get the next ID for the given event. - EventEmitterListenerID next_id = (current_id_ + 1) % std::numeric_limits::max(); - while (listeners_[event].count(next_id) > 0) { - next_id = (next_id + 1) % std::numeric_limits::max(); - } - current_id_ = next_id; - return current_id_; - } - - private: - std::unordered_map>> listeners_; - EventEmitterListenerID current_id_ = 0; -}; - -} // namespace event_emitter -} // namespace esphome diff --git a/esphome/components/external_components/__init__.py b/esphome/components/external_components/__init__.py index a09217fd21..ceb402c5b7 100644 --- a/esphome/components/external_components/__init__.py +++ b/esphome/components/external_components/__init__.py @@ -39,11 +39,13 @@ async def to_code(config): pass -def _process_git_config(config: dict, refresh) -> str: +def _process_git_config(config: dict, refresh, skip_update: bool = False) -> str: + # When skip_update is True, use NEVER_REFRESH to prevent updates + actual_refresh = git.NEVER_REFRESH if skip_update else refresh repo_dir, _ = git.clone_or_update( url=config[CONF_URL], ref=config.get(CONF_REF), - refresh=refresh, + refresh=actual_refresh, domain=DOMAIN, username=config.get(CONF_USERNAME), password=config.get(CONF_PASSWORD), @@ -70,12 +72,12 @@ def _process_git_config(config: dict, refresh) -> str: return components_dir -def _process_single_config(config: dict): +def _process_single_config(config: dict, skip_update: bool = False): conf = config[CONF_SOURCE] if conf[CONF_TYPE] == TYPE_GIT: with cv.prepend_path([CONF_SOURCE]): components_dir = _process_git_config( - config[CONF_SOURCE], config[CONF_REFRESH] + config[CONF_SOURCE], config[CONF_REFRESH], skip_update ) elif conf[CONF_TYPE] == TYPE_LOCAL: components_dir = Path(CORE.relative_config_path(conf[CONF_PATH])) @@ -105,7 +107,7 @@ def _process_single_config(config: dict): loader.install_meta_finder(components_dir, allowed_components=allowed_components) -def do_external_components_pass(config: dict) -> None: +def do_external_components_pass(config: dict, skip_update: bool = False) -> None: conf = config.get(DOMAIN) if conf is None: return @@ -113,4 +115,4 @@ def do_external_components_pass(config: dict) -> None: conf = CONFIG_SCHEMA(conf) for i, c in enumerate(conf): with cv.prepend_path(i): - _process_single_config(c) + _process_single_config(c, skip_update) diff --git a/esphome/components/factory_reset/__init__.py b/esphome/components/factory_reset/__init__.py index f1bcfd8c55..f3cefe6970 100644 --- a/esphome/components/factory_reset/__init__.py +++ b/esphome/components/factory_reset/__init__.py @@ -1,5 +1,97 @@ +from esphome.automation import Trigger, build_automation, validate_automation import esphome.codegen as cg +from esphome.components.esp8266 import CONF_RESTORE_FROM_FLASH, KEY_ESP8266 +import esphome.config_validation as cv +from esphome.const import ( + CONF_ID, + CONF_TRIGGER_ID, + PLATFORM_BK72XX, + PLATFORM_ESP32, + PLATFORM_ESP8266, + PLATFORM_LN882X, + PLATFORM_RTL87XX, +) +from esphome.core import CORE +from esphome.final_validate import full_config CODEOWNERS = ["@anatoly-savchenkov"] factory_reset_ns = cg.esphome_ns.namespace("factory_reset") +FactoryResetComponent = factory_reset_ns.class_("FactoryResetComponent", cg.Component) +FastBootTrigger = factory_reset_ns.class_("FastBootTrigger", Trigger, cg.Component) + +CONF_MAX_DELAY = "max_delay" +CONF_RESETS_REQUIRED = "resets_required" +CONF_ON_INCREMENT = "on_increment" + + +def _validate(config): + if CONF_RESETS_REQUIRED in config: + return cv.only_on( + [ + PLATFORM_BK72XX, + PLATFORM_ESP32, + PLATFORM_ESP8266, + PLATFORM_LN882X, + PLATFORM_RTL87XX, + ] + )(config) + + if CONF_ON_INCREMENT in config: + raise cv.Invalid( + f"'{CONF_ON_INCREMENT}' requires a value for '{CONF_RESETS_REQUIRED}'" + ) + return config + + +CONFIG_SCHEMA = cv.All( + cv.Schema( + { + cv.GenerateID(): cv.declare_id(FactoryResetComponent), + cv.Optional(CONF_MAX_DELAY, default="10s"): cv.All( + cv.positive_time_period_seconds, + cv.Range(min=cv.TimePeriod(milliseconds=1000)), + ), + cv.Optional(CONF_RESETS_REQUIRED): cv.positive_not_null_int, + cv.Optional(CONF_ON_INCREMENT): validate_automation( + { + cv.GenerateID(CONF_TRIGGER_ID): cv.declare_id(FastBootTrigger), + } + ), + } + ).extend(cv.COMPONENT_SCHEMA), + _validate, +) + + +def _final_validate(config): + if CORE.is_esp8266 and CONF_RESETS_REQUIRED in config: + fconfig = full_config.get() + if not fconfig.get_config_for_path([KEY_ESP8266, CONF_RESTORE_FROM_FLASH]): + raise cv.Invalid( + "'resets_required' needs 'restore_from_flash' to be enabled in the 'esp8266' configuration" + ) + return config + + +FINAL_VALIDATE_SCHEMA = _final_validate + + +async def to_code(config): + if reset_count := config.get(CONF_RESETS_REQUIRED): + var = cg.new_Pvariable( + config[CONF_ID], + reset_count, + config[CONF_MAX_DELAY].total_milliseconds, + ) + await cg.register_component(var, config) + for conf in config.get(CONF_ON_INCREMENT, []): + trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var) + await build_automation( + trigger, + [ + (cg.uint8, "x"), + (cg.uint8, "target"), + ], + conf, + ) diff --git a/esphome/components/factory_reset/button/factory_reset_button.cpp b/esphome/components/factory_reset/button/factory_reset_button.cpp index 585975c043..d582317767 100644 --- a/esphome/components/factory_reset/button/factory_reset_button.cpp +++ b/esphome/components/factory_reset/button/factory_reset_button.cpp @@ -1,7 +1,13 @@ #include "factory_reset_button.h" + +#include "esphome/core/defines.h" + +#ifdef USE_OPENTHREAD +#include "esphome/components/openthread/openthread.h" +#endif +#include "esphome/core/application.h" #include "esphome/core/hal.h" #include "esphome/core/log.h" -#include "esphome/core/application.h" namespace esphome { namespace factory_reset { @@ -13,9 +19,20 @@ void FactoryResetButton::press_action() { ESP_LOGI(TAG, "Resetting"); // Let MQTT settle a bit delay(100); // NOLINT +#ifdef USE_OPENTHREAD + openthread::global_openthread_component->on_factory_reset(FactoryResetButton::factory_reset_callback); +#else + global_preferences->reset(); + App.safe_reboot(); +#endif +} + +#ifdef USE_OPENTHREAD +void FactoryResetButton::factory_reset_callback() { global_preferences->reset(); App.safe_reboot(); } +#endif } // namespace factory_reset } // namespace esphome diff --git a/esphome/components/factory_reset/button/factory_reset_button.h b/esphome/components/factory_reset/button/factory_reset_button.h index 9996a860d9..c68da2ca74 100644 --- a/esphome/components/factory_reset/button/factory_reset_button.h +++ b/esphome/components/factory_reset/button/factory_reset_button.h @@ -1,7 +1,9 @@ #pragma once -#include "esphome/core/component.h" +#include "esphome/core/defines.h" + #include "esphome/components/button/button.h" +#include "esphome/core/component.h" namespace esphome { namespace factory_reset { @@ -9,6 +11,9 @@ namespace factory_reset { class FactoryResetButton : public button::Button, public Component { public: void dump_config() override; +#ifdef USE_OPENTHREAD + static void factory_reset_callback(); +#endif protected: void press_action() override; diff --git a/esphome/components/factory_reset/factory_reset.cpp b/esphome/components/factory_reset/factory_reset.cpp new file mode 100644 index 0000000000..c900759d90 --- /dev/null +++ b/esphome/components/factory_reset/factory_reset.cpp @@ -0,0 +1,76 @@ +#include "factory_reset.h" + +#include "esphome/core/application.h" +#include "esphome/core/hal.h" +#include "esphome/core/log.h" + +#include + +#if !defined(USE_RP2040) && !defined(USE_HOST) + +namespace esphome { +namespace factory_reset { + +static const char *const TAG = "factory_reset"; +static const uint32_t POWER_CYCLES_KEY = 0xFA5C0DE; + +static bool was_power_cycled() { +#ifdef USE_ESP32 + return esp_reset_reason() == ESP_RST_POWERON; +#endif +#ifdef USE_ESP8266 + auto reset_reason = EspClass::getResetReason(); + return strcasecmp(reset_reason.c_str(), "power On") == 0 || strcasecmp(reset_reason.c_str(), "external system") == 0; +#endif +#ifdef USE_LIBRETINY + auto reason = lt_get_reboot_reason(); + return reason == REBOOT_REASON_POWER || reason == REBOOT_REASON_HARDWARE; +#endif +} + +void FactoryResetComponent::dump_config() { + uint8_t count = 0; + this->flash_.load(&count); + ESP_LOGCONFIG(TAG, "Factory Reset by Reset:"); + ESP_LOGCONFIG(TAG, + " Max interval between resets %" PRIu32 " seconds\n" + " Current count: %u\n" + " Factory reset after %u resets", + this->max_interval_ / 1000, count, this->required_count_); +} + +void FactoryResetComponent::save_(uint8_t count) { + this->flash_.save(&count); + global_preferences->sync(); + this->defer([count, this] { this->increment_callback_.call(count, this->required_count_); }); +} + +void FactoryResetComponent::setup() { + this->flash_ = global_preferences->make_preference(POWER_CYCLES_KEY, true); + if (was_power_cycled()) { + uint8_t count = 0; + this->flash_.load(&count); + // this is a power on reset or external system reset + count++; + if (count == this->required_count_) { + ESP_LOGW(TAG, "Reset count reached, factory resetting"); + global_preferences->reset(); + // delay to allow log to be sent + delay(100); // NOLINT + App.safe_reboot(); // should not return + } + this->save_(count); + ESP_LOGD(TAG, "Power on reset detected, incremented count to %u", count); + this->set_timeout(this->max_interval_, [this]() { + ESP_LOGD(TAG, "No reset in the last %" PRIu32 " seconds, resetting count", this->max_interval_ / 1000); + this->save_(0); // reset count + }); + } else { + this->save_(0); // reset count if not a power cycle + } +} + +} // namespace factory_reset +} // namespace esphome + +#endif // !defined(USE_RP2040) && !defined(USE_HOST) diff --git a/esphome/components/factory_reset/factory_reset.h b/esphome/components/factory_reset/factory_reset.h new file mode 100644 index 0000000000..80942b29bd --- /dev/null +++ b/esphome/components/factory_reset/factory_reset.h @@ -0,0 +1,43 @@ +#pragma once + +#include "esphome/core/component.h" +#include "esphome/core/automation.h" +#include "esphome/core/preferences.h" +#if !defined(USE_RP2040) && !defined(USE_HOST) + +#ifdef USE_ESP32 +#include +#endif + +namespace esphome { +namespace factory_reset { +class FactoryResetComponent : public Component { + public: + FactoryResetComponent(uint8_t required_count, uint32_t max_interval) + : required_count_(required_count), max_interval_(max_interval) {} + + void dump_config() override; + void setup() override; + void add_increment_callback(std::function &&callback) { + this->increment_callback_.add(std::move(callback)); + } + + protected: + ~FactoryResetComponent() = default; + void save_(uint8_t count); + ESPPreferenceObject flash_{}; // saves the number of fast power cycles + uint8_t required_count_; // The number of boot attempts before fast boot is enabled + uint32_t max_interval_; // max interval between power cycles + CallbackManager increment_callback_{}; +}; + +class FastBootTrigger : public Trigger { + public: + explicit FastBootTrigger(FactoryResetComponent *parent) { + parent->add_increment_callback([this](uint8_t current, uint8_t target) { this->trigger(current, target); }); + } +}; +} // namespace factory_reset +} // namespace esphome + +#endif // !defined(USE_RP2040) && !defined(USE_HOST) diff --git a/esphome/components/factory_reset/switch/factory_reset_switch.cpp b/esphome/components/factory_reset/switch/factory_reset_switch.cpp index 1282c73f4e..75449aa526 100644 --- a/esphome/components/factory_reset/switch/factory_reset_switch.cpp +++ b/esphome/components/factory_reset/switch/factory_reset_switch.cpp @@ -1,7 +1,13 @@ #include "factory_reset_switch.h" + +#include "esphome/core/defines.h" + +#ifdef USE_OPENTHREAD +#include "esphome/components/openthread/openthread.h" +#endif +#include "esphome/core/application.h" #include "esphome/core/hal.h" #include "esphome/core/log.h" -#include "esphome/core/application.h" namespace esphome { namespace factory_reset { @@ -17,10 +23,21 @@ void FactoryResetSwitch::write_state(bool state) { ESP_LOGI(TAG, "Resetting"); // Let MQTT settle a bit delay(100); // NOLINT +#ifdef USE_OPENTHREAD + openthread::global_openthread_component->on_factory_reset(FactoryResetSwitch::factory_reset_callback); +#else global_preferences->reset(); App.safe_reboot(); +#endif } } +#ifdef USE_OPENTHREAD +void FactoryResetSwitch::factory_reset_callback() { + global_preferences->reset(); + App.safe_reboot(); +} +#endif + } // namespace factory_reset } // namespace esphome diff --git a/esphome/components/factory_reset/switch/factory_reset_switch.h b/esphome/components/factory_reset/switch/factory_reset_switch.h index 2c914ea76d..8ea0c79108 100644 --- a/esphome/components/factory_reset/switch/factory_reset_switch.h +++ b/esphome/components/factory_reset/switch/factory_reset_switch.h @@ -1,7 +1,8 @@ #pragma once -#include "esphome/core/component.h" #include "esphome/components/switch/switch.h" +#include "esphome/core/component.h" +#include "esphome/core/defines.h" namespace esphome { namespace factory_reset { @@ -9,6 +10,9 @@ namespace factory_reset { class FactoryResetSwitch : public switch_::Switch, public Component { public: void dump_config() override; +#ifdef USE_OPENTHREAD + static void factory_reset_callback(); +#endif protected: void write_state(bool state) override; diff --git a/esphome/components/fan/__init__.py b/esphome/components/fan/__init__.py index 0b1d39575d..da8bf850c7 100644 --- a/esphome/components/fan/__init__.py +++ b/esphome/components/fan/__init__.py @@ -31,7 +31,7 @@ from esphome.const import ( CONF_TRIGGER_ID, CONF_WEB_SERVER, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity IS_PLATFORM_COMPONENT = True @@ -398,7 +398,6 @@ async def fan_is_on_off_to_code(config, condition_id, template_arg, args): return cg.new_Pvariable(condition_id, template_arg, paren) -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_FAN") cg.add_global(fan_ns.using) diff --git a/esphome/components/fan/fan.cpp b/esphome/components/fan/fan.cpp index 82fc5319e0..26065ed644 100644 --- a/esphome/components/fan/fan.cpp +++ b/esphome/components/fan/fan.cpp @@ -148,7 +148,8 @@ void Fan::publish_state() { constexpr uint32_t RESTORE_STATE_VERSION = 0x71700ABA; optional Fan::restore_state_() { FanRestoreState recovered{}; - this->rtc_ = global_preferences->make_preference(this->get_object_id_hash() ^ RESTORE_STATE_VERSION); + this->rtc_ = + global_preferences->make_preference(this->get_preference_hash() ^ RESTORE_STATE_VERSION); bool restored = this->rtc_.load(&recovered); switch (this->restore_mode_) { diff --git a/esphome/components/fan/fan_traits.h b/esphome/components/fan/fan_traits.h index 2ef6f8b7cc..48509e5705 100644 --- a/esphome/components/fan/fan_traits.h +++ b/esphome/components/fan/fan_traits.h @@ -4,6 +4,13 @@ #pragma once namespace esphome { + +#ifdef USE_API +namespace api { +class APIConnection; +} // namespace api +#endif + namespace fan { class FanTraits { @@ -36,6 +43,15 @@ class FanTraits { bool supports_preset_modes() const { return !this->preset_modes_.empty(); } protected: +#ifdef USE_API + // The API connection is a friend class to access internal methods + friend class api::APIConnection; + // This method returns a reference to the internal preset modes set. + // It is used by the API to avoid copying data when encoding messages. + // Warning: Do not use this method outside of the API connection code. + // It returns a reference to internal data that can be invalidated. + const std::set &supported_preset_modes_for_api_() const { return this->preset_modes_; } +#endif bool oscillation_{false}; bool speed_{false}; bool direction_{false}; diff --git a/esphome/components/fastled_base/fastled_light.cpp b/esphome/components/fastled_base/fastled_light.cpp index bca7de811a..b3946a34b5 100644 --- a/esphome/components/fastled_base/fastled_light.cpp +++ b/esphome/components/fastled_base/fastled_light.cpp @@ -9,7 +9,6 @@ namespace fastled_base { static const char *const TAG = "fastled"; void FastLEDLightOutput::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->controller_->init(); this->controller_->setLeds(this->leds_, this->num_leds_); this->effect_data_ = new uint8_t[this->num_leds_]; // NOLINT diff --git a/esphome/components/fastled_clockless/light.py b/esphome/components/fastled_clockless/light.py index 49a6d390be..aa2172bf88 100644 --- a/esphome/components/fastled_clockless/light.py +++ b/esphome/components/fastled_clockless/light.py @@ -2,7 +2,13 @@ from esphome import pins import esphome.codegen as cg from esphome.components import fastled_base import esphome.config_validation as cv -from esphome.const import CONF_CHIPSET, CONF_NUM_LEDS, CONF_PIN, CONF_RGB_ORDER +from esphome.const import ( + CONF_CHIPSET, + CONF_NUM_LEDS, + CONF_PIN, + CONF_RGB_ORDER, + Framework, +) AUTO_LOAD = ["fastled_base"] @@ -48,13 +54,22 @@ CONFIG_SCHEMA = cv.All( cv.Required(CONF_PIN): pins.internal_gpio_output_pin_number, } ), - _validate, + cv.only_with_framework( + frameworks=Framework.ARDUINO, + suggestions={ + Framework.ESP_IDF: ( + "esp32_rmt_led_strip", + "light/esp32_rmt_led_strip", + ) + }, + ), cv.require_framework_version( esp8266_arduino=cv.Version(2, 7, 4), esp32_arduino=cv.Version(99, 0, 0), max_version=True, extra_message="Please see note on documentation for FastLED", ), + _validate, ) diff --git a/esphome/components/fastled_spi/light.py b/esphome/components/fastled_spi/light.py index ac30721eb4..e863d33846 100644 --- a/esphome/components/fastled_spi/light.py +++ b/esphome/components/fastled_spi/light.py @@ -9,6 +9,7 @@ from esphome.const import ( CONF_DATA_RATE, CONF_NUM_LEDS, CONF_RGB_ORDER, + Framework, ) AUTO_LOAD = ["fastled_base"] @@ -33,6 +34,15 @@ CONFIG_SCHEMA = cv.All( cv.Optional(CONF_DATA_RATE): cv.frequency, } ), + cv.only_with_framework( + frameworks=Framework.ARDUINO, + suggestions={ + Framework.ESP_IDF: ( + "spi_led_strip", + "light/spi_led_strip", + ) + }, + ), cv.require_framework_version( esp8266_arduino=cv.Version(2, 7, 4), esp32_arduino=cv.Version(99, 0, 0), @@ -45,9 +55,7 @@ CONFIG_SCHEMA = cv.All( async def to_code(config): var = await fastled_base.new_fastled_light(config) - rgb_order = cg.RawExpression( - config[CONF_RGB_ORDER] if CONF_RGB_ORDER in config else "RGB" - ) + rgb_order = cg.RawExpression(config.get(CONF_RGB_ORDER, "RGB")) data_rate = None if CONF_DATA_RATE in config: diff --git a/esphome/components/fingerprint_grow/fingerprint_grow.cpp b/esphome/components/fingerprint_grow/fingerprint_grow.cpp index e28548428c..eb7ede8fe9 100644 --- a/esphome/components/fingerprint_grow/fingerprint_grow.cpp +++ b/esphome/components/fingerprint_grow/fingerprint_grow.cpp @@ -57,8 +57,6 @@ void FingerprintGrowComponent::update() { } void FingerprintGrowComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->has_sensing_pin_ = (this->sensing_pin_ != nullptr); this->has_power_pin_ = (this->sensor_power_pin_ != nullptr); @@ -82,7 +80,7 @@ void FingerprintGrowComponent::setup() { delay(20); // This delay guarantees the sensor will in fact be powered power. if (this->check_password_()) { - if (this->new_password_ != -1) { + if (this->new_password_ != std::numeric_limits::max()) { if (this->set_password_()) return; } else { diff --git a/esphome/components/fingerprint_grow/fingerprint_grow.h b/esphome/components/fingerprint_grow/fingerprint_grow.h index 1c3098ef14..590c709c22 100644 --- a/esphome/components/fingerprint_grow/fingerprint_grow.h +++ b/esphome/components/fingerprint_grow/fingerprint_grow.h @@ -6,6 +6,7 @@ #include "esphome/components/binary_sensor/binary_sensor.h" #include "esphome/components/uart/uart.h" +#include #include namespace esphome { @@ -177,7 +178,7 @@ class FingerprintGrowComponent : public PollingComponent, public uart::UARTDevic uint8_t address_[4] = {0xFF, 0xFF, 0xFF, 0xFF}; uint16_t capacity_ = 64; uint32_t password_ = 0x0; - uint32_t new_password_ = -1; + uint32_t new_password_ = std::numeric_limits::max(); GPIOPin *sensing_pin_{nullptr}; GPIOPin *sensor_power_pin_{nullptr}; uint8_t enrollment_image_ = 0; diff --git a/esphome/components/font/__init__.py b/esphome/components/font/__init__.py index 7d9a35647e..ddcee14635 100644 --- a/esphome/components/font/__init__.py +++ b/esphome/components/font/__init__.py @@ -3,7 +3,6 @@ import functools import hashlib from itertools import accumulate import logging -import os from pathlib import Path import re @@ -15,6 +14,7 @@ from freetype import ( FT_LOAD_RENDER, FT_LOAD_TARGET_MONO, Face, + FT_Exception, ft_pixel_mode_mono, ) import requests @@ -37,6 +37,7 @@ from esphome.const import ( ) from esphome.core import CORE, HexInt from esphome.helpers import cpp_string_escape +from esphome.types import ConfigType _LOGGER = logging.getLogger(__name__) @@ -94,7 +95,14 @@ class FontCache(MutableMapping): return self.store[self._keytransform(item)] def __setitem__(self, key, value): - self.store[self._keytransform(key)] = Face(str(value)) + transformed = self._keytransform(key) + try: + self.store[transformed] = Face(str(value)) + except FT_Exception as exc: + file = transformed.split(":", 1) + raise cv.Invalid( + f"{file[0].capitalize()} {file[1]} is not a valid font file" + ) from exc FONT_CACHE = FontCache() @@ -245,11 +253,11 @@ def validate_truetype_file(value): return CORE.relative_config_path(cv.file_(value)) -def add_local_file(value): +def add_local_file(value: ConfigType) -> ConfigType: if value in FONT_CACHE: return value - path = value[CONF_PATH] - if not os.path.isfile(path): + path = Path(value[CONF_PATH]) + if not path.is_file(): raise cv.Invalid(f"File '{path}' not found.") FONT_CACHE[value] = path return value @@ -310,7 +318,7 @@ def download_gfont(value): external_files.compute_local_file_dir(DOMAIN) / f"{value[CONF_FAMILY]}@{value[CONF_WEIGHT]}@{value[CONF_ITALIC]}@v1.ttf" ) - if not external_files.is_file_recent(str(path), value[CONF_REFRESH]): + if not external_files.is_file_recent(path, value[CONF_REFRESH]): _LOGGER.debug("download_gfont: path=%s", path) try: req = requests.get(url, timeout=external_files.NETWORK_TIMEOUT) diff --git a/esphome/components/fs3000/fs3000.cpp b/esphome/components/fs3000/fs3000.cpp index c99772a23d..cea599211d 100644 --- a/esphome/components/fs3000/fs3000.cpp +++ b/esphome/components/fs3000/fs3000.cpp @@ -7,8 +7,6 @@ namespace fs3000 { static const char *const TAG = "fs3000"; void FS3000Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - if (model_ == FIVE) { // datasheet gives 9 points to interpolate from for the 1005 model static const uint16_t RAW_DATA_POINTS_1005[9] = {409, 915, 1522, 2066, 2523, 2908, 3256, 3572, 3686}; diff --git a/esphome/components/ft5x06/touchscreen/ft5x06_touchscreen.cpp b/esphome/components/ft5x06/touchscreen/ft5x06_touchscreen.cpp index 9873a88fde..505c3cffc0 100644 --- a/esphome/components/ft5x06/touchscreen/ft5x06_touchscreen.cpp +++ b/esphome/components/ft5x06/touchscreen/ft5x06_touchscreen.cpp @@ -9,7 +9,6 @@ namespace ft5x06 { static const char *const TAG = "ft5x06.touchscreen"; void FT5x06Touchscreen::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (this->interrupt_pin_ != nullptr) { this->interrupt_pin_->setup(); this->interrupt_pin_->pin_mode(gpio::FLAG_INPUT | gpio::FLAG_PULLUP); @@ -50,7 +49,6 @@ void FT5x06Touchscreen::continue_setup_() { this->y_raw_max_ = this->display_->get_native_height(); } } - ESP_LOGCONFIG(TAG, "FT5x06 Touchscreen setup complete"); } void FT5x06Touchscreen::update_touches() { diff --git a/esphome/components/ft63x6/ft63x6.cpp b/esphome/components/ft63x6/ft63x6.cpp index ba5b2094a5..f7c4f255a0 100644 --- a/esphome/components/ft63x6/ft63x6.cpp +++ b/esphome/components/ft63x6/ft63x6.cpp @@ -28,7 +28,6 @@ static const uint8_t FT63X6_ADDR_CHIP_ID = 0xA3; static const char *const TAG = "FT63X6"; void FT63X6Touchscreen::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (this->interrupt_pin_ != nullptr) { this->interrupt_pin_->pin_mode(gpio::FLAG_INPUT | gpio::FLAG_PULLUP); this->interrupt_pin_->setup(); diff --git a/esphome/components/gdk101/gdk101.cpp b/esphome/components/gdk101/gdk101.cpp index e8401aa09b..4c156ab24b 100644 --- a/esphome/components/gdk101/gdk101.cpp +++ b/esphome/components/gdk101/gdk101.cpp @@ -11,22 +11,22 @@ static const uint8_t NUMBER_OF_READ_RETRIES = 5; void GDK101Component::update() { uint8_t data[2]; if (!this->read_dose_1m_(data)) { - this->status_set_warning("Failed to read dose 1m"); + this->status_set_warning(LOG_STR("Failed to read dose 1m")); return; } if (!this->read_dose_10m_(data)) { - this->status_set_warning("Failed to read dose 10m"); + this->status_set_warning(LOG_STR("Failed to read dose 10m")); return; } if (!this->read_status_(data)) { - this->status_set_warning("Failed to read status"); + this->status_set_warning(LOG_STR("Failed to read status")); return; } if (!this->read_measurement_duration_(data)) { - this->status_set_warning("Failed to read measurement duration"); + this->status_set_warning(LOG_STR("Failed to read measurement duration")); return; } this->status_clear_warning(); @@ -34,7 +34,6 @@ void GDK101Component::update() { void GDK101Component::setup() { uint8_t data[2]; - ESP_LOGCONFIG(TAG, "Running setup"); // first, reset the sensor if (!this->reset_sensor_(data)) { this->status_set_error("Reset failed!"); diff --git a/esphome/components/gl_r01_i2c/gl_r01_i2c.cpp b/esphome/components/gl_r01_i2c/gl_r01_i2c.cpp index 5a24c63525..e2a64b6877 100644 --- a/esphome/components/gl_r01_i2c/gl_r01_i2c.cpp +++ b/esphome/components/gl_r01_i2c/gl_r01_i2c.cpp @@ -17,7 +17,6 @@ static const uint8_t RESTART_CMD2 = 0xA5; static const uint8_t READ_DELAY = 40; // minimum milliseconds from datasheet to safely read measurement result void GLR01I2CComponent::setup() { - ESP_LOGCONFIG(TAG, "Setting up GL-R01 I2C..."); // Verify sensor presence if (!this->read_byte_16(REG_VERSION, &this->version_)) { ESP_LOGE(TAG, "Failed to communicate with GL-R01 I2C sensor!"); diff --git a/esphome/components/globals/__init__.py b/esphome/components/globals/__init__.py index e4bce99b0b..633ccea66b 100644 --- a/esphome/components/globals/__init__.py +++ b/esphome/components/globals/__init__.py @@ -8,7 +8,7 @@ from esphome.const import ( CONF_TYPE, CONF_VALUE, ) -from esphome.core import coroutine_with_priority +from esphome.core import CoroPriority, coroutine_with_priority CODEOWNERS = ["@esphome/core"] globals_ns = cg.esphome_ns.namespace("globals") @@ -35,7 +35,7 @@ CONFIG_SCHEMA = cv.Schema( # Run with low priority so that namespaces are registered first -@coroutine_with_priority(-100.0) +@coroutine_with_priority(CoroPriority.LATE) async def to_code(config): type_ = cg.RawExpression(config[CONF_TYPE]) restore = config[CONF_RESTORE_VALUE] diff --git a/esphome/components/gpio/binary_sensor/__init__.py b/esphome/components/gpio/binary_sensor/__init__.py index 867a8efe49..8372bc7e08 100644 --- a/esphome/components/gpio/binary_sensor/__init__.py +++ b/esphome/components/gpio/binary_sensor/__init__.py @@ -4,7 +4,13 @@ from esphome import pins import esphome.codegen as cg from esphome.components import binary_sensor import esphome.config_validation as cv -from esphome.const import CONF_ID, CONF_NAME, CONF_NUMBER, CONF_PIN +from esphome.const import ( + CONF_ALLOW_OTHER_USES, + CONF_ID, + CONF_NAME, + CONF_NUMBER, + CONF_PIN, +) from esphome.core import CORE from .. import gpio_ns @@ -29,7 +35,21 @@ CONFIG_SCHEMA = ( .extend( { cv.Required(CONF_PIN): pins.gpio_input_pin_schema, - cv.Optional(CONF_USE_INTERRUPT, default=True): cv.boolean, + # Interrupts are disabled by default for bk72xx, ln882x, and rtl87xx platforms + # due to hardware limitations or lack of reliable interrupt support. This ensures + # stable operation on these platforms. Future maintainers should verify platform + # capabilities before changing this default behavior. + cv.SplitDefault( + CONF_USE_INTERRUPT, + bk72xx=False, + esp32=True, + esp8266=True, + host=True, + ln882x=False, + nrf52=True, + rp2040=True, + rtl87xx=False, + ): cv.boolean, cv.Optional(CONF_INTERRUPT_TYPE, default="ANY"): cv.enum( INTERRUPT_TYPES, upper=True ), @@ -62,6 +82,18 @@ async def to_code(config): ) use_interrupt = False + # Check if pin is shared with other components (allow_other_uses) + # When a pin is shared, interrupts can interfere with other components + # (e.g., duty_cycle sensor) that need to monitor the pin's state changes + if use_interrupt and config[CONF_PIN].get(CONF_ALLOW_OTHER_USES, False): + _LOGGER.info( + "GPIO binary_sensor '%s': Disabling interrupts because pin %s is shared with other components. " + "The sensor will use polling mode for compatibility with other pin uses.", + config.get(CONF_NAME, config[CONF_ID]), + config[CONF_PIN][CONF_NUMBER], + ) + use_interrupt = False + cg.add(var.set_use_interrupt(use_interrupt)) if use_interrupt: cg.add(var.set_interrupt_type(config[CONF_INTERRUPT_TYPE])) diff --git a/esphome/components/gpio/binary_sensor/gpio_binary_sensor.cpp b/esphome/components/gpio/binary_sensor/gpio_binary_sensor.cpp index 4b8369cd59..7a35596194 100644 --- a/esphome/components/gpio/binary_sensor/gpio_binary_sensor.cpp +++ b/esphome/components/gpio/binary_sensor/gpio_binary_sensor.cpp @@ -6,6 +6,25 @@ namespace gpio { static const char *const TAG = "gpio.binary_sensor"; +#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_DEBUG +static const LogString *interrupt_type_to_string(gpio::InterruptType type) { + switch (type) { + case gpio::INTERRUPT_RISING_EDGE: + return LOG_STR("RISING_EDGE"); + case gpio::INTERRUPT_FALLING_EDGE: + return LOG_STR("FALLING_EDGE"); + case gpio::INTERRUPT_ANY_EDGE: + return LOG_STR("ANY_EDGE"); + default: + return LOG_STR("UNKNOWN"); + } +} + +static const LogString *gpio_mode_to_string(bool use_interrupt) { + return use_interrupt ? LOG_STR("interrupt") : LOG_STR("polling"); +} +#endif + void IRAM_ATTR GPIOBinarySensorStore::gpio_intr(GPIOBinarySensorStore *arg) { bool new_state = arg->isr_pin_.digital_read(); if (new_state != arg->last_state_) { @@ -51,25 +70,9 @@ void GPIOBinarySensor::setup() { void GPIOBinarySensor::dump_config() { LOG_BINARY_SENSOR("", "GPIO Binary Sensor", this); LOG_PIN(" Pin: ", this->pin_); - const char *mode = this->use_interrupt_ ? "interrupt" : "polling"; - ESP_LOGCONFIG(TAG, " Mode: %s", mode); + ESP_LOGCONFIG(TAG, " Mode: %s", LOG_STR_ARG(gpio_mode_to_string(this->use_interrupt_))); if (this->use_interrupt_) { - const char *interrupt_type; - switch (this->interrupt_type_) { - case gpio::INTERRUPT_RISING_EDGE: - interrupt_type = "RISING_EDGE"; - break; - case gpio::INTERRUPT_FALLING_EDGE: - interrupt_type = "FALLING_EDGE"; - break; - case gpio::INTERRUPT_ANY_EDGE: - interrupt_type = "ANY_EDGE"; - break; - default: - interrupt_type = "UNKNOWN"; - break; - } - ESP_LOGCONFIG(TAG, " Interrupt Type: %s", interrupt_type); + ESP_LOGCONFIG(TAG, " Interrupt Type: %s", LOG_STR_ARG(interrupt_type_to_string(this->interrupt_type_))); } } diff --git a/esphome/components/gpio/one_wire/gpio_one_wire.cpp b/esphome/components/gpio/one_wire/gpio_one_wire.cpp index ee80fde6fa..4191c45de1 100644 --- a/esphome/components/gpio/one_wire/gpio_one_wire.cpp +++ b/esphome/components/gpio/one_wire/gpio_one_wire.cpp @@ -8,7 +8,6 @@ namespace gpio { static const char *const TAG = "gpio.one_wire"; void GPIOOneWireBus::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->t_pin_->setup(); this->t_pin_->pin_mode(gpio::FLAG_INPUT | gpio::FLAG_PULLUP); // clear bus with 480µs high, otherwise initial reset in search might fail diff --git a/esphome/components/gpio/switch/gpio_switch.cpp b/esphome/components/gpio/switch/gpio_switch.cpp index 6f901d602d..b67af5e95d 100644 --- a/esphome/components/gpio/switch/gpio_switch.cpp +++ b/esphome/components/gpio/switch/gpio_switch.cpp @@ -8,8 +8,6 @@ static const char *const TAG = "switch.gpio"; float GPIOSwitch::get_setup_priority() const { return setup_priority::HARDWARE; } void GPIOSwitch::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); - bool initial_state = this->get_initial_state_with_restore_mode().value_or(false); // write state before setup diff --git a/esphome/components/gpio_expander/cached_gpio.h b/esphome/components/gpio_expander/cached_gpio.h index 78c675cdb2..eeff98cb6e 100644 --- a/esphome/components/gpio_expander/cached_gpio.h +++ b/esphome/components/gpio_expander/cached_gpio.h @@ -2,52 +2,81 @@ #include #include +#include +#include +#include #include "esphome/core/hal.h" -namespace esphome { -namespace gpio_expander { +namespace esphome::gpio_expander { /// @brief A class to cache the read state of a GPIO expander. /// This class caches reads between GPIO Pins which are on the same bank. /// This means that for reading whole Port (ex. 8 pins) component needs only one -/// I2C/SPI read per main loop call. It assumes, that one bit in byte identifies one GPIO pin +/// I2C/SPI read per main loop call. It assumes that one bit in byte identifies one GPIO pin. +/// /// Template parameters: -/// T - Type which represents internal register. Could be uint8_t or uint16_t. Adjust to -/// match size of your internal GPIO bank register. -/// N - Number of pins -template class CachedGpioExpander { +/// T - Type which represents internal bank register. Could be uint8_t or uint16_t. +/// Choose based on how your I/O expander reads pins: +/// * uint8_t: For chips that read banks separately (8 pins at a time) +/// Examples: MCP23017 (2x8-bit banks), TCA9555 (2x8-bit banks) +/// * uint16_t: For chips that read all pins at once (up to 16 pins) +/// Examples: PCF8574/8575 (8/16 pins), PCA9554/9555 (8/16 pins) +/// N - Total number of pins (maximum 65535) +/// P - Type for pin number parameters (automatically selected based on N: +/// uint8_t for N<=256, uint16_t for N>256). Can be explicitly specified +/// if needed (e.g., for components like SN74HC165 with >256 pins) +template 256), uint16_t, uint8_t>::type> +class CachedGpioExpander { public: - bool digital_read(T pin) { - uint8_t bank = pin / (sizeof(T) * BITS_PER_BYTE); - if (this->read_cache_invalidated_[bank]) { - this->read_cache_invalidated_[bank] = false; + /// @brief Read the state of the given pin. This will invalidate the cache for the given pin number. + /// @param pin Pin number to read + /// @return Pin state + bool digital_read(P pin) { + const P bank = pin / BANK_SIZE; + const T pin_mask = (1 << (pin % BANK_SIZE)); + // Check if specific pin cache is valid + if (this->read_cache_valid_[bank] & pin_mask) { + // Invalidate pin + this->read_cache_valid_[bank] &= ~pin_mask; + } else { + // Read whole bank from hardware if (!this->digital_read_hw(pin)) return false; + // Mark bank cache as valid except the pin that is being returned now + this->read_cache_valid_[bank] = std::numeric_limits::max() & ~pin_mask; } return this->digital_read_cache(pin); } - void digital_write(T pin, bool value) { this->digital_write_hw(pin, value); } + void digital_write(P pin, bool value) { this->digital_write_hw(pin, value); } protected: - /// @brief Call component low level function to read GPIO state from device - virtual bool digital_read_hw(T pin) = 0; - /// @brief Call component read function from internal cache. - virtual bool digital_read_cache(T pin) = 0; - /// @brief Call component low level function to write GPIO state to device - virtual void digital_write_hw(T pin, bool value) = 0; - const uint8_t cache_byte_size_ = N / (sizeof(T) * BITS_PER_BYTE); + /// @brief Read GPIO bank from hardware into internal state + /// @param pin Pin number (used to determine which bank to read) + /// @return true if read succeeded, false on communication error + /// @note This does NOT return the pin state. It returns whether the read operation succeeded. + /// The actual pin state should be returned by digital_read_cache(). + virtual bool digital_read_hw(P pin) = 0; + + /// @brief Get cached pin value from internal state + /// @param pin Pin number to read + /// @return Pin state (true = HIGH, false = LOW) + virtual bool digital_read_cache(P pin) = 0; + + /// @brief Write GPIO state to hardware + /// @param pin Pin number to write + /// @param value Pin state to write (true = HIGH, false = LOW) + virtual void digital_write_hw(P pin, bool value) = 0; /// @brief Invalidate cache. This function should be called in component loop(). - void reset_pin_cache_() { - for (T i = 0; i < this->cache_byte_size_; i++) { - this->read_cache_invalidated_[i] = true; - } - } + void reset_pin_cache_() { memset(this->read_cache_valid_, 0x00, CACHE_SIZE_BYTES); } - static const uint8_t BITS_PER_BYTE = 8; - std::array read_cache_invalidated_{}; + static constexpr uint16_t BITS_PER_BYTE = 8; + static constexpr uint16_t BANK_SIZE = sizeof(T) * BITS_PER_BYTE; + static constexpr size_t BANKS = N / BANK_SIZE; + static constexpr size_t CACHE_SIZE_BYTES = BANKS * sizeof(T); + + T read_cache_valid_[BANKS]{0}; }; -} // namespace gpio_expander -} // namespace esphome +} // namespace esphome::gpio_expander diff --git a/esphome/components/gps/__init__.py b/esphome/components/gps/__init__.py index 7ccd82dec3..a872cf7015 100644 --- a/esphome/components/gps/__init__.py +++ b/esphome/components/gps/__init__.py @@ -84,7 +84,6 @@ CONFIG_SCHEMA = cv.All( ) .extend(cv.polling_component_schema("20s")) .extend(uart.UART_DEVICE_SCHEMA), - cv.only_with_arduino, ) FINAL_VALIDATE_SCHEMA = uart.final_validate_device_schema("gps", require_rx=True) @@ -123,4 +122,9 @@ async def to_code(config): cg.add(var.set_hdop_sensor(sens)) # https://platformio.org/lib/show/1655/TinyGPSPlus - cg.add_library("mikalhart/TinyGPSPlus", "1.1.0") + # Using fork of TinyGPSPlus patched to build on ESP-IDF + cg.add_library( + "TinyGPSPlus", + None, + "https://github.com/esphome/TinyGPSPlus.git#v1.1.0", + ) diff --git a/esphome/components/gps/gps.cpp b/esphome/components/gps/gps.cpp index 9dcb351b39..65cddcd984 100644 --- a/esphome/components/gps/gps.cpp +++ b/esphome/components/gps/gps.cpp @@ -1,5 +1,3 @@ -#ifdef USE_ARDUINO - #include "gps.h" #include "esphome/core/log.h" @@ -22,73 +20,76 @@ void GPS::dump_config() { } void GPS::update() { - if (this->latitude_sensor_ != nullptr) + if (this->latitude_sensor_ != nullptr) { this->latitude_sensor_->publish_state(this->latitude_); + } - if (this->longitude_sensor_ != nullptr) + if (this->longitude_sensor_ != nullptr) { this->longitude_sensor_->publish_state(this->longitude_); + } - if (this->speed_sensor_ != nullptr) + if (this->speed_sensor_ != nullptr) { this->speed_sensor_->publish_state(this->speed_); + } - if (this->course_sensor_ != nullptr) + if (this->course_sensor_ != nullptr) { this->course_sensor_->publish_state(this->course_); + } - if (this->altitude_sensor_ != nullptr) + if (this->altitude_sensor_ != nullptr) { this->altitude_sensor_->publish_state(this->altitude_); + } - if (this->satellites_sensor_ != nullptr) + if (this->satellites_sensor_ != nullptr) { this->satellites_sensor_->publish_state(this->satellites_); + } - if (this->hdop_sensor_ != nullptr) + if (this->hdop_sensor_ != nullptr) { this->hdop_sensor_->publish_state(this->hdop_); + } } void GPS::loop() { while (this->available() > 0 && !this->has_time_) { - if (this->tiny_gps_.encode(this->read())) { - if (this->tiny_gps_.location.isUpdated()) { - this->latitude_ = this->tiny_gps_.location.lat(); - this->longitude_ = this->tiny_gps_.location.lng(); + if (!this->tiny_gps_.encode(this->read())) { + continue; + } + if (this->tiny_gps_.location.isUpdated()) { + this->latitude_ = this->tiny_gps_.location.lat(); + this->longitude_ = this->tiny_gps_.location.lng(); + ESP_LOGV(TAG, "Latitude, Longitude: %.6f°, %.6f°", this->latitude_, this->longitude_); + } - ESP_LOGD(TAG, "Location:"); - ESP_LOGD(TAG, " Lat: %.6f °", this->latitude_); - ESP_LOGD(TAG, " Lon: %.6f °", this->longitude_); - } + if (this->tiny_gps_.speed.isUpdated()) { + this->speed_ = this->tiny_gps_.speed.kmph(); + ESP_LOGV(TAG, "Speed: %.3f km/h", this->speed_); + } - if (this->tiny_gps_.speed.isUpdated()) { - this->speed_ = this->tiny_gps_.speed.kmph(); - ESP_LOGD(TAG, "Speed: %.3f km/h", this->speed_); - } + if (this->tiny_gps_.course.isUpdated()) { + this->course_ = this->tiny_gps_.course.deg(); + ESP_LOGV(TAG, "Course: %.2f°", this->course_); + } - if (this->tiny_gps_.course.isUpdated()) { - this->course_ = this->tiny_gps_.course.deg(); - ESP_LOGD(TAG, "Course: %.2f °", this->course_); - } + if (this->tiny_gps_.altitude.isUpdated()) { + this->altitude_ = this->tiny_gps_.altitude.meters(); + ESP_LOGV(TAG, "Altitude: %.2f m", this->altitude_); + } - if (this->tiny_gps_.altitude.isUpdated()) { - this->altitude_ = this->tiny_gps_.altitude.meters(); - ESP_LOGD(TAG, "Altitude: %.2f m", this->altitude_); - } + if (this->tiny_gps_.satellites.isUpdated()) { + this->satellites_ = this->tiny_gps_.satellites.value(); + ESP_LOGV(TAG, "Satellites: %d", this->satellites_); + } - if (this->tiny_gps_.satellites.isUpdated()) { - this->satellites_ = this->tiny_gps_.satellites.value(); - ESP_LOGD(TAG, "Satellites: %d", this->satellites_); - } + if (this->tiny_gps_.hdop.isUpdated()) { + this->hdop_ = this->tiny_gps_.hdop.hdop(); + ESP_LOGV(TAG, "HDOP: %.3f", this->hdop_); + } - if (this->tiny_gps_.hdop.isUpdated()) { - this->hdop_ = this->tiny_gps_.hdop.hdop(); - ESP_LOGD(TAG, "HDOP: %.3f", this->hdop_); - } - - for (auto *listener : this->listeners_) { - listener->on_update(this->tiny_gps_); - } + for (auto *listener : this->listeners_) { + listener->on_update(this->tiny_gps_); } } } } // namespace gps } // namespace esphome - -#endif // USE_ARDUINO diff --git a/esphome/components/gps/gps.h b/esphome/components/gps/gps.h index 7bc23ed1e0..36923c68be 100644 --- a/esphome/components/gps/gps.h +++ b/esphome/components/gps/gps.h @@ -1,10 +1,8 @@ #pragma once -#ifdef USE_ARDUINO - -#include "esphome/core/component.h" -#include "esphome/components/uart/uart.h" #include "esphome/components/sensor/sensor.h" +#include "esphome/components/uart/uart.h" +#include "esphome/core/component.h" #include #include @@ -53,8 +51,9 @@ class GPS : public PollingComponent, public uart::UARTDevice { float speed_{NAN}; float course_{NAN}; float altitude_{NAN}; - uint16_t satellites_{0}; float hdop_{NAN}; + uint16_t satellites_{0}; + bool has_time_{false}; sensor::Sensor *latitude_sensor_{nullptr}; sensor::Sensor *longitude_sensor_{nullptr}; @@ -64,12 +63,9 @@ class GPS : public PollingComponent, public uart::UARTDevice { sensor::Sensor *satellites_sensor_{nullptr}; sensor::Sensor *hdop_sensor_{nullptr}; - bool has_time_{false}; TinyGPSPlus tiny_gps_; std::vector listeners_{}; }; } // namespace gps } // namespace esphome - -#endif // USE_ARDUINO diff --git a/esphome/components/gps/time/gps_time.cpp b/esphome/components/gps/time/gps_time.cpp index 0f1b989f77..cff8c1fb07 100644 --- a/esphome/components/gps/time/gps_time.cpp +++ b/esphome/components/gps/time/gps_time.cpp @@ -1,5 +1,3 @@ -#ifdef USE_ARDUINO - #include "gps_time.h" #include "esphome/core/log.h" @@ -9,12 +7,10 @@ namespace gps { static const char *const TAG = "gps.time"; void GPSTime::from_tiny_gps_(TinyGPSPlus &tiny_gps) { - if (!tiny_gps.time.isValid() || !tiny_gps.date.isValid()) - return; - if (!tiny_gps.time.isUpdated() || !tiny_gps.date.isUpdated()) - return; - if (tiny_gps.date.year() < 2019) + if (!tiny_gps.time.isValid() || !tiny_gps.date.isValid() || !tiny_gps.time.isUpdated() || + !tiny_gps.date.isUpdated() || tiny_gps.date.year() < 2025) { return; + } ESPTime val{}; val.year = tiny_gps.date.year(); @@ -34,5 +30,3 @@ void GPSTime::from_tiny_gps_(TinyGPSPlus &tiny_gps) { } // namespace gps } // namespace esphome - -#endif // USE_ARDUINO diff --git a/esphome/components/gps/time/gps_time.h b/esphome/components/gps/time/gps_time.h index d0d1db83b5..a8414f0015 100644 --- a/esphome/components/gps/time/gps_time.h +++ b/esphome/components/gps/time/gps_time.h @@ -1,10 +1,8 @@ #pragma once -#ifdef USE_ARDUINO - -#include "esphome/core/component.h" -#include "esphome/components/time/real_time_clock.h" #include "esphome/components/gps/gps.h" +#include "esphome/components/time/real_time_clock.h" +#include "esphome/core/component.h" namespace esphome { namespace gps { @@ -13,8 +11,9 @@ class GPSTime : public time::RealTimeClock, public GPSListener { public: void update() override { this->from_tiny_gps_(this->get_tiny_gps()); }; void on_update(TinyGPSPlus &tiny_gps) override { - if (!this->has_time_) + if (!this->has_time_) { this->from_tiny_gps_(tiny_gps); + } } protected: @@ -24,5 +23,3 @@ class GPSTime : public time::RealTimeClock, public GPSListener { } // namespace gps } // namespace esphome - -#endif // USE_ARDUINO diff --git a/esphome/components/graph/__init__.py b/esphome/components/graph/__init__.py index 6e8ba44bec..d72fe40dd2 100644 --- a/esphome/components/graph/__init__.py +++ b/esphome/components/graph/__init__.py @@ -116,7 +116,7 @@ GRAPH_SCHEMA = cv.Schema( def _relocate_fields_to_subfolder(config, subfolder, subschema): - fields = [k.schema for k in subschema.schema.keys()] + fields = [k.schema for k in subschema.schema] fields.remove(CONF_ID) if subfolder in config: # Ensure no ambiguous fields in base of config diff --git a/esphome/components/graph/graph.cpp b/esphome/components/graph/graph.cpp index 5abf2ade0d..ac6ace96ee 100644 --- a/esphome/components/graph/graph.cpp +++ b/esphome/components/graph/graph.cpp @@ -179,7 +179,7 @@ void Graph::draw(Display *buff, uint16_t x_offset, uint16_t y_offset, Color colo if (b) { int16_t y = (int16_t) roundf((this->height_ - 1) * (1.0 - v)) - thick / 2 + y_offset; auto draw_pixel_at = [&buff, c, y_offset, this](int16_t x, int16_t y) { - if (y >= y_offset && y < y_offset + this->height_) + if (y >= y_offset && static_cast(y) < y_offset + this->height_) buff->draw_pixel_at(x, y, c); }; if (!continuous || !has_prev || !prev_b || (abs(y - prev_y) <= thick)) { diff --git a/esphome/components/graphical_display_menu/graphical_display_menu.cpp b/esphome/components/graphical_display_menu/graphical_display_menu.cpp index 1a29536b46..2b120a746f 100644 --- a/esphome/components/graphical_display_menu/graphical_display_menu.cpp +++ b/esphome/components/graphical_display_menu/graphical_display_menu.cpp @@ -116,7 +116,7 @@ void GraphicalDisplayMenu::draw_menu_internal_(display::Display *display, const int number_items_fit_to_screen = 0; const int max_item_index = this->displayed_item_->items_size() - 1; - for (size_t i = 0; i <= max_item_index; i++) { + for (size_t i = 0; max_item_index >= 0 && i <= static_cast(max_item_index); i++) { const auto *item = this->displayed_item_->get_item(i); const bool selected = i == this->cursor_index_; const display::Rect item_dimensions = this->measure_item(display, item, bounds, selected); @@ -174,7 +174,8 @@ void GraphicalDisplayMenu::draw_menu_internal_(display::Display *display, const display->filled_rectangle(bounds->x, bounds->y, max_width, total_height, this->background_color_); auto y_offset = bounds->y; - for (size_t i = first_item_index; i <= last_item_index; i++) { + for (size_t i = static_cast(first_item_index); + last_item_index >= 0 && i <= static_cast(last_item_index); i++) { const auto *item = this->displayed_item_->get_item(i); const bool selected = i == this->cursor_index_; display::Rect dimensions = menu_dimensions[i]; diff --git a/esphome/components/grove_gas_mc_v2/grove_gas_mc_v2.cpp b/esphome/components/grove_gas_mc_v2/grove_gas_mc_v2.cpp index 361f3e04fd..b0f3429314 100644 --- a/esphome/components/grove_gas_mc_v2/grove_gas_mc_v2.cpp +++ b/esphome/components/grove_gas_mc_v2/grove_gas_mc_v2.cpp @@ -33,7 +33,6 @@ bool GroveGasMultichannelV2Component::read_sensor_(uint8_t address, sensor::Sens } void GroveGasMultichannelV2Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // Before reading sensor values, must preheat sensor if (!(this->write_bytes(GROVE_GAS_MC_V2_HEAT_ON, {}))) { this->mark_failed(); @@ -58,11 +57,11 @@ void GroveGasMultichannelV2Component::update() { void GroveGasMultichannelV2Component::dump_config() { ESP_LOGCONFIG(TAG, "Grove Multichannel Gas Sensor V2"); LOG_I2C_DEVICE(this) - LOG_UPDATE_INTERVAL(this) - LOG_SENSOR(" ", "Nitrogen Dioxide", this->nitrogen_dioxide_sensor_) - LOG_SENSOR(" ", "Ethanol", this->ethanol_sensor_) - LOG_SENSOR(" ", "Carbon Monoxide", this->carbon_monoxide_sensor_) - LOG_SENSOR(" ", "TVOC", this->tvoc_sensor_) + LOG_UPDATE_INTERVAL(this); + LOG_SENSOR(" ", "Nitrogen Dioxide", this->nitrogen_dioxide_sensor_); + LOG_SENSOR(" ", "Ethanol", this->ethanol_sensor_); + LOG_SENSOR(" ", "Carbon Monoxide", this->carbon_monoxide_sensor_); + LOG_SENSOR(" ", "TVOC", this->tvoc_sensor_); if (this->is_failed()) { switch (this->error_code_) { diff --git a/esphome/components/grove_tb6612fng/grove_tb6612fng.cpp b/esphome/components/grove_tb6612fng/grove_tb6612fng.cpp index 0dfb8478e7..a249984647 100644 --- a/esphome/components/grove_tb6612fng/grove_tb6612fng.cpp +++ b/esphome/components/grove_tb6612fng/grove_tb6612fng.cpp @@ -24,7 +24,6 @@ void GroveMotorDriveTB6612FNG::dump_config() { } void GroveMotorDriveTB6612FNG::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (!this->standby()) { this->mark_failed(); return; diff --git a/esphome/components/gt911/touchscreen/gt911_touchscreen.cpp b/esphome/components/gt911/touchscreen/gt911_touchscreen.cpp index 1cead70181..4810867d4b 100644 --- a/esphome/components/gt911/touchscreen/gt911_touchscreen.cpp +++ b/esphome/components/gt911/touchscreen/gt911_touchscreen.cpp @@ -8,6 +8,8 @@ namespace gt911 { static const char *const TAG = "gt911.touchscreen"; +static const uint8_t PRIMARY_ADDRESS = 0x5D; // default I2C address for GT911 +static const uint8_t SECONDARY_ADDRESS = 0x14; // secondary I2C address for GT911 static const uint8_t GET_TOUCH_STATE[2] = {0x81, 0x4E}; static const uint8_t CLEAR_TOUCH_STATE[3] = {0x81, 0x4E, 0x00}; static const uint8_t GET_TOUCHES[2] = {0x81, 0x4F}; @@ -18,43 +20,45 @@ static const size_t MAX_BUTTONS = 4; // max number of buttons scanned #define ERROR_CHECK(err) \ if ((err) != i2c::ERROR_OK) { \ - ESP_LOGE(TAG, "Failed to communicate!"); \ - this->status_set_warning(); \ + this->status_set_warning(LOG_STR(ESP_LOG_MSG_COMM_FAIL)); \ return; \ } void GT911Touchscreen::setup() { - i2c::ErrorCode err; - ESP_LOGCONFIG(TAG, "Running setup"); if (this->reset_pin_ != nullptr) { this->reset_pin_->setup(); this->reset_pin_->digital_write(false); if (this->interrupt_pin_ != nullptr) { - // The interrupt pin is used as an input during reset to select the I2C address. + // temporarily set the interrupt pin to output to control address selection this->interrupt_pin_->pin_mode(gpio::FLAG_OUTPUT); - this->interrupt_pin_->setup(); this->interrupt_pin_->digital_write(false); } delay(2); - this->reset_pin_->digital_write(true); - delay(50); // NOLINT - if (this->interrupt_pin_ != nullptr) { - this->interrupt_pin_->pin_mode(gpio::FLAG_INPUT); - this->interrupt_pin_->setup(); - } + this->reset_pin_->digital_write(true); // wait 50ms after reset + this->set_timeout(50, [this] { this->setup_internal_(); }); + return; + } + this->setup_internal_(); +} + +void GT911Touchscreen::setup_internal_() { + if (this->interrupt_pin_ != nullptr) { + // set pre-configured input mode + this->interrupt_pin_->setup(); } // check the configuration of the int line. uint8_t data[4]; - err = this->write(GET_SWITCHES, 2); + i2c::ErrorCode err = this->write(GET_SWITCHES, sizeof(GET_SWITCHES)); + if (err != i2c::ERROR_OK && this->address_ == PRIMARY_ADDRESS) { + this->address_ = SECONDARY_ADDRESS; + err = this->write(GET_SWITCHES, sizeof(GET_SWITCHES)); + } if (err == i2c::ERROR_OK) { err = this->read(data, 1); if (err == i2c::ERROR_OK) { - ESP_LOGD(TAG, "Read from switches: 0x%02X", data[0]); + ESP_LOGD(TAG, "Switches ADDR: 0x%02X DATA: 0x%02X", this->address_, data[0]); if (this->interrupt_pin_ != nullptr) { - // datasheet says NOT to use pullup/down on the int line. - this->interrupt_pin_->pin_mode(gpio::FLAG_INPUT); - this->interrupt_pin_->setup(); this->attach_interrupt_(this->interrupt_pin_, (data[0] & 1) ? gpio::INTERRUPT_FALLING_EDGE : gpio::INTERRUPT_RISING_EDGE); } @@ -63,7 +67,7 @@ void GT911Touchscreen::setup() { if (this->x_raw_max_ == 0 || this->y_raw_max_ == 0) { // no calibration? Attempt to read the max values from the touchscreen. if (err == i2c::ERROR_OK) { - err = this->write(GET_MAX_VALUES, 2); + err = this->write(GET_MAX_VALUES, sizeof(GET_MAX_VALUES)); if (err == i2c::ERROR_OK) { err = this->read(data, sizeof(data)); if (err == i2c::ERROR_OK) { @@ -75,26 +79,29 @@ void GT911Touchscreen::setup() { } } if (err != i2c::ERROR_OK) { - ESP_LOGE(TAG, "Failed to read calibration values from touchscreen!"); - this->mark_failed(); + this->mark_failed("Calibration error"); return; } } + if (err != i2c::ERROR_OK) { - ESP_LOGE(TAG, "Failed to communicate!"); - this->mark_failed(); + this->mark_failed(ESP_LOG_MSG_COMM_FAIL); return; } - - ESP_LOGCONFIG(TAG, "GT911 Touchscreen setup complete"); + this->setup_done_ = true; } void GT911Touchscreen::update_touches() { + this->skip_update_ = true; // skip send touch events by default, set to false after successful error checks + if (!this->setup_done_) { + return; + } + i2c::ErrorCode err; uint8_t touch_state = 0; uint8_t data[MAX_TOUCHES + 1][8]; // 8 bytes each for each point, plus extra space for the key byte - err = this->write(GET_TOUCH_STATE, sizeof(GET_TOUCH_STATE), false); + err = this->write(GET_TOUCH_STATE, sizeof(GET_TOUCH_STATE)); ERROR_CHECK(err); err = this->read(&touch_state, 1); ERROR_CHECK(err); @@ -102,16 +109,16 @@ void GT911Touchscreen::update_touches() { uint8_t num_of_touches = touch_state & 0x07; if ((touch_state & 0x80) == 0 || num_of_touches > MAX_TOUCHES) { - this->skip_update_ = true; // skip send touch events, touchscreen is not ready yet. return; } - err = this->write(GET_TOUCHES, sizeof(GET_TOUCHES), false); + err = this->write(GET_TOUCHES, sizeof(GET_TOUCHES)); ERROR_CHECK(err); // num_of_touches is guaranteed to be 0..5. Also read the key data err = this->read(data[0], sizeof(data[0]) * num_of_touches + 1); ERROR_CHECK(err); + this->skip_update_ = false; // All error checks passed, send touch events for (uint8_t i = 0; i != num_of_touches; i++) { uint16_t id = data[i][0]; uint16_t x = encode_uint16(data[i][2], data[i][1]); @@ -132,6 +139,7 @@ void GT911Touchscreen::dump_config() { ESP_LOGCONFIG(TAG, "GT911 Touchscreen:"); LOG_I2C_DEVICE(this); LOG_PIN(" Interrupt Pin: ", this->interrupt_pin_); + LOG_PIN(" Reset Pin: ", this->reset_pin_); } } // namespace gt911 diff --git a/esphome/components/gt911/touchscreen/gt911_touchscreen.h b/esphome/components/gt911/touchscreen/gt911_touchscreen.h index 17636a2ada..85025b5522 100644 --- a/esphome/components/gt911/touchscreen/gt911_touchscreen.h +++ b/esphome/components/gt911/touchscreen/gt911_touchscreen.h @@ -15,8 +15,20 @@ class GT911ButtonListener { class GT911Touchscreen : public touchscreen::Touchscreen, public i2c::I2CDevice { public: + /// @brief Initialize the GT911 touchscreen. + /// + /// If @ref reset_pin_ is set, the touchscreen will be hardware reset, + /// and the rest of the setup will be scheduled to run 50ms later using @ref set_timeout() + /// to allow the device to stabilize after reset. + /// + /// If @ref interrupt_pin_ is set, it will be temporarily configured during reset + /// to control I2C address selection. + /// + /// After the timeout, or immediately if no reset is performed, @ref setup_internal_() + /// is called to complete the initialization. void setup() override; void dump_config() override; + bool can_proceed() override { return this->setup_done_; } void set_interrupt_pin(InternalGPIOPin *pin) { this->interrupt_pin_ = pin; } void set_reset_pin(GPIOPin *pin) { this->reset_pin_ = pin; } @@ -25,8 +37,20 @@ class GT911Touchscreen : public touchscreen::Touchscreen, public i2c::I2CDevice protected: void update_touches() override; - InternalGPIOPin *interrupt_pin_{}; - GPIOPin *reset_pin_{}; + /// @brief Perform the internal setup routine for the GT911 touchscreen. + /// + /// This function checks the I2C address, configures the interrupt pin (if available), + /// reads the touchscreen mode from the controller, and attempts to read calibration + /// data (maximum X and Y values) if not already set. + /// + /// On success, sets @ref setup_done_ to true. + /// On failure, calls @ref mark_failed() with an appropriate error message. + void setup_internal_(); + /// @brief True if the touchscreen setup has completed successfully. + bool setup_done_{false}; + + InternalGPIOPin *interrupt_pin_{nullptr}; + GPIOPin *reset_pin_{nullptr}; std::vector button_listeners_; uint8_t button_state_{0xFF}; // last button state. Initial FF guarantees first update. }; diff --git a/esphome/components/haier/climate.py b/esphome/components/haier/climate.py index 0393c263d4..8c3649058f 100644 --- a/esphome/components/haier/climate.py +++ b/esphome/components/haier/climate.py @@ -330,8 +330,7 @@ HAIER_HON_BASE_ACTION_SCHEMA = automation.maybe_simple_id( ) async def display_action_to_code(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + return cg.new_Pvariable(action_id, template_arg, paren) @automation.register_action( @@ -342,8 +341,7 @@ async def display_action_to_code(config, action_id, template_arg, args): ) async def beeper_action_to_code(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + return cg.new_Pvariable(action_id, template_arg, paren) # Start self cleaning or steri-cleaning action action @@ -359,8 +357,7 @@ async def beeper_action_to_code(config, action_id, template_arg, args): ) async def start_cleaning_to_code(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + return cg.new_Pvariable(action_id, template_arg, paren) # Set vertical airflow direction action @@ -417,8 +414,7 @@ async def haier_set_horizontal_airflow_to_code(config, action_id, template_arg, ) async def health_action_to_code(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + return cg.new_Pvariable(action_id, template_arg, paren) @automation.register_action( @@ -432,8 +428,7 @@ async def health_action_to_code(config, action_id, template_arg, args): ) async def power_action_to_code(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + return cg.new_Pvariable(action_id, template_arg, paren) def _final_validate(config): diff --git a/esphome/components/haier/haier_base.cpp b/esphome/components/haier/haier_base.cpp index a784accdf4..55a2454fca 100644 --- a/esphome/components/haier/haier_base.cpp +++ b/esphome/components/haier/haier_base.cpp @@ -242,7 +242,6 @@ haier_protocol::HandlerError HaierClimateBase::timeout_default_handler_(haier_pr } void HaierClimateBase::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // Set timestamp here to give AC time to boot this->last_request_timestamp_ = std::chrono::steady_clock::now(); this->set_phase(ProtocolPhases::SENDING_INIT_1); @@ -352,7 +351,7 @@ ClimateTraits HaierClimateBase::traits() { return traits_; } void HaierClimateBase::initialization() { constexpr uint32_t restore_settings_version = 0xA77D21EF; this->base_rtc_ = - global_preferences->make_preference(this->get_object_id_hash() ^ restore_settings_version); + global_preferences->make_preference(this->get_preference_hash() ^ restore_settings_version); HaierBaseSettings recovered; if (!this->base_rtc_.load(&recovered)) { recovered = {false, true}; diff --git a/esphome/components/haier/hon_climate.cpp b/esphome/components/haier/hon_climate.cpp index fd2d6a5800..76558f2ebb 100644 --- a/esphome/components/haier/hon_climate.cpp +++ b/esphome/components/haier/hon_climate.cpp @@ -213,7 +213,7 @@ haier_protocol::HandlerError HonClimate::status_handler_(haier_protocol::FrameTy this->real_control_packet_size_); this->status_message_callback_.call((const char *) data, data_size); } else { - ESP_LOGW(TAG, "Status packet too small: %d (should be >= %d)", data_size, this->real_control_packet_size_); + ESP_LOGW(TAG, "Status packet too small: %zu (should be >= %zu)", data_size, this->real_control_packet_size_); } switch (this->protocol_phase_) { case ProtocolPhases::SENDING_FIRST_STATUS_REQUEST: @@ -516,7 +516,7 @@ void HonClimate::initialization() { HaierClimateBase::initialization(); constexpr uint32_t restore_settings_version = 0x57EB59DDUL; this->hon_rtc_ = - global_preferences->make_preference(this->get_object_id_hash() ^ restore_settings_version); + global_preferences->make_preference(this->get_preference_hash() ^ restore_settings_version); HonSettings recovered; if (this->hon_rtc_.load(&recovered)) { this->settings_ = recovered; @@ -827,7 +827,7 @@ haier_protocol::HandlerError HonClimate::process_status_message_(const uint8_t * size_t expected_size = 2 + this->status_message_header_size_ + this->real_control_packet_size_ + this->real_sensors_packet_size_; if (size < expected_size) { - ESP_LOGW(TAG, "Unexpected message size %d (expexted >= %d)", size, expected_size); + ESP_LOGW(TAG, "Unexpected message size %u (expexted >= %zu)", size, expected_size); return haier_protocol::HandlerError::WRONG_MESSAGE_STRUCTURE; } uint16_t subtype = (((uint16_t) packet_buffer[0]) << 8) + packet_buffer[1]; diff --git a/esphome/components/haier/hon_climate.h b/esphome/components/haier/hon_climate.h index 58173f8154..a567ab1d89 100644 --- a/esphome/components/haier/hon_climate.h +++ b/esphome/components/haier/hon_climate.h @@ -178,7 +178,7 @@ class HonClimate : public HaierClimateBase { int extra_control_packet_bytes_{0}; int extra_sensors_packet_bytes_{4}; int status_message_header_size_{0}; - int real_control_packet_size_{sizeof(hon_protocol::HaierPacketControl)}; + size_t real_control_packet_size_{sizeof(hon_protocol::HaierPacketControl)}; int real_sensors_packet_size_{sizeof(hon_protocol::HaierPacketSensors) + 4}; HonControlMethod control_method_; std::queue control_messages_queue_; diff --git a/esphome/components/hbridge/switch/hbridge_switch.cpp b/esphome/components/hbridge/switch/hbridge_switch.cpp index 2a1afa48c5..55012fed21 100644 --- a/esphome/components/hbridge/switch/hbridge_switch.cpp +++ b/esphome/components/hbridge/switch/hbridge_switch.cpp @@ -10,8 +10,6 @@ static const char *const TAG = "switch.hbridge"; float HBridgeSwitch::get_setup_priority() const { return setup_priority::HARDWARE; } void HBridgeSwitch::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); - optional initial_state = this->get_initial_state_with_restore_mode(); // Like GPIOSwitch does, set the pin state both before and after pin setup() diff --git a/esphome/components/hdc1080/hdc1080.cpp b/esphome/components/hdc1080/hdc1080.cpp index 956d01ed82..71b7cd7e6e 100644 --- a/esphome/components/hdc1080/hdc1080.cpp +++ b/esphome/components/hdc1080/hdc1080.cpp @@ -7,26 +7,20 @@ namespace hdc1080 { static const char *const TAG = "hdc1080"; -static const uint8_t HDC1080_ADDRESS = 0x40; // 0b1000000 from datasheet static const uint8_t HDC1080_CMD_CONFIGURATION = 0x02; static const uint8_t HDC1080_CMD_TEMPERATURE = 0x00; static const uint8_t HDC1080_CMD_HUMIDITY = 0x01; void HDC1080Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); + const uint8_t config[2] = {0x00, 0x00}; // resolution 14bit for both humidity and temperature - const uint8_t data[2] = { - 0b00000000, // resolution 14bit for both humidity and temperature - 0b00000000 // reserved - }; - - if (!this->write_bytes(HDC1080_CMD_CONFIGURATION, data, 2)) { - // as instruction is same as powerup defaults (for now), interpret as warning if this fails - ESP_LOGW(TAG, "HDC1080 initial config instruction error"); - this->status_set_warning(); + // if configuration fails - there is a problem + if (this->write_register(HDC1080_CMD_CONFIGURATION, config, 2) != i2c::ERROR_OK) { + this->mark_failed(); return; } } + void HDC1080Component::dump_config() { ESP_LOGCONFIG(TAG, "HDC1080:"); LOG_I2C_DEVICE(this); @@ -37,39 +31,51 @@ void HDC1080Component::dump_config() { LOG_SENSOR(" ", "Temperature", this->temperature_); LOG_SENSOR(" ", "Humidity", this->humidity_); } + void HDC1080Component::update() { - uint16_t raw_temp; + // regardless of what sensor/s are defined in yaml configuration + // the hdc1080 setup configuration used, requires both temperature and humidity to be read + + this->status_clear_warning(); + if (this->write(&HDC1080_CMD_TEMPERATURE, 1) != i2c::ERROR_OK) { this->status_set_warning(); return; } - delay(20); - if (this->read(reinterpret_cast(&raw_temp), 2) != i2c::ERROR_OK) { - this->status_set_warning(); - return; - } - raw_temp = i2c::i2ctohs(raw_temp); - float temp = raw_temp * 0.0025177f - 40.0f; // raw * 2^-16 * 165 - 40 - this->temperature_->publish_state(temp); - uint16_t raw_humidity; - if (this->write(&HDC1080_CMD_HUMIDITY, 1) != i2c::ERROR_OK) { - this->status_set_warning(); - return; - } - delay(20); - if (this->read(reinterpret_cast(&raw_humidity), 2) != i2c::ERROR_OK) { - this->status_set_warning(); - return; - } - raw_humidity = i2c::i2ctohs(raw_humidity); - float humidity = raw_humidity * 0.001525879f; // raw * 2^-16 * 100 - this->humidity_->publish_state(humidity); + this->set_timeout(20, [this]() { + uint16_t raw_temperature; + if (this->read(reinterpret_cast(&raw_temperature), 2) != i2c::ERROR_OK) { + this->status_set_warning(); + return; + } - ESP_LOGD(TAG, "Got temperature=%.1f°C humidity=%.1f%%", temp, humidity); - this->status_clear_warning(); + if (this->temperature_ != nullptr) { + raw_temperature = i2c::i2ctohs(raw_temperature); + float temperature = raw_temperature * 0.0025177f - 40.0f; // raw * 2^-16 * 165 - 40 + this->temperature_->publish_state(temperature); + } + + if (this->write(&HDC1080_CMD_HUMIDITY, 1) != i2c::ERROR_OK) { + this->status_set_warning(); + return; + } + + this->set_timeout(20, [this]() { + uint16_t raw_humidity; + if (this->read(reinterpret_cast(&raw_humidity), 2) != i2c::ERROR_OK) { + this->status_set_warning(); + return; + } + + if (this->humidity_ != nullptr) { + raw_humidity = i2c::i2ctohs(raw_humidity); + float humidity = raw_humidity * 0.001525879f; // raw * 2^-16 * 100 + this->humidity_->publish_state(humidity); + } + }); + }); } -float HDC1080Component::get_setup_priority() const { return setup_priority::DATA; } } // namespace hdc1080 } // namespace esphome diff --git a/esphome/components/hdc1080/hdc1080.h b/esphome/components/hdc1080/hdc1080.h index 2ff7b6dc33..7ad0764f1f 100644 --- a/esphome/components/hdc1080/hdc1080.h +++ b/esphome/components/hdc1080/hdc1080.h @@ -12,13 +12,11 @@ class HDC1080Component : public PollingComponent, public i2c::I2CDevice { void set_temperature(sensor::Sensor *temperature) { temperature_ = temperature; } void set_humidity(sensor::Sensor *humidity) { humidity_ = humidity; } - /// Setup the sensor and check for connection. void setup() override; void dump_config() override; - /// Retrieve the latest sensor values. This operation takes approximately 16ms. void update() override; - float get_setup_priority() const override; + float get_setup_priority() const override { return setup_priority::DATA; } protected: sensor::Sensor *temperature_{nullptr}; diff --git a/esphome/components/heatpumpir/climate.py b/esphome/components/heatpumpir/climate.py index 0f9f146ae9..ec6eac670f 100644 --- a/esphome/components/heatpumpir/climate.py +++ b/esphome/components/heatpumpir/climate.py @@ -126,6 +126,6 @@ async def to_code(config): cg.add(var.set_max_temperature(config[CONF_MAX_TEMPERATURE])) cg.add(var.set_min_temperature(config[CONF_MIN_TEMPERATURE])) - cg.add_library("tonia/HeatpumpIR", "1.0.35") - if CORE.is_libretiny: - CORE.add_platformio_option("lib_ignore", "IRremoteESP8266") + cg.add_library("tonia/HeatpumpIR", "1.0.37") + if CORE.is_libretiny or CORE.is_esp32: + CORE.add_platformio_option("lib_ignore", ["IRremoteESP8266"]) diff --git a/esphome/components/hlw8012/hlw8012.cpp b/esphome/components/hlw8012/hlw8012.cpp index ea1d081790..73696bd2a5 100644 --- a/esphome/components/hlw8012/hlw8012.cpp +++ b/esphome/components/hlw8012/hlw8012.cpp @@ -11,7 +11,6 @@ static const uint32_t HLW8012_CLOCK_FREQUENCY = 3579000; void HLW8012Component::setup() { float reference_voltage = 0; - ESP_LOGCONFIG(TAG, "Running setup"); this->sel_pin_->setup(); this->sel_pin_->digital_write(this->current_mode_); this->cf_store_.pulse_counter_setup(this->cf_pin_); @@ -43,11 +42,11 @@ void HLW8012Component::dump_config() { " Current resistor: %.1f mΩ\n" " Voltage Divider: %.1f", this->change_mode_every_, this->current_resistor_ * 1000.0f, this->voltage_divider_); - LOG_UPDATE_INTERVAL(this) - LOG_SENSOR(" ", "Voltage", this->voltage_sensor_) - LOG_SENSOR(" ", "Current", this->current_sensor_) - LOG_SENSOR(" ", "Power", this->power_sensor_) - LOG_SENSOR(" ", "Energy", this->energy_sensor_) + LOG_UPDATE_INTERVAL(this); + LOG_SENSOR(" ", "Voltage", this->voltage_sensor_); + LOG_SENSOR(" ", "Current", this->current_sensor_); + LOG_SENSOR(" ", "Power", this->power_sensor_); + LOG_SENSOR(" ", "Energy", this->energy_sensor_); } float HLW8012Component::get_setup_priority() const { return setup_priority::DATA; } void HLW8012Component::update() { diff --git a/esphome/components/hm3301/hm3301.cpp b/esphome/components/hm3301/hm3301.cpp index b165e361ff..a19d9dd09f 100644 --- a/esphome/components/hm3301/hm3301.cpp +++ b/esphome/components/hm3301/hm3301.cpp @@ -11,7 +11,6 @@ static const uint8_t PM_2_5_VALUE_INDEX = 6; static const uint8_t PM_10_0_VALUE_INDEX = 7; void HM3301Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (i2c::ERROR_OK != this->write(&SELECT_COMM_CMD, 1)) { error_code_ = ERROR_COMM; this->mark_failed(); diff --git a/esphome/components/hmc5883l/hmc5883l.cpp b/esphome/components/hmc5883l/hmc5883l.cpp index fe90b25af2..101493ad91 100644 --- a/esphome/components/hmc5883l/hmc5883l.cpp +++ b/esphome/components/hmc5883l/hmc5883l.cpp @@ -22,7 +22,6 @@ static const uint8_t HMC5883L_REGISTER_IDENTIFICATION_B = 0x0B; static const uint8_t HMC5883L_REGISTER_IDENTIFICATION_C = 0x0C; void HMC5883LComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t id[3]; if (!this->read_byte(HMC5883L_REGISTER_IDENTIFICATION_A, &id[0]) || !this->read_byte(HMC5883L_REGISTER_IDENTIFICATION_B, &id[1]) || diff --git a/esphome/components/homeassistant/__init__.py b/esphome/components/homeassistant/__init__.py index 223d6c18c3..7b23775b47 100644 --- a/esphome/components/homeassistant/__init__.py +++ b/esphome/components/homeassistant/__init__.py @@ -38,3 +38,4 @@ def setup_home_assistant_entity(var, config): cg.add(var.set_entity_id(config[CONF_ENTITY_ID])) if CONF_ATTRIBUTE in config: cg.add(var.set_attribute(config[CONF_ATTRIBUTE])) + cg.add_define("USE_API_HOMEASSISTANT_STATES") diff --git a/esphome/components/homeassistant/number/__init__.py b/esphome/components/homeassistant/number/__init__.py index a6cc615a64..8f760772c3 100644 --- a/esphome/components/homeassistant/number/__init__.py +++ b/esphome/components/homeassistant/number/__init__.py @@ -23,6 +23,7 @@ CONFIG_SCHEMA = ( async def to_code(config): + cg.add_define("USE_API_HOMEASSISTANT_SERVICES") var = await number.new_number( config, min_value=0, diff --git a/esphome/components/homeassistant/number/homeassistant_number.cpp b/esphome/components/homeassistant/number/homeassistant_number.cpp index a7f71c3244..9963f3431d 100644 --- a/esphome/components/homeassistant/number/homeassistant_number.cpp +++ b/esphome/components/homeassistant/number/homeassistant_number.cpp @@ -83,20 +83,23 @@ void HomeassistantNumber::control(float value) { this->publish_state(value); - api::HomeassistantServiceResponse resp; - resp.service = "number.set_value"; + static constexpr auto SERVICE_NAME = StringRef::from_lit("number.set_value"); + static constexpr auto ENTITY_ID_KEY = StringRef::from_lit("entity_id"); + static constexpr auto VALUE_KEY = StringRef::from_lit("value"); - api::HomeassistantServiceMap entity_id; - entity_id.key = "entity_id"; + api::HomeassistantActionRequest resp; + resp.set_service(SERVICE_NAME); + + resp.data.init(2); + auto &entity_id = resp.data.emplace_back(); + entity_id.set_key(ENTITY_ID_KEY); entity_id.value = this->entity_id_; - resp.data.push_back(entity_id); - api::HomeassistantServiceMap entity_value; - entity_value.key = "value"; + auto &entity_value = resp.data.emplace_back(); + entity_value.set_key(VALUE_KEY); entity_value.value = to_string(value); - resp.data.push_back(entity_value); - api::global_api_server->send_homeassistant_service_call(resp); + api::global_api_server->send_homeassistant_action(resp); } } // namespace homeassistant diff --git a/esphome/components/homeassistant/switch/__init__.py b/esphome/components/homeassistant/switch/__init__.py index 384f82bbad..c299a731f2 100644 --- a/esphome/components/homeassistant/switch/__init__.py +++ b/esphome/components/homeassistant/switch/__init__.py @@ -37,6 +37,7 @@ CONFIG_SCHEMA = cv.All( async def to_code(config): + cg.add_define("USE_API_HOMEASSISTANT_SERVICES") var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) await switch.register_switch(var, config) diff --git a/esphome/components/homeassistant/switch/homeassistant_switch.cpp b/esphome/components/homeassistant/switch/homeassistant_switch.cpp index 0451c95069..27d3705fc2 100644 --- a/esphome/components/homeassistant/switch/homeassistant_switch.cpp +++ b/esphome/components/homeassistant/switch/homeassistant_switch.cpp @@ -40,19 +40,23 @@ void HomeassistantSwitch::write_state(bool state) { return; } - api::HomeassistantServiceResponse resp; + static constexpr auto SERVICE_ON = StringRef::from_lit("homeassistant.turn_on"); + static constexpr auto SERVICE_OFF = StringRef::from_lit("homeassistant.turn_off"); + static constexpr auto ENTITY_ID_KEY = StringRef::from_lit("entity_id"); + + api::HomeassistantActionRequest resp; if (state) { - resp.service = "homeassistant.turn_on"; + resp.set_service(SERVICE_ON); } else { - resp.service = "homeassistant.turn_off"; + resp.set_service(SERVICE_OFF); } - api::HomeassistantServiceMap entity_id_kv; - entity_id_kv.key = "entity_id"; + resp.data.init(1); + auto &entity_id_kv = resp.data.emplace_back(); + entity_id_kv.set_key(ENTITY_ID_KEY); entity_id_kv.value = this->entity_id_; - resp.data.push_back(entity_id_kv); - api::global_api_server->send_homeassistant_service_call(resp); + api::global_api_server->send_homeassistant_action(resp); } } // namespace homeassistant diff --git a/esphome/components/honeywellabp/honeywellabp.cpp b/esphome/components/honeywellabp/honeywellabp.cpp index 9252e613dd..4c00f034aa 100644 --- a/esphome/components/honeywellabp/honeywellabp.cpp +++ b/esphome/components/honeywellabp/honeywellabp.cpp @@ -9,10 +9,7 @@ static const char *const TAG = "honeywellabp"; const float MIN_COUNT = 1638.4; // 1638 counts (10% of 2^14 counts or 0x0666) const float MAX_COUNT = 14745.6; // 14745 counts (90% of 2^14 counts or 0x3999) -void HONEYWELLABPSensor::setup() { - ESP_LOGD(TAG, "Setting up Honeywell ABP Sensor "); - this->spi_setup(); -} +void HONEYWELLABPSensor::setup() { this->spi_setup(); } uint8_t HONEYWELLABPSensor::readsensor_() { // Polls the sensor for new data. diff --git a/esphome/components/honeywellabp2_i2c/honeywellabp2.cpp b/esphome/components/honeywellabp2_i2c/honeywellabp2.cpp index 11f5dbc314..f173a1afbd 100644 --- a/esphome/components/honeywellabp2_i2c/honeywellabp2.cpp +++ b/esphome/components/honeywellabp2_i2c/honeywellabp2.cpp @@ -15,7 +15,7 @@ static const char *const TAG = "honeywellabp2"; void HONEYWELLABP2Sensor::read_sensor_data() { if (this->read(raw_data_, 7) != i2c::ERROR_OK) { ESP_LOGE(TAG, ESP_LOG_MSG_COMM_FAIL); - this->status_set_warning("couldn't read sensor data"); + this->status_set_warning(LOG_STR("couldn't read sensor data")); return; } float press_counts = encode_uint24(raw_data_[1], raw_data_[2], raw_data_[3]); // calculate digital pressure counts @@ -31,7 +31,7 @@ void HONEYWELLABP2Sensor::read_sensor_data() { void HONEYWELLABP2Sensor::start_measurement() { if (this->write(i2c_cmd_, 3) != i2c::ERROR_OK) { ESP_LOGE(TAG, ESP_LOG_MSG_COMM_FAIL); - this->status_set_warning("couldn't start measurement"); + this->status_set_warning(LOG_STR("couldn't start measurement")); return; } this->measurement_running_ = true; @@ -40,7 +40,7 @@ void HONEYWELLABP2Sensor::start_measurement() { bool HONEYWELLABP2Sensor::is_measurement_ready() { if (this->read(raw_data_, 1) != i2c::ERROR_OK) { ESP_LOGE(TAG, ESP_LOG_MSG_COMM_FAIL); - this->status_set_warning("couldn't check measurement"); + this->status_set_warning(LOG_STR("couldn't check measurement")); return false; } if ((raw_data_[0] & (0x1 << STATUS_BIT_BUSY)) > 0) { @@ -53,7 +53,7 @@ bool HONEYWELLABP2Sensor::is_measurement_ready() { void HONEYWELLABP2Sensor::measurement_timeout() { ESP_LOGE(TAG, "Timeout!"); this->measurement_running_ = false; - this->status_set_warning("measurement timed out"); + this->status_set_warning(LOG_STR("measurement timed out")); } float HONEYWELLABP2Sensor::get_pressure() { return this->last_pressure_; } diff --git a/esphome/components/host/__init__.py b/esphome/components/host/__init__.py index a67d73fbb7..ba05e497c8 100644 --- a/esphome/components/host/__init__.py +++ b/esphome/components/host/__init__.py @@ -7,6 +7,7 @@ from esphome.const import ( KEY_TARGET_FRAMEWORK, KEY_TARGET_PLATFORM, PLATFORM_HOST, + ThreadModel, ) from esphome.core import CORE @@ -43,6 +44,7 @@ async def to_code(config): cg.add_define("USE_ESPHOME_HOST_MAC_ADDRESS", config[CONF_MAC_ADDRESS].parts) cg.add_build_flag("-std=gnu++20") cg.add_define("ESPHOME_BOARD", "host") + cg.add_define(ThreadModel.MULTI_ATOMICS) cg.add_platformio_option("platform", "platformio/native") cg.add_platformio_option("lib_ldf_mode", "off") cg.add_platformio_option("lib_compat_mode", "strict") diff --git a/esphome/components/host/preferences.h b/esphome/components/host/preferences.h index 6707366517..6b2e7eb8f9 100644 --- a/esphome/components/host/preferences.h +++ b/esphome/components/host/preferences.h @@ -42,9 +42,10 @@ class HostPreferences : public ESPPreferences { if (len > 255) return false; this->setup_(); - if (this->data.count(key) == 0) + auto it = this->data.find(key); + if (it == this->data.end()) return false; - auto vec = this->data[key]; + const auto &vec = it->second; if (vec.size() != len) return false; memcpy(data, vec.data(), len); diff --git a/esphome/components/hte501/hte501.cpp b/esphome/components/hte501/hte501.cpp index 0f97c67f9e..b7d3be63fe 100644 --- a/esphome/components/hte501/hte501.cpp +++ b/esphome/components/hte501/hte501.cpp @@ -8,12 +8,10 @@ namespace hte501 { static const char *const TAG = "hte501"; void HTE501Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t address[] = {0x70, 0x29}; - this->write(address, 2, false); uint8_t identification[9]; - this->read(identification, 9); - if (identification[8] != calc_crc8_(identification, 0, 7)) { + this->write_read(address, sizeof address, identification, sizeof identification); + if (identification[8] != crc8(identification, 8, 0xFF, 0x31, true)) { this->error_code_ = CRC_CHECK_FAILED; this->mark_failed(); return; @@ -43,11 +41,12 @@ void HTE501Component::dump_config() { float HTE501Component::get_setup_priority() const { return setup_priority::DATA; } void HTE501Component::update() { uint8_t address_1[] = {0x2C, 0x1B}; - this->write(address_1, 2, true); + this->write(address_1, 2); this->set_timeout(50, [this]() { uint8_t i2c_response[6]; this->read(i2c_response, 6); - if (i2c_response[2] != calc_crc8_(i2c_response, 0, 1) && i2c_response[5] != calc_crc8_(i2c_response, 3, 4)) { + if (i2c_response[2] != crc8(i2c_response, 2, 0xFF, 0x31, true) && + i2c_response[5] != crc8(i2c_response + 3, 2, 0xFF, 0x31, true)) { this->error_code_ = CRC_CHECK_FAILED; this->status_set_warning(); return; @@ -68,24 +67,5 @@ void HTE501Component::update() { this->status_clear_warning(); }); } - -unsigned char HTE501Component::calc_crc8_(const unsigned char buf[], unsigned char from, unsigned char to) { - unsigned char crc_val = 0xFF; - unsigned char i = 0; - unsigned char j = 0; - for (i = from; i <= to; i++) { - int cur_val = buf[i]; - for (j = 0; j < 8; j++) { - if (((crc_val ^ cur_val) & 0x80) != 0) // If MSBs are not equal - { - crc_val = ((crc_val << 1) ^ 0x31); - } else { - crc_val = (crc_val << 1); - } - cur_val = cur_val << 1; - } - } - return crc_val; -} } // namespace hte501 } // namespace esphome diff --git a/esphome/components/hte501/hte501.h b/esphome/components/hte501/hte501.h index 0d2c952e81..a7072d5bdb 100644 --- a/esphome/components/hte501/hte501.h +++ b/esphome/components/hte501/hte501.h @@ -1,8 +1,8 @@ #pragma once -#include "esphome/core/component.h" -#include "esphome/components/sensor/sensor.h" #include "esphome/components/i2c/i2c.h" +#include "esphome/components/sensor/sensor.h" +#include "esphome/core/component.h" namespace esphome { namespace hte501 { @@ -19,7 +19,6 @@ class HTE501Component : public PollingComponent, public i2c::I2CDevice { void update() override; protected: - unsigned char calc_crc8_(const unsigned char buf[], unsigned char from, unsigned char to); sensor::Sensor *temperature_sensor_; sensor::Sensor *humidity_sensor_; diff --git a/esphome/components/http_request/__init__.py b/esphome/components/http_request/__init__.py index 0d32bc97c2..e428838c83 100644 --- a/esphome/components/http_request/__init__.py +++ b/esphome/components/http_request/__init__.py @@ -5,10 +5,12 @@ from esphome.components.const import CONF_REQUEST_HEADERS from esphome.config_helpers import filter_source_files_from_platform import esphome.config_validation as cv from esphome.const import ( + CONF_CAPTURE_RESPONSE, CONF_ESP8266_DISABLE_SSL_SUPPORT, CONF_ID, CONF_METHOD, CONF_ON_ERROR, + CONF_ON_RESPONSE, CONF_TIMEOUT, CONF_TRIGGER_ID, CONF_URL, @@ -52,12 +54,10 @@ CONF_BUFFER_SIZE_TX = "buffer_size_tx" CONF_CA_CERTIFICATE_PATH = "ca_certificate_path" CONF_MAX_RESPONSE_BUFFER_SIZE = "max_response_buffer_size" -CONF_ON_RESPONSE = "on_response" CONF_HEADERS = "headers" CONF_COLLECT_HEADERS = "collect_headers" CONF_BODY = "body" CONF_JSON = "json" -CONF_CAPTURE_RESPONSE = "capture_response" def validate_url(value): @@ -70,9 +70,8 @@ def validate_url(value): def validate_ssl_verification(config): error_message = "" - if CORE.is_esp32: - if not CORE.using_esp_idf and config[CONF_VERIFY_SSL]: - error_message = "ESPHome supports certificate verification only via ESP-IDF" + if CORE.is_esp32 and not CORE.using_esp_idf and config[CONF_VERIFY_SSL]: + error_message = "ESPHome supports certificate verification only via ESP-IDF" if CORE.is_rp2040 and config[CONF_VERIFY_SSL]: error_message = "ESPHome does not support certificate verification on RP2040" @@ -195,7 +194,7 @@ async def to_code(config): cg.add_define("CPPHTTPLIB_OPENSSL_SUPPORT") elif path := config.get(CONF_CA_CERTIFICATE_PATH): cg.add_define("CPPHTTPLIB_OPENSSL_SUPPORT") - cg.add(var.set_ca_path(path)) + cg.add(var.set_ca_path(str(path))) cg.add_build_flag("-lssl") cg.add_build_flag("-lcrypto") diff --git a/esphome/components/http_request/http_request.h b/esphome/components/http_request/http_request.h index 95515f731a..bb14cc6f51 100644 --- a/esphome/components/http_request/http_request.h +++ b/esphome/components/http_request/http_request.h @@ -167,8 +167,8 @@ class HttpRequestComponent : public Component { } protected: - virtual std::shared_ptr perform(std::string url, std::string method, std::string body, - std::list
request_headers, + virtual std::shared_ptr perform(const std::string &url, const std::string &method, + const std::string &body, const std::list
&request_headers, std::set collect_headers) = 0; const char *useragent_{nullptr}; bool follow_redirects_{}; diff --git a/esphome/components/http_request/http_request_arduino.cpp b/esphome/components/http_request/http_request_arduino.cpp index c009b33c2d..dfdbbd3fab 100644 --- a/esphome/components/http_request/http_request_arduino.cpp +++ b/esphome/components/http_request/http_request_arduino.cpp @@ -14,8 +14,9 @@ namespace http_request { static const char *const TAG = "http_request.arduino"; -std::shared_ptr HttpRequestArduino::perform(std::string url, std::string method, std::string body, - std::list
request_headers, +std::shared_ptr HttpRequestArduino::perform(const std::string &url, const std::string &method, + const std::string &body, + const std::list
&request_headers, std::set collect_headers) { if (!network::is_connected()) { this->status_momentary_error("failed", 1000); diff --git a/esphome/components/http_request/http_request_arduino.h b/esphome/components/http_request/http_request_arduino.h index 44744f8c78..c8208c74d8 100644 --- a/esphome/components/http_request/http_request_arduino.h +++ b/esphome/components/http_request/http_request_arduino.h @@ -31,8 +31,8 @@ class HttpContainerArduino : public HttpContainer { class HttpRequestArduino : public HttpRequestComponent { protected: - std::shared_ptr perform(std::string url, std::string method, std::string body, - std::list
request_headers, + std::shared_ptr perform(const std::string &url, const std::string &method, const std::string &body, + const std::list
&request_headers, std::set collect_headers) override; }; diff --git a/esphome/components/http_request/http_request_host.cpp b/esphome/components/http_request/http_request_host.cpp index 192032c1ac..c20ea552b7 100644 --- a/esphome/components/http_request/http_request_host.cpp +++ b/esphome/components/http_request/http_request_host.cpp @@ -1,7 +1,10 @@ -#include "http_request_host.h" - #ifdef USE_HOST +#define USE_HTTP_REQUEST_HOST_H +#define CPPHTTPLIB_NO_EXCEPTIONS +#include "httplib.h" +#include "http_request_host.h" + #include #include "esphome/components/network/util.h" #include "esphome/components/watchdog/watchdog.h" @@ -14,8 +17,9 @@ namespace http_request { static const char *const TAG = "http_request.host"; -std::shared_ptr HttpRequestHost::perform(std::string url, std::string method, std::string body, - std::list
request_headers, +std::shared_ptr HttpRequestHost::perform(const std::string &url, const std::string &method, + const std::string &body, + const std::list
&request_headers, std::set response_headers) { if (!network::is_connected()) { this->status_momentary_error("failed", 1000); diff --git a/esphome/components/http_request/http_request_host.h b/esphome/components/http_request/http_request_host.h index 49fd3b43fe..fdd72e7ea5 100644 --- a/esphome/components/http_request/http_request_host.h +++ b/esphome/components/http_request/http_request_host.h @@ -1,11 +1,7 @@ #pragma once -#include "http_request.h" - #ifdef USE_HOST - -#define CPPHTTPLIB_NO_EXCEPTIONS -#include "httplib.h" +#include "http_request.h" namespace esphome { namespace http_request { @@ -22,8 +18,8 @@ class HttpContainerHost : public HttpContainer { class HttpRequestHost : public HttpRequestComponent { public: - std::shared_ptr perform(std::string url, std::string method, std::string body, - std::list
request_headers, + std::shared_ptr perform(const std::string &url, const std::string &method, const std::string &body, + const std::list
&request_headers, std::set response_headers) override; void set_ca_path(const char *ca_path) { this->ca_path_ = ca_path; } diff --git a/esphome/components/http_request/http_request_idf.cpp b/esphome/components/http_request/http_request_idf.cpp index 68c06d28f2..a91c0bfc25 100644 --- a/esphome/components/http_request/http_request_idf.cpp +++ b/esphome/components/http_request/http_request_idf.cpp @@ -52,8 +52,9 @@ esp_err_t HttpRequestIDF::http_event_handler(esp_http_client_event_t *evt) { return ESP_OK; } -std::shared_ptr HttpRequestIDF::perform(std::string url, std::string method, std::string body, - std::list
request_headers, +std::shared_ptr HttpRequestIDF::perform(const std::string &url, const std::string &method, + const std::string &body, + const std::list
&request_headers, std::set collect_headers) { if (!network::is_connected()) { this->status_momentary_error("failed", 1000); @@ -157,8 +158,8 @@ std::shared_ptr HttpRequestIDF::perform(std::string url, std::str container->status_code = esp_http_client_get_status_code(client); container->feed_wdt(); container->set_response_headers(user_data.response_headers); + container->duration_ms = millis() - start; if (is_success(container->status_code)) { - container->duration_ms = millis() - start; return container; } @@ -191,8 +192,8 @@ std::shared_ptr HttpRequestIDF::perform(std::string url, std::str container->feed_wdt(); container->status_code = esp_http_client_get_status_code(client); container->feed_wdt(); + container->duration_ms = millis() - start; if (is_success(container->status_code)) { - container->duration_ms = millis() - start; return container; } diff --git a/esphome/components/http_request/http_request_idf.h b/esphome/components/http_request/http_request_idf.h index 5c5b784853..90dee0be68 100644 --- a/esphome/components/http_request/http_request_idf.h +++ b/esphome/components/http_request/http_request_idf.h @@ -37,8 +37,8 @@ class HttpRequestIDF : public HttpRequestComponent { void set_buffer_size_tx(uint16_t buffer_size_tx) { this->buffer_size_tx_ = buffer_size_tx; } protected: - std::shared_ptr perform(std::string url, std::string method, std::string body, - std::list
request_headers, + std::shared_ptr perform(const std::string &url, const std::string &method, const std::string &body, + const std::list
&request_headers, std::set collect_headers) override; // if zero ESP-IDF will use DEFAULT_HTTP_BUF_SIZE uint16_t buffer_size_rx_{}; diff --git a/esphome/components/http_request/httplib.h b/esphome/components/http_request/httplib.h index a2f4436ec7..8b08699702 100644 --- a/esphome/components/http_request/httplib.h +++ b/esphome/components/http_request/httplib.h @@ -3,12 +3,10 @@ /** * NOTE: This is a copy of httplib.h from https://github.com/yhirose/cpp-httplib * - * It has been modified only to add ifdefs for USE_HOST. While it contains many functions unused in ESPHome, + * It has been modified to add ifdefs for USE_HOST. While it contains many functions unused in ESPHome, * it was considered preferable to use it with as few changes as possible, to facilitate future updates. */ -#include "esphome/core/defines.h" - // // httplib.h // @@ -17,6 +15,11 @@ // #ifdef USE_HOST +// Prevent this code being included in main.cpp +#ifdef USE_HTTP_REQUEST_HOST_H + +#include "esphome/core/defines.h" + #ifndef CPPHTTPLIB_HTTPLIB_H #define CPPHTTPLIB_HTTPLIB_H @@ -9687,5 +9690,6 @@ inline SSL_CTX *Client::ssl_context() const { #endif #endif // CPPHTTPLIB_HTTPLIB_H +#endif // USE_HTTP_REQUEST_HOST_H #endif diff --git a/esphome/components/http_request/ota/__init__.py b/esphome/components/http_request/ota/__init__.py index a3f6d5840c..d2c574d8c6 100644 --- a/esphome/components/http_request/ota/__init__.py +++ b/esphome/components/http_request/ota/__init__.py @@ -4,6 +4,7 @@ from esphome.components.ota import BASE_OTA_SCHEMA, OTAComponent, ota_to_code import esphome.config_validation as cv from esphome.const import CONF_ID, CONF_PASSWORD, CONF_URL, CONF_USERNAME from esphome.core import coroutine_with_priority +from esphome.coroutine import CoroPriority from .. import CONF_HTTP_REQUEST_ID, HttpRequestComponent, http_request_ns @@ -40,7 +41,7 @@ CONFIG_SCHEMA = cv.All( ) -@coroutine_with_priority(52.0) +@coroutine_with_priority(CoroPriority.OTA_UPDATES) async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) await ota_to_code(var, config) diff --git a/esphome/components/htu21d/htu21d.cpp b/esphome/components/htu21d/htu21d.cpp index b5d6ad45d5..c5d91d3dd0 100644 --- a/esphome/components/htu21d/htu21d.cpp +++ b/esphome/components/htu21d/htu21d.cpp @@ -18,8 +18,6 @@ static const uint8_t HTU21D_READHEATER_REG_CMD = 0x11; /**< Read Heater Control static const uint8_t HTU21D_REG_HTRE_BIT = 0x02; /**< Control Register Heater Bit */ void HTU21DComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - if (!this->write_bytes(HTU21D_REGISTER_RESET, nullptr, 0)) { this->mark_failed(); return; @@ -59,7 +57,6 @@ void HTU21DComponent::update() { if (this->temperature_ != nullptr) this->temperature_->publish_state(temperature); - this->status_clear_warning(); if (this->write(&HTU21D_REGISTER_HUMIDITY, 1) != i2c::ERROR_OK) { this->status_set_warning(); @@ -81,10 +78,11 @@ void HTU21DComponent::update() { if (this->humidity_ != nullptr) this->humidity_->publish_state(humidity); - int8_t heater_level; + this->status_clear_warning(); // HTU21D does have a heater module but does not have heater level // Setting heater level to 1 in case the heater is ON + uint8_t heater_level = 0; if (this->sensor_model_ == HTU21D_SENSOR_MODEL_HTU21D) { if (this->is_heater_enabled()) { heater_level = 1; @@ -99,34 +97,30 @@ void HTU21DComponent::update() { if (this->heater_ != nullptr) this->heater_->publish_state(heater_level); - this->status_clear_warning(); }); }); } bool HTU21DComponent::is_heater_enabled() { uint8_t raw_heater; - if (this->read_register(HTU21D_REGISTER_STATUS, reinterpret_cast(&raw_heater), 2) != i2c::ERROR_OK) { + if (this->read_register(HTU21D_REGISTER_STATUS, &raw_heater, 1) != i2c::ERROR_OK) { this->status_set_warning(); return false; } - raw_heater = i2c::i2ctohs(raw_heater); - return (bool) (((raw_heater) >> (HTU21D_REG_HTRE_BIT)) & 0x01); + return (bool) ((raw_heater >> HTU21D_REG_HTRE_BIT) & 0x01); } void HTU21DComponent::set_heater(bool status) { uint8_t raw_heater; - if (this->read_register(HTU21D_REGISTER_STATUS, reinterpret_cast(&raw_heater), 2) != i2c::ERROR_OK) { + if (this->read_register(HTU21D_REGISTER_STATUS, &raw_heater, 1) != i2c::ERROR_OK) { this->status_set_warning(); return; } - raw_heater = i2c::i2ctohs(raw_heater); if (status) { - raw_heater |= (1 << (HTU21D_REG_HTRE_BIT)); + raw_heater |= (1 << HTU21D_REG_HTRE_BIT); } else { - raw_heater &= ~(1 << (HTU21D_REG_HTRE_BIT)); + raw_heater &= ~(1 << HTU21D_REG_HTRE_BIT); } - if (this->write_register(HTU21D_WRITERHT_REG_CMD, &raw_heater, 1) != i2c::ERROR_OK) { this->status_set_warning(); return; @@ -140,14 +134,13 @@ void HTU21DComponent::set_heater_level(uint8_t level) { } } -int8_t HTU21DComponent::get_heater_level() { - int8_t raw_heater; - if (this->read_register(HTU21D_READHEATER_REG_CMD, reinterpret_cast(&raw_heater), 2) != i2c::ERROR_OK) { +uint8_t HTU21DComponent::get_heater_level() { + uint8_t raw_heater; + if (this->read_register(HTU21D_READHEATER_REG_CMD, &raw_heater, 1) != i2c::ERROR_OK) { this->status_set_warning(); return 0; } - raw_heater = i2c::i2ctohs(raw_heater); - return raw_heater; + return raw_heater & 0xF; } float HTU21DComponent::get_setup_priority() const { return setup_priority::DATA; } diff --git a/esphome/components/htu21d/htu21d.h b/esphome/components/htu21d/htu21d.h index 8533875d43..9b3831b784 100644 --- a/esphome/components/htu21d/htu21d.h +++ b/esphome/components/htu21d/htu21d.h @@ -26,7 +26,7 @@ class HTU21DComponent : public PollingComponent, public i2c::I2CDevice { bool is_heater_enabled(); void set_heater(bool status); void set_heater_level(uint8_t level); - int8_t get_heater_level(); + uint8_t get_heater_level(); float get_setup_priority() const override; diff --git a/esphome/components/htu31d/htu31d.cpp b/esphome/components/htu31d/htu31d.cpp index 284548ed96..562078aacb 100644 --- a/esphome/components/htu31d/htu31d.cpp +++ b/esphome/components/htu31d/htu31d.cpp @@ -75,8 +75,6 @@ uint8_t compute_crc(uint32_t value) { * I2C. */ void HTU31DComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - if (!this->reset_()) { this->mark_failed(); return; diff --git a/esphome/components/hx711/hx711.cpp b/esphome/components/hx711/hx711.cpp index 0fc8b29604..67ec4549df 100644 --- a/esphome/components/hx711/hx711.cpp +++ b/esphome/components/hx711/hx711.cpp @@ -8,7 +8,6 @@ namespace hx711 { static const char *const TAG = "hx711"; void HX711Sensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); this->sck_pin_->setup(); this->dout_pin_->setup(); this->sck_pin_->digital_write(false); diff --git a/esphome/components/hydreon_rgxx/hydreon_rgxx.cpp b/esphome/components/hydreon_rgxx/hydreon_rgxx.cpp index 9d4680fdf4..4872d68610 100644 --- a/esphome/components/hydreon_rgxx/hydreon_rgxx.cpp +++ b/esphome/components/hydreon_rgxx/hydreon_rgxx.cpp @@ -41,7 +41,6 @@ void HydreonRGxxComponent::dump_config() { } void HydreonRGxxComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); while (this->available() != 0) { this->read(); } diff --git a/esphome/components/hydreon_rgxx/sensor.py b/esphome/components/hydreon_rgxx/sensor.py index 6c9f1e2877..f270b72e24 100644 --- a/esphome/components/hydreon_rgxx/sensor.py +++ b/esphome/components/hydreon_rgxx/sensor.py @@ -66,11 +66,10 @@ PROTOCOL_NAMES = { def _validate(config): for conf, models in SUPPORTED_OPTIONS.items(): - if conf in config: - if config[CONF_MODEL] not in models: - raise cv.Invalid( - f"{conf} is only available on {' and '.join(models)}, not {config[CONF_MODEL]}" - ) + if conf in config and config[CONF_MODEL] not in models: + raise cv.Invalid( + f"{conf} is only available on {' and '.join(models)}, not {config[CONF_MODEL]}" + ) return config diff --git a/esphome/components/i2c/__init__.py b/esphome/components/i2c/__init__.py index 4172b23845..3cfec1e94d 100644 --- a/esphome/components/i2c/__init__.py +++ b/esphome/components/i2c/__init__.py @@ -2,7 +2,6 @@ import logging from esphome import pins import esphome.codegen as cg -from esphome.components import esp32 from esphome.config_helpers import filter_source_files_from_platform import esphome.config_validation as cv from esphome.const import ( @@ -14,14 +13,12 @@ from esphome.const import ( CONF_SCL, CONF_SDA, CONF_TIMEOUT, - KEY_CORE, - KEY_FRAMEWORK_VERSION, PLATFORM_ESP32, PLATFORM_ESP8266, PLATFORM_RP2040, PlatformFramework, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority import esphome.final_validate as fv LOGGER = logging.getLogger(__name__) @@ -48,28 +45,8 @@ def _bus_declare_type(value): def validate_config(config): - if ( - config[CONF_SCAN] - and CORE.is_esp32 - and CORE.using_esp_idf - and esp32.get_esp32_variant() - in [ - esp32.const.VARIANT_ESP32C5, - esp32.const.VARIANT_ESP32C6, - esp32.const.VARIANT_ESP32P4, - ] - ): - version: cv.Version = CORE.data[KEY_CORE][KEY_FRAMEWORK_VERSION] - if version.major == 5 and ( - (version.minor == 3 and version.patch <= 3) - or (version.minor == 4 and version.patch <= 1) - ): - LOGGER.warning( - "There is a bug in esp-idf version %s that breaks I2C scan, I2C scan " - "has been disabled, see https://github.com/esphome/issues/issues/7128", - str(version), - ) - config[CONF_SCAN] = False + if CORE.using_esp_idf: + return cv.require_framework_version(esp_idf=cv.Version(5, 4, 2))(config) return config @@ -97,7 +74,7 @@ CONFIG_SCHEMA = cv.All( ) -@coroutine_with_priority(1.0) +@coroutine_with_priority(CoroPriority.BUS) async def to_code(config): cg.add_global(i2c_ns.using) cg.add_define("USE_I2C") diff --git a/esphome/components/i2c/i2c.cpp b/esphome/components/i2c/i2c.cpp index 2b2190d28b..31c21f398c 100644 --- a/esphome/components/i2c/i2c.cpp +++ b/esphome/components/i2c/i2c.cpp @@ -1,4 +1,6 @@ #include "i2c.h" + +#include "esphome/core/defines.h" #include "esphome/core/log.h" #include @@ -7,38 +9,52 @@ namespace i2c { static const char *const TAG = "i2c"; -ErrorCode I2CDevice::read_register(uint8_t a_register, uint8_t *data, size_t len, bool stop) { - ErrorCode err = this->write(&a_register, 1, stop); - if (err != ERROR_OK) - return err; - return bus_->read(address_, data, len); +void I2CBus::i2c_scan_() { + // suppress logs from the IDF I2C library during the scan +#if defined(USE_ESP32) && defined(USE_LOGGER) + auto previous = esp_log_level_get("*"); + esp_log_level_set("*", ESP_LOG_NONE); +#endif + + for (uint8_t address = 8; address != 120; address++) { + auto err = write_readv(address, nullptr, 0, nullptr, 0); + if (err == ERROR_OK) { + scan_results_.emplace_back(address, true); + } else if (err == ERROR_UNKNOWN) { + scan_results_.emplace_back(address, false); + } + } +#if defined(USE_ESP32) && defined(USE_LOGGER) + esp_log_level_set("*", previous); +#endif } -ErrorCode I2CDevice::read_register16(uint16_t a_register, uint8_t *data, size_t len, bool stop) { +ErrorCode I2CDevice::read_register(uint8_t a_register, uint8_t *data, size_t len) { + return bus_->write_readv(this->address_, &a_register, 1, data, len); +} + +ErrorCode I2CDevice::read_register16(uint16_t a_register, uint8_t *data, size_t len) { a_register = convert_big_endian(a_register); - ErrorCode const err = this->write(reinterpret_cast(&a_register), 2, stop); - if (err != ERROR_OK) - return err; - return bus_->read(address_, data, len); + return bus_->write_readv(this->address_, reinterpret_cast(&a_register), 2, data, len); } -ErrorCode I2CDevice::write_register(uint8_t a_register, const uint8_t *data, size_t len, bool stop) { - WriteBuffer buffers[2]; - buffers[0].data = &a_register; - buffers[0].len = 1; - buffers[1].data = data; - buffers[1].len = len; - return bus_->writev(address_, buffers, 2, stop); +ErrorCode I2CDevice::write_register(uint8_t a_register, const uint8_t *data, size_t len) const { + SmallBufferWithHeapFallback<17> buffer_alloc; // Most I2C writes are <= 16 bytes + uint8_t *buffer = buffer_alloc.get(len + 1); + + buffer[0] = a_register; + std::copy(data, data + len, buffer + 1); + return this->bus_->write_readv(this->address_, buffer, len + 1, nullptr, 0); } -ErrorCode I2CDevice::write_register16(uint16_t a_register, const uint8_t *data, size_t len, bool stop) { - a_register = convert_big_endian(a_register); - WriteBuffer buffers[2]; - buffers[0].data = reinterpret_cast(&a_register); - buffers[0].len = 2; - buffers[1].data = data; - buffers[1].len = len; - return bus_->writev(address_, buffers, 2, stop); +ErrorCode I2CDevice::write_register16(uint16_t a_register, const uint8_t *data, size_t len) const { + SmallBufferWithHeapFallback<18> buffer_alloc; // Most I2C writes are <= 16 bytes + 2 for register + uint8_t *buffer = buffer_alloc.get(len + 2); + + buffer[0] = a_register >> 8; + buffer[1] = a_register; + std::copy(data, data + len, buffer + 2); + return this->bus_->write_readv(this->address_, buffer, len + 2, nullptr, 0); } bool I2CDevice::read_bytes_16(uint8_t a_register, uint16_t *data, uint8_t len) { @@ -49,7 +65,7 @@ bool I2CDevice::read_bytes_16(uint8_t a_register, uint16_t *data, uint8_t len) { return true; } -bool I2CDevice::write_bytes_16(uint8_t a_register, const uint16_t *data, uint8_t len) { +bool I2CDevice::write_bytes_16(uint8_t a_register, const uint16_t *data, uint8_t len) const { // we have to copy in order to be able to change byte order std::unique_ptr temp{new uint16_t[len]}; for (size_t i = 0; i < len; i++) diff --git a/esphome/components/i2c/i2c.h b/esphome/components/i2c/i2c.h index 15f786245b..48a6e751cf 100644 --- a/esphome/components/i2c/i2c.h +++ b/esphome/components/i2c/i2c.h @@ -1,10 +1,10 @@ #pragma once -#include "i2c_bus.h" -#include "esphome/core/helpers.h" -#include "esphome/core/optional.h" #include #include +#include "esphome/core/helpers.h" +#include "esphome/core/optional.h" +#include "i2c_bus.h" namespace esphome { namespace i2c { @@ -161,51 +161,53 @@ class I2CDevice { /// @param data pointer to an array to store the bytes /// @param len length of the buffer = number of bytes to read /// @return an i2c::ErrorCode - ErrorCode read(uint8_t *data, size_t len) { return bus_->read(address_, data, len); } + ErrorCode read(uint8_t *data, size_t len) const { return bus_->write_readv(this->address_, nullptr, 0, data, len); } /// @brief reads an array of bytes from a specific register in the I²C device /// @param a_register an 8 bits internal address of the I²C register to read from /// @param data pointer to an array to store the bytes /// @param len length of the buffer = number of bytes to read - /// @param stop (true/false): True will send a stop message, releasing the bus after - /// transmission. False will send a restart, keeping the connection active. /// @return an i2c::ErrorCode - ErrorCode read_register(uint8_t a_register, uint8_t *data, size_t len, bool stop = true); + ErrorCode read_register(uint8_t a_register, uint8_t *data, size_t len); /// @brief reads an array of bytes from a specific register in the I²C device /// @param a_register the 16 bits internal address of the I²C register to read from /// @param data pointer to an array of bytes to store the information /// @param len length of the buffer = number of bytes to read - /// @param stop (true/false): True will send a stop message, releasing the bus after - /// transmission. False will send a restart, keeping the connection active. /// @return an i2c::ErrorCode - ErrorCode read_register16(uint16_t a_register, uint8_t *data, size_t len, bool stop = true); + ErrorCode read_register16(uint16_t a_register, uint8_t *data, size_t len); /// @brief writes an array of bytes to a device using an I2CBus /// @param data pointer to an array that contains the bytes to send /// @param len length of the buffer = number of bytes to write - /// @param stop (true/false): True will send a stop message, releasing the bus after - /// transmission. False will send a restart, keeping the connection active. /// @return an i2c::ErrorCode - ErrorCode write(const uint8_t *data, size_t len, bool stop = true) { return bus_->write(address_, data, len, stop); } + ErrorCode write(const uint8_t *data, size_t len) const { + return bus_->write_readv(this->address_, data, len, nullptr, 0); + } + + /// @brief writes an array of bytes to a device, then reads an array, as a single transaction + /// @param write_data pointer to an array that contains the bytes to send + /// @param write_len length of the buffer = number of bytes to write + /// @param read_data pointer to an array to store the bytes read + /// @param read_len length of the buffer = number of bytes to read + /// @return an i2c::ErrorCode + ErrorCode write_read(const uint8_t *write_data, size_t write_len, uint8_t *read_data, size_t read_len) const { + return bus_->write_readv(this->address_, write_data, write_len, read_data, read_len); + } /// @brief writes an array of bytes to a specific register in the I²C device /// @param a_register the internal address of the register to read from /// @param data pointer to an array to store the bytes /// @param len length of the buffer = number of bytes to read - /// @param stop (true/false): True will send a stop message, releasing the bus after - /// transmission. False will send a restart, keeping the connection active. /// @return an i2c::ErrorCode - ErrorCode write_register(uint8_t a_register, const uint8_t *data, size_t len, bool stop = true); + ErrorCode write_register(uint8_t a_register, const uint8_t *data, size_t len) const; /// @brief write an array of bytes to a specific register in the I²C device /// @param a_register the 16 bits internal address of the register to read from /// @param data pointer to an array to store the bytes /// @param len length of the buffer = number of bytes to read - /// @param stop (true/false): True will send a stop message, releasing the bus after - /// transmission. False will send a restart, keeping the connection active. /// @return an i2c::ErrorCode - ErrorCode write_register16(uint16_t a_register, const uint8_t *data, size_t len, bool stop = true); + ErrorCode write_register16(uint16_t a_register, const uint8_t *data, size_t len) const; /// /// Compat APIs @@ -217,7 +219,7 @@ class I2CDevice { return read_register(a_register, data, len) == ERROR_OK; } - bool read_bytes_raw(uint8_t *data, uint8_t len) { return read(data, len) == ERROR_OK; } + bool read_bytes_raw(uint8_t *data, uint8_t len) const { return read(data, len) == ERROR_OK; } template optional> read_bytes(uint8_t a_register) { std::array res; @@ -236,9 +238,7 @@ class I2CDevice { bool read_bytes_16(uint8_t a_register, uint16_t *data, uint8_t len); - bool read_byte(uint8_t a_register, uint8_t *data, bool stop = true) { - return read_register(a_register, data, 1, stop) == ERROR_OK; - } + bool read_byte(uint8_t a_register, uint8_t *data) { return read_register(a_register, data, 1) == ERROR_OK; } optional read_byte(uint8_t a_register) { uint8_t data; @@ -249,11 +249,11 @@ class I2CDevice { bool read_byte_16(uint8_t a_register, uint16_t *data) { return read_bytes_16(a_register, data, 1); } - bool write_bytes(uint8_t a_register, const uint8_t *data, uint8_t len, bool stop = true) { - return write_register(a_register, data, len, stop) == ERROR_OK; + bool write_bytes(uint8_t a_register, const uint8_t *data, uint8_t len) const { + return write_register(a_register, data, len) == ERROR_OK; } - bool write_bytes(uint8_t a_register, const std::vector &data) { + bool write_bytes(uint8_t a_register, const std::vector &data) const { return write_bytes(a_register, data.data(), data.size()); } @@ -261,13 +261,42 @@ class I2CDevice { return write_bytes(a_register, data.data(), data.size()); } - bool write_bytes_16(uint8_t a_register, const uint16_t *data, uint8_t len); + bool write_bytes_16(uint8_t a_register, const uint16_t *data, uint8_t len) const; - bool write_byte(uint8_t a_register, uint8_t data, bool stop = true) { - return write_bytes(a_register, &data, 1, stop); + bool write_byte(uint8_t a_register, uint8_t data) const { return write_bytes(a_register, &data, 1); } + + bool write_byte_16(uint8_t a_register, uint16_t data) const { return write_bytes_16(a_register, &data, 1); } + + // Deprecated functions + + ESPDEPRECATED("The stop argument is no longer used. This will be removed from ESPHome 2026.3.0", "2025.9.0") + ErrorCode read_register(uint8_t a_register, uint8_t *data, size_t len, bool stop) { + return this->read_register(a_register, data, len); } - bool write_byte_16(uint8_t a_register, uint16_t data) { return write_bytes_16(a_register, &data, 1); } + ESPDEPRECATED("The stop argument is no longer used. This will be removed from ESPHome 2026.3.0", "2025.9.0") + ErrorCode read_register16(uint16_t a_register, uint8_t *data, size_t len, bool stop) { + return this->read_register16(a_register, data, len); + } + + ESPDEPRECATED("The stop argument is no longer used; use write_read() for consecutive write and read. This will be " + "removed from ESPHome 2026.3.0", + "2025.9.0") + ErrorCode write(const uint8_t *data, size_t len, bool stop) const { return this->write(data, len); } + + ESPDEPRECATED("The stop argument is no longer used; use write_read() for consecutive write and read. This will be " + "removed from ESPHome 2026.3.0", + "2025.9.0") + ErrorCode write_register(uint8_t a_register, const uint8_t *data, size_t len, bool stop) const { + return this->write_register(a_register, data, len); + } + + ESPDEPRECATED("The stop argument is no longer used; use write_read() for consecutive write and read. This will be " + "removed from ESPHome 2026.3.0", + "2025.9.0") + ErrorCode write_register16(uint16_t a_register, const uint8_t *data, size_t len, bool stop) const { + return this->write_register16(a_register, data, len); + } protected: uint8_t address_{0x00}; ///< store the address of the device on the bus diff --git a/esphome/components/i2c/i2c_bus.h b/esphome/components/i2c/i2c_bus.h index 5c1e15d814..1acbe506a3 100644 --- a/esphome/components/i2c/i2c_bus.h +++ b/esphome/components/i2c/i2c_bus.h @@ -1,12 +1,32 @@ #pragma once #include #include +#include +#include #include #include +#include "esphome/core/helpers.h" + namespace esphome { namespace i2c { +/// @brief Helper class for efficient buffer allocation - uses stack for small sizes, heap for large +template class SmallBufferWithHeapFallback { + public: + uint8_t *get(size_t size) { + if (size <= STACK_SIZE) { + return this->stack_buffer_; + } + this->heap_buffer_ = std::unique_ptr(new uint8_t[size]); + return this->heap_buffer_.get(); + } + + private: + uint8_t stack_buffer_[STACK_SIZE]; + std::unique_ptr heap_buffer_; +}; + /// @brief Error codes returned by I2CBus and I2CDevice methods enum ErrorCode { NO_ERROR = 0, ///< No error found during execution of method @@ -39,71 +59,79 @@ struct WriteBuffer { /// note https://www.nxp.com/docs/en/application-note/AN10216.pdf class I2CBus { public: - /// @brief Creates a ReadBuffer and calls the virtual readv() method to read bytes into this buffer - /// @param address address of the I²C component on the i2c bus - /// @param buffer pointer to an array of bytes that will be used to store the data received - /// @param len length of the buffer = number of bytes to read - /// @return an i2c::ErrorCode - virtual ErrorCode read(uint8_t address, uint8_t *buffer, size_t len) { - ReadBuffer buf; - buf.data = buffer; - buf.len = len; - return readv(address, &buf, 1); - } + virtual ~I2CBus() = default; - /// @brief This virtual method reads bytes from an I2CBus into an array of ReadBuffer. - /// @param address address of the I²C component on the i2c bus - /// @param buffers pointer to an array of ReadBuffer - /// @param count number of ReadBuffer to read - /// @return an i2c::ErrorCode - /// @details This is a pure virtual method that must be implemented in a subclass. - virtual ErrorCode readv(uint8_t address, ReadBuffer *buffers, size_t count) = 0; - - virtual ErrorCode write(uint8_t address, const uint8_t *buffer, size_t len) { - return write(address, buffer, len, true); - } - - /// @brief Creates a WriteBuffer and calls the writev() method to send the bytes from this buffer - /// @param address address of the I²C component on the i2c bus - /// @param buffer pointer to an array of bytes that contains the data to be sent - /// @param len length of the buffer = number of bytes to write - /// @param stop true or false: True will send a stop message, releasing the bus after - /// transmission. False will send a restart, keeping the connection active. - /// @return an i2c::ErrorCode - virtual ErrorCode write(uint8_t address, const uint8_t *buffer, size_t len, bool stop) { - WriteBuffer buf; - buf.data = buffer; - buf.len = len; - return writev(address, &buf, 1, stop); - } - - virtual ErrorCode writev(uint8_t address, WriteBuffer *buffers, size_t cnt) { - return writev(address, buffers, cnt, true); - } - - /// @brief This virtual method writes bytes to an I2CBus from an array of WriteBuffer. - /// @param address address of the I²C component on the i2c bus - /// @param buffers pointer to an array of WriteBuffer - /// @param count number of WriteBuffer to write - /// @param stop true or false: True will send a stop message, releasing the bus after + /// @brief This virtual method writes bytes to an I2CBus from an array, + /// then reads bytes into an array of ReadBuffer. + /// @param address address of the I²C device on the i2c bus + /// @param write_buffer pointer to data + /// @param write_count number of bytes to write + /// @param read_buffer pointer to an array to receive data + /// @param read_count number of bytes to read /// transmission. False will send a restart, keeping the connection active. /// @return an i2c::ErrorCode /// @details This is a pure virtual method that must be implemented in the subclass. - virtual ErrorCode writev(uint8_t address, WriteBuffer *buffers, size_t count, bool stop) = 0; + virtual ErrorCode write_readv(uint8_t address, const uint8_t *write_buffer, size_t write_count, uint8_t *read_buffer, + size_t read_count) = 0; + + // Legacy functions for compatibility + + ErrorCode read(uint8_t address, uint8_t *buffer, size_t len) { + return this->write_readv(address, nullptr, 0, buffer, len); + } + + ErrorCode write(uint8_t address, const uint8_t *buffer, size_t len, bool stop = true) { + return this->write_readv(address, buffer, len, nullptr, 0); + } + + ESPDEPRECATED("This method is deprecated and will be removed in ESPHome 2026.3.0. Use write_readv() instead.", + "2025.9.0") + ErrorCode readv(uint8_t address, ReadBuffer *read_buffers, size_t count) { + size_t total_len = 0; + for (size_t i = 0; i != count; i++) { + total_len += read_buffers[i].len; + } + + SmallBufferWithHeapFallback<128> buffer_alloc; // Most I2C reads are small + uint8_t *buffer = buffer_alloc.get(total_len); + + auto err = this->write_readv(address, nullptr, 0, buffer, total_len); + if (err != ERROR_OK) + return err; + size_t pos = 0; + for (size_t i = 0; i != count; i++) { + if (read_buffers[i].len != 0) { + std::memcpy(read_buffers[i].data, buffer + pos, read_buffers[i].len); + pos += read_buffers[i].len; + } + } + return ERROR_OK; + } + + ESPDEPRECATED("This method is deprecated and will be removed in ESPHome 2026.3.0. Use write_readv() instead.", + "2025.9.0") + ErrorCode writev(uint8_t address, const WriteBuffer *write_buffers, size_t count, bool stop = true) { + size_t total_len = 0; + for (size_t i = 0; i != count; i++) { + total_len += write_buffers[i].len; + } + + SmallBufferWithHeapFallback<128> buffer_alloc; // Most I2C writes are small + uint8_t *buffer = buffer_alloc.get(total_len); + + size_t pos = 0; + for (size_t i = 0; i != count; i++) { + std::memcpy(buffer + pos, write_buffers[i].data, write_buffers[i].len); + pos += write_buffers[i].len; + } + + return this->write_readv(address, buffer, total_len, nullptr, 0); + } protected: /// @brief Scans the I2C bus for devices. Devices presence is kept in an array of std::pair /// that contains the address and the corresponding bool presence flag. - void i2c_scan_() { - for (uint8_t address = 8; address < 120; address++) { - auto err = writev(address, nullptr, 0); - if (err == ERROR_OK) { - scan_results_.emplace_back(address, true); - } else if (err == ERROR_UNKNOWN) { - scan_results_.emplace_back(address, false); - } - } - } + void i2c_scan_(); std::vector> scan_results_; ///< array containing scan results bool scan_{false}; ///< Should we scan ? Can be set in the yaml }; diff --git a/esphome/components/i2c/i2c_bus_arduino.cpp b/esphome/components/i2c/i2c_bus_arduino.cpp index a85df0a4cd..221423418b 100644 --- a/esphome/components/i2c/i2c_bus_arduino.cpp +++ b/esphome/components/i2c/i2c_bus_arduino.cpp @@ -13,7 +13,6 @@ namespace i2c { static const char *const TAG = "i2c.arduino"; void ArduinoI2CBus::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); recover_(); #if defined(USE_ESP32) @@ -112,88 +111,37 @@ void ArduinoI2CBus::dump_config() { } } -ErrorCode ArduinoI2CBus::readv(uint8_t address, ReadBuffer *buffers, size_t cnt) { +ErrorCode ArduinoI2CBus::write_readv(uint8_t address, const uint8_t *write_buffer, size_t write_count, + uint8_t *read_buffer, size_t read_count) { #if defined(USE_ESP8266) this->set_pins_and_clock_(); // reconfigure Wire global state in case there are multiple instances #endif - - // logging is only enabled with vv level, if warnings are shown the caller - // should log them if (!initialized_) { - ESP_LOGVV(TAG, "i2c bus not initialized!"); - return ERROR_NOT_INITIALIZED; - } - size_t to_request = 0; - for (size_t i = 0; i < cnt; i++) - to_request += buffers[i].len; - size_t ret = wire_->requestFrom(address, to_request, true); - if (ret != to_request) { - ESP_LOGVV(TAG, "RX %u from %02X failed with error %u", to_request, address, ret); - return ERROR_TIMEOUT; - } - - for (size_t i = 0; i < cnt; i++) { - const auto &buf = buffers[i]; - for (size_t j = 0; j < buf.len; j++) - buf.data[j] = wire_->read(); - } - -#ifdef ESPHOME_LOG_HAS_VERY_VERBOSE - char debug_buf[4]; - std::string debug_hex; - - for (size_t i = 0; i < cnt; i++) { - const auto &buf = buffers[i]; - for (size_t j = 0; j < buf.len; j++) { - snprintf(debug_buf, sizeof(debug_buf), "%02X", buf.data[j]); - debug_hex += debug_buf; - } - } - ESP_LOGVV(TAG, "0x%02X RX %s", address, debug_hex.c_str()); -#endif - - return ERROR_OK; -} -ErrorCode ArduinoI2CBus::writev(uint8_t address, WriteBuffer *buffers, size_t cnt, bool stop) { -#if defined(USE_ESP8266) - this->set_pins_and_clock_(); // reconfigure Wire global state in case there are multiple instances -#endif - - // logging is only enabled with vv level, if warnings are shown the caller - // should log them - if (!initialized_) { - ESP_LOGVV(TAG, "i2c bus not initialized!"); + ESP_LOGD(TAG, "i2c bus not initialized!"); return ERROR_NOT_INITIALIZED; } -#ifdef ESPHOME_LOG_HAS_VERY_VERBOSE - char debug_buf[4]; - std::string debug_hex; + ESP_LOGV(TAG, "0x%02X TX %s", address, format_hex_pretty(write_buffer, write_count).c_str()); - for (size_t i = 0; i < cnt; i++) { - const auto &buf = buffers[i]; - for (size_t j = 0; j < buf.len; j++) { - snprintf(debug_buf, sizeof(debug_buf), "%02X", buf.data[j]); - debug_hex += debug_buf; - } - } - ESP_LOGVV(TAG, "0x%02X TX %s", address, debug_hex.c_str()); -#endif - - wire_->beginTransmission(address); - size_t written = 0; - for (size_t i = 0; i < cnt; i++) { - const auto &buf = buffers[i]; - if (buf.len == 0) - continue; - size_t ret = wire_->write(buf.data, buf.len); - written += ret; - if (ret != buf.len) { - ESP_LOGVV(TAG, "TX failed at %u", written); + uint8_t status = 0; + if (write_count != 0 || read_count == 0) { + wire_->beginTransmission(address); + size_t ret = wire_->write(write_buffer, write_count); + if (ret != write_count) { + ESP_LOGV(TAG, "TX failed"); return ERROR_UNKNOWN; } + status = wire_->endTransmission(read_count == 0); + } + if (status == 0 && read_count != 0) { + size_t ret2 = wire_->requestFrom(address, read_count, true); + if (ret2 != read_count) { + ESP_LOGVV(TAG, "RX %u from %02X failed with error %u", read_count, address, ret2); + return ERROR_TIMEOUT; + } + for (size_t j = 0; j != read_count; j++) + read_buffer[j] = wire_->read(); } - uint8_t status = wire_->endTransmission(stop); switch (status) { case 0: return ERROR_OK; diff --git a/esphome/components/i2c/i2c_bus_arduino.h b/esphome/components/i2c/i2c_bus_arduino.h index 7e6616cbce..b441828353 100644 --- a/esphome/components/i2c/i2c_bus_arduino.h +++ b/esphome/components/i2c/i2c_bus_arduino.h @@ -19,8 +19,8 @@ class ArduinoI2CBus : public InternalI2CBus, public Component { public: void setup() override; void dump_config() override; - ErrorCode readv(uint8_t address, ReadBuffer *buffers, size_t cnt) override; - ErrorCode writev(uint8_t address, WriteBuffer *buffers, size_t cnt, bool stop) override; + ErrorCode write_readv(uint8_t address, const uint8_t *write_buffer, size_t write_count, uint8_t *read_buffer, + size_t read_count) override; float get_setup_priority() const override { return setup_priority::BUS; } void set_scan(bool scan) { scan_ = scan; } diff --git a/esphome/components/i2c/i2c_bus_esp_idf.cpp b/esphome/components/i2c/i2c_bus_esp_idf.cpp index e4643405ce..bf50ea0586 100644 --- a/esphome/components/i2c/i2c_bus_esp_idf.cpp +++ b/esphome/components/i2c/i2c_bus_esp_idf.cpp @@ -1,6 +1,8 @@ #ifdef USE_ESP_IDF #include "i2c_bus_esp_idf.h" + +#include #include #include #include "esphome/core/application.h" @@ -8,77 +10,73 @@ #include "esphome/core/helpers.h" #include "esphome/core/log.h" -#if ESP_IDF_VERSION < ESP_IDF_VERSION_VAL(5, 3, 0) -#define SOC_HP_I2C_NUM SOC_I2C_NUM -#endif - namespace esphome { namespace i2c { static const char *const TAG = "i2c.idf"; void IDFI2CBus::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); static i2c_port_t next_port = I2C_NUM_0; - port_ = next_port; -#if SOC_HP_I2C_NUM > 1 - next_port = (next_port == I2C_NUM_0) ? I2C_NUM_1 : I2C_NUM_MAX; + this->port_ = next_port; + if (this->port_ == I2C_NUM_MAX) { + ESP_LOGE(TAG, "No more than %u buses supported", I2C_NUM_MAX); + this->mark_failed(); + return; + } + + if (this->timeout_ > 13000) { + ESP_LOGW(TAG, "Using max allowed timeout: 13 ms"); + this->timeout_ = 13000; + } + + this->recover_(); + + next_port = (i2c_port_t) (next_port + 1); + + i2c_master_bus_config_t bus_conf{}; + memset(&bus_conf, 0, sizeof(bus_conf)); + bus_conf.sda_io_num = gpio_num_t(sda_pin_); + bus_conf.scl_io_num = gpio_num_t(scl_pin_); + bus_conf.i2c_port = this->port_; + bus_conf.glitch_ignore_cnt = 7; +#if SOC_LP_I2C_SUPPORTED + if (this->port_ < SOC_HP_I2C_NUM) { + bus_conf.clk_source = I2C_CLK_SRC_DEFAULT; + } else { + bus_conf.lp_source_clk = LP_I2C_SCLK_DEFAULT; + } #else - next_port = I2C_NUM_MAX; + bus_conf.clk_source = I2C_CLK_SRC_DEFAULT; #endif - - if (port_ == I2C_NUM_MAX) { - ESP_LOGE(TAG, "No more than %u buses supported", SOC_HP_I2C_NUM); - this->mark_failed(); - return; - } - - recover_(); - - i2c_config_t conf{}; - memset(&conf, 0, sizeof(conf)); - conf.mode = I2C_MODE_MASTER; - conf.sda_io_num = sda_pin_; - conf.sda_pullup_en = sda_pullup_enabled_; - conf.scl_io_num = scl_pin_; - conf.scl_pullup_en = scl_pullup_enabled_; - conf.master.clk_speed = frequency_; -#ifdef USE_ESP32_VARIANT_ESP32S2 - // workaround for https://github.com/esphome/issues/issues/6718 - conf.clk_flags = I2C_SCLK_SRC_FLAG_AWARE_DFS; -#endif - esp_err_t err = i2c_param_config(port_, &conf); + bus_conf.flags.enable_internal_pullup = sda_pullup_enabled_ || scl_pullup_enabled_; + esp_err_t err = i2c_new_master_bus(&bus_conf, &this->bus_); if (err != ESP_OK) { - ESP_LOGW(TAG, "i2c_param_config failed: %s", esp_err_to_name(err)); + ESP_LOGW(TAG, "i2c_new_master_bus failed: %s", esp_err_to_name(err)); this->mark_failed(); return; } - if (timeout_ > 0) { // if timeout specified in yaml: - if (timeout_ > 13000) { - ESP_LOGW(TAG, "i2c timeout of %" PRIu32 "us greater than max of 13ms on esp-idf, setting to max", timeout_); - timeout_ = 13000; - } - err = i2c_set_timeout(port_, timeout_ * 80); // unit: APB 80MHz clock cycle - if (err != ESP_OK) { - ESP_LOGW(TAG, "i2c_set_timeout failed: %s", esp_err_to_name(err)); - this->mark_failed(); - return; - } else { - ESP_LOGV(TAG, "i2c_timeout set to %" PRIu32 " ticks (%" PRIu32 " us)", timeout_ * 80, timeout_); - } - } - err = i2c_driver_install(port_, I2C_MODE_MASTER, 0, 0, 0); + + i2c_device_config_t dev_conf{}; + memset(&dev_conf, 0, sizeof(dev_conf)); + dev_conf.dev_addr_length = I2C_ADDR_BIT_LEN_7; + dev_conf.device_address = I2C_DEVICE_ADDRESS_NOT_USED; + dev_conf.scl_speed_hz = this->frequency_; + dev_conf.scl_wait_us = this->timeout_; + err = i2c_master_bus_add_device(this->bus_, &dev_conf, &this->dev_); if (err != ESP_OK) { - ESP_LOGW(TAG, "i2c_driver_install failed: %s", esp_err_to_name(err)); + ESP_LOGW(TAG, "i2c_master_bus_add_device failed: %s", esp_err_to_name(err)); this->mark_failed(); return; } - initialized_ = true; + + this->initialized_ = true; + if (this->scan_) { - ESP_LOGV(TAG, "Scanning bus for active devices"); + ESP_LOGV(TAG, "Scanning for devices"); this->i2c_scan_(); } } + void IDFI2CBus::dump_config() { ESP_LOGCONFIG(TAG, "I2C Bus:"); ESP_LOGCONFIG(TAG, @@ -101,13 +99,13 @@ void IDFI2CBus::dump_config() { break; } if (this->scan_) { - ESP_LOGI(TAG, "Results from bus scan:"); + ESP_LOGCONFIG(TAG, "Results from bus scan:"); if (scan_results_.empty()) { - ESP_LOGI(TAG, "Found no devices"); + ESP_LOGCONFIG(TAG, "Found no devices"); } else { for (const auto &s : scan_results_) { if (s.second) { - ESP_LOGI(TAG, "Found device at address 0x%02X", s.first); + ESP_LOGCONFIG(TAG, "Found device at address 0x%02X", s.first); } else { ESP_LOGE(TAG, "Unknown error at address 0x%02X", s.first); } @@ -116,140 +114,71 @@ void IDFI2CBus::dump_config() { } } -ErrorCode IDFI2CBus::readv(uint8_t address, ReadBuffer *buffers, size_t cnt) { - // logging is only enabled with vv level, if warnings are shown the caller +ErrorCode IDFI2CBus::write_readv(uint8_t address, const uint8_t *write_buffer, size_t write_count, uint8_t *read_buffer, + size_t read_count) { + // logging is only enabled with v level, if warnings are shown the caller // should log them if (!initialized_) { - ESP_LOGVV(TAG, "i2c bus not initialized!"); - return ERROR_NOT_INITIALIZED; - } - i2c_cmd_handle_t cmd = i2c_cmd_link_create(); - esp_err_t err = i2c_master_start(cmd); - if (err != ESP_OK) { - ESP_LOGVV(TAG, "RX from %02X master start failed: %s", address, esp_err_to_name(err)); - i2c_cmd_link_delete(cmd); - return ERROR_UNKNOWN; - } - err = i2c_master_write_byte(cmd, (address << 1) | I2C_MASTER_READ, true); - if (err != ESP_OK) { - ESP_LOGVV(TAG, "RX from %02X address write failed: %s", address, esp_err_to_name(err)); - i2c_cmd_link_delete(cmd); - return ERROR_UNKNOWN; - } - for (size_t i = 0; i < cnt; i++) { - const auto &buf = buffers[i]; - if (buf.len == 0) - continue; - err = i2c_master_read(cmd, buf.data, buf.len, i == cnt - 1 ? I2C_MASTER_LAST_NACK : I2C_MASTER_ACK); - if (err != ESP_OK) { - ESP_LOGVV(TAG, "RX from %02X data read failed: %s", address, esp_err_to_name(err)); - i2c_cmd_link_delete(cmd); - return ERROR_UNKNOWN; - } - } - err = i2c_master_stop(cmd); - if (err != ESP_OK) { - ESP_LOGVV(TAG, "RX from %02X stop failed: %s", address, esp_err_to_name(err)); - i2c_cmd_link_delete(cmd); - return ERROR_UNKNOWN; - } - err = i2c_master_cmd_begin(port_, cmd, 20 / portTICK_PERIOD_MS); - // i2c_master_cmd_begin() will block for a whole second if no ack: - // https://github.com/espressif/esp-idf/issues/4999 - i2c_cmd_link_delete(cmd); - if (err == ESP_FAIL) { - // transfer not acked - ESP_LOGVV(TAG, "RX from %02X failed: not acked", address); - return ERROR_NOT_ACKNOWLEDGED; - } else if (err == ESP_ERR_TIMEOUT) { - ESP_LOGVV(TAG, "RX from %02X failed: timeout", address); - return ERROR_TIMEOUT; - } else if (err != ESP_OK) { - ESP_LOGVV(TAG, "RX from %02X failed: %s", address, esp_err_to_name(err)); - return ERROR_UNKNOWN; - } - -#ifdef ESPHOME_LOG_HAS_VERY_VERBOSE - char debug_buf[4]; - std::string debug_hex; - - for (size_t i = 0; i < cnt; i++) { - const auto &buf = buffers[i]; - for (size_t j = 0; j < buf.len; j++) { - snprintf(debug_buf, sizeof(debug_buf), "%02X", buf.data[j]); - debug_hex += debug_buf; - } - } - ESP_LOGVV(TAG, "0x%02X RX %s", address, debug_hex.c_str()); -#endif - - return ERROR_OK; -} -ErrorCode IDFI2CBus::writev(uint8_t address, WriteBuffer *buffers, size_t cnt, bool stop) { - // logging is only enabled with vv level, if warnings are shown the caller - // should log them - if (!initialized_) { - ESP_LOGVV(TAG, "i2c bus not initialized!"); + ESP_LOGW(TAG, "i2c bus not initialized!"); return ERROR_NOT_INITIALIZED; } -#ifdef ESPHOME_LOG_HAS_VERY_VERBOSE - char debug_buf[4]; - std::string debug_hex; - - for (size_t i = 0; i < cnt; i++) { - const auto &buf = buffers[i]; - for (size_t j = 0; j < buf.len; j++) { - snprintf(debug_buf, sizeof(debug_buf), "%02X", buf.data[j]); - debug_hex += debug_buf; + i2c_operation_job_t jobs[8]{}; + size_t num_jobs = 0; + uint8_t write_addr = (address << 1) | I2C_MASTER_WRITE; + uint8_t read_addr = (address << 1) | I2C_MASTER_READ; + ESP_LOGV(TAG, "Writing %zu bytes, reading %zu bytes", write_count, read_count); + if (read_count == 0 && write_count == 0) { + // basically just a bus probe. Send a start, address and stop + ESP_LOGV(TAG, "0x%02X BUS PROBE", address); + jobs[num_jobs++].command = I2C_MASTER_CMD_START; + jobs[num_jobs].command = I2C_MASTER_CMD_WRITE; + jobs[num_jobs].write.ack_check = true; + jobs[num_jobs].write.data = &write_addr; + jobs[num_jobs++].write.total_bytes = 1; + } else { + if (write_count != 0) { + ESP_LOGV(TAG, "0x%02X TX %s", address, format_hex_pretty(write_buffer, write_count).c_str()); + jobs[num_jobs++].command = I2C_MASTER_CMD_START; + jobs[num_jobs].command = I2C_MASTER_CMD_WRITE; + jobs[num_jobs].write.ack_check = true; + jobs[num_jobs].write.data = &write_addr; + jobs[num_jobs++].write.total_bytes = 1; + jobs[num_jobs].command = I2C_MASTER_CMD_WRITE; + jobs[num_jobs].write.ack_check = true; + jobs[num_jobs].write.data = (uint8_t *) write_buffer; + jobs[num_jobs++].write.total_bytes = write_count; + } + if (read_count != 0) { + ESP_LOGV(TAG, "0x%02X RX bytes %zu", address, read_count); + jobs[num_jobs++].command = I2C_MASTER_CMD_START; + jobs[num_jobs].command = I2C_MASTER_CMD_WRITE; + jobs[num_jobs].write.ack_check = true; + jobs[num_jobs].write.data = &read_addr; + jobs[num_jobs++].write.total_bytes = 1; + if (read_count > 1) { + jobs[num_jobs].command = I2C_MASTER_CMD_READ; + jobs[num_jobs].read.ack_value = I2C_ACK_VAL; + jobs[num_jobs].read.data = read_buffer; + jobs[num_jobs++].read.total_bytes = read_count - 1; + } + jobs[num_jobs].command = I2C_MASTER_CMD_READ; + jobs[num_jobs].read.ack_value = I2C_NACK_VAL; + jobs[num_jobs].read.data = read_buffer + read_count - 1; + jobs[num_jobs++].read.total_bytes = 1; } } - ESP_LOGVV(TAG, "0x%02X TX %s", address, debug_hex.c_str()); -#endif - - i2c_cmd_handle_t cmd = i2c_cmd_link_create(); - esp_err_t err = i2c_master_start(cmd); - if (err != ESP_OK) { - ESP_LOGVV(TAG, "TX to %02X master start failed: %s", address, esp_err_to_name(err)); - i2c_cmd_link_delete(cmd); - return ERROR_UNKNOWN; - } - err = i2c_master_write_byte(cmd, (address << 1) | I2C_MASTER_WRITE, true); - if (err != ESP_OK) { - ESP_LOGVV(TAG, "TX to %02X address write failed: %s", address, esp_err_to_name(err)); - i2c_cmd_link_delete(cmd); - return ERROR_UNKNOWN; - } - for (size_t i = 0; i < cnt; i++) { - const auto &buf = buffers[i]; - if (buf.len == 0) - continue; - err = i2c_master_write(cmd, buf.data, buf.len, true); - if (err != ESP_OK) { - ESP_LOGVV(TAG, "TX to %02X data write failed: %s", address, esp_err_to_name(err)); - i2c_cmd_link_delete(cmd); - return ERROR_UNKNOWN; - } - } - if (stop) { - err = i2c_master_stop(cmd); - if (err != ESP_OK) { - ESP_LOGVV(TAG, "TX to %02X master stop failed: %s", address, esp_err_to_name(err)); - i2c_cmd_link_delete(cmd); - return ERROR_UNKNOWN; - } - } - err = i2c_master_cmd_begin(port_, cmd, 20 / portTICK_PERIOD_MS); - i2c_cmd_link_delete(cmd); - if (err == ESP_FAIL) { - // transfer not acked - ESP_LOGVV(TAG, "TX to %02X failed: not acked", address); + jobs[num_jobs++].command = I2C_MASTER_CMD_STOP; + ESP_LOGV(TAG, "Sending %zu jobs", num_jobs); + esp_err_t err = i2c_master_execute_defined_operations(this->dev_, jobs, num_jobs, 20); + if (err == ESP_ERR_INVALID_STATE) { + ESP_LOGV(TAG, "TX to %02X failed: not acked", address); return ERROR_NOT_ACKNOWLEDGED; } else if (err == ESP_ERR_TIMEOUT) { - ESP_LOGVV(TAG, "TX to %02X failed: timeout", address); + ESP_LOGV(TAG, "TX to %02X failed: timeout", address); return ERROR_TIMEOUT; } else if (err != ESP_OK) { - ESP_LOGVV(TAG, "TX to %02X failed: %s", address, esp_err_to_name(err)); + ESP_LOGV(TAG, "TX to %02X failed: %s", address, esp_err_to_name(err)); return ERROR_UNKNOWN; } return ERROR_OK; @@ -261,8 +190,8 @@ ErrorCode IDFI2CBus::writev(uint8_t address, WriteBuffer *buffers, size_t cnt, b void IDFI2CBus::recover_() { ESP_LOGI(TAG, "Performing bus recovery"); - const gpio_num_t scl_pin = static_cast(scl_pin_); - const gpio_num_t sda_pin = static_cast(sda_pin_); + const auto scl_pin = static_cast(scl_pin_); + const auto sda_pin = static_cast(sda_pin_); // For the upcoming operations, target for a 60kHz toggle frequency. // 1000kHz is the maximum frequency for I2C running in standard-mode, @@ -370,5 +299,4 @@ void IDFI2CBus::recover_() { } // namespace i2c } // namespace esphome - #endif // USE_ESP_IDF diff --git a/esphome/components/i2c/i2c_bus_esp_idf.h b/esphome/components/i2c/i2c_bus_esp_idf.h index ee29578944..f565be4535 100644 --- a/esphome/components/i2c/i2c_bus_esp_idf.h +++ b/esphome/components/i2c/i2c_bus_esp_idf.h @@ -2,9 +2,9 @@ #ifdef USE_ESP_IDF -#include #include "esphome/core/component.h" #include "i2c_bus.h" +#include namespace esphome { namespace i2c { @@ -19,31 +19,33 @@ class IDFI2CBus : public InternalI2CBus, public Component { public: void setup() override; void dump_config() override; - ErrorCode readv(uint8_t address, ReadBuffer *buffers, size_t cnt) override; - ErrorCode writev(uint8_t address, WriteBuffer *buffers, size_t cnt, bool stop) override; + ErrorCode write_readv(uint8_t address, const uint8_t *write_buffer, size_t write_count, uint8_t *read_buffer, + size_t read_count) override; float get_setup_priority() const override { return setup_priority::BUS; } - void set_scan(bool scan) { scan_ = scan; } - void set_sda_pin(uint8_t sda_pin) { sda_pin_ = sda_pin; } - void set_sda_pullup_enabled(bool sda_pullup_enabled) { sda_pullup_enabled_ = sda_pullup_enabled; } - void set_scl_pin(uint8_t scl_pin) { scl_pin_ = scl_pin; } - void set_scl_pullup_enabled(bool scl_pullup_enabled) { scl_pullup_enabled_ = scl_pullup_enabled; } - void set_frequency(uint32_t frequency) { frequency_ = frequency; } - void set_timeout(uint32_t timeout) { timeout_ = timeout; } + void set_scan(bool scan) { this->scan_ = scan; } + void set_sda_pin(uint8_t sda_pin) { this->sda_pin_ = sda_pin; } + void set_sda_pullup_enabled(bool sda_pullup_enabled) { this->sda_pullup_enabled_ = sda_pullup_enabled; } + void set_scl_pin(uint8_t scl_pin) { this->scl_pin_ = scl_pin; } + void set_scl_pullup_enabled(bool scl_pullup_enabled) { this->scl_pullup_enabled_ = scl_pullup_enabled; } + void set_frequency(uint32_t frequency) { this->frequency_ = frequency; } + void set_timeout(uint32_t timeout) { this->timeout_ = timeout; } - int get_port() const override { return static_cast(this->port_); } + int get_port() const override { return this->port_; } private: void recover_(); - RecoveryCode recovery_result_; + RecoveryCode recovery_result_{}; protected: - i2c_port_t port_; - uint8_t sda_pin_; - bool sda_pullup_enabled_; - uint8_t scl_pin_; - bool scl_pullup_enabled_; - uint32_t frequency_; + i2c_master_dev_handle_t dev_{}; + i2c_master_bus_handle_t bus_{}; + i2c_port_t port_{}; + uint8_t sda_pin_{}; + bool sda_pullup_enabled_{}; + uint8_t scl_pin_{}; + bool scl_pullup_enabled_{}; + uint32_t frequency_{}; uint32_t timeout_ = 0; bool initialized_ = false; }; diff --git a/esphome/components/i2s_audio/__init__.py b/esphome/components/i2s_audio/__init__.py index 9a2aa0362f..907429ee0e 100644 --- a/esphome/components/i2s_audio/__init__.py +++ b/esphome/components/i2s_audio/__init__.py @@ -1,9 +1,12 @@ from esphome import pins import esphome.codegen as cg -from esphome.components.esp32 import get_esp32_variant +from esphome.components.esp32 import add_idf_sdkconfig_option, get_esp32_variant from esphome.components.esp32.const import ( VARIANT_ESP32, VARIANT_ESP32C3, + VARIANT_ESP32C5, + VARIANT_ESP32C6, + VARIANT_ESP32H2, VARIANT_ESP32P4, VARIANT_ESP32S2, VARIANT_ESP32S3, @@ -62,12 +65,15 @@ I2S_ROLE_OPTIONS = { CONF_SECONDARY: i2s_role_t.I2S_ROLE_SLAVE, # NOLINT } -# https://github.com/espressif/esp-idf/blob/master/components/soc/{variant}/include/soc/soc_caps.h +# https://github.com/espressif/esp-idf/blob/master/components/soc/{variant}/include/soc/soc_caps.h (SOC_I2S_NUM) I2S_PORTS = { VARIANT_ESP32: 2, VARIANT_ESP32S2: 1, VARIANT_ESP32S3: 2, VARIANT_ESP32C3: 1, + VARIANT_ESP32C5: 1, + VARIANT_ESP32C6: 1, + VARIANT_ESP32H2: 1, VARIANT_ESP32P4: 3, } @@ -137,7 +143,18 @@ def validate_mclk_divisible_by_3(config): return config -_use_legacy_driver = None +# Key for storing legacy driver setting in CORE.data +I2S_USE_LEGACY_DRIVER_KEY = "i2s_use_legacy_driver" + + +def _get_use_legacy_driver(): + """Get the legacy driver setting from CORE.data.""" + return CORE.data.get(I2S_USE_LEGACY_DRIVER_KEY) + + +def _set_use_legacy_driver(value: bool) -> None: + """Set the legacy driver setting in CORE.data.""" + CORE.data[I2S_USE_LEGACY_DRIVER_KEY] = value def i2s_audio_component_schema( @@ -203,17 +220,15 @@ async def register_i2s_audio_component(var, config): def validate_use_legacy(value): - global _use_legacy_driver # noqa: PLW0603 if CONF_USE_LEGACY in value: - if (_use_legacy_driver is not None) and ( - _use_legacy_driver != value[CONF_USE_LEGACY] - ): + existing_value = _get_use_legacy_driver() + if (existing_value is not None) and (existing_value != value[CONF_USE_LEGACY]): raise cv.Invalid( f"All i2s_audio components must set {CONF_USE_LEGACY} to the same value." ) if (not value[CONF_USE_LEGACY]) and (CORE.using_arduino): - raise cv.Invalid("Arduino supports only the legacy i2s driver.") - _use_legacy_driver = value[CONF_USE_LEGACY] + raise cv.Invalid("Arduino supports only the legacy i2s driver") + _set_use_legacy_driver(value[CONF_USE_LEGACY]) return value @@ -243,10 +258,8 @@ def _final_validate(_): def use_legacy(): - if CORE.using_esp_idf: - if not _use_legacy_driver: - return False - return True + legacy_driver = _get_use_legacy_driver() + return not (CORE.using_esp_idf and not legacy_driver) FINAL_VALIDATE_SCHEMA = _final_validate @@ -258,6 +271,9 @@ async def to_code(config): if use_legacy(): cg.add_define("USE_I2S_LEGACY") + # Helps avoid callbacks being skipped due to processor load + add_idf_sdkconfig_option("CONFIG_I2S_ISR_IRAM_SAFE", True) + cg.add(var.set_lrclk_pin(config[CONF_I2S_LRCLK_PIN])) if CONF_I2S_BCLK_PIN in config: cg.add(var.set_bclk_pin(config[CONF_I2S_BCLK_PIN])) diff --git a/esphome/components/i2s_audio/i2s_audio.cpp b/esphome/components/i2s_audio/i2s_audio.cpp index 7f233516e6..43064498cc 100644 --- a/esphome/components/i2s_audio/i2s_audio.cpp +++ b/esphome/components/i2s_audio/i2s_audio.cpp @@ -10,8 +10,6 @@ namespace i2s_audio { static const char *const TAG = "i2s_audio"; void I2SAudioComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - static i2s_port_t next_port_num = I2S_NUM_0; if (next_port_num >= SOC_I2S_NUM) { ESP_LOGE(TAG, "Too many components"); diff --git a/esphome/components/i2s_audio/media_player/__init__.py b/esphome/components/i2s_audio/media_player/__init__.py index ad6665a5f5..316ce7c48b 100644 --- a/esphome/components/i2s_audio/media_player/__init__.py +++ b/esphome/components/i2s_audio/media_player/__init__.py @@ -92,7 +92,7 @@ CONFIG_SCHEMA = cv.All( def _final_validate(_): if not use_legacy(): - raise cv.Invalid("I2S media player is only compatible with legacy i2s driver.") + raise cv.Invalid("I2S media player is only compatible with legacy i2s driver") FINAL_VALIDATE_SCHEMA = _final_validate diff --git a/esphome/components/i2s_audio/media_player/i2s_audio_media_player.cpp b/esphome/components/i2s_audio/media_player/i2s_audio_media_player.cpp index 57e184d7f8..39301220d5 100644 --- a/esphome/components/i2s_audio/media_player/i2s_audio_media_player.cpp +++ b/esphome/components/i2s_audio/media_player/i2s_audio_media_player.cpp @@ -119,10 +119,7 @@ void I2SAudioMediaPlayer::set_volume_(float volume, bool publish) { this->volume = volume; } -void I2SAudioMediaPlayer::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->state = media_player::MEDIA_PLAYER_STATE_IDLE; -} +void I2SAudioMediaPlayer::setup() { this->state = media_player::MEDIA_PLAYER_STATE_IDLE; } void I2SAudioMediaPlayer::loop() { switch (this->i2s_state_) { diff --git a/esphome/components/i2s_audio/microphone/__init__.py b/esphome/components/i2s_audio/microphone/__init__.py index 7bbb94f6e3..f919199c60 100644 --- a/esphome/components/i2s_audio/microphone/__init__.py +++ b/esphome/components/i2s_audio/microphone/__init__.py @@ -44,9 +44,8 @@ PDM_VARIANTS = [esp32.const.VARIANT_ESP32, esp32.const.VARIANT_ESP32S3] def _validate_esp32_variant(config): variant = esp32.get_esp32_variant() if config[CONF_ADC_TYPE] == "external": - if config[CONF_PDM]: - if variant not in PDM_VARIANTS: - raise cv.Invalid(f"{variant} does not support PDM") + if config[CONF_PDM] and variant not in PDM_VARIANTS: + raise cv.Invalid(f"{variant} does not support PDM") return config if config[CONF_ADC_TYPE] == "internal": if variant not in INTERNAL_ADC_VARIANTS: @@ -122,9 +121,8 @@ CONFIG_SCHEMA = cv.All( def _final_validate(config): - if not use_legacy(): - if config[CONF_ADC_TYPE] == "internal": - raise cv.Invalid("Internal ADC is only compatible with legacy i2s driver.") + if not use_legacy() and config[CONF_ADC_TYPE] == "internal": + raise cv.Invalid("Internal ADC is only compatible with legacy i2s driver") FINAL_VALIDATE_SCHEMA = _final_validate diff --git a/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp b/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp index 0477e0682d..cdebc214e2 100644 --- a/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp +++ b/esphome/components/i2s_audio/microphone/i2s_audio_microphone.cpp @@ -24,9 +24,6 @@ static const uint32_t READ_DURATION_MS = 16; static const size_t TASK_STACK_SIZE = 4096; static const ssize_t TASK_PRIORITY = 23; -// Use an exponential moving average to correct a DC offset with weight factor 1/1000 -static const int32_t DC_OFFSET_MOVING_AVERAGE_COEFFICIENT_DENOMINATOR = 1000; - static const char *const TAG = "i2s_audio.microphone"; enum MicrophoneEventGroupBits : uint32_t { @@ -40,7 +37,6 @@ enum MicrophoneEventGroupBits : uint32_t { }; void I2SAudioMicrophone::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); #ifdef USE_I2S_LEGACY #if SOC_I2S_SUPPORTS_ADC if (this->adc_) { @@ -382,26 +378,57 @@ void I2SAudioMicrophone::mic_task(void *params) { } void I2SAudioMicrophone::fix_dc_offset_(std::vector &data) { + /** + * From https://www.musicdsp.org/en/latest/Filters/135-dc-filter.html: + * + * y(n) = x(n) - x(n-1) + R * y(n-1) + * R = 1 - (pi * 2 * frequency / samplerate) + * + * From https://en.wikipedia.org/wiki/Hearing_range: + * The human range is commonly given as 20Hz up. + * + * From https://en.wikipedia.org/wiki/High-resolution_audio: + * A reasonable upper bound for sample rate seems to be 96kHz. + * + * Calculate R value for 20Hz on a 96kHz sample rate: + * R = 1 - (pi * 2 * 20 / 96000) + * R = 0.9986910031 + * + * Transform floating point to bit-shifting approximation: + * output = input - prev_input + R * prev_output + * output = input - prev_input + (prev_output - (prev_output >> S)) + * + * Approximate bit-shift value S from R: + * R = 1 - (1 >> S) + * R = 1 - (1 / 2^S) + * R = 1 - 2^-S + * 0.9986910031 = 1 - 2^-S + * S = 9.57732 ~= 10 + * + * Actual R from S: + * R = 1 - 2^-10 = 0.9990234375 + * + * Confirm this has effect outside human hearing on 96000kHz sample: + * 0.9990234375 = 1 - (pi * 2 * f / 96000) + * f = 14.9208Hz + * + * Confirm this has effect outside human hearing on PDM 16kHz sample: + * 0.9990234375 = 1 - (pi * 2 * f / 16000) + * f = 2.4868Hz + * + */ + const uint8_t dc_filter_shift = 10; const size_t bytes_per_sample = this->audio_stream_info_.samples_to_bytes(1); const uint32_t total_samples = this->audio_stream_info_.bytes_to_samples(data.size()); - - if (total_samples == 0) { - return; - } - - int64_t offset_accumulator = 0; for (uint32_t sample_index = 0; sample_index < total_samples; ++sample_index) { const uint32_t byte_index = sample_index * bytes_per_sample; - int32_t sample = audio::unpack_audio_sample_to_q31(&data[byte_index], bytes_per_sample); - offset_accumulator += sample; - sample -= this->dc_offset_; - audio::pack_q31_as_audio_sample(sample, &data[byte_index], bytes_per_sample); + int32_t input = audio::unpack_audio_sample_to_q31(&data[byte_index], bytes_per_sample); + int32_t output = input - this->dc_offset_prev_input_ + + (this->dc_offset_prev_output_ - (this->dc_offset_prev_output_ >> dc_filter_shift)); + this->dc_offset_prev_input_ = input; + this->dc_offset_prev_output_ = output; + audio::pack_q31_as_audio_sample(output, &data[byte_index], bytes_per_sample); } - - const int32_t new_offset = offset_accumulator / total_samples; - this->dc_offset_ = new_offset / DC_OFFSET_MOVING_AVERAGE_COEFFICIENT_DENOMINATOR + - (DC_OFFSET_MOVING_AVERAGE_COEFFICIENT_DENOMINATOR - 1) * this->dc_offset_ / - DC_OFFSET_MOVING_AVERAGE_COEFFICIENT_DENOMINATOR; } size_t I2SAudioMicrophone::read_(uint8_t *buf, size_t len, TickType_t ticks_to_wait) { diff --git a/esphome/components/i2s_audio/microphone/i2s_audio_microphone.h b/esphome/components/i2s_audio/microphone/i2s_audio_microphone.h index 5f66f2e962..de272ba23d 100644 --- a/esphome/components/i2s_audio/microphone/i2s_audio_microphone.h +++ b/esphome/components/i2s_audio/microphone/i2s_audio_microphone.h @@ -36,8 +36,8 @@ class I2SAudioMicrophone : public I2SAudioIn, public microphone::Microphone, pub #ifdef USE_I2S_LEGACY #if SOC_I2S_SUPPORTS_ADC - void set_adc_channel(adc1_channel_t channel) { - this->adc_channel_ = channel; + void set_adc_channel(adc_channel_t channel) { + this->adc_channel_ = (adc1_channel_t) channel; this->adc_ = true; } #endif @@ -82,7 +82,8 @@ class I2SAudioMicrophone : public I2SAudioIn, public microphone::Microphone, pub bool correct_dc_offset_; bool locked_driver_{false}; - int32_t dc_offset_{0}; + int32_t dc_offset_prev_input_{0}; + int32_t dc_offset_prev_output_{0}; }; } // namespace i2s_audio diff --git a/esphome/components/i2s_audio/speaker/__init__.py b/esphome/components/i2s_audio/speaker/__init__.py index cb7b876a40..98322d3a18 100644 --- a/esphome/components/i2s_audio/speaker/__init__.py +++ b/esphome/components/i2s_audio/speaker/__init__.py @@ -163,7 +163,7 @@ CONFIG_SCHEMA = cv.All( def _final_validate(config): if not use_legacy(): if config[CONF_DAC_TYPE] == "internal": - raise cv.Invalid("Internal DAC is only compatible with legacy i2s driver.") + raise cv.Invalid("Internal DAC is only compatible with legacy i2s driver") if config[CONF_I2S_COMM_FMT] == "stand_max": raise cv.Invalid( "I2S standard max format only implemented with legacy i2s driver." diff --git a/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp b/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp index 1042a7ebee..53e378c41e 100644 --- a/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp +++ b/esphome/components/i2s_audio/speaker/i2s_audio_speaker.cpp @@ -9,6 +9,7 @@ #endif #include "esphome/components/audio/audio.h" +#include "esphome/components/audio/audio_transfer_buffer.h" #include "esphome/core/application.h" #include "esphome/core/hal.h" @@ -19,72 +20,33 @@ namespace esphome { namespace i2s_audio { -static const uint8_t DMA_BUFFER_DURATION_MS = 15; +static const uint32_t DMA_BUFFER_DURATION_MS = 15; static const size_t DMA_BUFFERS_COUNT = 4; -static const size_t TASK_DELAY_MS = DMA_BUFFER_DURATION_MS * DMA_BUFFERS_COUNT / 2; - static const size_t TASK_STACK_SIZE = 4096; -static const ssize_t TASK_PRIORITY = 23; +static const ssize_t TASK_PRIORITY = 19; static const size_t I2S_EVENT_QUEUE_COUNT = DMA_BUFFERS_COUNT + 1; static const char *const TAG = "i2s_audio.speaker"; enum SpeakerEventGroupBits : uint32_t { - COMMAND_START = (1 << 0), // starts the speaker task + COMMAND_START = (1 << 0), // indicates loop should start speaker task COMMAND_STOP = (1 << 1), // stops the speaker task COMMAND_STOP_GRACEFULLY = (1 << 2), // Stops the speaker task once all data has been written - STATE_STARTING = (1 << 10), - STATE_RUNNING = (1 << 11), - STATE_STOPPING = (1 << 12), - STATE_STOPPED = (1 << 13), - ERR_TASK_FAILED_TO_START = (1 << 14), - ERR_ESP_INVALID_STATE = (1 << 15), - ERR_ESP_NOT_SUPPORTED = (1 << 16), - ERR_ESP_INVALID_ARG = (1 << 17), - ERR_ESP_INVALID_SIZE = (1 << 18), + + TASK_STARTING = (1 << 10), + TASK_RUNNING = (1 << 11), + TASK_STOPPING = (1 << 12), + TASK_STOPPED = (1 << 13), + ERR_ESP_NO_MEM = (1 << 19), - ERR_ESP_FAIL = (1 << 20), - ALL_ERR_ESP_BITS = ERR_ESP_INVALID_STATE | ERR_ESP_NOT_SUPPORTED | ERR_ESP_INVALID_ARG | ERR_ESP_INVALID_SIZE | - ERR_ESP_NO_MEM | ERR_ESP_FAIL, + + WARN_DROPPED_EVENT = (1 << 20), + ALL_BITS = 0x00FFFFFF, // All valid FreeRTOS event group bits }; -// Translates a SpeakerEventGroupBits ERR_ESP bit to the coressponding esp_err_t -static esp_err_t err_bit_to_esp_err(uint32_t bit) { - switch (bit) { - case SpeakerEventGroupBits::ERR_ESP_INVALID_STATE: - return ESP_ERR_INVALID_STATE; - case SpeakerEventGroupBits::ERR_ESP_INVALID_ARG: - return ESP_ERR_INVALID_ARG; - case SpeakerEventGroupBits::ERR_ESP_INVALID_SIZE: - return ESP_ERR_INVALID_SIZE; - case SpeakerEventGroupBits::ERR_ESP_NO_MEM: - return ESP_ERR_NO_MEM; - case SpeakerEventGroupBits::ERR_ESP_NOT_SUPPORTED: - return ESP_ERR_NOT_SUPPORTED; - default: - return ESP_FAIL; - } -} - -/// @brief Multiplies the input array of Q15 numbers by a Q15 constant factor -/// -/// Based on `dsps_mulc_s16_ansi` from the esp-dsp library: -/// https://github.com/espressif/esp-dsp/blob/master/modules/math/mulc/fixed/dsps_mulc_s16_ansi.c -/// (accessed on 2024-09-30). -/// @param input Array of Q15 numbers -/// @param output Array of Q15 numbers -/// @param len Length of array -/// @param c Q15 constant factor -static void q15_multiplication(const int16_t *input, int16_t *output, size_t len, int16_t c) { - for (int i = 0; i < len; i++) { - int32_t acc = (int32_t) input[i] * (int32_t) c; - output[i] = (int16_t) (acc >> 15); - } -} - // Lists the Q15 fixed point scaling factor for volume reduction. // Has 100 values representing silence and a reduction [49, 48.5, ... 0.5, 0] dB. // dB to PCM scaling factor formula: floating_point_scale_factor = 2^(-db/6.014) @@ -99,8 +61,6 @@ static const std::vector Q15_VOLUME_SCALING_FACTORS = { 19508, 20665, 21891, 23189, 24565, 26022, 27566, 29201, 30933, 32767}; void I2SAudioSpeaker::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->event_group_ = xEventGroupCreate(); if (this->event_group_ == nullptr) { @@ -132,51 +92,80 @@ void I2SAudioSpeaker::dump_config() { void I2SAudioSpeaker::loop() { uint32_t event_group_bits = xEventGroupGetBits(this->event_group_); - if (event_group_bits & SpeakerEventGroupBits::STATE_STARTING) { - ESP_LOGD(TAG, "Starting"); + if ((event_group_bits & SpeakerEventGroupBits::COMMAND_START) && (this->state_ == speaker::STATE_STOPPED)) { this->state_ = speaker::STATE_STARTING; - xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::STATE_STARTING); + xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::COMMAND_START); } - if (event_group_bits & SpeakerEventGroupBits::STATE_RUNNING) { + + // Handle the task's state + if (event_group_bits & SpeakerEventGroupBits::TASK_STARTING) { + ESP_LOGD(TAG, "Starting"); + xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::TASK_STARTING); + } + if (event_group_bits & SpeakerEventGroupBits::TASK_RUNNING) { ESP_LOGD(TAG, "Started"); + xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::TASK_RUNNING); this->state_ = speaker::STATE_RUNNING; - xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::STATE_RUNNING); - this->status_clear_warning(); - this->status_clear_error(); } - if (event_group_bits & SpeakerEventGroupBits::STATE_STOPPING) { + if (event_group_bits & SpeakerEventGroupBits::TASK_STOPPING) { ESP_LOGD(TAG, "Stopping"); + xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::TASK_STOPPING); this->state_ = speaker::STATE_STOPPING; - xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::STATE_STOPPING); } - if (event_group_bits & SpeakerEventGroupBits::STATE_STOPPED) { - if (!this->task_created_) { - ESP_LOGD(TAG, "Stopped"); - this->state_ = speaker::STATE_STOPPED; - xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::ALL_BITS); - this->speaker_task_handle_ = nullptr; - } + if (event_group_bits & SpeakerEventGroupBits::TASK_STOPPED) { + ESP_LOGD(TAG, "Stopped"); + + vTaskDelete(this->speaker_task_handle_); + this->speaker_task_handle_ = nullptr; + + this->stop_i2s_driver_(); + xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::ALL_BITS); + this->status_clear_error(); + + this->state_ = speaker::STATE_STOPPED; } - if (event_group_bits & SpeakerEventGroupBits::ERR_TASK_FAILED_TO_START) { - this->status_set_error("Failed to start task"); - xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::ERR_TASK_FAILED_TO_START); + // Log any errors encounted by the task + if (event_group_bits & SpeakerEventGroupBits::ERR_ESP_NO_MEM) { + ESP_LOGE(TAG, "Not enough memory"); + xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::ERR_ESP_NO_MEM); } - if (event_group_bits & SpeakerEventGroupBits::ALL_ERR_ESP_BITS) { - uint32_t error_bits = event_group_bits & SpeakerEventGroupBits::ALL_ERR_ESP_BITS; - ESP_LOGW(TAG, "Writing failed: %s", esp_err_to_name(err_bit_to_esp_err(error_bits))); - this->status_set_warning(); + // Warn if any playback timestamp events are dropped, which drastically reduces synced playback accuracy + if (event_group_bits & SpeakerEventGroupBits::WARN_DROPPED_EVENT) { + ESP_LOGW(TAG, "Event dropped, synchronized playback accuracy is reduced"); + xEventGroupClearBits(this->event_group_, SpeakerEventGroupBits::WARN_DROPPED_EVENT); } - if (event_group_bits & SpeakerEventGroupBits::ERR_ESP_NOT_SUPPORTED) { - this->status_set_error("Failed to adjust bus to match incoming audio"); - ESP_LOGE(TAG, "Incompatible audio format: sample rate = %" PRIu32 ", channels = %u, bits per sample = %u", - this->audio_stream_info_.get_sample_rate(), this->audio_stream_info_.get_channels(), - this->audio_stream_info_.get_bits_per_sample()); - } + // Handle the speaker's state + switch (this->state_) { + case speaker::STATE_STARTING: + if (this->status_has_error()) { + break; + } - xEventGroupClearBits(this->event_group_, ALL_ERR_ESP_BITS); + if (this->start_i2s_driver_(this->audio_stream_info_) != ESP_OK) { + ESP_LOGE(TAG, "Driver failed to start; retrying in 1 second"); + this->status_momentary_error("driver-faiure", 1000); + break; + } + + if (this->speaker_task_handle_ == nullptr) { + xTaskCreate(I2SAudioSpeaker::speaker_task, "speaker_task", TASK_STACK_SIZE, (void *) this, TASK_PRIORITY, + &this->speaker_task_handle_); + + if (this->speaker_task_handle_ == nullptr) { + ESP_LOGE(TAG, "Task failed to start, retrying in 1 second"); + this->status_momentary_error("task-failure", 1000); + this->stop_i2s_driver_(); // Stops the driver to return the lock; will be reloaded in next attempt + } + } + break; + case speaker::STATE_RUNNING: // Intentional fallthrough + case speaker::STATE_STOPPING: // Intentional fallthrough + case speaker::STATE_STOPPED: + break; + } } void I2SAudioSpeaker::set_volume(float volume) { @@ -227,83 +216,76 @@ size_t I2SAudioSpeaker::play(const uint8_t *data, size_t length, TickType_t tick this->start(); } - if ((this->state_ != speaker::STATE_RUNNING) || (this->audio_ring_buffer_.use_count() != 1)) { + if (this->state_ != speaker::STATE_RUNNING) { // Unable to write data to a running speaker, so delay the max amount of time so it can get ready vTaskDelay(ticks_to_wait); ticks_to_wait = 0; } size_t bytes_written = 0; - if ((this->state_ == speaker::STATE_RUNNING) && (this->audio_ring_buffer_.use_count() == 1)) { - // Only one owner of the ring buffer (the speaker task), so the ring buffer is allocated and no other components are - // attempting to write to it. - - // Temporarily share ownership of the ring buffer so it won't be deallocated while writing - std::shared_ptr temp_ring_buffer = this->audio_ring_buffer_; - bytes_written = temp_ring_buffer->write_without_replacement((void *) data, length, ticks_to_wait); + if (this->state_ == speaker::STATE_RUNNING) { + std::shared_ptr temp_ring_buffer = this->audio_ring_buffer_.lock(); + if (temp_ring_buffer.use_count() == 2) { + // Only the speaker task and this temp_ring_buffer own the ring buffer, so its safe to write to + bytes_written = temp_ring_buffer->write_without_replacement((void *) data, length, ticks_to_wait); + } } return bytes_written; } bool I2SAudioSpeaker::has_buffered_data() const { - if (this->audio_ring_buffer_ != nullptr) { - return this->audio_ring_buffer_->available() > 0; + if (this->audio_ring_buffer_.use_count() > 0) { + std::shared_ptr temp_ring_buffer = this->audio_ring_buffer_.lock(); + return temp_ring_buffer->available() > 0; } return false; } void I2SAudioSpeaker::speaker_task(void *params) { I2SAudioSpeaker *this_speaker = (I2SAudioSpeaker *) params; - this_speaker->task_created_ = true; - uint32_t event_group_bits = - xEventGroupWaitBits(this_speaker->event_group_, - SpeakerEventGroupBits::COMMAND_START | SpeakerEventGroupBits::COMMAND_STOP | - SpeakerEventGroupBits::COMMAND_STOP_GRACEFULLY, // Bit message to read - pdTRUE, // Clear the bits on exit - pdFALSE, // Don't wait for all the bits, - portMAX_DELAY); // Block indefinitely until a bit is set - - if (event_group_bits & (SpeakerEventGroupBits::COMMAND_STOP | SpeakerEventGroupBits::COMMAND_STOP_GRACEFULLY)) { - // Received a stop signal before the task was requested to start - this_speaker->delete_task_(0); - } - - xEventGroupSetBits(this_speaker->event_group_, SpeakerEventGroupBits::STATE_STARTING); - - audio::AudioStreamInfo audio_stream_info = this_speaker->audio_stream_info_; + xEventGroupSetBits(this_speaker->event_group_, SpeakerEventGroupBits::TASK_STARTING); const uint32_t dma_buffers_duration_ms = DMA_BUFFER_DURATION_MS * DMA_BUFFERS_COUNT; // Ensure ring buffer duration is at least the duration of all DMA buffers const uint32_t ring_buffer_duration = std::max(dma_buffers_duration_ms, this_speaker->buffer_duration_ms_); // The DMA buffers may have more bits per sample, so calculate buffer sizes based in the input audio stream info - const size_t data_buffer_size = audio_stream_info.ms_to_bytes(dma_buffers_duration_ms); - const size_t ring_buffer_size = audio_stream_info.ms_to_bytes(ring_buffer_duration); + const size_t ring_buffer_size = this_speaker->current_stream_info_.ms_to_bytes(ring_buffer_duration); - const size_t single_dma_buffer_input_size = data_buffer_size / DMA_BUFFERS_COUNT; + const uint32_t frames_to_fill_single_dma_buffer = + this_speaker->current_stream_info_.ms_to_frames(DMA_BUFFER_DURATION_MS); + const size_t bytes_to_fill_single_dma_buffer = + this_speaker->current_stream_info_.frames_to_bytes(frames_to_fill_single_dma_buffer); - if (this_speaker->send_esp_err_to_event_group_(this_speaker->allocate_buffers_(data_buffer_size, ring_buffer_size))) { - // Failed to allocate buffers - xEventGroupSetBits(this_speaker->event_group_, SpeakerEventGroupBits::ERR_ESP_NO_MEM); - this_speaker->delete_task_(data_buffer_size); + bool successful_setup = false; + std::unique_ptr transfer_buffer = + audio::AudioSourceTransferBuffer::create(bytes_to_fill_single_dma_buffer); + + if (transfer_buffer != nullptr) { + std::shared_ptr temp_ring_buffer = RingBuffer::create(ring_buffer_size); + if (temp_ring_buffer.use_count() == 1) { + transfer_buffer->set_source(temp_ring_buffer); + this_speaker->audio_ring_buffer_ = temp_ring_buffer; + successful_setup = true; + } } - if (!this_speaker->send_esp_err_to_event_group_(this_speaker->start_i2s_driver_(audio_stream_info))) { - xEventGroupSetBits(this_speaker->event_group_, SpeakerEventGroupBits::STATE_RUNNING); - + if (!successful_setup) { + xEventGroupSetBits(this_speaker->event_group_, SpeakerEventGroupBits::ERR_ESP_NO_MEM); + } else { bool stop_gracefully = false; + bool tx_dma_underflow = true; + + uint32_t frames_written = 0; uint32_t last_data_received_time = millis(); - bool tx_dma_underflow = false; - this_speaker->accumulated_frames_written_ = 0; + xEventGroupSetBits(this_speaker->event_group_, SpeakerEventGroupBits::TASK_RUNNING); - // Keep looping if paused, there is no timeout configured, or data was received more recently than the configured - // timeout while (this_speaker->pause_state_ || !this_speaker->timeout_.has_value() || (millis() - last_data_received_time) <= this_speaker->timeout_.value()) { - event_group_bits = xEventGroupGetBits(this_speaker->event_group_); + uint32_t event_group_bits = xEventGroupGetBits(this_speaker->event_group_); if (event_group_bits & SpeakerEventGroupBits::COMMAND_STOP) { xEventGroupClearBits(this_speaker->event_group_, SpeakerEventGroupBits::COMMAND_STOP); @@ -314,7 +296,7 @@ void I2SAudioSpeaker::speaker_task(void *params) { stop_gracefully = true; } - if (this_speaker->audio_stream_info_ != audio_stream_info) { + if (this_speaker->audio_stream_info_ != this_speaker->current_stream_info_) { // Audio stream info changed, stop the speaker task so it will restart with the proper settings. break; } @@ -326,99 +308,163 @@ void I2SAudioSpeaker::speaker_task(void *params) { } } #else - bool overflow; - while (xQueueReceive(this_speaker->i2s_event_queue_, &overflow, 0)) { - if (overflow) { + int64_t write_timestamp; + while (xQueueReceive(this_speaker->i2s_event_queue_, &write_timestamp, 0)) { + // Receives timing events from the I2S on_sent callback. If actual audio data was sent in this event, it passes + // on the timing info via the audio_output_callback. + uint32_t frames_sent = frames_to_fill_single_dma_buffer; + if (frames_to_fill_single_dma_buffer > frames_written) { tx_dma_underflow = true; + frames_sent = frames_written; + const uint32_t frames_zeroed = frames_to_fill_single_dma_buffer - frames_written; + write_timestamp -= this_speaker->current_stream_info_.frames_to_microseconds(frames_zeroed); + } else { + tx_dma_underflow = false; + } + frames_written -= frames_sent; + if (frames_sent > 0) { + this_speaker->audio_output_callback_(frames_sent, write_timestamp); } } #endif if (this_speaker->pause_state_) { // Pause state is accessed atomically, so thread safe - // Delay so the task can yields, then skip transferring audio data - delay(TASK_DELAY_MS); + // Delay so the task yields, then skip transferring audio data + vTaskDelay(pdMS_TO_TICKS(DMA_BUFFER_DURATION_MS)); continue; } - size_t bytes_read = this_speaker->audio_ring_buffer_->read((void *) this_speaker->data_buffer_, data_buffer_size, - pdMS_TO_TICKS(TASK_DELAY_MS)); + // Wait half the duration of the data already written to the DMA buffers for new audio data + // The millisecond helper modifies the frames_written variable, so use the microsecond helper and divide by 1000 + const uint32_t read_delay = + (this_speaker->current_stream_info_.frames_to_microseconds(frames_written) / 1000) / 2; + + uint8_t *new_data = transfer_buffer->get_buffer_end(); // track start of any newly copied bytes + size_t bytes_read = transfer_buffer->transfer_data_from_source(pdMS_TO_TICKS(read_delay)); if (bytes_read > 0) { - if ((audio_stream_info.get_bits_per_sample() == 16) && (this_speaker->q15_volume_factor_ < INT16_MAX)) { - // Scale samples by the volume factor in place - q15_multiplication((int16_t *) this_speaker->data_buffer_, (int16_t *) this_speaker->data_buffer_, - bytes_read / sizeof(int16_t), this_speaker->q15_volume_factor_); + if (this_speaker->q15_volume_factor_ < INT16_MAX) { + // Apply the software volume adjustment by unpacking the sample into a Q31 fixed-point number, shifting it, + // multiplying by the volume factor, and packing the sample back into the original bytes per sample. + + const size_t bytes_per_sample = this_speaker->current_stream_info_.samples_to_bytes(1); + const uint32_t len = bytes_read / bytes_per_sample; + + // Use Q16 for samples with 1 or 2 bytes: shifted_sample * gain_factor is Q16 * Q15 -> Q31 + int32_t shift = 15; // Q31 -> Q16 + int32_t gain_factor = this_speaker->q15_volume_factor_; // Q15 + + if (bytes_per_sample >= 3) { + // Use Q23 for samples with 3 or 4 bytes: shifted_sample * gain_factor is Q23 * Q8 -> Q31 + + shift = 8; // Q31 -> Q23 + gain_factor >>= 7; // Q15 -> Q8 + } + + for (uint32_t i = 0; i < len; ++i) { + int32_t sample = + audio::unpack_audio_sample_to_q31(&new_data[i * bytes_per_sample], bytes_per_sample); // Q31 + sample >>= shift; + sample *= gain_factor; // Q31 + audio::pack_q31_as_audio_sample(sample, &new_data[i * bytes_per_sample], bytes_per_sample); + } } #ifdef USE_ESP32_VARIANT_ESP32 // For ESP32 8/16 bit mono mode samples need to be switched. - if (audio_stream_info.get_channels() == 1 && audio_stream_info.get_bits_per_sample() <= 16) { + if (this_speaker->current_stream_info_.get_channels() == 1 && + this_speaker->current_stream_info_.get_bits_per_sample() <= 16) { size_t len = bytes_read / sizeof(int16_t); - int16_t *tmp_buf = (int16_t *) this_speaker->data_buffer_; - for (int i = 0; i < len; i += 2) { + int16_t *tmp_buf = (int16_t *) new_data; + for (size_t i = 0; i < len; i += 2) { int16_t tmp = tmp_buf[i]; tmp_buf[i] = tmp_buf[i + 1]; tmp_buf[i + 1] = tmp; } } #endif - // Write the audio data to a single DMA buffer at a time to reduce latency for the audio duration played - // callback. - const uint32_t batches = (bytes_read + single_dma_buffer_input_size - 1) / single_dma_buffer_input_size; + } - for (uint32_t i = 0; i < batches; ++i) { - size_t bytes_written = 0; - size_t bytes_to_write = std::min(single_dma_buffer_input_size, bytes_read); - -#ifdef USE_I2S_LEGACY - if (audio_stream_info.get_bits_per_sample() == (uint8_t) this_speaker->bits_per_sample_) { - i2s_write(this_speaker->parent_->get_port(), this_speaker->data_buffer_ + i * single_dma_buffer_input_size, - bytes_to_write, &bytes_written, pdMS_TO_TICKS(DMA_BUFFER_DURATION_MS * 5)); - } else if (audio_stream_info.get_bits_per_sample() < (uint8_t) this_speaker->bits_per_sample_) { - i2s_write_expand(this_speaker->parent_->get_port(), - this_speaker->data_buffer_ + i * single_dma_buffer_input_size, bytes_to_write, - audio_stream_info.get_bits_per_sample(), this_speaker->bits_per_sample_, &bytes_written, - pdMS_TO_TICKS(DMA_BUFFER_DURATION_MS * 5)); - } -#else - i2s_channel_write(this_speaker->tx_handle_, this_speaker->data_buffer_ + i * single_dma_buffer_input_size, - bytes_to_write, &bytes_written, pdMS_TO_TICKS(DMA_BUFFER_DURATION_MS * 5)); -#endif - - int64_t now = esp_timer_get_time(); - - if (bytes_written != bytes_to_write) { - xEventGroupSetBits(this_speaker->event_group_, SpeakerEventGroupBits::ERR_ESP_INVALID_SIZE); - } - bytes_read -= bytes_written; - - this_speaker->audio_output_callback_(audio_stream_info.bytes_to_frames(bytes_written), - now + dma_buffers_duration_ms * 1000); - - tx_dma_underflow = false; - last_data_received_time = millis(); - } - } else { - // No data received + if (transfer_buffer->available() == 0) { if (stop_gracefully && tx_dma_underflow) { break; } + vTaskDelay(pdMS_TO_TICKS(DMA_BUFFER_DURATION_MS / 2)); + } else { + size_t bytes_written = 0; +#ifdef USE_I2S_LEGACY + if (this_speaker->current_stream_info_.get_bits_per_sample() == (uint8_t) this_speaker->bits_per_sample_) { + i2s_write(this_speaker->parent_->get_port(), transfer_buffer->get_buffer_start(), + transfer_buffer->available(), &bytes_written, pdMS_TO_TICKS(DMA_BUFFER_DURATION_MS)); + } else if (this_speaker->current_stream_info_.get_bits_per_sample() < + (uint8_t) this_speaker->bits_per_sample_) { + i2s_write_expand(this_speaker->parent_->get_port(), transfer_buffer->get_buffer_start(), + transfer_buffer->available(), this_speaker->current_stream_info_.get_bits_per_sample(), + this_speaker->bits_per_sample_, &bytes_written, pdMS_TO_TICKS(DMA_BUFFER_DURATION_MS)); + } +#else + if (tx_dma_underflow) { + // Temporarily disable channel and callback to reset the I2S driver's internal DMA buffer queue so timing + // callbacks are accurate. Preload the data. + i2s_channel_disable(this_speaker->tx_handle_); + const i2s_event_callbacks_t callbacks = { + .on_sent = nullptr, + }; + + i2s_channel_register_event_callback(this_speaker->tx_handle_, &callbacks, this_speaker); + i2s_channel_preload_data(this_speaker->tx_handle_, transfer_buffer->get_buffer_start(), + transfer_buffer->available(), &bytes_written); + } else { + // Audio is already playing, use regular I2S write to add to the DMA buffers + i2s_channel_write(this_speaker->tx_handle_, transfer_buffer->get_buffer_start(), transfer_buffer->available(), + &bytes_written, DMA_BUFFER_DURATION_MS); + } +#endif + if (bytes_written > 0) { + last_data_received_time = millis(); + frames_written += this_speaker->current_stream_info_.bytes_to_frames(bytes_written); + transfer_buffer->decrease_buffer_length(bytes_written); + if (tx_dma_underflow) { + tx_dma_underflow = false; +#ifndef USE_I2S_LEGACY + // Reset the event queue timestamps + // Enable the on_sent callback to accurately track the timestamps of played audio + // Enable the I2S channel to start sending the preloaded audio + + xQueueReset(this_speaker->i2s_event_queue_); + + const i2s_event_callbacks_t callbacks = { + .on_sent = i2s_on_sent_cb, + }; + i2s_channel_register_event_callback(this_speaker->tx_handle_, &callbacks, this_speaker); + + i2s_channel_enable(this_speaker->tx_handle_); +#endif + } +#ifdef USE_I2S_LEGACY + // The legacy driver doesn't easily support the callback approach for timestamps, so fall back to a direct but + // less accurate approach. + this_speaker->audio_output_callback_(this_speaker->current_stream_info_.bytes_to_frames(bytes_written), + esp_timer_get_time() + dma_buffers_duration_ms * 1000); +#endif + } } } - - xEventGroupSetBits(this_speaker->event_group_, SpeakerEventGroupBits::STATE_STOPPING); -#ifdef USE_I2S_LEGACY - i2s_driver_uninstall(this_speaker->parent_->get_port()); -#else - i2s_channel_disable(this_speaker->tx_handle_); - i2s_del_channel(this_speaker->tx_handle_); -#endif - - this_speaker->parent_->unlock(); } - this_speaker->delete_task_(data_buffer_size); + xEventGroupSetBits(this_speaker->event_group_, SpeakerEventGroupBits::TASK_STOPPING); + + if (transfer_buffer != nullptr) { + transfer_buffer.reset(); + } + + xEventGroupSetBits(this_speaker->event_group_, SpeakerEventGroupBits::TASK_STOPPED); + + while (true) { + // Continuously delay until the loop method deletes the task + vTaskDelay(pdMS_TO_TICKS(10)); + } } void I2SAudioSpeaker::start() { @@ -427,16 +473,7 @@ void I2SAudioSpeaker::start() { if ((this->state_ == speaker::STATE_STARTING) || (this->state_ == speaker::STATE_RUNNING)) return; - if (!this->task_created_ && (this->speaker_task_handle_ == nullptr)) { - xTaskCreate(I2SAudioSpeaker::speaker_task, "speaker_task", TASK_STACK_SIZE, (void *) this, TASK_PRIORITY, - &this->speaker_task_handle_); - - if (this->speaker_task_handle_ != nullptr) { - xEventGroupSetBits(this->event_group_, SpeakerEventGroupBits::COMMAND_START); - } else { - xEventGroupSetBits(this->event_group_, SpeakerEventGroupBits::ERR_TASK_FAILED_TO_START); - } - } + xEventGroupSetBits(this->event_group_, SpeakerEventGroupBits::COMMAND_START); } void I2SAudioSpeaker::stop() { this->stop_(false); } @@ -456,61 +493,16 @@ void I2SAudioSpeaker::stop_(bool wait_on_empty) { } } -bool I2SAudioSpeaker::send_esp_err_to_event_group_(esp_err_t err) { - switch (err) { - case ESP_OK: - return false; - case ESP_ERR_INVALID_STATE: - xEventGroupSetBits(this->event_group_, SpeakerEventGroupBits::ERR_ESP_INVALID_STATE); - return true; - case ESP_ERR_INVALID_ARG: - xEventGroupSetBits(this->event_group_, SpeakerEventGroupBits::ERR_ESP_INVALID_ARG); - return true; - case ESP_ERR_INVALID_SIZE: - xEventGroupSetBits(this->event_group_, SpeakerEventGroupBits::ERR_ESP_INVALID_SIZE); - return true; - case ESP_ERR_NO_MEM: - xEventGroupSetBits(this->event_group_, SpeakerEventGroupBits::ERR_ESP_NO_MEM); - return true; - case ESP_ERR_NOT_SUPPORTED: - xEventGroupSetBits(this->event_group_, SpeakerEventGroupBits::ERR_ESP_NOT_SUPPORTED); - return true; - default: - xEventGroupSetBits(this->event_group_, SpeakerEventGroupBits::ERR_ESP_FAIL); - return true; - } -} - -esp_err_t I2SAudioSpeaker::allocate_buffers_(size_t data_buffer_size, size_t ring_buffer_size) { - if (this->data_buffer_ == nullptr) { - // Allocate data buffer for temporarily storing audio from the ring buffer before writing to the I2S bus - RAMAllocator allocator; - this->data_buffer_ = allocator.allocate(data_buffer_size); - } - - if (this->data_buffer_ == nullptr) { - return ESP_ERR_NO_MEM; - } - - if (this->audio_ring_buffer_.use_count() == 0) { - // Allocate ring buffer. Uses a shared_ptr to ensure it isn't improperly deallocated. - this->audio_ring_buffer_ = RingBuffer::create(ring_buffer_size); - } - - if (this->audio_ring_buffer_ == nullptr) { - return ESP_ERR_NO_MEM; - } - - return ESP_OK; -} - esp_err_t I2SAudioSpeaker::start_i2s_driver_(audio::AudioStreamInfo &audio_stream_info) { + this->current_stream_info_ = audio_stream_info; // store the stream info settings the driver will use + #ifdef USE_I2S_LEGACY if ((this->i2s_mode_ & I2S_MODE_SLAVE) && (this->sample_rate_ != audio_stream_info.get_sample_rate())) { // NOLINT #else if ((this->i2s_role_ & I2S_ROLE_SLAVE) && (this->sample_rate_ != audio_stream_info.get_sample_rate())) { // NOLINT #endif // Can't reconfigure I2S bus, so the sample rate must match the configured value + ESP_LOGE(TAG, "Audio stream settings are not compatible with this I2S configuration"); return ESP_ERR_NOT_SUPPORTED; } @@ -521,10 +513,12 @@ esp_err_t I2SAudioSpeaker::start_i2s_driver_(audio::AudioStreamInfo &audio_strea (i2s_slot_bit_width_t) audio_stream_info.get_bits_per_sample() > this->slot_bit_width_) { #endif // Currently can't handle the case when the incoming audio has more bits per sample than the configured value + ESP_LOGE(TAG, "Audio streams with more bits per sample than the I2S speaker's configuration is not supported"); return ESP_ERR_NOT_SUPPORTED; } if (!this->parent_->try_lock()) { + ESP_LOGE(TAG, "Parent I2S bus not free"); return ESP_ERR_INVALID_STATE; } @@ -575,6 +569,7 @@ esp_err_t I2SAudioSpeaker::start_i2s_driver_(audio::AudioStreamInfo &audio_strea esp_err_t err = i2s_driver_install(this->parent_->get_port(), &config, I2S_EVENT_QUEUE_COUNT, &this->i2s_event_queue_); if (err != ESP_OK) { + ESP_LOGE(TAG, "Failed to install I2S legacy driver"); // Failed to install the driver, so unlock the I2S port this->parent_->unlock(); return err; @@ -595,6 +590,7 @@ esp_err_t I2SAudioSpeaker::start_i2s_driver_(audio::AudioStreamInfo &audio_strea if (err != ESP_OK) { // Failed to set the data out pin, so uninstall the driver and unlock the I2S port + ESP_LOGE(TAG, "Failed to set the data out pin"); i2s_driver_uninstall(this->parent_->get_port()); this->parent_->unlock(); } @@ -605,10 +601,12 @@ esp_err_t I2SAudioSpeaker::start_i2s_driver_(audio::AudioStreamInfo &audio_strea .dma_desc_num = DMA_BUFFERS_COUNT, .dma_frame_num = dma_buffer_length, .auto_clear = true, + .intr_priority = 3, }; /* Allocate a new TX channel and get the handle of this channel */ esp_err_t err = i2s_new_channel(&chan_cfg, &this->tx_handle_, NULL); if (err != ESP_OK) { + ESP_LOGE(TAG, "Failed to allocate new I2S channel"); this->parent_->unlock(); return err; } @@ -652,7 +650,11 @@ esp_err_t I2SAudioSpeaker::start_i2s_driver_(audio::AudioStreamInfo &audio_strea // per sample causes the audio to play too fast. Setting the ws_width to the configured slot bit width seems to // make it play at the correct speed while sending more bits per slot. if (this->slot_bit_width_ != I2S_SLOT_BIT_WIDTH_AUTO) { - std_slot_cfg.ws_width = static_cast(this->slot_bit_width_); + uint32_t configured_bit_width = static_cast(this->slot_bit_width_); + std_slot_cfg.ws_width = configured_bit_width; + if (configured_bit_width > 16) { + std_slot_cfg.msb_right = false; + } } #else std_slot_cfg.slot_bit_width = this->slot_bit_width_; @@ -670,54 +672,56 @@ esp_err_t I2SAudioSpeaker::start_i2s_driver_(audio::AudioStreamInfo &audio_strea err = i2s_channel_init_std_mode(this->tx_handle_, &std_cfg); if (err != ESP_OK) { + ESP_LOGE(TAG, "Failed to initialize channel"); i2s_del_channel(this->tx_handle_); + this->tx_handle_ = nullptr; this->parent_->unlock(); return err; } if (this->i2s_event_queue_ == nullptr) { - this->i2s_event_queue_ = xQueueCreate(1, sizeof(bool)); + this->i2s_event_queue_ = xQueueCreate(I2S_EVENT_QUEUE_COUNT, sizeof(int64_t)); } - const i2s_event_callbacks_t callbacks = { - .on_send_q_ovf = i2s_overflow_cb, - }; - i2s_channel_register_event_callback(this->tx_handle_, &callbacks, this); - - /* Before reading data, start the TX channel first */ i2s_channel_enable(this->tx_handle_); - if (err != ESP_OK) { - i2s_del_channel(this->tx_handle_); - this->parent_->unlock(); - } #endif return err; } -void I2SAudioSpeaker::delete_task_(size_t buffer_size) { - this->audio_ring_buffer_.reset(); // Releases ownership of the shared_ptr +#ifndef USE_I2S_LEGACY +bool IRAM_ATTR I2SAudioSpeaker::i2s_on_sent_cb(i2s_chan_handle_t handle, i2s_event_data_t *event, void *user_ctx) { + int64_t now = esp_timer_get_time(); - if (this->data_buffer_ != nullptr) { - RAMAllocator allocator; - allocator.deallocate(this->data_buffer_, buffer_size); - this->data_buffer_ = nullptr; + BaseType_t need_yield1 = pdFALSE; + BaseType_t need_yield2 = pdFALSE; + BaseType_t need_yield3 = pdFALSE; + + I2SAudioSpeaker *this_speaker = (I2SAudioSpeaker *) user_ctx; + + if (xQueueIsQueueFullFromISR(this_speaker->i2s_event_queue_)) { + // Queue is full, so discard the oldest event and set the warning flag to inform the user + int64_t dummy; + xQueueReceiveFromISR(this_speaker->i2s_event_queue_, &dummy, &need_yield1); + xEventGroupSetBitsFromISR(this_speaker->event_group_, SpeakerEventGroupBits::WARN_DROPPED_EVENT, &need_yield2); } - xEventGroupSetBits(this->event_group_, SpeakerEventGroupBits::STATE_STOPPED); + xQueueSendToBackFromISR(this_speaker->i2s_event_queue_, &now, &need_yield3); - this->task_created_ = false; - vTaskDelete(nullptr); -} - -#ifndef USE_I2S_LEGACY -bool IRAM_ATTR I2SAudioSpeaker::i2s_overflow_cb(i2s_chan_handle_t handle, i2s_event_data_t *event, void *user_ctx) { - I2SAudioSpeaker *this_speaker = (I2SAudioSpeaker *) user_ctx; - bool overflow = true; - xQueueOverwrite(this_speaker->i2s_event_queue_, &overflow); - return false; + return need_yield1 | need_yield2 | need_yield3; } #endif +void I2SAudioSpeaker::stop_i2s_driver_() { +#ifdef USE_I2S_LEGACY + i2s_driver_uninstall(this->parent_->get_port()); +#else + i2s_channel_disable(this->tx_handle_); + i2s_del_channel(this->tx_handle_); + this->tx_handle_ = nullptr; +#endif + this->parent_->unlock(); +} + } // namespace i2s_audio } // namespace esphome diff --git a/esphome/components/i2s_audio/speaker/i2s_audio_speaker.h b/esphome/components/i2s_audio/speaker/i2s_audio_speaker.h index eb2a0ae756..1d03a4c495 100644 --- a/esphome/components/i2s_audio/speaker/i2s_audio_speaker.h +++ b/esphome/components/i2s_audio/speaker/i2s_audio_speaker.h @@ -72,70 +72,57 @@ class I2SAudioSpeaker : public I2SAudioOut, public speaker::Speaker, public Comp protected: /// @brief Function for the FreeRTOS task handling audio output. - /// After receiving the COMMAND_START signal, allocates space for the buffers, starts the I2S driver, and reads - /// audio from the ring buffer and writes audio to the I2S port. Stops immmiately after receiving the COMMAND_STOP - /// signal and stops only after the ring buffer is empty after receiving the COMMAND_STOP_GRACEFULLY signal. Stops if - /// the ring buffer hasn't read data for more than timeout_ milliseconds. When stopping, it deallocates the buffers, - /// stops the I2S driver, unlocks the I2S port, and deletes the task. It communicates the state and any errors via - /// event_group_. + /// Allocates space for the buffers, reads audio from the ring buffer and writes audio to the I2S port. Stops + /// immmiately after receiving the COMMAND_STOP signal and stops only after the ring buffer is empty after receiving + /// the COMMAND_STOP_GRACEFULLY signal. Stops if the ring buffer hasn't read data for more than timeout_ milliseconds. + /// When stopping, it deallocates the buffers. It communicates its state and any errors via ``event_group_``. /// @param params I2SAudioSpeaker component static void speaker_task(void *params); - /// @brief Sends a stop command to the speaker task via event_group_. + /// @brief Sends a stop command to the speaker task via ``event_group_``. /// @param wait_on_empty If false, sends the COMMAND_STOP signal. If true, sends the COMMAND_STOP_GRACEFULLY signal. void stop_(bool wait_on_empty); - /// @brief Sets the corresponding ERR_ESP event group bits. - /// @param err esp_err_t error code. - /// @return True if an ERR_ESP bit is set and false if err == ESP_OK - bool send_esp_err_to_event_group_(esp_err_t err); - #ifndef USE_I2S_LEGACY - static bool i2s_overflow_cb(i2s_chan_handle_t handle, i2s_event_data_t *event, void *user_ctx); + /// @brief Callback function used to send playback timestamps the to the speaker task. + /// @param handle (i2s_chan_handle_t) + /// @param event (i2s_event_data_t) + /// @param user_ctx (void*) User context pointer that the callback accesses + /// @return True if a higher priority task was interrupted + static bool i2s_on_sent_cb(i2s_chan_handle_t handle, i2s_event_data_t *event, void *user_ctx); #endif - /// @brief Allocates the data buffer and ring buffer - /// @param data_buffer_size Number of bytes to allocate for the data buffer. - /// @param ring_buffer_size Number of bytes to allocate for the ring buffer. - /// @return ESP_ERR_NO_MEM if either buffer fails to allocate - /// ESP_OK if successful - esp_err_t allocate_buffers_(size_t data_buffer_size, size_t ring_buffer_size); - /// @brief Starts the ESP32 I2S driver. /// Attempts to lock the I2S port, starts the I2S driver using the passed in stream information, and sets the data out - /// pin. If it fails, it will unlock the I2S port and uninstall the driver, if necessary. + /// pin. If it fails, it will unlock the I2S port and uninstalls the driver, if necessary. /// @param audio_stream_info Stream information for the I2S driver. /// @return ESP_ERR_NOT_ALLOWED if the I2S port can't play the incoming audio stream. /// ESP_ERR_INVALID_STATE if the I2S port is already locked. - /// ESP_ERR_INVALID_ARG if nstalling the driver or setting the data outpin fails due to a parameter error. + /// ESP_ERR_INVALID_ARG if installing the driver or setting the data outpin fails due to a parameter error. /// ESP_ERR_NO_MEM if the driver fails to install due to a memory allocation error. - /// ESP_FAIL if setting the data out pin fails due to an IO error ESP_OK if successful + /// ESP_FAIL if setting the data out pin fails due to an IO error + /// ESP_OK if successful esp_err_t start_i2s_driver_(audio::AudioStreamInfo &audio_stream_info); - /// @brief Deletes the speaker's task. - /// Deallocates the data_buffer_ and audio_ring_buffer_, if necessary, and deletes the task. Should only be called by - /// the speaker_task itself. - /// @param buffer_size The allocated size of the data_buffer_. - void delete_task_(size_t buffer_size); + /// @brief Stops the I2S driver and unlocks the I2S port + void stop_i2s_driver_(); TaskHandle_t speaker_task_handle_{nullptr}; EventGroupHandle_t event_group_{nullptr}; QueueHandle_t i2s_event_queue_; - uint8_t *data_buffer_; - std::shared_ptr audio_ring_buffer_; + std::weak_ptr audio_ring_buffer_; uint32_t buffer_duration_ms_; optional timeout_; - bool task_created_{false}; bool pause_state_{false}; int16_t q15_volume_factor_{INT16_MAX}; - size_t bytes_written_{0}; + audio::AudioStreamInfo current_stream_info_; // The currently loaded driver's stream info #ifdef USE_I2S_LEGACY #if SOC_I2S_SUPPORTS_DAC @@ -148,8 +135,6 @@ class I2SAudioSpeaker : public I2SAudioOut, public speaker::Speaker, public Comp std::string i2s_comm_fmt_; i2s_chan_handle_t tx_handle_; #endif - - uint32_t accumulated_frames_written_{0}; }; } // namespace i2s_audio diff --git a/esphome/components/iaqcore/iaqcore.cpp b/esphome/components/iaqcore/iaqcore.cpp index 2a84eabf75..274f9086b6 100644 --- a/esphome/components/iaqcore/iaqcore.cpp +++ b/esphome/components/iaqcore/iaqcore.cpp @@ -35,7 +35,7 @@ void IAQCore::setup() { void IAQCore::update() { uint8_t buffer[sizeof(SensorData)]; - if (this->read_register(0xB5, buffer, sizeof(buffer), false) != i2c::ERROR_OK) { + if (this->read_register(0xB5, buffer, sizeof(buffer)) != i2c::ERROR_OK) { ESP_LOGD(TAG, "Read failed"); this->status_set_warning(); this->publish_nans_(); diff --git a/esphome/components/ili9xxx/display.py b/esphome/components/ili9xxx/display.py index 14b7f15218..9588bccd55 100644 --- a/esphome/components/ili9xxx/display.py +++ b/esphome/components/ili9xxx/display.py @@ -138,9 +138,10 @@ def _validate(config): ]: raise cv.Invalid("Selected model can't run on ESP8266.") - if model == "CUSTOM": - if CONF_INIT_SEQUENCE not in config or CONF_DIMENSIONS not in config: - raise cv.Invalid("CUSTOM model requires init_sequence and dimensions") + if model == "CUSTOM" and ( + CONF_INIT_SEQUENCE not in config or CONF_DIMENSIONS not in config + ): + raise cv.Invalid("CUSTOM model requires init_sequence and dimensions") return config diff --git a/esphome/components/ili9xxx/ili9xxx_display.cpp b/esphome/components/ili9xxx/ili9xxx_display.cpp index 41fd89cc58..2a3d0edca7 100644 --- a/esphome/components/ili9xxx/ili9xxx_display.cpp +++ b/esphome/components/ili9xxx/ili9xxx_display.cpp @@ -31,8 +31,6 @@ void ILI9XXXDisplay::set_madctl() { } void ILI9XXXDisplay::setup() { - ESP_LOGD(TAG, "Setting up ILI9xxx"); - this->setup_pins_(); this->init_lcd_(this->init_sequence_); this->init_lcd_(this->extra_init_sequence_.data()); @@ -327,7 +325,7 @@ void ILI9XXXDisplay::draw_pixels_at(int x_start, int y_start, int w, int h, cons // we could deal here with a non-zero y_offset, but if x_offset is zero, y_offset probably will be so don't bother this->write_array(ptr, w * h * 2); } else { - for (size_t y = 0; y != h; y++) { + for (size_t y = 0; y != static_cast(h); y++) { this->write_array(ptr + (y + y_offset) * stride + x_offset, w * 2); } } @@ -351,7 +349,7 @@ void ILI9XXXDisplay::draw_pixels_at(int x_start, int y_start, int w, int h, cons App.feed_wdt(); } // end of line? Skip to the next. - if (++pixel == w) { + if (++pixel == static_cast(w)) { pixel = 0; ptr += (x_pad + x_offset) * 2; } diff --git a/esphome/components/image/__init__.py b/esphome/components/image/__init__.py index f6d8673a08..f880b5f736 100644 --- a/esphome/components/image/__init__.py +++ b/esphome/components/image/__init__.py @@ -1,5 +1,6 @@ from __future__ import annotations +import contextlib import hashlib import io import logging @@ -107,6 +108,24 @@ class ImageEncoder: :return: """ + @classmethod + def is_endian(cls) -> bool: + """ + Check if the image encoder supports endianness configuration + """ + return getattr(cls, "set_big_endian", None) is not None + + @classmethod + def get_options(cls) -> list[str]: + """ + Get the available options for this image encoder + """ + options = [*OPTIONS] + if not cls.is_endian(): + options.remove(CONF_BYTE_ORDER) + options.append(CONF_RAW_DATA_ID) + return options + def is_alpha_only(image: Image): """ @@ -174,9 +193,8 @@ class ImageGrayscale(ImageEncoder): b = 1 if self.invert_alpha: b ^= 0xFF - if self.transparency == CONF_ALPHA_CHANNEL: - if a != 0xFF: - b = a + if self.transparency == CONF_ALPHA_CHANNEL and a != 0xFF: + b = a self.data[self.index] = b self.index += 1 @@ -446,13 +464,14 @@ def validate_type(image_types): return validate -def validate_settings(value): +def validate_settings(value, path=()): """ Validate the settings for a single image configuration. """ conf_type = value[CONF_TYPE] type_class = IMAGE_TYPE[conf_type] - transparency = value[CONF_TRANSPARENCY].lower() + + transparency = value.get(CONF_TRANSPARENCY, CONF_OPAQUE).lower() if transparency not in type_class.allow_config: raise cv.Invalid( f"Image format '{conf_type}' cannot have transparency: {transparency}" @@ -464,11 +483,10 @@ def validate_settings(value): and CONF_INVERT_ALPHA not in type_class.allow_config ): raise cv.Invalid("No alpha channel to invert") - if value.get(CONF_BYTE_ORDER) is not None and not callable( - getattr(type_class, "set_big_endian", None) - ): + if value.get(CONF_BYTE_ORDER) is not None and not type_class.is_endian(): raise cv.Invalid( - f"Image format '{conf_type}' does not support byte order configuration" + f"Image format '{conf_type}' does not support byte order configuration", + path=path, ) if file := value.get(CONF_FILE): file = Path(file) @@ -479,7 +497,7 @@ def validate_settings(value): Image.open(file) except UnidentifiedImageError as exc: raise cv.Invalid( - f"File can't be opened as image: {file.absolute()}" + f"File can't be opened as image: {file.absolute()}", path=path ) from exc return value @@ -499,6 +517,10 @@ OPTIONS_SCHEMA = { cv.Optional(CONF_INVERT_ALPHA, default=False): cv.boolean, cv.Optional(CONF_BYTE_ORDER): cv.one_of("BIG_ENDIAN", "LITTLE_ENDIAN", upper=True), cv.Optional(CONF_TRANSPARENCY, default=CONF_OPAQUE): validate_transparency(), +} + +DEFAULTS_SCHEMA = { + **OPTIONS_SCHEMA, cv.Optional(CONF_TYPE): validate_type(IMAGE_TYPE), } @@ -510,47 +532,61 @@ IMAGE_SCHEMA_NO_DEFAULTS = { **{cv.Optional(key): OPTIONS_SCHEMA[key] for key in OPTIONS}, } -BASE_SCHEMA = cv.Schema( +IMAGE_SCHEMA = cv.Schema( { **IMAGE_ID_SCHEMA, **OPTIONS_SCHEMA, - } -).add_extra(validate_settings) - -IMAGE_SCHEMA = BASE_SCHEMA.extend( - { cv.Required(CONF_TYPE): validate_type(IMAGE_TYPE), } ) +def apply_defaults(image, defaults, path): + """ + Apply defaults to an image configuration + """ + type = image.get(CONF_TYPE, defaults.get(CONF_TYPE)) + if type is None: + raise cv.Invalid( + "Type is required either in the image config or in the defaults", path=path + ) + type_class = IMAGE_TYPE[type] + config = { + **{key: image.get(key, defaults.get(key)) for key in type_class.get_options()}, + **{key.schema: image[key.schema] for key in IMAGE_ID_SCHEMA}, + CONF_TYPE: image.get(CONF_TYPE, defaults.get(CONF_TYPE)), + } + validate_settings(config, path) + return config + + def validate_defaults(value): """ - Validate the options for images with defaults + Apply defaults to the images in the configuration and flatten to a single list. """ defaults = value[CONF_DEFAULTS] result = [] - for index, image in enumerate(value[CONF_IMAGES]): - type = image.get(CONF_TYPE, defaults.get(CONF_TYPE)) - if type is None: - raise cv.Invalid( - "Type is required either in the image config or in the defaults", - path=[CONF_IMAGES, index], - ) - type_class = IMAGE_TYPE[type] - # A default byte order should be simply ignored if the type does not support it - available_options = [*OPTIONS] - if ( - not callable(getattr(type_class, "set_big_endian", None)) - and CONF_BYTE_ORDER not in image - ): - available_options.remove(CONF_BYTE_ORDER) - config = { - **{key: image.get(key, defaults.get(key)) for key in available_options}, - **{key.schema: image[key.schema] for key in IMAGE_ID_SCHEMA}, - } - validate_settings(config) - result.append(config) + # Apply defaults to the images: list and add the list entries to the result + for index, image in enumerate(value.get(CONF_IMAGES, [])): + result.append(apply_defaults(image, defaults, [CONF_IMAGES, index])) + + # Apply defaults to images under the type keys and add them to the result + for image_type, type_config in value.items(): + type_upper = image_type.upper() + if type_upper not in IMAGE_TYPE: + continue + type_class = IMAGE_TYPE[type_upper] + if isinstance(type_config, list): + # If the type is a list, apply defaults to each entry + for index, image in enumerate(type_config): + result.append(apply_defaults(image, defaults, [image_type, index])) + else: + # Handle transparency options for the type + for trans_type in set(type_class.allow_config).intersection(type_config): + for index, image in enumerate(type_config[trans_type]): + result.append( + apply_defaults(image, defaults, [image_type, trans_type, index]) + ) return result @@ -562,16 +598,20 @@ def typed_image_schema(image_type): cv.Schema( { cv.Optional(t.lower()): cv.ensure_list( - BASE_SCHEMA.extend( - { - cv.Optional( - CONF_TRANSPARENCY, default=t - ): validate_transparency((t,)), - cv.Optional(CONF_TYPE, default=image_type): validate_type( - (image_type,) - ), - } - ) + { + **IMAGE_ID_SCHEMA, + **{ + cv.Optional(key): OPTIONS_SCHEMA[key] + for key in OPTIONS + if key != CONF_TRANSPARENCY + }, + cv.Optional( + CONF_TRANSPARENCY, default=t + ): validate_transparency((t,)), + cv.Optional(CONF_TYPE, default=image_type): validate_type( + (image_type,) + ), + } ) for t in IMAGE_TYPE[image_type].allow_config.intersection( TRANSPARENCY_TYPES @@ -580,46 +620,44 @@ def typed_image_schema(image_type): ), # Allow a default configuration with no transparency preselected cv.ensure_list( - BASE_SCHEMA.extend( - { - cv.Optional( - CONF_TRANSPARENCY, default=CONF_OPAQUE - ): validate_transparency(), - cv.Optional(CONF_TYPE, default=image_type): validate_type( - (image_type,) - ), - } - ) + { + **IMAGE_SCHEMA_NO_DEFAULTS, + cv.Optional(CONF_TYPE, default=image_type): validate_type( + (image_type,) + ), + } ), ) # The config schema can be a (possibly empty) single list of images, -# or a dictionary of image types each with a list of images -# or a dictionary with keys `defaults:` and `images:` +# or a dictionary with optional keys `defaults:`, `images:` and the image types -def _config_schema(config): - if isinstance(config, list): - return cv.Schema([IMAGE_SCHEMA])(config) - if not isinstance(config, dict): +def _config_schema(value): + if isinstance(value, list) or ( + isinstance(value, dict) and (CONF_ID in value or CONF_FILE in value) + ): + return cv.ensure_list(cv.All(IMAGE_SCHEMA, validate_settings))(value) + if not isinstance(value, dict): raise cv.Invalid( - "Badly formed image configuration, expected a list or a dictionary" + "Badly formed image configuration, expected a list or a dictionary", ) - if CONF_DEFAULTS in config or CONF_IMAGES in config: - return validate_defaults( - cv.Schema( - { - cv.Required(CONF_DEFAULTS): OPTIONS_SCHEMA, - cv.Required(CONF_IMAGES): cv.ensure_list(IMAGE_SCHEMA_NO_DEFAULTS), - } - )(config) - ) - if CONF_ID in config or CONF_FILE in config: - return cv.ensure_list(IMAGE_SCHEMA)([config]) - return cv.Schema( - {cv.Optional(t.lower()): typed_image_schema(t) for t in IMAGE_TYPE} - )(config) + return cv.All( + cv.Schema( + { + cv.Optional(CONF_DEFAULTS, default={}): DEFAULTS_SCHEMA, + cv.Optional(CONF_IMAGES, default=[]): cv.ensure_list( + { + **IMAGE_SCHEMA_NO_DEFAULTS, + cv.Optional(CONF_TYPE): validate_type(IMAGE_TYPE), + } + ), + **{cv.Optional(t.lower()): typed_image_schema(t) for t in IMAGE_TYPE}, + } + ), + validate_defaults, + )(value) CONFIG_SCHEMA = _config_schema @@ -668,14 +706,12 @@ async def write_image(config, all_frames=False): else Image.Dither.FLOYDSTEINBERG ) type = config[CONF_TYPE] - transparency = config[CONF_TRANSPARENCY] + transparency = config.get(CONF_TRANSPARENCY, CONF_OPAQUE) invert_alpha = config[CONF_INVERT_ALPHA] frame_count = 1 if all_frames: - try: + with contextlib.suppress(AttributeError): frame_count = image.n_frames - except AttributeError: - pass if frame_count <= 1: _LOGGER.warning("Image file %s has no animation frames", path) @@ -701,14 +737,9 @@ async def write_image(config, all_frames=False): async def to_code(config): - if isinstance(config, list): - for entry in config: - await to_code(entry) - elif CONF_ID not in config: - for entry in config.values(): - await to_code(entry) - else: - prog_arr, width, height, image_type, trans_value, _ = await write_image(config) + # By now the config should be a simple list. + for entry in config: + prog_arr, width, height, image_type, trans_value, _ = await write_image(entry) cg.new_Pvariable( - config[CONF_ID], prog_arr, width, height, image_type, trans_value + entry[CONF_ID], prog_arr, width, height, image_type, trans_value ) diff --git a/esphome/components/improv_serial/__init__.py b/esphome/components/improv_serial/__init__.py index 544af212e0..568b200a85 100644 --- a/esphome/components/improv_serial/__init__.py +++ b/esphome/components/improv_serial/__init__.py @@ -27,14 +27,13 @@ def validate_logger(config): logger_conf = fv.full_config.get()[CONF_LOGGER] if logger_conf[CONF_BAUD_RATE] == 0: raise cv.Invalid("improv_serial requires the logger baud_rate to be not 0") - if CORE.using_esp_idf: - if ( - logger_conf[CONF_HARDWARE_UART] == USB_CDC - and get_esp32_variant() == VARIANT_ESP32S3 - ): - raise cv.Invalid( - "improv_serial does not support the selected logger hardware_uart" - ) + if CORE.using_esp_idf and ( + logger_conf[CONF_HARDWARE_UART] == USB_CDC + and get_esp32_variant() == VARIANT_ESP32S3 + ): + raise cv.Invalid( + "improv_serial does not support the selected logger hardware_uart" + ) return config diff --git a/esphome/components/improv_serial/improv_serial_component.cpp b/esphome/components/improv_serial/improv_serial_component.cpp index ae4927828b..28245dcfdf 100644 --- a/esphome/components/improv_serial/improv_serial_component.cpp +++ b/esphome/components/improv_serial/improv_serial_component.cpp @@ -15,11 +15,10 @@ static const char *const TAG = "improv_serial"; void ImprovSerialComponent::setup() { global_improv_serial_component = this; -#ifdef USE_ARDUINO - this->hw_serial_ = logger::global_logger->get_hw_serial(); -#endif -#ifdef USE_ESP_IDF +#ifdef USE_ESP32 this->uart_num_ = logger::global_logger->get_uart_num(); +#elif defined(USE_ARDUINO) + this->hw_serial_ = logger::global_logger->get_hw_serial(); #endif if (wifi::global_wifi_component->has_sta()) { @@ -34,13 +33,7 @@ void ImprovSerialComponent::dump_config() { ESP_LOGCONFIG(TAG, "Improv Serial:") optional ImprovSerialComponent::read_byte_() { optional byte; uint8_t data = 0; -#ifdef USE_ARDUINO - if (this->hw_serial_->available()) { - this->hw_serial_->readBytes(&data, 1); - byte = data; - } -#endif -#ifdef USE_ESP_IDF +#ifdef USE_ESP32 switch (logger::global_logger->get_uart()) { case logger::UART_SELECTION_UART0: case logger::UART_SELECTION_UART1: @@ -76,16 +69,18 @@ optional ImprovSerialComponent::read_byte_() { default: break; } +#elif defined(USE_ARDUINO) + if (this->hw_serial_->available()) { + this->hw_serial_->readBytes(&data, 1); + byte = data; + } #endif return byte; } void ImprovSerialComponent::write_data_(std::vector &data) { data.push_back('\n'); -#ifdef USE_ARDUINO - this->hw_serial_->write(data.data(), data.size()); -#endif -#ifdef USE_ESP_IDF +#ifdef USE_ESP32 switch (logger::global_logger->get_uart()) { case logger::UART_SELECTION_UART0: case logger::UART_SELECTION_UART1: @@ -112,6 +107,8 @@ void ImprovSerialComponent::write_data_(std::vector &data) { default: break; } +#elif defined(USE_ARDUINO) + this->hw_serial_->write(data.data(), data.size()); #endif } @@ -221,7 +218,7 @@ bool ImprovSerialComponent::parse_improv_payload_(improv::ImprovCommand &command } case improv::GET_WIFI_NETWORKS: { std::vector networks; - auto results = wifi::global_wifi_component->get_scan_result(); + const auto &results = wifi::global_wifi_component->get_scan_result(); for (auto &scan : results) { if (scan.get_is_hidden()) continue; diff --git a/esphome/components/improv_serial/improv_serial_component.h b/esphome/components/improv_serial/improv_serial_component.h index 5d2534c2fc..c3c9aee24e 100644 --- a/esphome/components/improv_serial/improv_serial_component.h +++ b/esphome/components/improv_serial/improv_serial_component.h @@ -9,10 +9,7 @@ #include #include -#ifdef USE_ARDUINO -#include -#endif -#ifdef USE_ESP_IDF +#ifdef USE_ESP32 #include #if defined(USE_ESP32_VARIANT_ESP32C3) || defined(USE_ESP32_VARIANT_ESP32C6) || defined(USE_ESP32_VARIANT_ESP32S3) || \ defined(USE_ESP32_VARIANT_ESP32H2) @@ -22,6 +19,8 @@ #if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) #include #endif +#elif defined(USE_ARDUINO) +#include #endif namespace esphome { @@ -60,11 +59,10 @@ class ImprovSerialComponent : public Component, public improv_base::ImprovBase { optional read_byte_(); void write_data_(std::vector &data); -#ifdef USE_ARDUINO - Stream *hw_serial_{nullptr}; -#endif -#ifdef USE_ESP_IDF +#ifdef USE_ESP32 uart_port_t uart_num_; +#elif defined(USE_ARDUINO) + Stream *hw_serial_{nullptr}; #endif std::vector rx_buffer_; diff --git a/esphome/components/ina219/ina219.cpp b/esphome/components/ina219/ina219.cpp index 52a3b1e067..ea8c5cea9d 100644 --- a/esphome/components/ina219/ina219.cpp +++ b/esphome/components/ina219/ina219.cpp @@ -34,7 +34,6 @@ static const uint8_t INA219_REGISTER_CURRENT = 0x04; static const uint8_t INA219_REGISTER_CALIBRATION = 0x05; void INA219Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // Config Register // 0bx000000000000000 << 15 RESET Bit (1 -> trigger reset) if (!this->write_byte_16(INA219_REGISTER_CONFIG, 0x8000)) { diff --git a/esphome/components/ina226/ina226.cpp b/esphome/components/ina226/ina226.cpp index 52e7127708..c4d4fb896e 100644 --- a/esphome/components/ina226/ina226.cpp +++ b/esphome/components/ina226/ina226.cpp @@ -37,8 +37,6 @@ static const uint16_t INA226_ADC_TIMES[] = {140, 204, 332, 588, 1100, 2116, 4156 static const uint16_t INA226_ADC_AVG_SAMPLES[] = {1, 4, 16, 64, 128, 256, 512, 1024}; void INA226Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - ConfigurationRegister config; config.reset = 1; diff --git a/esphome/components/ina260/ina260.cpp b/esphome/components/ina260/ina260.cpp index 2b6208f60f..9dd922cec2 100644 --- a/esphome/components/ina260/ina260.cpp +++ b/esphome/components/ina260/ina260.cpp @@ -35,8 +35,6 @@ static const uint8_t INA260_REGISTER_MANUFACTURE_ID = 0xFE; static const uint8_t INA260_REGISTER_DEVICE_ID = 0xFF; void INA260Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // Reset device on setup if (!this->write_byte_16(INA260_REGISTER_CONFIG, 0x8000)) { this->error_code_ = DEVICE_RESET_FAILED; diff --git a/esphome/components/ina2xx_base/__init__.py b/esphome/components/ina2xx_base/__init__.py index 7c3d3c8899..ce68ad2726 100644 --- a/esphome/components/ina2xx_base/__init__.py +++ b/esphome/components/ina2xx_base/__init__.py @@ -18,6 +18,7 @@ from esphome.const import ( DEVICE_CLASS_TEMPERATURE, DEVICE_CLASS_VOLTAGE, STATE_CLASS_MEASUREMENT, + STATE_CLASS_TOTAL_INCREASING, UNIT_AMPERE, UNIT_CELSIUS, UNIT_VOLT, @@ -34,6 +35,7 @@ CONF_CHARGE = "charge" CONF_CHARGE_COULOMBS = "charge_coulombs" CONF_ENERGY_JOULES = "energy_joules" CONF_TEMPERATURE_COEFFICIENT = "temperature_coefficient" +CONF_RESET_ON_BOOT = "reset_on_boot" UNIT_AMPERE_HOURS = "Ah" UNIT_COULOMB = "C" UNIT_JOULE = "J" @@ -78,11 +80,8 @@ def validate_model_config(config): model = config[CONF_MODEL] for key in config: - if key in SENSOR_MODEL_OPTIONS: - if model not in SENSOR_MODEL_OPTIONS[key]: - raise cv.Invalid( - f"Device model '{model}' does not support '{key}' sensor" - ) + if key in SENSOR_MODEL_OPTIONS and model not in SENSOR_MODEL_OPTIONS[key]: + raise cv.Invalid(f"Device model '{model}' does not support '{key}' sensor") tempco = config[CONF_TEMPERATURE_COEFFICIENT] if tempco > 0 and model not in ["INA228", "INA229"]: @@ -115,6 +114,7 @@ INA2XX_SCHEMA = cv.Schema( cv.Optional(CONF_TEMPERATURE_COEFFICIENT, default=0): cv.int_range( min=0, max=16383 ), + cv.Optional(CONF_RESET_ON_BOOT, default=True): cv.boolean, cv.Optional(CONF_SHUNT_VOLTAGE): cv.maybe_simple_value( sensor.sensor_schema( unit_of_measurement=UNIT_MILLIVOLT, @@ -165,7 +165,7 @@ INA2XX_SCHEMA = cv.Schema( unit_of_measurement=UNIT_WATT_HOURS, accuracy_decimals=8, device_class=DEVICE_CLASS_ENERGY, - state_class=STATE_CLASS_MEASUREMENT, + state_class=STATE_CLASS_TOTAL_INCREASING, ), key=CONF_NAME, ), @@ -173,7 +173,8 @@ INA2XX_SCHEMA = cv.Schema( sensor.sensor_schema( unit_of_measurement=UNIT_JOULE, accuracy_decimals=8, - state_class=STATE_CLASS_MEASUREMENT, + device_class=DEVICE_CLASS_ENERGY, + state_class=STATE_CLASS_TOTAL_INCREASING, ), key=CONF_NAME, ), @@ -207,6 +208,7 @@ async def setup_ina2xx(var, config): cg.add(var.set_adc_range(config[CONF_ADC_RANGE])) cg.add(var.set_adc_avg_samples(config[CONF_ADC_AVERAGING])) cg.add(var.set_shunt_tempco(config[CONF_TEMPERATURE_COEFFICIENT])) + cg.add(var.set_reset_on_boot(config[CONF_RESET_ON_BOOT])) adc_time_config = config[CONF_ADC_TIME] if isinstance(adc_time_config, dict): diff --git a/esphome/components/ina2xx_base/ina2xx_base.cpp b/esphome/components/ina2xx_base/ina2xx_base.cpp index 2112a28b02..4ab02703e8 100644 --- a/esphome/components/ina2xx_base/ina2xx_base.cpp +++ b/esphome/components/ina2xx_base/ina2xx_base.cpp @@ -50,8 +50,6 @@ static bool check_model_and_device_match(INAModel model, uint16_t dev_id) { } void INA2XX::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - if (!this->reset_config_()) { ESP_LOGE(TAG, "Reset failed, check connection"); this->mark_failed(); @@ -259,7 +257,12 @@ bool INA2XX::reset_energy_counters() { bool INA2XX::reset_config_() { ESP_LOGV(TAG, "Reset"); ConfigurationRegister cfg{0}; - cfg.RST = true; + if (!this->reset_on_boot_) { + ESP_LOGI(TAG, "Skipping on-boot device reset"); + cfg.RST = false; + } else { + cfg.RST = true; + } return this->write_unsigned_16_(RegisterMap::REG_CONFIG, cfg.raw_u16); } diff --git a/esphome/components/ina2xx_base/ina2xx_base.h b/esphome/components/ina2xx_base/ina2xx_base.h index 261c5321bf..ba0999b28e 100644 --- a/esphome/components/ina2xx_base/ina2xx_base.h +++ b/esphome/components/ina2xx_base/ina2xx_base.h @@ -127,6 +127,7 @@ class INA2XX : public PollingComponent { void set_adc_time_die_temperature(AdcTime time) { this->adc_time_die_temperature_ = time; } void set_adc_avg_samples(AdcAvgSamples samples) { this->adc_avg_samples_ = samples; } void set_shunt_tempco(uint16_t coeff) { this->shunt_tempco_ppm_c_ = coeff; } + void set_reset_on_boot(bool reset) { this->reset_on_boot_ = reset; } void set_shunt_voltage_sensor(sensor::Sensor *sensor) { this->shunt_voltage_sensor_ = sensor; } void set_bus_voltage_sensor(sensor::Sensor *sensor) { this->bus_voltage_sensor_ = sensor; } @@ -172,6 +173,7 @@ class INA2XX : public PollingComponent { AdcTime adc_time_die_temperature_{AdcTime::ADC_TIME_4120US}; AdcAvgSamples adc_avg_samples_{AdcAvgSamples::ADC_AVG_SAMPLES_128}; uint16_t shunt_tempco_ppm_c_{0}; + bool reset_on_boot_{true}; // // Calculated coefficients diff --git a/esphome/components/ina2xx_i2c/ina2xx_i2c.cpp b/esphome/components/ina2xx_i2c/ina2xx_i2c.cpp index d28525635d..a363a9c12f 100644 --- a/esphome/components/ina2xx_i2c/ina2xx_i2c.cpp +++ b/esphome/components/ina2xx_i2c/ina2xx_i2c.cpp @@ -21,7 +21,7 @@ void INA2XXI2C::dump_config() { } bool INA2XXI2C::read_ina_register(uint8_t reg, uint8_t *data, size_t len) { - auto ret = this->read_register(reg, data, len, false); + auto ret = this->read_register(reg, data, len); if (ret != i2c::ERROR_OK) { ESP_LOGE(TAG, "read_ina_register_ failed. Reg=0x%02X Err=%d", reg, ret); } diff --git a/esphome/components/ina3221/ina3221.cpp b/esphome/components/ina3221/ina3221.cpp index 35e79462ab..8243764147 100644 --- a/esphome/components/ina3221/ina3221.cpp +++ b/esphome/components/ina3221/ina3221.cpp @@ -22,7 +22,6 @@ static const uint8_t INA3221_REGISTER_CHANNEL3_BUS_VOLTAGE = 0x06; // A0 = SCL -> 0x43 void INA3221Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // Config Register // 0bx000000000000000 << 15 RESET Bit (1 -> trigger reset) if (!this->write_byte_16(INA3221_REGISTER_CONFIG, 0x8000)) { diff --git a/esphome/components/inkplate/__init__.py b/esphome/components/inkplate/__init__.py new file mode 100644 index 0000000000..1c6013793a --- /dev/null +++ b/esphome/components/inkplate/__init__.py @@ -0,0 +1 @@ +CODEOWNERS = ["@jesserockz", "@JosipKuci"] diff --git a/esphome/components/inkplate/const.py b/esphome/components/inkplate/const.py new file mode 100644 index 0000000000..77bf933320 --- /dev/null +++ b/esphome/components/inkplate/const.py @@ -0,0 +1,105 @@ +WAVEFORMS = { + "inkplate_6": ( + (0, 1, 1, 0, 0, 1, 1, 0, 0), + (0, 1, 2, 1, 1, 2, 1, 0, 0), + (1, 1, 1, 2, 2, 1, 0, 0, 0), + (0, 0, 0, 1, 1, 1, 2, 0, 0), + (2, 1, 1, 1, 2, 1, 2, 0, 0), + (2, 2, 1, 1, 2, 1, 2, 0, 0), + (1, 1, 1, 2, 1, 2, 2, 0, 0), + (0, 0, 0, 0, 0, 0, 2, 0, 0), + ), + "inkplate_10": ( + (0, 0, 0, 0, 0, 0, 0, 1, 0), + (0, 0, 0, 2, 2, 2, 1, 1, 0), + (0, 0, 2, 1, 1, 2, 2, 1, 0), + (0, 1, 2, 2, 1, 2, 2, 1, 0), + (0, 0, 2, 1, 2, 2, 2, 1, 0), + (0, 2, 2, 2, 2, 2, 2, 1, 0), + (0, 0, 0, 0, 0, 2, 1, 2, 0), + (0, 0, 0, 2, 2, 2, 2, 2, 0), + ), + "inkplate_6_plus": ( + (0, 0, 0, 0, 0, 2, 1, 1, 0), + (0, 0, 2, 1, 1, 1, 2, 1, 0), + (0, 2, 2, 2, 1, 1, 2, 1, 0), + (0, 0, 2, 2, 2, 1, 2, 1, 0), + (0, 0, 0, 0, 2, 2, 2, 1, 0), + (0, 0, 2, 1, 2, 1, 1, 2, 0), + (0, 0, 2, 2, 2, 1, 1, 2, 0), + (0, 0, 0, 0, 2, 2, 2, 2, 0), + ), + "inkplate_6_v2": ( + (1, 0, 1, 0, 1, 1, 1, 0, 0), + (0, 0, 0, 1, 1, 1, 1, 0, 0), + (1, 1, 1, 1, 0, 2, 1, 0, 0), + (1, 1, 1, 2, 2, 1, 1, 0, 0), + (1, 1, 1, 1, 2, 2, 1, 0, 0), + (0, 1, 1, 1, 2, 2, 1, 0, 0), + (0, 0, 0, 0, 1, 1, 2, 0, 0), + (0, 0, 0, 0, 0, 1, 2, 0, 0), + ), + "inkplate_5": ( + (0, 0, 1, 1, 0, 1, 1, 1, 0), + (0, 1, 1, 1, 1, 2, 0, 1, 0), + (1, 2, 2, 0, 2, 1, 1, 1, 0), + (1, 1, 1, 2, 0, 1, 1, 2, 0), + (0, 1, 1, 1, 2, 0, 1, 2, 0), + (0, 0, 0, 1, 1, 2, 1, 2, 0), + (1, 1, 1, 2, 0, 2, 1, 2, 0), + (0, 0, 0, 0, 0, 0, 0, 0, 0), + ), + "inkplate_5_v2": ( + (0, 0, 1, 1, 2, 1, 1, 1, 0), + (1, 1, 2, 2, 1, 2, 1, 1, 0), + (0, 1, 2, 2, 1, 1, 2, 1, 0), + (0, 0, 1, 1, 1, 1, 1, 2, 0), + (1, 2, 1, 2, 1, 1, 1, 2, 0), + (0, 1, 1, 1, 2, 0, 1, 2, 0), + (1, 1, 1, 2, 2, 2, 1, 2, 0), + (0, 0, 0, 0, 0, 0, 0, 0, 0), + ), +} + +INKPLATE_10_CUSTOM_WAVEFORMS = ( + ( + (0, 0, 0, 0, 0, 0, 0, 0, 0), + (0, 0, 0, 2, 1, 2, 1, 1, 0), + (0, 0, 0, 2, 2, 1, 2, 1, 0), + (0, 0, 2, 2, 1, 2, 2, 1, 0), + (0, 0, 0, 2, 1, 1, 1, 2, 0), + (0, 0, 2, 2, 2, 1, 1, 2, 0), + (0, 0, 0, 0, 0, 1, 2, 2, 0), + (0, 0, 0, 0, 2, 2, 2, 2, 0), + ), + ( + (0, 3, 3, 3, 3, 3, 3, 3, 0), + (0, 1, 2, 1, 1, 2, 2, 1, 0), + (0, 2, 2, 2, 1, 2, 2, 1, 0), + (0, 0, 2, 2, 2, 2, 2, 1, 0), + (0, 3, 3, 2, 1, 1, 1, 2, 0), + (0, 3, 3, 2, 2, 1, 1, 2, 0), + (0, 2, 1, 2, 1, 2, 1, 2, 0), + (0, 3, 3, 3, 2, 2, 2, 2, 0), + ), + ( + (0, 0, 0, 0, 0, 0, 0, 1, 0), + (0, 0, 0, 2, 2, 2, 1, 1, 0), + (0, 0, 2, 1, 1, 2, 2, 1, 0), + (1, 1, 2, 2, 1, 2, 2, 1, 0), + (0, 0, 2, 1, 2, 2, 2, 1, 0), + (0, 1, 2, 2, 2, 2, 2, 1, 0), + (0, 0, 0, 2, 2, 2, 1, 2, 0), + (0, 0, 0, 2, 2, 2, 2, 2, 0), + ), + ( + (0, 0, 0, 0, 0, 0, 0, 1, 0), + (0, 0, 0, 2, 2, 2, 1, 1, 0), + (2, 2, 2, 1, 0, 2, 1, 0, 0), + (2, 1, 1, 2, 1, 1, 1, 2, 0), + (2, 2, 2, 1, 1, 1, 0, 2, 0), + (2, 2, 2, 1, 1, 2, 1, 2, 0), + (0, 0, 0, 0, 2, 1, 2, 2, 0), + (0, 0, 0, 0, 2, 2, 2, 2, 0), + ), +) diff --git a/esphome/components/inkplate/display.py b/esphome/components/inkplate/display.py new file mode 100644 index 0000000000..a0b0265cf1 --- /dev/null +++ b/esphome/components/inkplate/display.py @@ -0,0 +1,238 @@ +from esphome import pins +import esphome.codegen as cg +from esphome.components import display, i2c +from esphome.components.esp32 import CONF_CPU_FREQUENCY +import esphome.config_validation as cv +from esphome.const import ( + CONF_FULL_UPDATE_EVERY, + CONF_ID, + CONF_LAMBDA, + CONF_MIRROR_X, + CONF_MIRROR_Y, + CONF_MODEL, + CONF_OE_PIN, + CONF_PAGES, + CONF_TRANSFORM, + CONF_WAKEUP_PIN, + PLATFORM_ESP32, +) +import esphome.final_validate as fv + +from .const import INKPLATE_10_CUSTOM_WAVEFORMS, WAVEFORMS + +DEPENDENCIES = ["i2c", "esp32"] +AUTO_LOAD = ["psram"] + +CONF_DISPLAY_DATA_0_PIN = "display_data_0_pin" +CONF_DISPLAY_DATA_1_PIN = "display_data_1_pin" +CONF_DISPLAY_DATA_2_PIN = "display_data_2_pin" +CONF_DISPLAY_DATA_3_PIN = "display_data_3_pin" +CONF_DISPLAY_DATA_4_PIN = "display_data_4_pin" +CONF_DISPLAY_DATA_5_PIN = "display_data_5_pin" +CONF_DISPLAY_DATA_6_PIN = "display_data_6_pin" +CONF_DISPLAY_DATA_7_PIN = "display_data_7_pin" + +CONF_CL_PIN = "cl_pin" +CONF_CKV_PIN = "ckv_pin" +CONF_GREYSCALE = "greyscale" +CONF_GMOD_PIN = "gmod_pin" +CONF_GPIO0_ENABLE_PIN = "gpio0_enable_pin" +CONF_LE_PIN = "le_pin" +CONF_PARTIAL_UPDATING = "partial_updating" +CONF_POWERUP_PIN = "powerup_pin" +CONF_SPH_PIN = "sph_pin" +CONF_SPV_PIN = "spv_pin" +CONF_VCOM_PIN = "vcom_pin" + +inkplate_ns = cg.esphome_ns.namespace("inkplate") +Inkplate = inkplate_ns.class_( + "Inkplate", + cg.PollingComponent, + i2c.I2CDevice, + display.Display, + display.DisplayBuffer, +) + +InkplateModel = inkplate_ns.enum("InkplateModel") + +MODELS = { + "inkplate_6": InkplateModel.INKPLATE_6, + "inkplate_10": InkplateModel.INKPLATE_10, + "inkplate_6_plus": InkplateModel.INKPLATE_6_PLUS, + "inkplate_6_v2": InkplateModel.INKPLATE_6_V2, + "inkplate_5": InkplateModel.INKPLATE_5, + "inkplate_5_v2": InkplateModel.INKPLATE_5_V2, +} + +CONF_CUSTOM_WAVEFORM = "custom_waveform" + + +def _validate_custom_waveform(config): + if CONF_CUSTOM_WAVEFORM in config and config[CONF_MODEL] != "inkplate_10": + raise cv.Invalid("Custom waveforms are only supported on the Inkplate 10") + return config + + +CONFIG_SCHEMA = cv.All( + display.FULL_DISPLAY_SCHEMA.extend( + { + cv.GenerateID(): cv.declare_id(Inkplate), + cv.Optional(CONF_GREYSCALE, default=False): cv.boolean, + cv.Optional(CONF_CUSTOM_WAVEFORM): cv.All( + cv.uint8_t, cv.Range(min=1, max=len(INKPLATE_10_CUSTOM_WAVEFORMS)) + ), + cv.Optional(CONF_TRANSFORM): cv.Schema( + { + cv.Optional(CONF_MIRROR_X, default=False): cv.boolean, + cv.Optional(CONF_MIRROR_Y, default=False): cv.boolean, + } + ), + cv.Optional(CONF_PARTIAL_UPDATING, default=True): cv.boolean, + cv.Optional(CONF_FULL_UPDATE_EVERY, default=10): cv.uint32_t, + cv.Optional(CONF_MODEL, default="inkplate_6"): cv.enum( + MODELS, lower=True, space="_" + ), + # Control pins + cv.Required(CONF_CKV_PIN): pins.gpio_output_pin_schema, + cv.Required(CONF_GMOD_PIN): pins.gpio_output_pin_schema, + cv.Required(CONF_GPIO0_ENABLE_PIN): pins.gpio_output_pin_schema, + cv.Required(CONF_OE_PIN): pins.gpio_output_pin_schema, + cv.Required(CONF_POWERUP_PIN): pins.gpio_output_pin_schema, + cv.Required(CONF_SPH_PIN): pins.gpio_output_pin_schema, + cv.Required(CONF_SPV_PIN): pins.gpio_output_pin_schema, + cv.Required(CONF_VCOM_PIN): pins.gpio_output_pin_schema, + cv.Required(CONF_WAKEUP_PIN): pins.gpio_output_pin_schema, + cv.Optional(CONF_CL_PIN, default=0): pins.internal_gpio_output_pin_schema, + cv.Optional(CONF_LE_PIN, default=2): pins.internal_gpio_output_pin_schema, + # Data pins + cv.Optional( + CONF_DISPLAY_DATA_0_PIN, default=4 + ): pins.internal_gpio_output_pin_schema, + cv.Optional( + CONF_DISPLAY_DATA_1_PIN, default=5 + ): pins.internal_gpio_output_pin_schema, + cv.Optional( + CONF_DISPLAY_DATA_2_PIN, default=18 + ): pins.internal_gpio_output_pin_schema, + cv.Optional( + CONF_DISPLAY_DATA_3_PIN, default=19 + ): pins.internal_gpio_output_pin_schema, + cv.Optional( + CONF_DISPLAY_DATA_4_PIN, default=23 + ): pins.internal_gpio_output_pin_schema, + cv.Optional( + CONF_DISPLAY_DATA_5_PIN, default=25 + ): pins.internal_gpio_output_pin_schema, + cv.Optional( + CONF_DISPLAY_DATA_6_PIN, default=26 + ): pins.internal_gpio_output_pin_schema, + cv.Optional( + CONF_DISPLAY_DATA_7_PIN, default=27 + ): pins.internal_gpio_output_pin_schema, + } + ) + .extend(cv.polling_component_schema("5s")) + .extend(i2c.i2c_device_schema(0x48)), + cv.has_at_most_one_key(CONF_PAGES, CONF_LAMBDA), + _validate_custom_waveform, +) + + +def _validate_cpu_frequency(config): + esp32_config = fv.full_config.get()[PLATFORM_ESP32] + if esp32_config[CONF_CPU_FREQUENCY] != "240MHZ": + raise cv.Invalid( + "Inkplate requires 240MHz CPU frequency (set in esp32 component)" + ) + return config + + +FINAL_VALIDATE_SCHEMA = _validate_cpu_frequency + + +async def to_code(config): + var = cg.new_Pvariable(config[CONF_ID]) + + await display.register_display(var, config) + await i2c.register_i2c_device(var, config) + + if CONF_LAMBDA in config: + lambda_ = await cg.process_lambda( + config[CONF_LAMBDA], [(display.DisplayRef, "it")], return_type=cg.void + ) + cg.add(var.set_writer(lambda_)) + + cg.add(var.set_greyscale(config[CONF_GREYSCALE])) + if transform := config.get(CONF_TRANSFORM): + cg.add(var.set_mirror_x(transform[CONF_MIRROR_X])) + cg.add(var.set_mirror_y(transform[CONF_MIRROR_Y])) + cg.add(var.set_partial_updating(config[CONF_PARTIAL_UPDATING])) + cg.add(var.set_full_update_every(config[CONF_FULL_UPDATE_EVERY])) + + cg.add(var.set_model(config[CONF_MODEL])) + + if custom_waveform := config.get(CONF_CUSTOM_WAVEFORM): + waveform = INKPLATE_10_CUSTOM_WAVEFORMS[custom_waveform - 1] + waveform = [element for tupl in waveform for element in tupl] + cg.add(var.set_waveform(waveform, True)) + else: + waveform = WAVEFORMS[config[CONF_MODEL]] + waveform = [element for tupl in waveform for element in tupl] + cg.add(var.set_waveform(waveform, False)) + + ckv = await cg.gpio_pin_expression(config[CONF_CKV_PIN]) + cg.add(var.set_ckv_pin(ckv)) + + gmod = await cg.gpio_pin_expression(config[CONF_GMOD_PIN]) + cg.add(var.set_gmod_pin(gmod)) + + gpio0_enable = await cg.gpio_pin_expression(config[CONF_GPIO0_ENABLE_PIN]) + cg.add(var.set_gpio0_enable_pin(gpio0_enable)) + + oe = await cg.gpio_pin_expression(config[CONF_OE_PIN]) + cg.add(var.set_oe_pin(oe)) + + powerup = await cg.gpio_pin_expression(config[CONF_POWERUP_PIN]) + cg.add(var.set_powerup_pin(powerup)) + + sph = await cg.gpio_pin_expression(config[CONF_SPH_PIN]) + cg.add(var.set_sph_pin(sph)) + + spv = await cg.gpio_pin_expression(config[CONF_SPV_PIN]) + cg.add(var.set_spv_pin(spv)) + + vcom = await cg.gpio_pin_expression(config[CONF_VCOM_PIN]) + cg.add(var.set_vcom_pin(vcom)) + + wakeup = await cg.gpio_pin_expression(config[CONF_WAKEUP_PIN]) + cg.add(var.set_wakeup_pin(wakeup)) + + cl = await cg.gpio_pin_expression(config[CONF_CL_PIN]) + cg.add(var.set_cl_pin(cl)) + + le = await cg.gpio_pin_expression(config[CONF_LE_PIN]) + cg.add(var.set_le_pin(le)) + + display_data_0 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_0_PIN]) + cg.add(var.set_display_data_0_pin(display_data_0)) + + display_data_1 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_1_PIN]) + cg.add(var.set_display_data_1_pin(display_data_1)) + + display_data_2 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_2_PIN]) + cg.add(var.set_display_data_2_pin(display_data_2)) + + display_data_3 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_3_PIN]) + cg.add(var.set_display_data_3_pin(display_data_3)) + + display_data_4 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_4_PIN]) + cg.add(var.set_display_data_4_pin(display_data_4)) + + display_data_5 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_5_PIN]) + cg.add(var.set_display_data_5_pin(display_data_5)) + + display_data_6 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_6_PIN]) + cg.add(var.set_display_data_6_pin(display_data_6)) + + display_data_7 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_7_PIN]) + cg.add(var.set_display_data_7_pin(display_data_7)) diff --git a/esphome/components/inkplate6/inkplate.cpp b/esphome/components/inkplate/inkplate.cpp similarity index 82% rename from esphome/components/inkplate6/inkplate.cpp rename to esphome/components/inkplate/inkplate.cpp index b3d0b87e83..f96fb6905e 100644 --- a/esphome/components/inkplate6/inkplate.cpp +++ b/esphome/components/inkplate/inkplate.cpp @@ -6,11 +6,11 @@ #include namespace esphome { -namespace inkplate6 { +namespace inkplate { static const char *const TAG = "inkplate"; -void Inkplate6::setup() { +void Inkplate::setup() { for (uint32_t i = 0; i < 256; i++) { this->pin_lut_[i] = ((i & 0b00000011) << 4) | (((i & 0b00001100) >> 2) << 18) | (((i & 0b00010000) >> 4) << 23) | (((i & 0b11100000) >> 5) << 25); @@ -56,7 +56,7 @@ void Inkplate6::setup() { /** * Allocate buffers. May be called after setup to re-initialise if e.g. greyscale is changed. */ -void Inkplate6::initialize_() { +void Inkplate::initialize_() { RAMAllocator allocator; RAMAllocator allocator32; uint32_t buffer_size = this->get_buffer_length_(); @@ -81,29 +81,25 @@ void Inkplate6::initialize_() { return; } if (this->greyscale_) { - uint8_t glut_size = 9; - - this->glut_ = allocator32.allocate(256 * glut_size); + this->glut_ = allocator32.allocate(256 * GLUT_SIZE); if (this->glut_ == nullptr) { ESP_LOGE(TAG, "Could not allocate glut!"); this->mark_failed(); return; } - this->glut2_ = allocator32.allocate(256 * glut_size); + this->glut2_ = allocator32.allocate(256 * GLUT_SIZE); if (this->glut2_ == nullptr) { ESP_LOGE(TAG, "Could not allocate glut2!"); this->mark_failed(); return; } - const auto *const waveform3_bit = waveform3BitAll[this->model_]; - - for (int i = 0; i < glut_size; i++) { + for (uint8_t i = 0; i < GLUT_SIZE; i++) { for (uint32_t j = 0; j < 256; j++) { - uint8_t z = (waveform3_bit[j & 0x07][i] << 2) | (waveform3_bit[(j >> 4) & 0x07][i]); + uint8_t z = (this->waveform_[j & 0x07][i] << 2) | (this->waveform_[(j >> 4) & 0x07][i]); this->glut_[i * 256 + j] = ((z & 0b00000011) << 4) | (((z & 0b00001100) >> 2) << 18) | (((z & 0b00010000) >> 4) << 23) | (((z & 0b11100000) >> 5) << 25); - z = ((waveform3_bit[j & 0x07][i] << 2) | (waveform3_bit[(j >> 4) & 0x07][i])) << 4; + z = ((this->waveform_[j & 0x07][i] << 2) | (this->waveform_[(j >> 4) & 0x07][i])) << 4; this->glut2_[i * 256 + j] = ((z & 0b00000011) << 4) | (((z & 0b00001100) >> 2) << 18) | (((z & 0b00010000) >> 4) << 23) | (((z & 0b11100000) >> 5) << 25); } @@ -130,9 +126,9 @@ void Inkplate6::initialize_() { memset(this->buffer_, 0, buffer_size); } -float Inkplate6::get_setup_priority() const { return setup_priority::PROCESSOR; } +float Inkplate::get_setup_priority() const { return setup_priority::PROCESSOR; } -size_t Inkplate6::get_buffer_length_() { +size_t Inkplate::get_buffer_length_() { if (this->greyscale_) { return size_t(this->get_width_internal()) * size_t(this->get_height_internal()) / 2u; } else { @@ -140,7 +136,7 @@ size_t Inkplate6::get_buffer_length_() { } } -void Inkplate6::update() { +void Inkplate::update() { this->do_update_(); if (this->full_update_every_ > 0 && this->partial_updates_ >= this->full_update_every_) { @@ -150,7 +146,7 @@ void Inkplate6::update() { this->display(); } -void HOT Inkplate6::draw_absolute_pixel_internal(int x, int y, Color color) { +void HOT Inkplate::draw_absolute_pixel_internal(int x, int y, Color color) { if (x >= this->get_width_internal() || y >= this->get_height_internal() || x < 0 || y < 0) return; @@ -171,18 +167,18 @@ void HOT Inkplate6::draw_absolute_pixel_internal(int x, int y, Color color) { // uint8_t gs = (uint8_t)(px*7); uint8_t gs = ((color.red * 2126 / 10000) + (color.green * 7152 / 10000) + (color.blue * 722 / 10000)) >> 5; - this->buffer_[pos] = (pixelMaskGLUT[x_sub] & current) | (x_sub ? gs : gs << 4); + this->buffer_[pos] = (PIXEL_MASK_GLUT[x_sub] & current) | (x_sub ? gs : gs << 4); } else { int x1 = x / 8; int x_sub = x % 8; uint32_t pos = (x1 + y * (this->get_width_internal() / 8)); uint8_t current = this->partial_buffer_[pos]; - this->partial_buffer_[pos] = (~pixelMaskLUT[x_sub] & current) | (color.is_on() ? 0 : pixelMaskLUT[x_sub]); + this->partial_buffer_[pos] = (~PIXEL_MASK_LUT[x_sub] & current) | (color.is_on() ? 0 : PIXEL_MASK_LUT[x_sub]); } } -void Inkplate6::dump_config() { +void Inkplate::dump_config() { LOG_DISPLAY("", "Inkplate", this); ESP_LOGCONFIG(TAG, " Greyscale: %s\n" @@ -214,7 +210,7 @@ void Inkplate6::dump_config() { LOG_UPDATE_INTERVAL(this); } -void Inkplate6::eink_off_() { +void Inkplate::eink_off_() { ESP_LOGV(TAG, "Eink off called"); if (!panel_on_) return; @@ -242,7 +238,7 @@ void Inkplate6::eink_off_() { pins_z_state_(); } -void Inkplate6::eink_on_() { +void Inkplate::eink_on_() { ESP_LOGV(TAG, "Eink on called"); if (panel_on_) return; @@ -284,7 +280,7 @@ void Inkplate6::eink_on_() { this->oe_pin_->digital_write(true); } -bool Inkplate6::read_power_status_() { +bool Inkplate::read_power_status_() { uint8_t data; auto err = this->read_register(0x0F, &data, 1); if (err == i2c::ERROR_OK) { @@ -293,7 +289,7 @@ bool Inkplate6::read_power_status_() { return false; } -void Inkplate6::fill(Color color) { +void Inkplate::fill(Color color) { ESP_LOGV(TAG, "Fill called"); uint32_t start_time = millis(); @@ -308,7 +304,7 @@ void Inkplate6::fill(Color color) { ESP_LOGV(TAG, "Fill finished (%ums)", millis() - start_time); } -void Inkplate6::display() { +void Inkplate::display() { ESP_LOGV(TAG, "Display called"); uint32_t start_time = millis(); @@ -324,7 +320,7 @@ void Inkplate6::display() { ESP_LOGV(TAG, "Display finished (full) (%ums)", millis() - start_time); } -void Inkplate6::display1b_() { +void Inkplate::display1b_() { ESP_LOGV(TAG, "Display1b called"); uint32_t start_time = millis(); @@ -334,32 +330,71 @@ void Inkplate6::display1b_() { uint8_t buffer_value; const uint8_t *buffer_ptr; eink_on_(); - if (this->model_ == INKPLATE_6_PLUS) { - clean_fast_(0, 1); - clean_fast_(1, 15); - clean_fast_(2, 1); - clean_fast_(0, 5); - clean_fast_(2, 1); - clean_fast_(1, 15); - } else { - clean_fast_(0, 1); - clean_fast_(1, 21); - clean_fast_(2, 1); - clean_fast_(0, 12); - clean_fast_(2, 1); - clean_fast_(1, 21); - clean_fast_(2, 1); - clean_fast_(0, 12); - clean_fast_(2, 1); + uint8_t rep = 4; + switch (this->model_) { + case INKPLATE_10: + clean_fast_(0, 1); + clean_fast_(1, 10); + clean_fast_(2, 1); + clean_fast_(0, 10); + clean_fast_(2, 1); + clean_fast_(1, 10); + clean_fast_(2, 1); + clean_fast_(0, 10); + rep = 5; + break; + case INKPLATE_6_PLUS: + clean_fast_(0, 1); + clean_fast_(1, 15); + clean_fast_(2, 1); + clean_fast_(0, 5); + clean_fast_(2, 1); + clean_fast_(1, 15); + break; + case INKPLATE_6: + case INKPLATE_6_V2: + clean_fast_(0, 1); + clean_fast_(1, 18); + clean_fast_(2, 1); + clean_fast_(0, 18); + clean_fast_(2, 1); + clean_fast_(1, 18); + clean_fast_(2, 1); + clean_fast_(0, 18); + clean_fast_(2, 1); + if (this->model_ == INKPLATE_6_V2) + rep = 5; + break; + case INKPLATE_5: + clean_fast_(0, 1); + clean_fast_(1, 14); + clean_fast_(2, 1); + clean_fast_(0, 14); + clean_fast_(2, 1); + clean_fast_(1, 14); + clean_fast_(2, 1); + clean_fast_(0, 14); + clean_fast_(2, 1); + rep = 5; + break; + case INKPLATE_5_V2: + clean_fast_(0, 1); + clean_fast_(1, 11); + clean_fast_(2, 1); + clean_fast_(0, 11); + clean_fast_(2, 1); + clean_fast_(1, 11); + clean_fast_(2, 1); + clean_fast_(0, 11); + rep = 3; + break; } uint32_t clock = (1 << this->cl_pin_->get_pin()); uint32_t data_mask = this->get_data_pin_mask_(); ESP_LOGV(TAG, "Display1b start loops (%ums)", millis() - start_time); - int rep = (this->model_ == INKPLATE_6_V2) ? 5 : 4; - - for (int k = 0; k < rep; k++) { + for (uint8_t k = 0; k < rep; k++) { buffer_ptr = &this->buffer_[this->get_buffer_length_() - 1]; vscan_start_(); for (int i = 0, im = this->get_height_internal(); i < im; i++) { @@ -452,28 +487,75 @@ void Inkplate6::display1b_() { ESP_LOGV(TAG, "Display1b finished (%ums)", millis() - start_time); } -void Inkplate6::display3b_() { +void Inkplate::display3b_() { ESP_LOGV(TAG, "Display3b called"); uint32_t start_time = millis(); eink_on_(); - if (this->model_ == INKPLATE_6_PLUS) { - clean_fast_(0, 1); - clean_fast_(1, 15); - clean_fast_(2, 1); - clean_fast_(0, 5); - clean_fast_(2, 1); - clean_fast_(1, 15); - } else { - clean_fast_(0, 1); - clean_fast_(1, 21); - clean_fast_(2, 1); - clean_fast_(0, 12); - clean_fast_(2, 1); - clean_fast_(1, 21); - clean_fast_(2, 1); - clean_fast_(0, 12); - clean_fast_(2, 1); + + switch (this->model_) { + case INKPLATE_10: + if (this->custom_waveform_) { + clean_fast_(1, 1); + clean_fast_(0, 7); + clean_fast_(2, 1); + clean_fast_(1, 12); + clean_fast_(2, 1); + clean_fast_(0, 7); + clean_fast_(2, 1); + clean_fast_(1, 12); + } else { + clean_fast_(1, 1); + clean_fast_(0, 10); + clean_fast_(2, 1); + clean_fast_(1, 10); + clean_fast_(2, 1); + clean_fast_(0, 10); + clean_fast_(2, 1); + clean_fast_(1, 10); + } + break; + case INKPLATE_6_PLUS: + clean_fast_(0, 1); + clean_fast_(1, 15); + clean_fast_(2, 1); + clean_fast_(0, 5); + clean_fast_(2, 1); + clean_fast_(1, 15); + break; + case INKPLATE_6: + case INKPLATE_6_V2: + clean_fast_(0, 1); + clean_fast_(1, 18); + clean_fast_(2, 1); + clean_fast_(0, 18); + clean_fast_(2, 1); + clean_fast_(1, 18); + clean_fast_(2, 1); + clean_fast_(0, 18); + clean_fast_(2, 1); + break; + case INKPLATE_5: + clean_fast_(0, 1); + clean_fast_(1, 14); + clean_fast_(2, 1); + clean_fast_(0, 14); + clean_fast_(2, 1); + clean_fast_(1, 14); + clean_fast_(2, 1); + clean_fast_(0, 14); + clean_fast_(2, 1); + break; + case INKPLATE_5_V2: + clean_fast_(0, 1); + clean_fast_(1, 11); + clean_fast_(2, 1); + clean_fast_(0, 11); + clean_fast_(2, 1); + clean_fast_(1, 11); + clean_fast_(2, 1); + clean_fast_(0, 11); + break; } uint32_t clock = (1 << this->cl_pin_->get_pin()); @@ -518,7 +600,7 @@ void Inkplate6::display3b_() { ESP_LOGV(TAG, "Display3b finished (%ums)", millis() - start_time); } -bool Inkplate6::partial_update_() { +bool Inkplate::partial_update_() { ESP_LOGV(TAG, "Partial update called"); uint32_t start_time = millis(); if (this->greyscale_) @@ -560,7 +642,7 @@ bool Inkplate6::partial_update_() { GPIO.out_w1ts = this->pin_lut_[data] | clock; GPIO.out_w1tc = data_mask | clock; } - // New Inkplate6 panel doesn't need last clock + // New Inkplate panel doesn't need last clock if (this->model_ != INKPLATE_6_V2) { GPIO.out_w1ts = clock; GPIO.out_w1tc = data_mask | clock; @@ -580,7 +662,7 @@ bool Inkplate6::partial_update_() { return true; } -void Inkplate6::vscan_start_() { +void Inkplate::vscan_start_() { this->ckv_pin_->digital_write(true); delayMicroseconds(7); this->spv_pin_->digital_write(false); @@ -604,7 +686,7 @@ void Inkplate6::vscan_start_() { this->ckv_pin_->digital_write(true); } -void Inkplate6::hscan_start_(uint32_t d) { +void Inkplate::hscan_start_(uint32_t d) { uint8_t clock = (1 << this->cl_pin_->get_pin()); this->sph_pin_->digital_write(false); GPIO.out_w1ts = d | clock; @@ -613,14 +695,14 @@ void Inkplate6::hscan_start_(uint32_t d) { this->ckv_pin_->digital_write(true); } -void Inkplate6::vscan_end_() { +void Inkplate::vscan_end_() { this->ckv_pin_->digital_write(false); this->le_pin_->digital_write(true); this->le_pin_->digital_write(false); delayMicroseconds(0); } -void Inkplate6::clean() { +void Inkplate::clean() { ESP_LOGV(TAG, "Clean called"); uint32_t start_time = millis(); @@ -634,7 +716,7 @@ void Inkplate6::clean() { ESP_LOGV(TAG, "Clean finished (%ums)", millis() - start_time); } -void Inkplate6::clean_fast_(uint8_t c, uint8_t rep) { +void Inkplate::clean_fast_(uint8_t c, uint8_t rep) { ESP_LOGV(TAG, "Clean fast called with: (%d, %d)", c, rep); uint32_t start_time = millis(); @@ -666,7 +748,7 @@ void Inkplate6::clean_fast_(uint8_t c, uint8_t rep) { GPIO.out_w1ts = clock; GPIO.out_w1tc = clock; } - // New Inkplate6 panel doesn't need last clock + // New Inkplate panel doesn't need last clock if (this->model_ != INKPLATE_6_V2) { GPIO.out_w1ts = send | clock; GPIO.out_w1tc = clock; @@ -679,7 +761,7 @@ void Inkplate6::clean_fast_(uint8_t c, uint8_t rep) { ESP_LOGV(TAG, "Clean fast finished (%ums)", millis() - start_time); } -void Inkplate6::pins_z_state_() { +void Inkplate::pins_z_state_() { this->cl_pin_->pin_mode(gpio::FLAG_INPUT); this->le_pin_->pin_mode(gpio::FLAG_INPUT); this->ckv_pin_->pin_mode(gpio::FLAG_INPUT); @@ -699,7 +781,7 @@ void Inkplate6::pins_z_state_() { this->display_data_7_pin_->pin_mode(gpio::FLAG_INPUT); } -void Inkplate6::pins_as_outputs_() { +void Inkplate::pins_as_outputs_() { this->cl_pin_->pin_mode(gpio::FLAG_OUTPUT); this->le_pin_->pin_mode(gpio::FLAG_OUTPUT); this->ckv_pin_->pin_mode(gpio::FLAG_OUTPUT); @@ -719,5 +801,5 @@ void Inkplate6::pins_as_outputs_() { this->display_data_7_pin_->pin_mode(gpio::FLAG_OUTPUT); } -} // namespace inkplate6 +} // namespace inkplate } // namespace esphome diff --git a/esphome/components/inkplate6/inkplate.h b/esphome/components/inkplate/inkplate.h similarity index 56% rename from esphome/components/inkplate6/inkplate.h rename to esphome/components/inkplate/inkplate.h index d8918bdf2a..fb4674b522 100644 --- a/esphome/components/inkplate6/inkplate.h +++ b/esphome/components/inkplate/inkplate.h @@ -5,8 +5,10 @@ #include "esphome/core/component.h" #include "esphome/core/hal.h" +#include + namespace esphome { -namespace inkplate6 { +namespace inkplate { enum InkplateModel : uint8_t { INKPLATE_6 = 0, @@ -17,79 +19,35 @@ enum InkplateModel : uint8_t { INKPLATE_5_V2 = 5, }; -class Inkplate6 : public display::DisplayBuffer, public i2c::I2CDevice { +static constexpr uint8_t GLUT_SIZE = 9; +static constexpr uint8_t GLUT_COUNT = 8; + +static constexpr uint8_t LUT2[16] = {0xAA, 0xA9, 0xA6, 0xA5, 0x9A, 0x99, 0x96, 0x95, + 0x6A, 0x69, 0x66, 0x65, 0x5A, 0x59, 0x56, 0x55}; +static constexpr uint8_t LUTW[16] = {0xFF, 0xFE, 0xFB, 0xFA, 0xEF, 0xEE, 0xEB, 0xEA, + 0xBF, 0xBE, 0xBB, 0xBA, 0xAF, 0xAE, 0xAB, 0xAA}; +static constexpr uint8_t LUTB[16] = {0xFF, 0xFD, 0xF7, 0xF5, 0xDF, 0xDD, 0xD7, 0xD5, + 0x7F, 0x7D, 0x77, 0x75, 0x5F, 0x5D, 0x57, 0x55}; + +static constexpr uint8_t PIXEL_MASK_LUT[8] = {0x1, 0x2, 0x4, 0x8, 0x10, 0x20, 0x40, 0x80}; +static constexpr uint8_t PIXEL_MASK_GLUT[2] = {0x0F, 0xF0}; + +class Inkplate : public display::DisplayBuffer, public i2c::I2CDevice { public: - const uint8_t LUT2[16] = {0xAA, 0xA9, 0xA6, 0xA5, 0x9A, 0x99, 0x96, 0x95, - 0x6A, 0x69, 0x66, 0x65, 0x5A, 0x59, 0x56, 0x55}; - const uint8_t LUTW[16] = {0xFF, 0xFE, 0xFB, 0xFA, 0xEF, 0xEE, 0xEB, 0xEA, - 0xBF, 0xBE, 0xBB, 0xBA, 0xAF, 0xAE, 0xAB, 0xAA}; - const uint8_t LUTB[16] = {0xFF, 0xFD, 0xF7, 0xF5, 0xDF, 0xDD, 0xD7, 0xD5, - 0x7F, 0x7D, 0x77, 0x75, 0x5F, 0x5D, 0x57, 0x55}; - - const uint8_t pixelMaskLUT[8] = {0x1, 0x2, 0x4, 0x8, 0x10, 0x20, 0x40, 0x80}; - const uint8_t pixelMaskGLUT[2] = {0x0F, 0xF0}; - - const uint8_t waveform3BitAll[6][8][9] = {// INKPLATE_6 - {{0, 1, 1, 0, 0, 1, 1, 0, 0}, - {0, 1, 2, 1, 1, 2, 1, 0, 0}, - {1, 1, 1, 2, 2, 1, 0, 0, 0}, - {0, 0, 0, 1, 1, 1, 2, 0, 0}, - {2, 1, 1, 1, 2, 1, 2, 0, 0}, - {2, 2, 1, 1, 2, 1, 2, 0, 0}, - {1, 1, 1, 2, 1, 2, 2, 0, 0}, - {0, 0, 0, 0, 0, 0, 2, 0, 0}}, - // INKPLATE_10 - {{0, 0, 0, 0, 0, 0, 0, 1, 0}, - {0, 0, 0, 2, 2, 2, 1, 1, 0}, - {0, 0, 2, 1, 1, 2, 2, 1, 0}, - {0, 1, 2, 2, 1, 2, 2, 1, 0}, - {0, 0, 2, 1, 2, 2, 2, 1, 0}, - {0, 2, 2, 2, 2, 2, 2, 1, 0}, - {0, 0, 0, 0, 0, 2, 1, 2, 0}, - {0, 0, 0, 2, 2, 2, 2, 2, 0}}, - // INKPLATE_6_PLUS - {{0, 0, 0, 0, 0, 2, 1, 1, 0}, - {0, 0, 2, 1, 1, 1, 2, 1, 0}, - {0, 2, 2, 2, 1, 1, 2, 1, 0}, - {0, 0, 2, 2, 2, 1, 2, 1, 0}, - {0, 0, 0, 0, 2, 2, 2, 1, 0}, - {0, 0, 2, 1, 2, 1, 1, 2, 0}, - {0, 0, 2, 2, 2, 1, 1, 2, 0}, - {0, 0, 0, 0, 2, 2, 2, 2, 0}}, - // INKPLATE_6_V2 - {{1, 0, 1, 0, 1, 1, 1, 0, 0}, - {0, 0, 0, 1, 1, 1, 1, 0, 0}, - {1, 1, 1, 1, 0, 2, 1, 0, 0}, - {1, 1, 1, 2, 2, 1, 1, 0, 0}, - {1, 1, 1, 1, 2, 2, 1, 0, 0}, - {0, 1, 1, 1, 2, 2, 1, 0, 0}, - {0, 0, 0, 0, 1, 1, 2, 0, 0}, - {0, 0, 0, 0, 0, 1, 2, 0, 0}}, - // INKPLATE_5 - {{0, 0, 1, 1, 0, 1, 1, 1, 0}, - {0, 1, 1, 1, 1, 2, 0, 1, 0}, - {1, 2, 2, 0, 2, 1, 1, 1, 0}, - {1, 1, 1, 2, 0, 1, 1, 2, 0}, - {0, 1, 1, 1, 2, 0, 1, 2, 0}, - {0, 0, 0, 1, 1, 2, 1, 2, 0}, - {1, 1, 1, 2, 0, 2, 1, 2, 0}, - {0, 0, 0, 0, 0, 0, 0, 0, 0}}, - // INKPLATE_5_V2 - {{0, 0, 1, 1, 2, 1, 1, 1, 0}, - {1, 1, 2, 2, 1, 2, 1, 1, 0}, - {0, 1, 2, 2, 1, 1, 2, 1, 0}, - {0, 0, 1, 1, 1, 1, 1, 2, 0}, - {1, 2, 1, 2, 1, 1, 1, 2, 0}, - {0, 1, 1, 1, 2, 0, 1, 2, 0}, - {1, 1, 1, 2, 2, 2, 1, 2, 0}, - {0, 0, 0, 0, 0, 0, 0, 0, 0}}}; - void set_greyscale(bool greyscale) { this->greyscale_ = greyscale; this->block_partial_ = true; if (this->is_ready()) this->initialize_(); } + + void set_waveform(const std::array &waveform, bool is_custom) { + static_assert(sizeof(this->waveform_) == sizeof(uint8_t) * GLUT_COUNT * GLUT_SIZE, + "waveform_ buffer size must match input waveform array size"); + memmove(this->waveform_, waveform.data(), sizeof(this->waveform_)); + this->custom_waveform_ = is_custom; + } + void set_mirror_y(bool mirror_y) { this->mirror_y_ = mirror_y; } void set_mirror_x(bool mirror_x) { this->mirror_x_ = mirror_x; } @@ -225,6 +183,8 @@ class Inkplate6 : public display::DisplayBuffer, public i2c::I2CDevice { bool mirror_y_{false}; bool mirror_x_{false}; bool partial_updating_; + bool custom_waveform_{false}; + uint8_t waveform_[GLUT_COUNT][GLUT_SIZE]; InkplateModel model_; @@ -250,5 +210,5 @@ class Inkplate6 : public display::DisplayBuffer, public i2c::I2CDevice { GPIOPin *wakeup_pin_; }; -} // namespace inkplate6 +} // namespace inkplate } // namespace esphome diff --git a/esphome/components/inkplate6/__init__.py b/esphome/components/inkplate6/__init__.py index b1de57df8f..e69de29bb2 100644 --- a/esphome/components/inkplate6/__init__.py +++ b/esphome/components/inkplate6/__init__.py @@ -1 +0,0 @@ -CODEOWNERS = ["@jesserockz"] diff --git a/esphome/components/inkplate6/display.py b/esphome/components/inkplate6/display.py index 063fc8b0aa..ff14be5491 100644 --- a/esphome/components/inkplate6/display.py +++ b/esphome/components/inkplate6/display.py @@ -1,214 +1,5 @@ -from esphome import pins -import esphome.codegen as cg -from esphome.components import display, i2c -from esphome.components.esp32 import CONF_CPU_FREQUENCY import esphome.config_validation as cv -from esphome.const import ( - CONF_FULL_UPDATE_EVERY, - CONF_ID, - CONF_LAMBDA, - CONF_MIRROR_X, - CONF_MIRROR_Y, - CONF_MODEL, - CONF_OE_PIN, - CONF_PAGES, - CONF_TRANSFORM, - CONF_WAKEUP_PIN, - PLATFORM_ESP32, + +CONFIG_SCHEMA = cv.invalid( + "The inkplate6 display component has been renamed to inkplate." ) -import esphome.final_validate as fv - -DEPENDENCIES = ["i2c", "esp32"] -AUTO_LOAD = ["psram"] - -CONF_DISPLAY_DATA_0_PIN = "display_data_0_pin" -CONF_DISPLAY_DATA_1_PIN = "display_data_1_pin" -CONF_DISPLAY_DATA_2_PIN = "display_data_2_pin" -CONF_DISPLAY_DATA_3_PIN = "display_data_3_pin" -CONF_DISPLAY_DATA_4_PIN = "display_data_4_pin" -CONF_DISPLAY_DATA_5_PIN = "display_data_5_pin" -CONF_DISPLAY_DATA_6_PIN = "display_data_6_pin" -CONF_DISPLAY_DATA_7_PIN = "display_data_7_pin" - -CONF_CL_PIN = "cl_pin" -CONF_CKV_PIN = "ckv_pin" -CONF_GREYSCALE = "greyscale" -CONF_GMOD_PIN = "gmod_pin" -CONF_GPIO0_ENABLE_PIN = "gpio0_enable_pin" -CONF_LE_PIN = "le_pin" -CONF_PARTIAL_UPDATING = "partial_updating" -CONF_POWERUP_PIN = "powerup_pin" -CONF_SPH_PIN = "sph_pin" -CONF_SPV_PIN = "spv_pin" -CONF_VCOM_PIN = "vcom_pin" - -inkplate6_ns = cg.esphome_ns.namespace("inkplate6") -Inkplate6 = inkplate6_ns.class_( - "Inkplate6", - cg.PollingComponent, - i2c.I2CDevice, - display.Display, - display.DisplayBuffer, -) - -InkplateModel = inkplate6_ns.enum("InkplateModel") - -MODELS = { - "inkplate_6": InkplateModel.INKPLATE_6, - "inkplate_10": InkplateModel.INKPLATE_10, - "inkplate_6_plus": InkplateModel.INKPLATE_6_PLUS, - "inkplate_6_v2": InkplateModel.INKPLATE_6_V2, - "inkplate_5": InkplateModel.INKPLATE_5, - "inkplate_5_v2": InkplateModel.INKPLATE_5_V2, -} - -CONFIG_SCHEMA = cv.All( - display.FULL_DISPLAY_SCHEMA.extend( - { - cv.GenerateID(): cv.declare_id(Inkplate6), - cv.Optional(CONF_GREYSCALE, default=False): cv.boolean, - cv.Optional(CONF_TRANSFORM): cv.Schema( - { - cv.Optional(CONF_MIRROR_X, default=False): cv.boolean, - cv.Optional(CONF_MIRROR_Y, default=False): cv.boolean, - } - ), - cv.Optional(CONF_PARTIAL_UPDATING, default=True): cv.boolean, - cv.Optional(CONF_FULL_UPDATE_EVERY, default=10): cv.uint32_t, - cv.Optional(CONF_MODEL, default="inkplate_6"): cv.enum( - MODELS, lower=True, space="_" - ), - # Control pins - cv.Required(CONF_CKV_PIN): pins.gpio_output_pin_schema, - cv.Required(CONF_GMOD_PIN): pins.gpio_output_pin_schema, - cv.Required(CONF_GPIO0_ENABLE_PIN): pins.gpio_output_pin_schema, - cv.Required(CONF_OE_PIN): pins.gpio_output_pin_schema, - cv.Required(CONF_POWERUP_PIN): pins.gpio_output_pin_schema, - cv.Required(CONF_SPH_PIN): pins.gpio_output_pin_schema, - cv.Required(CONF_SPV_PIN): pins.gpio_output_pin_schema, - cv.Required(CONF_VCOM_PIN): pins.gpio_output_pin_schema, - cv.Required(CONF_WAKEUP_PIN): pins.gpio_output_pin_schema, - cv.Optional(CONF_CL_PIN, default=0): pins.internal_gpio_output_pin_schema, - cv.Optional(CONF_LE_PIN, default=2): pins.internal_gpio_output_pin_schema, - # Data pins - cv.Optional( - CONF_DISPLAY_DATA_0_PIN, default=4 - ): pins.internal_gpio_output_pin_schema, - cv.Optional( - CONF_DISPLAY_DATA_1_PIN, default=5 - ): pins.internal_gpio_output_pin_schema, - cv.Optional( - CONF_DISPLAY_DATA_2_PIN, default=18 - ): pins.internal_gpio_output_pin_schema, - cv.Optional( - CONF_DISPLAY_DATA_3_PIN, default=19 - ): pins.internal_gpio_output_pin_schema, - cv.Optional( - CONF_DISPLAY_DATA_4_PIN, default=23 - ): pins.internal_gpio_output_pin_schema, - cv.Optional( - CONF_DISPLAY_DATA_5_PIN, default=25 - ): pins.internal_gpio_output_pin_schema, - cv.Optional( - CONF_DISPLAY_DATA_6_PIN, default=26 - ): pins.internal_gpio_output_pin_schema, - cv.Optional( - CONF_DISPLAY_DATA_7_PIN, default=27 - ): pins.internal_gpio_output_pin_schema, - } - ) - .extend(cv.polling_component_schema("5s")) - .extend(i2c.i2c_device_schema(0x48)), - cv.has_at_most_one_key(CONF_PAGES, CONF_LAMBDA), -) - - -def _validate_cpu_frequency(config): - esp32_config = fv.full_config.get()[PLATFORM_ESP32] - if esp32_config[CONF_CPU_FREQUENCY] != "240MHZ": - raise cv.Invalid( - "Inkplate requires 240MHz CPU frequency (set in esp32 component)" - ) - return config - - -FINAL_VALIDATE_SCHEMA = _validate_cpu_frequency - - -async def to_code(config): - var = cg.new_Pvariable(config[CONF_ID]) - - await display.register_display(var, config) - await i2c.register_i2c_device(var, config) - - if CONF_LAMBDA in config: - lambda_ = await cg.process_lambda( - config[CONF_LAMBDA], [(display.DisplayRef, "it")], return_type=cg.void - ) - cg.add(var.set_writer(lambda_)) - - cg.add(var.set_greyscale(config[CONF_GREYSCALE])) - if transform := config.get(CONF_TRANSFORM): - cg.add(var.set_mirror_x(transform[CONF_MIRROR_X])) - cg.add(var.set_mirror_y(transform[CONF_MIRROR_Y])) - cg.add(var.set_partial_updating(config[CONF_PARTIAL_UPDATING])) - cg.add(var.set_full_update_every(config[CONF_FULL_UPDATE_EVERY])) - - cg.add(var.set_model(config[CONF_MODEL])) - - ckv = await cg.gpio_pin_expression(config[CONF_CKV_PIN]) - cg.add(var.set_ckv_pin(ckv)) - - gmod = await cg.gpio_pin_expression(config[CONF_GMOD_PIN]) - cg.add(var.set_gmod_pin(gmod)) - - gpio0_enable = await cg.gpio_pin_expression(config[CONF_GPIO0_ENABLE_PIN]) - cg.add(var.set_gpio0_enable_pin(gpio0_enable)) - - oe = await cg.gpio_pin_expression(config[CONF_OE_PIN]) - cg.add(var.set_oe_pin(oe)) - - powerup = await cg.gpio_pin_expression(config[CONF_POWERUP_PIN]) - cg.add(var.set_powerup_pin(powerup)) - - sph = await cg.gpio_pin_expression(config[CONF_SPH_PIN]) - cg.add(var.set_sph_pin(sph)) - - spv = await cg.gpio_pin_expression(config[CONF_SPV_PIN]) - cg.add(var.set_spv_pin(spv)) - - vcom = await cg.gpio_pin_expression(config[CONF_VCOM_PIN]) - cg.add(var.set_vcom_pin(vcom)) - - wakeup = await cg.gpio_pin_expression(config[CONF_WAKEUP_PIN]) - cg.add(var.set_wakeup_pin(wakeup)) - - cl = await cg.gpio_pin_expression(config[CONF_CL_PIN]) - cg.add(var.set_cl_pin(cl)) - - le = await cg.gpio_pin_expression(config[CONF_LE_PIN]) - cg.add(var.set_le_pin(le)) - - display_data_0 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_0_PIN]) - cg.add(var.set_display_data_0_pin(display_data_0)) - - display_data_1 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_1_PIN]) - cg.add(var.set_display_data_1_pin(display_data_1)) - - display_data_2 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_2_PIN]) - cg.add(var.set_display_data_2_pin(display_data_2)) - - display_data_3 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_3_PIN]) - cg.add(var.set_display_data_3_pin(display_data_3)) - - display_data_4 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_4_PIN]) - cg.add(var.set_display_data_4_pin(display_data_4)) - - display_data_5 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_5_PIN]) - cg.add(var.set_display_data_5_pin(display_data_5)) - - display_data_6 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_6_PIN]) - cg.add(var.set_display_data_6_pin(display_data_6)) - - display_data_7 = await cg.gpio_pin_expression(config[CONF_DISPLAY_DATA_7_PIN]) - cg.add(var.set_display_data_7_pin(display_data_7)) diff --git a/esphome/components/integration/integration_sensor.cpp b/esphome/components/integration/integration_sensor.cpp index c09778e79e..80c718dc8d 100644 --- a/esphome/components/integration/integration_sensor.cpp +++ b/esphome/components/integration/integration_sensor.cpp @@ -10,7 +10,7 @@ static const char *const TAG = "integration"; void IntegrationSensor::setup() { if (this->restore_) { - this->pref_ = global_preferences->make_preference(this->get_object_id_hash()); + this->pref_ = global_preferences->make_preference(this->get_preference_hash()); float preference_value = 0; this->pref_.load(&preference_value); this->result_ = preference_value; diff --git a/esphome/components/internal_temperature/internal_temperature.cpp b/esphome/components/internal_temperature/internal_temperature.cpp index 85844647f2..28ac55d6de 100644 --- a/esphome/components/internal_temperature/internal_temperature.cpp +++ b/esphome/components/internal_temperature/internal_temperature.cpp @@ -84,8 +84,6 @@ void InternalTemperatureSensor::setup() { #if defined(USE_ESP32_VARIANT_ESP32C3) || defined(USE_ESP32_VARIANT_ESP32C6) || defined(USE_ESP32_VARIANT_ESP32S2) || \ defined(USE_ESP32_VARIANT_ESP32S3) || defined(USE_ESP32_VARIANT_ESP32H2) || defined(USE_ESP32_VARIANT_ESP32C2) || \ defined(USE_ESP32_VARIANT_ESP32P4) - ESP_LOGCONFIG(TAG, "Running setup"); - temperature_sensor_config_t tsens_config = TEMPERATURE_SENSOR_CONFIG_DEFAULT(-10, 80); esp_err_t result = temperature_sensor_install(&tsens_config, &tsensNew); diff --git a/esphome/components/interval/interval.h b/esphome/components/interval/interval.h index 8f904b104d..e419841e6c 100644 --- a/esphome/components/interval/interval.h +++ b/esphome/components/interval/interval.h @@ -1,6 +1,7 @@ #pragma once #include "esphome/core/component.h" +#include "esphome/core/log.h" #include "esphome/core/automation.h" namespace esphome { @@ -8,16 +9,12 @@ namespace interval { class IntervalTrigger : public Trigger<>, public PollingComponent { public: - void update() override { - if (this->started_) - this->trigger(); - } + void update() override { this->trigger(); } void setup() override { - if (this->startup_delay_ == 0) { - this->started_ = true; - } else { - this->set_timeout(this->startup_delay_, [this] { this->started_ = true; }); + if (this->startup_delay_ != 0) { + this->stop_poller(); + this->set_timeout(this->startup_delay_, [this] { this->start_poller(); }); } } @@ -25,7 +22,6 @@ class IntervalTrigger : public Trigger<>, public PollingComponent { protected: uint32_t startup_delay_{0}; - bool started_{false}; }; } // namespace interval diff --git a/esphome/components/json/__init__.py b/esphome/components/json/__init__.py index 9773bf67ce..4cd737c60d 100644 --- a/esphome/components/json/__init__.py +++ b/esphome/components/json/__init__.py @@ -1,8 +1,8 @@ import esphome.codegen as cg import esphome.config_validation as cv -from esphome.core import coroutine_with_priority +from esphome.core import CoroPriority, coroutine_with_priority -CODEOWNERS = ["@OttoWinter"] +CODEOWNERS = ["@esphome/core"] json_ns = cg.esphome_ns.namespace("json") CONFIG_SCHEMA = cv.All( @@ -10,7 +10,7 @@ CONFIG_SCHEMA = cv.All( ) -@coroutine_with_priority(1.0) +@coroutine_with_priority(CoroPriority.BUS) async def to_code(config): cg.add_library("bblanchon/ArduinoJson", "7.4.2") cg.add_define("USE_JSON") diff --git a/esphome/components/json/json_util.cpp b/esphome/components/json/json_util.cpp index 94c531222a..869d29f92e 100644 --- a/esphome/components/json/json_util.cpp +++ b/esphome/components/json/json_util.cpp @@ -8,70 +8,68 @@ namespace json { static const char *const TAG = "json"; -// Build an allocator for the JSON Library using the RAMAllocator class -struct SpiRamAllocator : ArduinoJson::Allocator { - void *allocate(size_t size) override { return this->allocator_.allocate(size); } - - void deallocate(void *pointer) override { - // ArduinoJson's Allocator interface doesn't provide the size parameter in deallocate. - // RAMAllocator::deallocate() requires the size, which we don't have access to here. - // RAMAllocator::deallocate implementation just calls free() regardless of whether - // the memory was allocated with heap_caps_malloc or malloc. - // This is safe because ESP-IDF's heap implementation internally tracks the memory region - // and routes free() to the appropriate heap. - free(pointer); // NOLINT(cppcoreguidelines-owning-memory,cppcoreguidelines-no-malloc) - } - - void *reallocate(void *ptr, size_t new_size) override { - return this->allocator_.reallocate(static_cast(ptr), new_size); - } - - protected: - RAMAllocator allocator_{RAMAllocator(RAMAllocator::NONE)}; -}; +#ifdef USE_PSRAM +// Global allocator that outlives all JsonDocuments returned by parse_json() +// This prevents dangling pointer issues when JsonDocuments are returned from functions +// NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables) - Must be mutable for ArduinoJson::Allocator +static SpiRamAllocator global_json_allocator; +#endif std::string build_json(const json_build_t &f) { // NOLINTBEGIN(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson - auto doc_allocator = SpiRamAllocator(); - JsonDocument json_document(&doc_allocator); - if (json_document.overflowed()) { - ESP_LOGE(TAG, "Could not allocate memory for JSON document!"); - return "{}"; - } - JsonObject root = json_document.to(); + JsonBuilder builder; + JsonObject root = builder.root(); f(root); - if (json_document.overflowed()) { - ESP_LOGE(TAG, "Could not allocate memory for JSON document!"); - return "{}"; - } - std::string output; - serializeJson(json_document, output); - return output; + return builder.serialize(); // NOLINTEND(clang-analyzer-cplusplus.NewDeleteLeaks) } bool parse_json(const std::string &data, const json_parse_t &f) { // NOLINTBEGIN(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson - auto doc_allocator = SpiRamAllocator(); - JsonDocument json_document(&doc_allocator); + JsonDocument doc = parse_json(reinterpret_cast(data.c_str()), data.size()); + if (doc.overflowed() || doc.isNull()) + return false; + return f(doc.as()); + // NOLINTEND(clang-analyzer-cplusplus.NewDeleteLeaks) +} + +JsonDocument parse_json(const uint8_t *data, size_t len) { + // NOLINTBEGIN(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson + if (data == nullptr || len == 0) { + ESP_LOGE(TAG, "No data to parse"); + return JsonObject(); // return unbound object + } +#ifdef USE_PSRAM + JsonDocument json_document(&global_json_allocator); +#else + JsonDocument json_document; +#endif if (json_document.overflowed()) { ESP_LOGE(TAG, "Could not allocate memory for JSON document!"); - return false; + return JsonObject(); // return unbound object } - DeserializationError err = deserializeJson(json_document, data); - - JsonObject root = json_document.as(); + DeserializationError err = deserializeJson(json_document, data, len); if (err == DeserializationError::Ok) { - return f(root); + return json_document; } else if (err == DeserializationError::NoMemory) { ESP_LOGE(TAG, "Can not allocate more memory for deserialization. Consider making source string smaller"); - return false; + return JsonObject(); // return unbound object } ESP_LOGE(TAG, "Parse error: %s", err.c_str()); - return false; + return JsonObject(); // return unbound object // NOLINTEND(clang-analyzer-cplusplus.NewDeleteLeaks) } +std::string JsonBuilder::serialize() { + if (doc_.overflowed()) { + ESP_LOGE(TAG, "JSON document overflow"); + return "{}"; + } + std::string output; + serializeJson(doc_, output); + return output; +} + } // namespace json } // namespace esphome diff --git a/esphome/components/json/json_util.h b/esphome/components/json/json_util.h index 72d31c8afe..91cc84dc14 100644 --- a/esphome/components/json/json_util.h +++ b/esphome/components/json/json_util.h @@ -2,6 +2,7 @@ #include +#include "esphome/core/defines.h" #include "esphome/core/helpers.h" #define ARDUINOJSON_ENABLE_STD_STRING 1 // NOLINT @@ -13,6 +14,31 @@ namespace esphome { namespace json { +#ifdef USE_PSRAM +// Build an allocator for the JSON Library using the RAMAllocator class +// This is only compiled when PSRAM is enabled +struct SpiRamAllocator : ArduinoJson::Allocator { + void *allocate(size_t size) override { return allocator_.allocate(size); } + + void deallocate(void *ptr) override { + // ArduinoJson's Allocator interface doesn't provide the size parameter in deallocate. + // RAMAllocator::deallocate() requires the size, which we don't have access to here. + // RAMAllocator::deallocate implementation just calls free() regardless of whether + // the memory was allocated with heap_caps_malloc or malloc. + // This is safe because ESP-IDF's heap implementation internally tracks the memory region + // and routes free() to the appropriate heap. + free(ptr); // NOLINT(cppcoreguidelines-owning-memory,cppcoreguidelines-no-malloc) + } + + void *reallocate(void *ptr, size_t new_size) override { + return allocator_.reallocate(static_cast(ptr), new_size); + } + + protected: + RAMAllocator allocator_{RAMAllocator::NONE}; +}; +#endif + /// Callback function typedef for parsing JsonObjects. using json_parse_t = std::function; @@ -25,5 +51,36 @@ std::string build_json(const json_build_t &f); /// Parse a JSON string and run the provided json parse function if it's valid. bool parse_json(const std::string &data, const json_parse_t &f); +/// Parse a JSON string and return the root JsonDocument (or an unbound object on error) +JsonDocument parse_json(const uint8_t *data, size_t len); +/// Parse a JSON string and return the root JsonDocument (or an unbound object on error) +inline JsonDocument parse_json(const std::string &data) { + return parse_json(reinterpret_cast(data.c_str()), data.size()); +} + +/// Builder class for creating JSON documents without lambdas +class JsonBuilder { + public: + JsonObject root() { + if (!root_created_) { + root_ = doc_.to(); + root_created_ = true; + } + return root_; + } + + std::string serialize(); + + private: +#ifdef USE_PSRAM + SpiRamAllocator allocator_; + JsonDocument doc_{&allocator_}; +#else + JsonDocument doc_; +#endif + JsonObject root_; + bool root_created_{false}; +}; + } // namespace json } // namespace esphome diff --git a/esphome/components/kamstrup_kmp/kamstrup_kmp.cpp b/esphome/components/kamstrup_kmp/kamstrup_kmp.cpp index c058c7b3aa..e5fa035682 100644 --- a/esphome/components/kamstrup_kmp/kamstrup_kmp.cpp +++ b/esphome/components/kamstrup_kmp/kamstrup_kmp.cpp @@ -22,7 +22,7 @@ void KamstrupKMPComponent::dump_config() { LOG_SENSOR(" ", "Flow", this->flow_sensor_); LOG_SENSOR(" ", "Volume", this->volume_sensor_); - for (int i = 0; i < this->custom_sensors_.size(); i++) { + for (size_t i = 0; i < this->custom_sensors_.size(); i++) { LOG_SENSOR(" ", "Custom Sensor", this->custom_sensors_[i]); ESP_LOGCONFIG(TAG, " Command: 0x%04X", this->custom_commands_[i]); } @@ -268,7 +268,7 @@ void KamstrupKMPComponent::set_sensor_value_(uint16_t command, float value, uint } // Custom sensors - for (int i = 0; i < this->custom_commands_.size(); i++) { + for (size_t i = 0; i < this->custom_commands_.size(); i++) { if (command == this->custom_commands_[i]) { this->custom_sensors_[i]->publish_state(value); } diff --git a/esphome/components/key_collector/key_collector.h b/esphome/components/key_collector/key_collector.h index 6e585ddd8e..35e8141ce5 100644 --- a/esphome/components/key_collector/key_collector.h +++ b/esphome/components/key_collector/key_collector.h @@ -13,8 +13,8 @@ class KeyCollector : public Component { void loop() override; void dump_config() override; void set_provider(key_provider::KeyProvider *provider); - void set_min_length(int min_length) { this->min_length_ = min_length; }; - void set_max_length(int max_length) { this->max_length_ = max_length; }; + void set_min_length(uint32_t min_length) { this->min_length_ = min_length; }; + void set_max_length(uint32_t max_length) { this->max_length_ = max_length; }; void set_start_keys(std::string start_keys) { this->start_keys_ = std::move(start_keys); }; void set_end_keys(std::string end_keys) { this->end_keys_ = std::move(end_keys); }; void set_end_key_required(bool end_key_required) { this->end_key_required_ = end_key_required; }; @@ -33,8 +33,8 @@ class KeyCollector : public Component { protected: void key_pressed_(uint8_t key); - int min_length_{0}; - int max_length_{0}; + uint32_t min_length_{0}; + uint32_t max_length_{0}; std::string start_keys_; std::string end_keys_; bool end_key_required_{false}; diff --git a/esphome/components/kmeteriso/kmeteriso.cpp b/esphome/components/kmeteriso/kmeteriso.cpp index 714df0b538..36f6d74ba0 100644 --- a/esphome/components/kmeteriso/kmeteriso.cpp +++ b/esphome/components/kmeteriso/kmeteriso.cpp @@ -14,7 +14,6 @@ static const uint8_t KMETER_INTERNAL_TEMP_VAL_REG = 0x10; static const uint8_t KMETER_FIRMWARE_VERSION_REG = 0xFE; void KMeterISOComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->error_code_ = NONE; // Mark as not failed before initializing. Some devices will turn off sensors to save on batteries @@ -23,7 +22,7 @@ void KMeterISOComponent::setup() { this->reset_to_construction_state(); } - auto err = this->bus_->writev(this->address_, nullptr, 0); + auto err = this->bus_->write_readv(this->address_, nullptr, 0, nullptr, 0); if (err == esphome::i2c::ERROR_OK) { ESP_LOGCONFIG(TAG, "Could write to the address %d.", this->address_); } else { @@ -46,7 +45,6 @@ void KMeterISOComponent::setup() { this->mark_failed(); return; } - ESP_LOGCONFIG(TAG, "The device was successfully setup."); } float KMeterISOComponent::get_setup_priority() const { return setup_priority::DATA; } diff --git a/esphome/components/lc709203f/lc709203f.cpp b/esphome/components/lc709203f/lc709203f.cpp index d95a2c1d5e..7e6ac878f8 100644 --- a/esphome/components/lc709203f/lc709203f.cpp +++ b/esphome/components/lc709203f/lc709203f.cpp @@ -1,5 +1,6 @@ -#include "esphome/core/log.h" #include "lc709203f.h" +#include "esphome/core/helpers.h" +#include "esphome/core/log.h" namespace esphome { namespace lc709203f { @@ -49,8 +50,6 @@ void Lc709203f::setup() { // initialization code checks the return code from those functions. If they don't return // NO_ERROR (0x00), that part of the initialization aborts and will be retried on the next // call to update(). - ESP_LOGCONFIG(TAG, "Running setup"); - // Set power mode to on. Note that, unlike some other similar devices, in sleep mode the IC // does not record power usage. If there is significant power consumption during sleep mode, // the pack RSOC will likely no longer be correct. Because of that, I do not implement @@ -186,12 +185,12 @@ uint8_t Lc709203f::get_register_(uint8_t register_to_read, uint16_t *register_va // function will send a stop between the read and the write portion of the I2C // transaction. This is bad in this case and will result in reading nothing but 0xFFFF // from the registers. - return_code = this->read_register(register_to_read, &read_buffer[3], 3, false); + return_code = this->read_register(register_to_read, &read_buffer[3], 3); if (return_code != i2c::NO_ERROR) { // Error on the i2c bus this->status_set_warning( str_sprintf("Error code %d when reading from register 0x%02X", return_code, register_to_read).c_str()); - } else if (this->crc8_(read_buffer, 5) != read_buffer[5]) { + } else if (crc8(read_buffer, 5, 0x00, 0x07, true) != read_buffer[5]) { // I2C indicated OK, but the CRC of the data does not matcth. this->status_set_warning(str_sprintf("CRC error reading from register 0x%02X", register_to_read).c_str()); } else { @@ -222,12 +221,12 @@ uint8_t Lc709203f::set_register_(uint8_t register_to_set, uint16_t value_to_set) write_buffer[1] = register_to_set; write_buffer[2] = value_to_set & 0xFF; // Low byte write_buffer[3] = (value_to_set >> 8) & 0xFF; // High byte - write_buffer[4] = this->crc8_(write_buffer, 4); + write_buffer[4] = crc8(write_buffer, 4, 0x00, 0x07, true); for (uint8_t i = 0; i <= LC709203F_I2C_RETRY_COUNT; i++) { // Note: we don't write the first byte of the write buffer to the device. // This is done automatically by the write() function. - return_code = this->write(&write_buffer[1], 4, true); + return_code = this->write(&write_buffer[1], 4); if (return_code == i2c::NO_ERROR) { return return_code; } else { @@ -241,20 +240,6 @@ uint8_t Lc709203f::set_register_(uint8_t register_to_set, uint16_t value_to_set) return return_code; } -uint8_t Lc709203f::crc8_(uint8_t *byte_buffer, uint8_t length_of_crc) { - uint8_t crc = 0x00; - const uint8_t polynomial(0x07); - - for (uint8_t j = length_of_crc; j; --j) { - crc ^= *byte_buffer++; - - for (uint8_t i = 8; i; --i) { - crc = (crc & 0x80) ? (crc << 1) ^ polynomial : (crc << 1); - } - } - return crc; -} - void Lc709203f::set_pack_size(uint16_t pack_size) { static const uint16_t PACK_SIZE_ARRAY[6] = {100, 200, 500, 1000, 2000, 3000}; static const uint16_t APA_ARRAY[6] = {0x08, 0x0B, 0x10, 0x19, 0x2D, 0x36}; diff --git a/esphome/components/lc709203f/lc709203f.h b/esphome/components/lc709203f/lc709203f.h index 3b5b04775f..59988a0079 100644 --- a/esphome/components/lc709203f/lc709203f.h +++ b/esphome/components/lc709203f/lc709203f.h @@ -1,8 +1,8 @@ #pragma once -#include "esphome/core/component.h" -#include "esphome/components/sensor/sensor.h" #include "esphome/components/i2c/i2c.h" +#include "esphome/components/sensor/sensor.h" +#include "esphome/core/component.h" namespace esphome { namespace lc709203f { @@ -38,7 +38,6 @@ class Lc709203f : public sensor::Sensor, public PollingComponent, public i2c::I2 private: uint8_t get_register_(uint8_t register_to_read, uint16_t *register_value); uint8_t set_register_(uint8_t register_to_set, uint16_t value_to_set); - uint8_t crc8_(uint8_t *byte_buffer, uint8_t length_of_crc); protected: sensor::Sensor *voltage_sensor_{nullptr}; diff --git a/esphome/components/lcd_gpio/display.py b/esphome/components/lcd_gpio/display.py index caa73194c9..0a77daf336 100644 --- a/esphome/components/lcd_gpio/display.py +++ b/esphome/components/lcd_gpio/display.py @@ -41,9 +41,7 @@ CONFIG_SCHEMA = lcd_base.LCD_SCHEMA.extend( async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) await lcd_base.setup_lcd_display(var, config) - pins_ = [] - for conf in config[CONF_DATA_PINS]: - pins_.append(await cg.gpio_pin_expression(conf)) + pins_ = [await cg.gpio_pin_expression(conf) for conf in config[CONF_DATA_PINS]] cg.add(var.set_data_pins(*pins_)) enable = await cg.gpio_pin_expression(config[CONF_ENABLE_PIN]) cg.add(var.set_enable_pin(enable)) diff --git a/esphome/components/lcd_gpio/gpio_lcd_display.cpp b/esphome/components/lcd_gpio/gpio_lcd_display.cpp index afa74643fb..ae6e1194b8 100644 --- a/esphome/components/lcd_gpio/gpio_lcd_display.cpp +++ b/esphome/components/lcd_gpio/gpio_lcd_display.cpp @@ -7,7 +7,6 @@ namespace lcd_gpio { static const char *const TAG = "lcd_gpio"; void GPIOLCDDisplay::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->rs_pin_->setup(); // OUTPUT this->rs_pin_->digital_write(false); if (this->rw_pin_ != nullptr) { diff --git a/esphome/components/lcd_pcf8574/pcf8574_display.cpp b/esphome/components/lcd_pcf8574/pcf8574_display.cpp index 0f06548b13..d582eead91 100644 --- a/esphome/components/lcd_pcf8574/pcf8574_display.cpp +++ b/esphome/components/lcd_pcf8574/pcf8574_display.cpp @@ -11,7 +11,6 @@ static const uint8_t LCD_DISPLAY_BACKLIGHT_ON = 0x08; static const uint8_t LCD_DISPLAY_BACKLIGHT_OFF = 0x00; void PCF8574LCDDisplay::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->backlight_value_ = LCD_DISPLAY_BACKLIGHT_ON; if (!this->write_bytes(this->backlight_value_, nullptr, 0)) { this->mark_failed(); diff --git a/esphome/components/ld2410/__init__.py b/esphome/components/ld2410/__init__.py index c58b9a4017..b492bbcd14 100644 --- a/esphome/components/ld2410/__init__.py +++ b/esphome/components/ld2410/__init__.py @@ -5,6 +5,7 @@ from esphome.components import uart import esphome.config_validation as cv from esphome.const import CONF_ID, CONF_PASSWORD, CONF_THROTTLE, CONF_TIMEOUT +AUTO_LOAD = ["ld24xx"] DEPENDENCIES = ["uart"] CODEOWNERS = ["@sebcaps", "@regevbr"] MULTI_CONF = True @@ -13,18 +14,16 @@ ld2410_ns = cg.esphome_ns.namespace("ld2410") LD2410Component = ld2410_ns.class_("LD2410Component", cg.Component, uart.UARTDevice) CONF_LD2410_ID = "ld2410_id" - CONF_MAX_MOVE_DISTANCE = "max_move_distance" CONF_MAX_STILL_DISTANCE = "max_still_distance" -CONF_STILL_THRESHOLDS = [f"g{x}_still_threshold" for x in range(9)] CONF_MOVE_THRESHOLDS = [f"g{x}_move_threshold" for x in range(9)] +CONF_STILL_THRESHOLDS = [f"g{x}_still_threshold" for x in range(9)] CONFIG_SCHEMA = cv.Schema( { cv.GenerateID(): cv.declare_id(LD2410Component), - cv.Optional(CONF_THROTTLE, default="1000ms"): cv.All( - cv.positive_time_period_milliseconds, - cv.Range(min=cv.TimePeriod(milliseconds=1)), + cv.Optional(CONF_THROTTLE): cv.invalid( + f"{CONF_THROTTLE} has been removed; use per-sensor filters, instead" ), cv.Optional(CONF_MAX_MOVE_DISTANCE): cv.invalid( f"The '{CONF_MAX_MOVE_DISTANCE}' option has been moved to the '{CONF_MAX_MOVE_DISTANCE}'" @@ -74,7 +73,6 @@ async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) await uart.register_uart_device(var, config) - cg.add(var.set_throttle(config[CONF_THROTTLE])) CALIBRATION_ACTION_SCHEMA = maybe_simple_id( diff --git a/esphome/components/ld2410/binary_sensor.py b/esphome/components/ld2410/binary_sensor.py index d2938754e9..4e35f67fbe 100644 --- a/esphome/components/ld2410/binary_sensor.py +++ b/esphome/components/ld2410/binary_sensor.py @@ -22,19 +22,23 @@ CONFIG_SCHEMA = { cv.GenerateID(CONF_LD2410_ID): cv.use_id(LD2410Component), cv.Optional(CONF_HAS_TARGET): binary_sensor.binary_sensor_schema( device_class=DEVICE_CLASS_OCCUPANCY, + filters=[{"settle": cv.TimePeriod(milliseconds=1000)}], icon=ICON_ACCOUNT, ), cv.Optional(CONF_HAS_MOVING_TARGET): binary_sensor.binary_sensor_schema( device_class=DEVICE_CLASS_MOTION, + filters=[{"settle": cv.TimePeriod(milliseconds=1000)}], icon=ICON_MOTION_SENSOR, ), cv.Optional(CONF_HAS_STILL_TARGET): binary_sensor.binary_sensor_schema( device_class=DEVICE_CLASS_OCCUPANCY, + filters=[{"settle": cv.TimePeriod(milliseconds=1000)}], icon=ICON_MOTION_SENSOR, ), cv.Optional(CONF_OUT_PIN_PRESENCE_STATUS): binary_sensor.binary_sensor_schema( device_class=DEVICE_CLASS_PRESENCE, entity_category=ENTITY_CATEGORY_DIAGNOSTIC, + filters=[{"settle": cv.TimePeriod(milliseconds=1000)}], icon=ICON_ACCOUNT, ), } diff --git a/esphome/components/ld2410/ld2410.cpp b/esphome/components/ld2410/ld2410.cpp index 8f1fb0ee9d..5c3af54ad8 100644 --- a/esphome/components/ld2410/ld2410.cpp +++ b/esphome/components/ld2410/ld2410.cpp @@ -1,6 +1,5 @@ #include "ld2410.h" -#include #ifdef USE_NUMBER #include "esphome/components/number/number.h" #endif @@ -10,10 +9,6 @@ #include "esphome/core/application.h" -#define CHECK_BIT(var, pos) (((var) >> (pos)) & 1) -#define highbyte(val) (uint8_t)((val) >> 8) -#define lowbyte(val) (uint8_t)((val) &0xff) - namespace esphome { namespace ld2410 { @@ -165,6 +160,9 @@ static constexpr uint8_t CMD_BLUETOOTH = 0xA4; static constexpr uint8_t CMD_MAX_MOVE_VALUE = 0x00; static constexpr uint8_t CMD_MAX_STILL_VALUE = 0x01; static constexpr uint8_t CMD_DURATION_VALUE = 0x02; +// Bitmasks for target states +static constexpr uint8_t MOVE_BITMASK = 0x01; +static constexpr uint8_t STILL_BITMASK = 0x02; // Header & Footer size static constexpr uint8_t HEADER_FOOTER_SIZE = 4; // Command Header & Footer @@ -190,9 +188,8 @@ void LD2410Component::dump_config() { ESP_LOGCONFIG(TAG, "LD2410:\n" " Firmware version: %s\n" - " MAC address: %s\n" - " Throttle: %u ms", - version.c_str(), mac_str.c_str(), this->throttle_); + " MAC address: %s", + version.c_str(), mac_str.c_str()); #ifdef USE_BINARY_SENSOR ESP_LOGCONFIG(TAG, "Binary Sensors:"); LOG_BINARY_SENSOR(" ", "Target", this->target_binary_sensor_); @@ -202,17 +199,17 @@ void LD2410Component::dump_config() { #endif #ifdef USE_SENSOR ESP_LOGCONFIG(TAG, "Sensors:"); - LOG_SENSOR(" ", "Light", this->light_sensor_); - LOG_SENSOR(" ", "DetectionDistance", this->detection_distance_sensor_); - LOG_SENSOR(" ", "MovingTargetDistance", this->moving_target_distance_sensor_); - LOG_SENSOR(" ", "MovingTargetEnergy", this->moving_target_energy_sensor_); - LOG_SENSOR(" ", "StillTargetDistance", this->still_target_distance_sensor_); - LOG_SENSOR(" ", "StillTargetEnergy", this->still_target_energy_sensor_); - for (sensor::Sensor *s : this->gate_move_sensors_) { - LOG_SENSOR(" ", "GateMove", s); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "Light", this->light_sensor_); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "DetectionDistance", this->detection_distance_sensor_); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "MovingTargetDistance", this->moving_target_distance_sensor_); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "MovingTargetEnergy", this->moving_target_energy_sensor_); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "StillTargetDistance", this->still_target_distance_sensor_); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "StillTargetEnergy", this->still_target_energy_sensor_); + for (auto &s : this->gate_move_sensors_) { + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "GateMove", s); } - for (sensor::Sensor *s : this->gate_still_sensors_) { - LOG_SENSOR(" ", "GateStill", s); + for (auto &s : this->gate_still_sensors_) { + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "GateStill", s); } #endif #ifdef USE_TEXT_SENSOR @@ -253,10 +250,7 @@ void LD2410Component::dump_config() { #endif } -void LD2410Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->read_all_info(); -} +void LD2410Component::setup() { this->read_all_info(); } void LD2410Component::read_all_info() { this->set_config_mode_(true); @@ -304,16 +298,13 @@ void LD2410Component::send_command_(uint8_t command, const uint8_t *command_valu } // frame footer bytes this->write_array(CMD_FRAME_FOOTER, sizeof(CMD_FRAME_FOOTER)); - // FIXME to remove - delay(50); // NOLINT + + if (command != CMD_ENABLE_CONF && command != CMD_DISABLE_CONF) { + delay(50); // NOLINT + } } void LD2410Component::handle_periodic_data_() { - // Reduce data update rate to reduce home assistant database growth - // Check this first to prevent unnecessary processing done in later checks/parsing - if (App.get_loop_component_start_time() - this->last_periodic_millis_ < this->throttle_) { - return; - } // 4 frame header bytes + 2 length bytes + 1 data end byte + 1 crc byte + 4 frame footer bytes // data header=0xAA, data footer=0x55, crc=0x00 if (this->buffer_pos_ < 12 || !ld2410::validate_header_footer(DATA_FRAME_HEADER, this->buffer_data_) || @@ -321,9 +312,6 @@ void LD2410Component::handle_periodic_data_() { this->buffer_data_[this->buffer_pos_ - 5] != CHECK) { return; } - // Save the timestamp after validating the frame so, if invalid, we'll take the next frame immediately - this->last_periodic_millis_ = App.get_loop_component_start_time(); - /* Data Type: 7th 0x01: Engineering mode @@ -348,10 +336,10 @@ void LD2410Component::handle_periodic_data_() { this->target_binary_sensor_->publish_state(target_state != 0x00); } if (this->moving_target_binary_sensor_ != nullptr) { - this->moving_target_binary_sensor_->publish_state(CHECK_BIT(target_state, 0)); + this->moving_target_binary_sensor_->publish_state(target_state & MOVE_BITMASK); } if (this->still_target_binary_sensor_ != nullptr) { - this->still_target_binary_sensor_->publish_state(CHECK_BIT(target_state, 1)); + this->still_target_binary_sensor_->publish_state(target_state & STILL_BITMASK); } #endif /* @@ -362,89 +350,51 @@ void LD2410Component::handle_periodic_data_() { Detect distance: 16~17th bytes */ #ifdef USE_SENSOR - if (this->moving_target_distance_sensor_ != nullptr) { - int new_moving_target_distance = - ld2410::two_byte_to_int(this->buffer_data_[MOVING_TARGET_LOW], this->buffer_data_[MOVING_TARGET_HIGH]); - if (this->moving_target_distance_sensor_->get_state() != new_moving_target_distance) - this->moving_target_distance_sensor_->publish_state(new_moving_target_distance); - } - if (this->moving_target_energy_sensor_ != nullptr) { - int new_moving_target_energy = this->buffer_data_[MOVING_ENERGY]; - if (this->moving_target_energy_sensor_->get_state() != new_moving_target_energy) - this->moving_target_energy_sensor_->publish_state(new_moving_target_energy); - } - if (this->still_target_distance_sensor_ != nullptr) { - int new_still_target_distance = - ld2410::two_byte_to_int(this->buffer_data_[STILL_TARGET_LOW], this->buffer_data_[STILL_TARGET_HIGH]); - if (this->still_target_distance_sensor_->get_state() != new_still_target_distance) - this->still_target_distance_sensor_->publish_state(new_still_target_distance); - } - if (this->still_target_energy_sensor_ != nullptr) { - int new_still_target_energy = this->buffer_data_[STILL_ENERGY]; - if (this->still_target_energy_sensor_->get_state() != new_still_target_energy) - this->still_target_energy_sensor_->publish_state(new_still_target_energy); - } - if (this->detection_distance_sensor_ != nullptr) { - int new_detect_distance = - ld2410::two_byte_to_int(this->buffer_data_[DETECT_DISTANCE_LOW], this->buffer_data_[DETECT_DISTANCE_HIGH]); - if (this->detection_distance_sensor_->get_state() != new_detect_distance) - this->detection_distance_sensor_->publish_state(new_detect_distance); - } + SAFE_PUBLISH_SENSOR( + this->moving_target_distance_sensor_, + ld2410::two_byte_to_int(this->buffer_data_[MOVING_TARGET_LOW], this->buffer_data_[MOVING_TARGET_HIGH])) + SAFE_PUBLISH_SENSOR(this->moving_target_energy_sensor_, this->buffer_data_[MOVING_ENERGY]) + SAFE_PUBLISH_SENSOR( + this->still_target_distance_sensor_, + ld2410::two_byte_to_int(this->buffer_data_[STILL_TARGET_LOW], this->buffer_data_[STILL_TARGET_HIGH])); + SAFE_PUBLISH_SENSOR(this->still_target_energy_sensor_, this->buffer_data_[STILL_ENERGY]); + SAFE_PUBLISH_SENSOR( + this->detection_distance_sensor_, + ld2410::two_byte_to_int(this->buffer_data_[DETECT_DISTANCE_LOW], this->buffer_data_[DETECT_DISTANCE_HIGH])); + if (engineering_mode) { /* Moving distance range: 18th byte Still distance range: 19th byte Moving energy: 20~28th bytes */ - for (std::vector::size_type i = 0; i != this->gate_move_sensors_.size(); i++) { - sensor::Sensor *s = this->gate_move_sensors_[i]; - if (s != nullptr) { - s->publish_state(this->buffer_data_[MOVING_SENSOR_START + i]); - } + for (uint8_t i = 0; i < TOTAL_GATES; i++) { + SAFE_PUBLISH_SENSOR(this->gate_move_sensors_[i], this->buffer_data_[MOVING_SENSOR_START + i]) } /* Still energy: 29~37th bytes */ - for (std::vector::size_type i = 0; i != this->gate_still_sensors_.size(); i++) { - sensor::Sensor *s = this->gate_still_sensors_[i]; - if (s != nullptr) { - s->publish_state(this->buffer_data_[STILL_SENSOR_START + i]); - } + for (uint8_t i = 0; i < TOTAL_GATES; i++) { + SAFE_PUBLISH_SENSOR(this->gate_still_sensors_[i], this->buffer_data_[STILL_SENSOR_START + i]) } /* Light sensor: 38th bytes */ - if (this->light_sensor_ != nullptr) { - int new_light_sensor = this->buffer_data_[LIGHT_SENSOR]; - if (this->light_sensor_->get_state() != new_light_sensor) { - this->light_sensor_->publish_state(new_light_sensor); - } - } + SAFE_PUBLISH_SENSOR(this->light_sensor_, this->buffer_data_[LIGHT_SENSOR]) } else { - for (auto *s : this->gate_move_sensors_) { - if (s != nullptr && !std::isnan(s->get_state())) { - s->publish_state(NAN); - } + for (auto &gate_move_sensor : this->gate_move_sensors_) { + SAFE_PUBLISH_SENSOR_UNKNOWN(gate_move_sensor) } - for (auto *s : this->gate_still_sensors_) { - if (s != nullptr && !std::isnan(s->get_state())) { - s->publish_state(NAN); - } - } - if (this->light_sensor_ != nullptr && !std::isnan(this->light_sensor_->get_state())) { - this->light_sensor_->publish_state(NAN); + for (auto &gate_still_sensor : this->gate_still_sensors_) { + SAFE_PUBLISH_SENSOR_UNKNOWN(gate_still_sensor) } + SAFE_PUBLISH_SENSOR_UNKNOWN(this->light_sensor_) } #endif #ifdef USE_BINARY_SENSOR - if (engineering_mode) { - if (this->out_pin_presence_status_binary_sensor_ != nullptr) { - this->out_pin_presence_status_binary_sensor_->publish_state(this->buffer_data_[OUT_PIN_SENSOR] == 0x01); - } - } else { - if (this->out_pin_presence_status_binary_sensor_ != nullptr) { - this->out_pin_presence_status_binary_sensor_->publish_state(false); - } + if (this->out_pin_presence_status_binary_sensor_ != nullptr) { + this->out_pin_presence_status_binary_sensor_->publish_state( + engineering_mode ? this->buffer_data_[OUT_PIN_SENSOR] == 0x01 : false); } #endif } @@ -824,8 +774,14 @@ void LD2410Component::set_light_out_control() { } #ifdef USE_SENSOR -void LD2410Component::set_gate_move_sensor(uint8_t gate, sensor::Sensor *s) { this->gate_move_sensors_[gate] = s; } -void LD2410Component::set_gate_still_sensor(uint8_t gate, sensor::Sensor *s) { this->gate_still_sensors_[gate] = s; } +// These could leak memory, but they are only set once prior to 'setup()' and should never be used again. +void LD2410Component::set_gate_move_sensor(uint8_t gate, sensor::Sensor *s) { + this->gate_move_sensors_[gate] = new SensorWithDedup(s); +} + +void LD2410Component::set_gate_still_sensor(uint8_t gate, sensor::Sensor *s) { + this->gate_still_sensors_[gate] = new SensorWithDedup(s); +} #endif } // namespace ld2410 diff --git a/esphome/components/ld2410/ld2410.h b/esphome/components/ld2410/ld2410.h index 8bd1dbcb5a..54fe1ce14d 100644 --- a/esphome/components/ld2410/ld2410.h +++ b/esphome/components/ld2410/ld2410.h @@ -22,15 +22,20 @@ #ifdef USE_TEXT_SENSOR #include "esphome/components/text_sensor/text_sensor.h" #endif +#include "esphome/components/ld24xx/ld24xx.h" #include "esphome/components/uart/uart.h" #include "esphome/core/automation.h" #include "esphome/core/helpers.h" +#include + namespace esphome { namespace ld2410 { -static const uint8_t MAX_LINE_LENGTH = 46; // Max characters for serial buffer -static const uint8_t TOTAL_GATES = 9; // Total number of gates supported by the LD2410 +using namespace ld24xx; + +static constexpr uint8_t MAX_LINE_LENGTH = 46; // Max characters for serial buffer +static constexpr uint8_t TOTAL_GATES = 9; // Total number of gates supported by the LD2410 class LD2410Component : public Component, public uart::UARTDevice { #ifdef USE_BINARY_SENSOR @@ -40,12 +45,12 @@ class LD2410Component : public Component, public uart::UARTDevice { SUB_BINARY_SENSOR(target) #endif #ifdef USE_SENSOR - SUB_SENSOR(light) - SUB_SENSOR(detection_distance) - SUB_SENSOR(moving_target_distance) - SUB_SENSOR(moving_target_energy) - SUB_SENSOR(still_target_distance) - SUB_SENSOR(still_target_energy) + SUB_SENSOR_WITH_DEDUP(light, uint8_t) + SUB_SENSOR_WITH_DEDUP(detection_distance, int) + SUB_SENSOR_WITH_DEDUP(moving_target_distance, int) + SUB_SENSOR_WITH_DEDUP(moving_target_energy, uint8_t) + SUB_SENSOR_WITH_DEDUP(still_target_distance, int) + SUB_SENSOR_WITH_DEDUP(still_target_energy, uint8_t) #endif #ifdef USE_TEXT_SENSOR SUB_TEXT_SENSOR(version) @@ -88,7 +93,6 @@ class LD2410Component : public Component, public uart::UARTDevice { void set_gate_move_sensor(uint8_t gate, sensor::Sensor *s); void set_gate_still_sensor(uint8_t gate, sensor::Sensor *s); #endif - void set_throttle(uint16_t value) { this->throttle_ = value; }; void set_bluetooth_password(const std::string &password); void set_engineering_mode(bool enable); void read_all_info(); @@ -111,8 +115,6 @@ class LD2410Component : public Component, public uart::UARTDevice { void query_light_control_(); void restart_(); - uint32_t last_periodic_millis_ = 0; - uint16_t throttle_ = 0; uint8_t light_function_ = 0; uint8_t light_threshold_ = 0; uint8_t out_pin_level_ = 0; @@ -122,12 +124,12 @@ class LD2410Component : public Component, public uart::UARTDevice { uint8_t version_[6] = {0, 0, 0, 0, 0, 0}; bool bluetooth_on_{false}; #ifdef USE_NUMBER - std::vector gate_move_threshold_numbers_ = std::vector(TOTAL_GATES); - std::vector gate_still_threshold_numbers_ = std::vector(TOTAL_GATES); + std::array gate_move_threshold_numbers_{}; + std::array gate_still_threshold_numbers_{}; #endif #ifdef USE_SENSOR - std::vector gate_move_sensors_ = std::vector(TOTAL_GATES); - std::vector gate_still_sensors_ = std::vector(TOTAL_GATES); + std::array *, TOTAL_GATES> gate_move_sensors_{}; + std::array *, TOTAL_GATES> gate_still_sensors_{}; #endif }; diff --git a/esphome/components/ld2410/sensor.py b/esphome/components/ld2410/sensor.py index 92245ea9a6..fca2b2ceca 100644 --- a/esphome/components/ld2410/sensor.py +++ b/esphome/components/ld2410/sensor.py @@ -18,42 +18,50 @@ from esphome.const import ( from . import CONF_LD2410_ID, LD2410Component DEPENDENCIES = ["ld2410"] -CONF_STILL_DISTANCE = "still_distance" -CONF_MOVING_ENERGY = "moving_energy" -CONF_STILL_ENERGY = "still_energy" + CONF_DETECTION_DISTANCE = "detection_distance" CONF_MOVE_ENERGY = "move_energy" +CONF_MOVING_ENERGY = "moving_energy" +CONF_STILL_DISTANCE = "still_distance" +CONF_STILL_ENERGY = "still_energy" + CONFIG_SCHEMA = cv.Schema( { cv.GenerateID(CONF_LD2410_ID): cv.use_id(LD2410Component), cv.Optional(CONF_MOVING_DISTANCE): sensor.sensor_schema( device_class=DEVICE_CLASS_DISTANCE, - unit_of_measurement=UNIT_CENTIMETER, + filters=[{"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}], icon=ICON_SIGNAL, + unit_of_measurement=UNIT_CENTIMETER, ), cv.Optional(CONF_STILL_DISTANCE): sensor.sensor_schema( device_class=DEVICE_CLASS_DISTANCE, - unit_of_measurement=UNIT_CENTIMETER, + filters=[{"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}], icon=ICON_SIGNAL, + unit_of_measurement=UNIT_CENTIMETER, ), cv.Optional(CONF_MOVING_ENERGY): sensor.sensor_schema( - unit_of_measurement=UNIT_PERCENT, + filters=[{"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}], icon=ICON_MOTION_SENSOR, + unit_of_measurement=UNIT_PERCENT, ), cv.Optional(CONF_STILL_ENERGY): sensor.sensor_schema( - unit_of_measurement=UNIT_PERCENT, + filters=[{"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}], icon=ICON_FLASH, + unit_of_measurement=UNIT_PERCENT, ), cv.Optional(CONF_LIGHT): sensor.sensor_schema( device_class=DEVICE_CLASS_ILLUMINANCE, entity_category=ENTITY_CATEGORY_DIAGNOSTIC, + filters=[{"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}], icon=ICON_LIGHTBULB, ), cv.Optional(CONF_DETECTION_DISTANCE): sensor.sensor_schema( device_class=DEVICE_CLASS_DISTANCE, - unit_of_measurement=UNIT_CENTIMETER, + filters=[{"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}], icon=ICON_SIGNAL, + unit_of_measurement=UNIT_CENTIMETER, ), } ) @@ -63,14 +71,20 @@ CONFIG_SCHEMA = CONFIG_SCHEMA.extend( cv.Optional(f"g{x}"): cv.Schema( { cv.Optional(CONF_MOVE_ENERGY): sensor.sensor_schema( - unit_of_measurement=UNIT_PERCENT, entity_category=ENTITY_CATEGORY_DIAGNOSTIC, + filters=[ + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)} + ], icon=ICON_MOTION_SENSOR, + unit_of_measurement=UNIT_PERCENT, ), cv.Optional(CONF_STILL_ENERGY): sensor.sensor_schema( - unit_of_measurement=UNIT_PERCENT, entity_category=ENTITY_CATEGORY_DIAGNOSTIC, + filters=[ + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)} + ], icon=ICON_FLASH, + unit_of_measurement=UNIT_PERCENT, ), } ) diff --git a/esphome/components/ld2412/__init__.py b/esphome/components/ld2412/__init__.py new file mode 100644 index 0000000000..e701d0bda9 --- /dev/null +++ b/esphome/components/ld2412/__init__.py @@ -0,0 +1,46 @@ +import esphome.codegen as cg +from esphome.components import uart +import esphome.config_validation as cv +from esphome.const import CONF_ID, CONF_THROTTLE + +AUTO_LOAD = ["ld24xx"] +CODEOWNERS = ["@Rihan9"] +DEPENDENCIES = ["uart"] +MULTI_CONF = True + +LD2412_ns = cg.esphome_ns.namespace("ld2412") +LD2412Component = LD2412_ns.class_("LD2412Component", cg.Component, uart.UARTDevice) + +CONF_LD2412_ID = "ld2412_id" + +CONF_MAX_MOVE_DISTANCE = "max_move_distance" +CONF_MAX_STILL_DISTANCE = "max_still_distance" +CONF_MOVE_THRESHOLDS = [f"g{x}_move_threshold" for x in range(9)] +CONF_STILL_THRESHOLDS = [f"g{x}_still_threshold" for x in range(9)] + +CONFIG_SCHEMA = ( + cv.Schema( + { + cv.GenerateID(): cv.declare_id(LD2412Component), + cv.Optional(CONF_THROTTLE): cv.invalid( + f"{CONF_THROTTLE} has been removed; use per-sensor filters, instead" + ), + } + ) + .extend(uart.UART_DEVICE_SCHEMA) + .extend(cv.COMPONENT_SCHEMA) +) + +FINAL_VALIDATE_SCHEMA = uart.final_validate_device_schema( + "ld2412", + require_tx=True, + require_rx=True, + parity="NONE", + stop_bits=1, +) + + +async def to_code(config): + var = cg.new_Pvariable(config[CONF_ID]) + await cg.register_component(var, config) + await uart.register_uart_device(var, config) diff --git a/esphome/components/ld2412/binary_sensor.py b/esphome/components/ld2412/binary_sensor.py new file mode 100644 index 0000000000..aa1b0d2cd8 --- /dev/null +++ b/esphome/components/ld2412/binary_sensor.py @@ -0,0 +1,70 @@ +import esphome.codegen as cg +from esphome.components import binary_sensor +import esphome.config_validation as cv +from esphome.const import ( + CONF_HAS_MOVING_TARGET, + CONF_HAS_STILL_TARGET, + CONF_HAS_TARGET, + DEVICE_CLASS_MOTION, + DEVICE_CLASS_OCCUPANCY, + DEVICE_CLASS_RUNNING, + ENTITY_CATEGORY_DIAGNOSTIC, + ICON_ACCOUNT, + ICON_MOTION_SENSOR, +) + +from . import CONF_LD2412_ID, LD2412Component + +DEPENDENCIES = ["ld2412"] + +CONF_DYNAMIC_BACKGROUND_CORRECTION_STATUS = "dynamic_background_correction_status" + +CONFIG_SCHEMA = { + cv.GenerateID(CONF_LD2412_ID): cv.use_id(LD2412Component), + cv.Optional( + CONF_DYNAMIC_BACKGROUND_CORRECTION_STATUS + ): binary_sensor.binary_sensor_schema( + device_class=DEVICE_CLASS_RUNNING, + entity_category=ENTITY_CATEGORY_DIAGNOSTIC, + icon=ICON_ACCOUNT, + ), + cv.Optional(CONF_HAS_TARGET): binary_sensor.binary_sensor_schema( + device_class=DEVICE_CLASS_OCCUPANCY, + filters=[{"settle": cv.TimePeriod(milliseconds=1000)}], + icon=ICON_ACCOUNT, + ), + cv.Optional(CONF_HAS_MOVING_TARGET): binary_sensor.binary_sensor_schema( + device_class=DEVICE_CLASS_MOTION, + filters=[{"settle": cv.TimePeriod(milliseconds=1000)}], + icon=ICON_MOTION_SENSOR, + ), + cv.Optional(CONF_HAS_STILL_TARGET): binary_sensor.binary_sensor_schema( + device_class=DEVICE_CLASS_OCCUPANCY, + filters=[{"settle": cv.TimePeriod(milliseconds=1000)}], + icon=ICON_MOTION_SENSOR, + ), +} + + +async def to_code(config): + LD2412_component = await cg.get_variable(config[CONF_LD2412_ID]) + if dynamic_background_correction_status_config := config.get( + CONF_DYNAMIC_BACKGROUND_CORRECTION_STATUS + ): + sens = await binary_sensor.new_binary_sensor( + dynamic_background_correction_status_config + ) + cg.add( + LD2412_component.set_dynamic_background_correction_status_binary_sensor( + sens + ) + ) + if has_target_config := config.get(CONF_HAS_TARGET): + sens = await binary_sensor.new_binary_sensor(has_target_config) + cg.add(LD2412_component.set_target_binary_sensor(sens)) + if has_moving_target_config := config.get(CONF_HAS_MOVING_TARGET): + sens = await binary_sensor.new_binary_sensor(has_moving_target_config) + cg.add(LD2412_component.set_moving_target_binary_sensor(sens)) + if has_still_target_config := config.get(CONF_HAS_STILL_TARGET): + sens = await binary_sensor.new_binary_sensor(has_still_target_config) + cg.add(LD2412_component.set_still_target_binary_sensor(sens)) diff --git a/esphome/components/ld2412/button/__init__.py b/esphome/components/ld2412/button/__init__.py new file mode 100644 index 0000000000..e78cad4b88 --- /dev/null +++ b/esphome/components/ld2412/button/__init__.py @@ -0,0 +1,74 @@ +import esphome.codegen as cg +from esphome.components import button +import esphome.config_validation as cv +from esphome.const import ( + CONF_FACTORY_RESET, + CONF_RESTART, + DEVICE_CLASS_RESTART, + ENTITY_CATEGORY_CONFIG, + ENTITY_CATEGORY_DIAGNOSTIC, + ICON_DATABASE, + ICON_PULSE, + ICON_RESTART, + ICON_RESTART_ALERT, +) + +from .. import CONF_LD2412_ID, LD2412_ns, LD2412Component + +FactoryResetButton = LD2412_ns.class_("FactoryResetButton", button.Button) +QueryButton = LD2412_ns.class_("QueryButton", button.Button) +RestartButton = LD2412_ns.class_("RestartButton", button.Button) +StartDynamicBackgroundCorrectionButton = LD2412_ns.class_( + "StartDynamicBackgroundCorrectionButton", button.Button +) + +CONF_QUERY_PARAMS = "query_params" +CONF_START_DYNAMIC_BACKGROUND_CORRECTION = "start_dynamic_background_correction" + +CONFIG_SCHEMA = { + cv.GenerateID(CONF_LD2412_ID): cv.use_id(LD2412Component), + cv.Optional(CONF_FACTORY_RESET): button.button_schema( + FactoryResetButton, + device_class=DEVICE_CLASS_RESTART, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_RESTART_ALERT, + ), + cv.Optional(CONF_QUERY_PARAMS): button.button_schema( + QueryButton, + entity_category=ENTITY_CATEGORY_DIAGNOSTIC, + icon=ICON_DATABASE, + ), + cv.Optional(CONF_RESTART): button.button_schema( + RestartButton, + device_class=DEVICE_CLASS_RESTART, + entity_category=ENTITY_CATEGORY_DIAGNOSTIC, + icon=ICON_RESTART, + ), + cv.Optional(CONF_START_DYNAMIC_BACKGROUND_CORRECTION): button.button_schema( + StartDynamicBackgroundCorrectionButton, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_PULSE, + ), +} + + +async def to_code(config): + LD2412_component = await cg.get_variable(config[CONF_LD2412_ID]) + if factory_reset_config := config.get(CONF_FACTORY_RESET): + b = await button.new_button(factory_reset_config) + await cg.register_parented(b, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_factory_reset_button(b)) + if query_params_config := config.get(CONF_QUERY_PARAMS): + b = await button.new_button(query_params_config) + await cg.register_parented(b, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_query_button(b)) + if restart_config := config.get(CONF_RESTART): + b = await button.new_button(restart_config) + await cg.register_parented(b, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_restart_button(b)) + if start_dynamic_background_correction_config := config.get( + CONF_START_DYNAMIC_BACKGROUND_CORRECTION + ): + b = await button.new_button(start_dynamic_background_correction_config) + await cg.register_parented(b, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_start_dynamic_background_correction_button(b)) diff --git a/esphome/components/ld2412/button/factory_reset_button.cpp b/esphome/components/ld2412/button/factory_reset_button.cpp new file mode 100644 index 0000000000..7ee85bc8f9 --- /dev/null +++ b/esphome/components/ld2412/button/factory_reset_button.cpp @@ -0,0 +1,9 @@ +#include "factory_reset_button.h" + +namespace esphome { +namespace ld2412 { + +void FactoryResetButton::press_action() { this->parent_->factory_reset(); } + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/button/factory_reset_button.h b/esphome/components/ld2412/button/factory_reset_button.h new file mode 100644 index 0000000000..36a3fffcd5 --- /dev/null +++ b/esphome/components/ld2412/button/factory_reset_button.h @@ -0,0 +1,18 @@ +#pragma once + +#include "esphome/components/button/button.h" +#include "../ld2412.h" + +namespace esphome { +namespace ld2412 { + +class FactoryResetButton : public button::Button, public Parented { + public: + FactoryResetButton() = default; + + protected: + void press_action() override; +}; + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/button/query_button.cpp b/esphome/components/ld2412/button/query_button.cpp new file mode 100644 index 0000000000..536f74427f --- /dev/null +++ b/esphome/components/ld2412/button/query_button.cpp @@ -0,0 +1,9 @@ +#include "query_button.h" + +namespace esphome { +namespace ld2412 { + +void QueryButton::press_action() { this->parent_->read_all_info(); } + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/button/query_button.h b/esphome/components/ld2412/button/query_button.h new file mode 100644 index 0000000000..595ef6d1e9 --- /dev/null +++ b/esphome/components/ld2412/button/query_button.h @@ -0,0 +1,18 @@ +#pragma once + +#include "esphome/components/button/button.h" +#include "../ld2412.h" + +namespace esphome { +namespace ld2412 { + +class QueryButton : public button::Button, public Parented { + public: + QueryButton() = default; + + protected: + void press_action() override; +}; + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/button/restart_button.cpp b/esphome/components/ld2412/button/restart_button.cpp new file mode 100644 index 0000000000..aca0d17841 --- /dev/null +++ b/esphome/components/ld2412/button/restart_button.cpp @@ -0,0 +1,9 @@ +#include "restart_button.h" + +namespace esphome { +namespace ld2412 { + +void RestartButton::press_action() { this->parent_->restart_and_read_all_info(); } + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/button/restart_button.h b/esphome/components/ld2412/button/restart_button.h new file mode 100644 index 0000000000..5cd582e2a3 --- /dev/null +++ b/esphome/components/ld2412/button/restart_button.h @@ -0,0 +1,18 @@ +#pragma once + +#include "esphome/components/button/button.h" +#include "../ld2412.h" + +namespace esphome { +namespace ld2412 { + +class RestartButton : public button::Button, public Parented { + public: + RestartButton() = default; + + protected: + void press_action() override; +}; + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/button/start_dynamic_background_correction_button.cpp b/esphome/components/ld2412/button/start_dynamic_background_correction_button.cpp new file mode 100644 index 0000000000..9b37243b82 --- /dev/null +++ b/esphome/components/ld2412/button/start_dynamic_background_correction_button.cpp @@ -0,0 +1,11 @@ +#include "start_dynamic_background_correction_button.h" + +#include "restart_button.h" + +namespace esphome { +namespace ld2412 { + +void StartDynamicBackgroundCorrectionButton::press_action() { this->parent_->start_dynamic_background_correction(); } + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/button/start_dynamic_background_correction_button.h b/esphome/components/ld2412/button/start_dynamic_background_correction_button.h new file mode 100644 index 0000000000..3af0a8a149 --- /dev/null +++ b/esphome/components/ld2412/button/start_dynamic_background_correction_button.h @@ -0,0 +1,18 @@ +#pragma once + +#include "esphome/components/button/button.h" +#include "../ld2412.h" + +namespace esphome { +namespace ld2412 { + +class StartDynamicBackgroundCorrectionButton : public button::Button, public Parented { + public: + StartDynamicBackgroundCorrectionButton() = default; + + protected: + void press_action() override; +}; + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/ld2412.cpp b/esphome/components/ld2412/ld2412.cpp new file mode 100644 index 0000000000..63af69ce0d --- /dev/null +++ b/esphome/components/ld2412/ld2412.cpp @@ -0,0 +1,861 @@ +#include "ld2412.h" + +#ifdef USE_NUMBER +#include "esphome/components/number/number.h" +#endif +#ifdef USE_SENSOR +#include "esphome/components/sensor/sensor.h" +#endif + +#include "esphome/core/application.h" +#include "esphome/core/helpers.h" + +namespace esphome { +namespace ld2412 { + +static const char *const TAG = "ld2412"; +static const char *const UNKNOWN_MAC = "unknown"; +static const char *const VERSION_FMT = "%u.%02X.%02X%02X%02X%02X"; + +enum BaudRate : uint8_t { + BAUD_RATE_9600 = 1, + BAUD_RATE_19200 = 2, + BAUD_RATE_38400 = 3, + BAUD_RATE_57600 = 4, + BAUD_RATE_115200 = 5, + BAUD_RATE_230400 = 6, + BAUD_RATE_256000 = 7, + BAUD_RATE_460800 = 8, +}; + +enum DistanceResolution : uint8_t { + DISTANCE_RESOLUTION_0_2 = 0x03, + DISTANCE_RESOLUTION_0_5 = 0x01, + DISTANCE_RESOLUTION_0_75 = 0x00, +}; + +enum LightFunction : uint8_t { + LIGHT_FUNCTION_OFF = 0x00, + LIGHT_FUNCTION_BELOW = 0x01, + LIGHT_FUNCTION_ABOVE = 0x02, +}; + +enum OutPinLevel : uint8_t { + OUT_PIN_LEVEL_LOW = 0x01, + OUT_PIN_LEVEL_HIGH = 0x00, +}; + +/* +Data Type: 6th byte +Target states: 9th byte + Moving target distance: 10~11th bytes + Moving target energy: 12th byte + Still target distance: 13~14th bytes + Still target energy: 15th byte + Detect distance: 16~17th bytes +*/ +enum PeriodicData : uint8_t { + DATA_TYPES = 6, + TARGET_STATES = 8, + MOVING_TARGET_LOW = 9, + MOVING_TARGET_HIGH = 10, + MOVING_ENERGY = 11, + STILL_TARGET_LOW = 12, + STILL_TARGET_HIGH = 13, + STILL_ENERGY = 14, + MOVING_SENSOR_START = 17, + STILL_SENSOR_START = 31, + LIGHT_SENSOR = 45, + OUT_PIN_SENSOR = 38, +}; + +enum PeriodicDataValue : uint8_t { + HEADER = 0XAA, + FOOTER = 0x55, + CHECK = 0x00, +}; + +enum AckData : uint8_t { + COMMAND = 6, + COMMAND_STATUS = 7, +}; + +// Memory-efficient lookup tables +struct StringToUint8 { + const char *str; + const uint8_t value; +}; + +struct Uint8ToString { + const uint8_t value; + const char *str; +}; + +constexpr StringToUint8 BAUD_RATES_BY_STR[] = { + {"9600", BAUD_RATE_9600}, {"19200", BAUD_RATE_19200}, {"38400", BAUD_RATE_38400}, + {"57600", BAUD_RATE_57600}, {"115200", BAUD_RATE_115200}, {"230400", BAUD_RATE_230400}, + {"256000", BAUD_RATE_256000}, {"460800", BAUD_RATE_460800}, +}; + +constexpr StringToUint8 DISTANCE_RESOLUTIONS_BY_STR[] = { + {"0.2m", DISTANCE_RESOLUTION_0_2}, + {"0.5m", DISTANCE_RESOLUTION_0_5}, + {"0.75m", DISTANCE_RESOLUTION_0_75}, +}; + +constexpr Uint8ToString DISTANCE_RESOLUTIONS_BY_UINT[] = { + {DISTANCE_RESOLUTION_0_2, "0.2m"}, + {DISTANCE_RESOLUTION_0_5, "0.5m"}, + {DISTANCE_RESOLUTION_0_75, "0.75m"}, +}; + +constexpr StringToUint8 LIGHT_FUNCTIONS_BY_STR[] = { + {"off", LIGHT_FUNCTION_OFF}, + {"below", LIGHT_FUNCTION_BELOW}, + {"above", LIGHT_FUNCTION_ABOVE}, +}; + +constexpr Uint8ToString LIGHT_FUNCTIONS_BY_UINT[] = { + {LIGHT_FUNCTION_OFF, "off"}, + {LIGHT_FUNCTION_BELOW, "below"}, + {LIGHT_FUNCTION_ABOVE, "above"}, +}; + +constexpr StringToUint8 OUT_PIN_LEVELS_BY_STR[] = { + {"low", OUT_PIN_LEVEL_LOW}, + {"high", OUT_PIN_LEVEL_HIGH}, +}; + +constexpr Uint8ToString OUT_PIN_LEVELS_BY_UINT[] = { + {OUT_PIN_LEVEL_LOW, "low"}, + {OUT_PIN_LEVEL_HIGH, "high"}, +}; + +// Helper functions for lookups +template uint8_t find_uint8(const StringToUint8 (&arr)[N], const std::string &str) { + for (const auto &entry : arr) { + if (str == entry.str) { + return entry.value; + } + } + return 0xFF; // Not found +} + +template const char *find_str(const Uint8ToString (&arr)[N], uint8_t value) { + for (const auto &entry : arr) { + if (value == entry.value) { + return entry.str; + } + } + return ""; // Not found +} + +static constexpr uint8_t DEFAULT_PRESENCE_TIMEOUT = 5; // Default used when number component is not defined +// Commands +static constexpr uint8_t CMD_ENABLE_CONF = 0xFF; +static constexpr uint8_t CMD_DISABLE_CONF = 0xFE; +static constexpr uint8_t CMD_ENABLE_ENG = 0x62; +static constexpr uint8_t CMD_DISABLE_ENG = 0x63; +static constexpr uint8_t CMD_QUERY_BASIC_CONF = 0x12; +static constexpr uint8_t CMD_BASIC_CONF = 0x02; +static constexpr uint8_t CMD_QUERY_VERSION = 0xA0; +static constexpr uint8_t CMD_QUERY_DISTANCE_RESOLUTION = 0x11; +static constexpr uint8_t CMD_SET_DISTANCE_RESOLUTION = 0x01; +static constexpr uint8_t CMD_QUERY_LIGHT_CONTROL = 0x1C; +static constexpr uint8_t CMD_SET_LIGHT_CONTROL = 0x0C; +static constexpr uint8_t CMD_SET_BAUD_RATE = 0xA1; +static constexpr uint8_t CMD_QUERY_MAC_ADDRESS = 0xA5; +static constexpr uint8_t CMD_FACTORY_RESET = 0xA2; +static constexpr uint8_t CMD_RESTART = 0xA3; +static constexpr uint8_t CMD_BLUETOOTH = 0xA4; +static constexpr uint8_t CMD_DYNAMIC_BACKGROUND_CORRECTION = 0x0B; +static constexpr uint8_t CMD_QUERY_DYNAMIC_BACKGROUND_CORRECTION = 0x1B; +static constexpr uint8_t CMD_MOTION_GATE_SENS = 0x03; +static constexpr uint8_t CMD_QUERY_MOTION_GATE_SENS = 0x13; +static constexpr uint8_t CMD_STATIC_GATE_SENS = 0x04; +static constexpr uint8_t CMD_QUERY_STATIC_GATE_SENS = 0x14; +static constexpr uint8_t CMD_NONE = 0x00; +// Commands values +static constexpr uint8_t CMD_MAX_MOVE_VALUE = 0x00; +static constexpr uint8_t CMD_MAX_STILL_VALUE = 0x01; +static constexpr uint8_t CMD_DURATION_VALUE = 0x02; +// Bitmasks for target states +static constexpr uint8_t MOVE_BITMASK = 0x01; +static constexpr uint8_t STILL_BITMASK = 0x02; +// Header & Footer size +static constexpr uint8_t HEADER_FOOTER_SIZE = 4; +// Command Header & Footer +static constexpr uint8_t CMD_FRAME_HEADER[HEADER_FOOTER_SIZE] = {0xFD, 0xFC, 0xFB, 0xFA}; +static constexpr uint8_t CMD_FRAME_FOOTER[HEADER_FOOTER_SIZE] = {0x04, 0x03, 0x02, 0x01}; +// Data Header & Footer +static constexpr uint8_t DATA_FRAME_HEADER[HEADER_FOOTER_SIZE] = {0xF4, 0xF3, 0xF2, 0xF1}; +static constexpr uint8_t DATA_FRAME_FOOTER[HEADER_FOOTER_SIZE] = {0xF8, 0xF7, 0xF6, 0xF5}; +// MAC address the module uses when Bluetooth is disabled +static constexpr uint8_t NO_MAC[] = {0x08, 0x05, 0x04, 0x03, 0x02, 0x01}; + +static inline int two_byte_to_int(char firstbyte, char secondbyte) { return (int16_t) (secondbyte << 8) + firstbyte; } + +static inline bool validate_header_footer(const uint8_t *header_footer, const uint8_t *buffer) { + return std::memcmp(header_footer, buffer, HEADER_FOOTER_SIZE) == 0; +} + +void LD2412Component::dump_config() { + std::string mac_str = + mac_address_is_valid(this->mac_address_) ? format_mac_address_pretty(this->mac_address_) : UNKNOWN_MAC; + std::string version = str_sprintf(VERSION_FMT, this->version_[1], this->version_[0], this->version_[5], + this->version_[4], this->version_[3], this->version_[2]); + ESP_LOGCONFIG(TAG, + "LD2412:\n" + " Firmware version: %s\n" + " MAC address: %s", + version.c_str(), mac_str.c_str()); +#ifdef USE_BINARY_SENSOR + ESP_LOGCONFIG(TAG, "Binary Sensors:"); + LOG_BINARY_SENSOR(" ", "DynamicBackgroundCorrectionStatus", + this->dynamic_background_correction_status_binary_sensor_); + LOG_BINARY_SENSOR(" ", "MovingTarget", this->moving_target_binary_sensor_); + LOG_BINARY_SENSOR(" ", "StillTarget", this->still_target_binary_sensor_); + LOG_BINARY_SENSOR(" ", "Target", this->target_binary_sensor_); +#endif +#ifdef USE_SENSOR + ESP_LOGCONFIG(TAG, "Sensors:"); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "Light", this->light_sensor_); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "DetectionDistance", this->detection_distance_sensor_); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "MovingTargetDistance", this->moving_target_distance_sensor_); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "MovingTargetEnergy", this->moving_target_energy_sensor_); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "StillTargetDistance", this->still_target_distance_sensor_); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "StillTargetEnergy", this->still_target_energy_sensor_); + for (auto &s : this->gate_still_sensors_) { + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "GateStill", s); + } + for (auto &s : this->gate_move_sensors_) { + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "GateMove", s); + } +#endif +#ifdef USE_TEXT_SENSOR + ESP_LOGCONFIG(TAG, "Text Sensors:"); + LOG_TEXT_SENSOR(" ", "MAC address", this->mac_text_sensor_); + LOG_TEXT_SENSOR(" ", "Version", this->version_text_sensor_); +#endif +#ifdef USE_NUMBER + ESP_LOGCONFIG(TAG, "Numbers:"); + LOG_NUMBER(" ", "LightThreshold", this->light_threshold_number_); + LOG_NUMBER(" ", "MaxDistanceGate", this->max_distance_gate_number_); + LOG_NUMBER(" ", "MinDistanceGate", this->min_distance_gate_number_); + LOG_NUMBER(" ", "Timeout", this->timeout_number_); + for (number::Number *n : this->gate_move_threshold_numbers_) { + LOG_NUMBER(" ", "Move Thresholds", n); + } + for (number::Number *n : this->gate_still_threshold_numbers_) { + LOG_NUMBER(" ", "Still Thresholds", n); + } +#endif +#ifdef USE_SELECT + ESP_LOGCONFIG(TAG, "Selects:"); + LOG_SELECT(" ", "BaudRate", this->baud_rate_select_); + LOG_SELECT(" ", "DistanceResolution", this->distance_resolution_select_); + LOG_SELECT(" ", "LightFunction", this->light_function_select_); + LOG_SELECT(" ", "OutPinLevel", this->out_pin_level_select_); +#endif +#ifdef USE_SWITCH + ESP_LOGCONFIG(TAG, "Switches:"); + LOG_SWITCH(" ", "Bluetooth", this->bluetooth_switch_); + LOG_SWITCH(" ", "EngineeringMode", this->engineering_mode_switch_); +#endif +#ifdef USE_BUTTON + ESP_LOGCONFIG(TAG, "Buttons:"); + LOG_BUTTON(" ", "FactoryReset", this->factory_reset_button_); + LOG_BUTTON(" ", "Query", this->query_button_); + LOG_BUTTON(" ", "Restart", this->restart_button_); + LOG_BUTTON(" ", "StartDynamicBackgroundCorrection", this->start_dynamic_background_correction_button_); +#endif +} + +void LD2412Component::setup() { + ESP_LOGCONFIG(TAG, "Running setup"); + this->read_all_info(); +} + +void LD2412Component::read_all_info() { + this->set_config_mode_(true); + this->get_version_(); + delay(10); // NOLINT + this->get_mac_(); + delay(10); // NOLINT + this->get_distance_resolution_(); + delay(10); // NOLINT + this->query_parameters_(); + delay(10); // NOLINT + this->query_dynamic_background_correction_(); + delay(10); // NOLINT + this->query_light_control_(); + delay(10); // NOLINT +#ifdef USE_NUMBER + this->get_gate_threshold(); + delay(10); // NOLINT +#endif + this->set_config_mode_(false); +#ifdef USE_SELECT + const auto baud_rate = std::to_string(this->parent_->get_baud_rate()); + if (this->baud_rate_select_ != nullptr) { + this->baud_rate_select_->publish_state(baud_rate); + } +#endif +} + +void LD2412Component::restart_and_read_all_info() { + this->set_config_mode_(true); + this->restart_(); + this->set_timeout(1000, [this]() { this->read_all_info(); }); +} + +void LD2412Component::loop() { + while (this->available()) { + this->readline_(this->read()); + } +} + +void LD2412Component::send_command_(uint8_t command, const uint8_t *command_value, uint8_t command_value_len) { + ESP_LOGV(TAG, "Sending COMMAND %02X", command); + // frame header bytes + this->write_array(CMD_FRAME_HEADER, HEADER_FOOTER_SIZE); + // length bytes + uint8_t len = 2; + if (command_value != nullptr) { + len += command_value_len; + } + // 2 length bytes (low, high) + 2 command bytes (low, high) + uint8_t len_cmd[] = {len, 0x00, command, 0x00}; + this->write_array(len_cmd, sizeof(len_cmd)); + + // command value bytes + if (command_value != nullptr) { + this->write_array(command_value, command_value_len); + } + // frame footer bytes + this->write_array(CMD_FRAME_FOOTER, HEADER_FOOTER_SIZE); + + if (command != CMD_ENABLE_CONF && command != CMD_DISABLE_CONF) { + delay(30); // NOLINT + } + delay(20); // NOLINT +} + +void LD2412Component::handle_periodic_data_() { + // 4 frame header bytes + 2 length bytes + 1 data end byte + 1 crc byte + 4 frame footer bytes + // data header=0xAA, data footer=0x55, crc=0x00 + if (this->buffer_pos_ < 12 || !ld2412::validate_header_footer(DATA_FRAME_HEADER, this->buffer_data_) || + this->buffer_data_[7] != HEADER || this->buffer_data_[this->buffer_pos_ - 6] != FOOTER) { + return; + } + /* + Data Type: 7th + 0x01: Engineering mode + 0x02: Normal mode + */ + bool engineering_mode = this->buffer_data_[DATA_TYPES] == 0x01; +#ifdef USE_SWITCH + if (this->engineering_mode_switch_ != nullptr) { + this->engineering_mode_switch_->publish_state(engineering_mode); + } +#endif + +#ifdef USE_BINARY_SENSOR + /* + Target states: 9th + 0x00 = No target + 0x01 = Moving targets + 0x02 = Still targets + 0x03 = Moving+Still targets + */ + char target_state = this->buffer_data_[TARGET_STATES]; + if (this->target_binary_sensor_ != nullptr) { + this->target_binary_sensor_->publish_state(target_state != 0x00); + } + if (this->moving_target_binary_sensor_ != nullptr) { + this->moving_target_binary_sensor_->publish_state(target_state & MOVE_BITMASK); + } + if (this->still_target_binary_sensor_ != nullptr) { + this->still_target_binary_sensor_->publish_state(target_state & STILL_BITMASK); + } +#endif + /* + Moving target distance: 10~11th bytes + Moving target energy: 12th byte + Still target distance: 13~14th bytes + Still target energy: 15th byte + Detect distance: 16~17th bytes + */ +#ifdef USE_SENSOR + SAFE_PUBLISH_SENSOR( + this->moving_target_distance_sensor_, + ld2412::two_byte_to_int(this->buffer_data_[MOVING_TARGET_LOW], this->buffer_data_[MOVING_TARGET_HIGH])) + SAFE_PUBLISH_SENSOR(this->moving_target_energy_sensor_, this->buffer_data_[MOVING_ENERGY]) + SAFE_PUBLISH_SENSOR( + this->still_target_distance_sensor_, + ld2412::two_byte_to_int(this->buffer_data_[STILL_TARGET_LOW], this->buffer_data_[STILL_TARGET_HIGH])) + SAFE_PUBLISH_SENSOR(this->still_target_energy_sensor_, this->buffer_data_[STILL_ENERGY]) + if (this->detection_distance_sensor_ != nullptr) { + int new_detect_distance = 0; + if (target_state != 0x00 && (target_state & MOVE_BITMASK)) { + new_detect_distance = + ld2412::two_byte_to_int(this->buffer_data_[MOVING_TARGET_LOW], this->buffer_data_[MOVING_TARGET_HIGH]); + } else if (target_state != 0x00) { + new_detect_distance = + ld2412::two_byte_to_int(this->buffer_data_[STILL_TARGET_LOW], this->buffer_data_[STILL_TARGET_HIGH]); + } + this->detection_distance_sensor_->publish_state_if_not_dup(new_detect_distance); + } + if (engineering_mode) { + /* + Moving distance range: 18th byte + Still distance range: 19th byte + Moving energy: 20~28th bytes + */ + for (uint8_t i = 0; i < TOTAL_GATES; i++) { + SAFE_PUBLISH_SENSOR(this->gate_move_sensors_[i], this->buffer_data_[MOVING_SENSOR_START + i]) + } + /* + Still energy: 29~37th bytes + */ + for (uint8_t i = 0; i < TOTAL_GATES; i++) { + SAFE_PUBLISH_SENSOR(this->gate_still_sensors_[i], this->buffer_data_[STILL_SENSOR_START + i]) + } + /* + Light sensor: 38th bytes + */ + SAFE_PUBLISH_SENSOR(this->light_sensor_, this->buffer_data_[LIGHT_SENSOR]) + } else { + for (auto &gate_move_sensor : this->gate_move_sensors_) { + SAFE_PUBLISH_SENSOR_UNKNOWN(gate_move_sensor) + } + for (auto &gate_still_sensor : this->gate_still_sensors_) { + SAFE_PUBLISH_SENSOR_UNKNOWN(gate_still_sensor) + } + SAFE_PUBLISH_SENSOR_UNKNOWN(this->light_sensor_) + } +#endif + // the radar module won't tell us when it's done, so we just have to keep polling... + if (this->dynamic_background_correction_active_) { + this->set_config_mode_(true); + this->query_dynamic_background_correction_(); + this->set_config_mode_(false); + } +} + +#ifdef USE_NUMBER +std::function set_number_value(number::Number *n, float value) { + if (n != nullptr && (!n->has_state() || n->state != value)) { + n->state = value; + return [n, value]() { n->publish_state(value); }; + } + return []() {}; +} +#endif + +bool LD2412Component::handle_ack_data_() { + ESP_LOGV(TAG, "Handling ACK DATA for COMMAND %02X", this->buffer_data_[COMMAND]); + if (this->buffer_pos_ < 10) { + ESP_LOGW(TAG, "Invalid length"); + return true; + } + if (!ld2412::validate_header_footer(CMD_FRAME_HEADER, this->buffer_data_)) { + ESP_LOGW(TAG, "Invalid header: %s", format_hex_pretty(this->buffer_data_, HEADER_FOOTER_SIZE).c_str()); + return true; + } + if (this->buffer_data_[COMMAND_STATUS] != 0x01) { + ESP_LOGW(TAG, "Invalid status"); + return true; + } + if (this->buffer_data_[8] || this->buffer_data_[9]) { + ESP_LOGW(TAG, "Invalid command: %02X, %02X", this->buffer_data_[8], this->buffer_data_[9]); + return true; + } + + switch (this->buffer_data_[COMMAND]) { + case CMD_ENABLE_CONF: + ESP_LOGV(TAG, "Enable conf"); + break; + + case CMD_DISABLE_CONF: + ESP_LOGV(TAG, "Disabled conf"); + break; + + case CMD_SET_BAUD_RATE: + ESP_LOGV(TAG, "Baud rate change"); +#ifdef USE_SELECT + if (this->baud_rate_select_ != nullptr) { + ESP_LOGW(TAG, "Change baud rate to %s and reinstall", this->baud_rate_select_->state.c_str()); + } +#endif + break; + + case CMD_QUERY_VERSION: { + std::memcpy(this->version_, &this->buffer_data_[12], sizeof(this->version_)); + std::string version = str_sprintf(VERSION_FMT, this->version_[1], this->version_[0], this->version_[5], + this->version_[4], this->version_[3], this->version_[2]); + ESP_LOGV(TAG, "Firmware version: %s", version.c_str()); +#ifdef USE_TEXT_SENSOR + if (this->version_text_sensor_ != nullptr) { + this->version_text_sensor_->publish_state(version); + } +#endif + break; + } + case CMD_QUERY_DISTANCE_RESOLUTION: { + const auto *distance_resolution = find_str(DISTANCE_RESOLUTIONS_BY_UINT, this->buffer_data_[10]); + ESP_LOGV(TAG, "Distance resolution: %s", distance_resolution); +#ifdef USE_SELECT + if (this->distance_resolution_select_ != nullptr) { + this->distance_resolution_select_->publish_state(distance_resolution); + } +#endif + break; + } + + case CMD_QUERY_LIGHT_CONTROL: { + this->light_function_ = this->buffer_data_[10]; + this->light_threshold_ = this->buffer_data_[11]; + const auto *light_function_str = find_str(LIGHT_FUNCTIONS_BY_UINT, this->light_function_); + ESP_LOGV(TAG, + "Light function: %s\n" + "Light threshold: %u", + light_function_str, this->light_threshold_); +#ifdef USE_SELECT + if (this->light_function_select_ != nullptr) { + this->light_function_select_->publish_state(light_function_str); + } +#endif +#ifdef USE_NUMBER + if (this->light_threshold_number_ != nullptr) { + this->light_threshold_number_->publish_state(static_cast(this->light_threshold_)); + } +#endif + break; + } + + case CMD_QUERY_MAC_ADDRESS: { + if (this->buffer_pos_ < 20) { + return false; + } + + this->bluetooth_on_ = std::memcmp(&this->buffer_data_[10], NO_MAC, sizeof(NO_MAC)) != 0; + if (this->bluetooth_on_) { + std::memcpy(this->mac_address_, &this->buffer_data_[10], sizeof(this->mac_address_)); + } + + std::string mac_str = + mac_address_is_valid(this->mac_address_) ? format_mac_address_pretty(this->mac_address_) : UNKNOWN_MAC; + ESP_LOGV(TAG, "MAC address: %s", mac_str.c_str()); +#ifdef USE_TEXT_SENSOR + if (this->mac_text_sensor_ != nullptr) { + this->mac_text_sensor_->publish_state(mac_str); + } +#endif +#ifdef USE_SWITCH + if (this->bluetooth_switch_ != nullptr) { + this->bluetooth_switch_->publish_state(this->bluetooth_on_); + } +#endif + break; + } + + case CMD_SET_DISTANCE_RESOLUTION: + ESP_LOGV(TAG, "Handled set distance resolution command"); + break; + + case CMD_QUERY_DYNAMIC_BACKGROUND_CORRECTION: { + ESP_LOGV(TAG, "Handled query dynamic background correction"); + bool dynamic_background_correction_active = (this->buffer_data_[10] != 0x00); +#ifdef USE_BINARY_SENSOR + if (this->dynamic_background_correction_status_binary_sensor_ != nullptr) { + this->dynamic_background_correction_status_binary_sensor_->publish_state(dynamic_background_correction_active); + } +#endif + this->dynamic_background_correction_active_ = dynamic_background_correction_active; + break; + } + + case CMD_BLUETOOTH: + ESP_LOGV(TAG, "Handled bluetooth command"); + break; + + case CMD_SET_LIGHT_CONTROL: + ESP_LOGV(TAG, "Handled set light control command"); + break; + + case CMD_QUERY_MOTION_GATE_SENS: { +#ifdef USE_NUMBER + std::vector> updates; + updates.reserve(this->gate_still_threshold_numbers_.size()); + for (size_t i = 0; i < this->gate_still_threshold_numbers_.size(); i++) { + updates.push_back(set_number_value(this->gate_move_threshold_numbers_[i], this->buffer_data_[10 + i])); + } + for (auto &update : updates) { + update(); + } +#endif + break; + } + + case CMD_QUERY_STATIC_GATE_SENS: { +#ifdef USE_NUMBER + std::vector> updates; + updates.reserve(this->gate_still_threshold_numbers_.size()); + for (size_t i = 0; i < this->gate_still_threshold_numbers_.size(); i++) { + updates.push_back(set_number_value(this->gate_still_threshold_numbers_[i], this->buffer_data_[10 + i])); + } + for (auto &update : updates) { + update(); + } +#endif + break; + } + + case CMD_QUERY_BASIC_CONF: // Query parameters response + { +#ifdef USE_NUMBER + /* + Moving distance range: 9th byte + Still distance range: 10th byte + */ + std::vector> updates; + updates.push_back(set_number_value(this->min_distance_gate_number_, this->buffer_data_[10])); + updates.push_back(set_number_value(this->max_distance_gate_number_, this->buffer_data_[11] - 1)); + ESP_LOGV(TAG, "min_distance_gate_number_: %u, max_distance_gate_number_ %u", this->buffer_data_[10], + this->buffer_data_[11]); + /* + None Duration: 11~12th bytes + */ + updates.push_back(set_number_value(this->timeout_number_, + ld2412::two_byte_to_int(this->buffer_data_[12], this->buffer_data_[13]))); + ESP_LOGV(TAG, "timeout_number_: %u", ld2412::two_byte_to_int(this->buffer_data_[12], this->buffer_data_[13])); + /* + Output pin configuration: 13th bytes + */ + this->out_pin_level_ = this->buffer_data_[14]; +#ifdef USE_SELECT + const auto *out_pin_level_str = find_str(OUT_PIN_LEVELS_BY_UINT, this->out_pin_level_); + if (this->out_pin_level_select_ != nullptr) { + this->out_pin_level_select_->publish_state(out_pin_level_str); + } +#endif + for (auto &update : updates) { + update(); + } +#endif + } break; + default: + break; + } + + return true; +} + +void LD2412Component::readline_(int readch) { + if (readch < 0) { + return; // No data available + } + if (this->buffer_pos_ < HEADER_FOOTER_SIZE && readch != DATA_FRAME_HEADER[this->buffer_pos_] && + readch != CMD_FRAME_HEADER[this->buffer_pos_]) { + this->buffer_pos_ = 0; + return; + } + if (this->buffer_pos_ < MAX_LINE_LENGTH - 1) { + this->buffer_data_[this->buffer_pos_++] = readch; + this->buffer_data_[this->buffer_pos_] = 0; + } else { + // We should never get here, but just in case... + ESP_LOGW(TAG, "Max command length exceeded; ignoring"); + this->buffer_pos_ = 0; + } + if (this->buffer_pos_ < 4) { + return; // Not enough data to process yet + } + if (ld2412::validate_header_footer(DATA_FRAME_FOOTER, &this->buffer_data_[this->buffer_pos_ - 4])) { + ESP_LOGV(TAG, "Handling Periodic Data: %s", format_hex_pretty(this->buffer_data_, this->buffer_pos_).c_str()); + this->handle_periodic_data_(); + this->buffer_pos_ = 0; // Reset position index for next message + } else if (ld2412::validate_header_footer(CMD_FRAME_FOOTER, &this->buffer_data_[this->buffer_pos_ - 4])) { + ESP_LOGV(TAG, "Handling Ack Data: %s", format_hex_pretty(this->buffer_data_, this->buffer_pos_).c_str()); + if (this->handle_ack_data_()) { + this->buffer_pos_ = 0; // Reset position index for next message + } else { + ESP_LOGV(TAG, "Ack Data incomplete"); + } + } +} + +void LD2412Component::set_config_mode_(bool enable) { + const uint8_t cmd = enable ? CMD_ENABLE_CONF : CMD_DISABLE_CONF; + const uint8_t cmd_value[2] = {0x01, 0x00}; + this->send_command_(cmd, enable ? cmd_value : nullptr, sizeof(cmd_value)); +} + +void LD2412Component::set_bluetooth(bool enable) { + this->set_config_mode_(true); + const uint8_t cmd_value[2] = {enable ? (uint8_t) 0x01 : (uint8_t) 0x00, 0x00}; + this->send_command_(CMD_BLUETOOTH, cmd_value, sizeof(cmd_value)); + this->set_timeout(200, [this]() { this->restart_and_read_all_info(); }); +} + +void LD2412Component::set_distance_resolution(const std::string &state) { + this->set_config_mode_(true); + const uint8_t cmd_value[6] = {find_uint8(DISTANCE_RESOLUTIONS_BY_STR, state), 0x00, 0x00, 0x00, 0x00, 0x00}; + this->send_command_(CMD_SET_DISTANCE_RESOLUTION, cmd_value, sizeof(cmd_value)); + this->set_timeout(200, [this]() { this->restart_and_read_all_info(); }); +} + +void LD2412Component::set_baud_rate(const std::string &state) { + this->set_config_mode_(true); + const uint8_t cmd_value[2] = {find_uint8(BAUD_RATES_BY_STR, state), 0x00}; + this->send_command_(CMD_SET_BAUD_RATE, cmd_value, sizeof(cmd_value)); + this->set_timeout(200, [this]() { this->restart_(); }); +} + +void LD2412Component::query_dynamic_background_correction_() { + this->send_command_(CMD_QUERY_DYNAMIC_BACKGROUND_CORRECTION, nullptr, 0); +} + +void LD2412Component::start_dynamic_background_correction() { + if (this->dynamic_background_correction_active_) { + return; // Already in progress + } +#ifdef USE_BINARY_SENSOR + if (this->dynamic_background_correction_status_binary_sensor_ != nullptr) { + this->dynamic_background_correction_status_binary_sensor_->publish_state(true); + } +#endif + this->dynamic_background_correction_active_ = true; + this->set_config_mode_(true); + this->send_command_(CMD_DYNAMIC_BACKGROUND_CORRECTION, nullptr, 0); + this->set_config_mode_(false); +} + +void LD2412Component::set_engineering_mode(bool enable) { + const uint8_t cmd = enable ? CMD_ENABLE_ENG : CMD_DISABLE_ENG; + this->set_config_mode_(true); + this->send_command_(cmd, nullptr, 0); + this->set_config_mode_(false); +} + +void LD2412Component::factory_reset() { + this->set_config_mode_(true); + this->send_command_(CMD_FACTORY_RESET, nullptr, 0); + this->set_timeout(2000, [this]() { this->restart_and_read_all_info(); }); +} + +void LD2412Component::restart_() { this->send_command_(CMD_RESTART, nullptr, 0); } + +void LD2412Component::query_parameters_() { this->send_command_(CMD_QUERY_BASIC_CONF, nullptr, 0); } + +void LD2412Component::get_version_() { this->send_command_(CMD_QUERY_VERSION, nullptr, 0); } + +void LD2412Component::get_mac_() { + const uint8_t cmd_value[2] = {0x01, 0x00}; + this->send_command_(CMD_QUERY_MAC_ADDRESS, cmd_value, sizeof(cmd_value)); +} + +void LD2412Component::get_distance_resolution_() { this->send_command_(CMD_QUERY_DISTANCE_RESOLUTION, nullptr, 0); } + +void LD2412Component::query_light_control_() { this->send_command_(CMD_QUERY_LIGHT_CONTROL, nullptr, 0); } + +void LD2412Component::set_basic_config() { +#ifdef USE_NUMBER + if (!this->min_distance_gate_number_->has_state() || !this->max_distance_gate_number_->has_state() || + !this->timeout_number_->has_state()) { + return; + } +#endif +#ifdef USE_SELECT + if (!this->out_pin_level_select_->has_state()) { + return; + } +#endif + + uint8_t value[5] = { +#ifdef USE_NUMBER + lowbyte(static_cast(this->min_distance_gate_number_->state)), + lowbyte(static_cast(this->max_distance_gate_number_->state) + 1), + lowbyte(static_cast(this->timeout_number_->state)), + highbyte(static_cast(this->timeout_number_->state)), +#else + 1, TOTAL_GATES, DEFAULT_PRESENCE_TIMEOUT, 0, +#endif +#ifdef USE_SELECT + find_uint8(OUT_PIN_LEVELS_BY_STR, this->out_pin_level_select_->state), +#else + 0x01, // Default value if not using select +#endif + }; + this->set_config_mode_(true); + this->send_command_(CMD_BASIC_CONF, value, sizeof(value)); + this->set_config_mode_(false); +} + +#ifdef USE_NUMBER +void LD2412Component::set_gate_threshold() { + if (this->gate_move_threshold_numbers_.empty() && this->gate_still_threshold_numbers_.empty()) { + return; // No gate threshold numbers set; nothing to do here + } + uint8_t value[TOTAL_GATES] = {0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0}; + this->set_config_mode_(true); + if (!this->gate_move_threshold_numbers_.empty()) { + for (size_t i = 0; i < this->gate_move_threshold_numbers_.size(); i++) { + value[i] = lowbyte(static_cast(this->gate_move_threshold_numbers_[i]->state)); + } + this->send_command_(CMD_MOTION_GATE_SENS, value, sizeof(value)); + } + if (!this->gate_still_threshold_numbers_.empty()) { + for (size_t i = 0; i < this->gate_still_threshold_numbers_.size(); i++) { + value[i] = lowbyte(static_cast(this->gate_still_threshold_numbers_[i]->state)); + } + this->send_command_(CMD_STATIC_GATE_SENS, value, sizeof(value)); + } + this->set_config_mode_(false); +} + +void LD2412Component::get_gate_threshold() { + this->send_command_(CMD_QUERY_MOTION_GATE_SENS, nullptr, 0); + this->send_command_(CMD_QUERY_STATIC_GATE_SENS, nullptr, 0); +} + +void LD2412Component::set_gate_still_threshold_number(uint8_t gate, number::Number *n) { + this->gate_still_threshold_numbers_[gate] = n; +} + +void LD2412Component::set_gate_move_threshold_number(uint8_t gate, number::Number *n) { + this->gate_move_threshold_numbers_[gate] = n; +} +#endif + +void LD2412Component::set_light_out_control() { +#ifdef USE_NUMBER + if (this->light_threshold_number_ != nullptr && this->light_threshold_number_->has_state()) { + this->light_threshold_ = static_cast(this->light_threshold_number_->state); + } +#endif +#ifdef USE_SELECT + if (this->light_function_select_ != nullptr && this->light_function_select_->has_state()) { + this->light_function_ = find_uint8(LIGHT_FUNCTIONS_BY_STR, this->light_function_select_->state); + } +#endif + uint8_t value[2] = {this->light_function_, this->light_threshold_}; + this->set_config_mode_(true); + this->send_command_(CMD_SET_LIGHT_CONTROL, value, sizeof(value)); + this->query_light_control_(); + this->set_timeout(200, [this]() { this->restart_and_read_all_info(); }); +} + +#ifdef USE_SENSOR +// These could leak memory, but they are only set once prior to 'setup()' and should never be used again. +void LD2412Component::set_gate_move_sensor(uint8_t gate, sensor::Sensor *s) { + this->gate_move_sensors_[gate] = new SensorWithDedup(s); +} +void LD2412Component::set_gate_still_sensor(uint8_t gate, sensor::Sensor *s) { + this->gate_still_sensors_[gate] = new SensorWithDedup(s); +} +#endif + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/ld2412.h b/esphome/components/ld2412/ld2412.h new file mode 100644 index 0000000000..41f96ab301 --- /dev/null +++ b/esphome/components/ld2412/ld2412.h @@ -0,0 +1,141 @@ +#pragma once +#include "esphome/core/defines.h" +#include "esphome/core/component.h" +#ifdef USE_BINARY_SENSOR +#include "esphome/components/binary_sensor/binary_sensor.h" +#endif +#ifdef USE_SENSOR +#include "esphome/components/sensor/sensor.h" +#endif +#ifdef USE_NUMBER +#include "esphome/components/number/number.h" +#endif +#ifdef USE_SWITCH +#include "esphome/components/switch/switch.h" +#endif +#ifdef USE_BUTTON +#include "esphome/components/button/button.h" +#endif +#ifdef USE_SELECT +#include "esphome/components/select/select.h" +#endif +#ifdef USE_TEXT_SENSOR +#include "esphome/components/text_sensor/text_sensor.h" +#endif +#include "esphome/components/ld24xx/ld24xx.h" +#include "esphome/components/uart/uart.h" +#include "esphome/core/automation.h" +#include "esphome/core/helpers.h" + +#include + +namespace esphome { +namespace ld2412 { + +using namespace ld24xx; + +static constexpr uint8_t MAX_LINE_LENGTH = 54; // Max characters for serial buffer +static constexpr uint8_t TOTAL_GATES = 14; // Total number of gates supported by the LD2412 + +class LD2412Component : public Component, public uart::UARTDevice { +#ifdef USE_BINARY_SENSOR + SUB_BINARY_SENSOR(dynamic_background_correction_status) + SUB_BINARY_SENSOR(moving_target) + SUB_BINARY_SENSOR(still_target) + SUB_BINARY_SENSOR(target) +#endif +#ifdef USE_SENSOR + SUB_SENSOR_WITH_DEDUP(light, uint8_t) + SUB_SENSOR_WITH_DEDUP(detection_distance, int) + SUB_SENSOR_WITH_DEDUP(moving_target_distance, int) + SUB_SENSOR_WITH_DEDUP(moving_target_energy, uint8_t) + SUB_SENSOR_WITH_DEDUP(still_target_distance, int) + SUB_SENSOR_WITH_DEDUP(still_target_energy, uint8_t) +#endif +#ifdef USE_TEXT_SENSOR + SUB_TEXT_SENSOR(mac) + SUB_TEXT_SENSOR(version) +#endif +#ifdef USE_NUMBER + SUB_NUMBER(light_threshold) + SUB_NUMBER(max_distance_gate) + SUB_NUMBER(min_distance_gate) + SUB_NUMBER(timeout) +#endif +#ifdef USE_SELECT + SUB_SELECT(baud_rate) + SUB_SELECT(distance_resolution) + SUB_SELECT(light_function) + SUB_SELECT(out_pin_level) +#endif +#ifdef USE_SWITCH + SUB_SWITCH(bluetooth) + SUB_SWITCH(engineering_mode) +#endif +#ifdef USE_BUTTON + SUB_BUTTON(factory_reset) + SUB_BUTTON(query) + SUB_BUTTON(restart) + SUB_BUTTON(start_dynamic_background_correction) +#endif + + public: + void setup() override; + void dump_config() override; + void loop() override; + void set_light_out_control(); + void set_basic_config(); +#ifdef USE_NUMBER + void set_gate_move_threshold_number(uint8_t gate, number::Number *n); + void set_gate_still_threshold_number(uint8_t gate, number::Number *n); + void set_gate_threshold(); + void get_gate_threshold(); +#endif +#ifdef USE_SENSOR + void set_gate_move_sensor(uint8_t gate, sensor::Sensor *s); + void set_gate_still_sensor(uint8_t gate, sensor::Sensor *s); +#endif + void set_engineering_mode(bool enable); + void read_all_info(); + void restart_and_read_all_info(); + void set_bluetooth(bool enable); + void set_distance_resolution(const std::string &state); + void set_baud_rate(const std::string &state); + void factory_reset(); + void start_dynamic_background_correction(); + + protected: + void send_command_(uint8_t command_str, const uint8_t *command_value, uint8_t command_value_len); + void set_config_mode_(bool enable); + void handle_periodic_data_(); + bool handle_ack_data_(); + void readline_(int readch); + void query_parameters_(); + void get_version_(); + void get_mac_(); + void get_distance_resolution_(); + void query_light_control_(); + void restart_(); + void query_dynamic_background_correction_(); + + uint8_t light_function_ = 0; + uint8_t light_threshold_ = 0; + uint8_t out_pin_level_ = 0; + uint8_t buffer_pos_ = 0; // where to resume processing/populating buffer + uint8_t buffer_data_[MAX_LINE_LENGTH]; + uint8_t mac_address_[6] = {0, 0, 0, 0, 0, 0}; + uint8_t version_[6] = {0, 0, 0, 0, 0, 0}; + bool bluetooth_on_{false}; + bool dynamic_background_correction_active_{false}; +#ifdef USE_NUMBER + std::array gate_move_threshold_numbers_{}; + std::array gate_still_threshold_numbers_{}; +#endif +#ifdef USE_SENSOR + std::array *, TOTAL_GATES> gate_move_sensors_{}; + std::array *, TOTAL_GATES> gate_still_sensors_{}; +#endif +}; + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/number/__init__.py b/esphome/components/ld2412/number/__init__.py new file mode 100644 index 0000000000..5b0d6d8749 --- /dev/null +++ b/esphome/components/ld2412/number/__init__.py @@ -0,0 +1,126 @@ +import esphome.codegen as cg +from esphome.components import number +import esphome.config_validation as cv +from esphome.const import ( + CONF_ID, + CONF_MOVE_THRESHOLD, + CONF_STILL_THRESHOLD, + CONF_TIMEOUT, + DEVICE_CLASS_DISTANCE, + DEVICE_CLASS_ILLUMINANCE, + DEVICE_CLASS_SIGNAL_STRENGTH, + ENTITY_CATEGORY_CONFIG, + ICON_LIGHTBULB, + ICON_MOTION_SENSOR, + ICON_TIMELAPSE, + UNIT_PERCENT, + UNIT_SECOND, +) + +from .. import CONF_LD2412_ID, LD2412_ns, LD2412Component + +GateThresholdNumber = LD2412_ns.class_("GateThresholdNumber", number.Number) +LightThresholdNumber = LD2412_ns.class_("LightThresholdNumber", number.Number) +MaxDistanceTimeoutNumber = LD2412_ns.class_("MaxDistanceTimeoutNumber", number.Number) + +CONF_LIGHT_THRESHOLD = "light_threshold" +CONF_MAX_DISTANCE_GATE = "max_distance_gate" +CONF_MIN_DISTANCE_GATE = "min_distance_gate" + +TIMEOUT_GROUP = "timeout" + +CONFIG_SCHEMA = cv.Schema( + { + cv.GenerateID(CONF_LD2412_ID): cv.use_id(LD2412Component), + cv.Optional(CONF_LIGHT_THRESHOLD): number.number_schema( + LightThresholdNumber, + device_class=DEVICE_CLASS_ILLUMINANCE, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_LIGHTBULB, + ), + cv.Optional(CONF_MAX_DISTANCE_GATE): number.number_schema( + MaxDistanceTimeoutNumber, + device_class=DEVICE_CLASS_DISTANCE, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_MOTION_SENSOR, + ), + cv.Optional(CONF_MIN_DISTANCE_GATE): number.number_schema( + MaxDistanceTimeoutNumber, + device_class=DEVICE_CLASS_DISTANCE, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_MOTION_SENSOR, + ), + cv.Optional(CONF_TIMEOUT): number.number_schema( + MaxDistanceTimeoutNumber, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_TIMELAPSE, + unit_of_measurement=UNIT_SECOND, + ), + } +) + +CONFIG_SCHEMA = CONFIG_SCHEMA.extend( + { + cv.Optional(f"gate_{x}"): ( + { + cv.Required(CONF_MOVE_THRESHOLD): number.number_schema( + GateThresholdNumber, + device_class=DEVICE_CLASS_SIGNAL_STRENGTH, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_MOTION_SENSOR, + unit_of_measurement=UNIT_PERCENT, + ), + cv.Required(CONF_STILL_THRESHOLD): number.number_schema( + GateThresholdNumber, + device_class=DEVICE_CLASS_SIGNAL_STRENGTH, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_MOTION_SENSOR, + unit_of_measurement=UNIT_PERCENT, + ), + } + ) + for x in range(14) + } +) + + +async def to_code(config): + LD2412_component = await cg.get_variable(config[CONF_LD2412_ID]) + if light_threshold_config := config.get(CONF_LIGHT_THRESHOLD): + n = await number.new_number( + light_threshold_config, min_value=0, max_value=255, step=1 + ) + await cg.register_parented(n, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_light_threshold_number(n)) + if max_distance_gate_config := config.get(CONF_MAX_DISTANCE_GATE): + n = await number.new_number( + max_distance_gate_config, min_value=2, max_value=13, step=1 + ) + await cg.register_parented(n, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_max_distance_gate_number(n)) + if min_distance_gate_config := config.get(CONF_MIN_DISTANCE_GATE): + n = await number.new_number( + min_distance_gate_config, min_value=1, max_value=12, step=1 + ) + await cg.register_parented(n, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_min_distance_gate_number(n)) + for x in range(14): + if gate_conf := config.get(f"gate_{x}"): + move_config = gate_conf[CONF_MOVE_THRESHOLD] + n = cg.new_Pvariable(move_config[CONF_ID], x) + await number.register_number( + n, move_config, min_value=0, max_value=100, step=1 + ) + await cg.register_parented(n, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_gate_move_threshold_number(x, n)) + still_config = gate_conf[CONF_STILL_THRESHOLD] + n = cg.new_Pvariable(still_config[CONF_ID], x) + await number.register_number( + n, still_config, min_value=0, max_value=100, step=1 + ) + await cg.register_parented(n, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_gate_still_threshold_number(x, n)) + if timeout_config := config.get(CONF_TIMEOUT): + n = await number.new_number(timeout_config, min_value=0, max_value=900, step=1) + await cg.register_parented(n, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_timeout_number(n)) diff --git a/esphome/components/ld2412/number/gate_threshold_number.cpp b/esphome/components/ld2412/number/gate_threshold_number.cpp new file mode 100644 index 0000000000..47f8cd9107 --- /dev/null +++ b/esphome/components/ld2412/number/gate_threshold_number.cpp @@ -0,0 +1,14 @@ +#include "gate_threshold_number.h" + +namespace esphome { +namespace ld2412 { + +GateThresholdNumber::GateThresholdNumber(uint8_t gate) : gate_(gate) {} + +void GateThresholdNumber::control(float value) { + this->publish_state(value); + this->parent_->set_gate_threshold(); +} + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/number/gate_threshold_number.h b/esphome/components/ld2412/number/gate_threshold_number.h new file mode 100644 index 0000000000..61d9945a0a --- /dev/null +++ b/esphome/components/ld2412/number/gate_threshold_number.h @@ -0,0 +1,19 @@ +#pragma once + +#include "esphome/components/number/number.h" +#include "../ld2412.h" + +namespace esphome { +namespace ld2412 { + +class GateThresholdNumber : public number::Number, public Parented { + public: + GateThresholdNumber(uint8_t gate); + + protected: + uint8_t gate_; + void control(float value) override; +}; + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/number/light_threshold_number.cpp b/esphome/components/ld2412/number/light_threshold_number.cpp new file mode 100644 index 0000000000..5dab1716bf --- /dev/null +++ b/esphome/components/ld2412/number/light_threshold_number.cpp @@ -0,0 +1,12 @@ +#include "light_threshold_number.h" + +namespace esphome { +namespace ld2412 { + +void LightThresholdNumber::control(float value) { + this->publish_state(value); + this->parent_->set_light_out_control(); +} + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/number/light_threshold_number.h b/esphome/components/ld2412/number/light_threshold_number.h new file mode 100644 index 0000000000..d8727d3c98 --- /dev/null +++ b/esphome/components/ld2412/number/light_threshold_number.h @@ -0,0 +1,18 @@ +#pragma once + +#include "esphome/components/number/number.h" +#include "../ld2412.h" + +namespace esphome { +namespace ld2412 { + +class LightThresholdNumber : public number::Number, public Parented { + public: + LightThresholdNumber() = default; + + protected: + void control(float value) override; +}; + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/number/max_distance_timeout_number.cpp b/esphome/components/ld2412/number/max_distance_timeout_number.cpp new file mode 100644 index 0000000000..1c6471bc72 --- /dev/null +++ b/esphome/components/ld2412/number/max_distance_timeout_number.cpp @@ -0,0 +1,12 @@ +#include "max_distance_timeout_number.h" + +namespace esphome { +namespace ld2412 { + +void MaxDistanceTimeoutNumber::control(float value) { + this->publish_state(value); + this->parent_->set_basic_config(); +} + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/number/max_distance_timeout_number.h b/esphome/components/ld2412/number/max_distance_timeout_number.h new file mode 100644 index 0000000000..af0dcf68c5 --- /dev/null +++ b/esphome/components/ld2412/number/max_distance_timeout_number.h @@ -0,0 +1,18 @@ +#pragma once + +#include "esphome/components/number/number.h" +#include "../ld2412.h" + +namespace esphome { +namespace ld2412 { + +class MaxDistanceTimeoutNumber : public number::Number, public Parented { + public: + MaxDistanceTimeoutNumber() = default; + + protected: + void control(float value) override; +}; + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/select/__init__.py b/esphome/components/ld2412/select/__init__.py new file mode 100644 index 0000000000..d71ce460d9 --- /dev/null +++ b/esphome/components/ld2412/select/__init__.py @@ -0,0 +1,82 @@ +import esphome.codegen as cg +from esphome.components import select +import esphome.config_validation as cv +from esphome.const import ( + CONF_BAUD_RATE, + ENTITY_CATEGORY_CONFIG, + ICON_LIGHTBULB, + ICON_RULER, + ICON_SCALE, + ICON_THERMOMETER, +) + +from .. import CONF_LD2412_ID, LD2412_ns, LD2412Component + +BaudRateSelect = LD2412_ns.class_("BaudRateSelect", select.Select) +DistanceResolutionSelect = LD2412_ns.class_("DistanceResolutionSelect", select.Select) +LightOutControlSelect = LD2412_ns.class_("LightOutControlSelect", select.Select) + +CONF_DISTANCE_RESOLUTION = "distance_resolution" +CONF_LIGHT_FUNCTION = "light_function" +CONF_OUT_PIN_LEVEL = "out_pin_level" + + +CONFIG_SCHEMA = { + cv.GenerateID(CONF_LD2412_ID): cv.use_id(LD2412Component), + cv.Optional(CONF_BAUD_RATE): select.select_schema( + BaudRateSelect, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_THERMOMETER, + ), + cv.Optional(CONF_DISTANCE_RESOLUTION): select.select_schema( + DistanceResolutionSelect, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_RULER, + ), + cv.Optional(CONF_LIGHT_FUNCTION): select.select_schema( + LightOutControlSelect, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_LIGHTBULB, + ), + cv.Optional(CONF_OUT_PIN_LEVEL): select.select_schema( + LightOutControlSelect, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_SCALE, + ), +} + + +async def to_code(config): + LD2412_component = await cg.get_variable(config[CONF_LD2412_ID]) + if baud_rate_config := config.get(CONF_BAUD_RATE): + s = await select.new_select( + baud_rate_config, + options=[ + "9600", + "19200", + "38400", + "57600", + "115200", + "230400", + "256000", + "460800", + ], + ) + await cg.register_parented(s, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_baud_rate_select(s)) + if distance_resolution_config := config.get(CONF_DISTANCE_RESOLUTION): + s = await select.new_select( + distance_resolution_config, options=["0.2m", "0.5m", "0.75m"] + ) + await cg.register_parented(s, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_distance_resolution_select(s)) + if light_function_config := config.get(CONF_LIGHT_FUNCTION): + s = await select.new_select( + light_function_config, options=["off", "below", "above"] + ) + await cg.register_parented(s, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_light_function_select(s)) + if out_pin_level_config := config.get(CONF_OUT_PIN_LEVEL): + s = await select.new_select(out_pin_level_config, options=["low", "high"]) + await cg.register_parented(s, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_out_pin_level_select(s)) diff --git a/esphome/components/ld2412/select/baud_rate_select.cpp b/esphome/components/ld2412/select/baud_rate_select.cpp new file mode 100644 index 0000000000..2291a81896 --- /dev/null +++ b/esphome/components/ld2412/select/baud_rate_select.cpp @@ -0,0 +1,12 @@ +#include "baud_rate_select.h" + +namespace esphome { +namespace ld2412 { + +void BaudRateSelect::control(const std::string &value) { + this->publish_state(value); + this->parent_->set_baud_rate(state); +} + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/select/baud_rate_select.h b/esphome/components/ld2412/select/baud_rate_select.h new file mode 100644 index 0000000000..2ae33551fb --- /dev/null +++ b/esphome/components/ld2412/select/baud_rate_select.h @@ -0,0 +1,18 @@ +#pragma once + +#include "esphome/components/select/select.h" +#include "../ld2412.h" + +namespace esphome { +namespace ld2412 { + +class BaudRateSelect : public select::Select, public Parented { + public: + BaudRateSelect() = default; + + protected: + void control(const std::string &value) override; +}; + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/select/distance_resolution_select.cpp b/esphome/components/ld2412/select/distance_resolution_select.cpp new file mode 100644 index 0000000000..a282215fbd --- /dev/null +++ b/esphome/components/ld2412/select/distance_resolution_select.cpp @@ -0,0 +1,12 @@ +#include "distance_resolution_select.h" + +namespace esphome { +namespace ld2412 { + +void DistanceResolutionSelect::control(const std::string &value) { + this->publish_state(value); + this->parent_->set_distance_resolution(state); +} + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/select/distance_resolution_select.h b/esphome/components/ld2412/select/distance_resolution_select.h new file mode 100644 index 0000000000..0658f5d1a7 --- /dev/null +++ b/esphome/components/ld2412/select/distance_resolution_select.h @@ -0,0 +1,18 @@ +#pragma once + +#include "esphome/components/select/select.h" +#include "../ld2412.h" + +namespace esphome { +namespace ld2412 { + +class DistanceResolutionSelect : public select::Select, public Parented { + public: + DistanceResolutionSelect() = default; + + protected: + void control(const std::string &value) override; +}; + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/select/light_out_control_select.cpp b/esphome/components/ld2412/select/light_out_control_select.cpp new file mode 100644 index 0000000000..c331729d40 --- /dev/null +++ b/esphome/components/ld2412/select/light_out_control_select.cpp @@ -0,0 +1,12 @@ +#include "light_out_control_select.h" + +namespace esphome { +namespace ld2412 { + +void LightOutControlSelect::control(const std::string &value) { + this->publish_state(value); + this->parent_->set_light_out_control(); +} + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/select/light_out_control_select.h b/esphome/components/ld2412/select/light_out_control_select.h new file mode 100644 index 0000000000..a71bab1e14 --- /dev/null +++ b/esphome/components/ld2412/select/light_out_control_select.h @@ -0,0 +1,18 @@ +#pragma once + +#include "esphome/components/select/select.h" +#include "../ld2412.h" + +namespace esphome { +namespace ld2412 { + +class LightOutControlSelect : public select::Select, public Parented { + public: + LightOutControlSelect() = default; + + protected: + void control(const std::string &value) override; +}; + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/sensor.py b/esphome/components/ld2412/sensor.py new file mode 100644 index 0000000000..abb823faad --- /dev/null +++ b/esphome/components/ld2412/sensor.py @@ -0,0 +1,124 @@ +import esphome.codegen as cg +from esphome.components import sensor +import esphome.config_validation as cv +from esphome.const import ( + CONF_LIGHT, + CONF_MOVING_DISTANCE, + DEVICE_CLASS_DISTANCE, + DEVICE_CLASS_ILLUMINANCE, + ENTITY_CATEGORY_DIAGNOSTIC, + ICON_FLASH, + ICON_LIGHTBULB, + ICON_MOTION_SENSOR, + ICON_SIGNAL, + UNIT_CENTIMETER, + UNIT_EMPTY, + UNIT_PERCENT, +) + +from . import CONF_LD2412_ID, LD2412Component + +DEPENDENCIES = ["ld2412"] + +CONF_DETECTION_DISTANCE = "detection_distance" +CONF_MOVE_ENERGY = "move_energy" +CONF_MOVING_ENERGY = "moving_energy" +CONF_STILL_DISTANCE = "still_distance" +CONF_STILL_ENERGY = "still_energy" + +CONFIG_SCHEMA = cv.Schema( + { + cv.GenerateID(CONF_LD2412_ID): cv.use_id(LD2412Component), + cv.Optional(CONF_DETECTION_DISTANCE): sensor.sensor_schema( + device_class=DEVICE_CLASS_DISTANCE, + filters=[{"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}], + icon=ICON_SIGNAL, + unit_of_measurement=UNIT_CENTIMETER, + ), + cv.Optional(CONF_LIGHT): sensor.sensor_schema( + device_class=DEVICE_CLASS_ILLUMINANCE, + entity_category=ENTITY_CATEGORY_DIAGNOSTIC, + filters=[{"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}], + icon=ICON_LIGHTBULB, + unit_of_measurement=UNIT_EMPTY, # No standard unit for this light sensor + ), + cv.Optional(CONF_MOVING_DISTANCE): sensor.sensor_schema( + device_class=DEVICE_CLASS_DISTANCE, + filters=[{"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}], + icon=ICON_SIGNAL, + unit_of_measurement=UNIT_CENTIMETER, + ), + cv.Optional(CONF_MOVING_ENERGY): sensor.sensor_schema( + filters=[{"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}], + icon=ICON_MOTION_SENSOR, + unit_of_measurement=UNIT_PERCENT, + ), + cv.Optional(CONF_STILL_DISTANCE): sensor.sensor_schema( + device_class=DEVICE_CLASS_DISTANCE, + filters=[{"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}], + icon=ICON_SIGNAL, + unit_of_measurement=UNIT_CENTIMETER, + ), + cv.Optional(CONF_STILL_ENERGY): sensor.sensor_schema( + filters=[{"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}], + icon=ICON_FLASH, + unit_of_measurement=UNIT_PERCENT, + ), + } +) + +CONFIG_SCHEMA = CONFIG_SCHEMA.extend( + { + cv.Optional(f"gate_{x}"): ( + { + cv.Optional(CONF_MOVE_ENERGY): sensor.sensor_schema( + entity_category=ENTITY_CATEGORY_DIAGNOSTIC, + filters=[ + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)} + ], + icon=ICON_MOTION_SENSOR, + unit_of_measurement=UNIT_PERCENT, + ), + cv.Optional(CONF_STILL_ENERGY): sensor.sensor_schema( + entity_category=ENTITY_CATEGORY_DIAGNOSTIC, + filters=[ + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)} + ], + icon=ICON_FLASH, + unit_of_measurement=UNIT_PERCENT, + ), + } + ) + for x in range(14) + } +) + + +async def to_code(config): + LD2412_component = await cg.get_variable(config[CONF_LD2412_ID]) + if detection_distance_config := config.get(CONF_DETECTION_DISTANCE): + sens = await sensor.new_sensor(detection_distance_config) + cg.add(LD2412_component.set_detection_distance_sensor(sens)) + if light_config := config.get(CONF_LIGHT): + sens = await sensor.new_sensor(light_config) + cg.add(LD2412_component.set_light_sensor(sens)) + if moving_distance_config := config.get(CONF_MOVING_DISTANCE): + sens = await sensor.new_sensor(moving_distance_config) + cg.add(LD2412_component.set_moving_target_distance_sensor(sens)) + if moving_energy_config := config.get(CONF_MOVING_ENERGY): + sens = await sensor.new_sensor(moving_energy_config) + cg.add(LD2412_component.set_moving_target_energy_sensor(sens)) + if still_distance_config := config.get(CONF_STILL_DISTANCE): + sens = await sensor.new_sensor(still_distance_config) + cg.add(LD2412_component.set_still_target_distance_sensor(sens)) + if still_energy_config := config.get(CONF_STILL_ENERGY): + sens = await sensor.new_sensor(still_energy_config) + cg.add(LD2412_component.set_still_target_energy_sensor(sens)) + for x in range(14): + if gate_conf := config.get(f"gate_{x}"): + if move_config := gate_conf.get(CONF_MOVE_ENERGY): + sens = await sensor.new_sensor(move_config) + cg.add(LD2412_component.set_gate_move_sensor(x, sens)) + if still_config := gate_conf.get(CONF_STILL_ENERGY): + sens = await sensor.new_sensor(still_config) + cg.add(LD2412_component.set_gate_still_sensor(x, sens)) diff --git a/esphome/components/ld2412/switch/__init__.py b/esphome/components/ld2412/switch/__init__.py new file mode 100644 index 0000000000..df994687ec --- /dev/null +++ b/esphome/components/ld2412/switch/__init__.py @@ -0,0 +1,45 @@ +import esphome.codegen as cg +from esphome.components import switch +import esphome.config_validation as cv +from esphome.const import ( + CONF_BLUETOOTH, + DEVICE_CLASS_SWITCH, + ENTITY_CATEGORY_CONFIG, + ICON_BLUETOOTH, + ICON_PULSE, +) + +from .. import CONF_LD2412_ID, LD2412_ns, LD2412Component + +BluetoothSwitch = LD2412_ns.class_("BluetoothSwitch", switch.Switch) +EngineeringModeSwitch = LD2412_ns.class_("EngineeringModeSwitch", switch.Switch) + +CONF_ENGINEERING_MODE = "engineering_mode" + +CONFIG_SCHEMA = { + cv.GenerateID(CONF_LD2412_ID): cv.use_id(LD2412Component), + cv.Optional(CONF_BLUETOOTH): switch.switch_schema( + BluetoothSwitch, + device_class=DEVICE_CLASS_SWITCH, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_BLUETOOTH, + ), + cv.Optional(CONF_ENGINEERING_MODE): switch.switch_schema( + EngineeringModeSwitch, + device_class=DEVICE_CLASS_SWITCH, + entity_category=ENTITY_CATEGORY_CONFIG, + icon=ICON_PULSE, + ), +} + + +async def to_code(config): + LD2412_component = await cg.get_variable(config[CONF_LD2412_ID]) + if bluetooth_config := config.get(CONF_BLUETOOTH): + s = await switch.new_switch(bluetooth_config) + await cg.register_parented(s, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_bluetooth_switch(s)) + if engineering_mode_config := config.get(CONF_ENGINEERING_MODE): + s = await switch.new_switch(engineering_mode_config) + await cg.register_parented(s, config[CONF_LD2412_ID]) + cg.add(LD2412_component.set_engineering_mode_switch(s)) diff --git a/esphome/components/ld2412/switch/bluetooth_switch.cpp b/esphome/components/ld2412/switch/bluetooth_switch.cpp new file mode 100644 index 0000000000..14387aa276 --- /dev/null +++ b/esphome/components/ld2412/switch/bluetooth_switch.cpp @@ -0,0 +1,12 @@ +#include "bluetooth_switch.h" + +namespace esphome { +namespace ld2412 { + +void BluetoothSwitch::write_state(bool state) { + this->publish_state(state); + this->parent_->set_bluetooth(state); +} + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/switch/bluetooth_switch.h b/esphome/components/ld2412/switch/bluetooth_switch.h new file mode 100644 index 0000000000..730d338d87 --- /dev/null +++ b/esphome/components/ld2412/switch/bluetooth_switch.h @@ -0,0 +1,18 @@ +#pragma once + +#include "esphome/components/switch/switch.h" +#include "../ld2412.h" + +namespace esphome { +namespace ld2412 { + +class BluetoothSwitch : public switch_::Switch, public Parented { + public: + BluetoothSwitch() = default; + + protected: + void write_state(bool state) override; +}; + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/switch/engineering_mode_switch.cpp b/esphome/components/ld2412/switch/engineering_mode_switch.cpp new file mode 100644 index 0000000000..29ca0c22a8 --- /dev/null +++ b/esphome/components/ld2412/switch/engineering_mode_switch.cpp @@ -0,0 +1,12 @@ +#include "engineering_mode_switch.h" + +namespace esphome { +namespace ld2412 { + +void EngineeringModeSwitch::write_state(bool state) { + this->publish_state(state); + this->parent_->set_engineering_mode(state); +} + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/switch/engineering_mode_switch.h b/esphome/components/ld2412/switch/engineering_mode_switch.h new file mode 100644 index 0000000000..aaa404c673 --- /dev/null +++ b/esphome/components/ld2412/switch/engineering_mode_switch.h @@ -0,0 +1,18 @@ +#pragma once + +#include "esphome/components/switch/switch.h" +#include "../ld2412.h" + +namespace esphome { +namespace ld2412 { + +class EngineeringModeSwitch : public switch_::Switch, public Parented { + public: + EngineeringModeSwitch() = default; + + protected: + void write_state(bool state) override; +}; + +} // namespace ld2412 +} // namespace esphome diff --git a/esphome/components/ld2412/text_sensor.py b/esphome/components/ld2412/text_sensor.py new file mode 100644 index 0000000000..1074494933 --- /dev/null +++ b/esphome/components/ld2412/text_sensor.py @@ -0,0 +1,34 @@ +import esphome.codegen as cg +from esphome.components import text_sensor +import esphome.config_validation as cv +from esphome.const import ( + CONF_MAC_ADDRESS, + CONF_VERSION, + ENTITY_CATEGORY_DIAGNOSTIC, + ICON_BLUETOOTH, + ICON_CHIP, +) + +from . import CONF_LD2412_ID, LD2412Component + +DEPENDENCIES = ["ld2412"] + +CONFIG_SCHEMA = { + cv.GenerateID(CONF_LD2412_ID): cv.use_id(LD2412Component), + cv.Optional(CONF_VERSION): text_sensor.text_sensor_schema( + entity_category=ENTITY_CATEGORY_DIAGNOSTIC, icon=ICON_CHIP + ), + cv.Optional(CONF_MAC_ADDRESS): text_sensor.text_sensor_schema( + entity_category=ENTITY_CATEGORY_DIAGNOSTIC, icon=ICON_BLUETOOTH + ), +} + + +async def to_code(config): + LD2412_component = await cg.get_variable(config[CONF_LD2412_ID]) + if version_config := config.get(CONF_VERSION): + sens = await text_sensor.new_text_sensor(version_config) + cg.add(LD2412_component.set_version_text_sensor(sens)) + if mac_address_config := config.get(CONF_MAC_ADDRESS): + sens = await text_sensor.new_text_sensor(mac_address_config) + cg.add(LD2412_component.set_mac_text_sensor(sens)) diff --git a/esphome/components/ld2420/ld2420.cpp b/esphome/components/ld2420/ld2420.cpp index 0baff368c8..3842098c44 100644 --- a/esphome/components/ld2420/ld2420.cpp +++ b/esphome/components/ld2420/ld2420.cpp @@ -213,7 +213,6 @@ void LD2420Component::dump_config() { } void LD2420Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (this->set_config_mode(true) == LD2420_ERROR_TIMEOUT) { ESP_LOGE(TAG, ESP_LOG_MSG_COMM_FAIL); this->mark_failed(); diff --git a/esphome/components/ld2420/text_sensor/text_sensor.cpp b/esphome/components/ld2420/text_sensor/ld2420_text_sensor.cpp similarity index 91% rename from esphome/components/ld2420/text_sensor/text_sensor.cpp rename to esphome/components/ld2420/text_sensor/ld2420_text_sensor.cpp index 73af3b3660..f647a36936 100644 --- a/esphome/components/ld2420/text_sensor/text_sensor.cpp +++ b/esphome/components/ld2420/text_sensor/ld2420_text_sensor.cpp @@ -1,4 +1,4 @@ -#include "text_sensor.h" +#include "ld2420_text_sensor.h" #include "esphome/core/helpers.h" #include "esphome/core/log.h" diff --git a/esphome/components/ld2420/text_sensor/text_sensor.h b/esphome/components/ld2420/text_sensor/ld2420_text_sensor.h similarity index 100% rename from esphome/components/ld2420/text_sensor/text_sensor.h rename to esphome/components/ld2420/text_sensor/ld2420_text_sensor.h diff --git a/esphome/components/ld2450/__init__.py b/esphome/components/ld2450/__init__.py index 442fdaa125..bd6d697c90 100644 --- a/esphome/components/ld2450/__init__.py +++ b/esphome/components/ld2450/__init__.py @@ -3,6 +3,7 @@ from esphome.components import uart import esphome.config_validation as cv from esphome.const import CONF_ID, CONF_THROTTLE +AUTO_LOAD = ["ld24xx"] DEPENDENCIES = ["uart"] CODEOWNERS = ["@hareeshmu"] MULTI_CONF = True @@ -16,9 +17,8 @@ CONFIG_SCHEMA = cv.All( cv.Schema( { cv.GenerateID(): cv.declare_id(LD2450Component), - cv.Optional(CONF_THROTTLE, default="1000ms"): cv.All( - cv.positive_time_period_milliseconds, - cv.Range(min=cv.TimePeriod(milliseconds=1)), + cv.Optional(CONF_THROTTLE): cv.invalid( + f"{CONF_THROTTLE} has been removed; use per-sensor filters, instead" ), } ) @@ -45,4 +45,3 @@ async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) await uart.register_uart_device(var, config) - cg.add(var.set_throttle(config[CONF_THROTTLE])) diff --git a/esphome/components/ld2450/binary_sensor.py b/esphome/components/ld2450/binary_sensor.py index d0082ac21a..37f722b0fa 100644 --- a/esphome/components/ld2450/binary_sensor.py +++ b/esphome/components/ld2450/binary_sensor.py @@ -21,14 +21,17 @@ CONFIG_SCHEMA = { cv.GenerateID(CONF_LD2450_ID): cv.use_id(LD2450Component), cv.Optional(CONF_HAS_TARGET): binary_sensor.binary_sensor_schema( device_class=DEVICE_CLASS_OCCUPANCY, + filters=[{"settle": cv.TimePeriod(milliseconds=1000)}], icon=ICON_SHIELD_ACCOUNT, ), cv.Optional(CONF_HAS_MOVING_TARGET): binary_sensor.binary_sensor_schema( device_class=DEVICE_CLASS_MOTION, + filters=[{"settle": cv.TimePeriod(milliseconds=1000)}], icon=ICON_TARGET_ACCOUNT, ), cv.Optional(CONF_HAS_STILL_TARGET): binary_sensor.binary_sensor_schema( device_class=DEVICE_CLASS_OCCUPANCY, + filters=[{"settle": cv.TimePeriod(milliseconds=1000)}], icon=ICON_MEDITATION, ), } diff --git a/esphome/components/ld2450/ld2450.cpp b/esphome/components/ld2450/ld2450.cpp index 8f3b3a3f21..f30752e5a2 100644 --- a/esphome/components/ld2450/ld2450.cpp +++ b/esphome/components/ld2450/ld2450.cpp @@ -1,6 +1,5 @@ #include "ld2450.h" -#include -#include + #ifdef USE_NUMBER #include "esphome/components/number/number.h" #endif @@ -11,8 +10,8 @@ #include "esphome/core/component.h" #include "esphome/core/helpers.h" -#define highbyte(val) (uint8_t)((val) >> 8) -#define lowbyte(val) (uint8_t)((val) &0xff) +#include +#include namespace esphome { namespace ld2450 { @@ -170,28 +169,22 @@ static inline int16_t hex_to_signed_int(const uint8_t *buffer, uint8_t offset) { } static inline float calculate_angle(float base, float hypotenuse) { - if (base < 0.0 || hypotenuse <= 0.0) { - return 0.0; + if (base < 0.0f || hypotenuse <= 0.0f) { + return 0.0f; } - float angle_radians = std::acos(base / hypotenuse); - float angle_degrees = angle_radians * (180.0 / M_PI); + float angle_radians = acosf(base / hypotenuse); + float angle_degrees = angle_radians * (180.0f / std::numbers::pi_v); return angle_degrees; } -static bool validate_header_footer(const uint8_t *header_footer, const uint8_t *buffer) { - for (uint8_t i = 0; i < HEADER_FOOTER_SIZE; i++) { - if (header_footer[i] != buffer[i]) { - return false; // Mismatch in header/footer - } - } - return true; // Valid header/footer +static inline bool validate_header_footer(const uint8_t *header_footer, const uint8_t *buffer) { + return std::memcmp(header_footer, buffer, HEADER_FOOTER_SIZE) == 0; } void LD2450Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); #ifdef USE_NUMBER if (this->presence_timeout_number_ != nullptr) { - this->pref_ = global_preferences->make_preference(this->presence_timeout_number_->get_object_id_hash()); + this->pref_ = global_preferences->make_preference(this->presence_timeout_number_->get_preference_hash()); this->set_presence_timeout(); } #endif @@ -206,9 +199,8 @@ void LD2450Component::dump_config() { ESP_LOGCONFIG(TAG, "LD2450:\n" " Firmware version: %s\n" - " MAC address: %s\n" - " Throttle: %u ms", - version.c_str(), mac_str.c_str(), this->throttle_); + " MAC address: %s", + version.c_str(), mac_str.c_str()); #ifdef USE_BINARY_SENSOR ESP_LOGCONFIG(TAG, "Binary Sensors:"); LOG_BINARY_SENSOR(" ", "MovingTarget", this->moving_target_binary_sensor_); @@ -217,41 +209,41 @@ void LD2450Component::dump_config() { #endif #ifdef USE_SENSOR ESP_LOGCONFIG(TAG, "Sensors:"); - LOG_SENSOR(" ", "MovingTargetCount", this->moving_target_count_sensor_); - LOG_SENSOR(" ", "StillTargetCount", this->still_target_count_sensor_); - LOG_SENSOR(" ", "TargetCount", this->target_count_sensor_); - for (sensor::Sensor *s : this->move_x_sensors_) { - LOG_SENSOR(" ", "TargetX", s); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "MovingTargetCount", this->moving_target_count_sensor_); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "StillTargetCount", this->still_target_count_sensor_); + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "TargetCount", this->target_count_sensor_); + for (auto &s : this->move_x_sensors_) { + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "TargetX", s); } - for (sensor::Sensor *s : this->move_y_sensors_) { - LOG_SENSOR(" ", "TargetY", s); + for (auto &s : this->move_y_sensors_) { + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "TargetY", s); } - for (sensor::Sensor *s : this->move_angle_sensors_) { - LOG_SENSOR(" ", "TargetAngle", s); + for (auto &s : this->move_angle_sensors_) { + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "TargetAngle", s); } - for (sensor::Sensor *s : this->move_distance_sensors_) { - LOG_SENSOR(" ", "TargetDistance", s); + for (auto &s : this->move_distance_sensors_) { + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "TargetDistance", s); } - for (sensor::Sensor *s : this->move_resolution_sensors_) { - LOG_SENSOR(" ", "TargetResolution", s); + for (auto &s : this->move_resolution_sensors_) { + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "TargetResolution", s); } - for (sensor::Sensor *s : this->move_speed_sensors_) { - LOG_SENSOR(" ", "TargetSpeed", s); + for (auto &s : this->move_speed_sensors_) { + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "TargetSpeed", s); } - for (sensor::Sensor *s : this->zone_target_count_sensors_) { - LOG_SENSOR(" ", "ZoneTargetCount", s); + for (auto &s : this->zone_target_count_sensors_) { + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "ZoneTargetCount", s); } - for (sensor::Sensor *s : this->zone_moving_target_count_sensors_) { - LOG_SENSOR(" ", "ZoneMovingTargetCount", s); + for (auto &s : this->zone_moving_target_count_sensors_) { + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "ZoneMovingTargetCount", s); } - for (sensor::Sensor *s : this->zone_still_target_count_sensors_) { - LOG_SENSOR(" ", "ZoneStillTargetCount", s); + for (auto &s : this->zone_still_target_count_sensors_) { + LOG_SENSOR_WITH_DEDUP_SAFE(" ", "ZoneStillTargetCount", s); } #endif #ifdef USE_TEXT_SENSOR ESP_LOGCONFIG(TAG, "Text Sensors:"); LOG_TEXT_SENSOR(" ", "Version", this->version_text_sensor_); - LOG_TEXT_SENSOR(" ", "Mac", this->mac_text_sensor_); + LOG_TEXT_SENSOR(" ", "MAC address", this->mac_text_sensor_); for (text_sensor::TextSensor *s : this->direction_text_sensors_) { LOG_TEXT_SENSOR(" ", "Direction", s); } @@ -419,30 +411,25 @@ void LD2450Component::send_command_(uint8_t command, const uint8_t *command_valu if (command_value != nullptr) { len += command_value_len; } - uint8_t len_cmd[] = {lowbyte(len), highbyte(len), command, 0x00}; + // 2 length bytes (low, high) + 2 command bytes (low, high) + uint8_t len_cmd[] = {len, 0x00, command, 0x00}; this->write_array(len_cmd, sizeof(len_cmd)); - // command value bytes if (command_value != nullptr) { - for (uint8_t i = 0; i < command_value_len; i++) { - this->write_byte(command_value[i]); - } + this->write_array(command_value, command_value_len); } // frame footer bytes this->write_array(CMD_FRAME_FOOTER, sizeof(CMD_FRAME_FOOTER)); - // FIXME to remove - delay(50); // NOLINT + + if (command != CMD_ENABLE_CONF && command != CMD_DISABLE_CONF) { + delay(50); // NOLINT + } } // LD2450 Radar data message: // [AA FF 03 00] [0E 03 B1 86 10 00 40 01] [00 00 00 00 00 00 00 00] [00 00 00 00 00 00 00 00] [55 CC] // Header Target 1 Target 2 Target 3 End void LD2450Component::handle_periodic_data_() { - // Early throttle check - moved before any processing to save CPU cycles - if (App.get_loop_component_start_time() - this->last_periodic_millis_ < this->throttle_) { - return; - } - if (this->buffer_pos_ < 29) { // header (4 bytes) + 8 x 3 target data + footer (2 bytes) ESP_LOGE(TAG, "Invalid length"); return; @@ -453,14 +440,12 @@ void LD2450Component::handle_periodic_data_() { ESP_LOGE(TAG, "Invalid header/footer"); return; } - // Save the timestamp after validating the frame so, if invalid, we'll take the next frame immediately - this->last_periodic_millis_ = App.get_loop_component_start_time(); int16_t target_count = 0; int16_t still_target_count = 0; int16_t moving_target_count = 0; + int16_t res = 0; int16_t start = 0; - int16_t val = 0; int16_t tx = 0; int16_t ty = 0; int16_t td = 0; @@ -477,84 +462,44 @@ void LD2450Component::handle_periodic_data_() { // X start = TARGET_X + index * 8; is_moving = false; - sensor::Sensor *sx = this->move_x_sensors_[index]; - if (sx != nullptr) { - val = ld2450::decode_coordinate(this->buffer_data_[start], this->buffer_data_[start + 1]); - tx = val; - if (this->cached_target_data_[index].x != val) { - sx->publish_state(val); - this->cached_target_data_[index].x = val; - } - } + // tx is used for further calculations, so always needs to be populated + tx = ld2450::decode_coordinate(this->buffer_data_[start], this->buffer_data_[start + 1]); + SAFE_PUBLISH_SENSOR(this->move_x_sensors_[index], tx); // Y start = TARGET_Y + index * 8; - sensor::Sensor *sy = this->move_y_sensors_[index]; - if (sy != nullptr) { - val = ld2450::decode_coordinate(this->buffer_data_[start], this->buffer_data_[start + 1]); - ty = val; - if (this->cached_target_data_[index].y != val) { - sy->publish_state(val); - this->cached_target_data_[index].y = val; - } - } + ty = ld2450::decode_coordinate(this->buffer_data_[start], this->buffer_data_[start + 1]); + SAFE_PUBLISH_SENSOR(this->move_y_sensors_[index], ty); // RESOLUTION start = TARGET_RESOLUTION + index * 8; - sensor::Sensor *sr = this->move_resolution_sensors_[index]; - if (sr != nullptr) { - val = (this->buffer_data_[start + 1] << 8) | this->buffer_data_[start]; - if (this->cached_target_data_[index].resolution != val) { - sr->publish_state(val); - this->cached_target_data_[index].resolution = val; - } - } + res = (this->buffer_data_[start + 1] << 8) | this->buffer_data_[start]; + SAFE_PUBLISH_SENSOR(this->move_resolution_sensors_[index], res); #endif // SPEED start = TARGET_SPEED + index * 8; - val = ld2450::decode_speed(this->buffer_data_[start], this->buffer_data_[start + 1]); - ts = val; - if (val) { + ts = ld2450::decode_speed(this->buffer_data_[start], this->buffer_data_[start + 1]); + if (ts) { is_moving = true; moving_target_count++; } #ifdef USE_SENSOR - sensor::Sensor *ss = this->move_speed_sensors_[index]; - if (ss != nullptr) { - if (this->cached_target_data_[index].speed != val) { - ss->publish_state(val); - this->cached_target_data_[index].speed = val; - } - } + SAFE_PUBLISH_SENSOR(this->move_speed_sensors_[index], ts); #endif // DISTANCE // Optimized: use already decoded tx and ty values, replace pow() with multiplication int32_t x_squared = (int32_t) tx * tx; int32_t y_squared = (int32_t) ty * ty; - val = (uint16_t) sqrt(x_squared + y_squared); - td = val; - if (val > 0) { + td = (uint16_t) sqrtf(x_squared + y_squared); + if (td > 0) { target_count++; } #ifdef USE_SENSOR - sensor::Sensor *sd = this->move_distance_sensors_[index]; - if (sd != nullptr) { - if (this->cached_target_data_[index].distance != val) { - sd->publish_state(val); - this->cached_target_data_[index].distance = val; - } - } + SAFE_PUBLISH_SENSOR(this->move_distance_sensors_[index], td); // ANGLE angle = ld2450::calculate_angle(static_cast(ty), static_cast(td)); if (tx > 0) { angle = angle * -1; } - sensor::Sensor *sa = this->move_angle_sensors_[index]; - if (sa != nullptr) { - if (std::isnan(this->cached_target_data_[index].angle) || - std::abs(this->cached_target_data_[index].angle - angle) > 0.1f) { - sa->publish_state(angle); - this->cached_target_data_[index].angle = angle; - } - } + SAFE_PUBLISH_SENSOR(this->move_angle_sensors_[index], angle); #endif #ifdef USE_TEXT_SENSOR // DIRECTION @@ -568,11 +513,9 @@ void LD2450Component::handle_periodic_data_() { direction = DIRECTION_STATIONARY; } text_sensor::TextSensor *tsd = this->direction_text_sensors_[index]; - if (tsd != nullptr) { - if (this->cached_target_data_[index].direction != direction) { - tsd->publish_state(find_str(ld2450::DIRECTION_BY_UINT, direction)); - this->cached_target_data_[index].direction = direction; - } + const auto *dir_str = find_str(ld2450::DIRECTION_BY_UINT, direction); + if (tsd != nullptr && (!tsd->has_state() || tsd->get_state() != dir_str)) { + tsd->publish_state(dir_str); } #endif @@ -597,53 +540,19 @@ void LD2450Component::handle_periodic_data_() { zone_all_targets = zone_still_targets + zone_moving_targets; // Publish Still Target Count in Zones - sensor::Sensor *szstc = this->zone_still_target_count_sensors_[index]; - if (szstc != nullptr) { - if (this->cached_zone_data_[index].still_count != zone_still_targets) { - szstc->publish_state(zone_still_targets); - this->cached_zone_data_[index].still_count = zone_still_targets; - } - } + SAFE_PUBLISH_SENSOR(this->zone_still_target_count_sensors_[index], zone_still_targets); // Publish Moving Target Count in Zones - sensor::Sensor *szmtc = this->zone_moving_target_count_sensors_[index]; - if (szmtc != nullptr) { - if (this->cached_zone_data_[index].moving_count != zone_moving_targets) { - szmtc->publish_state(zone_moving_targets); - this->cached_zone_data_[index].moving_count = zone_moving_targets; - } - } + SAFE_PUBLISH_SENSOR(this->zone_moving_target_count_sensors_[index], zone_moving_targets); // Publish All Target Count in Zones - sensor::Sensor *sztc = this->zone_target_count_sensors_[index]; - if (sztc != nullptr) { - if (this->cached_zone_data_[index].total_count != zone_all_targets) { - sztc->publish_state(zone_all_targets); - this->cached_zone_data_[index].total_count = zone_all_targets; - } - } - + SAFE_PUBLISH_SENSOR(this->zone_target_count_sensors_[index], zone_all_targets); } // End loop thru zones // Target Count - if (this->target_count_sensor_ != nullptr) { - if (this->cached_global_data_.target_count != target_count) { - this->target_count_sensor_->publish_state(target_count); - this->cached_global_data_.target_count = target_count; - } - } + SAFE_PUBLISH_SENSOR(this->target_count_sensor_, target_count); // Still Target Count - if (this->still_target_count_sensor_ != nullptr) { - if (this->cached_global_data_.still_count != still_target_count) { - this->still_target_count_sensor_->publish_state(still_target_count); - this->cached_global_data_.still_count = still_target_count; - } - } + SAFE_PUBLISH_SENSOR(this->still_target_count_sensor_, still_target_count); // Moving Target Count - if (this->moving_target_count_sensor_ != nullptr) { - if (this->cached_global_data_.moving_count != moving_target_count) { - this->moving_target_count_sensor_->publish_state(moving_target_count); - this->cached_global_data_.moving_count = moving_target_count; - } - } + SAFE_PUBLISH_SENSOR(this->moving_target_count_sensor_, moving_target_count); #endif #ifdef USE_BINARY_SENSOR @@ -940,28 +849,33 @@ void LD2450Component::query_target_tracking_mode_() { this->send_command_(CMD_QU void LD2450Component::query_zone_() { this->send_command_(CMD_QUERY_ZONE, nullptr, 0); } #ifdef USE_SENSOR -void LD2450Component::set_move_x_sensor(uint8_t target, sensor::Sensor *s) { this->move_x_sensors_[target] = s; } -void LD2450Component::set_move_y_sensor(uint8_t target, sensor::Sensor *s) { this->move_y_sensors_[target] = s; } +// These could leak memory, but they are only set once prior to 'setup()' and should never be used again. +void LD2450Component::set_move_x_sensor(uint8_t target, sensor::Sensor *s) { + this->move_x_sensors_[target] = new SensorWithDedup(s); +} +void LD2450Component::set_move_y_sensor(uint8_t target, sensor::Sensor *s) { + this->move_y_sensors_[target] = new SensorWithDedup(s); +} void LD2450Component::set_move_speed_sensor(uint8_t target, sensor::Sensor *s) { - this->move_speed_sensors_[target] = s; + this->move_speed_sensors_[target] = new SensorWithDedup(s); } void LD2450Component::set_move_angle_sensor(uint8_t target, sensor::Sensor *s) { - this->move_angle_sensors_[target] = s; + this->move_angle_sensors_[target] = new SensorWithDedup(s); } void LD2450Component::set_move_distance_sensor(uint8_t target, sensor::Sensor *s) { - this->move_distance_sensors_[target] = s; + this->move_distance_sensors_[target] = new SensorWithDedup(s); } void LD2450Component::set_move_resolution_sensor(uint8_t target, sensor::Sensor *s) { - this->move_resolution_sensors_[target] = s; + this->move_resolution_sensors_[target] = new SensorWithDedup(s); } void LD2450Component::set_zone_target_count_sensor(uint8_t zone, sensor::Sensor *s) { - this->zone_target_count_sensors_[zone] = s; + this->zone_target_count_sensors_[zone] = new SensorWithDedup(s); } void LD2450Component::set_zone_still_target_count_sensor(uint8_t zone, sensor::Sensor *s) { - this->zone_still_target_count_sensors_[zone] = s; + this->zone_still_target_count_sensors_[zone] = new SensorWithDedup(s); } void LD2450Component::set_zone_moving_target_count_sensor(uint8_t zone, sensor::Sensor *s) { - this->zone_moving_target_count_sensors_[zone] = s; + this->zone_moving_target_count_sensors_[zone] = new SensorWithDedup(s); } #endif #ifdef USE_TEXT_SENSOR diff --git a/esphome/components/ld2450/ld2450.h b/esphome/components/ld2450/ld2450.h index ae72a0d8cb..9faa189019 100644 --- a/esphome/components/ld2450/ld2450.h +++ b/esphome/components/ld2450/ld2450.h @@ -1,12 +1,7 @@ #pragma once -#include "esphome/components/uart/uart.h" -#include "esphome/core/component.h" #include "esphome/core/defines.h" -#include "esphome/core/helpers.h" -#include "esphome/core/preferences.h" -#include -#include +#include "esphome/core/component.h" #ifdef USE_SENSOR #include "esphome/components/sensor/sensor.h" #endif @@ -29,18 +24,23 @@ #include "esphome/components/binary_sensor/binary_sensor.h" #endif -#ifndef M_PI -#define M_PI 3.14 -#endif +#include "esphome/components/ld24xx/ld24xx.h" +#include "esphome/components/uart/uart.h" +#include "esphome/core/helpers.h" +#include "esphome/core/preferences.h" + +#include namespace esphome { namespace ld2450 { +using namespace ld24xx; + // Constants -static const uint8_t DEFAULT_PRESENCE_TIMEOUT = 5; // Timeout to reset presense status 5 sec. -static const uint8_t MAX_LINE_LENGTH = 41; // Max characters for serial buffer -static const uint8_t MAX_TARGETS = 3; // Max 3 Targets in LD2450 -static const uint8_t MAX_ZONES = 3; // Max 3 Zones in LD2450 +static constexpr uint8_t DEFAULT_PRESENCE_TIMEOUT = 5; // Timeout to reset presense status 5 sec. +static constexpr uint8_t MAX_LINE_LENGTH = 41; // Max characters for serial buffer +static constexpr uint8_t MAX_TARGETS = 3; // Max 3 Targets in LD2450 +static constexpr uint8_t MAX_ZONES = 3; // Max 3 Zones in LD2450 enum Direction : uint8_t { DIRECTION_APPROACHING = 0, @@ -81,9 +81,9 @@ class LD2450Component : public Component, public uart::UARTDevice { SUB_BINARY_SENSOR(target) #endif #ifdef USE_SENSOR - SUB_SENSOR(moving_target_count) - SUB_SENSOR(still_target_count) - SUB_SENSOR(target_count) + SUB_SENSOR_WITH_DEDUP(moving_target_count, uint8_t) + SUB_SENSOR_WITH_DEDUP(still_target_count, uint8_t) + SUB_SENSOR_WITH_DEDUP(target_count, uint8_t) #endif #ifdef USE_TEXT_SENSOR SUB_TEXT_SENSOR(mac) @@ -110,7 +110,6 @@ class LD2450Component : public Component, public uart::UARTDevice { void dump_config() override; void loop() override; void set_presence_timeout(); - void set_throttle(uint16_t value) { this->throttle_ = value; } void read_all_info(); void query_zone_info(); void restart_and_read_all_info(); @@ -161,11 +160,9 @@ class LD2450Component : public Component, public uart::UARTDevice { bool get_timeout_status_(uint32_t check_millis); uint8_t count_targets_in_zone_(const Zone &zone, bool is_moving); - uint32_t last_periodic_millis_ = 0; uint32_t presence_millis_ = 0; uint32_t still_presence_millis_ = 0; uint32_t moving_presence_millis_ = 0; - uint16_t throttle_ = 0; uint16_t timeout_ = 5; uint8_t buffer_data_[MAX_LINE_LENGTH]; uint8_t mac_address_[6] = {0, 0, 0, 0, 0, 0}; @@ -176,48 +173,23 @@ class LD2450Component : public Component, public uart::UARTDevice { Target target_info_[MAX_TARGETS]; Zone zone_config_[MAX_ZONES]; - // Change detection - cache previous values to avoid redundant publishes - // All values are initialized to sentinel values that are outside the valid sensor ranges - // to ensure the first real measurement is always published - struct CachedTargetData { - int16_t x = std::numeric_limits::min(); // -32768, outside range of -4860 to 4860 - int16_t y = std::numeric_limits::min(); // -32768, outside range of 0 to 7560 - int16_t speed = std::numeric_limits::min(); // -32768, outside practical sensor range - uint16_t resolution = std::numeric_limits::max(); // 65535, unlikely resolution value - uint16_t distance = std::numeric_limits::max(); // 65535, outside range of 0 to ~8990 - Direction direction = DIRECTION_UNDEFINED; // Undefined, will differ from any real direction - float angle = NAN; // NAN, safe sentinel for floats - } cached_target_data_[MAX_TARGETS]; - - struct CachedZoneData { - uint8_t still_count = std::numeric_limits::max(); // 255, unlikely zone count - uint8_t moving_count = std::numeric_limits::max(); // 255, unlikely zone count - uint8_t total_count = std::numeric_limits::max(); // 255, unlikely zone count - } cached_zone_data_[MAX_ZONES]; - - struct CachedGlobalData { - uint8_t target_count = std::numeric_limits::max(); // 255, max 3 targets possible - uint8_t still_count = std::numeric_limits::max(); // 255, max 3 targets possible - uint8_t moving_count = std::numeric_limits::max(); // 255, max 3 targets possible - } cached_global_data_; - #ifdef USE_NUMBER ESPPreferenceObject pref_; // only used when numbers are in use ZoneOfNumbers zone_numbers_[MAX_ZONES]; #endif #ifdef USE_SENSOR - std::vector move_x_sensors_ = std::vector(MAX_TARGETS); - std::vector move_y_sensors_ = std::vector(MAX_TARGETS); - std::vector move_speed_sensors_ = std::vector(MAX_TARGETS); - std::vector move_angle_sensors_ = std::vector(MAX_TARGETS); - std::vector move_distance_sensors_ = std::vector(MAX_TARGETS); - std::vector move_resolution_sensors_ = std::vector(MAX_TARGETS); - std::vector zone_target_count_sensors_ = std::vector(MAX_ZONES); - std::vector zone_still_target_count_sensors_ = std::vector(MAX_ZONES); - std::vector zone_moving_target_count_sensors_ = std::vector(MAX_ZONES); + std::array *, MAX_TARGETS> move_x_sensors_{}; + std::array *, MAX_TARGETS> move_y_sensors_{}; + std::array *, MAX_TARGETS> move_speed_sensors_{}; + std::array *, MAX_TARGETS> move_angle_sensors_{}; + std::array *, MAX_TARGETS> move_distance_sensors_{}; + std::array *, MAX_TARGETS> move_resolution_sensors_{}; + std::array *, MAX_ZONES> zone_target_count_sensors_{}; + std::array *, MAX_ZONES> zone_still_target_count_sensors_{}; + std::array *, MAX_ZONES> zone_moving_target_count_sensors_{}; #endif #ifdef USE_TEXT_SENSOR - std::vector direction_text_sensors_ = std::vector(3); + std::array direction_text_sensors_{}; #endif }; diff --git a/esphome/components/ld2450/sensor.py b/esphome/components/ld2450/sensor.py index 071ce8aa32..4a3597d583 100644 --- a/esphome/components/ld2450/sensor.py +++ b/esphome/components/ld2450/sensor.py @@ -42,12 +42,42 @@ CONFIG_SCHEMA = cv.Schema( { cv.GenerateID(CONF_LD2450_ID): cv.use_id(LD2450Component), cv.Optional(CONF_TARGET_COUNT): sensor.sensor_schema( + accuracy_decimals=0, + filters=[ + { + "timeout": { + "timeout": cv.TimePeriod(milliseconds=1000), + "value": "last", + } + }, + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}, + ], icon=ICON_ACCOUNT_GROUP, ), cv.Optional(CONF_STILL_TARGET_COUNT): sensor.sensor_schema( + accuracy_decimals=0, + filters=[ + { + "timeout": { + "timeout": cv.TimePeriod(milliseconds=1000), + "value": "last", + } + }, + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}, + ], icon=ICON_HUMAN_GREETING_PROXIMITY, ), cv.Optional(CONF_MOVING_TARGET_COUNT): sensor.sensor_schema( + accuracy_decimals=0, + filters=[ + { + "timeout": { + "timeout": cv.TimePeriod(milliseconds=1000), + "value": "last", + } + }, + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}, + ], icon=ICON_ACCOUNT_SWITCH, ), } @@ -59,32 +89,86 @@ CONFIG_SCHEMA = CONFIG_SCHEMA.extend( { cv.Optional(CONF_X): sensor.sensor_schema( device_class=DEVICE_CLASS_DISTANCE, - unit_of_measurement=UNIT_MILLIMETER, + filters=[ + { + "timeout": { + "timeout": cv.TimePeriod(milliseconds=1000), + "value": "last", + } + }, + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}, + ], icon=ICON_ALPHA_X_BOX_OUTLINE, + unit_of_measurement=UNIT_MILLIMETER, ), cv.Optional(CONF_Y): sensor.sensor_schema( device_class=DEVICE_CLASS_DISTANCE, - unit_of_measurement=UNIT_MILLIMETER, + filters=[ + { + "timeout": { + "timeout": cv.TimePeriod(milliseconds=1000), + "value": "last", + } + }, + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}, + ], icon=ICON_ALPHA_Y_BOX_OUTLINE, + unit_of_measurement=UNIT_MILLIMETER, ), cv.Optional(CONF_SPEED): sensor.sensor_schema( device_class=DEVICE_CLASS_SPEED, - unit_of_measurement=UNIT_MILLIMETER_PER_SECOND, + filters=[ + { + "timeout": { + "timeout": cv.TimePeriod(milliseconds=1000), + "value": "last", + } + }, + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}, + ], icon=ICON_SPEEDOMETER_SLOW, + unit_of_measurement=UNIT_MILLIMETER_PER_SECOND, ), cv.Optional(CONF_ANGLE): sensor.sensor_schema( - unit_of_measurement=UNIT_DEGREES, + filters=[ + { + "timeout": { + "timeout": cv.TimePeriod(milliseconds=1000), + "value": "last", + } + }, + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}, + ], icon=ICON_FORMAT_TEXT_ROTATION_ANGLE_UP, + unit_of_measurement=UNIT_DEGREES, ), cv.Optional(CONF_DISTANCE): sensor.sensor_schema( device_class=DEVICE_CLASS_DISTANCE, - unit_of_measurement=UNIT_MILLIMETER, + filters=[ + { + "timeout": { + "timeout": cv.TimePeriod(milliseconds=1000), + "value": "last", + } + }, + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}, + ], icon=ICON_MAP_MARKER_DISTANCE, + unit_of_measurement=UNIT_MILLIMETER, ), cv.Optional(CONF_RESOLUTION): sensor.sensor_schema( device_class=DEVICE_CLASS_DISTANCE, - unit_of_measurement=UNIT_MILLIMETER, + filters=[ + { + "timeout": { + "timeout": cv.TimePeriod(milliseconds=1000), + "value": "last", + } + }, + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}, + ], icon=ICON_RELATION_ZERO_OR_ONE_TO_ZERO_OR_ONE, + unit_of_measurement=UNIT_MILLIMETER, ), } ) @@ -94,12 +178,42 @@ CONFIG_SCHEMA = CONFIG_SCHEMA.extend( cv.Optional(f"zone_{n + 1}"): cv.Schema( { cv.Optional(CONF_TARGET_COUNT): sensor.sensor_schema( + accuracy_decimals=0, + filters=[ + { + "timeout": { + "timeout": cv.TimePeriod(milliseconds=1000), + "value": "last", + } + }, + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}, + ], icon=ICON_MAP_MARKER_ACCOUNT, ), cv.Optional(CONF_STILL_TARGET_COUNT): sensor.sensor_schema( + accuracy_decimals=0, + filters=[ + { + "timeout": { + "timeout": cv.TimePeriod(milliseconds=1000), + "value": "last", + } + }, + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}, + ], icon=ICON_MAP_MARKER_ACCOUNT, ), cv.Optional(CONF_MOVING_TARGET_COUNT): sensor.sensor_schema( + accuracy_decimals=0, + filters=[ + { + "timeout": { + "timeout": cv.TimePeriod(milliseconds=1000), + "value": "last", + } + }, + {"throttle_with_priority": cv.TimePeriod(milliseconds=1000)}, + ], icon=ICON_MAP_MARKER_ACCOUNT, ), } diff --git a/esphome/components/ld2450/text_sensor.py b/esphome/components/ld2450/text_sensor.py index 6c11024b89..89b6939a29 100644 --- a/esphome/components/ld2450/text_sensor.py +++ b/esphome/components/ld2450/text_sensor.py @@ -56,7 +56,8 @@ async def to_code(config): sens = await text_sensor.new_text_sensor(mac_address_config) cg.add(ld2450_component.set_mac_text_sensor(sens)) for n in range(MAX_TARGETS): - if direction_conf := config.get(f"target_{n + 1}"): - if direction_config := direction_conf.get(CONF_DIRECTION): - sens = await text_sensor.new_text_sensor(direction_config) - cg.add(ld2450_component.set_direction_text_sensor(n, sens)) + if (direction_conf := config.get(f"target_{n + 1}")) and ( + direction_config := direction_conf.get(CONF_DIRECTION) + ): + sens = await text_sensor.new_text_sensor(direction_config) + cg.add(ld2450_component.set_direction_text_sensor(n, sens)) diff --git a/esphome/components/ld24xx/__init__.py b/esphome/components/ld24xx/__init__.py new file mode 100644 index 0000000000..516af84856 --- /dev/null +++ b/esphome/components/ld24xx/__init__.py @@ -0,0 +1 @@ +CODEOWNERS = ["@kbx81"] diff --git a/esphome/components/ld24xx/ld24xx.h b/esphome/components/ld24xx/ld24xx.h new file mode 100644 index 0000000000..1cd5e01163 --- /dev/null +++ b/esphome/components/ld24xx/ld24xx.h @@ -0,0 +1,65 @@ +#pragma once + +#include "esphome/core/defines.h" + +#include + +#ifdef USE_SENSOR +#include "esphome/core/helpers.h" +#include "esphome/components/sensor/sensor.h" + +#define SUB_SENSOR_WITH_DEDUP(name, dedup_type) \ + protected: \ + ld24xx::SensorWithDedup *name##_sensor_{nullptr}; \ +\ + public: \ + void set_##name##_sensor(sensor::Sensor *sensor) { \ + this->name##_sensor_ = new ld24xx::SensorWithDedup(sensor); \ + } +#endif + +#define LOG_SENSOR_WITH_DEDUP_SAFE(tag, name, sensor) \ + if ((sensor) != nullptr) { \ + LOG_SENSOR(tag, name, (sensor)->sens); \ + } + +#define SAFE_PUBLISH_SENSOR(sensor, value) \ + if ((sensor) != nullptr) { \ + (sensor)->publish_state_if_not_dup(value); \ + } + +#define SAFE_PUBLISH_SENSOR_UNKNOWN(sensor) \ + if ((sensor) != nullptr) { \ + (sensor)->publish_state_unknown(); \ + } + +#define highbyte(val) (uint8_t)((val) >> 8) +#define lowbyte(val) (uint8_t)((val) &0xff) + +namespace esphome { +namespace ld24xx { + +#ifdef USE_SENSOR +// Helper class to store a sensor with a deduplicator & publish state only when the value changes +template class SensorWithDedup { + public: + SensorWithDedup(sensor::Sensor *sens) : sens(sens) {} + + void publish_state_if_not_dup(T state) { + if (this->publish_dedup.next(state)) { + this->sens->publish_state(static_cast(state)); + } + } + + void publish_state_unknown() { + if (this->publish_dedup.next_unknown()) { + this->sens->publish_state(NAN); + } + } + + sensor::Sensor *sens; + Deduplicator publish_dedup; +}; +#endif +} // namespace ld24xx +} // namespace esphome diff --git a/esphome/components/ledc/ledc_output.cpp b/esphome/components/ledc/ledc_output.cpp index 2ae2656f54..aaa4794586 100644 --- a/esphome/components/ledc/ledc_output.cpp +++ b/esphome/components/ledc/ledc_output.cpp @@ -116,7 +116,6 @@ void LEDCOutput::write_state(float state) { } void LEDCOutput::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); auto speed_mode = get_speed_mode(this->channel_); auto timer_num = static_cast((this->channel_ % 8) / 2); auto chan_num = static_cast(this->channel_ % 8); diff --git a/esphome/components/libretiny/__init__.py b/esphome/components/libretiny/__init__.py index 17d5d46ffd..c63d6d7faa 100644 --- a/esphome/components/libretiny/__init__.py +++ b/esphome/components/libretiny/__init__.py @@ -1,6 +1,5 @@ import json import logging -from os.path import dirname, isfile, join import esphome.codegen as cg import esphome.config_validation as cv @@ -20,9 +19,11 @@ from esphome.const import ( KEY_FRAMEWORK_VERSION, KEY_TARGET_FRAMEWORK, KEY_TARGET_PLATFORM, + ThreadModel, __version__, ) from esphome.core import CORE +from esphome.storage_json import StorageJSON from . import gpio # noqa from .const import ( @@ -128,7 +129,7 @@ def only_on_family(*, supported=None, unsupported=None): return validator_ -def get_download_types(storage_json=None): +def get_download_types(storage_json: StorageJSON = None): types = [ { "title": "UF2 package (recommended)", @@ -138,11 +139,11 @@ def get_download_types(storage_json=None): }, ] - build_dir = dirname(storage_json.firmware_bin_path) - outputs = join(build_dir, "firmware.json") - if not isfile(outputs): + build_dir = storage_json.firmware_bin_path.parent + outputs = build_dir / "firmware.json" + if not outputs.is_file(): return types - with open(outputs, encoding="utf-8") as f: + with outputs.open(encoding="utf-8") as f: outputs = json.load(f) for output in outputs: if not output["public"]: @@ -260,6 +261,7 @@ async def component_to_code(config): cg.add_build_flag(f"-DUSE_LIBRETINY_VARIANT_{config[CONF_FAMILY]}") cg.add_define("ESPHOME_BOARD", config[CONF_BOARD]) cg.add_define("ESPHOME_VARIANT", FAMILY_FRIENDLY[config[CONF_FAMILY]]) + cg.add_define(ThreadModel.MULTI_NO_ATOMICS) # force using arduino framework cg.add_platformio_option("framework", "arduino") diff --git a/esphome/components/libretiny/helpers.cpp b/esphome/components/libretiny/helpers.cpp index b6451860d5..37ae0fb455 100644 --- a/esphome/components/libretiny/helpers.cpp +++ b/esphome/components/libretiny/helpers.cpp @@ -26,6 +26,10 @@ void Mutex::unlock() { xSemaphoreGive(this->handle_); } IRAM_ATTR InterruptLock::InterruptLock() { portDISABLE_INTERRUPTS(); } IRAM_ATTR InterruptLock::~InterruptLock() { portENABLE_INTERRUPTS(); } +// LibreTiny doesn't support lwIP core locking, so this is a no-op +LwIPLock::LwIPLock() {} +LwIPLock::~LwIPLock() {} + void get_mac_address_raw(uint8_t *mac) { // NOLINT(readability-non-const-parameter) WiFi.macAddress(mac); } diff --git a/esphome/components/libretiny/preferences.cpp b/esphome/components/libretiny/preferences.cpp index ce4ed915c0..871b186d8e 100644 --- a/esphome/components/libretiny/preferences.cpp +++ b/esphome/components/libretiny/preferences.cpp @@ -5,7 +5,7 @@ #include "esphome/core/preferences.h" #include #include -#include +#include #include namespace esphome { @@ -15,7 +15,14 @@ static const char *const TAG = "lt.preferences"; struct NVSData { std::string key; - std::vector data; + std::unique_ptr data; + size_t len; + + void set_data(const uint8_t *src, size_t size) { + data = std::make_unique(size); + memcpy(data.get(), src, size); + len = size; + } }; static std::vector s_pending_save; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) @@ -30,15 +37,15 @@ class LibreTinyPreferenceBackend : public ESPPreferenceBackend { // try find in pending saves and update that for (auto &obj : s_pending_save) { if (obj.key == key) { - obj.data.assign(data, data + len); + obj.set_data(data, len); return true; } } NVSData save{}; save.key = key; - save.data.assign(data, data + len); - s_pending_save.emplace_back(save); - ESP_LOGVV(TAG, "s_pending_save: key: %s, len: %d", key.c_str(), len); + save.set_data(data, len); + s_pending_save.emplace_back(std::move(save)); + ESP_LOGVV(TAG, "s_pending_save: key: %s, len: %zu", key.c_str(), len); return true; } @@ -46,11 +53,11 @@ class LibreTinyPreferenceBackend : public ESPPreferenceBackend { // try find in pending saves and load from that for (auto &obj : s_pending_save) { if (obj.key == key) { - if (obj.data.size() != len) { + if (obj.len != len) { // size mismatch return false; } - memcpy(data, obj.data.data(), len); + memcpy(data, obj.data.get(), len); return true; } } @@ -58,10 +65,10 @@ class LibreTinyPreferenceBackend : public ESPPreferenceBackend { fdb_blob_make(blob, data, len); size_t actual_len = fdb_kv_get_blob(db, key.c_str(), blob); if (actual_len != len) { - ESP_LOGVV(TAG, "NVS length does not match (%u!=%u)", actual_len, len); + ESP_LOGVV(TAG, "NVS length does not match (%zu!=%zu)", actual_len, len); return false; } else { - ESP_LOGVV(TAG, "fdb_kv_get_blob: key: %s, len: %d", key.c_str(), len); + ESP_LOGVV(TAG, "fdb_kv_get_blob: key: %s, len: %zu", key.c_str(), len); } return true; } @@ -101,7 +108,7 @@ class LibreTinyPreferences : public ESPPreferences { if (s_pending_save.empty()) return true; - ESP_LOGV(TAG, "Saving %d items...", s_pending_save.size()); + ESP_LOGV(TAG, "Saving %zu items...", s_pending_save.size()); // goal try write all pending saves even if one fails int cached = 0, written = 0, failed = 0; fdb_err_t last_err = FDB_NO_ERR; @@ -112,11 +119,11 @@ class LibreTinyPreferences : public ESPPreferences { const auto &save = s_pending_save[i]; ESP_LOGVV(TAG, "Checking if FDB data %s has changed", save.key.c_str()); if (is_changed(&db, save)) { - ESP_LOGV(TAG, "sync: key: %s, len: %d", save.key.c_str(), save.data.size()); - fdb_blob_make(&blob, save.data.data(), save.data.size()); + ESP_LOGV(TAG, "sync: key: %s, len: %zu", save.key.c_str(), save.len); + fdb_blob_make(&blob, save.data.get(), save.len); fdb_err_t err = fdb_kv_set_blob(&db, save.key.c_str(), &blob); if (err != FDB_NO_ERR) { - ESP_LOGV(TAG, "fdb_kv_set_blob('%s', len=%u) failed: %d", save.key.c_str(), save.data.size(), err); + ESP_LOGV(TAG, "fdb_kv_set_blob('%s', len=%zu) failed: %d", save.key.c_str(), save.len, err); failed++; last_err = err; last_key = save.key; @@ -124,7 +131,7 @@ class LibreTinyPreferences : public ESPPreferences { } written++; } else { - ESP_LOGD(TAG, "FDB data not changed; skipping %s len=%u", save.key.c_str(), save.data.size()); + ESP_LOGD(TAG, "FDB data not changed; skipping %s len=%zu", save.key.c_str(), save.len); cached++; } s_pending_save.erase(s_pending_save.begin() + i); @@ -139,21 +146,29 @@ class LibreTinyPreferences : public ESPPreferences { } bool is_changed(const fdb_kvdb_t db, const NVSData &to_save) { - NVSData stored_data{}; struct fdb_kv kv; fdb_kv_t kvp = fdb_kv_get_obj(db, to_save.key.c_str(), &kv); if (kvp == nullptr) { ESP_LOGV(TAG, "fdb_kv_get_obj('%s'): nullptr - the key might not be set yet", to_save.key.c_str()); return true; } - stored_data.data.resize(kv.value_len); - fdb_blob_make(&blob, stored_data.data.data(), kv.value_len); + + // Check size first - if different, data has changed + if (kv.value_len != to_save.len) { + return true; + } + + // Allocate buffer on heap to avoid stack allocation for large data + auto stored_data = std::make_unique(kv.value_len); + fdb_blob_make(&blob, stored_data.get(), kv.value_len); size_t actual_len = fdb_kv_get_blob(db, to_save.key.c_str(), &blob); if (actual_len != kv.value_len) { ESP_LOGV(TAG, "fdb_kv_get_blob('%s') len mismatch: %u != %u", to_save.key.c_str(), actual_len, kv.value_len); return true; } - return to_save.data != stored_data.data; + + // Compare the actual data + return memcmp(to_save.data.get(), stored_data.get(), kv.value_len) != 0; } bool reset() override { diff --git a/esphome/components/light/__init__.py b/esphome/components/light/__init__.py index 7ab899edb2..f1089ad64f 100644 --- a/esphome/components/light/__init__.py +++ b/esphome/components/light/__init__.py @@ -37,7 +37,7 @@ from esphome.const import ( CONF_WEB_SERVER, CONF_WHITE, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass @@ -283,7 +283,6 @@ async def new_light(config, *args): return output_var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_LIGHT") cg.add_global(light_ns.using) diff --git a/esphome/components/light/addressable_light_effect.h b/esphome/components/light/addressable_light_effect.h index d622ec0375..fcf76b3cb0 100644 --- a/esphome/components/light/addressable_light_effect.h +++ b/esphome/components/light/addressable_light_effect.h @@ -44,6 +44,13 @@ class AddressableLightEffect : public LightEffect { this->apply(*this->get_addressable_(), current_color); } + /// Get effect index specifically for addressable effects. + /// Can be used by effects to modify behavior based on their position in the list. + uint32_t get_effect_index() const { return this->get_index(); } + + /// Check if this is the currently running addressable effect. + bool is_current_effect() const { return this->is_active() && this->get_addressable_()->is_effect_active(); } + protected: AddressableLight *get_addressable_() const { return (AddressableLight *) this->state_->get_output(); } }; diff --git a/esphome/components/light/base_light_effects.h b/esphome/components/light/base_light_effects.h index 9e02e889c9..ff6cd1ccfe 100644 --- a/esphome/components/light/base_light_effects.h +++ b/esphome/components/light/base_light_effects.h @@ -125,6 +125,10 @@ class LambdaLightEffect : public LightEffect { } } + /// Get the current effect index for use in lambda functions. + /// This can be useful for lambda effects that need to know their own index. + uint32_t get_current_index() const { return this->get_index(); } + protected: std::function f_; uint32_t update_interval_; @@ -143,6 +147,10 @@ class AutomationLightEffect : public LightEffect { } Trigger<> *get_trig() const { return trig_; } + /// Get the current effect index for use in automations. + /// Useful for automations that need to know which effect is running. + uint32_t get_current_index() const { return this->get_index(); } + protected: Trigger<> *trig_{new Trigger<>}; }; diff --git a/esphome/components/light/effects.py b/esphome/components/light/effects.py index 67c318eb8e..15d9272d1a 100644 --- a/esphome/components/light/effects.py +++ b/esphome/components/light/effects.py @@ -29,6 +29,7 @@ from esphome.const import ( CONF_WHITE, CONF_WIDTH, ) +from esphome.cpp_generator import MockObjClass from esphome.schema_extractors import SCHEMA_EXTRACT, schema_extractor from esphome.util import Registry @@ -88,8 +89,15 @@ ADDRESSABLE_EFFECTS = [] EFFECTS_REGISTRY = Registry() -def register_effect(name, effect_type, default_name, schema, *extra_validators): - schema = cv.Schema(schema).extend( +def register_effect( + name: str, + effect_type: MockObjClass, + default_name: str, + schema: cv.Schema | dict, + *extra_validators, +): + schema = schema if isinstance(schema, cv.Schema) else cv.Schema(schema) + schema = schema.extend( { cv.Optional(CONF_NAME, default=default_name): cv.string_strict, } @@ -98,7 +106,13 @@ def register_effect(name, effect_type, default_name, schema, *extra_validators): return EFFECTS_REGISTRY.register(name, effect_type, validator) -def register_binary_effect(name, effect_type, default_name, schema, *extra_validators): +def register_binary_effect( + name: str, + effect_type: MockObjClass, + default_name: str, + schema: cv.Schema | dict, + *extra_validators, +): # binary effect can be used for all lights BINARY_EFFECTS.append(name) MONOCHROMATIC_EFFECTS.append(name) @@ -109,7 +123,11 @@ def register_binary_effect(name, effect_type, default_name, schema, *extra_valid def register_monochromatic_effect( - name, effect_type, default_name, schema, *extra_validators + name: str, + effect_type: MockObjClass, + default_name: str, + schema: cv.Schema | dict, + *extra_validators, ): # monochromatic effect can be used for all lights expect binary MONOCHROMATIC_EFFECTS.append(name) @@ -119,7 +137,13 @@ def register_monochromatic_effect( return register_effect(name, effect_type, default_name, schema, *extra_validators) -def register_rgb_effect(name, effect_type, default_name, schema, *extra_validators): +def register_rgb_effect( + name: str, + effect_type: MockObjClass, + default_name: str, + schema: cv.Schema | dict, + *extra_validators, +): # RGB effect can be used for RGB and addressable lights RGB_EFFECTS.append(name) ADDRESSABLE_EFFECTS.append(name) @@ -128,7 +152,11 @@ def register_rgb_effect(name, effect_type, default_name, schema, *extra_validato def register_addressable_effect( - name, effect_type, default_name, schema, *extra_validators + name: str, + effect_type: MockObjClass, + default_name: str, + schema: cv.Schema | dict, + *extra_validators, ): # addressable effect can be used only in addressable ADDRESSABLE_EFFECTS.append(name) @@ -291,31 +319,30 @@ async def random_effect_to_code(config, effect_id): ) async def strobe_effect_to_code(config, effect_id): var = cg.new_Pvariable(effect_id, config[CONF_NAME]) - colors = [] - for color in config.get(CONF_COLORS, []): - colors.append( - cg.StructInitializer( - StrobeLightEffectColor, - ( - "color", - LightColorValues( - color.get(CONF_COLOR_MODE, ColorMode.UNKNOWN), - color[CONF_STATE], - color[CONF_BRIGHTNESS], - color[CONF_COLOR_BRIGHTNESS], - color[CONF_RED], - color[CONF_GREEN], - color[CONF_BLUE], - color[CONF_WHITE], - color.get(CONF_COLOR_TEMPERATURE, 0.0), - color[CONF_COLD_WHITE], - color[CONF_WARM_WHITE], - ), + colors = [ + cg.StructInitializer( + StrobeLightEffectColor, + ( + "color", + LightColorValues( + color.get(CONF_COLOR_MODE, ColorMode.UNKNOWN), + color[CONF_STATE], + color[CONF_BRIGHTNESS], + color[CONF_COLOR_BRIGHTNESS], + color[CONF_RED], + color[CONF_GREEN], + color[CONF_BLUE], + color[CONF_WHITE], + color.get(CONF_COLOR_TEMPERATURE, 0.0), + color[CONF_COLD_WHITE], + color[CONF_WARM_WHITE], ), - ("duration", color[CONF_DURATION]), - ("transition_length", color[CONF_TRANSITION_LENGTH]), - ) + ), + ("duration", color[CONF_DURATION]), + ("transition_length", color[CONF_TRANSITION_LENGTH]), ) + for color in config.get(CONF_COLORS, []) + ] cg.add(var.set_colors(colors)) return var @@ -354,10 +381,9 @@ async def addressable_lambda_effect_to_code(config, effect_id): (bool, "initial_run"), ] lambda_ = await cg.process_lambda(config[CONF_LAMBDA], args, return_type=cg.void) - var = cg.new_Pvariable( + return cg.new_Pvariable( effect_id, config[CONF_NAME], lambda_, config[CONF_UPDATE_INTERVAL] ) - return var @register_addressable_effect( @@ -404,20 +430,19 @@ async def addressable_color_wipe_effect_to_code(config, effect_id): var = cg.new_Pvariable(effect_id, config[CONF_NAME]) cg.add(var.set_add_led_interval(config[CONF_ADD_LED_INTERVAL])) cg.add(var.set_reverse(config[CONF_REVERSE])) - colors = [] - for color in config.get(CONF_COLORS, []): - colors.append( - cg.StructInitializer( - AddressableColorWipeEffectColor, - ("r", int(round(color[CONF_RED] * 255))), - ("g", int(round(color[CONF_GREEN] * 255))), - ("b", int(round(color[CONF_BLUE] * 255))), - ("w", int(round(color[CONF_WHITE] * 255))), - ("random", color[CONF_RANDOM]), - ("num_leds", color[CONF_NUM_LEDS]), - ("gradient", color[CONF_GRADIENT]), - ) + colors = [ + cg.StructInitializer( + AddressableColorWipeEffectColor, + ("r", int(round(color[CONF_RED] * 255))), + ("g", int(round(color[CONF_GREEN] * 255))), + ("b", int(round(color[CONF_BLUE] * 255))), + ("w", int(round(color[CONF_WHITE] * 255))), + ("random", color[CONF_RANDOM]), + ("num_leds", color[CONF_NUM_LEDS]), + ("gradient", color[CONF_GRADIENT]), ) + for color in config.get(CONF_COLORS, []) + ] cg.add(var.set_colors(colors)) return var @@ -526,7 +551,7 @@ def validate_effects(allowed_effects): errors = [] names = set() for i, x in enumerate(value): - key = next(it for it in x.keys()) + key = next(it for it in x) if key not in allowed_effects: errors.append( cv.Invalid( diff --git a/esphome/components/light/light_call.cpp b/esphome/components/light/light_call.cpp index a3ffe22591..915b8fdf89 100644 --- a/esphome/components/light/light_call.cpp +++ b/esphome/components/light/light_call.cpp @@ -9,6 +9,33 @@ namespace light { static const char *const TAG = "light"; +// Helper functions to reduce code size for logging +static void clamp_and_log_if_invalid(const char *name, float &value, const LogString *param_name, float min = 0.0f, + float max = 1.0f) { + if (value < min || value > max) { + ESP_LOGW(TAG, "'%s': %s value %.2f is out of range [%.1f - %.1f]", name, LOG_STR_ARG(param_name), value, min, max); + value = clamp(value, min, max); + } +} + +#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_WARN +static void log_feature_not_supported(const char *name, const LogString *feature) { + ESP_LOGW(TAG, "'%s': %s not supported", name, LOG_STR_ARG(feature)); +} + +static void log_color_mode_not_supported(const char *name, const LogString *feature) { + ESP_LOGW(TAG, "'%s': color mode does not support setting %s", name, LOG_STR_ARG(feature)); +} + +static void log_invalid_parameter(const char *name, const LogString *message) { + ESP_LOGW(TAG, "'%s': %s", name, LOG_STR_ARG(message)); +} +#else +#define log_feature_not_supported(name, feature) +#define log_color_mode_not_supported(name, feature) +#define log_invalid_parameter(name, message) +#endif + // Macro to reduce repetitive setter code #define IMPLEMENT_LIGHT_CALL_SETTER(name, type, flag) \ LightCall &LightCall::set_##name(optional(name)) { \ @@ -20,7 +47,7 @@ static const char *const TAG = "light"; } \ LightCall &LightCall::set_##name(type name) { \ this->name##_ = name; \ - this->set_flag_(flag, true); \ + this->set_flag_(flag); \ return *this; \ } @@ -44,11 +71,21 @@ static const LogString *color_mode_to_human(ColorMode color_mode) { return LOG_STR(""); } +// Helper to log percentage values +#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_DEBUG +static void log_percent(const char *name, const char *param, float value) { + ESP_LOGD(TAG, " %s: %.0f%%", param, value * 100.0f); +} +#else +#define log_percent(name, param, value) +#endif + void LightCall::perform() { const char *name = this->parent_->get_name().c_str(); LightColorValues v = this->validate_(); + const bool publish = this->get_publish_(); - if (this->get_publish_()) { + if (publish) { ESP_LOGD(TAG, "'%s' Setting:", name); // Only print color mode when it's being changed @@ -66,11 +103,11 @@ void LightCall::perform() { } if (this->has_brightness()) { - ESP_LOGD(TAG, " Brightness: %.0f%%", v.get_brightness() * 100.0f); + log_percent(name, "Brightness", v.get_brightness()); } if (this->has_color_brightness()) { - ESP_LOGD(TAG, " Color brightness: %.0f%%", v.get_color_brightness() * 100.0f); + log_percent(name, "Color brightness", v.get_color_brightness()); } if (this->has_red() || this->has_green() || this->has_blue()) { ESP_LOGD(TAG, " Red: %.0f%%, Green: %.0f%%, Blue: %.0f%%", v.get_red() * 100.0f, v.get_green() * 100.0f, @@ -78,7 +115,7 @@ void LightCall::perform() { } if (this->has_white()) { - ESP_LOGD(TAG, " White: %.0f%%", v.get_white() * 100.0f); + log_percent(name, "White", v.get_white()); } if (this->has_color_temperature()) { ESP_LOGD(TAG, " Color temperature: %.1f mireds", v.get_color_temperature()); @@ -92,26 +129,26 @@ void LightCall::perform() { if (this->has_flash_()) { // FLASH - if (this->get_publish_()) { + if (publish) { ESP_LOGD(TAG, " Flash length: %.1fs", this->flash_length_ / 1e3f); } - this->parent_->start_flash_(v, this->flash_length_, this->get_publish_()); + this->parent_->start_flash_(v, this->flash_length_, publish); } else if (this->has_transition_()) { // TRANSITION - if (this->get_publish_()) { + if (publish) { ESP_LOGD(TAG, " Transition length: %.1fs", this->transition_length_ / 1e3f); } // Special case: Transition and effect can be set when turning off if (this->has_effect_()) { - if (this->get_publish_()) { + if (publish) { ESP_LOGD(TAG, " Effect: 'None'"); } this->parent_->stop_effect_(); } - this->parent_->start_transition_(v, this->transition_length_, this->get_publish_()); + this->parent_->start_transition_(v, this->transition_length_, publish); } else if (this->has_effect_()) { // EFFECT @@ -122,7 +159,7 @@ void LightCall::perform() { effect_s = this->parent_->effects_[this->effect_ - 1]->get_name().c_str(); } - if (this->get_publish_()) { + if (publish) { ESP_LOGD(TAG, " Effect: '%s'", effect_s); } @@ -133,13 +170,13 @@ void LightCall::perform() { this->parent_->set_immediately_(v, true); } else { // INSTANT CHANGE - this->parent_->set_immediately_(v, this->get_publish_()); + this->parent_->set_immediately_(v, publish); } if (!this->has_transition_()) { this->parent_->target_state_reached_callback_.call(); } - if (this->get_publish_()) { + if (publish) { this->parent_->publish_state(); } if (this->get_save_()) { @@ -147,6 +184,16 @@ void LightCall::perform() { } } +void LightCall::log_and_clear_unsupported_(FieldFlags flag, const LogString *feature, bool use_color_mode_log) { + auto *name = this->parent_->get_name().c_str(); + if (use_color_mode_log) { + log_color_mode_not_supported(name, feature); + } else { + log_feature_not_supported(name, feature); + } + this->clear_flag_(flag); +} + LightColorValues LightCall::validate_() { auto *name = this->parent_->get_name().c_str(); auto traits = this->parent_->get_traits(); @@ -154,142 +201,108 @@ LightColorValues LightCall::validate_() { // Color mode check if (this->has_color_mode() && !traits.supports_color_mode(this->color_mode_)) { ESP_LOGW(TAG, "'%s' does not support color mode %s", name, LOG_STR_ARG(color_mode_to_human(this->color_mode_))); - this->set_flag_(FLAG_HAS_COLOR_MODE, false); + this->clear_flag_(FLAG_HAS_COLOR_MODE); } // Ensure there is always a color mode set if (!this->has_color_mode()) { this->color_mode_ = this->compute_color_mode_(); - this->set_flag_(FLAG_HAS_COLOR_MODE, true); + this->set_flag_(FLAG_HAS_COLOR_MODE); } auto color_mode = this->color_mode_; // Transform calls that use non-native parameters for the current mode. this->transform_parameters_(); - // Brightness exists check - if (this->has_brightness() && this->brightness_ > 0.0f && !(color_mode & ColorCapability::BRIGHTNESS)) { - ESP_LOGW(TAG, "'%s': setting brightness not supported", name); - this->set_flag_(FLAG_HAS_BRIGHTNESS, false); - } - - // Transition length possible check - if (this->has_transition_() && this->transition_length_ != 0 && !(color_mode & ColorCapability::BRIGHTNESS)) { - ESP_LOGW(TAG, "'%s': transitions not supported", name); - this->set_flag_(FLAG_HAS_TRANSITION, false); - } - - // Color brightness exists check - if (this->has_color_brightness() && this->color_brightness_ > 0.0f && !(color_mode & ColorCapability::RGB)) { - ESP_LOGW(TAG, "'%s': color mode does not support setting RGB brightness", name); - this->set_flag_(FLAG_HAS_COLOR_BRIGHTNESS, false); - } - - // RGB exists check - if ((this->has_red() && this->red_ > 0.0f) || (this->has_green() && this->green_ > 0.0f) || - (this->has_blue() && this->blue_ > 0.0f)) { - if (!(color_mode & ColorCapability::RGB)) { - ESP_LOGW(TAG, "'%s': color mode does not support setting RGB color", name); - this->set_flag_(FLAG_HAS_RED, false); - this->set_flag_(FLAG_HAS_GREEN, false); - this->set_flag_(FLAG_HAS_BLUE, false); - } - } - - // White value exists check - if (this->has_white() && this->white_ > 0.0f && - !(color_mode & ColorCapability::WHITE || color_mode & ColorCapability::COLD_WARM_WHITE)) { - ESP_LOGW(TAG, "'%s': color mode does not support setting white value", name); - this->set_flag_(FLAG_HAS_WHITE, false); - } - - // Color temperature exists check - if (this->has_color_temperature() && - !(color_mode & ColorCapability::COLOR_TEMPERATURE || color_mode & ColorCapability::COLD_WARM_WHITE)) { - ESP_LOGW(TAG, "'%s': color mode does not support setting color temperature", name); - this->set_flag_(FLAG_HAS_COLOR_TEMPERATURE, false); - } - - // Cold/warm white value exists check - if ((this->has_cold_white() && this->cold_white_ > 0.0f) || (this->has_warm_white() && this->warm_white_ > 0.0f)) { - if (!(color_mode & ColorCapability::COLD_WARM_WHITE)) { - ESP_LOGW(TAG, "'%s': color mode does not support setting cold/warm white value", name); - this->set_flag_(FLAG_HAS_COLD_WHITE, false); - this->set_flag_(FLAG_HAS_WARM_WHITE, false); - } - } - -#define VALIDATE_RANGE_(name_, upper_name, min, max) \ - if (this->has_##name_()) { \ - auto val = this->name_##_; \ - if (val < (min) || val > (max)) { \ - ESP_LOGW(TAG, "'%s': %s value %.2f is out of range [%.1f - %.1f]", name, LOG_STR_LITERAL(upper_name), val, \ - (min), (max)); \ - this->name_##_ = clamp(val, (min), (max)); \ - } \ - } -#define VALIDATE_RANGE(name, upper_name) VALIDATE_RANGE_(name, upper_name, 0.0f, 1.0f) - - // Range checks - VALIDATE_RANGE(brightness, "Brightness") - VALIDATE_RANGE(color_brightness, "Color brightness") - VALIDATE_RANGE(red, "Red") - VALIDATE_RANGE(green, "Green") - VALIDATE_RANGE(blue, "Blue") - VALIDATE_RANGE(white, "White") - VALIDATE_RANGE(cold_white, "Cold white") - VALIDATE_RANGE(warm_white, "Warm white") - VALIDATE_RANGE_(color_temperature, "Color temperature", traits.get_min_mireds(), traits.get_max_mireds()) - + // Business logic adjustments before validation // Flag whether an explicit turn off was requested, in which case we'll also stop the effect. bool explicit_turn_off_request = this->has_state() && !this->state_; // Turn off when brightness is set to zero, and reset brightness (so that it has nonzero brightness when turned on). if (this->has_brightness() && this->brightness_ == 0.0f) { this->state_ = false; - this->set_flag_(FLAG_HAS_STATE, true); + this->set_flag_(FLAG_HAS_STATE); this->brightness_ = 1.0f; } // Set color brightness to 100% if currently zero and a color is set. - if (this->has_red() || this->has_green() || this->has_blue()) { - if (!this->has_color_brightness() && this->parent_->remote_values.get_color_brightness() == 0.0f) { - this->color_brightness_ = 1.0f; - this->set_flag_(FLAG_HAS_COLOR_BRIGHTNESS, true); - } + if ((this->has_red() || this->has_green() || this->has_blue()) && !this->has_color_brightness() && + this->parent_->remote_values.get_color_brightness() == 0.0f) { + this->color_brightness_ = 1.0f; + this->set_flag_(FLAG_HAS_COLOR_BRIGHTNESS); } - // Create color values for the light with this call applied. + // Capability validation + if (this->has_brightness() && this->brightness_ > 0.0f && !(color_mode & ColorCapability::BRIGHTNESS)) + this->log_and_clear_unsupported_(FLAG_HAS_BRIGHTNESS, LOG_STR("brightness"), false); + + // Transition length possible check + if (this->has_transition_() && this->transition_length_ != 0 && !(color_mode & ColorCapability::BRIGHTNESS)) + this->log_and_clear_unsupported_(FLAG_HAS_TRANSITION, LOG_STR("transitions"), false); + + if (this->has_color_brightness() && this->color_brightness_ > 0.0f && !(color_mode & ColorCapability::RGB)) + this->log_and_clear_unsupported_(FLAG_HAS_COLOR_BRIGHTNESS, LOG_STR("RGB brightness"), true); + + // RGB exists check + if (((this->has_red() && this->red_ > 0.0f) || (this->has_green() && this->green_ > 0.0f) || + (this->has_blue() && this->blue_ > 0.0f)) && + !(color_mode & ColorCapability::RGB)) { + log_color_mode_not_supported(name, LOG_STR("RGB color")); + this->clear_flag_(FLAG_HAS_RED); + this->clear_flag_(FLAG_HAS_GREEN); + this->clear_flag_(FLAG_HAS_BLUE); + } + + // White value exists check + if (this->has_white() && this->white_ > 0.0f && + !(color_mode & ColorCapability::WHITE || color_mode & ColorCapability::COLD_WARM_WHITE)) + this->log_and_clear_unsupported_(FLAG_HAS_WHITE, LOG_STR("white value"), true); + + // Color temperature exists check + if (this->has_color_temperature() && + !(color_mode & ColorCapability::COLOR_TEMPERATURE || color_mode & ColorCapability::COLD_WARM_WHITE)) + this->log_and_clear_unsupported_(FLAG_HAS_COLOR_TEMPERATURE, LOG_STR("color temperature"), true); + + // Cold/warm white value exists check + if (((this->has_cold_white() && this->cold_white_ > 0.0f) || (this->has_warm_white() && this->warm_white_ > 0.0f)) && + !(color_mode & ColorCapability::COLD_WARM_WHITE)) { + log_color_mode_not_supported(name, LOG_STR("cold/warm white value")); + this->clear_flag_(FLAG_HAS_COLD_WHITE); + this->clear_flag_(FLAG_HAS_WARM_WHITE); + } + + // Create color values and validate+apply ranges in one step to eliminate duplicate checks auto v = this->parent_->remote_values; if (this->has_color_mode()) v.set_color_mode(this->color_mode_); if (this->has_state()) v.set_state(this->state_); - if (this->has_brightness()) - v.set_brightness(this->brightness_); - if (this->has_color_brightness()) - v.set_color_brightness(this->color_brightness_); - if (this->has_red()) - v.set_red(this->red_); - if (this->has_green()) - v.set_green(this->green_); - if (this->has_blue()) - v.set_blue(this->blue_); - if (this->has_white()) - v.set_white(this->white_); - if (this->has_color_temperature()) - v.set_color_temperature(this->color_temperature_); - if (this->has_cold_white()) - v.set_cold_white(this->cold_white_); - if (this->has_warm_white()) - v.set_warm_white(this->warm_white_); + +#define VALIDATE_AND_APPLY(field, setter, name_str, ...) \ + if (this->has_##field()) { \ + clamp_and_log_if_invalid(name, this->field##_, LOG_STR(name_str), ##__VA_ARGS__); \ + v.setter(this->field##_); \ + } + + VALIDATE_AND_APPLY(brightness, set_brightness, "Brightness") + VALIDATE_AND_APPLY(color_brightness, set_color_brightness, "Color brightness") + VALIDATE_AND_APPLY(red, set_red, "Red") + VALIDATE_AND_APPLY(green, set_green, "Green") + VALIDATE_AND_APPLY(blue, set_blue, "Blue") + VALIDATE_AND_APPLY(white, set_white, "White") + VALIDATE_AND_APPLY(cold_white, set_cold_white, "Cold white") + VALIDATE_AND_APPLY(warm_white, set_warm_white, "Warm white") + VALIDATE_AND_APPLY(color_temperature, set_color_temperature, "Color temperature", traits.get_min_mireds(), + traits.get_max_mireds()) + +#undef VALIDATE_AND_APPLY v.normalize_color(); // Flash length check if (this->has_flash_() && this->flash_length_ == 0) { - ESP_LOGW(TAG, "'%s': flash length must be greater than zero", name); - this->set_flag_(FLAG_HAS_FLASH, false); + log_invalid_parameter(name, LOG_STR("flash length must be >0")); + this->clear_flag_(FLAG_HAS_FLASH); } // validate transition length/flash length/effect not used at the same time @@ -297,42 +310,40 @@ LightColorValues LightCall::validate_() { // If effect is already active, remove effect start if (this->has_effect_() && this->effect_ == this->parent_->active_effect_index_) { - this->set_flag_(FLAG_HAS_EFFECT, false); + this->clear_flag_(FLAG_HAS_EFFECT); } // validate effect index if (this->has_effect_() && this->effect_ > this->parent_->effects_.size()) { ESP_LOGW(TAG, "'%s': invalid effect index %" PRIu32, name, this->effect_); - this->set_flag_(FLAG_HAS_EFFECT, false); + this->clear_flag_(FLAG_HAS_EFFECT); } if (this->has_effect_() && (this->has_transition_() || this->has_flash_())) { - ESP_LOGW(TAG, "'%s': effect cannot be used with transition/flash", name); - this->set_flag_(FLAG_HAS_TRANSITION, false); - this->set_flag_(FLAG_HAS_FLASH, false); + log_invalid_parameter(name, LOG_STR("effect cannot be used with transition/flash")); + this->clear_flag_(FLAG_HAS_TRANSITION); + this->clear_flag_(FLAG_HAS_FLASH); } if (this->has_flash_() && this->has_transition_()) { - ESP_LOGW(TAG, "'%s': flash cannot be used with transition", name); - this->set_flag_(FLAG_HAS_TRANSITION, false); + log_invalid_parameter(name, LOG_STR("flash cannot be used with transition")); + this->clear_flag_(FLAG_HAS_TRANSITION); } if (!this->has_transition_() && !this->has_flash_() && (!this->has_effect_() || this->effect_ == 0) && supports_transition) { // nothing specified and light supports transitions, set default transition length this->transition_length_ = this->parent_->default_transition_length_; - this->set_flag_(FLAG_HAS_TRANSITION, true); + this->set_flag_(FLAG_HAS_TRANSITION); } if (this->has_transition_() && this->transition_length_ == 0) { // 0 transition is interpreted as no transition (instant change) - this->set_flag_(FLAG_HAS_TRANSITION, false); + this->clear_flag_(FLAG_HAS_TRANSITION); } - if (this->has_transition_() && !supports_transition) { - ESP_LOGW(TAG, "'%s': transitions not supported", name); - this->set_flag_(FLAG_HAS_TRANSITION, false); - } + if (this->has_transition_() && !supports_transition) + this->log_and_clear_unsupported_(FLAG_HAS_TRANSITION, LOG_STR("transitions"), false); // If not a flash and turning the light off, then disable the light // Do not use light color values directly, so that effects can set 0% brightness @@ -340,18 +351,18 @@ LightColorValues LightCall::validate_() { bool target_state = this->has_state() ? this->state_ : v.is_on(); if (!this->has_flash_() && !target_state) { if (this->has_effect_()) { - ESP_LOGW(TAG, "'%s': cannot start effect when turning off", name); - this->set_flag_(FLAG_HAS_EFFECT, false); + log_invalid_parameter(name, LOG_STR("cannot start effect when turning off")); + this->clear_flag_(FLAG_HAS_EFFECT); } else if (this->parent_->active_effect_index_ != 0 && explicit_turn_off_request) { // Auto turn off effect this->effect_ = 0; - this->set_flag_(FLAG_HAS_EFFECT, true); + this->set_flag_(FLAG_HAS_EFFECT); } } // Disable saving for flashes if (this->has_flash_()) - this->set_flag_(FLAG_SAVE, false); + this->clear_flag_(FLAG_SAVE); return v; } @@ -364,27 +375,33 @@ void LightCall::transform_parameters_() { // - RGBWW lights with color_interlock=true, which also sets "brightness" and // "color_temperature" (without color_interlock, CW/WW are set directly) // - Legacy Home Assistant (pre-colormode), which sets "white" and "color_temperature" + + // Cache min/max mireds to avoid repeated calls + const float min_mireds = traits.get_min_mireds(); + const float max_mireds = traits.get_max_mireds(); + if (((this->has_white() && this->white_ > 0.0f) || this->has_color_temperature()) && // (this->color_mode_ & ColorCapability::COLD_WARM_WHITE) && // !(this->color_mode_ & ColorCapability::WHITE) && // !(this->color_mode_ & ColorCapability::COLOR_TEMPERATURE) && // - traits.get_min_mireds() > 0.0f && traits.get_max_mireds() > 0.0f) { + min_mireds > 0.0f && max_mireds > 0.0f) { ESP_LOGD(TAG, "'%s': setting cold/warm white channels using white/color temperature values", this->parent_->get_name().c_str()); if (this->has_color_temperature()) { - const float color_temp = clamp(this->color_temperature_, traits.get_min_mireds(), traits.get_max_mireds()); - const float ww_fraction = - (color_temp - traits.get_min_mireds()) / (traits.get_max_mireds() - traits.get_min_mireds()); + const float color_temp = clamp(this->color_temperature_, min_mireds, max_mireds); + const float range = max_mireds - min_mireds; + const float ww_fraction = (color_temp - min_mireds) / range; const float cw_fraction = 1.0f - ww_fraction; const float max_cw_ww = std::max(ww_fraction, cw_fraction); - this->cold_white_ = gamma_uncorrect(cw_fraction / max_cw_ww, this->parent_->get_gamma_correct()); - this->warm_white_ = gamma_uncorrect(ww_fraction / max_cw_ww, this->parent_->get_gamma_correct()); - this->set_flag_(FLAG_HAS_COLD_WHITE, true); - this->set_flag_(FLAG_HAS_WARM_WHITE, true); + const float gamma = this->parent_->get_gamma_correct(); + this->cold_white_ = gamma_uncorrect(cw_fraction / max_cw_ww, gamma); + this->warm_white_ = gamma_uncorrect(ww_fraction / max_cw_ww, gamma); + this->set_flag_(FLAG_HAS_COLD_WHITE); + this->set_flag_(FLAG_HAS_WARM_WHITE); } if (this->has_white()) { this->brightness_ = this->white_; - this->set_flag_(FLAG_HAS_BRIGHTNESS, true); + this->set_flag_(FLAG_HAS_BRIGHTNESS); } } } @@ -442,41 +459,39 @@ std::set LightCall::get_suitable_color_modes_() { bool has_rgb = (this->has_color_brightness() && this->color_brightness_ > 0.0f) || (this->has_red() || this->has_green() || this->has_blue()); +// Build key from flags: [rgb][cwww][ct][white] #define KEY(white, ct, cwww, rgb) ((white) << 0 | (ct) << 1 | (cwww) << 2 | (rgb) << 3) -#define ENTRY(white, ct, cwww, rgb, ...) \ - std::make_tuple>(KEY(white, ct, cwww, rgb), __VA_ARGS__) - // Flag order: white, color temperature, cwww, rgb - std::array>, 10> lookup_table{ - ENTRY(true, false, false, false, - {ColorMode::WHITE, ColorMode::RGB_WHITE, ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::COLD_WARM_WHITE, - ColorMode::RGB_COLD_WARM_WHITE}), - ENTRY(false, true, false, false, - {ColorMode::COLOR_TEMPERATURE, ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::COLD_WARM_WHITE, - ColorMode::RGB_COLD_WARM_WHITE}), - ENTRY(true, true, false, false, - {ColorMode::COLD_WARM_WHITE, ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::RGB_COLD_WARM_WHITE}), - ENTRY(false, false, true, false, {ColorMode::COLD_WARM_WHITE, ColorMode::RGB_COLD_WARM_WHITE}), - ENTRY(false, false, false, false, - {ColorMode::RGB_WHITE, ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::RGB_COLD_WARM_WHITE, ColorMode::RGB, - ColorMode::WHITE, ColorMode::COLOR_TEMPERATURE, ColorMode::COLD_WARM_WHITE}), - ENTRY(true, false, false, true, - {ColorMode::RGB_WHITE, ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::RGB_COLD_WARM_WHITE}), - ENTRY(false, true, false, true, {ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::RGB_COLD_WARM_WHITE}), - ENTRY(true, true, false, true, {ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::RGB_COLD_WARM_WHITE}), - ENTRY(false, false, true, true, {ColorMode::RGB_COLD_WARM_WHITE}), - ENTRY(false, false, false, true, - {ColorMode::RGB, ColorMode::RGB_WHITE, ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::RGB_COLD_WARM_WHITE}), - }; + uint8_t key = KEY(has_white, has_ct, has_cwww, has_rgb); - auto key = KEY(has_white, has_ct, has_cwww, has_rgb); - for (auto &item : lookup_table) { - if (std::get<0>(item) == key) - return std::get<1>(item); + switch (key) { + case KEY(true, false, false, false): // white only + return {ColorMode::WHITE, ColorMode::RGB_WHITE, ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::COLD_WARM_WHITE, + ColorMode::RGB_COLD_WARM_WHITE}; + case KEY(false, true, false, false): // ct only + return {ColorMode::COLOR_TEMPERATURE, ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::COLD_WARM_WHITE, + ColorMode::RGB_COLD_WARM_WHITE}; + case KEY(true, true, false, false): // white + ct + return {ColorMode::COLD_WARM_WHITE, ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::RGB_COLD_WARM_WHITE}; + case KEY(false, false, true, false): // cwww only + return {ColorMode::COLD_WARM_WHITE, ColorMode::RGB_COLD_WARM_WHITE}; + case KEY(false, false, false, false): // none + return {ColorMode::RGB_WHITE, ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::RGB_COLD_WARM_WHITE, ColorMode::RGB, + ColorMode::WHITE, ColorMode::COLOR_TEMPERATURE, ColorMode::COLD_WARM_WHITE}; + case KEY(true, false, false, true): // rgb + white + return {ColorMode::RGB_WHITE, ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::RGB_COLD_WARM_WHITE}; + case KEY(false, true, false, true): // rgb + ct + case KEY(true, true, false, true): // rgb + white + ct + return {ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::RGB_COLD_WARM_WHITE}; + case KEY(false, false, true, true): // rgb + cwww + return {ColorMode::RGB_COLD_WARM_WHITE}; + case KEY(false, false, false, true): // rgb only + return {ColorMode::RGB, ColorMode::RGB_WHITE, ColorMode::RGB_COLOR_TEMPERATURE, ColorMode::RGB_COLD_WARM_WHITE}; + default: + return {}; // conflicting flags } - // This happens if there are conflicting flags given. - return {}; +#undef KEY } LightCall &LightCall::set_effect(const std::string &effect) { @@ -593,7 +608,7 @@ LightCall &LightCall::set_effect(optional effect) { } LightCall &LightCall::set_effect(uint32_t effect_number) { this->effect_ = effect_number; - this->set_flag_(FLAG_HAS_EFFECT, true); + this->set_flag_(FLAG_HAS_EFFECT); return *this; } LightCall &LightCall::set_effect(optional effect_number) { diff --git a/esphome/components/light/light_call.h b/esphome/components/light/light_call.h index 7e04e1a767..d3a526b136 100644 --- a/esphome/components/light/light_call.h +++ b/esphome/components/light/light_call.h @@ -4,6 +4,10 @@ #include namespace esphome { + +// Forward declaration +struct LogString; + namespace light { class LightState; @@ -207,14 +211,14 @@ class LightCall { FLAG_SAVE = 1 << 15, }; - bool has_transition_() { return (this->flags_ & FLAG_HAS_TRANSITION) != 0; } - bool has_flash_() { return (this->flags_ & FLAG_HAS_FLASH) != 0; } - bool has_effect_() { return (this->flags_ & FLAG_HAS_EFFECT) != 0; } - bool get_publish_() { return (this->flags_ & FLAG_PUBLISH) != 0; } - bool get_save_() { return (this->flags_ & FLAG_SAVE) != 0; } + inline bool has_transition_() { return (this->flags_ & FLAG_HAS_TRANSITION) != 0; } + inline bool has_flash_() { return (this->flags_ & FLAG_HAS_FLASH) != 0; } + inline bool has_effect_() { return (this->flags_ & FLAG_HAS_EFFECT) != 0; } + inline bool get_publish_() { return (this->flags_ & FLAG_PUBLISH) != 0; } + inline bool get_save_() { return (this->flags_ & FLAG_SAVE) != 0; } - // Helper to set flag - void set_flag_(FieldFlags flag, bool value) { + // Helper to set flag - defaults to true for common case + void set_flag_(FieldFlags flag, bool value = true) { if (value) { this->flags_ |= flag; } else { @@ -222,6 +226,12 @@ class LightCall { } } + // Helper to clear flag - reduces code size for common case + void clear_flag_(FieldFlags flag) { this->flags_ &= ~flag; } + + // Helper to log unsupported feature and clear flag - reduces code duplication + void log_and_clear_unsupported_(FieldFlags flag, const LogString *feature, bool use_color_mode_log); + LightState *parent_; // Light state values - use flags_ to check if a value has been set. diff --git a/esphome/components/light/light_color_values.h b/esphome/components/light/light_color_values.h index 5653a8d2a5..04d7d1e7d8 100644 --- a/esphome/components/light/light_color_values.h +++ b/esphome/components/light/light_color_values.h @@ -84,18 +84,23 @@ class LightColorValues { * @return The linearly interpolated LightColorValues. */ static LightColorValues lerp(const LightColorValues &start, const LightColorValues &end, float completion) { + // Directly interpolate the raw values to avoid getter/setter overhead. + // This is safe because: + // - All LightColorValues have their values clamped when set via the setters + // - std::lerp guarantees output is in the same range as inputs + // - Therefore the output doesn't need clamping, so we can skip the setters LightColorValues v; - v.set_color_mode(end.color_mode_); - v.set_state(std::lerp(start.get_state(), end.get_state(), completion)); - v.set_brightness(std::lerp(start.get_brightness(), end.get_brightness(), completion)); - v.set_color_brightness(std::lerp(start.get_color_brightness(), end.get_color_brightness(), completion)); - v.set_red(std::lerp(start.get_red(), end.get_red(), completion)); - v.set_green(std::lerp(start.get_green(), end.get_green(), completion)); - v.set_blue(std::lerp(start.get_blue(), end.get_blue(), completion)); - v.set_white(std::lerp(start.get_white(), end.get_white(), completion)); - v.set_color_temperature(std::lerp(start.get_color_temperature(), end.get_color_temperature(), completion)); - v.set_cold_white(std::lerp(start.get_cold_white(), end.get_cold_white(), completion)); - v.set_warm_white(std::lerp(start.get_warm_white(), end.get_warm_white(), completion)); + v.color_mode_ = end.color_mode_; + v.state_ = std::lerp(start.state_, end.state_, completion); + v.brightness_ = std::lerp(start.brightness_, end.brightness_, completion); + v.color_brightness_ = std::lerp(start.color_brightness_, end.color_brightness_, completion); + v.red_ = std::lerp(start.red_, end.red_, completion); + v.green_ = std::lerp(start.green_, end.green_, completion); + v.blue_ = std::lerp(start.blue_, end.blue_, completion); + v.white_ = std::lerp(start.white_, end.white_, completion); + v.color_temperature_ = std::lerp(start.color_temperature_, end.color_temperature_, completion); + v.cold_white_ = std::lerp(start.cold_white_, end.cold_white_, completion); + v.warm_white_ = std::lerp(start.warm_white_, end.warm_white_, completion); return v; } diff --git a/esphome/components/light/light_effect.cpp b/esphome/components/light/light_effect.cpp new file mode 100644 index 0000000000..a210b48e5b --- /dev/null +++ b/esphome/components/light/light_effect.cpp @@ -0,0 +1,36 @@ +#include "light_effect.h" +#include "light_state.h" + +namespace esphome { +namespace light { + +uint32_t LightEffect::get_index() const { + if (this->state_ == nullptr) { + return 0; + } + return this->get_index_in_parent_(); +} + +bool LightEffect::is_active() const { + if (this->state_ == nullptr) { + return false; + } + return this->get_index() != 0 && this->state_->get_current_effect_index() == this->get_index(); +} + +uint32_t LightEffect::get_index_in_parent_() const { + if (this->state_ == nullptr) { + return 0; + } + + const auto &effects = this->state_->get_effects(); + for (size_t i = 0; i < effects.size(); i++) { + if (effects[i] == this) { + return i + 1; // Effects are 1-indexed in the API + } + } + return 0; // Not found +} + +} // namespace light +} // namespace esphome diff --git a/esphome/components/light/light_effect.h b/esphome/components/light/light_effect.h index 8da51fe8b3..dbaf1faf24 100644 --- a/esphome/components/light/light_effect.h +++ b/esphome/components/light/light_effect.h @@ -34,9 +34,23 @@ class LightEffect { this->init(); } + /// Get the index of this effect in the parent light's effect list. + /// Returns 0 if not found or not initialized. + uint32_t get_index() const; + + /// Check if this effect is currently active. + bool is_active() const; + + /// Get a reference to the parent light state. + /// Returns nullptr if not initialized. + LightState *get_light_state() const { return this->state_; } + protected: LightState *state_{nullptr}; std::string name_; + + /// Internal method to find this effect's index in the parent light's effect list. + uint32_t get_index_in_parent_() const; }; } // namespace light diff --git a/esphome/components/light/light_json_schema.cpp b/esphome/components/light/light_json_schema.cpp index 26615bae5c..010e130612 100644 --- a/esphome/components/light/light_json_schema.cpp +++ b/esphome/components/light/light_json_schema.cpp @@ -8,68 +8,73 @@ namespace light { // See https://www.home-assistant.io/integrations/light.mqtt/#json-schema for documentation on the schema +// Lookup table for color mode strings +static constexpr const char *get_color_mode_json_str(ColorMode mode) { + switch (mode) { + case ColorMode::ON_OFF: + return "onoff"; + case ColorMode::BRIGHTNESS: + return "brightness"; + case ColorMode::WHITE: + return "white"; // not supported by HA in MQTT + case ColorMode::COLOR_TEMPERATURE: + return "color_temp"; + case ColorMode::COLD_WARM_WHITE: + return "cwww"; // not supported by HA + case ColorMode::RGB: + return "rgb"; + case ColorMode::RGB_WHITE: + return "rgbw"; + case ColorMode::RGB_COLOR_TEMPERATURE: + return "rgbct"; // not supported by HA + case ColorMode::RGB_COLD_WARM_WHITE: + return "rgbww"; + default: + return nullptr; + } +} + void LightJSONSchema::dump_json(LightState &state, JsonObject root) { // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson - if (state.supports_effects()) + if (state.supports_effects()) { root["effect"] = state.get_effect_name(); + root["effect_index"] = state.get_current_effect_index(); + root["effect_count"] = state.get_effect_count(); + } auto values = state.remote_values; auto traits = state.get_output()->get_traits(); - switch (values.get_color_mode()) { - case ColorMode::UNKNOWN: // don't need to set color mode if we don't know it - break; - case ColorMode::ON_OFF: - root["color_mode"] = "onoff"; - break; - case ColorMode::BRIGHTNESS: - root["color_mode"] = "brightness"; - break; - case ColorMode::WHITE: // not supported by HA in MQTT - root["color_mode"] = "white"; - break; - case ColorMode::COLOR_TEMPERATURE: - root["color_mode"] = "color_temp"; - break; - case ColorMode::COLD_WARM_WHITE: // not supported by HA - root["color_mode"] = "cwww"; - break; - case ColorMode::RGB: - root["color_mode"] = "rgb"; - break; - case ColorMode::RGB_WHITE: - root["color_mode"] = "rgbw"; - break; - case ColorMode::RGB_COLOR_TEMPERATURE: // not supported by HA - root["color_mode"] = "rgbct"; - break; - case ColorMode::RGB_COLD_WARM_WHITE: - root["color_mode"] = "rgbww"; - break; + const auto color_mode = values.get_color_mode(); + const char *mode_str = get_color_mode_json_str(color_mode); + if (mode_str != nullptr) { + root["color_mode"] = mode_str; } - if (values.get_color_mode() & ColorCapability::ON_OFF) + if (color_mode & ColorCapability::ON_OFF) root["state"] = (values.get_state() != 0.0f) ? "ON" : "OFF"; - if (values.get_color_mode() & ColorCapability::BRIGHTNESS) - root["brightness"] = uint8_t(values.get_brightness() * 255); + if (color_mode & ColorCapability::BRIGHTNESS) + root["brightness"] = to_uint8_scale(values.get_brightness()); JsonObject color = root["color"].to(); - if (values.get_color_mode() & ColorCapability::RGB) { - color["r"] = uint8_t(values.get_color_brightness() * values.get_red() * 255); - color["g"] = uint8_t(values.get_color_brightness() * values.get_green() * 255); - color["b"] = uint8_t(values.get_color_brightness() * values.get_blue() * 255); + if (color_mode & ColorCapability::RGB) { + float color_brightness = values.get_color_brightness(); + color["r"] = to_uint8_scale(color_brightness * values.get_red()); + color["g"] = to_uint8_scale(color_brightness * values.get_green()); + color["b"] = to_uint8_scale(color_brightness * values.get_blue()); } - if (values.get_color_mode() & ColorCapability::WHITE) { - color["w"] = uint8_t(values.get_white() * 255); - root["white_value"] = uint8_t(values.get_white() * 255); // legacy API + if (color_mode & ColorCapability::WHITE) { + uint8_t white_val = to_uint8_scale(values.get_white()); + color["w"] = white_val; + root["white_value"] = white_val; // legacy API } - if (values.get_color_mode() & ColorCapability::COLOR_TEMPERATURE) { + if (color_mode & ColorCapability::COLOR_TEMPERATURE) { // this one isn't under the color subkey for some reason root["color_temp"] = uint32_t(values.get_color_temperature()); } - if (values.get_color_mode() & ColorCapability::COLD_WARM_WHITE) { - color["c"] = uint8_t(values.get_cold_white() * 255); - color["w"] = uint8_t(values.get_warm_white() * 255); + if (color_mode & ColorCapability::COLD_WARM_WHITE) { + color["c"] = to_uint8_scale(values.get_cold_white()); + color["w"] = to_uint8_scale(values.get_warm_white()); } } @@ -158,6 +163,11 @@ void LightJSONSchema::parse_json(LightState &state, LightCall &call, JsonObject const char *effect = root["effect"]; call.set_effect(effect); } + + if (root["effect_index"].is()) { + uint32_t effect_index = root["effect_index"]; + call.set_effect(effect_index); + } } } // namespace light diff --git a/esphome/components/light/light_state.cpp b/esphome/components/light/light_state.cpp index 0aae6aed15..1d139e49e7 100644 --- a/esphome/components/light/light_state.cpp +++ b/esphome/components/light/light_state.cpp @@ -18,15 +18,14 @@ LightCall LightState::toggle() { return this->make_call().set_state(!this->remot LightCall LightState::make_call() { return LightCall(this); } void LightState::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->get_name().c_str()); - this->output_->setup_state(this); for (auto *effect : this->effects_) { effect->init_internal(this); } // When supported color temperature range is known, initialize color temperature setting within bounds. - float min_mireds = this->get_traits().get_min_mireds(); + auto traits = this->get_traits(); + float min_mireds = traits.get_min_mireds(); if (min_mireds > 0) { this->remote_values.set_color_temperature(min_mireds); this->current_values.set_color_temperature(min_mireds); @@ -42,14 +41,11 @@ void LightState::setup() { case LIGHT_RESTORE_DEFAULT_ON: case LIGHT_RESTORE_INVERTED_DEFAULT_OFF: case LIGHT_RESTORE_INVERTED_DEFAULT_ON: - this->rtc_ = global_preferences->make_preference(this->get_object_id_hash()); + this->rtc_ = global_preferences->make_preference(this->get_preference_hash()); // Attempt to load from preferences, else fall back to default values if (!this->rtc_.load(&recovered)) { - recovered.state = false; - if (this->restore_mode_ == LIGHT_RESTORE_DEFAULT_ON || - this->restore_mode_ == LIGHT_RESTORE_INVERTED_DEFAULT_ON) { - recovered.state = true; - } + recovered.state = (this->restore_mode_ == LIGHT_RESTORE_DEFAULT_ON || + this->restore_mode_ == LIGHT_RESTORE_INVERTED_DEFAULT_ON); } else if (this->restore_mode_ == LIGHT_RESTORE_INVERTED_DEFAULT_OFF || this->restore_mode_ == LIGHT_RESTORE_INVERTED_DEFAULT_ON) { // Inverted restore state @@ -58,7 +54,7 @@ void LightState::setup() { break; case LIGHT_RESTORE_AND_OFF: case LIGHT_RESTORE_AND_ON: - this->rtc_ = global_preferences->make_preference(this->get_object_id_hash()); + this->rtc_ = global_preferences->make_preference(this->get_preference_hash()); this->rtc_.load(&recovered); recovered.state = (this->restore_mode_ == LIGHT_RESTORE_AND_ON); break; @@ -90,17 +86,18 @@ void LightState::setup() { } void LightState::dump_config() { ESP_LOGCONFIG(TAG, "Light '%s'", this->get_name().c_str()); - if (this->get_traits().supports_color_capability(ColorCapability::BRIGHTNESS)) { + auto traits = this->get_traits(); + if (traits.supports_color_capability(ColorCapability::BRIGHTNESS)) { ESP_LOGCONFIG(TAG, " Default Transition Length: %.1fs\n" " Gamma Correct: %.2f", this->default_transition_length_ / 1e3f, this->gamma_correct_); } - if (this->get_traits().supports_color_capability(ColorCapability::COLOR_TEMPERATURE)) { + if (traits.supports_color_capability(ColorCapability::COLOR_TEMPERATURE)) { ESP_LOGCONFIG(TAG, " Min Mireds: %.1f\n" " Max Mireds: %.1f", - this->get_traits().get_min_mireds(), this->get_traits().get_max_mireds()); + traits.get_min_mireds(), traits.get_max_mireds()); } } void LightState::loop() { @@ -143,12 +140,22 @@ float LightState::get_setup_priority() const { return setup_priority::HARDWARE - void LightState::publish_state() { this->remote_values_callback_.call(); } LightOutput *LightState::get_output() const { return this->output_; } + +static constexpr const char *EFFECT_NONE = "None"; +static constexpr auto EFFECT_NONE_REF = StringRef::from_lit("None"); + std::string LightState::get_effect_name() { if (this->active_effect_index_ > 0) { return this->effects_[this->active_effect_index_ - 1]->get_name(); - } else { - return "None"; } + return EFFECT_NONE; +} + +StringRef LightState::get_effect_name_ref() { + if (this->active_effect_index_ > 0) { + return StringRef(this->effects_[this->active_effect_index_ - 1]->get_name()); + } + return EFFECT_NONE_REF; } void LightState::add_new_remote_values_callback(std::function &&send_callback) { @@ -170,9 +177,10 @@ void LightState::set_gamma_correct(float gamma_correct) { this->gamma_correct_ = void LightState::set_restore_mode(LightRestoreMode restore_mode) { this->restore_mode_ = restore_mode; } void LightState::set_initial_state(const LightStateRTCState &initial_state) { this->initial_state_ = initial_state; } bool LightState::supports_effects() { return !this->effects_.empty(); } -const std::vector &LightState::get_effects() const { return this->effects_; } +const FixedVector &LightState::get_effects() const { return this->effects_; } void LightState::add_effects(const std::vector &effects) { - this->effects_.reserve(this->effects_.size() + effects.size()); + // Called once from Python codegen during setup with all effects from YAML config + this->effects_.init(effects.size()); for (auto *effect : effects) { this->effects_.push_back(effect); } diff --git a/esphome/components/light/light_state.h b/esphome/components/light/light_state.h index 72cb99223e..a07aeb6ae5 100644 --- a/esphome/components/light/light_state.h +++ b/esphome/components/light/light_state.h @@ -4,12 +4,15 @@ #include "esphome/core/entity_base.h" #include "esphome/core/optional.h" #include "esphome/core/preferences.h" +#include "esphome/core/string_ref.h" #include "light_call.h" #include "light_color_values.h" #include "light_effect.h" #include "light_traits.h" #include "light_transformer.h" +#include "esphome/core/helpers.h" +#include #include namespace esphome { @@ -116,6 +119,8 @@ class LightState : public EntityBase, public Component { /// Return the name of the current effect, or if no effect is active "None". std::string get_effect_name(); + /// Return the name of the current effect as StringRef (for API usage) + StringRef get_effect_name_ref(); /** * This lets front-end components subscribe to light change events. This callback is called once @@ -155,11 +160,49 @@ class LightState : public EntityBase, public Component { bool supports_effects(); /// Get all effects for this light state. - const std::vector &get_effects() const; + const FixedVector &get_effects() const; /// Add effects for this light state. void add_effects(const std::vector &effects); + /// Get the total number of effects available for this light. + size_t get_effect_count() const { return this->effects_.size(); } + + /// Get the currently active effect index (0 = no effect, 1+ = effect index). + uint32_t get_current_effect_index() const { return this->active_effect_index_; } + + /// Get effect index by name. Returns 0 if effect not found. + uint32_t get_effect_index(const std::string &effect_name) const { + if (strcasecmp(effect_name.c_str(), "none") == 0) { + return 0; + } + for (size_t i = 0; i < this->effects_.size(); i++) { + if (strcasecmp(effect_name.c_str(), this->effects_[i]->get_name().c_str()) == 0) { + return i + 1; // Effects are 1-indexed in active_effect_index_ + } + } + return 0; // Effect not found + } + + /// Get effect by index. Returns nullptr if index is invalid. + LightEffect *get_effect_by_index(uint32_t index) const { + if (index == 0 || index > this->effects_.size()) { + return nullptr; + } + return this->effects_[index - 1]; // Effects are 1-indexed in active_effect_index_ + } + + /// Get effect name by index. Returns "None" for index 0, empty string for invalid index. + std::string get_effect_name_by_index(uint32_t index) const { + if (index == 0) { + return "None"; + } + if (index > this->effects_.size()) { + return ""; // Invalid index + } + return this->effects_[index - 1]->get_name(); + } + /// The result of all the current_values_as_* methods have gamma correction applied. void current_values_as_binary(bool *binary); @@ -218,7 +261,7 @@ class LightState : public EntityBase, public Component { /// The currently active transformer for this light (transition/flash). std::unique_ptr transformer_{nullptr}; /// List of effects for this light. - std::vector effects_; + FixedVector effects_; /// Object used to store the persisted values of the light. ESPPreferenceObject rtc_; /// Value for storing the index of the currently active effect. 0 if no effect is active diff --git a/esphome/components/light/light_traits.h b/esphome/components/light/light_traits.h index 7c99d721f0..a45301d148 100644 --- a/esphome/components/light/light_traits.h +++ b/esphome/components/light/light_traits.h @@ -5,6 +5,13 @@ #include namespace esphome { + +#ifdef USE_API +namespace api { +class APIConnection; +} // namespace api +#endif + namespace light { /// This class is used to represent the capabilities of a light. @@ -52,6 +59,16 @@ class LightTraits { void set_max_mireds(float max_mireds) { this->max_mireds_ = max_mireds; } protected: +#ifdef USE_API + // The API connection is a friend class to access internal methods + friend class api::APIConnection; + // This method returns a reference to the internal color modes set. + // It is used by the API to avoid copying data when encoding messages. + // Warning: Do not use this method outside of the API connection code. + // It returns a reference to internal data that can be invalidated. + const std::set &get_supported_color_modes_for_api_() const { return this->supported_color_modes_; } +#endif + std::set supported_color_modes_{}; float min_mireds_{0}; float max_mireds_{0}; diff --git a/esphome/components/lightwaverf/lightwaverf.cpp b/esphome/components/lightwaverf/lightwaverf.cpp index 626e5747b7..31ac1fc576 100644 --- a/esphome/components/lightwaverf/lightwaverf.cpp +++ b/esphome/components/lightwaverf/lightwaverf.cpp @@ -14,8 +14,6 @@ static const bool DEFAULT_INVERT = false; static const uint32_t DEFAULT_TICK = 330; void LightWaveRF::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->lwtx_.lwtx_setup(pin_tx_, DEFAULT_REPEAT, DEFAULT_INVERT, DEFAULT_TICK); this->lwrx_.lwrx_setup(pin_rx_); } diff --git a/esphome/components/lilygo_t5_47/touchscreen/lilygo_t5_47_touchscreen.cpp b/esphome/components/lilygo_t5_47/touchscreen/lilygo_t5_47_touchscreen.cpp index c472a9f669..b29e4c2154 100644 --- a/esphome/components/lilygo_t5_47/touchscreen/lilygo_t5_47_touchscreen.cpp +++ b/esphome/components/lilygo_t5_47/touchscreen/lilygo_t5_47_touchscreen.cpp @@ -24,7 +24,6 @@ static const uint8_t READ_TOUCH[1] = {0x07}; } void LilygoT547Touchscreen::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->interrupt_pin_->pin_mode(gpio::FLAG_INPUT | gpio::FLAG_PULLUP); this->interrupt_pin_->setup(); diff --git a/esphome/components/lm75b/__init__.py b/esphome/components/lm75b/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/esphome/components/lm75b/lm75b.cpp b/esphome/components/lm75b/lm75b.cpp new file mode 100644 index 0000000000..19398eda85 --- /dev/null +++ b/esphome/components/lm75b/lm75b.cpp @@ -0,0 +1,39 @@ +#include "lm75b.h" +#include "esphome/core/log.h" +#include "esphome/core/hal.h" + +namespace esphome { +namespace lm75b { + +static const char *const TAG = "lm75b"; + +void LM75BComponent::dump_config() { + ESP_LOGCONFIG(TAG, "LM75B:"); + LOG_I2C_DEVICE(this); + if (this->is_failed()) { + ESP_LOGE(TAG, "Setting up LM75B failed!"); + } + LOG_UPDATE_INTERVAL(this); + LOG_SENSOR(" ", "Temperature", this); +} + +void LM75BComponent::update() { + // Create a temporary buffer + uint8_t buff[2]; + if (this->read_register(LM75B_REG_TEMPERATURE, buff, 2) != i2c::ERROR_OK) { + this->status_set_warning(); + return; + } + // Obtain combined 16-bit value + int16_t raw_temperature = (buff[0] << 8) | buff[1]; + // Read the 11-bit raw temperature value + raw_temperature >>= 5; + // Publish the temperature in °C + this->publish_state(raw_temperature * 0.125); + if (this->status_has_warning()) { + this->status_clear_warning(); + } +} + +} // namespace lm75b +} // namespace esphome diff --git a/esphome/components/lm75b/lm75b.h b/esphome/components/lm75b/lm75b.h new file mode 100644 index 0000000000..79d9fa3f32 --- /dev/null +++ b/esphome/components/lm75b/lm75b.h @@ -0,0 +1,19 @@ +#pragma once + +#include "esphome/core/component.h" +#include "esphome/components/sensor/sensor.h" +#include "esphome/components/i2c/i2c.h" + +namespace esphome { +namespace lm75b { + +static const uint8_t LM75B_REG_TEMPERATURE = 0x00; + +class LM75BComponent : public PollingComponent, public i2c::I2CDevice, public sensor::Sensor { + public: + void dump_config() override; + void update() override; +}; + +} // namespace lm75b +} // namespace esphome diff --git a/esphome/components/lm75b/sensor.py b/esphome/components/lm75b/sensor.py new file mode 100644 index 0000000000..335446b62f --- /dev/null +++ b/esphome/components/lm75b/sensor.py @@ -0,0 +1,34 @@ +import esphome.codegen as cg +from esphome.components import i2c, sensor +import esphome.config_validation as cv +from esphome.const import ( + DEVICE_CLASS_TEMPERATURE, + STATE_CLASS_MEASUREMENT, + UNIT_CELSIUS, +) + +CODEOWNERS = ["@beormund"] +DEPENDENCIES = ["i2c"] + +lm75b_ns = cg.esphome_ns.namespace("lm75b") +LM75BComponent = lm75b_ns.class_( + "LM75BComponent", cg.PollingComponent, i2c.I2CDevice, sensor.Sensor +) + +CONFIG_SCHEMA = ( + sensor.sensor_schema( + LM75BComponent, + unit_of_measurement=UNIT_CELSIUS, + accuracy_decimals=3, + device_class=DEVICE_CLASS_TEMPERATURE, + state_class=STATE_CLASS_MEASUREMENT, + ) + .extend(cv.polling_component_schema("60s")) + .extend(i2c.i2c_device_schema(0x48)) +) + + +async def to_code(config): + var = await sensor.new_sensor(config) + await cg.register_component(var, config) + await i2c.register_i2c_device(var, config) diff --git a/esphome/components/lock/__init__.py b/esphome/components/lock/__init__.py index e62d9f3e2b..04c1586ddd 100644 --- a/esphome/components/lock/__init__.py +++ b/esphome/components/lock/__init__.py @@ -13,7 +13,7 @@ from esphome.const import ( CONF_TRIGGER_ID, CONF_WEB_SERVER, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass @@ -155,7 +155,6 @@ async def lock_is_off_to_code(config, condition_id, template_arg, args): return cg.new_Pvariable(condition_id, template_arg, paren, False) -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): cg.add_global(lock_ns.using) - cg.add_define("USE_LOCK") diff --git a/esphome/components/lock/lock.h b/esphome/components/lock/lock.h index 2173c84903..9737569921 100644 --- a/esphome/components/lock/lock.h +++ b/esphome/components/lock/lock.h @@ -5,7 +5,7 @@ #include "esphome/core/helpers.h" #include "esphome/core/log.h" #include "esphome/core/preferences.h" -#include +#include namespace esphome { namespace lock { @@ -15,8 +15,8 @@ class Lock; #define LOG_LOCK(prefix, type, obj) \ if ((obj) != nullptr) { \ ESP_LOGCONFIG(TAG, "%s%s '%s'", prefix, LOG_STR_LITERAL(type), (obj)->get_name().c_str()); \ - if (!(obj)->get_icon().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon().c_str()); \ + if (!(obj)->get_icon_ref().empty()) { \ + ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon_ref().c_str()); \ } \ if ((obj)->traits.get_assumed_state()) { \ ESP_LOGCONFIG(TAG, "%s Assumed State: YES", prefix); \ @@ -44,16 +44,22 @@ class LockTraits { bool get_assumed_state() const { return this->assumed_state_; } void set_assumed_state(bool assumed_state) { this->assumed_state_ = assumed_state; } - bool supports_state(LockState state) const { return supported_states_.count(state); } - std::set get_supported_states() const { return supported_states_; } - void set_supported_states(std::set states) { supported_states_ = std::move(states); } - void add_supported_state(LockState state) { supported_states_.insert(state); } + bool supports_state(LockState state) const { return supported_states_mask_ & (1 << state); } + void set_supported_states(std::initializer_list states) { + supported_states_mask_ = 0; + for (auto state : states) { + supported_states_mask_ |= (1 << state); + } + } + uint8_t get_supported_states_mask() const { return supported_states_mask_; } + void set_supported_states_mask(uint8_t mask) { supported_states_mask_ = mask; } + void add_supported_state(LockState state) { supported_states_mask_ |= (1 << state); } protected: bool supports_open_{false}; bool requires_code_{false}; bool assumed_state_{false}; - std::set supported_states_ = {LOCK_STATE_NONE, LOCK_STATE_LOCKED, LOCK_STATE_UNLOCKED}; + uint8_t supported_states_mask_{(1 << LOCK_STATE_NONE) | (1 << LOCK_STATE_LOCKED) | (1 << LOCK_STATE_UNLOCKED)}; }; /** This class is used to encode all control actions on a lock device. diff --git a/esphome/components/logger/__init__.py b/esphome/components/logger/__init__.py index 9ac2999696..1d02073d27 100644 --- a/esphome/components/logger/__init__.py +++ b/esphome/components/logger/__init__.py @@ -21,6 +21,11 @@ from esphome.components.libretiny.const import ( COMPONENT_LN882X, COMPONENT_RTL87XX, ) +from esphome.components.zephyr import ( + zephyr_add_cdc_acm, + zephyr_add_overlay, + zephyr_add_prj_conf, +) from esphome.config_helpers import filter_source_files_from_platform import esphome.config_validation as cv from esphome.const import ( @@ -41,11 +46,12 @@ from esphome.const import ( PLATFORM_ESP32, PLATFORM_ESP8266, PLATFORM_LN882X, + PLATFORM_NRF52, PLATFORM_RP2040, PLATFORM_RTL87XX, PlatformFramework, ) -from esphome.core import CORE, Lambda, coroutine_with_priority +from esphome.core import CORE, CoroPriority, Lambda, coroutine_with_priority CODEOWNERS = ["@esphome/core"] logger_ns = cg.esphome_ns.namespace("logger") @@ -89,6 +95,7 @@ DEFAULT = "DEFAULT" CONF_INITIAL_LEVEL = "initial_level" CONF_LOGGER_ID = "logger_id" +CONF_RUNTIME_TAG_LEVELS = "runtime_tag_levels" CONF_TASK_LOG_BUFFER_SIZE = "task_log_buffer_size" UART_SELECTION_ESP32 = { @@ -111,10 +118,10 @@ UART_SELECTION_LIBRETINY = { COMPONENT_RTL87XX: [DEFAULT, UART0, UART1, UART2], } -ESP_ARDUINO_UNSUPPORTED_USB_UARTS = [USB_SERIAL_JTAG] - UART_SELECTION_RP2040 = [USB_CDC, UART0, UART1] +UART_SELECTION_NRF52 = [USB_CDC, UART0] + HARDWARE_UART_TO_UART_SELECTION = { UART0: logger_ns.UART_SELECTION_UART0, UART0_SWAP: logger_ns.UART_SELECTION_UART0_SWAP, @@ -145,13 +152,7 @@ is_log_level = cv.one_of(*LOG_LEVELS, upper=True) def uart_selection(value): if CORE.is_esp32: - if CORE.using_arduino and value.upper() in ESP_ARDUINO_UNSUPPORTED_USB_UARTS: - raise cv.Invalid(f"Arduino framework does not support {value}.") variant = get_esp32_variant() - if CORE.using_esp_idf and variant == VARIANT_ESP32C3 and value == USB_CDC: - raise cv.Invalid( - f"{value} is not supported for variant {variant} when using ESP-IDF." - ) if variant in UART_SELECTION_ESP32: return cv.one_of(*UART_SELECTION_ESP32[variant], upper=True)(value) if CORE.is_esp8266: @@ -167,6 +168,8 @@ def uart_selection(value): return cv.one_of(*UART_SELECTION_LIBRETINY[component], upper=True)(value) if CORE.is_host: raise cv.Invalid("Uart selection not valid for host platform") + if CORE.is_nrf52: + return cv.one_of(*UART_SELECTION_NRF52, upper=True)(value) raise NotImplementedError @@ -183,9 +186,10 @@ def validate_local_no_higher_than_global(value): Logger = logger_ns.class_("Logger", cg.Component) LoggerMessageTrigger = logger_ns.class_( "LoggerMessageTrigger", - automation.Trigger.template(cg.int_, cg.const_char_ptr, cg.const_char_ptr), + automation.Trigger.template(cg.uint8, cg.const_char_ptr, cg.const_char_ptr), ) + CONF_ESP8266_STORE_LOG_STRINGS_IN_FLASH = "esp8266_store_log_strings_in_flash" CONFIG_SCHEMA = cv.All( cv.Schema( @@ -215,18 +219,16 @@ CONFIG_SCHEMA = cv.All( esp8266=UART0, esp32=UART0, esp32_s2=USB_CDC, - esp32_s3_arduino=USB_CDC, - esp32_s3_idf=USB_SERIAL_JTAG, - esp32_c3_arduino=USB_CDC, - esp32_c3_idf=USB_SERIAL_JTAG, - esp32_c5_idf=USB_SERIAL_JTAG, - esp32_c6_arduino=USB_CDC, - esp32_c6_idf=USB_SERIAL_JTAG, - esp32_p4_idf=USB_SERIAL_JTAG, + esp32_s3=USB_SERIAL_JTAG, + esp32_c3=USB_SERIAL_JTAG, + esp32_c5=USB_SERIAL_JTAG, + esp32_c6=USB_SERIAL_JTAG, + esp32_p4=USB_SERIAL_JTAG, rp2040=USB_CDC, bk72xx=DEFAULT, ln882x=DEFAULT, rtl87xx=DEFAULT, + nrf52=USB_CDC, ): cv.All( cv.only_on( [ @@ -236,6 +238,7 @@ CONFIG_SCHEMA = cv.All( PLATFORM_BK72XX, PLATFORM_LN882X, PLATFORM_RTL87XX, + PLATFORM_NRF52, ] ), uart_selection, @@ -247,6 +250,7 @@ CONFIG_SCHEMA = cv.All( } ), cv.Optional(CONF_INITIAL_LEVEL): is_log_level, + cv.Optional(CONF_RUNTIME_TAG_LEVELS, default=False): cv.boolean, cv.Optional(CONF_ON_MESSAGE): automation.validate_automation( { cv.GenerateID(CONF_TRIGGER_ID): cv.declare_id(LoggerMessageTrigger), @@ -262,7 +266,7 @@ CONFIG_SCHEMA = cv.All( ) -@coroutine_with_priority(90.0) +@coroutine_with_priority(CoroPriority.DIAGNOSTICS) async def to_code(config): baud_rate = config[CONF_BAUD_RATE] level = config[CONF_LEVEL] @@ -289,8 +293,12 @@ async def to_code(config): ) cg.add(log.pre_setup()) - for tag, log_level in config[CONF_LOGS].items(): - cg.add(log.set_log_level(tag, LOG_LEVELS[log_level])) + # Enable runtime tag levels if logs are configured or explicitly enabled + logs_config = config[CONF_LOGS] + if logs_config or config[CONF_RUNTIME_TAG_LEVELS]: + cg.add_define("USE_LOGGER_RUNTIME_TAG_LEVELS") + for tag, log_level in logs_config.items(): + cg.add(log.set_log_level(tag, LOG_LEVELS[log_level])) cg.add_define("USE_LOGGER") this_severity = LOG_LEVEL_SEVERITY.index(level) @@ -333,16 +341,7 @@ async def to_code(config): if config.get(CONF_ESP8266_STORE_LOG_STRINGS_IN_FLASH): cg.add_build_flag("-DUSE_STORE_LOG_STR_IN_FLASH") - if CORE.using_arduino: - if config[CONF_HARDWARE_UART] == USB_CDC: - cg.add_build_flag("-DARDUINO_USB_CDC_ON_BOOT=1") - if CORE.is_esp32 and get_esp32_variant() in ( - VARIANT_ESP32C3, - VARIANT_ESP32C6, - ): - cg.add_build_flag("-DARDUINO_USB_MODE=1") - - if CORE.using_esp_idf: + if CORE.is_esp32: if config[CONF_HARDWARE_UART] == USB_CDC: add_idf_sdkconfig_option("CONFIG_ESP_CONSOLE_USB_CDC", True) elif config[CONF_HARDWARE_UART] == USB_SERIAL_JTAG: @@ -358,6 +357,15 @@ async def to_code(config): except cv.Invalid: pass + if CORE.using_zephyr: + if config[CONF_HARDWARE_UART] == UART0: + zephyr_add_overlay("""&uart0 { status = "okay";};""") + if config[CONF_HARDWARE_UART] == UART1: + zephyr_add_overlay("""&uart1 { status = "okay";};""") + if config[CONF_HARDWARE_UART] == USB_CDC: + zephyr_add_prj_conf("UART_LINE_CTRL", True) + zephyr_add_cdc_acm(config, 0) + # Register at end for safe mode await cg.register_component(log, config) @@ -368,7 +376,7 @@ async def to_code(config): await automation.build_automation( trigger, [ - (cg.int_, "level"), + (cg.uint8, "level"), (cg.const_char_ptr, "tag"), (cg.const_char_ptr, "message"), ], @@ -388,7 +396,7 @@ def validate_printf(value): [cCdiouxXeEfgGaAnpsSZ] # type ) """ # noqa - matches = re.findall(cfmt, value[CONF_FORMAT], flags=re.X) + matches = re.findall(cfmt, value[CONF_FORMAT], flags=re.VERBOSE) if len(matches) != len(value[CONF_ARGS]): raise cv.Invalid( f"Found {len(matches)} printf-patterns ({', '.join(matches)}), but {len(value[CONF_ARGS])} args were given!" @@ -400,6 +408,7 @@ CONF_LOGGER_LOG = "logger.log" LOGGER_LOG_ACTION_SCHEMA = cv.All( cv.maybe_simple_value( { + cv.GenerateID(CONF_LOGGER_ID): cv.use_id(Logger), cv.Required(CONF_FORMAT): cv.string, cv.Optional(CONF_ARGS, default=list): cv.ensure_list(cv.lambda_), cv.Optional(CONF_LEVEL, default="DEBUG"): cv.one_of( @@ -440,6 +449,7 @@ async def logger_set_level_to_code(config, action_id, template_arg, args): level = LOG_LEVELS[config[CONF_LEVEL]] logger = await cg.get_variable(config[CONF_LOGGER_ID]) if tag := config.get(CONF_TAG): + cg.add_define("USE_LOGGER_RUNTIME_TAG_LEVELS") text = str(cg.statement(logger.set_log_level(tag, level))) else: text = str(cg.statement(logger.set_log_level(level))) @@ -462,6 +472,7 @@ FILTER_SOURCE_FILES = filter_source_files_from_platform( PlatformFramework.RTL87XX_ARDUINO, PlatformFramework.LN882X_ARDUINO, }, + "logger_zephyr.cpp": {PlatformFramework.NRF52_ZEPHYR}, "task_log_buffer.cpp": { PlatformFramework.ESP32_ARDUINO, PlatformFramework.ESP32_IDF, diff --git a/esphome/components/logger/logger.cpp b/esphome/components/logger/logger.cpp index db807f7e53..9a9bf89fe3 100644 --- a/esphome/components/logger/logger.cpp +++ b/esphome/components/logger/logger.cpp @@ -4,12 +4,11 @@ #include // For unique_ptr #endif +#include "esphome/core/application.h" #include "esphome/core/hal.h" #include "esphome/core/log.h" -#include "esphome/core/application.h" -namespace esphome { -namespace logger { +namespace esphome::logger { static const char *const TAG = "logger"; @@ -149,9 +148,11 @@ void Logger::log_vprintf_(uint8_t level, const char *tag, int line, const __Flas #endif // USE_STORE_LOG_STR_IN_FLASH inline uint8_t Logger::level_for(const char *tag) { +#ifdef USE_LOGGER_RUNTIME_TAG_LEVELS auto it = this->log_levels_.find(tag); if (it != this->log_levels_.end()) return it->second; +#endif return this->current_level_; } @@ -160,6 +161,8 @@ Logger::Logger(uint32_t baud_rate, size_t tx_buffer_size) : baud_rate_(baud_rate this->tx_buffer_ = new char[this->tx_buffer_size_ + 1]; // NOLINT #if defined(USE_ESP32) || defined(USE_LIBRETINY) this->main_task_ = xTaskGetCurrentTaskHandle(); +#elif defined(USE_ZEPHYR) + this->main_task_ = k_current_get(); #endif } #ifdef USE_ESPHOME_TASK_LOG_BUFFER @@ -172,21 +175,11 @@ void Logger::init_log_buffer(size_t total_buffer_size) { } #endif -#if defined(USE_LOGGER_USB_CDC) || defined(USE_ESP32) -void Logger::loop() { -#if defined(USE_LOGGER_USB_CDC) && defined(USE_ARDUINO) - if (this->uart_ == UART_SELECTION_USB_CDC) { - static bool opened = false; - if (opened == Serial) { - return; - } - if (false == opened) { - App.schedule_dump_config(); - } - opened = !opened; - } +#ifdef USE_ESPHOME_TASK_LOG_BUFFER +void Logger::loop() { this->process_messages_(); } #endif +void Logger::process_messages_() { #ifdef USE_ESPHOME_TASK_LOG_BUFFER // Process any buffered messages when available if (this->log_buffer_->has_messages()) { @@ -227,12 +220,13 @@ void Logger::loop() { } #endif } -#endif void Logger::set_baud_rate(uint32_t baud_rate) { this->baud_rate_ = baud_rate; } -void Logger::set_log_level(const std::string &tag, uint8_t log_level) { this->log_levels_[tag] = log_level; } +#ifdef USE_LOGGER_RUNTIME_TAG_LEVELS +void Logger::set_log_level(const char *tag, uint8_t log_level) { this->log_levels_[tag] = log_level; } +#endif -#if defined(USE_ESP32) || defined(USE_ESP8266) || defined(USE_RP2040) || defined(USE_LIBRETINY) +#if defined(USE_ESP32) || defined(USE_ESP8266) || defined(USE_RP2040) || defined(USE_LIBRETINY) || defined(USE_ZEPHYR) UARTSelection Logger::get_uart() const { return this->uart_; } #endif @@ -240,19 +234,40 @@ void Logger::add_on_log_callback(std::functionlog_callback_.add(std::move(callback)); } float Logger::get_setup_priority() const { return setup_priority::BUS + 500.0f; } + +#ifdef USE_STORE_LOG_STR_IN_FLASH +// ESP8266: PSTR() cannot be used in array initializers, so we need to declare +// each string separately as a global constant first +static const char LOG_LEVEL_NONE[] PROGMEM = "NONE"; +static const char LOG_LEVEL_ERROR[] PROGMEM = "ERROR"; +static const char LOG_LEVEL_WARN[] PROGMEM = "WARN"; +static const char LOG_LEVEL_INFO[] PROGMEM = "INFO"; +static const char LOG_LEVEL_CONFIG[] PROGMEM = "CONFIG"; +static const char LOG_LEVEL_DEBUG[] PROGMEM = "DEBUG"; +static const char LOG_LEVEL_VERBOSE[] PROGMEM = "VERBOSE"; +static const char LOG_LEVEL_VERY_VERBOSE[] PROGMEM = "VERY_VERBOSE"; + +static const LogString *const LOG_LEVELS[] = { + reinterpret_cast(LOG_LEVEL_NONE), reinterpret_cast(LOG_LEVEL_ERROR), + reinterpret_cast(LOG_LEVEL_WARN), reinterpret_cast(LOG_LEVEL_INFO), + reinterpret_cast(LOG_LEVEL_CONFIG), reinterpret_cast(LOG_LEVEL_DEBUG), + reinterpret_cast(LOG_LEVEL_VERBOSE), reinterpret_cast(LOG_LEVEL_VERY_VERBOSE), +}; +#else static const char *const LOG_LEVELS[] = {"NONE", "ERROR", "WARN", "INFO", "CONFIG", "DEBUG", "VERBOSE", "VERY_VERBOSE"}; +#endif void Logger::dump_config() { ESP_LOGCONFIG(TAG, "Logger:\n" " Max Level: %s\n" " Initial Level: %s", - LOG_LEVELS[ESPHOME_LOG_LEVEL], LOG_LEVELS[this->current_level_]); + LOG_STR_ARG(LOG_LEVELS[ESPHOME_LOG_LEVEL]), LOG_STR_ARG(LOG_LEVELS[this->current_level_])); #ifndef USE_HOST ESP_LOGCONFIG(TAG, " Log Baud Rate: %" PRIu32 "\n" " Hardware UART: %s", - this->baud_rate_, get_uart_selection_()); + this->baud_rate_, LOG_STR_ARG(get_uart_selection_())); #endif #ifdef USE_ESPHOME_TASK_LOG_BUFFER if (this->log_buffer_) { @@ -260,15 +275,17 @@ void Logger::dump_config() { } #endif +#ifdef USE_LOGGER_RUNTIME_TAG_LEVELS for (auto &it : this->log_levels_) { - ESP_LOGCONFIG(TAG, " Level for '%s': %s", it.first.c_str(), LOG_LEVELS[it.second]); + ESP_LOGCONFIG(TAG, " Level for '%s': %s", it.first, LOG_STR_ARG(LOG_LEVELS[it.second])); } +#endif } void Logger::set_log_level(uint8_t level) { if (level > ESPHOME_LOG_LEVEL) { level = ESPHOME_LOG_LEVEL; - ESP_LOGW(TAG, "Cannot set log level higher than pre-compiled %s", LOG_LEVELS[ESPHOME_LOG_LEVEL]); + ESP_LOGW(TAG, "Cannot set log level higher than pre-compiled %s", LOG_STR_ARG(LOG_LEVELS[ESPHOME_LOG_LEVEL])); } this->current_level_ = level; this->level_callback_.call(level); @@ -276,5 +293,4 @@ void Logger::set_log_level(uint8_t level) { Logger *global_logger = nullptr; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) -} // namespace logger -} // namespace esphome +} // namespace esphome::logger diff --git a/esphome/components/logger/logger.h b/esphome/components/logger/logger.h index fb68e75a51..dc8e06e0c9 100644 --- a/esphome/components/logger/logger.h +++ b/esphome/components/logger/logger.h @@ -16,47 +16,62 @@ #endif #ifdef USE_ARDUINO -#if defined(USE_ESP8266) || defined(USE_ESP32) +#if defined(USE_ESP8266) #include -#endif // USE_ESP8266 || USE_ESP32 +#endif // USE_ESP8266 #ifdef USE_RP2040 #include #include #endif // USE_RP2040 #endif // USE_ARDUINO -#ifdef USE_ESP_IDF +#ifdef USE_ESP32 #include -#endif // USE_ESP_IDF +#endif // USE_ESP32 -namespace esphome { +#ifdef USE_ZEPHYR +#include +struct device; +#endif -namespace logger { +namespace esphome::logger { -// Color and letter constants for log levels -static const char *const LOG_LEVEL_COLORS[] = { - "", // NONE - ESPHOME_LOG_BOLD(ESPHOME_LOG_COLOR_RED), // ERROR - ESPHOME_LOG_COLOR(ESPHOME_LOG_COLOR_YELLOW), // WARNING - ESPHOME_LOG_COLOR(ESPHOME_LOG_COLOR_GREEN), // INFO - ESPHOME_LOG_COLOR(ESPHOME_LOG_COLOR_MAGENTA), // CONFIG - ESPHOME_LOG_COLOR(ESPHOME_LOG_COLOR_CYAN), // DEBUG - ESPHOME_LOG_COLOR(ESPHOME_LOG_COLOR_GRAY), // VERBOSE - ESPHOME_LOG_COLOR(ESPHOME_LOG_COLOR_WHITE), // VERY_VERBOSE +#ifdef USE_LOGGER_RUNTIME_TAG_LEVELS +// Comparison function for const char* keys in log_levels_ map +struct CStrCompare { + bool operator()(const char *a, const char *b) const { return strcmp(a, b) < 0; } +}; +#endif + +// ANSI color code last digit (30-38 range, store only last digit to save RAM) +static constexpr char LOG_LEVEL_COLOR_DIGIT[] = { + '\0', // NONE + '1', // ERROR (31 = red) + '3', // WARNING (33 = yellow) + '2', // INFO (32 = green) + '5', // CONFIG (35 = magenta) + '6', // DEBUG (36 = cyan) + '7', // VERBOSE (37 = gray) + '8', // VERY_VERBOSE (38 = white) }; -static const char *const LOG_LEVEL_LETTERS[] = { - "", // NONE - "E", // ERROR - "W", // WARNING - "I", // INFO - "C", // CONFIG - "D", // DEBUG - "V", // VERBOSE - "VV", // VERY_VERBOSE +static constexpr char LOG_LEVEL_LETTER_CHARS[] = { + '\0', // NONE + 'E', // ERROR + 'W', // WARNING + 'I', // INFO + 'C', // CONFIG + 'D', // DEBUG + 'V', // VERBOSE (VERY_VERBOSE uses two 'V's) }; -#if defined(USE_ESP32) || defined(USE_ESP8266) || defined(USE_RP2040) || defined(USE_LIBRETINY) +// Maximum header size: 35 bytes fixed + 32 bytes tag + 16 bytes thread name = 83 bytes (45 byte safety margin) +static constexpr uint16_t MAX_HEADER_SIZE = 128; + +// "0x" + 2 hex digits per byte + '\0' +static constexpr size_t MAX_POINTER_REPRESENTATION = 2 + sizeof(void *) * 2 + 1; + +#if defined(USE_ESP32) || defined(USE_ESP8266) || defined(USE_RP2040) || defined(USE_LIBRETINY) || defined(USE_ZEPHYR) /** Enum for logging UART selection * * Advanced configuration (pin selection, etc) is not supported. @@ -82,7 +97,7 @@ enum UARTSelection : uint8_t { UART_SELECTION_UART0_SWAP, #endif // USE_ESP8266 }; -#endif // USE_ESP32 || USE_ESP8266 || USE_RP2040 || USE_LIBRETINY +#endif // USE_ESP32 || USE_ESP8266 || USE_RP2040 || USE_LIBRETINY || USE_ZEPHYR /** * @brief Logger component for all ESPHome logging. @@ -107,22 +122,20 @@ class Logger : public Component { #ifdef USE_ESPHOME_TASK_LOG_BUFFER void init_log_buffer(size_t total_buffer_size); #endif -#if defined(USE_LOGGER_USB_CDC) || defined(USE_ESP32) +#if defined(USE_ESPHOME_TASK_LOG_BUFFER) || (defined(USE_ZEPHYR) && defined(USE_LOGGER_USB_CDC)) void loop() override; #endif /// Manually set the baud rate for serial, set to 0 to disable. void set_baud_rate(uint32_t baud_rate); uint32_t get_baud_rate() const { return baud_rate_; } -#ifdef USE_ARDUINO +#if defined(USE_ARDUINO) && !defined(USE_ESP32) Stream *get_hw_serial() const { return hw_serial_; } #endif -#ifdef USE_ESP_IDF - uart_port_t get_uart_num() const { return uart_num_; } -#endif #ifdef USE_ESP32 + uart_port_t get_uart_num() const { return uart_num_; } void create_pthread_key() { pthread_key_create(&log_recursion_key_, nullptr); } #endif -#if defined(USE_ESP32) || defined(USE_ESP8266) || defined(USE_RP2040) || defined(USE_LIBRETINY) +#if defined(USE_ESP32) || defined(USE_ESP8266) || defined(USE_RP2040) || defined(USE_LIBRETINY) || defined(USE_ZEPHYR) void set_uart_selection(UARTSelection uart_selection) { uart_ = uart_selection; } /// Get the UART used by the logger. UARTSelection get_uart() const; @@ -130,8 +143,10 @@ class Logger : public Component { /// Set the default log level for this logger. void set_log_level(uint8_t level); +#ifdef USE_LOGGER_RUNTIME_TAG_LEVELS /// Set the log level of the specified tag. - void set_log_level(const std::string &tag, uint8_t log_level); + void set_log_level(const char *tag, uint8_t log_level); +#endif uint8_t get_log_level() { return this->current_level_; } // ========== INTERNAL METHODS ========== @@ -157,6 +172,7 @@ class Logger : public Component { #endif protected: + void process_messages_(); void write_msg_(const char *msg); // Format a log message with printf-style arguments and write it to a buffer with header, footer, and null terminator @@ -166,6 +182,9 @@ class Logger : public Component { uint16_t buffer_size) { #if defined(USE_ESP32) || defined(USE_LIBRETINY) this->write_header_to_buffer_(level, tag, line, this->get_thread_name_(), buffer, buffer_at, buffer_size); +#elif defined(USE_ZEPHYR) + char buff[MAX_POINTER_REPRESENTATION]; + this->write_header_to_buffer_(level, tag, line, this->get_thread_name_(buff), buffer, buffer_at, buffer_size); #else this->write_header_to_buffer_(level, tag, line, nullptr, buffer, buffer_at, buffer_size); #endif @@ -213,25 +232,20 @@ class Logger : public Component { } } - // Format string to explicit buffer with varargs - inline void printf_to_buffer_(char *buffer, uint16_t *buffer_at, uint16_t buffer_size, const char *format, ...) { - va_list arg; - va_start(arg, format); - this->format_body_to_buffer_(buffer, buffer_at, buffer_size, format, arg); - va_end(arg); - } - #ifndef USE_HOST - const char *get_uart_selection_(); + const LogString *get_uart_selection_(); #endif // Group 4-byte aligned members first uint32_t baud_rate_; char *tx_buffer_{nullptr}; -#ifdef USE_ARDUINO +#if defined(USE_ARDUINO) && !defined(USE_ESP32) Stream *hw_serial_{nullptr}; #endif -#if defined(USE_ESP32) || defined(USE_LIBRETINY) +#if defined(USE_ZEPHYR) + const device *uart_dev_{nullptr}; +#endif +#if defined(USE_ESP32) || defined(USE_LIBRETINY) || defined(USE_ZEPHYR) void *main_task_ = nullptr; // Only used for thread name identification #endif #ifdef USE_ESP32 @@ -239,13 +253,13 @@ class Logger : public Component { // - Main task uses a dedicated member variable for efficiency // - Other tasks use pthread TLS with a dynamically created key via pthread_key_create pthread_key_t log_recursion_key_; // 4 bytes -#endif -#ifdef USE_ESP_IDF - uart_port_t uart_num_; // 4 bytes (enum defaults to int size) + uart_port_t uart_num_; // 4 bytes (enum defaults to int size) #endif // Large objects (internally aligned) - std::map log_levels_{}; +#ifdef USE_LOGGER_RUNTIME_TAG_LEVELS + std::map log_levels_{}; +#endif CallbackManager log_callback_{}; CallbackManager level_callback_{}; #ifdef USE_ESPHOME_TASK_LOG_BUFFER @@ -256,7 +270,7 @@ class Logger : public Component { uint16_t tx_buffer_at_{0}; uint16_t tx_buffer_size_{0}; uint8_t current_level_{ESPHOME_LOG_LEVEL_VERY_VERBOSE}; -#if defined(USE_ESP32) || defined(USE_ESP8266) || defined(USE_RP2040) +#if defined(USE_ESP32) || defined(USE_ESP8266) || defined(USE_RP2040) || defined(USE_ZEPHYR) UARTSelection uart_{UART_SELECTION_UART0}; #endif #ifdef USE_LIBRETINY @@ -268,9 +282,17 @@ class Logger : public Component { bool global_recursion_guard_{false}; // Simple global recursion guard for single-task platforms #endif -#if defined(USE_ESP32) || defined(USE_LIBRETINY) - const char *HOT get_thread_name_() { +#if defined(USE_ESP32) || defined(USE_LIBRETINY) || defined(USE_ZEPHYR) + const char *HOT get_thread_name_( +#ifdef USE_ZEPHYR + char *buff +#endif + ) { +#ifdef USE_ZEPHYR + k_tid_t current_task = k_current_get(); +#else TaskHandle_t current_task = xTaskGetCurrentTaskHandle(); +#endif if (current_task == main_task_) { return nullptr; // Main task } else { @@ -278,6 +300,14 @@ class Logger : public Component { return pcTaskGetName(current_task); #elif defined(USE_LIBRETINY) return pcTaskGetTaskName(current_task); +#elif defined(USE_ZEPHYR) + const char *name = k_thread_name_get(current_task); + if (name) { + // zephyr print task names only if debug component is present + return name; + } + std::snprintf(buff, MAX_POINTER_REPRESENTATION, "%p", current_task); + return buff; #endif } } @@ -309,26 +339,76 @@ class Logger : public Component { } #endif + static inline void copy_string(char *buffer, uint16_t &pos, const char *str) { + const size_t len = strlen(str); + // Intentionally no null terminator, building larger string + memcpy(buffer + pos, str, len); // NOLINT(bugprone-not-null-terminated-result) + pos += len; + } + + static inline void write_ansi_color_for_level(char *buffer, uint16_t &pos, uint8_t level) { + if (level == 0) + return; + // Construct ANSI escape sequence: "\033[{bold};3{color}m" + // Example: "\033[1;31m" for ERROR (bold red) + buffer[pos++] = '\033'; + buffer[pos++] = '['; + buffer[pos++] = (level == 1) ? '1' : '0'; // Only ERROR is bold + buffer[pos++] = ';'; + buffer[pos++] = '3'; + buffer[pos++] = LOG_LEVEL_COLOR_DIGIT[level]; + buffer[pos++] = 'm'; + } + inline void HOT write_header_to_buffer_(uint8_t level, const char *tag, int line, const char *thread_name, char *buffer, uint16_t *buffer_at, uint16_t buffer_size) { - // Format header - // uint8_t level is already bounded 0-255, just ensure it's <= 7 - if (level > 7) - level = 7; - - const char *color = esphome::logger::LOG_LEVEL_COLORS[level]; - const char *letter = esphome::logger::LOG_LEVEL_LETTERS[level]; - -#if defined(USE_ESP32) || defined(USE_LIBRETINY) - if (thread_name != nullptr) { - // Non-main task with thread name - this->printf_to_buffer_(buffer, buffer_at, buffer_size, "%s[%s][%s:%03u]%s[%s]%s: ", color, letter, tag, line, - ESPHOME_LOG_BOLD(ESPHOME_LOG_COLOR_RED), thread_name, color); + uint16_t pos = *buffer_at; + // Early return if insufficient space - intentionally don't update buffer_at to prevent partial writes + if (pos + MAX_HEADER_SIZE > buffer_size) return; + + // Construct: [LEVEL][tag:line]: + write_ansi_color_for_level(buffer, pos, level); + buffer[pos++] = '['; + if (level != 0) { + if (level >= 7) { + buffer[pos++] = 'V'; // VERY_VERBOSE = "VV" + buffer[pos++] = 'V'; + } else { + buffer[pos++] = LOG_LEVEL_LETTER_CHARS[level]; + } + } + buffer[pos++] = ']'; + buffer[pos++] = '['; + copy_string(buffer, pos, tag); + buffer[pos++] = ':'; + // Format line number without modulo operations (passed by value, safe to mutate) + if (line > 999) [[unlikely]] { + int thousands = line / 1000; + buffer[pos++] = '0' + thousands; + line -= thousands * 1000; + } + int hundreds = line / 100; + int remainder = line - hundreds * 100; + int tens = remainder / 10; + buffer[pos++] = '0' + hundreds; + buffer[pos++] = '0' + tens; + buffer[pos++] = '0' + (remainder - tens * 10); + buffer[pos++] = ']'; + +#if defined(USE_ESP32) || defined(USE_LIBRETINY) || defined(USE_ZEPHYR) + if (thread_name != nullptr) { + write_ansi_color_for_level(buffer, pos, 1); // Always use bold red for thread name + buffer[pos++] = '['; + copy_string(buffer, pos, thread_name); + buffer[pos++] = ']'; + write_ansi_color_for_level(buffer, pos, level); // Restore original color } #endif - // Main task or non ESP32/LibreTiny platform - this->printf_to_buffer_(buffer, buffer_at, buffer_size, "%s[%s][%s:%03u]: ", color, letter, tag, line); + + buffer[pos++] = ':'; + buffer[pos++] = ' '; + *buffer_at = pos; } inline void HOT format_body_to_buffer_(char *buffer, uint16_t *buffer_at, uint16_t buffer_size, const char *format, @@ -367,15 +447,7 @@ class Logger : public Component { // will be processed on the next main loop iteration since: // - disable_loop() takes effect immediately // - enable_loop_soon_any_context() sets a pending flag that's checked at loop start -#if defined(USE_LOGGER_USB_CDC) && defined(USE_ARDUINO) - // Only disable if not using USB CDC (which needs loop for connection detection) - if (this->uart_ != UART_SELECTION_USB_CDC) { - this->disable_loop(); - } -#else - // No USB CDC support, always safe to disable this->disable_loop(); -#endif } #endif }; @@ -396,6 +468,4 @@ class LoggerMessageTrigger : public Trigger uint8_t level_; }; -} // namespace logger - -} // namespace esphome +} // namespace esphome::logger diff --git a/esphome/components/logger/logger_esp32.cpp b/esphome/components/logger/logger_esp32.cpp index 2fde0f7d49..7fc79e6f54 100644 --- a/esphome/components/logger/logger_esp32.cpp +++ b/esphome/components/logger/logger_esp32.cpp @@ -1,11 +1,8 @@ #ifdef USE_ESP32 #include "logger.h" -#if defined(USE_ESP32_FRAMEWORK_ARDUINO) || defined(USE_ESP_IDF) #include -#endif // USE_ESP32_FRAMEWORK_ARDUINO || USE_ESP_IDF -#ifdef USE_ESP_IDF #include #ifdef USE_LOGGER_USB_SERIAL_JTAG @@ -25,17 +22,12 @@ #include #include -#endif // USE_ESP_IDF - #include "esphome/core/log.h" -namespace esphome { -namespace logger { +namespace esphome::logger { static const char *const TAG = "logger"; -#ifdef USE_ESP_IDF - #ifdef USE_LOGGER_USB_SERIAL_JTAG static void init_usb_serial_jtag_() { setvbuf(stdin, NULL, _IONBF, 0); // Disable buffering on stdin @@ -90,45 +82,8 @@ void init_uart(uart_port_t uart_num, uint32_t baud_rate, int tx_buffer_size) { uart_driver_install(uart_num, uart_buffer_size, uart_buffer_size, 10, nullptr, 0); } -#endif // USE_ESP_IDF - void Logger::pre_setup() { if (this->baud_rate_ > 0) { -#ifdef USE_ARDUINO - switch (this->uart_) { - case UART_SELECTION_UART0: -#if ARDUINO_USB_CDC_ON_BOOT - this->hw_serial_ = &Serial0; - Serial0.begin(this->baud_rate_); -#else - this->hw_serial_ = &Serial; - Serial.begin(this->baud_rate_); -#endif - break; - case UART_SELECTION_UART1: - this->hw_serial_ = &Serial1; - Serial1.begin(this->baud_rate_); - break; -#ifdef USE_ESP32_VARIANT_ESP32 - case UART_SELECTION_UART2: - this->hw_serial_ = &Serial2; - Serial2.begin(this->baud_rate_); - break; -#endif - -#ifdef USE_LOGGER_USB_CDC - case UART_SELECTION_USB_CDC: - this->hw_serial_ = &Serial; -#if ARDUINO_USB_CDC_ON_BOOT - Serial.setTxTimeoutMs(0); // workaround for 2.0.9 crash when there's no data connection -#endif - Serial.begin(this->baud_rate_); - break; -#endif - } -#endif // USE_ARDUINO - -#ifdef USE_ESP_IDF this->uart_num_ = UART_NUM_0; switch (this->uart_) { case UART_SELECTION_UART0: @@ -155,21 +110,17 @@ void Logger::pre_setup() { break; #endif } -#endif // USE_ESP_IDF } global_logger = this; -#if defined(USE_ESP_IDF) || defined(USE_ESP32_FRAMEWORK_ARDUINO) esp_log_set_vprintf(esp_idf_log_vprintf_); if (ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE) { esp_log_level_set("*", ESP_LOG_VERBOSE); } -#endif // USE_ESP_IDF || USE_ESP32_FRAMEWORK_ARDUINO ESP_LOGI(TAG, "Log initialized"); } -#ifdef USE_ESP_IDF void HOT Logger::write_msg_(const char *msg) { if ( #if defined(USE_LOGGER_USB_CDC) && !defined(USE_LOGGER_USB_SERIAL_JTAG) @@ -190,25 +141,29 @@ void HOT Logger::write_msg_(const char *msg) { uart_write_bytes(this->uart_num_, "\n", 1); } } -#else -void HOT Logger::write_msg_(const char *msg) { this->hw_serial_->println(msg); } -#endif -const char *const UART_SELECTIONS[] = { - "UART0", "UART1", +const LogString *Logger::get_uart_selection_() { + switch (this->uart_) { + case UART_SELECTION_UART0: + return LOG_STR("UART0"); + case UART_SELECTION_UART1: + return LOG_STR("UART1"); #ifdef USE_ESP32_VARIANT_ESP32 - "UART2", + case UART_SELECTION_UART2: + return LOG_STR("UART2"); #endif #ifdef USE_LOGGER_USB_CDC - "USB_CDC", + case UART_SELECTION_USB_CDC: + return LOG_STR("USB_CDC"); #endif #ifdef USE_LOGGER_USB_SERIAL_JTAG - "USB_SERIAL_JTAG", + case UART_SELECTION_USB_SERIAL_JTAG: + return LOG_STR("USB_SERIAL_JTAG"); #endif -}; + default: + return LOG_STR("UNKNOWN"); + } +} -const char *Logger::get_uart_selection_() { return UART_SELECTIONS[this->uart_]; } - -} // namespace logger -} // namespace esphome +} // namespace esphome::logger #endif diff --git a/esphome/components/logger/logger_esp8266.cpp b/esphome/components/logger/logger_esp8266.cpp index 5bfeb21007..5063d88b92 100644 --- a/esphome/components/logger/logger_esp8266.cpp +++ b/esphome/components/logger/logger_esp8266.cpp @@ -2,8 +2,7 @@ #include "logger.h" #include "esphome/core/log.h" -namespace esphome { -namespace logger { +namespace esphome::logger { static const char *const TAG = "logger"; @@ -36,10 +35,17 @@ void Logger::pre_setup() { void HOT Logger::write_msg_(const char *msg) { this->hw_serial_->println(msg); } -const char *const UART_SELECTIONS[] = {"UART0", "UART1", "UART0_SWAP"}; +const LogString *Logger::get_uart_selection_() { + switch (this->uart_) { + case UART_SELECTION_UART0: + return LOG_STR("UART0"); + case UART_SELECTION_UART1: + return LOG_STR("UART1"); + case UART_SELECTION_UART0_SWAP: + default: + return LOG_STR("UART0_SWAP"); + } +} -const char *Logger::get_uart_selection_() { return UART_SELECTIONS[this->uart_]; } - -} // namespace logger -} // namespace esphome +} // namespace esphome::logger #endif diff --git a/esphome/components/logger/logger_host.cpp b/esphome/components/logger/logger_host.cpp index edffb1a8c3..4abe92286a 100644 --- a/esphome/components/logger/logger_host.cpp +++ b/esphome/components/logger/logger_host.cpp @@ -1,8 +1,7 @@ #if defined(USE_HOST) #include "logger.h" -namespace esphome { -namespace logger { +namespace esphome::logger { void HOT Logger::write_msg_(const char *msg) { time_t rawtime; @@ -18,7 +17,6 @@ void HOT Logger::write_msg_(const char *msg) { void Logger::pre_setup() { global_logger = this; } -} // namespace logger -} // namespace esphome +} // namespace esphome::logger #endif diff --git a/esphome/components/logger/logger_libretiny.cpp b/esphome/components/logger/logger_libretiny.cpp index 12e55b7cef..3edfa74480 100644 --- a/esphome/components/logger/logger_libretiny.cpp +++ b/esphome/components/logger/logger_libretiny.cpp @@ -1,8 +1,7 @@ #ifdef USE_LIBRETINY #include "logger.h" -namespace esphome { -namespace logger { +namespace esphome::logger { static const char *const TAG = "logger"; @@ -52,11 +51,20 @@ void Logger::pre_setup() { void HOT Logger::write_msg_(const char *msg) { this->hw_serial_->println(msg); } -const char *const UART_SELECTIONS[] = {"DEFAULT", "UART0", "UART1", "UART2"}; +const LogString *Logger::get_uart_selection_() { + switch (this->uart_) { + case UART_SELECTION_DEFAULT: + return LOG_STR("DEFAULT"); + case UART_SELECTION_UART0: + return LOG_STR("UART0"); + case UART_SELECTION_UART1: + return LOG_STR("UART1"); + case UART_SELECTION_UART2: + default: + return LOG_STR("UART2"); + } +} -const char *Logger::get_uart_selection_() { return UART_SELECTIONS[this->uart_]; } - -} // namespace logger -} // namespace esphome +} // namespace esphome::logger #endif // USE_LIBRETINY diff --git a/esphome/components/logger/logger_rp2040.cpp b/esphome/components/logger/logger_rp2040.cpp index 2783d77ac1..63727c2cda 100644 --- a/esphome/components/logger/logger_rp2040.cpp +++ b/esphome/components/logger/logger_rp2040.cpp @@ -2,8 +2,7 @@ #include "logger.h" #include "esphome/core/log.h" -namespace esphome { -namespace logger { +namespace esphome::logger { static const char *const TAG = "logger"; @@ -30,10 +29,20 @@ void Logger::pre_setup() { void HOT Logger::write_msg_(const char *msg) { this->hw_serial_->println(msg); } -const char *const UART_SELECTIONS[] = {"UART0", "UART1", "USB_CDC"}; +const LogString *Logger::get_uart_selection_() { + switch (this->uart_) { + case UART_SELECTION_UART0: + return LOG_STR("UART0"); + case UART_SELECTION_UART1: + return LOG_STR("UART1"); +#ifdef USE_LOGGER_USB_CDC + case UART_SELECTION_USB_CDC: + return LOG_STR("USB_CDC"); +#endif + default: + return LOG_STR("UNKNOWN"); + } +} -const char *Logger::get_uart_selection_() { return UART_SELECTIONS[this->uart_]; } - -} // namespace logger -} // namespace esphome +} // namespace esphome::logger #endif // USE_RP2040 diff --git a/esphome/components/logger/logger_zephyr.cpp b/esphome/components/logger/logger_zephyr.cpp new file mode 100644 index 0000000000..fb0c7dcca3 --- /dev/null +++ b/esphome/components/logger/logger_zephyr.cpp @@ -0,0 +1,96 @@ +#ifdef USE_ZEPHYR + +#include "esphome/core/application.h" +#include "esphome/core/log.h" +#include "logger.h" + +#include +#include +#include + +namespace esphome::logger { + +static const char *const TAG = "logger"; + +#ifdef USE_LOGGER_USB_CDC +void Logger::loop() { + if (this->uart_ != UART_SELECTION_USB_CDC || nullptr == this->uart_dev_) { + return; + } + static bool opened = false; + uint32_t dtr = 0; + uart_line_ctrl_get(this->uart_dev_, UART_LINE_CTRL_DTR, &dtr); + + /* Poll if the DTR flag was set, optional */ + if (opened == dtr) { + return; + } + + if (!opened) { + App.schedule_dump_config(); + } + opened = !opened; +} +#endif + +void Logger::pre_setup() { + if (this->baud_rate_ > 0) { + static const struct device *uart_dev = nullptr; + switch (this->uart_) { + case UART_SELECTION_UART0: + uart_dev = DEVICE_DT_GET_OR_NULL(DT_NODELABEL(uart0)); + break; + case UART_SELECTION_UART1: + uart_dev = DEVICE_DT_GET_OR_NULL(DT_NODELABEL(uart1)); + break; +#ifdef USE_LOGGER_USB_CDC + case UART_SELECTION_USB_CDC: + uart_dev = DEVICE_DT_GET_OR_NULL(DT_NODELABEL(cdc_acm_uart0)); + if (device_is_ready(uart_dev)) { + usb_enable(nullptr); + } + break; +#endif + } + if (!device_is_ready(uart_dev)) { + ESP_LOGE(TAG, "%s is not ready.", LOG_STR_ARG(get_uart_selection_())); + } else { + this->uart_dev_ = uart_dev; + } + } + global_logger = this; + ESP_LOGI(TAG, "Log initialized"); +} + +void HOT Logger::write_msg_(const char *msg) { +#ifdef CONFIG_PRINTK + printk("%s\n", msg); +#endif + if (nullptr == this->uart_dev_) { + return; + } + while (*msg) { + uart_poll_out(this->uart_dev_, *msg); + ++msg; + } + uart_poll_out(this->uart_dev_, '\n'); +} + +const LogString *Logger::get_uart_selection_() { + switch (this->uart_) { + case UART_SELECTION_UART0: + return LOG_STR("UART0"); + case UART_SELECTION_UART1: + return LOG_STR("UART1"); +#ifdef USE_LOGGER_USB_CDC + case UART_SELECTION_USB_CDC: + return LOG_STR("USB_CDC"); +#endif + default: + return LOG_STR("UNKNOWN"); + } +} + +} // namespace esphome::logger + +#endif diff --git a/esphome/components/logger/select/logger_level_select.cpp b/esphome/components/logger/select/logger_level_select.cpp index b71a6e02a2..6d60a3ae47 100644 --- a/esphome/components/logger/select/logger_level_select.cpp +++ b/esphome/components/logger/select/logger_level_select.cpp @@ -1,14 +1,12 @@ #include "logger_level_select.h" -namespace esphome { -namespace logger { +namespace esphome::logger { void LoggerLevelSelect::publish_state(int level) { - auto value = this->at(level); - if (!value) { + const auto &option = this->at(level_to_index(level)); + if (!option) return; - } - Select::publish_state(value.value()); + Select::publish_state(option.value()); } void LoggerLevelSelect::setup() { @@ -17,11 +15,10 @@ void LoggerLevelSelect::setup() { } void LoggerLevelSelect::control(const std::string &value) { - auto level = this->index_of(value); - if (!level) + const auto index = this->index_of(value); + if (!index) return; - this->parent_->set_log_level(level.value()); + this->parent_->set_log_level(index_to_level(index.value())); } -} // namespace logger -} // namespace esphome +} // namespace esphome::logger diff --git a/esphome/components/logger/select/logger_level_select.h b/esphome/components/logger/select/logger_level_select.h index 2c92c84d13..0631eca45d 100644 --- a/esphome/components/logger/select/logger_level_select.h +++ b/esphome/components/logger/select/logger_level_select.h @@ -3,13 +3,18 @@ #include "esphome/components/select/select.h" #include "esphome/core/component.h" #include "esphome/components/logger/logger.h" -namespace esphome { -namespace logger { + +namespace esphome::logger { class LoggerLevelSelect : public Component, public select::Select, public Parented { public: void publish_state(int level); void setup() override; void control(const std::string &value) override; + + protected: + // Convert log level to option index (skip CONFIG at level 4) + static uint8_t level_to_index(uint8_t level) { return (level > ESPHOME_LOG_LEVEL_CONFIG) ? level - 1 : level; } + // Convert option index to log level (skip CONFIG at level 4) + static uint8_t index_to_level(uint8_t index) { return (index >= ESPHOME_LOG_LEVEL_CONFIG) ? index + 1 : index; } }; -} // namespace logger -} // namespace esphome +} // namespace esphome::logger diff --git a/esphome/components/logger/task_log_buffer.cpp b/esphome/components/logger/task_log_buffer.cpp index 24d9284f1a..b5dd9f0239 100644 --- a/esphome/components/logger/task_log_buffer.cpp +++ b/esphome/components/logger/task_log_buffer.cpp @@ -5,8 +5,7 @@ #ifdef USE_ESPHOME_TASK_LOG_BUFFER -namespace esphome { -namespace logger { +namespace esphome::logger { TaskLogBuffer::TaskLogBuffer(size_t total_buffer_size) { // Store the buffer size @@ -132,7 +131,6 @@ bool TaskLogBuffer::send_message_thread_safe(uint8_t level, const char *tag, uin return true; } -} // namespace logger -} // namespace esphome +} // namespace esphome::logger #endif // USE_ESPHOME_TASK_LOG_BUFFER diff --git a/esphome/components/logger/task_log_buffer.h b/esphome/components/logger/task_log_buffer.h index 1618a5a121..fdda07190d 100644 --- a/esphome/components/logger/task_log_buffer.h +++ b/esphome/components/logger/task_log_buffer.h @@ -11,8 +11,7 @@ #include #include -namespace esphome { -namespace logger { +namespace esphome::logger { class TaskLogBuffer { public: @@ -63,7 +62,6 @@ class TaskLogBuffer { mutable uint16_t last_processed_counter_{0}; // Tracks last processed message }; -} // namespace logger -} // namespace esphome +} // namespace esphome::logger #endif // USE_ESPHOME_TASK_LOG_BUFFER diff --git a/esphome/components/ltr390/ltr390.cpp b/esphome/components/ltr390/ltr390.cpp index cc7e686d13..c1885dcb6f 100644 --- a/esphome/components/ltr390/ltr390.cpp +++ b/esphome/components/ltr390/ltr390.cpp @@ -148,8 +148,6 @@ void LTR390Component::read_mode_(int mode_index) { } void LTR390Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // reset std::bitset<8> ctrl = this->reg(LTR390_MAIN_CTRL).get(); ctrl[LTR390_CTRL_RST] = true; diff --git a/esphome/components/ltr501/ltr501.cpp b/esphome/components/ltr501/ltr501.cpp index 12f227ab91..be5a4ddccf 100644 --- a/esphome/components/ltr501/ltr501.cpp +++ b/esphome/components/ltr501/ltr501.cpp @@ -2,6 +2,7 @@ #include "esphome/core/application.h" #include "esphome/core/helpers.h" #include "esphome/core/log.h" +#include using esphome::i2c::ErrorCode; @@ -28,30 +29,30 @@ bool operator!=(const GainTimePair &lhs, const GainTimePair &rhs) { template T get_next(const T (&array)[size], const T val) { size_t i = 0; - size_t idx = -1; - while (idx == -1 && i < size) { + size_t idx = std::numeric_limits::max(); + while (idx == std::numeric_limits::max() && i < size) { if (array[i] == val) { idx = i; break; } i++; } - if (idx == -1 || i + 1 >= size) + if (idx == std::numeric_limits::max() || i + 1 >= size) return val; return array[i + 1]; } template T get_prev(const T (&array)[size], const T val) { size_t i = size - 1; - size_t idx = -1; - while (idx == -1 && i > 0) { + size_t idx = std::numeric_limits::max(); + while (idx == std::numeric_limits::max() && i > 0) { if (array[i] == val) { idx = i; break; } i--; } - if (idx == -1 || i == 0) + if (idx == std::numeric_limits::max() || i == 0) return val; return array[i - 1]; } @@ -74,7 +75,6 @@ static float get_ps_gain_coeff(PsGain501 gain) { } void LTRAlsPs501Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // As per datasheet we need to wait at least 100ms after power on to get ALS chip responsive this->set_timeout(100, [this]() { this->state_ = State::DELAYED_SETUP; }); } diff --git a/esphome/components/ltr_als_ps/ltr_als_ps.cpp b/esphome/components/ltr_als_ps/ltr_als_ps.cpp index 9b635a12b1..c3ea5848c8 100644 --- a/esphome/components/ltr_als_ps/ltr_als_ps.cpp +++ b/esphome/components/ltr_als_ps/ltr_als_ps.cpp @@ -2,6 +2,7 @@ #include "esphome/core/application.h" #include "esphome/core/helpers.h" #include "esphome/core/log.h" +#include using esphome::i2c::ErrorCode; @@ -14,30 +15,30 @@ static const uint8_t MAX_TRIES = 5; template T get_next(const T (&array)[size], const T val) { size_t i = 0; - size_t idx = -1; - while (idx == -1 && i < size) { + size_t idx = std::numeric_limits::max(); + while (idx == std::numeric_limits::max() && i < size) { if (array[i] == val) { idx = i; break; } i++; } - if (idx == -1 || i + 1 >= size) + if (idx == std::numeric_limits::max() || i + 1 >= size) return val; return array[i + 1]; } template T get_prev(const T (&array)[size], const T val) { size_t i = size - 1; - size_t idx = -1; - while (idx == -1 && i > 0) { + size_t idx = std::numeric_limits::max(); + while (idx == std::numeric_limits::max() && i > 0) { if (array[i] == val) { idx = i; break; } i--; } - if (idx == -1 || i == 0) + if (idx == std::numeric_limits::max() || i == 0) return val; return array[i - 1]; } @@ -63,7 +64,6 @@ static float get_ps_gain_coeff(PsGain gain) { } void LTRAlsPsComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // As per datasheet we need to wait at least 100ms after power on to get ALS chip responsive this->set_timeout(100, [this]() { this->state_ = State::DELAYED_SETUP; }); } diff --git a/esphome/components/lvgl/__init__.py b/esphome/components/lvgl/__init__.py index 4a450375c4..5af61300da 100644 --- a/esphome/components/lvgl/__init__.py +++ b/esphome/components/lvgl/__init__.py @@ -2,8 +2,9 @@ import logging from esphome.automation import build_automation, register_action, validate_automation import esphome.codegen as cg -from esphome.components.const import CONF_DRAW_ROUNDING +from esphome.components.const import CONF_COLOR_DEPTH, CONF_DRAW_ROUNDING from esphome.components.display import Display +from esphome.components.psram import DOMAIN as PSRAM_DOMAIN import esphome.config_validation as cv from esphome.const import ( CONF_AUTO_CLEAR_ENABLED, @@ -11,6 +12,7 @@ from esphome.const import ( CONF_GROUP, CONF_ID, CONF_LAMBDA, + CONF_LOG_LEVEL, CONF_ON_BOOT, CONF_ON_IDLE, CONF_PAGES, @@ -185,8 +187,8 @@ def multi_conf_validate(configs: list[dict]): base_config = configs[0] for config in configs[1:]: for item in ( - df.CONF_LOG_LEVEL, - df.CONF_COLOR_DEPTH, + CONF_LOG_LEVEL, + CONF_COLOR_DEPTH, df.CONF_BYTE_ORDER, df.CONF_TRANSPARENCY_KEY, ): @@ -201,9 +203,8 @@ def final_validation(configs): multi_conf_validate(configs) global_config = full_config.get() for config in configs: - if pages := config.get(CONF_PAGES): - if all(p[df.CONF_SKIP] for p in pages): - raise cv.Invalid("At least one page must not be skipped") + if (pages := config.get(CONF_PAGES)) and all(p[df.CONF_SKIP] for p in pages): + raise cv.Invalid("At least one page must not be skipped") for display_id in config[df.CONF_DISPLAYS]: path = global_config.get_path_for_id(display_id)[:-1] display = global_config.get_config_for_path(path) @@ -220,7 +221,7 @@ def final_validation(configs): draw_rounding, config[CONF_DRAW_ROUNDING] ) buffer_frac = config[CONF_BUFFER_SIZE] - if CORE.is_esp32 and buffer_frac > 0.5 and "psram" not in global_config: + if CORE.is_esp32 and buffer_frac > 0.5 and PSRAM_DOMAIN not in global_config: LOGGER.warning("buffer_size: may need to be reduced without PSRAM") for image_id in lv_images_used: path = global_config.get_path_for_id(image_id)[:-1] @@ -269,17 +270,17 @@ async def to_code(configs): add_define( "LV_LOG_LEVEL", - f"LV_LOG_LEVEL_{df.LV_LOG_LEVELS[config_0[df.CONF_LOG_LEVEL]]}", + f"LV_LOG_LEVEL_{df.LV_LOG_LEVELS[config_0[CONF_LOG_LEVEL]]}", ) cg.add_define( "LVGL_LOG_LEVEL", - cg.RawExpression(f"ESPHOME_LOG_LEVEL_{config_0[df.CONF_LOG_LEVEL]}"), + cg.RawExpression(f"ESPHOME_LOG_LEVEL_{config_0[CONF_LOG_LEVEL]}"), ) - add_define("LV_COLOR_DEPTH", config_0[df.CONF_COLOR_DEPTH]) + add_define("LV_COLOR_DEPTH", config_0[CONF_COLOR_DEPTH]) for font in helpers.lv_fonts_used: add_define(f"LV_FONT_{font.upper()}") - if config_0[df.CONF_COLOR_DEPTH] == 16: + if config_0[CONF_COLOR_DEPTH] == 16: add_define( "LV_COLOR_16_SWAP", "1" if config_0[df.CONF_BYTE_ORDER] == "big_endian" else "0", @@ -416,14 +417,14 @@ LVGL_SCHEMA = cv.All( { cv.GenerateID(CONF_ID): cv.declare_id(LvglComponent), cv.GenerateID(df.CONF_DISPLAYS): display_schema, - cv.Optional(df.CONF_COLOR_DEPTH, default=16): cv.one_of(16), + cv.Optional(CONF_COLOR_DEPTH, default=16): cv.one_of(16), cv.Optional( df.CONF_DEFAULT_FONT, default="montserrat_14" ): lvalid.lv_font, cv.Optional(df.CONF_FULL_REFRESH, default=False): cv.boolean, cv.Optional(CONF_DRAW_ROUNDING, default=2): cv.positive_int, cv.Optional(CONF_BUFFER_SIZE, default=0): cv.percentage, - cv.Optional(df.CONF_LOG_LEVEL, default="WARN"): cv.one_of( + cv.Optional(CONF_LOG_LEVEL, default="WARN"): cv.one_of( *df.LV_LOG_LEVELS, upper=True ), cv.Optional(df.CONF_BYTE_ORDER, default="big_endian"): cv.one_of( diff --git a/esphome/components/lvgl/automation.py b/esphome/components/lvgl/automation.py index cc0f833ced..fc70b0f682 100644 --- a/esphome/components/lvgl/automation.py +++ b/esphome/components/lvgl/automation.py @@ -85,8 +85,7 @@ async def action_to_code( async with LambdaContext(parameters=args, where=action_id) as context: for widget in widgets: await action(widget) - var = cg.new_Pvariable(action_id, template_arg, await context.get_lambda()) - return var + return cg.new_Pvariable(action_id, template_arg, await context.get_lambda()) async def update_to_code(config, action_id, template_arg, args): @@ -354,8 +353,7 @@ async def widget_focus(config, action_id, template_arg, args): if config[CONF_FREEZE]: lv.group_focus_freeze(group, True) - var = cg.new_Pvariable(action_id, template_arg, await context.get_lambda()) - return var + return cg.new_Pvariable(action_id, template_arg, await context.get_lambda()) @automation.register_action( diff --git a/esphome/components/lvgl/defines.py b/esphome/components/lvgl/defines.py index baa9a19c51..6464824c64 100644 --- a/esphome/components/lvgl/defines.py +++ b/esphome/components/lvgl/defines.py @@ -418,7 +418,6 @@ CONF_BUTTONS = "buttons" CONF_BYTE_ORDER = "byte_order" CONF_CHANGE_RATE = "change_rate" CONF_CLOSE_BUTTON = "close_button" -CONF_COLOR_DEPTH = "color_depth" CONF_CONTROL = "control" CONF_DEFAULT_FONT = "default_font" CONF_DEFAULT_GROUP = "default_group" @@ -452,12 +451,12 @@ CONF_GRID_ROWS = "grid_rows" CONF_HEADER_MODE = "header_mode" CONF_HOME = "home" CONF_INITIAL_FOCUS = "initial_focus" +CONF_SELECTED_DIGIT = "selected_digit" CONF_KEY_CODE = "key_code" CONF_KEYPADS = "keypads" CONF_LAYOUT = "layout" CONF_LEFT_BUTTON = "left_button" CONF_LINE_WIDTH = "line_width" -CONF_LOG_LEVEL = "log_level" CONF_LONG_PRESS_TIME = "long_press_time" CONF_LONG_PRESS_REPEAT_TIME = "long_press_repeat_time" CONF_LVGL_ID = "lvgl_id" @@ -487,7 +486,6 @@ CONF_RESUME_ON_INPUT = "resume_on_input" CONF_RIGHT_BUTTON = "right_button" CONF_ROLLOVER = "rollover" CONF_ROOT_BACK_BTN = "root_back_btn" -CONF_ROWS = "rows" CONF_SCALE_LINES = "scale_lines" CONF_SCROLLBAR_MODE = "scrollbar_mode" CONF_SELECTED_INDEX = "selected_index" diff --git a/esphome/components/lvgl/hello_world.py b/esphome/components/lvgl/hello_world.py index 2c2ec6732c..f85da9d8e4 100644 --- a/esphome/components/lvgl/hello_world.py +++ b/esphome/components/lvgl/hello_world.py @@ -4,49 +4,112 @@ from esphome.yaml_util import parse_yaml CONFIG = """ - obj: - radius: 0 + id: hello_world_card_ pad_all: 12 - bg_color: 0xFFFFFF + bg_color: white height: 100% width: 100% + scrollable: false widgets: - - spinner: - id: hello_world_spinner_ - align: center - indicator: - arc_color: tomato - height: 100 - width: 100 - spin_time: 2s - arc_length: 60deg - - label: - id: hello_world_label_ - text: "Hello World!" + - obj: + align: top_mid + outline_width: 0 + border_width: 0 + pad_all: 4 + scrollable: false + height: size_content + width: 100% + layout: + type: flex + flex_flow: row + flex_align_cross: center + flex_align_track: start + flex_align_main: space_between + widgets: + - button: + checkable: true + radius: 4 + text_font: montserrat_20 + on_click: + lvgl.label.update: + id: hello_world_label_ + text: "Clicked!" + widgets: + - label: + text: "Button" + - label: + id: hello_world_title_ + text: ESPHome + text_font: montserrat_20 + width: 100% + text_align: center + on_boot: + lvgl.widget.refresh: hello_world_title_ + hidden: !lambda |- + return lv_obj_get_width(lv_scr_act()) < 400; + - checkbox: + text: Checkbox + id: hello_world_checkbox_ + on_boot: + lvgl.widget.refresh: hello_world_checkbox_ + hidden: !lambda |- + return lv_obj_get_width(lv_scr_act()) < 240; + on_click: + lvgl.label.update: + id: hello_world_label_ + text: "Checked!" + - obj: + id: hello_world_container_ align: center + y: 14 + pad_all: 0 + outline_width: 0 + border_width: 0 + width: 100% + height: size_content + scrollable: false on_click: lvgl.spinner.update: id: hello_world_spinner_ arc_color: springgreen - - checkbox: - pad_all: 8 - text: Checkbox - align: top_right - on_click: - lvgl.label.update: - id: hello_world_label_ - text: "Checked!" - - button: - pad_all: 8 - checkable: true - align: top_left - text_font: montserrat_20 - on_click: - lvgl.label.update: - id: hello_world_label_ - text: "Clicked!" + layout: + type: flex + flex_flow: row_wrap + flex_align_cross: center + flex_align_track: center + flex_align_main: space_evenly widgets: - - label: - text: "Button" + - spinner: + id: hello_world_spinner_ + indicator: + arc_color: tomato + height: 100 + width: 100 + spin_time: 2s + arc_length: 60deg + widgets: + - label: + id: hello_world_label_ + text: "Hello World!" + align: center + - obj: + id: hello_world_qrcode_ + outline_width: 0 + border_width: 0 + hidden: !lambda |- + return lv_obj_get_width(lv_scr_act()) < 300 && lv_obj_get_height(lv_scr_act()) < 400; + widgets: + - label: + text_font: montserrat_14 + text: esphome.io + align: top_mid + - qrcode: + text: "https://esphome.io" + size: 80 + align: bottom_mid + on_boot: + lvgl.widget.refresh: hello_world_qrcode_ + - slider: width: 80% align: bottom_mid diff --git a/esphome/components/lvgl/helpers.py b/esphome/components/lvgl/helpers.py index e04a0105d5..8d5b6354bb 100644 --- a/esphome/components/lvgl/helpers.py +++ b/esphome/components/lvgl/helpers.py @@ -33,7 +33,7 @@ def validate_printf(value): [cCdiouxXeEfgGaAnpsSZ] # type ) """ # noqa - matches = re.findall(cfmt, value[CONF_FORMAT], flags=re.X) + matches = re.findall(cfmt, value[CONF_FORMAT], flags=re.VERBOSE) if len(matches) != len(value[CONF_ARGS]): raise cv.Invalid( f"Found {len(matches)} printf-patterns ({', '.join(matches)}), but {len(value[CONF_ARGS])} args were given!" diff --git a/esphome/components/lvgl/lv_validation.py b/esphome/components/lvgl/lv_validation.py index 92fe74eb52..d345ac70f3 100644 --- a/esphome/components/lvgl/lv_validation.py +++ b/esphome/components/lvgl/lv_validation.py @@ -271,8 +271,7 @@ padding = LValidator(padding_validator, int32, retmapper=literal) def zoom_validator(value): - value = cv.float_range(0.1, 10.0)(value) - return value + return cv.float_range(0.1, 10.0)(value) def zoom_retmapper(value): @@ -288,10 +287,14 @@ def angle(value): :param value: The input in the range 0..360 :return: An angle in 1/10 degree units. """ - return int(cv.float_range(0.0, 360.0)(cv.angle(value)) * 10) + return cv.float_range(0.0, 360.0)(cv.angle(value)) -lv_angle = LValidator(angle, uint32) +# Validator for angles in LVGL expressed in 1/10 degree units. +lv_angle = LValidator(angle, uint32, retmapper=lambda x: int(x * 10)) + +# Validator for angles in LVGL expressed in whole degrees +lv_angle_degrees = LValidator(angle, uint32, retmapper=int) @schema_extractor("one_of") diff --git a/esphome/components/lvgl/lvgl_esphome.cpp b/esphome/components/lvgl/lvgl_esphome.cpp index dd877df0f0..7a32691b53 100644 --- a/esphome/components/lvgl/lvgl_esphome.cpp +++ b/esphome/components/lvgl/lvgl_esphome.cpp @@ -434,7 +434,6 @@ LvglComponent::LvglComponent(std::vector displays, float buf } void LvglComponent::setup() { - ESP_LOGCONFIG(TAG, "LVGL Setup starts"); auto *display = this->displays_[0]; auto width = display->get_width(); auto height = display->get_height(); @@ -452,7 +451,8 @@ void LvglComponent::setup() { if (buffer == nullptr && this->buffer_frac_ == 0) { frac = MIN_BUFFER_FRAC; buffer_pixels /= MIN_BUFFER_FRAC; - buffer = lv_custom_mem_alloc(buf_bytes / MIN_BUFFER_FRAC); // NOLINT + buf_bytes /= MIN_BUFFER_FRAC; + buffer = lv_custom_mem_alloc(buf_bytes); // NOLINT } if (buffer == nullptr) { this->status_set_error("Memory allocation failure"); @@ -489,7 +489,6 @@ void LvglComponent::setup() { disp->set_rotation(display::DISPLAY_ROTATION_0_DEGREES); this->show_page(0, LV_SCR_LOAD_ANIM_NONE, 0); lv_disp_trig_activity(this->disp_); - ESP_LOGCONFIG(TAG, "LVGL Setup complete"); } void LvglComponent::update() { diff --git a/esphome/components/lvgl/number/lvgl_number.h b/esphome/components/lvgl/number/lvgl_number.h index 277494673b..7bc44c9e20 100644 --- a/esphome/components/lvgl/number/lvgl_number.h +++ b/esphome/components/lvgl/number/lvgl_number.h @@ -21,7 +21,7 @@ class LVGLNumber : public number::Number, public Component { void setup() override { float value = this->value_lambda_(); if (this->restore_) { - this->pref_ = global_preferences->make_preference(this->get_object_id_hash()); + this->pref_ = global_preferences->make_preference(this->get_preference_hash()); if (this->pref_.load(&value)) { this->control_lambda_(value); } diff --git a/esphome/components/lvgl/select/lvgl_select.h b/esphome/components/lvgl/select/lvgl_select.h index 5b43209a5f..a0e60295a6 100644 --- a/esphome/components/lvgl/select/lvgl_select.h +++ b/esphome/components/lvgl/select/lvgl_select.h @@ -20,7 +20,7 @@ class LVGLSelect : public select::Select, public Component { this->set_options_(); if (this->restore_) { size_t index; - this->pref_ = global_preferences->make_preference(this->get_object_id_hash()); + this->pref_ = global_preferences->make_preference(this->get_preference_hash()); if (this->pref_.load(&index)) this->widget_->set_selected_index(index, LV_ANIM_OFF); } diff --git a/esphome/components/lvgl/styles.py b/esphome/components/lvgl/styles.py index 426dd3f229..3969c9f388 100644 --- a/esphome/components/lvgl/styles.py +++ b/esphome/components/lvgl/styles.py @@ -66,8 +66,7 @@ async def style_update_to_code(config, action_id, template_arg, args): async with LambdaContext(parameters=args, where=action_id) as context: await style_set(style, config) - var = cg.new_Pvariable(action_id, template_arg, await context.get_lambda()) - return var + return cg.new_Pvariable(action_id, template_arg, await context.get_lambda()) async def theme_to_code(config): @@ -76,6 +75,7 @@ async def theme_to_code(config): for w_name, style in theme.items(): # Work around Python 3.10 bug with nested async comprehensions # With Python 3.11 this could be simplified + # TODO: Now that we require Python 3.11+, this can be updated to use nested comprehensions styles = {} for part, states in collect_parts(style).items(): styles[part] = { diff --git a/esphome/components/lvgl/types.py b/esphome/components/lvgl/types.py index 40e69119f0..c19c89401a 100644 --- a/esphome/components/lvgl/types.py +++ b/esphome/components/lvgl/types.py @@ -161,7 +161,7 @@ class WidgetType: """ return [] - def obj_creator(self, parent: MockObjClass, config: dict): + async def obj_creator(self, parent: MockObjClass, config: dict): """ Create an instance of the widget type :param parent: The parent to which it should be attached @@ -192,7 +192,7 @@ class WidgetType: class NumberType(WidgetType): def get_max(self, config: dict): - return int(config[CONF_MAX_VALUE] or 100) + return int(config.get(CONF_MAX_VALUE, 100)) def get_min(self, config: dict): - return int(config[CONF_MIN_VALUE] or 0) + return int(config.get(CONF_MIN_VALUE, 0)) diff --git a/esphome/components/lvgl/widgets/__init__.py b/esphome/components/lvgl/widgets/__init__.py index a8cb8dce33..1f9cdde0a0 100644 --- a/esphome/components/lvgl/widgets/__init__.py +++ b/esphome/components/lvgl/widgets/__init__.py @@ -67,7 +67,6 @@ class Widget: self.type = wtype self.config = config self.scale = 1.0 - self.step = 1.0 self.range_from = -sys.maxsize self.range_to = sys.maxsize if wtype.is_compound(): @@ -189,7 +188,7 @@ class Widget: for matrix buttons :return: """ - return None + return def get_max(self): return self.type.get_max(self.config) @@ -439,7 +438,7 @@ async def widget_to_code(w_cnfig, w_type: WidgetType, parent): :return: """ spec: WidgetType = WIDGET_TYPES[w_type] - creator = spec.obj_creator(parent, w_cnfig) + creator = await spec.obj_creator(parent, w_cnfig) add_lv_use(spec.name) add_lv_use(*spec.get_uses()) wid = w_cnfig[CONF_ID] diff --git a/esphome/components/lvgl/widgets/arc.py b/esphome/components/lvgl/widgets/arc.py index 65f0e785b6..ef4da0d815 100644 --- a/esphome/components/lvgl/widgets/arc.py +++ b/esphome/components/lvgl/widgets/arc.py @@ -20,7 +20,7 @@ from ..defines import ( CONF_START_ANGLE, literal, ) -from ..lv_validation import angle, get_start_value, lv_float +from ..lv_validation import get_start_value, lv_angle_degrees, lv_float, lv_int from ..lvcode import lv, lv_expr, lv_obj from ..types import LvNumber, NumberType from . import Widget @@ -29,11 +29,11 @@ CONF_ARC = "arc" ARC_SCHEMA = cv.Schema( { cv.Optional(CONF_VALUE): lv_float, - cv.Optional(CONF_MIN_VALUE, default=0): cv.int_, - cv.Optional(CONF_MAX_VALUE, default=100): cv.int_, - cv.Optional(CONF_START_ANGLE, default=135): angle, - cv.Optional(CONF_END_ANGLE, default=45): angle, - cv.Optional(CONF_ROTATION, default=0.0): angle, + cv.Optional(CONF_MIN_VALUE, default=0): lv_int, + cv.Optional(CONF_MAX_VALUE, default=100): lv_int, + cv.Optional(CONF_START_ANGLE, default=135): lv_angle_degrees, + cv.Optional(CONF_END_ANGLE, default=45): lv_angle_degrees, + cv.Optional(CONF_ROTATION, default=0.0): lv_angle_degrees, cv.Optional(CONF_ADJUSTABLE, default=False): bool, cv.Optional(CONF_MODE, default="NORMAL"): ARC_MODES.one_of, cv.Optional(CONF_CHANGE_RATE, default=720): cv.uint16_t, @@ -59,11 +59,14 @@ class ArcType(NumberType): async def to_code(self, w: Widget, config): if CONF_MIN_VALUE in config: - lv.arc_set_range(w.obj, config[CONF_MIN_VALUE], config[CONF_MAX_VALUE]) - lv.arc_set_bg_angles( - w.obj, config[CONF_START_ANGLE] // 10, config[CONF_END_ANGLE] // 10 - ) - lv.arc_set_rotation(w.obj, config[CONF_ROTATION] // 10) + max_value = await lv_int.process(config[CONF_MAX_VALUE]) + min_value = await lv_int.process(config[CONF_MIN_VALUE]) + lv.arc_set_range(w.obj, min_value, max_value) + start = await lv_angle_degrees.process(config[CONF_START_ANGLE]) + end = await lv_angle_degrees.process(config[CONF_END_ANGLE]) + rotation = await lv_angle_degrees.process(config[CONF_ROTATION]) + lv.arc_set_bg_angles(w.obj, start, end) + lv.arc_set_rotation(w.obj, rotation) lv.arc_set_mode(w.obj, literal(config[CONF_MODE])) lv.arc_set_change_rate(w.obj, config[CONF_CHANGE_RATE]) diff --git a/esphome/components/lvgl/widgets/buttonmatrix.py b/esphome/components/lvgl/widgets/buttonmatrix.py index aa33be722c..baeb1c8e3e 100644 --- a/esphome/components/lvgl/widgets/buttonmatrix.py +++ b/esphome/components/lvgl/widgets/buttonmatrix.py @@ -2,7 +2,7 @@ from esphome import automation import esphome.codegen as cg from esphome.components.key_provider import KeyProvider import esphome.config_validation as cv -from esphome.const import CONF_ID, CONF_ITEMS, CONF_TEXT, CONF_WIDTH +from esphome.const import CONF_ID, CONF_ITEMS, CONF_ROWS, CONF_TEXT, CONF_WIDTH from esphome.cpp_generator import MockObj from ..automation import action_to_code @@ -15,7 +15,6 @@ from ..defines import ( CONF_ONE_CHECKED, CONF_PAD_COLUMN, CONF_PAD_ROW, - CONF_ROWS, CONF_SELECTED, ) from ..helpers import lvgl_components_required @@ -193,7 +192,7 @@ class ButtonMatrixType(WidgetType): async def to_code(self, w: Widget, config): lvgl_components_required.add("BUTTONMATRIX") if CONF_ROWS not in config: - return [] + return text_list, ctrl_list, width_list, key_list = await get_button_data( config[CONF_ROWS], w ) diff --git a/esphome/components/lvgl/widgets/canvas.py b/esphome/components/lvgl/widgets/canvas.py index 4fd81b6e4a..217e8935f1 100644 --- a/esphome/components/lvgl/widgets/canvas.py +++ b/esphome/components/lvgl/widgets/canvas.py @@ -24,7 +24,7 @@ from ..defines import ( literal, ) from ..lv_validation import ( - lv_angle, + lv_angle_degrees, lv_bool, lv_color, lv_image, @@ -395,15 +395,15 @@ ARC_PROPS = { DRAW_OPA_SCHEMA.extend( { cv.Required(CONF_RADIUS): pixels, - cv.Required(CONF_START_ANGLE): lv_angle, - cv.Required(CONF_END_ANGLE): lv_angle, + cv.Required(CONF_START_ANGLE): lv_angle_degrees, + cv.Required(CONF_END_ANGLE): lv_angle_degrees, } ).extend({cv.Optional(prop): validator for prop, validator in ARC_PROPS.items()}), ) async def canvas_draw_arc(config, action_id, template_arg, args): radius = await size.process(config[CONF_RADIUS]) - start_angle = await lv_angle.process(config[CONF_START_ANGLE]) - end_angle = await lv_angle.process(config[CONF_END_ANGLE]) + start_angle = await lv_angle_degrees.process(config[CONF_START_ANGLE]) + end_angle = await lv_angle_degrees.process(config[CONF_END_ANGLE]) async def do_draw_arc(w: Widget, x, y, dsc_addr): lv.canvas_draw_arc(w.obj, x, y, radius, start_angle, end_angle, dsc_addr) diff --git a/esphome/components/lvgl/widgets/meter.py b/esphome/components/lvgl/widgets/meter.py index f836a1eca5..aefda0e71a 100644 --- a/esphome/components/lvgl/widgets/meter.py +++ b/esphome/components/lvgl/widgets/meter.py @@ -31,7 +31,7 @@ from ..helpers import add_lv_use, lvgl_components_required from ..lv_validation import ( get_end_value, get_start_value, - lv_angle, + lv_angle_degrees, lv_bool, lv_color, lv_float, @@ -162,7 +162,7 @@ SCALE_SCHEMA = cv.Schema( cv.Optional(CONF_RANGE_FROM, default=0.0): cv.float_, cv.Optional(CONF_RANGE_TO, default=100.0): cv.float_, cv.Optional(CONF_ANGLE_RANGE, default=270): cv.int_range(0, 360), - cv.Optional(CONF_ROTATION): lv_angle, + cv.Optional(CONF_ROTATION): lv_angle_degrees, cv.Optional(CONF_INDICATORS): cv.ensure_list(INDICATOR_SCHEMA), } ) @@ -187,7 +187,7 @@ class MeterType(WidgetType): for scale_conf in config.get(CONF_SCALES, ()): rotation = 90 + (360 - scale_conf[CONF_ANGLE_RANGE]) / 2 if CONF_ROTATION in scale_conf: - rotation = await lv_angle.process(scale_conf[CONF_ROTATION]) + rotation = await lv_angle_degrees.process(scale_conf[CONF_ROTATION]) with LocalVariable( "meter_var", "lv_meter_scale_t", lv_expr.meter_add_scale(var) ) as meter_var: @@ -264,7 +264,7 @@ class MeterType(WidgetType): color_start, color_end, v[CONF_LOCAL], - size.process(v[CONF_WIDTH]), + await size.process(v[CONF_WIDTH]), ), ) if t == CONF_IMAGE: diff --git a/esphome/components/lvgl/widgets/qrcode.py b/esphome/components/lvgl/widgets/qrcode.py index 7d8d13d8c4..028a81b449 100644 --- a/esphome/components/lvgl/widgets/qrcode.py +++ b/esphome/components/lvgl/widgets/qrcode.py @@ -4,7 +4,7 @@ from esphome.const import CONF_SIZE, CONF_TEXT from esphome.cpp_generator import MockObjClass from ..defines import CONF_MAIN -from ..lv_validation import color, color_retmapper, lv_text +from ..lv_validation import lv_color, lv_text from ..lvcode import LocalVariable, lv, lv_expr from ..schemas import TEXT_SCHEMA from ..types import WidgetType, lv_obj_t @@ -16,8 +16,8 @@ CONF_LIGHT_COLOR = "light_color" QRCODE_SCHEMA = TEXT_SCHEMA.extend( { - cv.Optional(CONF_DARK_COLOR, default="black"): color, - cv.Optional(CONF_LIGHT_COLOR, default="white"): color, + cv.Optional(CONF_DARK_COLOR, default="black"): lv_color, + cv.Optional(CONF_LIGHT_COLOR, default="white"): lv_color, cv.Required(CONF_SIZE): cv.int_, } ) @@ -34,11 +34,11 @@ class QrCodeType(WidgetType): ) def get_uses(self): - return ("canvas", "img", "label") + return "canvas", "img", "label" - def obj_creator(self, parent: MockObjClass, config: dict): - dark_color = color_retmapper(config[CONF_DARK_COLOR]) - light_color = color_retmapper(config[CONF_LIGHT_COLOR]) + async def obj_creator(self, parent: MockObjClass, config: dict): + dark_color = await lv_color.process(config[CONF_DARK_COLOR]) + light_color = await lv_color.process(config[CONF_LIGHT_COLOR]) size = config[CONF_SIZE] return lv_expr.call("qrcode_create", parent, size, dark_color, light_color) diff --git a/esphome/components/lvgl/widgets/spinbox.py b/esphome/components/lvgl/widgets/spinbox.py index b84dc7cd23..26ad149c6f 100644 --- a/esphome/components/lvgl/widgets/spinbox.py +++ b/esphome/components/lvgl/widgets/spinbox.py @@ -11,6 +11,7 @@ from ..defines import ( CONF_ROLLOVER, CONF_SCROLLBAR, CONF_SELECTED, + CONF_SELECTED_DIGIT, CONF_TEXTAREA_PLACEHOLDER, ) from ..lv_validation import lv_bool, lv_float @@ -38,18 +39,24 @@ def validate_spinbox(config): min_val = -1 - max_val range_from = int(config[CONF_RANGE_FROM]) range_to = int(config[CONF_RANGE_TO]) - step = int(config[CONF_STEP]) + step = config[CONF_SELECTED_DIGIT] + digits = config[CONF_DIGITS] if ( range_from > max_val or range_from < min_val or range_to > max_val or range_to < min_val ): - raise cv.Invalid("Range outside allowed limits") - if step <= 0 or step >= (range_to - range_from) / 2: - raise cv.Invalid("Invalid step value") - if config[CONF_DIGITS] <= config[CONF_DECIMAL_PLACES]: - raise cv.Invalid("Number of digits must exceed number of decimal places") + raise cv.Invalid("Range outside allowed limits", path=[CONF_RANGE_FROM]) + if digits <= config[CONF_DECIMAL_PLACES]: + raise cv.Invalid( + "Number of digits must exceed number of decimal places", path=[CONF_DIGITS] + ) + if step >= digits: + raise cv.Invalid( + "Initial selected digit must be less than number of digits", + path=[CONF_SELECTED_DIGIT], + ) return config @@ -59,7 +66,10 @@ SPINBOX_SCHEMA = cv.Schema( cv.Optional(CONF_RANGE_FROM, default=0): cv.float_, cv.Optional(CONF_RANGE_TO, default=100): cv.float_, cv.Optional(CONF_DIGITS, default=4): cv.int_range(1, 10), - cv.Optional(CONF_STEP, default=1.0): cv.positive_float, + cv.Optional(CONF_STEP): cv.invalid( + f"{CONF_STEP} has been replaced by {CONF_SELECTED_DIGIT}" + ), + cv.Optional(CONF_SELECTED_DIGIT, default=0): cv.positive_int, cv.Optional(CONF_DECIMAL_PLACES, default=0): cv.int_range(0, 6), cv.Optional(CONF_ROLLOVER, default=False): lv_bool, } @@ -93,13 +103,12 @@ class SpinboxType(WidgetType): scale = 10 ** config[CONF_DECIMAL_PLACES] range_from = int(config[CONF_RANGE_FROM]) * scale range_to = int(config[CONF_RANGE_TO]) * scale - step = int(config[CONF_STEP]) * scale + step = config[CONF_SELECTED_DIGIT] w.scale = scale - w.step = step w.range_to = range_to w.range_from = range_from lv.spinbox_set_range(w.obj, range_from, range_to) - await w.set_property(CONF_STEP, step) + await w.set_property("step", 10**step) await w.set_property(CONF_ROLLOVER, config) lv.spinbox_set_digit_format( w.obj, digits, digits - config[CONF_DECIMAL_PLACES] @@ -120,7 +129,7 @@ class SpinboxType(WidgetType): return config[CONF_RANGE_FROM] def get_step(self, config: dict): - return config[CONF_STEP] + return 10 ** config[CONF_SELECTED_DIGIT] spinbox_spec = SpinboxType() diff --git a/esphome/components/lvgl/widgets/spinner.py b/esphome/components/lvgl/widgets/spinner.py index 2940feb594..83aac25a59 100644 --- a/esphome/components/lvgl/widgets/spinner.py +++ b/esphome/components/lvgl/widgets/spinner.py @@ -2,7 +2,7 @@ import esphome.config_validation as cv from esphome.cpp_generator import MockObjClass from ..defines import CONF_ARC_LENGTH, CONF_INDICATOR, CONF_MAIN, CONF_SPIN_TIME -from ..lv_validation import angle +from ..lv_validation import lv_angle_degrees, lv_milliseconds from ..lvcode import lv_expr from ..types import LvType from . import Widget, WidgetType @@ -12,8 +12,8 @@ CONF_SPINNER = "spinner" SPINNER_SCHEMA = cv.Schema( { - cv.Required(CONF_ARC_LENGTH): angle, - cv.Required(CONF_SPIN_TIME): cv.positive_time_period_milliseconds, + cv.Required(CONF_ARC_LENGTH): lv_angle_degrees, + cv.Required(CONF_SPIN_TIME): lv_milliseconds, } ) @@ -34,9 +34,9 @@ class SpinnerType(WidgetType): def get_uses(self): return (CONF_ARC,) - def obj_creator(self, parent: MockObjClass, config: dict): - spin_time = config[CONF_SPIN_TIME].total_milliseconds - arc_length = config[CONF_ARC_LENGTH] // 10 + async def obj_creator(self, parent: MockObjClass, config: dict): + spin_time = await lv_milliseconds.process(config[CONF_SPIN_TIME]) + arc_length = await lv_angle_degrees.process(config[CONF_ARC_LENGTH]) return lv_expr.call("spinner_create", parent, spin_time, arc_length) diff --git a/esphome/components/lvgl/widgets/switch.py b/esphome/components/lvgl/widgets/switch.py index a7c1356bf2..06738faae5 100644 --- a/esphome/components/lvgl/widgets/switch.py +++ b/esphome/components/lvgl/widgets/switch.py @@ -1,9 +1,9 @@ +from esphome.const import CONF_SWITCH + from ..defines import CONF_INDICATOR, CONF_KNOB, CONF_MAIN from ..types import LvBoolean from . import WidgetType -CONF_SWITCH = "switch" - class SwitchType(WidgetType): def __init__(self): diff --git a/esphome/components/lvgl/widgets/tabview.py b/esphome/components/lvgl/widgets/tabview.py index 42cf486e1c..e8931bab7c 100644 --- a/esphome/components/lvgl/widgets/tabview.py +++ b/esphome/components/lvgl/widgets/tabview.py @@ -87,12 +87,12 @@ class TabviewType(WidgetType): ) as content_obj: await set_obj_properties(Widget(content_obj, obj_spec), content_style) - def obj_creator(self, parent: MockObjClass, config: dict): + async def obj_creator(self, parent: MockObjClass, config: dict): return lv_expr.call( "tabview_create", parent, - literal(config[CONF_POSITION]), - literal(config[CONF_SIZE]), + await DIRECTIONS.process(config[CONF_POSITION]), + await size.process(config[CONF_SIZE]), ) diff --git a/esphome/components/lvgl/widgets/tileview.py b/esphome/components/lvgl/widgets/tileview.py index 3865d404e2..5e3a95f017 100644 --- a/esphome/components/lvgl/widgets/tileview.py +++ b/esphome/components/lvgl/widgets/tileview.py @@ -15,7 +15,7 @@ from ..defines import ( TILE_DIRECTIONS, literal, ) -from ..lv_validation import animated, lv_int +from ..lv_validation import animated, lv_int, lv_pct from ..lvcode import lv, lv_assign, lv_expr, lv_obj, lv_Pvariable from ..schemas import container_schema from ..types import LV_EVENT, LvType, ObjUpdateAction, lv_obj_t, lv_obj_t_ptr @@ -41,8 +41,8 @@ TILEVIEW_SCHEMA = cv.Schema( container_schema( obj_spec, { - cv.Required(CONF_ROW): lv_int, - cv.Required(CONF_COLUMN): lv_int, + cv.Required(CONF_ROW): cv.positive_int, + cv.Required(CONF_COLUMN): cv.positive_int, cv.GenerateID(): cv.declare_id(lv_tile_t), cv.Optional(CONF_DIR, default="ALL"): TILE_DIRECTIONS.several_of, }, @@ -63,21 +63,29 @@ class TileviewType(WidgetType): ) async def to_code(self, w: Widget, config: dict): - for tile_conf in config.get(CONF_TILES, ()): + tiles = config[CONF_TILES] + for tile_conf in tiles: w_id = tile_conf[CONF_ID] tile_obj = lv_Pvariable(lv_obj_t, w_id) tile = Widget.create(w_id, tile_obj, tile_spec, tile_conf) dirs = tile_conf[CONF_DIR] if isinstance(dirs, list): dirs = "|".join(dirs) + row_pos = tile_conf[CONF_ROW] + col_pos = tile_conf[CONF_COLUMN] lv_assign( tile_obj, - lv_expr.tileview_add_tile( - w.obj, tile_conf[CONF_COLUMN], tile_conf[CONF_ROW], literal(dirs) - ), + lv_expr.tileview_add_tile(w.obj, col_pos, row_pos, literal(dirs)), ) + # Bugfix for LVGL 8.x + lv_obj.set_pos(tile_obj, lv_pct(col_pos * 100), lv_pct(row_pos * 100)) await set_obj_properties(tile, tile_conf) await add_widgets(tile, tile_conf) + if tiles: + # Set the first tile as active + lv_obj.set_tile_id( + w.obj, tiles[0][CONF_COLUMN], tiles[0][CONF_ROW], literal("LV_ANIM_OFF") + ) tileview_spec = TileviewType() diff --git a/esphome/components/m5stack_8angle/binary_sensor/m5stack_8angle_binary_sensor.cpp b/esphome/components/m5stack_8angle/binary_sensor/m5stack_8angle_binary_sensor.cpp index 2f68d9f254..3eeba4a644 100644 --- a/esphome/components/m5stack_8angle/binary_sensor/m5stack_8angle_binary_sensor.cpp +++ b/esphome/components/m5stack_8angle/binary_sensor/m5stack_8angle_binary_sensor.cpp @@ -6,7 +6,7 @@ namespace m5stack_8angle { void M5Stack8AngleSwitchBinarySensor::update() { int8_t out = this->parent_->read_switch(); if (out == -1) { - this->status_set_warning("Could not read binary sensor state from M5Stack 8Angle."); + this->status_set_warning(LOG_STR("Could not read binary sensor state from M5Stack 8Angle.")); return; } this->publish_state(out != 0); diff --git a/esphome/components/m5stack_8angle/m5stack_8angle.cpp b/esphome/components/m5stack_8angle/m5stack_8angle.cpp index 416b903816..c542b4459e 100644 --- a/esphome/components/m5stack_8angle/m5stack_8angle.cpp +++ b/esphome/components/m5stack_8angle/m5stack_8angle.cpp @@ -8,7 +8,6 @@ namespace m5stack_8angle { static const char *const TAG = "m5stack_8angle"; void M5Stack8AngleComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); i2c::ErrorCode err; err = this->read(nullptr, 0); diff --git a/esphome/components/m5stack_8angle/sensor/m5stack_8angle_sensor.cpp b/esphome/components/m5stack_8angle/sensor/m5stack_8angle_sensor.cpp index 5e034f1dd3..d22b345141 100644 --- a/esphome/components/m5stack_8angle/sensor/m5stack_8angle_sensor.cpp +++ b/esphome/components/m5stack_8angle/sensor/m5stack_8angle_sensor.cpp @@ -7,7 +7,7 @@ void M5Stack8AngleKnobSensor::update() { if (this->parent_ != nullptr) { int32_t raw_pos = this->parent_->read_knob_pos_raw(this->channel_, this->bits_); if (raw_pos == -1) { - this->status_set_warning("Could not read knob position from M5Stack 8Angle."); + this->status_set_warning(LOG_STR("Could not read knob position from M5Stack 8Angle.")); return; } if (this->raw_) { diff --git a/esphome/components/mapping/__init__.py b/esphome/components/mapping/__init__.py index 79657084fa..94c7c10a82 100644 --- a/esphome/components/mapping/__init__.py +++ b/esphome/components/mapping/__init__.py @@ -10,7 +10,8 @@ from esphome.loader import get_component CODEOWNERS = ["@clydebarrow"] MULTI_CONF = True -map_ = cg.std_ns.class_("map") +mapping_ns = cg.esphome_ns.namespace("mapping") +mapping_class = mapping_ns.class_("Mapping") CONF_ENTRIES = "entries" CONF_CLASS = "class" @@ -29,7 +30,11 @@ class IndexType: INDEX_TYPES = { "int": IndexType(cv.int_, cg.int_, int), - "string": IndexType(cv.string, cg.std_string, str), + "string": IndexType( + cv.string, + cg.std_string, + str, + ), } @@ -47,7 +52,7 @@ def to_schema(value): BASE_SCHEMA = cv.Schema( { - cv.Required(CONF_ID): cv.declare_id(map_), + cv.Required(CONF_ID): cv.declare_id(mapping_class), cv.Required(CONF_FROM): cv.one_of(*INDEX_TYPES, lower=True), cv.Required(CONF_TO): cv.string, }, @@ -123,12 +128,15 @@ async def to_code(config): if list(entries.values())[0].op != ".": value_type = value_type.operator("ptr") varid = config[CONF_ID] - varid.type = map_.template(index_type, value_type) + varid.type = mapping_class.template( + index_type, + value_type, + ) var = MockObj(varid, ".") decl = VariableDeclarationExpression(varid.type, "", varid) add_global(decl) CORE.register_variable(varid, var) for key, value in entries.items(): - cg.add(var.insert((key, value))) + cg.add(var.set(key, value)) return var diff --git a/esphome/components/mapping/mapping.h b/esphome/components/mapping/mapping.h new file mode 100644 index 0000000000..99c1f38829 --- /dev/null +++ b/esphome/components/mapping/mapping.h @@ -0,0 +1,69 @@ +#pragma once + +#include "esphome/core/helpers.h" +#include "esphome/core/log.h" +#include +#include + +namespace esphome::mapping { + +using alloc_string_t = std::basic_string, RAMAllocator>; + +/** + * + * Mapping class with custom allocator. + * Additionally, when std::string is used as key or value, it will be replaced with a custom string type + * that uses RAMAllocator. + * @tparam K The type of the key in the mapping. + * @tparam V The type of the value in the mapping. Should be a basic type or pointer. + */ + +static const char *const TAG = "mapping"; + +template class Mapping { + public: + // Constructor + Mapping() = default; + + using key_t = const std::conditional_t, + alloc_string_t, // if K is std::string, custom string type + K>; + using value_t = std::conditional_t, + alloc_string_t, // if V is std::string, custom string type + V>; + + void set(const K &key, const V &value) { this->map_[key_t{key}] = value; } + + V get(const K &key) const { + auto it = this->map_.find(key_t{key}); + if (it != this->map_.end()) { + return V{it->second}; + } + if constexpr (std::is_pointer_v) { + esph_log_e(TAG, "Key '%p' not found in mapping", key); + } else if constexpr (std::is_same_v) { + esph_log_e(TAG, "Key '%s' not found in mapping", key.c_str()); + } else { + esph_log_e(TAG, "Key '%s' not found in mapping", to_string(key).c_str()); + } + return {}; + } + + // index map overload + V operator[](K key) { return this->get(key); } + + // convenience function for strings to get a C-style string + template, int> = 0> + const char *operator[](K key) const { + auto it = this->map_.find(key_t{key}); + if (it != this->map_.end()) { + return it->second.c_str(); // safe since value remains in map + } + return ""; + } + + protected: + std::map, RAMAllocator>> map_; +}; + +} // namespace esphome::mapping diff --git a/esphome/components/matrix_keypad/__init__.py b/esphome/components/matrix_keypad/__init__.py index b2bcde98ec..2e123323a0 100644 --- a/esphome/components/matrix_keypad/__init__.py +++ b/esphome/components/matrix_keypad/__init__.py @@ -2,7 +2,7 @@ from esphome import automation, pins import esphome.codegen as cg from esphome.components import key_provider import esphome.config_validation as cv -from esphome.const import CONF_ID, CONF_ON_KEY, CONF_PIN, CONF_TRIGGER_ID +from esphome.const import CONF_ID, CONF_ON_KEY, CONF_PIN, CONF_ROWS, CONF_TRIGGER_ID CODEOWNERS = ["@ssieb"] @@ -19,7 +19,6 @@ MatrixKeyTrigger = matrix_keypad_ns.class_( ) CONF_KEYPAD_ID = "keypad_id" -CONF_ROWS = "rows" CONF_COLUMNS = "columns" CONF_KEYS = "keys" CONF_DEBOUNCE_TIME = "debounce_time" @@ -28,9 +27,10 @@ CONF_HAS_PULLDOWNS = "has_pulldowns" def check_keys(obj): - if CONF_KEYS in obj: - if len(obj[CONF_KEYS]) != len(obj[CONF_ROWS]) * len(obj[CONF_COLUMNS]): - raise cv.Invalid("The number of key codes must equal the number of buttons") + if CONF_KEYS in obj and len(obj[CONF_KEYS]) != len(obj[CONF_ROWS]) * len( + obj[CONF_COLUMNS] + ): + raise cv.Invalid("The number of key codes must equal the number of buttons") return obj diff --git a/esphome/components/matrix_keypad/matrix_keypad.h b/esphome/components/matrix_keypad/matrix_keypad.h index 8b309b42c2..258ab4fadc 100644 --- a/esphome/components/matrix_keypad/matrix_keypad.h +++ b/esphome/components/matrix_keypad/matrix_keypad.h @@ -29,9 +29,9 @@ class MatrixKeypad : public key_provider::KeyProvider, public Component { void set_columns(std::vector pins) { columns_ = std::move(pins); }; void set_rows(std::vector pins) { rows_ = std::move(pins); }; void set_keys(std::string keys) { keys_ = std::move(keys); }; - void set_debounce_time(int debounce_time) { debounce_time_ = debounce_time; }; - void set_has_diodes(int has_diodes) { has_diodes_ = has_diodes; }; - void set_has_pulldowns(int has_pulldowns) { has_pulldowns_ = has_pulldowns; }; + void set_debounce_time(uint32_t debounce_time) { debounce_time_ = debounce_time; }; + void set_has_diodes(bool has_diodes) { has_diodes_ = has_diodes; }; + void set_has_pulldowns(bool has_pulldowns) { has_pulldowns_ = has_pulldowns; }; void register_listener(MatrixKeypadListener *listener); void register_key_trigger(MatrixKeyTrigger *trig); @@ -40,7 +40,7 @@ class MatrixKeypad : public key_provider::KeyProvider, public Component { std::vector rows_; std::vector columns_; std::string keys_; - int debounce_time_ = 0; + uint32_t debounce_time_ = 0; bool has_diodes_{false}; bool has_pulldowns_{false}; int pressed_key_ = -1; diff --git a/esphome/components/max17043/max17043.cpp b/esphome/components/max17043/max17043.cpp index dc61babc7e..f605fb1324 100644 --- a/esphome/components/max17043/max17043.cpp +++ b/esphome/components/max17043/max17043.cpp @@ -22,7 +22,7 @@ void MAX17043Component::update() { if (this->voltage_sensor_ != nullptr) { if (!this->read_byte_16(MAX17043_VCELL, &raw_voltage)) { - this->status_set_warning("Unable to read MAX17043_VCELL"); + this->status_set_warning(LOG_STR("Unable to read MAX17043_VCELL")); } else { float voltage = (1.25 * (float) (raw_voltage >> 4)) / 1000.0; this->voltage_sensor_->publish_state(voltage); @@ -31,7 +31,7 @@ void MAX17043Component::update() { } if (this->battery_remaining_sensor_ != nullptr) { if (!this->read_byte_16(MAX17043_SOC, &raw_percent)) { - this->status_set_warning("Unable to read MAX17043_SOC"); + this->status_set_warning(LOG_STR("Unable to read MAX17043_SOC")); } else { float percent = (float) ((raw_percent >> 8) + 0.003906f * (raw_percent & 0x00ff)); this->battery_remaining_sensor_->publish_state(percent); @@ -41,8 +41,6 @@ void MAX17043Component::update() { } void MAX17043Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - uint16_t config_reg; if (this->write(&MAX17043_CONFIG, 1) != i2c::ERROR_OK) { this->status_set_warning(); diff --git a/esphome/components/max31855/max31855.cpp b/esphome/components/max31855/max31855.cpp index 26fba428cc..b5be3106cf 100644 --- a/esphome/components/max31855/max31855.cpp +++ b/esphome/components/max31855/max31855.cpp @@ -19,10 +19,7 @@ void MAX31855Sensor::update() { this->set_timeout("value", 220, f); } -void MAX31855Sensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); - this->spi_setup(); -} +void MAX31855Sensor::setup() { this->spi_setup(); } void MAX31855Sensor::dump_config() { ESP_LOGCONFIG(TAG, "MAX31855:"); LOG_PIN(" CS Pin: ", this->cs_); diff --git a/esphome/components/max31856/max31856.cpp b/esphome/components/max31856/max31856.cpp index c30e2e1a31..cc573cbc53 100644 --- a/esphome/components/max31856/max31856.cpp +++ b/esphome/components/max31856/max31856.cpp @@ -11,14 +11,10 @@ static const char *const TAG = "max31856"; // Based on Adafruit's library: https://github.com/adafruit/Adafruit_MAX31856 void MAX31856Sensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); this->spi_setup(); // assert on any fault - ESP_LOGCONFIG(TAG, "Setting up assertion on all faults"); this->write_register_(MAX31856_MASK_REG, 0x0); - - ESP_LOGCONFIG(TAG, "Setting up open circuit fault detection"); this->write_register_(MAX31856_CR0_REG, MAX31856_CR0_OCFAULT01); this->set_thermocouple_type_(); diff --git a/esphome/components/max31865/max31865.cpp b/esphome/components/max31865/max31865.cpp index 4c9a4ae540..a9c5204cf5 100644 --- a/esphome/components/max31865/max31865.cpp +++ b/esphome/components/max31865/max31865.cpp @@ -65,7 +65,6 @@ void MAX31865Sensor::update() { } void MAX31865Sensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); this->spi_setup(); // Build base configuration diff --git a/esphome/components/max44009/max44009.cpp b/esphome/components/max44009/max44009.cpp index 6d1ce351d4..928fc47696 100644 --- a/esphome/components/max44009/max44009.cpp +++ b/esphome/components/max44009/max44009.cpp @@ -21,7 +21,6 @@ static const uint8_t MAX44009_ERROR_HIGH_BYTE = -30; static const uint8_t MAX44009_ERROR_LOW_BYTE = -31; void MAX44009Sensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); bool state_ok = false; if (this->mode_ == MAX44009Mode::MAX44009_MODE_LOW_POWER) { state_ok = this->set_low_power_mode(); diff --git a/esphome/components/max6675/max6675.cpp b/esphome/components/max6675/max6675.cpp index a2881911f2..54e0330ff7 100644 --- a/esphome/components/max6675/max6675.cpp +++ b/esphome/components/max6675/max6675.cpp @@ -17,10 +17,7 @@ void MAX6675Sensor::update() { this->set_timeout("value", 250, f); } -void MAX6675Sensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); - this->spi_setup(); -} +void MAX6675Sensor::setup() { this->spi_setup(); } void MAX6675Sensor::dump_config() { LOG_SENSOR("", "MAX6675", this); LOG_PIN(" CS Pin: ", this->cs_); diff --git a/esphome/components/max6956/max6956.cpp b/esphome/components/max6956/max6956.cpp index 5a1da9dc6f..a377a1a192 100644 --- a/esphome/components/max6956/max6956.cpp +++ b/esphome/components/max6956/max6956.cpp @@ -20,7 +20,6 @@ const uint8_t MASK_CURRENT_PIN = 0x0F; * MAX6956 * **************************************/ void MAX6956::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t configuration; if (!this->read_reg_(MAX6956_CONFIGURATION, &configuration)) { this->mark_failed(); diff --git a/esphome/components/max7219/max7219.cpp b/esphome/components/max7219/max7219.cpp index 3f78b35bbb..157b317c02 100644 --- a/esphome/components/max7219/max7219.cpp +++ b/esphome/components/max7219/max7219.cpp @@ -116,7 +116,6 @@ const uint8_t MAX7219_ASCII_TO_RAW[95] PROGMEM = { float MAX7219Component::get_setup_priority() const { return setup_priority::PROCESSOR; } void MAX7219Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); this->buffer_ = new uint8_t[this->num_chips_ * 8]; // NOLINT for (uint8_t i = 0; i < this->num_chips_ * 8; i++) diff --git a/esphome/components/max7219digit/display.py b/esphome/components/max7219digit/display.py index f195078c1a..fef121ff10 100644 --- a/esphome/components/max7219digit/display.py +++ b/esphome/components/max7219digit/display.py @@ -3,6 +3,7 @@ import esphome.codegen as cg from esphome.components import display, spi import esphome.config_validation as cv from esphome.const import ( + CONF_FLIP_X, CONF_ID, CONF_INTENSITY, CONF_LAMBDA, @@ -14,7 +15,6 @@ CODEOWNERS = ["@rspaargaren"] DEPENDENCIES = ["spi"] CONF_ROTATE_CHIP = "rotate_chip" -CONF_FLIP_X = "flip_x" CONF_SCROLL_SPEED = "scroll_speed" CONF_SCROLL_DWELL = "scroll_dwell" CONF_SCROLL_DELAY = "scroll_delay" diff --git a/esphome/components/max7219digit/max7219digit.cpp b/esphome/components/max7219digit/max7219digit.cpp index 1721dc80ce..6df3c4d7c8 100644 --- a/esphome/components/max7219digit/max7219digit.cpp +++ b/esphome/components/max7219digit/max7219digit.cpp @@ -26,7 +26,6 @@ constexpr uint8_t MAX7219_DISPLAY_TEST = 0x01; float MAX7219Component::get_setup_priority() const { return setup_priority::PROCESSOR; } void MAX7219Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); this->stepsleft_ = 0; for (int chip_line = 0; chip_line < this->num_chip_lines_; chip_line++) { @@ -91,7 +90,7 @@ void MAX7219Component::loop() { } if (this->scroll_mode_ == ScrollMode::STOP) { - if (this->stepsleft_ + get_width_internal() == first_line_size + 1) { + if (static_cast(this->stepsleft_ + get_width_internal()) == first_line_size + 1) { if (millis_since_last_scroll < this->scroll_dwell_) { ESP_LOGVV(TAG, "Dwell time at end of string in case of stop at end. Step %d, since last scroll %d, dwell %d.", this->stepsleft_, millis_since_last_scroll, this->scroll_dwell_); diff --git a/esphome/components/max9611/max9611.cpp b/esphome/components/max9611/max9611.cpp index e61a30ab99..f00f9d76be 100644 --- a/esphome/components/max9611/max9611.cpp +++ b/esphome/components/max9611/max9611.cpp @@ -31,7 +31,6 @@ static const float TEMP_LSB = 0.48; // 0.48C/LSB static const float MICRO_VOLTS_PER_VOLT = 1000000.0; void MAX9611Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // Perform dummy-read uint8_t value; this->read(&value, 1); diff --git a/esphome/components/mcp23008/__init__.py b/esphome/components/mcp23008/__init__.py index ed48eb06a6..8ff938114a 100644 --- a/esphome/components/mcp23008/__init__.py +++ b/esphome/components/mcp23008/__init__.py @@ -24,5 +24,5 @@ CONFIG_SCHEMA = ( async def to_code(config): - var = await mcp23xxx_base.register_mcp23xxx(config) + var = await mcp23xxx_base.register_mcp23xxx(config, mcp23x08_base.NUM_PINS) await i2c.register_i2c_device(var, config) diff --git a/esphome/components/mcp23008/mcp23008.cpp b/esphome/components/mcp23008/mcp23008.cpp index b93bec9e79..0c34e4971a 100644 --- a/esphome/components/mcp23008/mcp23008.cpp +++ b/esphome/components/mcp23008/mcp23008.cpp @@ -7,7 +7,6 @@ namespace mcp23008 { static const char *const TAG = "mcp23008"; void MCP23008::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t iocon; if (!this->read_reg(mcp23x08_base::MCP23X08_IOCON, &iocon)) { this->mark_failed(); diff --git a/esphome/components/mcp23016/__init__.py b/esphome/components/mcp23016/__init__.py index 3333e46c97..5a1f011617 100644 --- a/esphome/components/mcp23016/__init__.py +++ b/esphome/components/mcp23016/__init__.py @@ -11,6 +11,7 @@ from esphome.const import ( CONF_OUTPUT, ) +AUTO_LOAD = ["gpio_expander"] DEPENDENCIES = ["i2c"] MULTI_CONF = True diff --git a/esphome/components/mcp23016/mcp23016.cpp b/esphome/components/mcp23016/mcp23016.cpp index 17647e9915..be86cb2256 100644 --- a/esphome/components/mcp23016/mcp23016.cpp +++ b/esphome/components/mcp23016/mcp23016.cpp @@ -8,7 +8,6 @@ namespace mcp23016 { static const char *const TAG = "mcp23016"; void MCP23016::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t iocon; if (!this->read_reg_(MCP23016_IOCON0, &iocon)) { this->mark_failed(); @@ -23,14 +22,29 @@ void MCP23016::setup() { this->write_reg_(MCP23016_IODIR0, 0xFF); this->write_reg_(MCP23016_IODIR1, 0xFF); } -bool MCP23016::digital_read(uint8_t pin) { - uint8_t bit = pin % 8; + +void MCP23016::loop() { + // Invalidate cache at the start of each loop + this->reset_pin_cache_(); +} +bool MCP23016::digital_read_hw(uint8_t pin) { uint8_t reg_addr = pin < 8 ? MCP23016_GP0 : MCP23016_GP1; uint8_t value = 0; - this->read_reg_(reg_addr, &value); - return value & (1 << bit); + if (!this->read_reg_(reg_addr, &value)) { + return false; + } + + // Update the appropriate part of input_mask_ + if (pin < 8) { + this->input_mask_ = (this->input_mask_ & 0xFF00) | value; + } else { + this->input_mask_ = (this->input_mask_ & 0x00FF) | (uint16_t(value) << 8); + } + return true; } -void MCP23016::digital_write(uint8_t pin, bool value) { + +bool MCP23016::digital_read_cache(uint8_t pin) { return this->input_mask_ & (1 << pin); } +void MCP23016::digital_write_hw(uint8_t pin, bool value) { uint8_t reg_addr = pin < 8 ? MCP23016_OLAT0 : MCP23016_OLAT1; this->update_reg_(pin, value, reg_addr); } diff --git a/esphome/components/mcp23016/mcp23016.h b/esphome/components/mcp23016/mcp23016.h index e4ed47a3b2..781c207de0 100644 --- a/esphome/components/mcp23016/mcp23016.h +++ b/esphome/components/mcp23016/mcp23016.h @@ -3,6 +3,7 @@ #include "esphome/core/component.h" #include "esphome/core/hal.h" #include "esphome/components/i2c/i2c.h" +#include "esphome/components/gpio_expander/cached_gpio.h" namespace esphome { namespace mcp23016 { @@ -24,19 +25,22 @@ enum MCP23016GPIORegisters { MCP23016_IOCON1 = 0x0B, }; -class MCP23016 : public Component, public i2c::I2CDevice { +class MCP23016 : public Component, public i2c::I2CDevice, public gpio_expander::CachedGpioExpander { public: MCP23016() = default; void setup() override; - - bool digital_read(uint8_t pin); - void digital_write(uint8_t pin, bool value); + void loop() override; void pin_mode(uint8_t pin, gpio::Flags flags); float get_setup_priority() const override; protected: + // Virtual methods from CachedGpioExpander + bool digital_read_hw(uint8_t pin) override; + bool digital_read_cache(uint8_t pin) override; + void digital_write_hw(uint8_t pin, bool value) override; + // read a given register bool read_reg_(uint8_t reg, uint8_t *value); // write a value to a given register @@ -46,6 +50,8 @@ class MCP23016 : public Component, public i2c::I2CDevice { uint8_t olat_0_{0x00}; uint8_t olat_1_{0x00}; + // Cache for input values (16-bit combined for both banks) + uint16_t input_mask_{0x00}; }; class MCP23016GPIOPin : public GPIOPin { diff --git a/esphome/components/mcp23017/__init__.py b/esphome/components/mcp23017/__init__.py index 33b8a680cf..e5cc1856eb 100644 --- a/esphome/components/mcp23017/__init__.py +++ b/esphome/components/mcp23017/__init__.py @@ -24,5 +24,5 @@ CONFIG_SCHEMA = ( async def to_code(config): - var = await mcp23xxx_base.register_mcp23xxx(config) + var = await mcp23xxx_base.register_mcp23xxx(config, mcp23x17_base.NUM_PINS) await i2c.register_i2c_device(var, config) diff --git a/esphome/components/mcp23017/mcp23017.cpp b/esphome/components/mcp23017/mcp23017.cpp index 5c0c2c4703..1ad2036939 100644 --- a/esphome/components/mcp23017/mcp23017.cpp +++ b/esphome/components/mcp23017/mcp23017.cpp @@ -7,7 +7,6 @@ namespace mcp23017 { static const char *const TAG = "mcp23017"; void MCP23017::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t iocon; if (!this->read_reg(mcp23x17_base::MCP23X17_IOCONA, &iocon)) { this->mark_failed(); diff --git a/esphome/components/mcp23s08/__init__.py b/esphome/components/mcp23s08/__init__.py index c6152d58c0..3d4e304f9b 100644 --- a/esphome/components/mcp23s08/__init__.py +++ b/esphome/components/mcp23s08/__init__.py @@ -27,6 +27,6 @@ CONFIG_SCHEMA = ( async def to_code(config): - var = await mcp23xxx_base.register_mcp23xxx(config) + var = await mcp23xxx_base.register_mcp23xxx(config, mcp23x08_base.NUM_PINS) cg.add(var.set_device_address(config[CONF_DEVICEADDRESS])) await spi.register_spi_device(var, config) diff --git a/esphome/components/mcp23s08/mcp23s08.cpp b/esphome/components/mcp23s08/mcp23s08.cpp index 671506c79d..3d944b45d5 100644 --- a/esphome/components/mcp23s08/mcp23s08.cpp +++ b/esphome/components/mcp23s08/mcp23s08.cpp @@ -13,7 +13,6 @@ void MCP23S08::set_device_address(uint8_t device_addr) { } void MCP23S08::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); this->enable(); diff --git a/esphome/components/mcp23s17/__init__.py b/esphome/components/mcp23s17/__init__.py index 9a763d09b0..ea8433af2e 100644 --- a/esphome/components/mcp23s17/__init__.py +++ b/esphome/components/mcp23s17/__init__.py @@ -27,6 +27,6 @@ CONFIG_SCHEMA = ( async def to_code(config): - var = await mcp23xxx_base.register_mcp23xxx(config) + var = await mcp23xxx_base.register_mcp23xxx(config, mcp23x17_base.NUM_PINS) cg.add(var.set_device_address(config[CONF_DEVICEADDRESS])) await spi.register_spi_device(var, config) diff --git a/esphome/components/mcp23s17/mcp23s17.cpp b/esphome/components/mcp23s17/mcp23s17.cpp index 1b922a8130..1624eda9e4 100644 --- a/esphome/components/mcp23s17/mcp23s17.cpp +++ b/esphome/components/mcp23s17/mcp23s17.cpp @@ -13,7 +13,6 @@ void MCP23S17::set_device_address(uint8_t device_addr) { } void MCP23S17::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); this->enable(); diff --git a/esphome/components/mcp23x08_base/__init__.py b/esphome/components/mcp23x08_base/__init__.py index ba44917202..a3c12165f0 100644 --- a/esphome/components/mcp23x08_base/__init__.py +++ b/esphome/components/mcp23x08_base/__init__.py @@ -4,5 +4,7 @@ from esphome.components import mcp23xxx_base AUTO_LOAD = ["mcp23xxx_base"] CODEOWNERS = ["@jesserockz"] +NUM_PINS = 8 + mcp23x08_base_ns = cg.esphome_ns.namespace("mcp23x08_base") MCP23X08Base = mcp23x08_base_ns.class_("MCP23X08Base", mcp23xxx_base.MCP23XXXBase) diff --git a/esphome/components/mcp23x08_base/mcp23x08_base.cpp b/esphome/components/mcp23x08_base/mcp23x08_base.cpp index 0c20e902c4..1593c376cd 100644 --- a/esphome/components/mcp23x08_base/mcp23x08_base.cpp +++ b/esphome/components/mcp23x08_base/mcp23x08_base.cpp @@ -6,19 +6,21 @@ namespace mcp23x08_base { static const char *const TAG = "mcp23x08_base"; -bool MCP23X08Base::digital_read(uint8_t pin) { - uint8_t bit = pin % 8; - uint8_t reg_addr = mcp23x08_base::MCP23X08_GPIO; - uint8_t value = 0; - this->read_reg(reg_addr, &value); - return value & (1 << bit); +bool MCP23X08Base::digital_read_hw(uint8_t pin) { + if (!this->read_reg(mcp23x08_base::MCP23X08_GPIO, &this->input_mask_)) { + this->status_set_warning(LOG_STR(ESP_LOG_MSG_COMM_FAIL)); + return false; + } + return true; } -void MCP23X08Base::digital_write(uint8_t pin, bool value) { +void MCP23X08Base::digital_write_hw(uint8_t pin, bool value) { uint8_t reg_addr = mcp23x08_base::MCP23X08_OLAT; this->update_reg(pin, value, reg_addr); } +bool MCP23X08Base::digital_read_cache(uint8_t pin) { return this->input_mask_ & (1 << pin); } + void MCP23X08Base::pin_mode(uint8_t pin, gpio::Flags flags) { uint8_t iodir = mcp23x08_base::MCP23X08_IODIR; uint8_t gppu = mcp23x08_base::MCP23X08_GPPU; diff --git a/esphome/components/mcp23x08_base/mcp23x08_base.h b/esphome/components/mcp23x08_base/mcp23x08_base.h index 910519119b..6eee8274b1 100644 --- a/esphome/components/mcp23x08_base/mcp23x08_base.h +++ b/esphome/components/mcp23x08_base/mcp23x08_base.h @@ -1,7 +1,7 @@ #pragma once -#include "esphome/core/component.h" #include "esphome/components/mcp23xxx_base/mcp23xxx_base.h" +#include "esphome/core/component.h" #include "esphome/core/hal.h" namespace esphome { @@ -22,10 +22,12 @@ enum MCP23S08GPIORegisters { MCP23X08_OLAT = 0x0A, }; -class MCP23X08Base : public mcp23xxx_base::MCP23XXXBase { +class MCP23X08Base : public mcp23xxx_base::MCP23XXXBase<8> { public: - bool digital_read(uint8_t pin) override; - void digital_write(uint8_t pin, bool value) override; + bool digital_read_hw(uint8_t pin) override; + void digital_write_hw(uint8_t pin, bool value) override; + bool digital_read_cache(uint8_t pin) override; + void pin_mode(uint8_t pin, gpio::Flags flags) override; void pin_interrupt_mode(uint8_t pin, mcp23xxx_base::MCP23XXXInterruptMode interrupt_mode) override; @@ -33,6 +35,9 @@ class MCP23X08Base : public mcp23xxx_base::MCP23XXXBase { void update_reg(uint8_t pin, bool pin_value, uint8_t reg_a) override; uint8_t olat_{0x00}; + + /// State read in digital_read_hw + uint8_t input_mask_{0x00}; }; } // namespace mcp23x08_base diff --git a/esphome/components/mcp23x17_base/__init__.py b/esphome/components/mcp23x17_base/__init__.py index 97e0b3823d..1b93d16ff3 100644 --- a/esphome/components/mcp23x17_base/__init__.py +++ b/esphome/components/mcp23x17_base/__init__.py @@ -4,5 +4,7 @@ from esphome.components import mcp23xxx_base AUTO_LOAD = ["mcp23xxx_base"] CODEOWNERS = ["@jesserockz"] +NUM_PINS = 16 + mcp23x17_base_ns = cg.esphome_ns.namespace("mcp23x17_base") MCP23X17Base = mcp23x17_base_ns.class_("MCP23X17Base", mcp23xxx_base.MCP23XXXBase) diff --git a/esphome/components/mcp23x17_base/mcp23x17_base.cpp b/esphome/components/mcp23x17_base/mcp23x17_base.cpp index 99064f8880..b1f1f260b4 100644 --- a/esphome/components/mcp23x17_base/mcp23x17_base.cpp +++ b/esphome/components/mcp23x17_base/mcp23x17_base.cpp @@ -1,4 +1,5 @@ #include "mcp23x17_base.h" +#include "esphome/core/helpers.h" #include "esphome/core/log.h" namespace esphome { @@ -6,19 +7,31 @@ namespace mcp23x17_base { static const char *const TAG = "mcp23x17_base"; -bool MCP23X17Base::digital_read(uint8_t pin) { - uint8_t bit = pin % 8; - uint8_t reg_addr = pin < 8 ? mcp23x17_base::MCP23X17_GPIOA : mcp23x17_base::MCP23X17_GPIOB; - uint8_t value = 0; - this->read_reg(reg_addr, &value); - return value & (1 << bit); +bool MCP23X17Base::digital_read_hw(uint8_t pin) { + uint8_t data; + if (pin < 8) { + if (!this->read_reg(mcp23x17_base::MCP23X17_GPIOA, &data)) { + this->status_set_warning(LOG_STR(ESP_LOG_MSG_COMM_FAIL)); + return false; + } + this->input_mask_ = encode_uint16(this->input_mask_ >> 8, data); + } else { + if (!this->read_reg(mcp23x17_base::MCP23X17_GPIOB, &data)) { + this->status_set_warning(LOG_STR(ESP_LOG_MSG_COMM_FAIL)); + return false; + } + this->input_mask_ = encode_uint16(data, this->input_mask_ & 0xFF); + } + return true; } -void MCP23X17Base::digital_write(uint8_t pin, bool value) { +void MCP23X17Base::digital_write_hw(uint8_t pin, bool value) { uint8_t reg_addr = pin < 8 ? mcp23x17_base::MCP23X17_OLATA : mcp23x17_base::MCP23X17_OLATB; this->update_reg(pin, value, reg_addr); } +bool MCP23X17Base::digital_read_cache(uint8_t pin) { return this->input_mask_ & (1 << pin); } + void MCP23X17Base::pin_mode(uint8_t pin, gpio::Flags flags) { uint8_t iodir = pin < 8 ? mcp23x17_base::MCP23X17_IODIRA : mcp23x17_base::MCP23X17_IODIRB; uint8_t gppu = pin < 8 ? mcp23x17_base::MCP23X17_GPPUA : mcp23x17_base::MCP23X17_GPPUB; diff --git a/esphome/components/mcp23x17_base/mcp23x17_base.h b/esphome/components/mcp23x17_base/mcp23x17_base.h index 3d50ee8c03..bdd66503e2 100644 --- a/esphome/components/mcp23x17_base/mcp23x17_base.h +++ b/esphome/components/mcp23x17_base/mcp23x17_base.h @@ -1,7 +1,7 @@ #pragma once -#include "esphome/core/component.h" #include "esphome/components/mcp23xxx_base/mcp23xxx_base.h" +#include "esphome/core/component.h" #include "esphome/core/hal.h" namespace esphome { @@ -34,10 +34,12 @@ enum MCP23X17GPIORegisters { MCP23X17_OLATB = 0x15, }; -class MCP23X17Base : public mcp23xxx_base::MCP23XXXBase { +class MCP23X17Base : public mcp23xxx_base::MCP23XXXBase<16> { public: - bool digital_read(uint8_t pin) override; - void digital_write(uint8_t pin, bool value) override; + bool digital_read_hw(uint8_t pin) override; + void digital_write_hw(uint8_t pin, bool value) override; + bool digital_read_cache(uint8_t pin) override; + void pin_mode(uint8_t pin, gpio::Flags flags) override; void pin_interrupt_mode(uint8_t pin, mcp23xxx_base::MCP23XXXInterruptMode interrupt_mode) override; @@ -46,6 +48,9 @@ class MCP23X17Base : public mcp23xxx_base::MCP23XXXBase { uint8_t olat_a_{0x00}; uint8_t olat_b_{0x00}; + + /// State read in digital_read_hw + uint16_t input_mask_{0x00}; }; } // namespace mcp23x17_base diff --git a/esphome/components/mcp23xxx_base/__init__.py b/esphome/components/mcp23xxx_base/__init__.py index 8cf0ebcd44..d6e82101ad 100644 --- a/esphome/components/mcp23xxx_base/__init__.py +++ b/esphome/components/mcp23xxx_base/__init__.py @@ -12,8 +12,9 @@ from esphome.const import ( CONF_OUTPUT, CONF_PULLUP, ) -from esphome.core import coroutine +from esphome.core import CORE, ID, coroutine +AUTO_LOAD = ["gpio_expander"] CODEOWNERS = ["@jesserockz"] mcp23xxx_base_ns = cg.esphome_ns.namespace("mcp23xxx_base") @@ -36,9 +37,11 @@ MCP23XXX_CONFIG_SCHEMA = cv.Schema( @coroutine -async def register_mcp23xxx(config): - var = cg.new_Pvariable(config[CONF_ID]) +async def register_mcp23xxx(config, num_pins): + id: ID = config[CONF_ID] + var = cg.new_Pvariable(id) await cg.register_component(var, config) + CORE.data.setdefault(CONF_MCP23XXX, {})[id.id] = num_pins cg.add(var.set_open_drain_ints(config[CONF_OPEN_DRAIN_INTERRUPT])) return var @@ -73,9 +76,12 @@ MCP23XXX_PIN_SCHEMA = pins.gpio_base_schema( @pins.PIN_SCHEMA_REGISTRY.register(CONF_MCP23XXX, MCP23XXX_PIN_SCHEMA) async def mcp23xxx_pin_to_code(config): - var = cg.new_Pvariable(config[CONF_ID]) - parent = await cg.get_variable(config[CONF_MCP23XXX]) + parent_id: ID = config[CONF_MCP23XXX] + parent = await cg.get_variable(parent_id) + num_pins = cg.TemplateArguments(CORE.data[CONF_MCP23XXX][parent_id.id]) + + var = cg.new_Pvariable(config[CONF_ID], num_pins) cg.add(var.set_parent(parent)) num = config[CONF_NUMBER] diff --git a/esphome/components/mcp23xxx_base/mcp23xxx_base.cpp b/esphome/components/mcp23xxx_base/mcp23xxx_base.cpp index fc49f216ee..81324e794f 100644 --- a/esphome/components/mcp23xxx_base/mcp23xxx_base.cpp +++ b/esphome/components/mcp23xxx_base/mcp23xxx_base.cpp @@ -1,24 +1,27 @@ #include "mcp23xxx_base.h" +#include "esphome/core/helpers.h" #include "esphome/core/log.h" namespace esphome { namespace mcp23xxx_base { -float MCP23XXXBase::get_setup_priority() const { return setup_priority::IO; } - -void MCP23XXXGPIOPin::setup() { - pin_mode(flags_); +template void MCP23XXXGPIOPin::setup() { + this->pin_mode(flags_); this->parent_->pin_interrupt_mode(this->pin_, this->interrupt_mode_); } - -void MCP23XXXGPIOPin::pin_mode(gpio::Flags flags) { this->parent_->pin_mode(this->pin_, flags); } -bool MCP23XXXGPIOPin::digital_read() { return this->parent_->digital_read(this->pin_) != this->inverted_; } -void MCP23XXXGPIOPin::digital_write(bool value) { this->parent_->digital_write(this->pin_, value != this->inverted_); } -std::string MCP23XXXGPIOPin::dump_summary() const { - char buffer[32]; - snprintf(buffer, sizeof(buffer), "%u via MCP23XXX", pin_); - return buffer; +template void MCP23XXXGPIOPin::pin_mode(gpio::Flags flags) { this->parent_->pin_mode(this->pin_, flags); } +template bool MCP23XXXGPIOPin::digital_read() { + return this->parent_->digital_read(this->pin_) != this->inverted_; } +template void MCP23XXXGPIOPin::digital_write(bool value) { + this->parent_->digital_write(this->pin_, value != this->inverted_); +} +template std::string MCP23XXXGPIOPin::dump_summary() const { + return str_snprintf("%u via MCP23XXX", 15, pin_); +} + +template class MCP23XXXGPIOPin<8>; +template class MCP23XXXGPIOPin<16>; } // namespace mcp23xxx_base } // namespace esphome diff --git a/esphome/components/mcp23xxx_base/mcp23xxx_base.h b/esphome/components/mcp23xxx_base/mcp23xxx_base.h index 9686c9fd33..cf0ef5d41c 100644 --- a/esphome/components/mcp23xxx_base/mcp23xxx_base.h +++ b/esphome/components/mcp23xxx_base/mcp23xxx_base.h @@ -1,5 +1,6 @@ #pragma once +#include "esphome/components/gpio_expander/cached_gpio.h" #include "esphome/core/component.h" #include "esphome/core/hal.h" @@ -8,28 +9,28 @@ namespace mcp23xxx_base { enum MCP23XXXInterruptMode : uint8_t { MCP23XXX_NO_INTERRUPT = 0, MCP23XXX_CHANGE, MCP23XXX_RISING, MCP23XXX_FALLING }; -class MCP23XXXBase : public Component { +template class MCP23XXXBase : public Component, public gpio_expander::CachedGpioExpander { public: - virtual bool digital_read(uint8_t pin); - virtual void digital_write(uint8_t pin, bool value); virtual void pin_mode(uint8_t pin, gpio::Flags flags); virtual void pin_interrupt_mode(uint8_t pin, MCP23XXXInterruptMode interrupt_mode); void set_open_drain_ints(const bool value) { this->open_drain_ints_ = value; } - float get_setup_priority() const override; + float get_setup_priority() const override { return setup_priority::IO; } + + void loop() override { this->reset_pin_cache_(); } protected: // read a given register - virtual bool read_reg(uint8_t reg, uint8_t *value); + virtual bool read_reg(uint8_t reg, uint8_t *value) = 0; // write a value to a given register - virtual bool write_reg(uint8_t reg, uint8_t value); + virtual bool write_reg(uint8_t reg, uint8_t value) = 0; // update registers with given pin value. - virtual void update_reg(uint8_t pin, bool pin_value, uint8_t reg_a); + virtual void update_reg(uint8_t pin, bool pin_value, uint8_t reg_a) = 0; bool open_drain_ints_; }; -class MCP23XXXGPIOPin : public GPIOPin { +template class MCP23XXXGPIOPin : public GPIOPin { public: void setup() override; void pin_mode(gpio::Flags flags) override; @@ -37,7 +38,7 @@ class MCP23XXXGPIOPin : public GPIOPin { void digital_write(bool value) override; std::string dump_summary() const override; - void set_parent(MCP23XXXBase *parent) { parent_ = parent; } + void set_parent(MCP23XXXBase *parent) { parent_ = parent; } void set_pin(uint8_t pin) { pin_ = pin; } void set_inverted(bool inverted) { inverted_ = inverted; } void set_flags(gpio::Flags flags) { flags_ = flags; } @@ -46,7 +47,7 @@ class MCP23XXXGPIOPin : public GPIOPin { gpio::Flags get_flags() const override { return this->flags_; } protected: - MCP23XXXBase *parent_; + MCP23XXXBase *parent_; uint8_t pin_; bool inverted_; gpio::Flags flags_; diff --git a/esphome/components/mcp2515/mcp2515.cpp b/esphome/components/mcp2515/mcp2515.cpp index 23104f5aeb..1a17715315 100644 --- a/esphome/components/mcp2515/mcp2515.cpp +++ b/esphome/components/mcp2515/mcp2515.cpp @@ -20,6 +20,23 @@ bool MCP2515::setup_internal() { return false; if (this->set_bitrate_(this->bit_rate_, this->mcp_clock_) != canbus::ERROR_OK) return false; + + // setup hardware filter RXF0 accepting all standard CAN IDs + if (this->set_filter_(RXF::RXF0, false, 0) != canbus::ERROR_OK) { + return false; + } + if (this->set_filter_mask_(MASK::MASK0, false, 0) != canbus::ERROR_OK) { + return false; + } + + // setup hardware filter RXF1 accepting all extended CAN IDs + if (this->set_filter_(RXF::RXF1, true, 0) != canbus::ERROR_OK) { + return false; + } + if (this->set_filter_mask_(MASK::MASK1, true, 0) != canbus::ERROR_OK) { + return false; + } + if (this->set_mode_(this->mcp_mode_) != canbus::ERROR_OK) return false; uint8_t err_flags = this->get_error_flags_(); @@ -155,7 +172,7 @@ void MCP2515::prepare_id_(uint8_t *buffer, const bool extended, const uint32_t i canid = (uint16_t) (id >> 16); buffer[MCP_SIDL] = (uint8_t) (canid & 0x03); buffer[MCP_SIDL] += (uint8_t) ((canid & 0x1C) << 3); - buffer[MCP_SIDL] |= TXB_EXIDE_MASK; + buffer[MCP_SIDL] |= SIDL_EXIDE_MASK; buffer[MCP_SIDH] = (uint8_t) (canid >> 5); } else { buffer[MCP_SIDH] = (uint8_t) (canid >> 3); @@ -258,7 +275,7 @@ canbus::Error MCP2515::send_message(struct canbus::CanFrame *frame) { } } - return canbus::ERROR_FAILTX; + return canbus::ERROR_ALLTXBUSY; } canbus::Error MCP2515::read_message_(RXBn rxbn, struct canbus::CanFrame *frame) { @@ -272,7 +289,7 @@ canbus::Error MCP2515::read_message_(RXBn rxbn, struct canbus::CanFrame *frame) bool use_extended_id = false; bool remote_transmission_request = false; - if ((tbufdata[MCP_SIDL] & TXB_EXIDE_MASK) == TXB_EXIDE_MASK) { + if ((tbufdata[MCP_SIDL] & SIDL_EXIDE_MASK) == SIDL_EXIDE_MASK) { id = (id << 2) + (tbufdata[MCP_SIDL] & 0x03); id = (id << 8) + tbufdata[MCP_EID8]; id = (id << 8) + tbufdata[MCP_EID0]; @@ -315,6 +332,17 @@ canbus::Error MCP2515::read_message(struct canbus::CanFrame *frame) { rc = canbus::ERROR_NOMSG; } +#ifdef ESPHOME_LOG_HAS_DEBUG + uint8_t err = get_error_flags_(); + // The receive flowchart in the datasheet says that if rollover is set (BUKT), RX1OVR flag will be set + // once both buffers are full. However, the RX0OVR flag is actually set instead. + // We can just check for both though because it doesn't break anything. + if (err & (EFLG_RX0OVR | EFLG_RX1OVR)) { + ESP_LOGD(TAG, "receive buffer overrun"); + clear_rx_n_ovr_flags_(); + } +#endif + return rc; } diff --git a/esphome/components/mcp2515/mcp2515_defs.h b/esphome/components/mcp2515/mcp2515_defs.h index 2f5cf2a238..b33adcbba6 100644 --- a/esphome/components/mcp2515/mcp2515_defs.h +++ b/esphome/components/mcp2515/mcp2515_defs.h @@ -130,7 +130,9 @@ static const uint8_t CANSTAT_ICOD = 0x0E; static const uint8_t CNF3_SOF = 0x80; -static const uint8_t TXB_EXIDE_MASK = 0x08; +// applies to RXBn_SIDL, TXBn_SIDL and RXFn_SIDL +static const uint8_t SIDL_EXIDE_MASK = 0x08; + static const uint8_t DLC_MASK = 0x0F; static const uint8_t RTR_MASK = 0x40; diff --git a/esphome/components/mcp3008/mcp3008.cpp b/esphome/components/mcp3008/mcp3008.cpp index fb9bda35d0..812a3b0c83 100644 --- a/esphome/components/mcp3008/mcp3008.cpp +++ b/esphome/components/mcp3008/mcp3008.cpp @@ -10,10 +10,7 @@ static const char *const TAG = "mcp3008"; float MCP3008::get_setup_priority() const { return setup_priority::HARDWARE; } -void MCP3008::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->spi_setup(); -} +void MCP3008::setup() { this->spi_setup(); } void MCP3008::dump_config() { ESP_LOGCONFIG(TAG, "MCP3008:"); diff --git a/esphome/components/mcp3204/mcp3204.cpp b/esphome/components/mcp3204/mcp3204.cpp index 1f956612d7..4bb0cbed76 100644 --- a/esphome/components/mcp3204/mcp3204.cpp +++ b/esphome/components/mcp3204/mcp3204.cpp @@ -8,10 +8,7 @@ static const char *const TAG = "mcp3204"; float MCP3204::get_setup_priority() const { return setup_priority::HARDWARE; } -void MCP3204::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->spi_setup(); -} +void MCP3204::setup() { this->spi_setup(); } void MCP3204::dump_config() { ESP_LOGCONFIG(TAG, "MCP3204:"); diff --git a/esphome/components/mcp4461/mcp4461.cpp b/esphome/components/mcp4461/mcp4461.cpp index 39127a6c04..2f2c75e05a 100644 --- a/esphome/components/mcp4461/mcp4461.cpp +++ b/esphome/components/mcp4461/mcp4461.cpp @@ -10,7 +10,6 @@ static const char *const TAG = "mcp4461"; constexpr uint8_t EEPROM_WRITE_TIMEOUT_MS = 10; void Mcp4461Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup for address 0x%02X", this->address_); auto err = this->write(nullptr, 0); if (err != i2c::ERROR_OK) { this->error_code_ = MCP4461_STATUS_I2C_ERROR; @@ -123,7 +122,7 @@ uint8_t Mcp4461Component::get_status_register_() { uint8_t addr = static_cast(Mcp4461Addresses::MCP4461_STATUS); uint8_t reg = addr | static_cast(Mcp4461Commands::READ); uint16_t buf; - if (!this->read_byte_16(reg, &buf)) { + if (!this->read_16_(reg, &buf)) { this->error_code_ = MCP4461_STATUS_REGISTER_ERROR; this->mark_failed(); return 0; @@ -149,6 +148,20 @@ void Mcp4461Component::read_status_register_to_log() { ((status_register_value >> 3) & 0x01), ((status_register_value >> 2) & 0x01), ((status_register_value >> 1) & 0x01), ((status_register_value >> 0) & 0x01)); } +bool Mcp4461Component::read_16_(uint8_t address, uint16_t *buf) { + // read 16 bits and convert from big endian to host, + // Do this as two separate operations to ensure a stop condition between the write and read + i2c::ErrorCode err = this->write(&address, 1); + if (err != i2c::ERROR_OK) { + return false; + } + err = this->read(reinterpret_cast(buf), 2); + if (err != i2c::ERROR_OK) { + return false; + } + *buf = convert_big_endian(*buf); + return true; +} uint8_t Mcp4461Component::get_wiper_address_(uint8_t wiper) { uint8_t addr; @@ -199,14 +212,14 @@ uint16_t Mcp4461Component::get_wiper_level_(Mcp4461WiperIdx wiper) { uint16_t Mcp4461Component::read_wiper_level_(uint8_t wiper_idx) { uint8_t addr = this->get_wiper_address_(wiper_idx); - uint8_t reg = addr | static_cast(Mcp4461Commands::INCREMENT); + uint8_t reg = addr | static_cast(Mcp4461Commands::READ); if (wiper_idx > 3) { if (!this->is_eeprom_ready_for_writing_(true)) { return 0; } } uint16_t buf = 0; - if (!(this->read_byte_16(reg, &buf))) { + if (!(this->read_16_(reg, &buf))) { this->error_code_ = MCP4461_STATUS_I2C_ERROR; this->status_set_warning(); ESP_LOGW(TAG, "Error fetching %swiper %u value", (wiper_idx > 3) ? "nonvolatile " : "", wiper_idx); @@ -329,7 +342,7 @@ bool Mcp4461Component::increase_wiper_(Mcp4461WiperIdx wiper) { ESP_LOGV(TAG, "Increasing wiper %u", wiper_idx); uint8_t addr = this->get_wiper_address_(wiper_idx); uint8_t reg = addr | static_cast(Mcp4461Commands::INCREMENT); - auto err = this->write(&this->address_, reg, sizeof(reg)); + auto err = this->write(&this->address_, reg); if (err != i2c::ERROR_OK) { this->error_code_ = MCP4461_STATUS_I2C_ERROR; this->status_set_warning(); @@ -360,7 +373,7 @@ bool Mcp4461Component::decrease_wiper_(Mcp4461WiperIdx wiper) { ESP_LOGV(TAG, "Decreasing wiper %u", wiper_idx); uint8_t addr = this->get_wiper_address_(wiper_idx); uint8_t reg = addr | static_cast(Mcp4461Commands::DECREMENT); - auto err = this->write(&this->address_, reg, sizeof(reg)); + auto err = this->write(&this->address_, reg); if (err != i2c::ERROR_OK) { this->error_code_ = MCP4461_STATUS_I2C_ERROR; this->status_set_warning(); @@ -393,7 +406,7 @@ uint8_t Mcp4461Component::get_terminal_register_(Mcp4461TerminalIdx terminal_con : static_cast(Mcp4461Addresses::MCP4461_TCON1); reg |= static_cast(Mcp4461Commands::READ); uint16_t buf; - if (this->read_byte_16(reg, &buf)) { + if (this->read_16_(reg, &buf)) { return static_cast(buf & 0x00ff); } else { this->error_code_ = MCP4461_STATUS_I2C_ERROR; @@ -518,7 +531,7 @@ uint16_t Mcp4461Component::get_eeprom_value(Mcp4461EepromLocation location) { if (!this->is_eeprom_ready_for_writing_(true)) { return 0; } - if (!this->read_byte_16(reg, &buf)) { + if (!this->read_16_(reg, &buf)) { this->error_code_ = MCP4461_STATUS_I2C_ERROR; this->status_set_warning(); ESP_LOGW(TAG, "Error fetching EEPROM location value"); diff --git a/esphome/components/mcp4461/mcp4461.h b/esphome/components/mcp4461/mcp4461.h index 9b7f60f201..59f6358a56 100644 --- a/esphome/components/mcp4461/mcp4461.h +++ b/esphome/components/mcp4461/mcp4461.h @@ -96,6 +96,7 @@ class Mcp4461Component : public Component, public i2c::I2CDevice { protected: friend class Mcp4461Wiper; + bool read_16_(uint8_t address, uint16_t *buf); void update_write_protection_status_(); uint8_t get_wiper_address_(uint8_t wiper); uint16_t read_wiper_level_(uint8_t wiper); diff --git a/esphome/components/mcp4725/mcp4725.cpp b/esphome/components/mcp4725/mcp4725.cpp index 8b2f8524d8..137ac9cb61 100644 --- a/esphome/components/mcp4725/mcp4725.cpp +++ b/esphome/components/mcp4725/mcp4725.cpp @@ -7,7 +7,6 @@ namespace mcp4725 { static const char *const TAG = "mcp4725"; void MCP4725::setup() { - ESP_LOGCONFIG(TAG, "Running setup for address 0x%02X", this->address_); auto err = this->write(nullptr, 0); if (err != i2c::ERROR_OK) { this->error_code_ = COMMUNICATION_FAILED; diff --git a/esphome/components/mcp4728/mcp4728.cpp b/esphome/components/mcp4728/mcp4728.cpp index 7b2b43d4d8..bab94cb233 100644 --- a/esphome/components/mcp4728/mcp4728.cpp +++ b/esphome/components/mcp4728/mcp4728.cpp @@ -9,7 +9,6 @@ namespace mcp4728 { static const char *const TAG = "mcp4728"; void MCP4728Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup for address 0x%02X", this->address_); auto err = this->write(nullptr, 0); if (err != i2c::ERROR_OK) { this->mark_failed(); diff --git a/esphome/components/mcp9600/mcp9600.cpp b/esphome/components/mcp9600/mcp9600.cpp index 16c19326f2..e1a88988c4 100644 --- a/esphome/components/mcp9600/mcp9600.cpp +++ b/esphome/components/mcp9600/mcp9600.cpp @@ -28,8 +28,6 @@ static const uint8_t MCP9600_REGISTER_ALERT4_LIMIT = 0x13; static const uint8_t MCP9600_REGISTER_DEVICE_ID = 0x20; void MCP9600Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - uint16_t dev_id = 0; this->read_byte_16(MCP9600_REGISTER_DEVICE_ID, &dev_id); this->device_id_ = (uint8_t) (dev_id >> 8); diff --git a/esphome/components/mcp9808/mcp9808.cpp b/esphome/components/mcp9808/mcp9808.cpp index 02ddc1aceb..088d33887f 100644 --- a/esphome/components/mcp9808/mcp9808.cpp +++ b/esphome/components/mcp9808/mcp9808.cpp @@ -18,8 +18,6 @@ static const uint8_t MCP9808_AMBIENT_TEMP_NEGATIVE = 0x10; static const char *const TAG = "mcp9808"; void MCP9808Sensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); - uint16_t manu = 0; if (!this->read_byte_16(MCP9808_REG_MANUF_ID, &manu) || manu != MCP9808_MANUF_ID) { this->mark_failed(); diff --git a/esphome/components/md5/md5.cpp b/esphome/components/md5/md5.cpp index 980cb98699..866f00eda4 100644 --- a/esphome/components/md5/md5.cpp +++ b/esphome/components/md5/md5.cpp @@ -1,4 +1,3 @@ -#include #include #include "md5.h" #ifdef USE_MD5 @@ -40,30 +39,6 @@ void MD5Digest::add(const uint8_t *data, size_t len) { br_md5_update(&this->ctx_ void MD5Digest::calculate() { br_md5_out(&this->ctx_, this->digest_); } #endif // USE_RP2040 -void MD5Digest::get_bytes(uint8_t *output) { memcpy(output, this->digest_, 16); } - -void MD5Digest::get_hex(char *output) { - for (size_t i = 0; i < 16; i++) { - sprintf(output + i * 2, "%02x", this->digest_[i]); - } -} - -bool MD5Digest::equals_bytes(const uint8_t *expected) { - for (size_t i = 0; i < 16; i++) { - if (expected[i] != this->digest_[i]) { - return false; - } - } - return true; -} - -bool MD5Digest::equals_hex(const char *expected) { - uint8_t parsed[16]; - if (!parse_hex(expected, parsed, 16)) - return false; - return equals_bytes(parsed); -} - } // namespace md5 } // namespace esphome #endif diff --git a/esphome/components/md5/md5.h b/esphome/components/md5/md5.h index be1df40423..b0da2c0a3b 100644 --- a/esphome/components/md5/md5.h +++ b/esphome/components/md5/md5.h @@ -3,6 +3,8 @@ #include "esphome/core/defines.h" #ifdef USE_MD5 +#include "esphome/core/hash_base.h" + #ifdef USE_ESP32 #include "esp_rom_md5.h" #define MD5_CTX_TYPE md5_context_t @@ -26,38 +28,26 @@ namespace esphome { namespace md5 { -class MD5Digest { +class MD5Digest : public HashBase { public: MD5Digest() = default; - ~MD5Digest() = default; + ~MD5Digest() override = default; /// Initialize a new MD5 digest computation. - void init(); + void init() override; /// Add bytes of data for the digest. - void add(const uint8_t *data, size_t len); - void add(const char *data, size_t len) { this->add((const uint8_t *) data, len); } + void add(const uint8_t *data, size_t len) override; + using HashBase::add; // Bring base class overload into scope /// Compute the digest, based on the provided data. - void calculate(); + void calculate() override; - /// Retrieve the MD5 digest as bytes. - /// The output must be able to hold 16 bytes or more. - void get_bytes(uint8_t *output); - - /// Retrieve the MD5 digest as hex characters. - /// The output must be able to hold 32 bytes or more. - void get_hex(char *output); - - /// Compare the digest against a provided byte-encoded digest (16 bytes). - bool equals_bytes(const uint8_t *expected); - - /// Compare the digest against a provided hex-encoded digest (32 bytes). - bool equals_hex(const char *expected); + /// Get the size of the hash in bytes (16 for MD5) + size_t get_size() const override { return 16; } protected: MD5_CTX_TYPE ctx_{}; - uint8_t digest_[16]; }; } // namespace md5 diff --git a/esphome/components/mdns/__init__.py b/esphome/components/mdns/__init__.py index e32d39cede..c6a9ee1a0c 100644 --- a/esphome/components/mdns/__init__.py +++ b/esphome/components/mdns/__init__.py @@ -1,6 +1,6 @@ import esphome.codegen as cg from esphome.components.esp32 import add_idf_component -from esphome.config_helpers import filter_source_files_from_platform +from esphome.config_helpers import filter_source_files_from_platform, get_logger_level import esphome.config_validation as cv from esphome.const import ( CONF_DISABLED, @@ -11,11 +11,17 @@ from esphome.const import ( CONF_SERVICES, PlatformFramework, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, Lambda, coroutine_with_priority +from esphome.coroutine import CoroPriority CODEOWNERS = ["@esphome/core"] DEPENDENCIES = ["network"] +# Components that create mDNS services at runtime +# IMPORTANT: If you add a new component here, you must also update the corresponding +# #ifdef blocks in mdns_component.cpp compile_records_() method +COMPONENTS_WITH_MDNS_SERVICES = ("api", "prometheus", "web_server") + mdns_ns = cg.esphome_ns.namespace("mdns") MDNSComponent = mdns_ns.class_("MDNSComponent", cg.Component) MDNSTXTRecord = mdns_ns.struct("MDNSTXTRecord") @@ -52,27 +58,85 @@ CONFIG_SCHEMA = cv.All( ) -def mdns_txt_record(key: str, value: str): - return cg.StructInitializer( - MDNSTXTRecord, - ("key", key), - ("value", value), +def mdns_txt_record(key: str, value: str) -> cg.RawExpression: + """Create a mDNS TXT record. + + Public API for external components. Do not remove. + + Args: + key: The TXT record key + value: The TXT record value (static string only) + + Returns: + A RawExpression representing a MDNSTXTRecord struct + """ + return cg.RawExpression( + f"{{MDNS_STR({cg.safe_exp(key)}), MDNS_STR({cg.safe_exp(value)})}}" ) +async def _mdns_txt_record_templated( + mdns_comp: cg.Pvariable, key: str, value: Lambda | str +) -> cg.RawExpression: + """Create a mDNS TXT record with support for templated values. + + Internal helper function. + + Args: + mdns_comp: The MDNSComponent instance (from cg.get_variable()) + key: The TXT record key + value: The TXT record value (can be a static string or a lambda template) + + Returns: + A RawExpression representing a MDNSTXTRecord struct + """ + if not cg.is_template(value): + # It's a static string - use directly in flash, no need to store in vector + return mdns_txt_record(key, value) + # It's a lambda - evaluate and store using helper + templated_value = await cg.templatable(value, [], cg.std_string) + safe_key = cg.safe_exp(key) + dynamic_call = f"{mdns_comp}->add_dynamic_txt_value(({templated_value})())" + return cg.RawExpression(f"{{MDNS_STR({safe_key}), MDNS_STR({dynamic_call})}}") + + def mdns_service( - service: str, proto: str, port: int, txt_records: list[dict[str, str]] -): + service: str, proto: str, port: int, txt_records: list[cg.RawExpression] +) -> cg.StructInitializer: + """Create a mDNS service. + + Public API for external components. Do not remove. + + Args: + service: Service name (e.g., "_http") + proto: Protocol (e.g., "_tcp" or "_udp") + port: Port number + txt_records: List of MDNSTXTRecord expressions + + Returns: + A StructInitializer representing a MDNSService struct + """ return cg.StructInitializer( MDNSService, - ("service_type", service), - ("proto", proto), + ("service_type", cg.RawExpression(f"MDNS_STR({cg.safe_exp(service)})")), + ("proto", cg.RawExpression(f"MDNS_STR({cg.safe_exp(proto)})")), ("port", port), ("txt_records", txt_records), ) -@coroutine_with_priority(55.0) +def enable_mdns_storage(): + """Enable persistent storage of mDNS services in the MDNSComponent. + + Called by external components (like OpenThread) that need access to + services after setup() completes via get_services(). + + Public API for external components. Do not remove. + """ + cg.add_define("USE_MDNS_STORE_SERVICES") + + +@coroutine_with_priority(CoroPriority.NETWORK_SERVICES) async def to_code(config): if config[CONF_DISABLED] is True: return @@ -90,23 +154,54 @@ async def to_code(config): cg.add_define("USE_MDNS") + # Calculate compile-time service count + service_count = sum( + 1 for key in COMPONENTS_WITH_MDNS_SERVICES if key in CORE.config + ) + len(config[CONF_SERVICES]) + + if config[CONF_SERVICES]: + cg.add_define("USE_MDNS_EXTRA_SERVICES") + # Extra services need to be stored persistently + enable_mdns_storage() + + # Ensure at least 1 service (fallback service) + cg.add_define("MDNS_SERVICE_COUNT", max(1, service_count)) + + # Calculate compile-time dynamic TXT value count + # Dynamic values are those that cannot be stored in flash at compile time + dynamic_txt_count = 0 + if "api" in CORE.config: + # Always: get_mac_address() + dynamic_txt_count += 1 + # User-provided templatable TXT values (only lambdas, not static strings) + dynamic_txt_count += sum( + 1 + for service in config[CONF_SERVICES] + for txt_value in service[CONF_TXT].values() + if cg.is_template(txt_value) + ) + + # Ensure at least 1 to avoid zero-size array + cg.add_define("MDNS_DYNAMIC_TXT_COUNT", max(1, dynamic_txt_count)) + + # Enable storage if verbose logging is enabled (for dump_config) + if get_logger_level() in ("VERBOSE", "VERY_VERBOSE"): + enable_mdns_storage() + var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) for service in config[CONF_SERVICES]: - txt = [ - cg.StructInitializer( - MDNSTXTRecord, - ("key", txt_key), - ("value", await cg.templatable(txt_value, [], cg.std_string)), - ) + txt_records = [ + await _mdns_txt_record_templated(var, txt_key, txt_value) for txt_key, txt_value in service[CONF_TXT].items() ] + exp = mdns_service( service[CONF_SERVICE], service[CONF_PROTOCOL], await cg.templatable(service[CONF_PORT], [], cg.uint16), - txt, + txt_records, ) cg.add(var.add_extra_service(exp)) diff --git a/esphome/components/mdns/mdns_component.cpp b/esphome/components/mdns/mdns_component.cpp index 06ca99b402..d476136554 100644 --- a/esphome/components/mdns/mdns_component.cpp +++ b/esphome/components/mdns/mdns_component.cpp @@ -5,6 +5,15 @@ #include "esphome/core/version.h" #include "mdns_component.h" +#ifdef USE_ESP8266 +#include +// Macro to define strings in PROGMEM on ESP8266, regular memory on other platforms +#define MDNS_STATIC_CONST_CHAR(name, value) static const char name[] PROGMEM = value +#else +// On non-ESP8266 platforms, use regular const char* +#define MDNS_STATIC_CONST_CHAR(name, value) static constexpr const char name[] = value +#endif + #ifdef USE_API #include "esphome/components/api/api_server.h" #endif @@ -21,101 +30,149 @@ static const char *const TAG = "mdns"; #define USE_WEBSERVER_PORT 80 // NOLINT #endif -void MDNSComponent::compile_records_() { +// Define all constant strings using the macro +MDNS_STATIC_CONST_CHAR(SERVICE_TCP, "_tcp"); + +// Wrap build-time defines into flash storage +MDNS_STATIC_CONST_CHAR(VALUE_VERSION, ESPHOME_VERSION); + +void MDNSComponent::compile_records_(StaticVector &services) { this->hostname_ = App.get_name(); - this->services_.clear(); -#ifdef USE_API - if (api::global_api_server != nullptr) { - MDNSService service{}; - service.service_type = "_esphomelib"; - service.proto = "_tcp"; - service.port = api::global_api_server->get_port(); - if (!App.get_friendly_name().empty()) { - service.txt_records.push_back({"friendly_name", App.get_friendly_name()}); - } - service.txt_records.push_back({"version", ESPHOME_VERSION}); - service.txt_records.push_back({"mac", get_mac_address()}); - const char *platform = nullptr; -#ifdef USE_ESP8266 - platform = "ESP8266"; -#endif -#ifdef USE_ESP32 - platform = "ESP32"; -#endif -#ifdef USE_RP2040 - platform = "RP2040"; -#endif -#ifdef USE_LIBRETINY - platform = lt_cpu_get_model_name(); -#endif - if (platform != nullptr) { - service.txt_records.push_back({"platform", platform}); - } + // IMPORTANT: The #ifdef blocks below must match COMPONENTS_WITH_MDNS_SERVICES + // in mdns/__init__.py. If you add a new service here, update both locations. - service.txt_records.push_back({"board", ESPHOME_BOARD}); +#ifdef USE_API + MDNS_STATIC_CONST_CHAR(SERVICE_ESPHOMELIB, "_esphomelib"); + MDNS_STATIC_CONST_CHAR(TXT_FRIENDLY_NAME, "friendly_name"); + MDNS_STATIC_CONST_CHAR(TXT_VERSION, "version"); + MDNS_STATIC_CONST_CHAR(TXT_MAC, "mac"); + MDNS_STATIC_CONST_CHAR(TXT_PLATFORM, "platform"); + MDNS_STATIC_CONST_CHAR(TXT_BOARD, "board"); + MDNS_STATIC_CONST_CHAR(TXT_NETWORK, "network"); + MDNS_STATIC_CONST_CHAR(VALUE_BOARD, ESPHOME_BOARD); + + if (api::global_api_server != nullptr) { + auto &service = services.emplace_next(); + service.service_type = MDNS_STR(SERVICE_ESPHOMELIB); + service.proto = MDNS_STR(SERVICE_TCP); + service.port = api::global_api_server->get_port(); + + const std::string &friendly_name = App.get_friendly_name(); + bool friendly_name_empty = friendly_name.empty(); + + // Calculate exact capacity for txt_records + size_t txt_count = 3; // version, mac, board (always present) + if (!friendly_name_empty) { + txt_count++; // friendly_name + } +#if defined(USE_ESP8266) || defined(USE_ESP32) || defined(USE_RP2040) || defined(USE_LIBRETINY) + txt_count++; // platform +#endif +#if defined(USE_WIFI) || defined(USE_ETHERNET) || defined(USE_OPENTHREAD) + txt_count++; // network +#endif +#ifdef USE_API_NOISE + txt_count++; // api_encryption or api_encryption_supported +#endif +#ifdef ESPHOME_PROJECT_NAME + txt_count += 2; // project_name and project_version +#endif +#ifdef USE_DASHBOARD_IMPORT + txt_count++; // package_import_url +#endif + + auto &txt_records = service.txt_records; + txt_records.init(txt_count); + + if (!friendly_name_empty) { + txt_records.push_back({MDNS_STR(TXT_FRIENDLY_NAME), MDNS_STR(friendly_name.c_str())}); + } + txt_records.push_back({MDNS_STR(TXT_VERSION), MDNS_STR(VALUE_VERSION)}); + txt_records.push_back({MDNS_STR(TXT_MAC), MDNS_STR(this->add_dynamic_txt_value(get_mac_address()))}); + +#ifdef USE_ESP8266 + MDNS_STATIC_CONST_CHAR(PLATFORM_ESP8266, "ESP8266"); + txt_records.push_back({MDNS_STR(TXT_PLATFORM), MDNS_STR(PLATFORM_ESP8266)}); +#elif defined(USE_ESP32) + MDNS_STATIC_CONST_CHAR(PLATFORM_ESP32, "ESP32"); + txt_records.push_back({MDNS_STR(TXT_PLATFORM), MDNS_STR(PLATFORM_ESP32)}); +#elif defined(USE_RP2040) + MDNS_STATIC_CONST_CHAR(PLATFORM_RP2040, "RP2040"); + txt_records.push_back({MDNS_STR(TXT_PLATFORM), MDNS_STR(PLATFORM_RP2040)}); +#elif defined(USE_LIBRETINY) + txt_records.push_back({MDNS_STR(TXT_PLATFORM), MDNS_STR(lt_cpu_get_model_name())}); +#endif + + txt_records.push_back({MDNS_STR(TXT_BOARD), MDNS_STR(VALUE_BOARD)}); #if defined(USE_WIFI) - service.txt_records.push_back({"network", "wifi"}); + MDNS_STATIC_CONST_CHAR(NETWORK_WIFI, "wifi"); + txt_records.push_back({MDNS_STR(TXT_NETWORK), MDNS_STR(NETWORK_WIFI)}); #elif defined(USE_ETHERNET) - service.txt_records.push_back({"network", "ethernet"}); + MDNS_STATIC_CONST_CHAR(NETWORK_ETHERNET, "ethernet"); + txt_records.push_back({MDNS_STR(TXT_NETWORK), MDNS_STR(NETWORK_ETHERNET)}); #elif defined(USE_OPENTHREAD) - service.txt_records.push_back({"network", "thread"}); + MDNS_STATIC_CONST_CHAR(NETWORK_THREAD, "thread"); + txt_records.push_back({MDNS_STR(TXT_NETWORK), MDNS_STR(NETWORK_THREAD)}); #endif #ifdef USE_API_NOISE - if (api::global_api_server->get_noise_ctx()->has_psk()) { - service.txt_records.push_back({"api_encryption", "Noise_NNpsk0_25519_ChaChaPoly_SHA256"}); - } else { - service.txt_records.push_back({"api_encryption_supported", "Noise_NNpsk0_25519_ChaChaPoly_SHA256"}); - } + MDNS_STATIC_CONST_CHAR(TXT_API_ENCRYPTION, "api_encryption"); + MDNS_STATIC_CONST_CHAR(TXT_API_ENCRYPTION_SUPPORTED, "api_encryption_supported"); + MDNS_STATIC_CONST_CHAR(NOISE_ENCRYPTION, "Noise_NNpsk0_25519_ChaChaPoly_SHA256"); + bool has_psk = api::global_api_server->get_noise_ctx()->has_psk(); + const char *encryption_key = has_psk ? TXT_API_ENCRYPTION : TXT_API_ENCRYPTION_SUPPORTED; + txt_records.push_back({MDNS_STR(encryption_key), MDNS_STR(NOISE_ENCRYPTION)}); #endif #ifdef ESPHOME_PROJECT_NAME - service.txt_records.push_back({"project_name", ESPHOME_PROJECT_NAME}); - service.txt_records.push_back({"project_version", ESPHOME_PROJECT_VERSION}); + MDNS_STATIC_CONST_CHAR(TXT_PROJECT_NAME, "project_name"); + MDNS_STATIC_CONST_CHAR(TXT_PROJECT_VERSION, "project_version"); + MDNS_STATIC_CONST_CHAR(VALUE_PROJECT_NAME, ESPHOME_PROJECT_NAME); + MDNS_STATIC_CONST_CHAR(VALUE_PROJECT_VERSION, ESPHOME_PROJECT_VERSION); + txt_records.push_back({MDNS_STR(TXT_PROJECT_NAME), MDNS_STR(VALUE_PROJECT_NAME)}); + txt_records.push_back({MDNS_STR(TXT_PROJECT_VERSION), MDNS_STR(VALUE_PROJECT_VERSION)}); #endif // ESPHOME_PROJECT_NAME #ifdef USE_DASHBOARD_IMPORT - service.txt_records.push_back({"package_import_url", dashboard_import::get_package_import_url()}); + MDNS_STATIC_CONST_CHAR(TXT_PACKAGE_IMPORT_URL, "package_import_url"); + txt_records.push_back( + {MDNS_STR(TXT_PACKAGE_IMPORT_URL), MDNS_STR(dashboard_import::get_package_import_url().c_str())}); #endif - - this->services_.push_back(service); } #endif // USE_API #ifdef USE_PROMETHEUS - { - MDNSService service{}; - service.service_type = "_prometheus-http"; - service.proto = "_tcp"; - service.port = USE_WEBSERVER_PORT; - this->services_.push_back(service); - } + MDNS_STATIC_CONST_CHAR(SERVICE_PROMETHEUS, "_prometheus-http"); + + auto &prom_service = services.emplace_next(); + prom_service.service_type = MDNS_STR(SERVICE_PROMETHEUS); + prom_service.proto = MDNS_STR(SERVICE_TCP); + prom_service.port = USE_WEBSERVER_PORT; #endif #ifdef USE_WEBSERVER - { - MDNSService service{}; - service.service_type = "_http"; - service.proto = "_tcp"; - service.port = USE_WEBSERVER_PORT; - this->services_.push_back(service); - } + MDNS_STATIC_CONST_CHAR(SERVICE_HTTP, "_http"); + + auto &web_service = services.emplace_next(); + web_service.service_type = MDNS_STR(SERVICE_HTTP); + web_service.proto = MDNS_STR(SERVICE_TCP); + web_service.port = USE_WEBSERVER_PORT; #endif - this->services_.insert(this->services_.end(), this->services_extra_.begin(), this->services_extra_.end()); +#if !defined(USE_API) && !defined(USE_PROMETHEUS) && !defined(USE_WEBSERVER) && !defined(USE_MDNS_EXTRA_SERVICES) + MDNS_STATIC_CONST_CHAR(SERVICE_HTTP, "_http"); + MDNS_STATIC_CONST_CHAR(TXT_VERSION, "version"); - if (this->services_.empty()) { - // Publish "http" service if not using native API - // This is just to have *some* mDNS service so that .local resolution works - MDNSService service{}; - service.service_type = "_http"; - service.proto = "_tcp"; - service.port = USE_WEBSERVER_PORT; - service.txt_records.push_back({"version", ESPHOME_VERSION}); - this->services_.push_back(service); - } + // Publish "http" service if not using native API or any other services + // This is just to have *some* mDNS service so that .local resolution works + auto &fallback_service = services.emplace_next(); + fallback_service.service_type = MDNS_STR(SERVICE_HTTP); + fallback_service.proto = MDNS_STR(SERVICE_TCP); + fallback_service.port = USE_WEBSERVER_PORT; + fallback_service.txt_records = {{MDNS_STR(TXT_VERSION), MDNS_STR(VALUE_VERSION)}}; +#endif } void MDNSComponent::dump_config() { @@ -123,19 +180,18 @@ void MDNSComponent::dump_config() { "mDNS:\n" " Hostname: %s", this->hostname_.c_str()); +#ifdef USE_MDNS_STORE_SERVICES ESP_LOGV(TAG, " Services:"); for (const auto &service : this->services_) { - ESP_LOGV(TAG, " - %s, %s, %d", service.service_type.c_str(), service.proto.c_str(), + ESP_LOGV(TAG, " - %s, %s, %d", MDNS_STR_ARG(service.service_type), MDNS_STR_ARG(service.proto), const_cast &>(service.port).value()); for (const auto &record : service.txt_records) { - ESP_LOGV(TAG, " TXT: %s = %s", record.key.c_str(), - const_cast &>(record.value).value().c_str()); + ESP_LOGV(TAG, " TXT: %s = %s", MDNS_STR_ARG(record.key), MDNS_STR_ARG(record.value)); } } +#endif } -std::vector MDNSComponent::get_services() { return this->services_; } - } // namespace mdns } // namespace esphome #endif diff --git a/esphome/components/mdns/mdns_component.h b/esphome/components/mdns/mdns_component.h index 93a16f40d2..35371fd739 100644 --- a/esphome/components/mdns/mdns_component.h +++ b/esphome/components/mdns/mdns_component.h @@ -2,27 +2,43 @@ #include "esphome/core/defines.h" #ifdef USE_MDNS #include -#include #include "esphome/core/automation.h" #include "esphome/core/component.h" +#include "esphome/core/helpers.h" namespace esphome { namespace mdns { +// Helper struct that identifies strings that may be stored in flash storage (similar to LogString) +struct MDNSString; + +// Macro to cast string literals to MDNSString* (works on all platforms) +#define MDNS_STR(name) (reinterpret_cast(name)) + +#ifdef USE_ESP8266 +#include +#define MDNS_STR_ARG(s) ((PGM_P) (s)) +#else +#define MDNS_STR_ARG(s) (reinterpret_cast(s)) +#endif + +// Service count is calculated at compile time by Python codegen +// MDNS_SERVICE_COUNT will always be defined + struct MDNSTXTRecord { - std::string key; - TemplatableValue value; + const MDNSString *key; + const MDNSString *value; }; struct MDNSService { // service name _including_ underscore character prefix // as defined in RFC6763 Section 7 - std::string service_type; + const MDNSString *service_type; // second label indicating protocol _including_ underscore character prefix // as defined in RFC6763 Section 7, like "_tcp" or "_udp" - std::string proto; + const MDNSString *proto; TemplatableValue port; - std::vector txt_records; + FixedVector txt_records; }; class MDNSComponent : public Component { @@ -35,17 +51,33 @@ class MDNSComponent : public Component { #endif float get_setup_priority() const override { return setup_priority::AFTER_CONNECTION; } - void add_extra_service(MDNSService service) { services_extra_.push_back(std::move(service)); } +#ifdef USE_MDNS_EXTRA_SERVICES + void add_extra_service(MDNSService service) { this->services_.emplace_next() = std::move(service); } +#endif - std::vector get_services(); +#ifdef USE_MDNS_STORE_SERVICES + const StaticVector &get_services() const { return this->services_; } +#endif void on_shutdown() override; + /// Add a dynamic TXT value and return pointer to it for use in MDNSTXTRecord + const char *add_dynamic_txt_value(const std::string &value) { + this->dynamic_txt_values_.push_back(value); + return this->dynamic_txt_values_[this->dynamic_txt_values_.size() - 1].c_str(); + } + + /// Storage for runtime-generated TXT values (MAC address, user lambdas) + /// Pre-sized at compile time via MDNS_DYNAMIC_TXT_COUNT to avoid heap allocations. + /// Static/compile-time values (version, board, etc.) are stored directly in flash and don't use this. + StaticVector dynamic_txt_values_; + protected: - std::vector services_extra_{}; - std::vector services_{}; +#ifdef USE_MDNS_STORE_SERVICES + StaticVector services_{}; +#endif std::string hostname_; - void compile_records_(); + void compile_records_(StaticVector &services); }; } // namespace mdns diff --git a/esphome/components/mdns/mdns_esp32.cpp b/esphome/components/mdns/mdns_esp32.cpp index ffd86afec1..f2cb2d3ef5 100644 --- a/esphome/components/mdns/mdns_esp32.cpp +++ b/esphome/components/mdns/mdns_esp32.cpp @@ -2,7 +2,6 @@ #if defined(USE_ESP32) && defined(USE_MDNS) #include -#include #include "esphome/core/hal.h" #include "esphome/core/log.h" #include "mdns_component.h" @@ -13,7 +12,13 @@ namespace mdns { static const char *const TAG = "mdns"; void MDNSComponent::setup() { - this->compile_records_(); +#ifdef USE_MDNS_STORE_SERVICES + this->compile_records_(this->services_); + const auto &services = this->services_; +#else + StaticVector services; + this->compile_records_(services); +#endif esp_err_t err = mdns_init(); if (err != ESP_OK) { @@ -25,27 +30,22 @@ void MDNSComponent::setup() { mdns_hostname_set(this->hostname_.c_str()); mdns_instance_name_set(this->hostname_.c_str()); - for (const auto &service : this->services_) { + for (const auto &service : services) { std::vector txt_records; for (const auto &record : service.txt_records) { mdns_txt_item_t it{}; - // dup strings to ensure the pointer is valid even after the record loop - it.key = strdup(record.key.c_str()); - it.value = strdup(const_cast &>(record.value).value().c_str()); + // key and value are either compile-time string literals in flash or pointers to dynamic_txt_values_ + // Both remain valid for the lifetime of this function, and ESP-IDF makes internal copies + it.key = MDNS_STR_ARG(record.key); + it.value = MDNS_STR_ARG(record.value); txt_records.push_back(it); } uint16_t port = const_cast &>(service.port).value(); - err = mdns_service_add(nullptr, service.service_type.c_str(), service.proto.c_str(), port, txt_records.data(), - txt_records.size()); - - // free records - for (const auto &it : txt_records) { - delete it.key; // NOLINT(cppcoreguidelines-owning-memory) - delete it.value; // NOLINT(cppcoreguidelines-owning-memory) - } + err = mdns_service_add(nullptr, MDNS_STR_ARG(service.service_type), MDNS_STR_ARG(service.proto), port, + txt_records.data(), txt_records.size()); if (err != ESP_OK) { - ESP_LOGW(TAG, "Failed to register service %s: %s", service.service_type.c_str(), esp_err_to_name(err)); + ESP_LOGW(TAG, "Failed to register service %s: %s", MDNS_STR_ARG(service.service_type), esp_err_to_name(err)); } } } diff --git a/esphome/components/mdns/mdns_esp8266.cpp b/esphome/components/mdns/mdns_esp8266.cpp index 2c90d57021..25a3defa7b 100644 --- a/esphome/components/mdns/mdns_esp8266.cpp +++ b/esphome/components/mdns/mdns_esp8266.cpp @@ -12,28 +12,34 @@ namespace esphome { namespace mdns { void MDNSComponent::setup() { - this->compile_records_(); +#ifdef USE_MDNS_STORE_SERVICES + this->compile_records_(this->services_); + const auto &services = this->services_; +#else + StaticVector services; + this->compile_records_(services); +#endif MDNS.begin(this->hostname_.c_str()); - for (const auto &service : this->services_) { + for (const auto &service : services) { // Strip the leading underscore from the proto and service_type. While it is // part of the wire protocol to have an underscore, and for example ESP-IDF // expects the underscore to be there, the ESP8266 implementation always adds // the underscore itself. - auto *proto = service.proto.c_str(); - while (*proto == '_') { + auto *proto = MDNS_STR_ARG(service.proto); + while (progmem_read_byte((const uint8_t *) proto) == '_') { proto++; } - auto *service_type = service.service_type.c_str(); - while (*service_type == '_') { + auto *service_type = MDNS_STR_ARG(service.service_type); + while (progmem_read_byte((const uint8_t *) service_type) == '_') { service_type++; } uint16_t port = const_cast &>(service.port).value(); - MDNS.addService(service_type, proto, port); + MDNS.addService(FPSTR(service_type), FPSTR(proto), port); for (const auto &record : service.txt_records) { - MDNS.addServiceTxt(service_type, proto, record.key.c_str(), - const_cast &>(record.value).value().c_str()); + MDNS.addServiceTxt(FPSTR(service_type), FPSTR(proto), FPSTR(MDNS_STR_ARG(record.key)), + FPSTR(MDNS_STR_ARG(record.value))); } } } diff --git a/esphome/components/mdns/mdns_host.cpp b/esphome/components/mdns/mdns_host.cpp index 78767ed136..f645d8d068 100644 --- a/esphome/components/mdns/mdns_host.cpp +++ b/esphome/components/mdns/mdns_host.cpp @@ -9,7 +9,9 @@ namespace esphome { namespace mdns { -void MDNSComponent::setup() { this->compile_records_(); } +void MDNSComponent::setup() { + // Host platform doesn't have actual mDNS implementation +} void MDNSComponent::on_shutdown() {} diff --git a/esphome/components/mdns/mdns_libretiny.cpp b/esphome/components/mdns/mdns_libretiny.cpp index 7a41ec9dce..a3e317a2bf 100644 --- a/esphome/components/mdns/mdns_libretiny.cpp +++ b/esphome/components/mdns/mdns_libretiny.cpp @@ -12,28 +12,33 @@ namespace esphome { namespace mdns { void MDNSComponent::setup() { - this->compile_records_(); +#ifdef USE_MDNS_STORE_SERVICES + this->compile_records_(this->services_); + const auto &services = this->services_; +#else + StaticVector services; + this->compile_records_(services); +#endif MDNS.begin(this->hostname_.c_str()); - for (const auto &service : this->services_) { + for (const auto &service : services) { // Strip the leading underscore from the proto and service_type. While it is // part of the wire protocol to have an underscore, and for example ESP-IDF // expects the underscore to be there, the ESP8266 implementation always adds // the underscore itself. - auto *proto = service.proto.c_str(); + auto *proto = MDNS_STR_ARG(service.proto); while (*proto == '_') { proto++; } - auto *service_type = service.service_type.c_str(); + auto *service_type = MDNS_STR_ARG(service.service_type); while (*service_type == '_') { service_type++; } uint16_t port_ = const_cast &>(service.port).value(); MDNS.addService(service_type, proto, port_); for (const auto &record : service.txt_records) { - MDNS.addServiceTxt(service_type, proto, record.key.c_str(), - const_cast &>(record.value).value().c_str()); + MDNS.addServiceTxt(service_type, proto, MDNS_STR_ARG(record.key), MDNS_STR_ARG(record.value)); } } } diff --git a/esphome/components/mdns/mdns_rp2040.cpp b/esphome/components/mdns/mdns_rp2040.cpp index 95894323f4..791fa3934d 100644 --- a/esphome/components/mdns/mdns_rp2040.cpp +++ b/esphome/components/mdns/mdns_rp2040.cpp @@ -12,28 +12,33 @@ namespace esphome { namespace mdns { void MDNSComponent::setup() { - this->compile_records_(); +#ifdef USE_MDNS_STORE_SERVICES + this->compile_records_(this->services_); + const auto &services = this->services_; +#else + StaticVector services; + this->compile_records_(services); +#endif MDNS.begin(this->hostname_.c_str()); - for (const auto &service : this->services_) { + for (const auto &service : services) { // Strip the leading underscore from the proto and service_type. While it is // part of the wire protocol to have an underscore, and for example ESP-IDF // expects the underscore to be there, the ESP8266 implementation always adds // the underscore itself. - auto *proto = service.proto.c_str(); + auto *proto = MDNS_STR_ARG(service.proto); while (*proto == '_') { proto++; } - auto *service_type = service.service_type.c_str(); + auto *service_type = MDNS_STR_ARG(service.service_type); while (*service_type == '_') { service_type++; } uint16_t port = const_cast &>(service.port).value(); MDNS.addService(service_type, proto, port); for (const auto &record : service.txt_records) { - MDNS.addServiceTxt(service_type, proto, record.key.c_str(), - const_cast &>(record.value).value().c_str()); + MDNS.addServiceTxt(service_type, proto, MDNS_STR_ARG(record.key), MDNS_STR_ARG(record.value)); } } } diff --git a/esphome/components/media_player/__init__.py b/esphome/components/media_player/__init__.py index ccded1deb2..70c7cf7a56 100644 --- a/esphome/components/media_player/__init__.py +++ b/esphome/components/media_player/__init__.py @@ -7,12 +7,14 @@ from esphome.const import ( CONF_ID, CONF_ON_IDLE, CONF_ON_STATE, + CONF_ON_TURN_OFF, + CONF_ON_TURN_ON, CONF_TRIGGER_ID, CONF_VOLUME, ) from esphome.core import CORE from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity -from esphome.coroutine import coroutine_with_priority +from esphome.coroutine import CoroPriority, coroutine_with_priority from esphome.cpp_generator import MockObjClass CODEOWNERS = ["@jesserockz"] @@ -58,6 +60,12 @@ VolumeDownAction = media_player_ns.class_( VolumeSetAction = media_player_ns.class_( "VolumeSetAction", automation.Action, cg.Parented.template(MediaPlayer) ) +TurnOnAction = media_player_ns.class_( + "TurnOnAction", automation.Action, cg.Parented.template(MediaPlayer) +) +TurnOffAction = media_player_ns.class_( + "TurnOffAction", automation.Action, cg.Parented.template(MediaPlayer) +) CONF_ANNOUNCEMENT = "announcement" CONF_ON_PLAY = "on_play" @@ -72,12 +80,16 @@ PauseTrigger = media_player_ns.class_("PauseTrigger", automation.Trigger.templat AnnoucementTrigger = media_player_ns.class_( "AnnouncementTrigger", automation.Trigger.template() ) +OnTrigger = media_player_ns.class_("OnTrigger", automation.Trigger.template()) +OffTrigger = media_player_ns.class_("OffTrigger", automation.Trigger.template()) IsIdleCondition = media_player_ns.class_("IsIdleCondition", automation.Condition) IsPausedCondition = media_player_ns.class_("IsPausedCondition", automation.Condition) IsPlayingCondition = media_player_ns.class_("IsPlayingCondition", automation.Condition) IsAnnouncingCondition = media_player_ns.class_( "IsAnnouncingCondition", automation.Condition ) +IsOnCondition = media_player_ns.class_("IsOnCondition", automation.Condition) +IsOffCondition = media_player_ns.class_("IsOffCondition", automation.Condition) async def setup_media_player_core_(var, config): @@ -97,6 +109,12 @@ async def setup_media_player_core_(var, config): for conf in config.get(CONF_ON_ANNOUNCEMENT, []): trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var) await automation.build_automation(trigger, [], conf) + for conf in config.get(CONF_ON_TURN_ON, []): + trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var) + await automation.build_automation(trigger, [], conf) + for conf in config.get(CONF_ON_TURN_OFF, []): + trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var) + await automation.build_automation(trigger, [], conf) async def register_media_player(var, config): @@ -140,6 +158,16 @@ _MEDIA_PLAYER_SCHEMA = cv.ENTITY_BASE_SCHEMA.extend( cv.GenerateID(CONF_TRIGGER_ID): cv.declare_id(AnnoucementTrigger), } ), + cv.Optional(CONF_ON_TURN_ON): automation.validate_automation( + { + cv.GenerateID(CONF_TRIGGER_ID): cv.declare_id(OnTrigger), + } + ), + cv.Optional(CONF_ON_TURN_OFF): automation.validate_automation( + { + cv.GenerateID(CONF_TRIGGER_ID): cv.declare_id(OffTrigger), + } + ), } ) @@ -218,6 +246,12 @@ async def media_player_play_media_action(config, action_id, template_arg, args): @automation.register_action( "media_player.volume_down", VolumeDownAction, MEDIA_PLAYER_ACTION_SCHEMA ) +@automation.register_action( + "media_player.turn_on", TurnOnAction, MEDIA_PLAYER_ACTION_SCHEMA +) +@automation.register_action( + "media_player.turn_off", TurnOffAction, MEDIA_PLAYER_ACTION_SCHEMA +) async def media_player_action(config, action_id, template_arg, args): var = cg.new_Pvariable(action_id, template_arg) await cg.register_parented(var, config[CONF_ID]) @@ -238,6 +272,12 @@ async def media_player_action(config, action_id, template_arg, args): @automation.register_condition( "media_player.is_announcing", IsAnnouncingCondition, MEDIA_PLAYER_CONDITION_SCHEMA ) +@automation.register_condition( + "media_player.is_on", IsOnCondition, MEDIA_PLAYER_CONDITION_SCHEMA +) +@automation.register_condition( + "media_player.is_off", IsOffCondition, MEDIA_PLAYER_CONDITION_SCHEMA +) async def media_player_condition(config, action_id, template_arg, args): var = cg.new_Pvariable(action_id, template_arg) await cg.register_parented(var, config[CONF_ID]) @@ -263,7 +303,7 @@ async def media_player_volume_set_action(config, action_id, template_arg, args): return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): cg.add_global(media_player_ns.using) cg.add_define("USE_MEDIA_PLAYER") diff --git a/esphome/components/media_player/automation.h b/esphome/components/media_player/automation.h index 422c224a85..3af5959f32 100644 --- a/esphome/components/media_player/automation.h +++ b/esphome/components/media_player/automation.h @@ -28,6 +28,10 @@ template using VolumeUpAction = MediaPlayerCommandAction; template using VolumeDownAction = MediaPlayerCommandAction; +template +using TurnOnAction = MediaPlayerCommandAction; +template +using TurnOffAction = MediaPlayerCommandAction; template class PlayMediaAction : public Action, public Parented { TEMPLATABLE_VALUE(std::string, media_url) @@ -66,6 +70,8 @@ using IdleTrigger = MediaPlayerStateTrigger; using PauseTrigger = MediaPlayerStateTrigger; using AnnouncementTrigger = MediaPlayerStateTrigger; +using OnTrigger = MediaPlayerStateTrigger; +using OffTrigger = MediaPlayerStateTrigger; template class IsIdleCondition : public Condition, public Parented { public: @@ -87,5 +93,15 @@ template class IsAnnouncingCondition : public Condition, bool check(Ts... x) override { return this->parent_->state == MediaPlayerState::MEDIA_PLAYER_STATE_ANNOUNCING; } }; +template class IsOnCondition : public Condition, public Parented { + public: + bool check(Ts... x) override { return this->parent_->state == MediaPlayerState::MEDIA_PLAYER_STATE_ON; } +}; + +template class IsOffCondition : public Condition, public Parented { + public: + bool check(Ts... x) override { return this->parent_->state == MediaPlayerState::MEDIA_PLAYER_STATE_OFF; } +}; + } // namespace media_player } // namespace esphome diff --git a/esphome/components/media_player/media_player.cpp b/esphome/components/media_player/media_player.cpp index 32da7ee265..3f274bf73b 100644 --- a/esphome/components/media_player/media_player.cpp +++ b/esphome/components/media_player/media_player.cpp @@ -9,6 +9,10 @@ static const char *const TAG = "media_player"; const char *media_player_state_to_string(MediaPlayerState state) { switch (state) { + case MEDIA_PLAYER_STATE_ON: + return "ON"; + case MEDIA_PLAYER_STATE_OFF: + return "OFF"; case MEDIA_PLAYER_STATE_IDLE: return "IDLE"; case MEDIA_PLAYER_STATE_PLAYING: @@ -18,6 +22,7 @@ const char *media_player_state_to_string(MediaPlayerState state) { case MEDIA_PLAYER_STATE_ANNOUNCING: return "ANNOUNCING"; case MEDIA_PLAYER_STATE_NONE: + return "NONE"; default: return "UNKNOWN"; } @@ -49,6 +54,10 @@ const char *media_player_command_to_string(MediaPlayerCommand command) { return "REPEAT_OFF"; case MEDIA_PLAYER_COMMAND_CLEAR_PLAYLIST: return "CLEAR_PLAYLIST"; + case MEDIA_PLAYER_COMMAND_TURN_ON: + return "TURN_ON"; + case MEDIA_PLAYER_COMMAND_TURN_OFF: + return "TURN_OFF"; default: return "UNKNOWN"; } @@ -110,6 +119,10 @@ MediaPlayerCall &MediaPlayerCall::set_command(const std::string &command) { this->set_command(MEDIA_PLAYER_COMMAND_UNMUTE); } else if (str_equals_case_insensitive(command, "TOGGLE")) { this->set_command(MEDIA_PLAYER_COMMAND_TOGGLE); + } else if (str_equals_case_insensitive(command, "TURN_ON")) { + this->set_command(MEDIA_PLAYER_COMMAND_TURN_ON); + } else if (str_equals_case_insensitive(command, "TURN_OFF")) { + this->set_command(MEDIA_PLAYER_COMMAND_TURN_OFF); } else { ESP_LOGW(TAG, "'%s' - Unrecognized command %s", this->parent_->get_name().c_str(), command.c_str()); } diff --git a/esphome/components/media_player/media_player.h b/esphome/components/media_player/media_player.h index ee5889901c..2f1c99115f 100644 --- a/esphome/components/media_player/media_player.h +++ b/esphome/components/media_player/media_player.h @@ -6,12 +6,40 @@ namespace esphome { namespace media_player { +enum MediaPlayerEntityFeature : uint32_t { + PAUSE = 1 << 0, + SEEK = 1 << 1, + VOLUME_SET = 1 << 2, + VOLUME_MUTE = 1 << 3, + PREVIOUS_TRACK = 1 << 4, + NEXT_TRACK = 1 << 5, + + TURN_ON = 1 << 7, + TURN_OFF = 1 << 8, + PLAY_MEDIA = 1 << 9, + VOLUME_STEP = 1 << 10, + SELECT_SOURCE = 1 << 11, + STOP = 1 << 12, + CLEAR_PLAYLIST = 1 << 13, + PLAY = 1 << 14, + SHUFFLE_SET = 1 << 15, + SELECT_SOUND_MODE = 1 << 16, + BROWSE_MEDIA = 1 << 17, + REPEAT_SET = 1 << 18, + GROUPING = 1 << 19, + MEDIA_ANNOUNCE = 1 << 20, + MEDIA_ENQUEUE = 1 << 21, + SEARCH_MEDIA = 1 << 22, +}; + enum MediaPlayerState : uint8_t { MEDIA_PLAYER_STATE_NONE = 0, MEDIA_PLAYER_STATE_IDLE = 1, MEDIA_PLAYER_STATE_PLAYING = 2, MEDIA_PLAYER_STATE_PAUSED = 3, - MEDIA_PLAYER_STATE_ANNOUNCING = 4 + MEDIA_PLAYER_STATE_ANNOUNCING = 4, + MEDIA_PLAYER_STATE_OFF = 5, + MEDIA_PLAYER_STATE_ON = 6, }; const char *media_player_state_to_string(MediaPlayerState state); @@ -28,6 +56,8 @@ enum MediaPlayerCommand : uint8_t { MEDIA_PLAYER_COMMAND_REPEAT_ONE = 9, MEDIA_PLAYER_COMMAND_REPEAT_OFF = 10, MEDIA_PLAYER_COMMAND_CLEAR_PLAYLIST = 11, + MEDIA_PLAYER_COMMAND_TURN_ON = 12, + MEDIA_PLAYER_COMMAND_TURN_OFF = 13, }; const char *media_player_command_to_string(MediaPlayerCommand command); @@ -51,14 +81,31 @@ class MediaPlayerTraits { MediaPlayerTraits() = default; void set_supports_pause(bool supports_pause) { this->supports_pause_ = supports_pause; } - bool get_supports_pause() const { return this->supports_pause_; } + void set_supports_turn_off_on(bool supports_turn_off_on) { this->supports_turn_off_on_ = supports_turn_off_on; } + bool get_supports_turn_off_on() const { return this->supports_turn_off_on_; } + std::vector &get_supported_formats() { return this->supported_formats_; } + uint32_t get_feature_flags() const { + uint32_t flags = 0; + flags |= MediaPlayerEntityFeature::PLAY_MEDIA | MediaPlayerEntityFeature::BROWSE_MEDIA | + MediaPlayerEntityFeature::STOP | MediaPlayerEntityFeature::VOLUME_SET | + MediaPlayerEntityFeature::VOLUME_MUTE | MediaPlayerEntityFeature::MEDIA_ANNOUNCE; + if (this->get_supports_pause()) { + flags |= MediaPlayerEntityFeature::PAUSE | MediaPlayerEntityFeature::PLAY; + } + if (this->get_supports_turn_off_on()) { + flags |= MediaPlayerEntityFeature::TURN_OFF | MediaPlayerEntityFeature::TURN_ON; + } + return flags; + } + protected: - bool supports_pause_{false}; std::vector supported_formats_{}; + bool supports_pause_{false}; + bool supports_turn_off_on_{false}; }; class MediaPlayerCall { diff --git a/esphome/components/micro_wake_word/__init__.py b/esphome/components/micro_wake_word/__init__.py index cde8752157..8cd7115368 100644 --- a/esphome/components/micro_wake_word/__init__.py +++ b/esphome/components/micro_wake_word/__init__.py @@ -201,7 +201,7 @@ def _validate_manifest_version(manifest_data): else: raise cv.Invalid("Invalid manifest version") else: - raise cv.Invalid("Invalid manifest file, missing 'version' key.") + raise cv.Invalid("Invalid manifest file, missing 'version' key") def _process_http_source(config): @@ -421,7 +421,7 @@ def _feature_step_size_validate(config): if features_step_size is None: features_step_size = model_step_size elif features_step_size != model_step_size: - raise cv.Invalid("Cannot load models with different features step sizes.") + raise cv.Invalid("Cannot load models with different features step sizes") FINAL_VALIDATE_SCHEMA = cv.All( diff --git a/esphome/components/micro_wake_word/micro_wake_word.cpp b/esphome/components/micro_wake_word/micro_wake_word.cpp index 201d956a37..6fca48a5bd 100644 --- a/esphome/components/micro_wake_word/micro_wake_word.cpp +++ b/esphome/components/micro_wake_word/micro_wake_word.cpp @@ -72,8 +72,6 @@ void MicroWakeWord::dump_config() { } void MicroWakeWord::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->frontend_config_.window.size_ms = FEATURE_DURATION_MS; this->frontend_config_.window.step_size_ms = this->features_step_size_; this->frontend_config_.filterbank.num_channels = PREPROCESSOR_FEATURE_SIZE; @@ -130,7 +128,6 @@ void MicroWakeWord::setup() { } }); #endif - ESP_LOGCONFIG(TAG, "Micro Wake Word initialized"); } void MicroWakeWord::inference_task(void *params) { diff --git a/esphome/components/microphone/__init__.py b/esphome/components/microphone/__init__.py index 29bdcfa3f3..1fc0df88a3 100644 --- a/esphome/components/microphone/__init__.py +++ b/esphome/components/microphone/__init__.py @@ -12,7 +12,7 @@ from esphome.const import ( CONF_TRIGGER_ID, ) from esphome.core import CORE -from esphome.coroutine import coroutine_with_priority +from esphome.coroutine import CoroPriority, coroutine_with_priority AUTO_LOAD = ["audio"] CODEOWNERS = ["@jesserockz", "@kahrendt"] @@ -213,7 +213,7 @@ automation.register_condition( )(microphone_action) -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): cg.add_global(microphone_ns.using) cg.add_define("USE_MICROPHONE") diff --git a/esphome/components/mics_4514/mics_4514.cpp b/esphome/components/mics_4514/mics_4514.cpp index 3a2cf22914..8181ece94c 100644 --- a/esphome/components/mics_4514/mics_4514.cpp +++ b/esphome/components/mics_4514/mics_4514.cpp @@ -12,7 +12,6 @@ static const uint8_t SENSOR_REGISTER = 0x04; static const uint8_t POWER_MODE_REGISTER = 0x0a; void MICS4514Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t power_mode; this->read_register(POWER_MODE_REGISTER, &power_mode, 1); if (power_mode == 0x00) { @@ -20,16 +19,12 @@ void MICS4514Component::setup() { power_mode = 0x01; this->write_register(POWER_MODE_REGISTER, &power_mode, 1); delay(100); // NOLINT - this->set_timeout("warmup", 3 * 60 * 1000, [this]() { - this->warmed_up_ = true; - ESP_LOGCONFIG(TAG, "MICS 4514 setup complete."); - }); + this->set_timeout("warmup", 3 * 60 * 1000, [this]() { this->warmed_up_ = true; }); this->status_set_warning(); return; } ESP_LOGCONFIG(TAG, "Device already awake."); this->warmed_up_ = true; - ESP_LOGCONFIG(TAG, "MICS 4514 setup complete."); } void MICS4514Component::dump_config() { ESP_LOGCONFIG(TAG, "MICS 4514:"); diff --git a/esphome/components/midea/ir_transmitter.h b/esphome/components/midea/ir_transmitter.h index eba8fc87f7..a16aed2e72 100644 --- a/esphome/components/midea/ir_transmitter.h +++ b/esphome/components/midea/ir_transmitter.h @@ -54,15 +54,15 @@ class IrFollowMeData : public IrData { void set_fahrenheit(bool val) { this->set_mask_(2, val, 32); } protected: - static const uint8_t MIN_TEMP_C = 0; - static const uint8_t MAX_TEMP_C = 37; + inline static constexpr uint8_t MIN_TEMP_C = 0; + inline static constexpr uint8_t MAX_TEMP_C = 37; // see // https://github.com/crankyoldgit/IRremoteESP8266/blob/9bdf8abcb465268c5409db99dc83a26df64c7445/src/ir_Midea.h#L116 - static const uint8_t MIN_TEMP_F = 32; + inline static constexpr uint8_t MIN_TEMP_F = 32; // see // https://github.com/crankyoldgit/IRremoteESP8266/blob/9bdf8abcb465268c5409db99dc83a26df64c7445/src/ir_Midea.h#L117 - static const uint8_t MAX_TEMP_F = 99; + inline static constexpr uint8_t MAX_TEMP_F = 99; }; class IrSpecialData : public IrData { diff --git a/esphome/components/mipi/__init__.py b/esphome/components/mipi/__init__.py new file mode 100644 index 0000000000..4dff1af62a --- /dev/null +++ b/esphome/components/mipi/__init__.py @@ -0,0 +1,471 @@ +# Various constants used in MIPI DBI communication +# Various configuration constants for MIPI displays +# Various utility functions for MIPI DBI configuration + +from typing import Any, Self + +from esphome.components.const import CONF_COLOR_DEPTH +from esphome.components.display import CONF_SHOW_TEST_CARD, display_ns +import esphome.config_validation as cv +from esphome.const import ( + CONF_BRIGHTNESS, + CONF_COLOR_ORDER, + CONF_DIMENSIONS, + CONF_DISABLED, + CONF_HEIGHT, + CONF_INIT_SEQUENCE, + CONF_INVERT_COLORS, + CONF_LAMBDA, + CONF_MIRROR_X, + CONF_MIRROR_Y, + CONF_OFFSET_HEIGHT, + CONF_OFFSET_WIDTH, + CONF_PAGES, + CONF_ROTATION, + CONF_SWAP_XY, + CONF_TRANSFORM, + CONF_WIDTH, +) +from esphome.core import TimePeriod + +LOGGER = cv.logging.getLogger(__name__) + +ColorOrder = display_ns.enum("ColorMode") + +NOP = 0x00 +SWRESET = 0x01 +RDDID = 0x04 +RDDST = 0x09 +RDMODE = 0x0A +RDMADCTL = 0x0B +RDPIXFMT = 0x0C +RDIMGFMT = 0x0D +RDSELFDIAG = 0x0F +SLEEP_IN = 0x10 +SLPIN = 0x10 +SLEEP_OUT = 0x11 +SLPOUT = 0x11 +PTLON = 0x12 +NORON = 0x13 +INVERT_OFF = 0x20 +INVOFF = 0x20 +INVERT_ON = 0x21 +INVON = 0x21 +ALL_ON = 0x23 +WRAM = 0x24 +GAMMASET = 0x26 +MIPI = 0x26 +DISPOFF = 0x28 +DISPON = 0x29 +CASET = 0x2A +PASET = 0x2B +RASET = 0x2B +RAMWR = 0x2C +WDATA = 0x2C +RAMRD = 0x2E +PTLAR = 0x30 +VSCRDEF = 0x33 +TEON = 0x35 +MADCTL = 0x36 +MADCTL_CMD = 0x36 +VSCRSADD = 0x37 +IDMOFF = 0x38 +IDMON = 0x39 +COLMOD = 0x3A +PIXFMT = 0x3A +GETSCANLINE = 0x45 +BRIGHTNESS = 0x51 +WRDISBV = 0x51 +RDDISBV = 0x52 +WRCTRLD = 0x53 +WCE = 0x58 +SWIRE1 = 0x5A +SWIRE2 = 0x5B +IFMODE = 0xB0 +FRMCTR1 = 0xB1 +FRMCTR2 = 0xB2 +FRMCTR3 = 0xB3 +INVCTR = 0xB4 +DFUNCTR = 0xB6 +ETMOD = 0xB7 +PWCTR1 = 0xC0 +PWCTR2 = 0xC1 +PWCTR3 = 0xC2 +PWCTR4 = 0xC3 +PWCTR5 = 0xC4 +SPIMODESEL = 0xC4 +VMCTR1 = 0xC5 +IFCTR = 0xC6 +VMCTR2 = 0xC7 +GMCTR = 0xC8 +SETEXTC = 0xC8 +PWSET = 0xD0 +VMCTR = 0xD1 +PWSETN = 0xD2 +RDID4 = 0xD3 +RDINDEX = 0xD9 +RDID1 = 0xDA +RDID2 = 0xDB +RDID3 = 0xDC +RDIDX = 0xDD +GMCTRP1 = 0xE0 +GMCTRN1 = 0xE1 +CSCON = 0xF0 +PWCTR6 = 0xF6 +ADJCTL3 = 0xF7 +PAGESEL = 0xFE + +MADCTL_MY = 0x80 # Bit 7 Bottom to top +MADCTL_MX = 0x40 # Bit 6 Right to left +MADCTL_MV = 0x20 # Bit 5 Reverse Mode +MADCTL_ML = 0x10 # Bit 4 LCD refresh Bottom to top +MADCTL_RGB = 0x00 # Bit 3 Red-Green-Blue pixel order +MADCTL_BGR = 0x08 # Bit 3 Blue-Green-Red pixel order +MADCTL_MH = 0x04 # Bit 2 LCD refresh right to left + +# These bits are used instead of the above bits on some chips, where using MX and MY results in incorrect +# partial updates. +MADCTL_XFLIP = 0x02 # Mirror the display horizontally +MADCTL_YFLIP = 0x01 # Mirror the display vertically + +# Special constant for delays in command sequences +DELAY_FLAG = 0xFFF # Special flag to indicate a delay + +CONF_PIXEL_MODE = "pixel_mode" +CONF_USE_AXIS_FLIPS = "use_axis_flips" + +PIXEL_MODE_24BIT = "24bit" +PIXEL_MODE_18BIT = "18bit" +PIXEL_MODE_16BIT = "16bit" + +PIXEL_MODES = { + PIXEL_MODE_16BIT: 0x55, + PIXEL_MODE_18BIT: 0x66, + PIXEL_MODE_24BIT: 0x77, +} + +MODE_RGB = "RGB" +MODE_BGR = "BGR" +COLOR_ORDERS = { + MODE_RGB: ColorOrder.COLOR_ORDER_RGB, + MODE_BGR: ColorOrder.COLOR_ORDER_BGR, +} + +CONF_HSYNC_BACK_PORCH = "hsync_back_porch" +CONF_HSYNC_FRONT_PORCH = "hsync_front_porch" +CONF_HSYNC_PULSE_WIDTH = "hsync_pulse_width" +CONF_VSYNC_BACK_PORCH = "vsync_back_porch" +CONF_VSYNC_FRONT_PORCH = "vsync_front_porch" +CONF_VSYNC_PULSE_WIDTH = "vsync_pulse_width" +CONF_PCLK_FREQUENCY = "pclk_frequency" +CONF_PCLK_INVERTED = "pclk_inverted" +CONF_NATIVE_HEIGHT = "native_height" +CONF_NATIVE_WIDTH = "native_width" + +CONF_DE_PIN = "de_pin" +CONF_PCLK_PIN = "pclk_pin" + + +def power_of_two(value): + value = cv.int_range(1, 128)(value) + if value & (value - 1) != 0: + raise cv.Invalid("value must be a power of two") + return value + + +def validate_dimension(rounding): + def validator(value): + value = cv.positive_int(value) + if value % rounding != 0: + raise cv.Invalid(f"Dimensions and offsets must be divisible by {rounding}") + return value + + return validator + + +def dimension_schema(rounding): + return cv.Any( + cv.dimensions, + cv.Schema( + { + cv.Required(CONF_WIDTH): validate_dimension(rounding), + cv.Required(CONF_HEIGHT): validate_dimension(rounding), + cv.Optional(CONF_OFFSET_HEIGHT, default=0): validate_dimension( + rounding + ), + cv.Optional(CONF_OFFSET_WIDTH, default=0): validate_dimension(rounding), + } + ), + ) + + +def map_sequence(value): + """ + Maps one entry in a sequence to a command and data bytes. + The format is a repeated sequence of [CMD, ] where is s a sequence of bytes. The length is inferred + from the length of the sequence and should not be explicit. + A single integer can be provided where there are no data bytes, in which case it is treated as a command. + A delay can be inserted by specifying "- delay N" where N is in ms + """ + if isinstance(value, str) and value.lower().startswith("delay "): + value = value.lower()[6:] + delay_value = cv.All( + cv.positive_time_period_milliseconds, + cv.Range(TimePeriod(milliseconds=1), TimePeriod(milliseconds=255)), + )(value) + return DELAY_FLAG, delay_value.total_milliseconds + value = cv.All(cv.ensure_list(cv.int_range(0, 255)), cv.Length(1, 254))(value) + return tuple(value) + + +def delay(ms): + return DELAY_FLAG, ms + + +class DriverChip: + """ + A class representing a MIPI DBI driver chip model. + The parameters supplied as defaults will be used to provide default values for the display configuration. + Setting swap_xy to cv.UNDEFINED will indicate that the model does not support swapping X and Y axes. + """ + + models: dict[str, Self] = {} + + def __init__( + self, + name: str, + initsequence=None, + **defaults, + ): + name = name.upper() + self.name = name + self.initsequence = initsequence + self.defaults = defaults + DriverChip.models[name] = self + + @classmethod + def get_models(cls): + """ + Return the current set of models and reset the models dictionary. + """ + models = cls.models + cls.models = {} + return models + + def extend(self, name, **kwargs) -> "DriverChip": + """ + Extend the current model with additional parameters or a modified init sequence. + Parameters supplied here will override the defaults of the current model. + if the initsequence is not provided, the current model's initsequence will be used. + If add_init_sequence is provided, it will be appended to the current initsequence. + :param name: + :param kwargs: + :return: + """ + initsequence = list(kwargs.pop("initsequence", self.initsequence)) + initsequence.extend(kwargs.pop("add_init_sequence", ())) + defaults = self.defaults.copy() + if ( + CONF_WIDTH in defaults + and CONF_OFFSET_WIDTH in kwargs + and CONF_NATIVE_WIDTH not in defaults + ): + defaults[CONF_NATIVE_WIDTH] = defaults[CONF_WIDTH] + if ( + CONF_HEIGHT in defaults + and CONF_OFFSET_HEIGHT in kwargs + and CONF_NATIVE_HEIGHT not in defaults + ): + defaults[CONF_NATIVE_HEIGHT] = defaults[CONF_HEIGHT] + defaults.update(kwargs) + return self.__class__(name, initsequence=tuple(initsequence), **defaults) + + def get_default(self, key, fallback: Any = False) -> Any: + return self.defaults.get(key, fallback) + + @property + def transforms(self) -> set[str]: + """ + Return the available transforms for this model. + """ + if self.get_default("no_transform", False): + return set() + if self.get_default(CONF_SWAP_XY) != cv.UNDEFINED: + return {CONF_MIRROR_X, CONF_MIRROR_Y, CONF_SWAP_XY} + return {CONF_MIRROR_X, CONF_MIRROR_Y} + + def option(self, name, fallback=False) -> cv.Optional: + return cv.Optional(name, default=self.get_default(name, fallback)) + + def rotation_as_transform(self, config) -> bool: + """ + Check if a rotation can be implemented in hardware using the MADCTL register. + A rotation of 180 is always possible if x and y mirroring are supported, 90 and 270 are possible if the model supports swapping X and Y. + """ + if config.get(CONF_TRANSFORM) == CONF_DISABLED: + return False + transforms = self.transforms + rotation = config.get(CONF_ROTATION, 0) + if rotation == 0 or not transforms: + return False + if rotation == 180: + return CONF_MIRROR_X in transforms and CONF_MIRROR_Y in transforms + if rotation == 90: + return CONF_SWAP_XY in transforms and CONF_MIRROR_X in transforms + return CONF_SWAP_XY in transforms and CONF_MIRROR_Y in transforms + + def get_dimensions(self, config) -> tuple[int, int, int, int]: + if CONF_DIMENSIONS in config: + # Explicit dimensions, just use as is + dimensions = config[CONF_DIMENSIONS] + if isinstance(dimensions, dict): + width = dimensions[CONF_WIDTH] + height = dimensions[CONF_HEIGHT] + offset_width = dimensions[CONF_OFFSET_WIDTH] + offset_height = dimensions[CONF_OFFSET_HEIGHT] + return width, height, offset_width, offset_height + (width, height) = dimensions + return width, height, 0, 0 + + # Default dimensions, use model defaults + transform = self.get_transform(config) + + width = self.get_default(CONF_WIDTH) + height = self.get_default(CONF_HEIGHT) + offset_width = self.get_default(CONF_OFFSET_WIDTH, 0) + offset_height = self.get_default(CONF_OFFSET_HEIGHT, 0) + + # if mirroring axes and there are offsets, also mirror the offsets to cater for situations where + # the offset is asymmetric + if transform.get(CONF_MIRROR_X): + native_width = self.get_default(CONF_NATIVE_WIDTH, width + offset_width * 2) + offset_width = native_width - width - offset_width + if transform.get(CONF_MIRROR_Y): + native_height = self.get_default( + CONF_NATIVE_HEIGHT, height + offset_height * 2 + ) + offset_height = native_height - height - offset_height + # Swap default dimensions if swap_xy is set, or if rotation is 90/270 and we are not using a buffer + if transform.get(CONF_SWAP_XY) is True: + width, height = height, width + offset_height, offset_width = offset_width, offset_height + return width, height, offset_width, offset_height + + def get_transform(self, config) -> dict[str, bool]: + can_transform = self.rotation_as_transform(config) + transform = config.get( + CONF_TRANSFORM, + { + CONF_MIRROR_X: self.get_default(CONF_MIRROR_X), + CONF_MIRROR_Y: self.get_default(CONF_MIRROR_Y), + CONF_SWAP_XY: self.get_default(CONF_SWAP_XY), + }, + ) + if not isinstance(transform, dict): + # Presumably disabled + return { + CONF_MIRROR_X: False, + CONF_MIRROR_Y: False, + CONF_SWAP_XY: False, + CONF_TRANSFORM: False, + } + # Can we use the MADCTL register to set the rotation? + if can_transform and CONF_TRANSFORM not in config: + rotation = config[CONF_ROTATION] + if rotation == 180: + transform[CONF_MIRROR_X] = not transform[CONF_MIRROR_X] + transform[CONF_MIRROR_Y] = not transform[CONF_MIRROR_Y] + elif rotation == 90: + transform[CONF_SWAP_XY] = not transform[CONF_SWAP_XY] + transform[CONF_MIRROR_X] = not transform[CONF_MIRROR_X] + else: + transform[CONF_SWAP_XY] = not transform[CONF_SWAP_XY] + transform[CONF_MIRROR_Y] = not transform[CONF_MIRROR_Y] + transform[CONF_TRANSFORM] = True + return transform + + def add_madctl(self, sequence: list, config: dict): + # Add the MADCTL command to the sequence based on the configuration. + use_flip = config.get(CONF_USE_AXIS_FLIPS) + madctl = 0 + transform = self.get_transform(config) + if transform[CONF_MIRROR_X]: + madctl |= MADCTL_XFLIP if use_flip else MADCTL_MX + if transform[CONF_MIRROR_Y]: + madctl |= MADCTL_YFLIP if use_flip else MADCTL_MY + if transform.get(CONF_SWAP_XY) is True: # Exclude Undefined + madctl |= MADCTL_MV + if config[CONF_COLOR_ORDER] == MODE_BGR: + madctl |= MADCTL_BGR + sequence.append((MADCTL, madctl)) + return madctl + + def skip_command(self, command: str): + """ + Allow suppressing a standard command in the init sequence. + """ + return self.get_default(f"no_{command.lower()}", False) + + def get_sequence(self, config) -> tuple[tuple[int, ...], int]: + """ + Create the init sequence for the display. + Use the default sequence from the model, if any, and append any custom sequence provided in the config. + Append SLPOUT (if not already in the sequence) and DISPON to the end of the sequence + Pixel format, color order, and orientation will be set. + Returns a tuple of the init sequence and the computed MADCTL value. + """ + sequence = list(self.initsequence or ()) + custom_sequence = config.get(CONF_INIT_SEQUENCE, []) + sequence.extend(custom_sequence) + # Ensure each command is a tuple + sequence = [x if isinstance(x, tuple) else (x,) for x in sequence] + + # Set pixel format if not already in the custom sequence + pixel_mode = config[CONF_PIXEL_MODE] + if not isinstance(pixel_mode, int): + if not pixel_mode.endswith("bit"): + pixel_mode = f"{pixel_mode}bit" + pixel_mode = PIXEL_MODES[pixel_mode] + sequence.append((PIXFMT, pixel_mode)) + + if self.rotation_as_transform(config): + LOGGER.info("Using hardware transform to implement rotation") + madctl = self.add_madctl(sequence, config) + if config[CONF_INVERT_COLORS]: + sequence.append((INVON,)) + else: + sequence.append((INVOFF,)) + if brightness := config.get(CONF_BRIGHTNESS, self.get_default(CONF_BRIGHTNESS)): + sequence.append((BRIGHTNESS, brightness)) + # Add a SLPOUT command if required. + if not self.skip_command("SLPOUT"): + sequence.append((SLPOUT,)) + sequence.append((DISPON,)) + + # Flatten the sequence into a list of bytes, with the length of each command + # or the delay flag inserted where needed + return sum( + tuple( + (x[1], 0xFF) if x[0] == DELAY_FLAG else (x[0], len(x) - 1) + x[1:] + for x in sequence + ), + (), + ), madctl + + +def requires_buffer(config) -> bool: + """ + Check if the display configuration requires a buffer. It will do so if any drawing methods are configured. + :param config: + :return: True if a buffer is required, False otherwise + """ + return any( + config.get(key) for key in (CONF_LAMBDA, CONF_PAGES, CONF_SHOW_TEST_CARD) + ) + + +def get_color_depth(config) -> int: + """ + Get the color depth in bits from the configuration. + """ + return int(config[CONF_COLOR_DEPTH].removesuffix("bit")) diff --git a/esphome/components/mipi_dsi/__init__.py b/esphome/components/mipi_dsi/__init__.py new file mode 100644 index 0000000000..84dd66e2f4 --- /dev/null +++ b/esphome/components/mipi_dsi/__init__.py @@ -0,0 +1,5 @@ +import esphome.codegen as cg + +CODEOWNERS = ["@clydebarrow"] + +mipi_dsi_ns = cg.esphome_ns.namespace("mipi_dsi") diff --git a/esphome/components/mipi_dsi/display.py b/esphome/components/mipi_dsi/display.py new file mode 100644 index 0000000000..4fc837be67 --- /dev/null +++ b/esphome/components/mipi_dsi/display.py @@ -0,0 +1,233 @@ +import importlib +import logging +import pkgutil + +from esphome import pins +import esphome.codegen as cg +from esphome.components import display +from esphome.components.const import ( + BYTE_ORDER_BIG, + BYTE_ORDER_LITTLE, + CONF_BYTE_ORDER, + CONF_DRAW_ROUNDING, +) +from esphome.components.display import CONF_SHOW_TEST_CARD +from esphome.components.esp32 import const, only_on_variant +from esphome.components.mipi import ( + COLOR_ORDERS, + CONF_COLOR_DEPTH, + CONF_HSYNC_BACK_PORCH, + CONF_HSYNC_FRONT_PORCH, + CONF_HSYNC_PULSE_WIDTH, + CONF_PCLK_FREQUENCY, + CONF_PIXEL_MODE, + CONF_USE_AXIS_FLIPS, + CONF_VSYNC_BACK_PORCH, + CONF_VSYNC_FRONT_PORCH, + CONF_VSYNC_PULSE_WIDTH, + MODE_BGR, + PIXEL_MODE_16BIT, + PIXEL_MODE_24BIT, + DriverChip, + dimension_schema, + get_color_depth, + map_sequence, + power_of_two, + requires_buffer, +) +import esphome.config_validation as cv +from esphome.const import ( + CONF_COLOR_ORDER, + CONF_DIMENSIONS, + CONF_ENABLE_PIN, + CONF_ID, + CONF_INIT_SEQUENCE, + CONF_INVERT_COLORS, + CONF_LAMBDA, + CONF_MIRROR_X, + CONF_MIRROR_Y, + CONF_MODEL, + CONF_RESET_PIN, + CONF_ROTATION, + CONF_SWAP_XY, + CONF_TRANSFORM, + CONF_WIDTH, +) +from esphome.final_validate import full_config + +from . import mipi_dsi_ns, models + +# Currently only ESP32-P4 is supported, so esp_ldo and psram are required +DEPENDENCIES = ["esp32", "esp_ldo", "psram"] +DOMAIN = "mipi_dsi" + +LOGGER = logging.getLogger(DOMAIN) + +MIPI_DSI = mipi_dsi_ns.class_("MIPI_DSI", display.Display, cg.Component) +ColorOrder = display.display_ns.enum("ColorMode") +ColorBitness = display.display_ns.enum("ColorBitness") + +CONF_LANE_BIT_RATE = "lane_bit_rate" +CONF_LANES = "lanes" + +DriverChip("CUSTOM") + +# Import all models dynamically from the models package + +for module_info in pkgutil.iter_modules(models.__path__): + importlib.import_module(f".models.{module_info.name}", package=__package__) + +MODELS = DriverChip.get_models() + +COLOR_DEPTHS = { + 16: ColorBitness.COLOR_BITNESS_565, + 24: ColorBitness.COLOR_BITNESS_888, +} + + +def model_schema(config): + model = MODELS[config[CONF_MODEL].upper()] + transform = cv.Schema( + { + cv.Required(CONF_MIRROR_X): cv.boolean, + cv.Required(CONF_MIRROR_Y): cv.boolean, + } + ) + if model.get_default(CONF_SWAP_XY) != cv.UNDEFINED: + transform = transform.extend( + { + cv.Optional(CONF_SWAP_XY): cv.invalid( + "Axis swapping not supported by this model" + ) + } + ) + else: + transform = transform.extend( + { + cv.Required(CONF_SWAP_XY): cv.boolean, + } + ) + # CUSTOM model will need to provide a custom init sequence + iseqconf = ( + cv.Required(CONF_INIT_SEQUENCE) + if model.initsequence is None + else cv.Optional(CONF_INIT_SEQUENCE) + ) + swap_xy = config.get(CONF_TRANSFORM, {}).get(CONF_SWAP_XY, False) + + # Dimensions are optional if the model has a default width and the swap_xy transform is not overridden + cv_dimensions = ( + cv.Optional if model.get_default(CONF_WIDTH) and not swap_xy else cv.Required + ) + pixel_modes = (PIXEL_MODE_16BIT, PIXEL_MODE_24BIT, "16", "24") + schema = display.FULL_DISPLAY_SCHEMA.extend( + { + model.option(CONF_RESET_PIN, cv.UNDEFINED): pins.gpio_output_pin_schema, + cv.GenerateID(): cv.declare_id(MIPI_DSI), + cv_dimensions(CONF_DIMENSIONS): dimension_schema( + model.get_default(CONF_DRAW_ROUNDING, 1) + ), + model.option(CONF_ENABLE_PIN, cv.UNDEFINED): cv.ensure_list( + pins.gpio_output_pin_schema + ), + model.option(CONF_COLOR_ORDER, MODE_BGR): cv.enum(COLOR_ORDERS, upper=True), + model.option(CONF_DRAW_ROUNDING, 2): power_of_two, + model.option(CONF_PIXEL_MODE, PIXEL_MODE_16BIT): cv.one_of( + *pixel_modes, lower=True + ), + model.option(CONF_TRANSFORM, cv.UNDEFINED): transform, + cv.Required(CONF_MODEL): cv.one_of(model.name, upper=True), + model.option(CONF_INVERT_COLORS, False): cv.boolean, + model.option(CONF_COLOR_DEPTH, "16"): cv.one_of( + *[str(d) for d in COLOR_DEPTHS], + *[f"{d}bit" for d in COLOR_DEPTHS], + lower=True, + ), + model.option(CONF_USE_AXIS_FLIPS, True): cv.boolean, + model.option(CONF_PCLK_FREQUENCY, "40MHz"): cv.All( + cv.frequency, cv.Range(min=4e6, max=100e6) + ), + model.option(CONF_LANES, 2): cv.int_range(1, 2), + model.option(CONF_LANE_BIT_RATE, None): cv.All( + cv.bps, cv.Range(min=100e6, max=3200e6) + ), + iseqconf: cv.ensure_list(map_sequence), + model.option(CONF_BYTE_ORDER, BYTE_ORDER_LITTLE): cv.one_of( + BYTE_ORDER_LITTLE, BYTE_ORDER_BIG, lower=True + ), + model.option(CONF_HSYNC_PULSE_WIDTH): cv.int_, + model.option(CONF_HSYNC_BACK_PORCH): cv.int_, + model.option(CONF_HSYNC_FRONT_PORCH): cv.int_, + model.option(CONF_VSYNC_PULSE_WIDTH): cv.int_, + model.option(CONF_VSYNC_BACK_PORCH): cv.int_, + model.option(CONF_VSYNC_FRONT_PORCH): cv.int_, + } + ) + return cv.All( + schema, + only_on_variant(supported=[const.VARIANT_ESP32P4]), + cv.only_with_esp_idf, + ) + + +def _config_schema(config): + config = cv.Schema( + { + cv.Required(CONF_MODEL): cv.one_of(*MODELS, upper=True), + }, + extra=cv.ALLOW_EXTRA, + )(config) + return model_schema(config)(config) + + +def _final_validate(config): + global_config = full_config.get() + + from esphome.components.lvgl import DOMAIN as LVGL_DOMAIN + + if not requires_buffer(config) and LVGL_DOMAIN not in global_config: + # If no drawing methods are configured, and LVGL is not enabled, show a test card + config[CONF_SHOW_TEST_CARD] = True + return config + + +CONFIG_SCHEMA = _config_schema +FINAL_VALIDATE_SCHEMA = _final_validate + + +async def to_code(config): + model = MODELS[config[CONF_MODEL].upper()] + color_depth = COLOR_DEPTHS[get_color_depth(config)] + pixel_mode = int(config[CONF_PIXEL_MODE].removesuffix("bit")) + width, height, _offset_width, _offset_height = model.get_dimensions(config) + var = cg.new_Pvariable(config[CONF_ID], width, height, color_depth, pixel_mode) + + sequence, madctl = model.get_sequence(config) + cg.add(var.set_model(config[CONF_MODEL])) + cg.add(var.set_init_sequence(sequence)) + cg.add(var.set_madctl(madctl)) + cg.add(var.set_invert_colors(config[CONF_INVERT_COLORS])) + cg.add(var.set_hsync_pulse_width(config[CONF_HSYNC_PULSE_WIDTH])) + cg.add(var.set_hsync_back_porch(config[CONF_HSYNC_BACK_PORCH])) + cg.add(var.set_hsync_front_porch(config[CONF_HSYNC_FRONT_PORCH])) + cg.add(var.set_vsync_pulse_width(config[CONF_VSYNC_PULSE_WIDTH])) + cg.add(var.set_vsync_back_porch(config[CONF_VSYNC_BACK_PORCH])) + cg.add(var.set_vsync_front_porch(config[CONF_VSYNC_FRONT_PORCH])) + cg.add(var.set_pclk_frequency(int(config[CONF_PCLK_FREQUENCY] / 1e6))) + cg.add(var.set_lanes(int(config[CONF_LANES]))) + cg.add(var.set_lane_bit_rate(int(config[CONF_LANE_BIT_RATE] / 1e6))) + if reset_pin := config.get(CONF_RESET_PIN): + reset = await cg.gpio_pin_expression(reset_pin) + cg.add(var.set_reset_pin(reset)) + if enable_pin := config.get(CONF_ENABLE_PIN): + enable = [await cg.gpio_pin_expression(pin) for pin in enable_pin] + cg.add(var.set_enable_pins(enable)) + + if model.rotation_as_transform(config): + config[CONF_ROTATION] = 0 + await display.register_display(var, config) + if lamb := config.get(CONF_LAMBDA): + lambda_ = await cg.process_lambda( + lamb, [(display.DisplayRef, "it")], return_type=cg.void + ) + cg.add(var.set_writer(lambda_)) diff --git a/esphome/components/mipi_dsi/mipi_dsi.cpp b/esphome/components/mipi_dsi/mipi_dsi.cpp new file mode 100644 index 0000000000..fbe251de41 --- /dev/null +++ b/esphome/components/mipi_dsi/mipi_dsi.cpp @@ -0,0 +1,379 @@ +#ifdef USE_ESP32_VARIANT_ESP32P4 +#include +#include "mipi_dsi.h" + +namespace esphome { +namespace mipi_dsi { + +static bool notify_refresh_ready(esp_lcd_panel_handle_t panel, esp_lcd_dpi_panel_event_data_t *edata, void *user_ctx) { + auto *sem = static_cast(user_ctx); + BaseType_t need_yield = pdFALSE; + xSemaphoreGiveFromISR(sem, &need_yield); + return (need_yield == pdTRUE); +} +void MIPI_DSI::setup() { + ESP_LOGCONFIG(TAG, "Running Setup"); + + if (!this->enable_pins_.empty()) { + for (auto *pin : this->enable_pins_) { + pin->setup(); + pin->digital_write(true); + } + delay(10); + } + + esp_lcd_dsi_bus_config_t bus_config = { + .bus_id = 0, // index from 0, specify the DSI host to use + .num_data_lanes = + this->lanes_, // Number of data lanes to use, can't set a value that exceeds the chip's capability + .phy_clk_src = MIPI_DSI_PHY_CLK_SRC_DEFAULT, // Clock source for the DPHY + .lane_bit_rate_mbps = this->lane_bit_rate_, // Bit rate of the data lanes, in Mbps + }; + auto err = esp_lcd_new_dsi_bus(&bus_config, &this->bus_handle_); + if (err != ESP_OK) { + this->smark_failed("lcd_new_dsi_bus failed", err); + return; + } + esp_lcd_dbi_io_config_t dbi_config = { + .virtual_channel = 0, + .lcd_cmd_bits = 8, // according to the LCD spec + .lcd_param_bits = 8, // according to the LCD spec + }; + err = esp_lcd_new_panel_io_dbi(this->bus_handle_, &dbi_config, &this->io_handle_); + if (err != ESP_OK) { + this->smark_failed("new_panel_io_dbi failed", err); + return; + } + auto pixel_format = LCD_COLOR_PIXEL_FORMAT_RGB565; + if (this->color_depth_ == display::COLOR_BITNESS_888) { + pixel_format = LCD_COLOR_PIXEL_FORMAT_RGB888; + } + esp_lcd_dpi_panel_config_t dpi_config = {.virtual_channel = 0, + .dpi_clk_src = MIPI_DSI_DPI_CLK_SRC_DEFAULT, + .dpi_clock_freq_mhz = this->pclk_frequency_, + .pixel_format = pixel_format, + .num_fbs = 1, // number of frame buffers to allocate + .video_timing = + { + .h_size = this->width_, + .v_size = this->height_, + .hsync_pulse_width = this->hsync_pulse_width_, + .hsync_back_porch = this->hsync_back_porch_, + .hsync_front_porch = this->hsync_front_porch_, + .vsync_pulse_width = this->vsync_pulse_width_, + .vsync_back_porch = this->vsync_back_porch_, + .vsync_front_porch = this->vsync_front_porch_, + }, + .flags = { + .use_dma2d = true, + }}; + err = esp_lcd_new_panel_dpi(this->bus_handle_, &dpi_config, &this->handle_); + if (err != ESP_OK) { + this->smark_failed("esp_lcd_new_panel_dpi failed", err); + return; + } + if (this->reset_pin_ != nullptr) { + this->reset_pin_->setup(); + this->reset_pin_->digital_write(true); + delay(5); + this->reset_pin_->digital_write(false); + delay(5); + this->reset_pin_->digital_write(true); + } else { + esp_lcd_panel_io_tx_param(this->io_handle_, SW_RESET_CMD, nullptr, 0); + } + // need to know when the display is ready for SLPOUT command - will be 120ms after reset + auto when = millis() + 120; + err = esp_lcd_panel_init(this->handle_); + if (err != ESP_OK) { + this->smark_failed("esp_lcd_init failed", err); + return; + } + size_t index = 0; + auto &vec = this->init_sequence_; + while (index != vec.size()) { + if (vec.size() - index < 2) { + this->mark_failed("Malformed init sequence"); + return; + } + uint8_t cmd = vec[index++]; + uint8_t x = vec[index++]; + if (x == DELAY_FLAG) { + ESP_LOGD(TAG, "Delay %dms", cmd); + delay(cmd); + } else { + uint8_t num_args = x & 0x7F; + if (vec.size() - index < num_args) { + this->mark_failed("Malformed init sequence"); + return; + } + if (cmd == SLEEP_OUT) { + // are we ready, boots? + int duration = when - millis(); + if (duration > 0) { + delay(duration); + } + } + const auto *ptr = vec.data() + index; + ESP_LOGVV(TAG, "Command %02X, length %d, byte(s) %s", cmd, num_args, + format_hex_pretty(ptr, num_args, '.', false).c_str()); + err = esp_lcd_panel_io_tx_param(this->io_handle_, cmd, ptr, num_args); + if (err != ESP_OK) { + this->smark_failed("lcd_panel_io_tx_param failed", err); + return; + } + index += num_args; + if (cmd == SLEEP_OUT) + delay(10); + } + } + this->io_lock_ = xSemaphoreCreateBinary(); + esp_lcd_dpi_panel_event_callbacks_t cbs = { + .on_color_trans_done = notify_refresh_ready, + }; + + err = (esp_lcd_dpi_panel_register_event_callbacks(this->handle_, &cbs, this->io_lock_)); + if (err != ESP_OK) { + this->smark_failed("Failed to register callbacks", err); + return; + } + + ESP_LOGCONFIG(TAG, "MIPI DSI setup complete"); +} + +void MIPI_DSI::update() { + if (this->auto_clear_enabled_) { + this->clear(); + } + if (this->show_test_card_) { + this->test_card(); + } else if (this->page_ != nullptr) { + this->page_->get_writer()(*this); + } else if (this->writer_.has_value()) { + (*this->writer_)(*this); + } else { + this->stop_poller(); + } + if (this->buffer_ == nullptr || this->x_low_ > this->x_high_ || this->y_low_ > this->y_high_) + return; + ESP_LOGV(TAG, "x_low %d, y_low %d, x_high %d, y_high %d", this->x_low_, this->y_low_, this->x_high_, this->y_high_); + int w = this->x_high_ - this->x_low_ + 1; + int h = this->y_high_ - this->y_low_ + 1; + this->write_to_display_(this->x_low_, this->y_low_, w, h, this->buffer_, this->x_low_, this->y_low_, + this->width_ - w - this->x_low_); + // invalidate watermarks + this->x_low_ = this->width_; + this->y_low_ = this->height_; + this->x_high_ = 0; + this->y_high_ = 0; +} + +void MIPI_DSI::draw_pixels_at(int x_start, int y_start, int w, int h, const uint8_t *ptr, display::ColorOrder order, + display::ColorBitness bitness, bool big_endian, int x_offset, int y_offset, int x_pad) { + if (w <= 0 || h <= 0) + return; + // if color mapping is required, pass the buck. + // note that endianness is not considered here - it is assumed to match! + if (bitness != this->color_depth_) { + display::Display::draw_pixels_at(x_start, y_start, w, h, ptr, order, bitness, big_endian, x_offset, y_offset, + x_pad); + } + this->write_to_display_(x_start, y_start, w, h, ptr, x_offset, y_offset, x_pad); +} + +void MIPI_DSI::write_to_display_(int x_start, int y_start, int w, int h, const uint8_t *ptr, int x_offset, int y_offset, + int x_pad) { + esp_err_t err = ESP_OK; + auto bytes_per_pixel = 3 - this->color_depth_; + auto stride = (x_offset + w + x_pad) * bytes_per_pixel; + ptr += y_offset * stride + x_offset * bytes_per_pixel; // skip to the first pixel + // x_ and y_offset are offsets into the source buffer, unrelated to our own offsets into the display. + if (x_offset == 0 && x_pad == 0) { + err = esp_lcd_panel_draw_bitmap(this->handle_, x_start, y_start, x_start + w, y_start + h, ptr); + xSemaphoreTake(this->io_lock_, portMAX_DELAY); + + } else { + // draw line by line + for (int y = 0; y != h; y++) { + err = esp_lcd_panel_draw_bitmap(this->handle_, x_start, y + y_start, x_start + w, y + y_start + 1, ptr); + if (err != ESP_OK) + break; + ptr += stride; // next line + xSemaphoreTake(this->io_lock_, portMAX_DELAY); + } + } + if (err != ESP_OK) + ESP_LOGE(TAG, "lcd_lcd_panel_draw_bitmap failed: %s", esp_err_to_name(err)); +} + +bool MIPI_DSI::check_buffer_() { + if (this->is_failed()) + return false; + if (this->buffer_ != nullptr) + return true; + // this is dependent on the enum values. + auto bytes_per_pixel = 3 - this->color_depth_; + RAMAllocator allocator; + this->buffer_ = allocator.allocate(this->height_ * this->width_ * bytes_per_pixel); + if (this->buffer_ == nullptr) { + this->mark_failed("Could not allocate buffer for display!"); + return false; + } + return true; +} + +void MIPI_DSI::draw_pixel_at(int x, int y, Color color) { + if (!this->get_clipping().inside(x, y)) + return; + + switch (this->rotation_) { + case display::DISPLAY_ROTATION_0_DEGREES: + break; + case display::DISPLAY_ROTATION_90_DEGREES: + std::swap(x, y); + x = this->width_ - x - 1; + break; + case display::DISPLAY_ROTATION_180_DEGREES: + x = this->width_ - x - 1; + y = this->height_ - y - 1; + break; + case display::DISPLAY_ROTATION_270_DEGREES: + std::swap(x, y); + y = this->height_ - y - 1; + break; + } + if (x >= this->get_width_internal() || x < 0 || y >= this->get_height_internal() || y < 0) { + return; + } + auto pixel = convert_big_endian(display::ColorUtil::color_to_565(color)); + if (!this->check_buffer_()) + return; + size_t pos = (y * this->width_) + x; + switch (this->color_depth_) { + case display::COLOR_BITNESS_565: { + auto *ptr_16 = reinterpret_cast(this->buffer_); + uint8_t hi_byte = static_cast(color.r & 0xF8) | (color.g >> 5); + uint8_t lo_byte = static_cast((color.g & 0x1C) << 3) | (color.b >> 3); + uint16_t new_color = lo_byte | (hi_byte << 8); // little endian + if (ptr_16[pos] == new_color) + return; + ptr_16[pos] = new_color; + break; + } + case display::COLOR_BITNESS_888: + if (this->color_mode_ == display::COLOR_ORDER_BGR) { + this->buffer_[pos * 3] = color.b; + this->buffer_[pos * 3 + 1] = color.g; + this->buffer_[pos * 3 + 2] = color.r; + } else { + this->buffer_[pos * 3] = color.r; + this->buffer_[pos * 3 + 1] = color.g; + this->buffer_[pos * 3 + 2] = color.b; + } + break; + case display::COLOR_BITNESS_332: + break; + } + // low and high watermark may speed up drawing from buffer + if (x < this->x_low_) + this->x_low_ = x; + if (y < this->y_low_) + this->y_low_ = y; + if (x > this->x_high_) + this->x_high_ = x; + if (y > this->y_high_) + this->y_high_ = y; +} +void MIPI_DSI::fill(Color color) { + if (!this->check_buffer_()) + return; + switch (this->color_depth_) { + case display::COLOR_BITNESS_565: { + auto *ptr_16 = reinterpret_cast(this->buffer_); + uint8_t hi_byte = static_cast(color.r & 0xF8) | (color.g >> 5); + uint8_t lo_byte = static_cast((color.g & 0x1C) << 3) | (color.b >> 3); + uint16_t new_color = lo_byte | (hi_byte << 8); // little endian + std::fill_n(ptr_16, this->width_ * this->height_, new_color); + break; + } + + case display::COLOR_BITNESS_888: + if (this->color_mode_ == display::COLOR_ORDER_BGR) { + for (size_t i = 0; i != this->width_ * this->height_; i++) { + this->buffer_[i * 3 + 0] = color.b; + this->buffer_[i * 3 + 1] = color.g; + this->buffer_[i * 3 + 2] = color.r; + } + } else { + for (size_t i = 0; i != this->width_ * this->height_; i++) { + this->buffer_[i * 3 + 0] = color.r; + this->buffer_[i * 3 + 1] = color.g; + this->buffer_[i * 3 + 2] = color.b; + } + } + + default: + break; + } +} + +int MIPI_DSI::get_width() { + switch (this->rotation_) { + case display::DISPLAY_ROTATION_90_DEGREES: + case display::DISPLAY_ROTATION_270_DEGREES: + return this->get_height_internal(); + case display::DISPLAY_ROTATION_0_DEGREES: + case display::DISPLAY_ROTATION_180_DEGREES: + default: + return this->get_width_internal(); + } +} + +int MIPI_DSI::get_height() { + switch (this->rotation_) { + case display::DISPLAY_ROTATION_0_DEGREES: + case display::DISPLAY_ROTATION_180_DEGREES: + return this->get_height_internal(); + case display::DISPLAY_ROTATION_90_DEGREES: + case display::DISPLAY_ROTATION_270_DEGREES: + default: + return this->get_width_internal(); + } +} + +static const uint8_t PIXEL_MODES[] = {0, 16, 18, 24}; + +void MIPI_DSI::dump_config() { + ESP_LOGCONFIG(TAG, + "MIPI_DSI RGB LCD" + "\n Model: %s" + "\n Width: %u" + "\n Height: %u" + "\n Mirror X: %s" + "\n Mirror Y: %s" + "\n Swap X/Y: %s" + "\n Rotation: %d degrees" + "\n DSI Lanes: %u" + "\n Lane Bit Rate: %uMbps" + "\n HSync Pulse Width: %u" + "\n HSync Back Porch: %u" + "\n HSync Front Porch: %u" + "\n VSync Pulse Width: %u" + "\n VSync Back Porch: %u" + "\n VSync Front Porch: %u" + "\n Buffer Color Depth: %d bit" + "\n Display Pixel Mode: %d bit" + "\n Color Order: %s" + "\n Invert Colors: %s" + "\n Pixel Clock: %dMHz", + this->model_, this->width_, this->height_, YESNO(this->madctl_ & (MADCTL_XFLIP | MADCTL_MX)), + YESNO(this->madctl_ & (MADCTL_YFLIP | MADCTL_MY)), YESNO(this->madctl_ & MADCTL_MV), this->rotation_, + this->lanes_, this->lane_bit_rate_, this->hsync_pulse_width_, this->hsync_back_porch_, + this->hsync_front_porch_, this->vsync_pulse_width_, this->vsync_back_porch_, this->vsync_front_porch_, + (3 - this->color_depth_) * 8, this->pixel_mode_, this->madctl_ & MADCTL_BGR ? "BGR" : "RGB", + YESNO(this->invert_colors_), this->pclk_frequency_); + LOG_PIN(" Reset Pin ", this->reset_pin_); +} +} // namespace mipi_dsi +} // namespace esphome +#endif // USE_ESP32_VARIANT_ESP32P4 diff --git a/esphome/components/mipi_dsi/mipi_dsi.h b/esphome/components/mipi_dsi/mipi_dsi.h new file mode 100644 index 0000000000..ce8a2a2236 --- /dev/null +++ b/esphome/components/mipi_dsi/mipi_dsi.h @@ -0,0 +1,123 @@ +// +// Created by Clyde Stubbs on 29/10/2023. +// +#pragma once + +// only applicable on ESP32-P4 +#ifdef USE_ESP32_VARIANT_ESP32P4 +#include "esphome/core/component.h" +#include "esphome/core/application.h" +#include "esphome/core/log.h" +#include "esphome/core/gpio.h" + +#include "esphome/components/display/display.h" +#include "esp_lcd_panel_ops.h" +#include "esp_lcd_panel_io.h" + +#include "esp_lcd_mipi_dsi.h" + +namespace esphome { +namespace mipi_dsi { + +constexpr static const char *const TAG = "display.mipi_dsi"; +const uint8_t SW_RESET_CMD = 0x01; +const uint8_t SLEEP_OUT = 0x11; +const uint8_t SDIR_CMD = 0xC7; +const uint8_t MADCTL_CMD = 0x36; +const uint8_t INVERT_OFF = 0x20; +const uint8_t INVERT_ON = 0x21; +const uint8_t DISPLAY_ON = 0x29; +const uint8_t CMD2_BKSEL = 0xFF; +const uint8_t DELAY_FLAG = 0xFF; +const uint8_t MADCTL_BGR = 0x08; +const uint8_t MADCTL_MX = 0x40; +const uint8_t MADCTL_MY = 0x80; +const uint8_t MADCTL_MV = 0x20; // row/column swap +const uint8_t MADCTL_XFLIP = 0x02; // Mirror the display horizontally +const uint8_t MADCTL_YFLIP = 0x01; // Mirror the display vertically + +class MIPI_DSI : public display::Display { + public: + MIPI_DSI(size_t width, size_t height, display::ColorBitness color_depth, uint8_t pixel_mode) + : width_(width), height_(height), color_depth_(color_depth), pixel_mode_(pixel_mode) {} + display::ColorOrder get_color_mode() { return this->color_mode_; } + void set_color_mode(display::ColorOrder color_mode) { this->color_mode_ = color_mode; } + void set_invert_colors(bool invert_colors) { this->invert_colors_ = invert_colors; } + display::DisplayType get_display_type() override { return display::DisplayType::DISPLAY_TYPE_COLOR; } + + void set_reset_pin(GPIOPin *reset_pin) { this->reset_pin_ = reset_pin; } + void set_enable_pins(std::vector enable_pins) { this->enable_pins_ = std::move(enable_pins); } + void set_pclk_frequency(uint32_t pclk_frequency) { this->pclk_frequency_ = pclk_frequency; } + int get_width_internal() override { return this->width_; } + int get_height_internal() override { return this->height_; } + void set_hsync_back_porch(uint16_t hsync_back_porch) { this->hsync_back_porch_ = hsync_back_porch; } + void set_hsync_front_porch(uint16_t hsync_front_porch) { this->hsync_front_porch_ = hsync_front_porch; } + void set_hsync_pulse_width(uint16_t hsync_pulse_width) { this->hsync_pulse_width_ = hsync_pulse_width; } + void set_vsync_pulse_width(uint16_t vsync_pulse_width) { this->vsync_pulse_width_ = vsync_pulse_width; } + void set_vsync_back_porch(uint16_t vsync_back_porch) { this->vsync_back_porch_ = vsync_back_porch; } + void set_vsync_front_porch(uint16_t vsync_front_porch) { this->vsync_front_porch_ = vsync_front_porch; } + void set_init_sequence(const std::vector &init_sequence) { this->init_sequence_ = init_sequence; } + void set_model(const char *model) { this->model_ = model; } + void set_lane_bit_rate(uint16_t lane_bit_rate) { this->lane_bit_rate_ = lane_bit_rate; } + void set_lanes(uint8_t lanes) { this->lanes_ = lanes; } + void set_madctl(uint8_t madctl) { this->madctl_ = madctl; } + + void smark_failed(const char *message, esp_err_t err) { + auto str = str_sprintf("Setup failed: %s: %s", message, esp_err_to_name(err)); + this->mark_failed(str.c_str()); + } + + void update() override; + + void setup() override; + + void draw_pixels_at(int x_start, int y_start, int w, int h, const uint8_t *ptr, display::ColorOrder order, + display::ColorBitness bitness, bool big_endian, int x_offset, int y_offset, int x_pad) override; + + void draw_pixel_at(int x, int y, Color color) override; + void fill(Color color) override; + int get_width() override; + int get_height() override; + + void dump_config() override; + + protected: + void write_to_display_(int x_start, int y_start, int w, int h, const uint8_t *ptr, int x_offset, int y_offset, + int x_pad); + bool check_buffer_(); + GPIOPin *reset_pin_{nullptr}; + std::vector enable_pins_{}; + size_t width_{}; + size_t height_{}; + uint8_t madctl_{}; + uint16_t hsync_pulse_width_ = 10; + uint16_t hsync_back_porch_ = 10; + uint16_t hsync_front_porch_ = 20; + uint16_t vsync_pulse_width_ = 10; + uint16_t vsync_back_porch_ = 10; + uint16_t vsync_front_porch_ = 10; + const char *model_{"Unknown"}; + std::vector init_sequence_{}; + uint16_t pclk_frequency_ = 16; // in MHz + uint16_t lane_bit_rate_{1500}; // in Mbps + uint8_t lanes_{2}; // 1, 2, 3 or 4 lanes + + bool invert_colors_{}; + display::ColorOrder color_mode_{display::COLOR_ORDER_BGR}; + display::ColorBitness color_depth_; + uint8_t pixel_mode_{}; + + esp_lcd_panel_handle_t handle_{}; + esp_lcd_dsi_bus_handle_t bus_handle_{}; + esp_lcd_panel_io_handle_t io_handle_{}; + SemaphoreHandle_t io_lock_{}; + uint8_t *buffer_{nullptr}; + uint16_t x_low_{1}; + uint16_t y_low_{1}; + uint16_t x_high_{0}; + uint16_t y_high_{0}; +}; + +} // namespace mipi_dsi +} // namespace esphome +#endif diff --git a/esphome/components/mipi_dsi/models/__init__.py b/esphome/components/mipi_dsi/models/__init__.py new file mode 100644 index 0000000000..e69de29bb2 diff --git a/esphome/components/mipi_dsi/models/guition.py b/esphome/components/mipi_dsi/models/guition.py new file mode 100644 index 0000000000..5f7db4ebda --- /dev/null +++ b/esphome/components/mipi_dsi/models/guition.py @@ -0,0 +1,37 @@ +from esphome.components.mipi import DriverChip +import esphome.config_validation as cv + +# fmt: off +DriverChip( + "JC1060P470", + width=1024, + height=600, + hsync_back_porch=160, + hsync_pulse_width=40, + hsync_front_porch=160, + vsync_back_porch=23, + vsync_pulse_width=10, + vsync_front_porch=12, + pclk_frequency="54MHz", + lane_bit_rate="750Mbps", + swap_xy=cv.UNDEFINED, + color_order="RGB", + initsequence=[ + (0x30, 0x00), (0xF7, 0x49, 0x61, 0x02, 0x00), (0x30, 0x01), (0x04, 0x0C), (0x05, 0x00), (0x06, 0x00), + (0x0B, 0x11), (0x17, 0x00), (0x20, 0x04), (0x1F, 0x05), (0x23, 0x00), (0x25, 0x19), (0x28, 0x18), (0x29, 0x04), (0x2A, 0x01), + (0x2B, 0x04), (0x2C, 0x01), (0x30, 0x02), (0x01, 0x22), (0x03, 0x12), (0x04, 0x00), (0x05, 0x64), (0x0A, 0x08), + (0x0B, 0x0A, 0x1A, 0x0B, 0x0D, 0x0D, 0x11, 0x10, 0x06, 0x08, 0x1F, 0x1D), + (0x0C, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D), + (0x0D, 0x16, 0x1B, 0x0B, 0x0D, 0x0D, 0x11, 0x10, 0x07, 0x09, 0x1E, 0x1C), + (0x0E, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D), + (0x0F, 0x16, 0x1B, 0x0D, 0x0B, 0x0D, 0x11, 0x10, 0x1C, 0x1E, 0x09, 0x07), + (0x10, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D), + (0x11, 0x0A, 0x1A, 0x0D, 0x0B, 0x0D, 0x11, 0x10, 0x1D, 0x1F, 0x08, 0x06), + (0x12, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D, 0x0D), + (0x14, 0x00, 0x00, 0x11, 0x11), (0x18, 0x99), (0x30, 0x06), + (0x12, 0x36, 0x2C, 0x2E, 0x3C, 0x38, 0x35, 0x35, 0x32, 0x2E, 0x1D, 0x2B, 0x21, 0x16, 0x29,), + (0x13, 0x36, 0x2C, 0x2E, 0x3C, 0x38, 0x35, 0x35, 0x32, 0x2E, 0x1D, 0x2B, 0x21, 0x16, 0x29,), + (0x30, 0x0A), (0x02, 0x4F), (0x0B, 0x40), (0x12, 0x3E), (0x13, 0x78), (0x30, 0x0D), (0x0D, 0x04), + (0x10, 0x0C), (0x11, 0x0C), (0x12, 0x0C), (0x13, 0x0C), (0x30, 0x00), + ], +) diff --git a/esphome/components/mipi_dsi/models/m5stack.py b/esphome/components/mipi_dsi/models/m5stack.py new file mode 100644 index 0000000000..6055c77f8f --- /dev/null +++ b/esphome/components/mipi_dsi/models/m5stack.py @@ -0,0 +1,57 @@ +from esphome.components.mipi import DriverChip +import esphome.config_validation as cv + +# fmt: off +DriverChip( + "M5STACK-TAB5", + height=1280, + width=720, + hsync_back_porch=140, + hsync_pulse_width=40, + hsync_front_porch=40, + vsync_back_porch=20, + vsync_pulse_width=4, + vsync_front_porch=20, + pclk_frequency="60MHz", + lane_bit_rate="730Mbps", + swap_xy=cv.UNDEFINED, + color_order="RGB", + initsequence=[ + (0xFF, 0x98, 0x81, 0x01), # Select Page 1 + (0xB7, 0x03), # Pad control - 2 lane + (0xFF, 0x98, 0x81, 0x00), # Select Page 0 + # CMD_Page 3 + (0xFF, 0x98, 0x81, 0x03), # Select Page 3 + (0x01, 0x00), (0x02, 0x00), (0x03, 0x73), (0x04, 0x00), (0x05, 0x00), (0x06, 0x08), (0x07, 0x00), (0x08, 0x00), + (0x09, 0x1B), (0x0A, 0x01), (0x0B, 0x01), (0x0C, 0x0D), (0x0D, 0x01), (0x0E, 0x01), (0x0F, 0x26), (0x10, 0x26), + (0x11, 0x00), (0x12, 0x00), (0x13, 0x02), (0x14, 0x00), (0x15, 0x00), (0x16, 0x00), (0x17, 0x00), (0x18, 0x00), + (0x19, 0x00), (0x1A, 0x00), (0x1B, 0x00), (0x1C, 0x00), (0x1D, 0x00), (0x1E, 0x40), (0x1F, 0x00), (0x20, 0x06), + (0x21, 0x01), (0x22, 0x00), (0x23, 0x00), (0x24, 0x00), (0x25, 0x00), (0x26, 0x00), (0x27, 0x00), (0x28, 0x33), + (0x29, 0x03), (0x2A, 0x00), (0x2B, 0x00), (0x2C, 0x00), (0x2D, 0x00), (0x2E, 0x00), (0x2F, 0x00), (0x30, 0x00), + (0x31, 0x00), (0x32, 0x00), (0x33, 0x00), (0x34, 0x00), (0x35, 0x00), (0x36, 0x00), (0x37, 0x00), (0x38, 0x00), + (0x39, 0x00), (0x3A, 0x00), (0x3B, 0x00), (0x3C, 0x00), (0x3D, 0x00), (0x3E, 0x00), (0x3F, 0x00), (0x40, 0x00), + (0x41, 0x00), (0x42, 0x00), (0x43, 0x00), (0x44, 0x00), (0x50, 0x01), (0x51, 0x23), (0x52, 0x45), (0x53, 0x67), + (0x54, 0x89), (0x55, 0xAB), (0x56, 0x01), (0x57, 0x23), (0x58, 0x45), (0x59, 0x67), (0x5A, 0x89), (0x5B, 0xAB), + (0x5C, 0xCD), (0x5D, 0xEF), (0x5E, 0x11), (0x5F, 0x02), (0x60, 0x00), (0x61, 0x07), (0x62, 0x06), (0x63, 0x0E), + (0x64, 0x0F), (0x65, 0x0C), (0x66, 0x0D), (0x67, 0x02), (0x68, 0x02), (0x69, 0x02), (0x6A, 0x02), (0x6B, 0x02), + (0x6C, 0x02), (0x6D, 0x02), (0x6E, 0x02), (0x6F, 0x02), (0x70, 0x02), (0x71, 0x02), (0x72, 0x02), (0x73, 0x05), + (0x74, 0x01), (0x75, 0x02), (0x76, 0x00), (0x77, 0x07), (0x78, 0x06), (0x79, 0x0E), (0x7A, 0x0F), (0x7B, 0x0C), + (0x7C, 0x0D), (0x7D, 0x02), (0x7E, 0x02), (0x7F, 0x02), (0x80, 0x02), (0x81, 0x02), (0x82, 0x02), (0x83, 0x02), + (0x84, 0x02), (0x85, 0x02), (0x86, 0x02), (0x87, 0x02), (0x88, 0x02), (0x89, 0x05), (0x8A, 0x01), + (0xFF, 0x98, 0x81, 0x04), # Select Page 4 + (0x38, 0x01), (0x39, 0x00), (0x6C, 0x15), (0x6E, 0x1A), (0x6F, 0x25), (0x3A, 0xA4), (0x8D, 0x20), (0x87, 0xBA), (0x3B, 0x98), + (0xFF, 0x98, 0x81, 0x01), # Select Page 1 + (0x22, 0x0A), (0x31, 0x00), (0x50, 0x6B), (0x51, 0x66), (0x53, 0x73), (0x55, 0x8B), (0x60, 0x1B), (0x61, 0x01), (0x62, 0x0C), (0x63, 0x00), + # Gamma P + (0xA0, 0x00), (0xA1, 0x15), (0xA2, 0x1F), (0xA3, 0x13), (0xA4, 0x11), (0xA5, 0x21), (0xA6, 0x17), (0xA7, 0x1B), + (0xA8, 0x6B), (0xA9, 0x1E), (0xAA, 0x2B), (0xAB, 0x5D), (0xAC, 0x19), (0xAD, 0x14), (0xAE, 0x4B), (0xAF, 0x1D), + (0xB0, 0x27), (0xB1, 0x49), (0xB2, 0x5D), (0xB3, 0x39), + # Gamma N + (0xC0, 0x00), (0xC1, 0x01), (0xC2, 0x0C), (0xC3, 0x11), (0xC4, 0x15), (0xC5, 0x28), (0xC6, 0x1B), (0xC7, 0x1C), + (0xC8, 0x62), (0xC9, 0x1C), (0xCA, 0x29), (0xCB, 0x60), (0xCC, 0x16), (0xCD, 0x17), (0xCE, 0x4A), (0xCF, 0x23), + (0xD0, 0x24), (0xD1, 0x4F), (0xD2, 0x5F), (0xD3, 0x39), + # CMD_Page 0 + (0xFF, 0x98, 0x81, 0x00), # Select Page 0 + (0x35,), (0xFE,), + ], +) diff --git a/esphome/components/mipi_dsi/models/waveshare.py b/esphome/components/mipi_dsi/models/waveshare.py new file mode 100644 index 0000000000..c3d080f8b2 --- /dev/null +++ b/esphome/components/mipi_dsi/models/waveshare.py @@ -0,0 +1,96 @@ +from esphome.components.mipi import DriverChip +import esphome.config_validation as cv + +# fmt: off +DriverChip( + "WAVESHARE-P4-NANO-10.1", + height=1280, + width=800, + hsync_back_porch=20, + hsync_pulse_width=20, + hsync_front_porch=40, + vsync_back_porch=12, + vsync_pulse_width=4, + vsync_front_porch=30, + pclk_frequency="80MHz", + lane_bit_rate="1.5Gbps", + swap_xy=cv.UNDEFINED, + color_order="RGB", + initsequence=[ + (0xE0, 0x00), # select userpage + (0xE1, 0x93), (0xE2, 0x65), (0xE3, 0xF8), + (0x80, 0x01), # Select number of lanes (2) + (0xE0, 0x01), # select page 1 + (0x00, 0x00), (0x01, 0x38), (0x03, 0x10), (0x04, 0x38), (0x0C, 0x74), (0x17, 0x00), (0x18, 0xAF), (0x19, 0x00), + (0x1A, 0x00), (0x1B, 0xAF), (0x1C, 0x00), (0x35, 0x26), (0x37, 0x09), (0x38, 0x04), (0x39, 0x00), (0x3C, 0x78), + (0x3D, 0xFF), (0x3E, 0xFF), (0x3F, 0x7F), (0x40, 0x06), (0x41, 0xA0), (0x42, 0x81), (0x43, 0x1E), (0x44, 0x0D), + (0x45, 0x28), (0x55, 0x02), (0x57, 0x69), (0x59, 0x0A), (0x5A, 0x2A), (0x5B, 0x17), (0x5D, 0x7F), (0x5E, 0x6A), + (0x5F, 0x5B), (0x60, 0x4F), (0x61, 0x4A), (0x62, 0x3D), (0x63, 0x41), (0x64, 0x2A), (0x65, 0x44), (0x66, 0x43), + (0x67, 0x44), (0x68, 0x62), (0x69, 0x52), (0x6A, 0x59), (0x6B, 0x4C), (0x6C, 0x48), (0x6D, 0x3A), (0x6E, 0x26), + (0x6F, 0x00), (0x70, 0x7F), (0x71, 0x6A), (0x72, 0x5B), (0x73, 0x4F), (0x74, 0x4A), (0x75, 0x3D), (0x76, 0x41), + (0x77, 0x2A), (0x78, 0x44), (0x79, 0x43), (0x7A, 0x44), (0x7B, 0x62), (0x7C, 0x52), (0x7D, 0x59), (0x7E, 0x4C), + (0x7F, 0x48), (0x80, 0x3A), (0x81, 0x26), (0x82, 0x00), + (0xE0, 0x02), # select page 2 + (0x00, 0x42), (0x01, 0x42), (0x02, 0x40), (0x03, 0x40), (0x04, 0x5E), (0x05, 0x5E), (0x06, 0x5F), (0x07, 0x5F), + (0x08, 0x5F), (0x09, 0x57), (0x0A, 0x57), (0x0B, 0x77), (0x0C, 0x77), (0x0D, 0x47), (0x0E, 0x47), (0x0F, 0x45), + (0x10, 0x45), (0x11, 0x4B), (0x12, 0x4B), (0x13, 0x49), (0x14, 0x49), (0x15, 0x5F), (0x16, 0x41), (0x17, 0x41), + (0x18, 0x40), (0x19, 0x40), (0x1A, 0x5E), (0x1B, 0x5E), (0x1C, 0x5F), (0x1D, 0x5F), (0x1E, 0x5F), (0x1F, 0x57), + (0x20, 0x57), (0x21, 0x77), (0x22, 0x77), (0x23, 0x46), (0x24, 0x46), (0x25, 0x44), (0x26, 0x44), (0x27, 0x4A), + (0x28, 0x4A), (0x29, 0x48), (0x2A, 0x48), (0x2B, 0x5F), (0x2C, 0x01), (0x2D, 0x01), (0x2E, 0x00), (0x2F, 0x00), + (0x30, 0x1F), (0x31, 0x1F), (0x32, 0x1E), (0x33, 0x1E), (0x34, 0x1F), (0x35, 0x17), (0x36, 0x17), (0x37, 0x37), + (0x38, 0x37), (0x39, 0x08), (0x3A, 0x08), (0x3B, 0x0A), (0x3C, 0x0A), (0x3D, 0x04), (0x3E, 0x04), (0x3F, 0x06), + (0x40, 0x06), (0x41, 0x1F), (0x42, 0x02), (0x43, 0x02), (0x44, 0x00), (0x45, 0x00), (0x46, 0x1F), (0x47, 0x1F), + (0x48, 0x1E), (0x49, 0x1E), (0x4A, 0x1F), (0x4B, 0x17), (0x4C, 0x17), (0x4D, 0x37), (0x4E, 0x37), (0x4F, 0x09), + (0x50, 0x09), (0x51, 0x0B), (0x52, 0x0B), (0x53, 0x05), (0x54, 0x05), (0x55, 0x07), (0x56, 0x07), (0x57, 0x1F), + (0x58, 0x40), (0x5B, 0x30), (0x5C, 0x00), (0x5D, 0x34), (0x5E, 0x05), (0x5F, 0x02), (0x63, 0x00), (0x64, 0x6A), + (0x67, 0x73), (0x68, 0x07), (0x69, 0x08), (0x6A, 0x6A), (0x6B, 0x08), (0x6C, 0x00), (0x6D, 0x00), (0x6E, 0x00), + (0x6F, 0x88), (0x75, 0xFF), (0x77, 0xDD), (0x78, 0x2C), (0x79, 0x15), (0x7A, 0x17), (0x7D, 0x14), (0x7E, 0x82), + (0xE0, 0x04), # select page 4 + (0x00, 0x0E), (0x02, 0xB3), (0x09, 0x61), (0x0E, 0x48), (0x37, 0x58), (0x2B, 0x0F), + (0xE0, 0x00), # Select userpage + (0xE6, 0x02), (0xE7, 0x0C), + ], +) + +DriverChip( + "WAVESHARE-P4-86-PANEL", + height=720, + width=720, + hsync_back_porch=50, + hsync_pulse_width=20, + hsync_front_porch=50, + vsync_back_porch=20, + vsync_pulse_width=4, + vsync_front_porch=20, + pclk_frequency="38MHz", + lane_bit_rate="480Mbps", + swap_xy=cv.UNDEFINED, + color_order="RGB", + reset_pin=27, + initsequence=[ + (0xB9, 0xF1, 0x12, 0x83), + (0xB1, 0x00, 0x00, 0x00, 0xDA, 0x80), + (0xB2, 0x3C, 0x12, 0x30), + (0xB3, 0x10, 0x10, 0x28, 0x28, 0x03, 0xFF, 0x00, 0x00, 0x00, 0x00), + (0xB4, 0x80), + (0xB5, 0x0A, 0x0A), + (0xB6, 0x97, 0x97), + (0xB8, 0x26, 0x22, 0xF0, 0x13), + (0xBA, 0x31, 0x81, 0x0F, 0xF9, 0x0E, 0x06, 0x20, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x44, 0x25, 0x00, 0x90, 0x0A, 0x00, 0x00, 0x01, 0x4F, 0x01, 0x00, 0x00, 0x37), + (0xBC, 0x47), + (0xBF, 0x02, 0x11, 0x00), + (0xC0, 0x73, 0x73, 0x50, 0x50, 0x00, 0x00, 0x12, 0x70, 0x00), + (0xC1, 0x25, 0x00, 0x32, 0x32, 0x77, 0xE4, 0xFF, 0xFF, 0xCC, 0xCC, 0x77, 0x77), + (0xC6, 0x82, 0x00, 0xBF, 0xFF, 0x00, 0xFF), + (0xC7, 0xB8, 0x00, 0x0A, 0x10, 0x01, 0x09), + (0xC8, 0x10, 0x40, 0x1E, 0x02), + (0xCC, 0x0B), + (0xE0, 0x00, 0x0B, 0x10, 0x2C, 0x3D, 0x3F, 0x42, 0x3A, 0x07, 0x0D, 0x0F, 0x13, 0x15, 0x13, 0x14, 0x0F, 0x16, 0x00, 0x0B, 0x10, 0x2C, 0x3D, 0x3F, 0x42, 0x3A, 0x07, 0x0D, 0x0F, 0x13, 0x15, 0x13, 0x14, 0x0F, 0x16), + (0xE3, 0x07, 0x07, 0x0B, 0x0B, 0x0B, 0x0B, 0x00, 0x00, 0x00, 0x00, 0xFF, 0x00, 0xC0, 0x10), + (0xE9, 0xC8, 0x10, 0x0A, 0x00, 0x00, 0x80, 0x81, 0x12, 0x31, 0x23, 0x4F, 0x86, 0xA0, 0x00, 0x47, 0x08, 0x00, 0x00, 0x0C, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0C, 0x00, 0x00, 0x00, 0x98, 0x02, 0x8B, 0xAF, 0x46, 0x02, 0x88, 0x88, 0x88, 0x88, 0x88, 0x98, 0x13, 0x8B, 0xAF, 0x57, 0x13, 0x88, 0x88, 0x88, 0x88, 0x88, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00), + (0xEA, 0x97, 0x0C, 0x09, 0x09, 0x09, 0x78, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x9F, 0x31, 0x8B, 0xA8, 0x31, 0x75, 0x88, 0x88, 0x88, 0x88, 0x88, 0x9F, 0x20, 0x8B, 0xA8, 0x20, 0x64, 0x88, 0x88, 0x88, 0x88, 0x88, 0x23, 0x00, 0x00, 0x02, 0x71, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x40, 0x80, 0x81, 0x00, 0x00, 0x00, 0x00), + (0xEF, 0xFF, 0xFF, 0x01), + (0x11, 0x00), + (0x29, 0x00), + ], +) diff --git a/esphome/components/mipi_rgb/__init__.py b/esphome/components/mipi_rgb/__init__.py new file mode 100644 index 0000000000..4f9972c6e0 --- /dev/null +++ b/esphome/components/mipi_rgb/__init__.py @@ -0,0 +1,2 @@ +CODEOWNERS = ["@clydebarrow"] +DOMAIN = "mipi_rgb" diff --git a/esphome/components/mipi_rgb/display.py b/esphome/components/mipi_rgb/display.py new file mode 100644 index 0000000000..3001d33980 --- /dev/null +++ b/esphome/components/mipi_rgb/display.py @@ -0,0 +1,321 @@ +import importlib +import pkgutil + +from esphome import pins +import esphome.codegen as cg +from esphome.components import display, spi +from esphome.components.const import ( + BYTE_ORDER_BIG, + BYTE_ORDER_LITTLE, + CONF_BYTE_ORDER, + CONF_DRAW_ROUNDING, +) +from esphome.components.display import CONF_SHOW_TEST_CARD +from esphome.components.esp32 import const, only_on_variant +from esphome.components.mipi import ( + COLOR_ORDERS, + CONF_DE_PIN, + CONF_HSYNC_BACK_PORCH, + CONF_HSYNC_FRONT_PORCH, + CONF_HSYNC_PULSE_WIDTH, + CONF_PCLK_PIN, + CONF_PIXEL_MODE, + CONF_USE_AXIS_FLIPS, + CONF_VSYNC_BACK_PORCH, + CONF_VSYNC_FRONT_PORCH, + CONF_VSYNC_PULSE_WIDTH, + MODE_BGR, + PIXEL_MODE_16BIT, + PIXEL_MODE_18BIT, + DriverChip, + dimension_schema, + map_sequence, + power_of_two, + requires_buffer, +) +from esphome.components.rpi_dpi_rgb.display import ( + CONF_PCLK_FREQUENCY, + CONF_PCLK_INVERTED, +) +import esphome.config_validation as cv +from esphome.const import ( + CONF_BLUE, + CONF_COLOR_ORDER, + CONF_CS_PIN, + CONF_DATA_PINS, + CONF_DATA_RATE, + CONF_DC_PIN, + CONF_DIMENSIONS, + CONF_ENABLE_PIN, + CONF_GREEN, + CONF_HSYNC_PIN, + CONF_ID, + CONF_IGNORE_STRAPPING_WARNING, + CONF_INIT_SEQUENCE, + CONF_INVERT_COLORS, + CONF_LAMBDA, + CONF_MIRROR_X, + CONF_MIRROR_Y, + CONF_MODEL, + CONF_NUMBER, + CONF_RED, + CONF_RESET_PIN, + CONF_ROTATION, + CONF_SPI_ID, + CONF_SWAP_XY, + CONF_TRANSFORM, + CONF_VSYNC_PIN, + CONF_WIDTH, +) +from esphome.final_validate import full_config + +from ..spi import CONF_SPI_MODE, SPI_DATA_RATE_SCHEMA, SPI_MODE_OPTIONS, SPIComponent +from . import models + +DEPENDENCIES = ["esp32", "psram"] + +mipi_rgb_ns = cg.esphome_ns.namespace("mipi_rgb") +mipi_rgb = mipi_rgb_ns.class_("MipiRgb", display.Display, cg.Component) +mipi_rgb_spi = mipi_rgb_ns.class_( + "MipiRgbSpi", mipi_rgb, display.Display, cg.Component, spi.SPIDevice +) +ColorOrder = display.display_ns.enum("ColorMode") + +DATA_PIN_SCHEMA = pins.internal_gpio_output_pin_schema + +DriverChip("CUSTOM") + +# Import all models dynamically from the models package + +for module_info in pkgutil.iter_modules(models.__path__): + importlib.import_module(f".models.{module_info.name}", package=__package__) + +MODELS = DriverChip.get_models() + + +def data_pin_validate(value): + """ + It is safe to use strapping pins as RGB output data bits, as they are outputs only, + and not initialised until after boot. + """ + if not isinstance(value, dict): + try: + return DATA_PIN_SCHEMA( + {CONF_NUMBER: value, CONF_IGNORE_STRAPPING_WARNING: True} + ) + except cv.Invalid: + pass + return DATA_PIN_SCHEMA(value) + + +def data_pin_set(length): + return cv.All( + [data_pin_validate], + cv.Length(min=length, max=length, msg=f"Exactly {length} data pins required"), + ) + + +def model_schema(config): + model = MODELS[config[CONF_MODEL].upper()] + if transforms := model.transforms: + transform = cv.Schema({cv.Required(x): cv.boolean for x in transforms}) + for x in (CONF_SWAP_XY, CONF_MIRROR_X, CONF_MIRROR_Y): + if x not in transforms: + transform = transform.extend( + {cv.Optional(x): cv.invalid(f"{x} not supported by this model")} + ) + else: + transform = cv.invalid("This model does not support transforms") + + # RPI model does not use an init sequence, indicates with empty list + if model.initsequence is None: + # Custom model requires an init sequence + iseqconf = cv.Required(CONF_INIT_SEQUENCE) + uses_spi = True + else: + iseqconf = cv.Optional(CONF_INIT_SEQUENCE) + uses_spi = CONF_INIT_SEQUENCE in config or len(model.initsequence) != 0 + swap_xy = config.get(CONF_TRANSFORM, {}).get(CONF_SWAP_XY, False) + + # Dimensions are optional if the model has a default width and the swap_xy transform is not overridden + cv_dimensions = ( + cv.Optional if model.get_default(CONF_WIDTH) and not swap_xy else cv.Required + ) + pixel_modes = (PIXEL_MODE_16BIT, PIXEL_MODE_18BIT, "16", "18") + schema = display.FULL_DISPLAY_SCHEMA.extend( + { + model.option(CONF_RESET_PIN, cv.UNDEFINED): pins.gpio_output_pin_schema, + cv.GenerateID(): cv.declare_id(mipi_rgb_spi if uses_spi else mipi_rgb), + cv_dimensions(CONF_DIMENSIONS): dimension_schema( + model.get_default(CONF_DRAW_ROUNDING, 1) + ), + model.option(CONF_ENABLE_PIN, cv.UNDEFINED): cv.ensure_list( + pins.gpio_output_pin_schema + ), + model.option(CONF_COLOR_ORDER, MODE_BGR): cv.enum(COLOR_ORDERS, upper=True), + model.option(CONF_DRAW_ROUNDING, 2): power_of_two, + model.option(CONF_PIXEL_MODE, PIXEL_MODE_16BIT): cv.one_of( + *pixel_modes, lower=True + ), + model.option(CONF_TRANSFORM, cv.UNDEFINED): transform, + cv.Required(CONF_MODEL): cv.one_of(model.name, upper=True), + model.option(CONF_INVERT_COLORS, False): cv.boolean, + model.option(CONF_USE_AXIS_FLIPS, True): cv.boolean, + model.option(CONF_PCLK_FREQUENCY, "40MHz"): cv.All( + cv.frequency, cv.Range(min=4e6, max=100e6) + ), + model.option(CONF_PCLK_INVERTED, True): cv.boolean, + iseqconf: cv.ensure_list(map_sequence), + model.option(CONF_BYTE_ORDER, BYTE_ORDER_BIG): cv.one_of( + BYTE_ORDER_LITTLE, BYTE_ORDER_BIG, lower=True + ), + model.option(CONF_HSYNC_PULSE_WIDTH): cv.int_, + model.option(CONF_HSYNC_BACK_PORCH): cv.int_, + model.option(CONF_HSYNC_FRONT_PORCH): cv.int_, + model.option(CONF_VSYNC_PULSE_WIDTH): cv.int_, + model.option(CONF_VSYNC_BACK_PORCH): cv.int_, + model.option(CONF_VSYNC_FRONT_PORCH): cv.int_, + model.option(CONF_DATA_PINS): cv.Any( + data_pin_set(16), + cv.Schema( + { + cv.Required(CONF_RED): data_pin_set(5), + cv.Required(CONF_GREEN): data_pin_set(6), + cv.Required(CONF_BLUE): data_pin_set(5), + } + ), + ), + model.option( + CONF_DE_PIN, cv.UNDEFINED + ): pins.internal_gpio_output_pin_schema, + model.option(CONF_PCLK_PIN): pins.internal_gpio_output_pin_schema, + model.option(CONF_HSYNC_PIN): pins.internal_gpio_output_pin_schema, + model.option(CONF_VSYNC_PIN): pins.internal_gpio_output_pin_schema, + model.option(CONF_RESET_PIN, cv.UNDEFINED): pins.gpio_output_pin_schema, + } + ) + if uses_spi: + schema = schema.extend( + { + cv.GenerateID(CONF_SPI_ID): cv.use_id(SPIComponent), + model.option(CONF_DC_PIN, cv.UNDEFINED): pins.gpio_output_pin_schema, + model.option(CONF_DATA_RATE, "1MHz"): SPI_DATA_RATE_SCHEMA, + model.option(CONF_SPI_MODE, "MODE0"): cv.enum( + SPI_MODE_OPTIONS, upper=True + ), + model.option(CONF_CS_PIN, cv.UNDEFINED): pins.gpio_output_pin_schema, + } + ) + return schema + + +def _config_schema(config): + config = cv.Schema( + { + cv.Required(CONF_MODEL): cv.one_of(*MODELS, upper=True), + }, + extra=cv.ALLOW_EXTRA, + )(config) + schema = model_schema(config) + return cv.All( + schema, + only_on_variant(supported=[const.VARIANT_ESP32S3]), + cv.only_with_esp_idf, + )(config) + + +CONFIG_SCHEMA = _config_schema + + +def _final_validate(config): + global_config = full_config.get() + + from esphome.components.lvgl import DOMAIN as LVGL_DOMAIN + + if not requires_buffer(config) and LVGL_DOMAIN not in global_config: + # If no drawing methods are configured, and LVGL is not enabled, show a test card + config[CONF_SHOW_TEST_CARD] = True + if CONF_SPI_ID in config: + config = spi.final_validate_device_schema( + "mipi_rgb", require_miso=False, require_mosi=True + )(config) + return config + + +FINAL_VALIDATE_SCHEMA = _final_validate + + +async def to_code(config): + model = MODELS[config[CONF_MODEL].upper()] + width, height, _offset_width, _offset_height = model.get_dimensions(config) + var = cg.new_Pvariable(config[CONF_ID], width, height) + cg.add(var.set_model(model.name)) + if enable_pin := config.get(CONF_ENABLE_PIN): + enable = [await cg.gpio_pin_expression(pin) for pin in enable_pin] + cg.add(var.set_enable_pins(enable)) + + if CONF_SPI_ID in config: + await spi.register_spi_device(var, config) + sequence, madctl = model.get_sequence(config) + cg.add(var.set_init_sequence(sequence)) + cg.add(var.set_madctl(madctl)) + + cg.add(var.set_color_mode(COLOR_ORDERS[config[CONF_COLOR_ORDER]])) + cg.add(var.set_invert_colors(config[CONF_INVERT_COLORS])) + cg.add(var.set_hsync_pulse_width(config[CONF_HSYNC_PULSE_WIDTH])) + cg.add(var.set_hsync_back_porch(config[CONF_HSYNC_BACK_PORCH])) + cg.add(var.set_hsync_front_porch(config[CONF_HSYNC_FRONT_PORCH])) + cg.add(var.set_vsync_pulse_width(config[CONF_VSYNC_PULSE_WIDTH])) + cg.add(var.set_vsync_back_porch(config[CONF_VSYNC_BACK_PORCH])) + cg.add(var.set_vsync_front_porch(config[CONF_VSYNC_FRONT_PORCH])) + cg.add(var.set_pclk_inverted(config[CONF_PCLK_INVERTED])) + cg.add(var.set_pclk_frequency(config[CONF_PCLK_FREQUENCY])) + index = 0 + dpins = [] + if CONF_RED in config[CONF_DATA_PINS]: + red_pins = config[CONF_DATA_PINS][CONF_RED] + green_pins = config[CONF_DATA_PINS][CONF_GREEN] + blue_pins = config[CONF_DATA_PINS][CONF_BLUE] + if config[CONF_COLOR_ORDER] == "BGR": + dpins.extend(red_pins) + dpins.extend(green_pins) + dpins.extend(blue_pins) + else: + dpins.extend(blue_pins) + dpins.extend(green_pins) + dpins.extend(red_pins) + # swap bytes to match big-endian format + dpins = dpins[8:16] + dpins[0:8] + else: + dpins = config[CONF_DATA_PINS] + for index, pin in enumerate(dpins): + data_pin = await cg.gpio_pin_expression(pin) + cg.add(var.add_data_pin(data_pin, index)) + + if dc_pin := config.get(CONF_DC_PIN): + dc = await cg.gpio_pin_expression(dc_pin) + cg.add(var.set_dc_pin(dc)) + + if reset_pin := config.get(CONF_RESET_PIN): + reset = await cg.gpio_pin_expression(reset_pin) + cg.add(var.set_reset_pin(reset)) + + if model.rotation_as_transform(config): + config[CONF_ROTATION] = 0 + + if de_pin := config.get(CONF_DE_PIN): + pin = await cg.gpio_pin_expression(de_pin) + cg.add(var.set_de_pin(pin)) + pin = await cg.gpio_pin_expression(config[CONF_PCLK_PIN]) + cg.add(var.set_pclk_pin(pin)) + pin = await cg.gpio_pin_expression(config[CONF_HSYNC_PIN]) + cg.add(var.set_hsync_pin(pin)) + pin = await cg.gpio_pin_expression(config[CONF_VSYNC_PIN]) + cg.add(var.set_vsync_pin(pin)) + + await display.register_display(var, config) + if lamb := config.get(CONF_LAMBDA): + lambda_ = await cg.process_lambda( + lamb, [(display.DisplayRef, "it")], return_type=cg.void + ) + cg.add(var.set_writer(lambda_)) diff --git a/esphome/components/mipi_rgb/mipi_rgb.cpp b/esphome/components/mipi_rgb/mipi_rgb.cpp new file mode 100644 index 0000000000..00c9c8cbff --- /dev/null +++ b/esphome/components/mipi_rgb/mipi_rgb.cpp @@ -0,0 +1,388 @@ +#ifdef USE_ESP32_VARIANT_ESP32S3 +#include "mipi_rgb.h" +#include "esphome/core/log.h" +#include "esphome/core/hal.h" +#include "esp_lcd_panel_rgb.h" + +namespace esphome { +namespace mipi_rgb { + +static const uint8_t DELAY_FLAG = 0xFF; +static constexpr uint8_t MADCTL_MY = 0x80; // Bit 7 Bottom to top +static constexpr uint8_t MADCTL_MX = 0x40; // Bit 6 Right to left +static constexpr uint8_t MADCTL_MV = 0x20; // Bit 5 Swap axes +static constexpr uint8_t MADCTL_ML = 0x10; // Bit 4 Refresh bottom to top +static constexpr uint8_t MADCTL_BGR = 0x08; // Bit 3 Blue-Green-Red pixel order +static constexpr uint8_t MADCTL_XFLIP = 0x02; // Mirror the display horizontally +static constexpr uint8_t MADCTL_YFLIP = 0x01; // Mirror the display vertically + +void MipiRgb::setup_enables_() { + if (!this->enable_pins_.empty()) { + for (auto *pin : this->enable_pins_) { + pin->setup(); + pin->digital_write(true); + } + delay(10); + } + if (this->reset_pin_ != nullptr) { + this->reset_pin_->setup(); + this->reset_pin_->digital_write(true); + delay(5); + this->reset_pin_->digital_write(false); + delay(5); + this->reset_pin_->digital_write(true); + } +} + +#ifdef USE_SPI +void MipiRgbSpi::setup() { + this->setup_enables_(); + this->spi_setup(); + this->write_init_sequence_(); + this->common_setup_(); +} +void MipiRgbSpi::write_command_(uint8_t value) { + this->enable(); + if (this->dc_pin_ == nullptr) { + this->write(value, 9); + } else { + this->dc_pin_->digital_write(false); + this->write_byte(value); + this->dc_pin_->digital_write(true); + } + this->disable(); +} + +void MipiRgbSpi::write_data_(uint8_t value) { + this->enable(); + if (this->dc_pin_ == nullptr) { + this->write(value | 0x100, 9); + } else { + this->dc_pin_->digital_write(true); + this->write_byte(value); + } + this->disable(); +} + +/** + * this relies upon the init sequence being well-formed, which is guaranteed by the Python init code. + */ + +void MipiRgbSpi::write_init_sequence_() { + size_t index = 0; + auto &vec = this->init_sequence_; + while (index != vec.size()) { + if (vec.size() - index < 2) { + this->mark_failed("Malformed init sequence"); + return; + } + uint8_t cmd = vec[index++]; + uint8_t x = vec[index++]; + if (x == DELAY_FLAG) { + ESP_LOGD(TAG, "Delay %dms", cmd); + delay(cmd); + } else { + uint8_t num_args = x & 0x7F; + if (vec.size() - index < num_args) { + this->mark_failed("Malformed init sequence"); + return; + } + if (cmd == SLEEP_OUT) { + delay(120); // NOLINT + } + const auto *ptr = vec.data() + index; + ESP_LOGD(TAG, "Write command %02X, length %d, byte(s) %s", cmd, num_args, + format_hex_pretty(ptr, num_args, '.', false).c_str()); + index += num_args; + this->write_command_(cmd); + while (num_args-- != 0) + this->write_data_(*ptr++); + if (cmd == SLEEP_OUT) + delay(10); + } + } + // this->spi_teardown(); // SPI not needed after this + this->init_sequence_.clear(); + delay(10); +} + +void MipiRgbSpi::dump_config() { + MipiRgb::dump_config(); + LOG_PIN(" CS Pin: ", this->cs_); + LOG_PIN(" DC Pin: ", this->dc_pin_); + ESP_LOGCONFIG(TAG, + " SPI Data rate: %uMHz" + "\n Mirror X: %s" + "\n Mirror Y: %s" + "\n Swap X/Y: %s" + "\n Color Order: %s", + (unsigned) (this->data_rate_ / 1000000), YESNO(this->madctl_ & (MADCTL_XFLIP | MADCTL_MX)), + YESNO(this->madctl_ & (MADCTL_YFLIP | MADCTL_MY | MADCTL_ML)), YESNO(this->madctl_ & MADCTL_MV), + this->madctl_ & MADCTL_BGR ? "BGR" : "RGB"); +} + +#endif // USE_SPI + +void MipiRgb::setup() { + this->setup_enables_(); + this->common_setup_(); +} + +void MipiRgb::common_setup_() { + esp_lcd_rgb_panel_config_t config{}; + config.flags.fb_in_psram = 1; + config.bounce_buffer_size_px = this->width_ * 10; + config.num_fbs = 1; + config.timings.h_res = this->width_; + config.timings.v_res = this->height_; + config.timings.hsync_pulse_width = this->hsync_pulse_width_; + config.timings.hsync_back_porch = this->hsync_back_porch_; + config.timings.hsync_front_porch = this->hsync_front_porch_; + config.timings.vsync_pulse_width = this->vsync_pulse_width_; + config.timings.vsync_back_porch = this->vsync_back_porch_; + config.timings.vsync_front_porch = this->vsync_front_porch_; + config.timings.flags.pclk_active_neg = this->pclk_inverted_; + config.timings.pclk_hz = this->pclk_frequency_; + config.clk_src = LCD_CLK_SRC_PLL160M; + size_t data_pin_count = sizeof(this->data_pins_) / sizeof(this->data_pins_[0]); + for (size_t i = 0; i != data_pin_count; i++) { + config.data_gpio_nums[i] = this->data_pins_[i]->get_pin(); + } + config.data_width = data_pin_count; + config.disp_gpio_num = -1; + config.hsync_gpio_num = this->hsync_pin_->get_pin(); + config.vsync_gpio_num = this->vsync_pin_->get_pin(); + if (this->de_pin_) { + config.de_gpio_num = this->de_pin_->get_pin(); + } else { + config.de_gpio_num = -1; + } + config.pclk_gpio_num = this->pclk_pin_->get_pin(); + esp_err_t err = esp_lcd_new_rgb_panel(&config, &this->handle_); + if (err == ESP_OK) + err = esp_lcd_panel_reset(this->handle_); + if (err == ESP_OK) + err = esp_lcd_panel_init(this->handle_); + if (err != ESP_OK) { + auto msg = str_sprintf("lcd setup failed: %s", esp_err_to_name(err)); + this->mark_failed(msg.c_str()); + } + ESP_LOGCONFIG(TAG, "MipiRgb setup complete"); +} + +void MipiRgb::loop() { + if (this->handle_ != nullptr) + esp_lcd_rgb_panel_restart(this->handle_); +} + +void MipiRgb::update() { + if (this->is_failed()) + return; + if (this->auto_clear_enabled_) { + this->clear(); + } + if (this->show_test_card_) { + this->test_card(); + } else if (this->page_ != nullptr) { + this->page_->get_writer()(*this); + } else if (this->writer_.has_value()) { + (*this->writer_)(*this); + } else { + this->stop_poller(); + } + if (this->buffer_ == nullptr || this->x_low_ > this->x_high_ || this->y_low_ > this->y_high_) + return; + ESP_LOGV(TAG, "x_low %d, y_low %d, x_high %d, y_high %d", this->x_low_, this->y_low_, this->x_high_, this->y_high_); + int w = this->x_high_ - this->x_low_ + 1; + int h = this->y_high_ - this->y_low_ + 1; + this->write_to_display_(this->x_low_, this->y_low_, w, h, reinterpret_cast(this->buffer_), + this->x_low_, this->y_low_, this->width_ - w - this->x_low_); + // invalidate watermarks + this->x_low_ = this->width_; + this->y_low_ = this->height_; + this->x_high_ = 0; + this->y_high_ = 0; +} + +void MipiRgb::draw_pixels_at(int x_start, int y_start, int w, int h, const uint8_t *ptr, display::ColorOrder order, + display::ColorBitness bitness, bool big_endian, int x_offset, int y_offset, int x_pad) { + if (w <= 0 || h <= 0 || this->is_failed()) + return; + // if color mapping is required, pass the buck. + // note that endianness is not considered here - it is assumed to match! + if (bitness != display::COLOR_BITNESS_565) { + Display::draw_pixels_at(x_start, y_start, w, h, ptr, order, bitness, big_endian, x_offset, y_offset, x_pad); + this->write_to_display_(x_start, y_start, w, h, reinterpret_cast(this->buffer_), x_start, y_start, + this->width_ - w - x_start); + } else { + this->write_to_display_(x_start, y_start, w, h, ptr, x_offset, y_offset, x_pad); + } +} + +void MipiRgb::write_to_display_(int x_start, int y_start, int w, int h, const uint8_t *ptr, int x_offset, int y_offset, + int x_pad) { + esp_err_t err = ESP_OK; + auto stride = (x_offset + w + x_pad) * 2; + ptr += y_offset * stride + x_offset * 2; // skip to the first pixel + // x_ and y_offset are offsets into the source buffer, unrelated to our own offsets into the display. + if (x_offset == 0 && x_pad == 0) { + err = esp_lcd_panel_draw_bitmap(this->handle_, x_start, y_start, x_start + w, y_start + h, ptr); + } else { + // draw line by line + for (int y = 0; y != h; y++) { + err = esp_lcd_panel_draw_bitmap(this->handle_, x_start, y + y_start, x_start + w, y + y_start + 1, ptr); + if (err != ESP_OK) + break; + ptr += stride; // next line + } + } + if (err != ESP_OK) + ESP_LOGE(TAG, "lcd_lcd_panel_draw_bitmap failed: %s", esp_err_to_name(err)); +} + +bool MipiRgb::check_buffer_() { + if (this->is_failed()) + return false; + if (this->buffer_ != nullptr) + return true; + // this is dependent on the enum values. + RAMAllocator allocator; + this->buffer_ = allocator.allocate(this->height_ * this->width_); + if (this->buffer_ == nullptr) { + this->mark_failed("Could not allocate buffer for display!"); + return false; + } + return true; +} + +void MipiRgb::draw_pixel_at(int x, int y, Color color) { + if (!this->get_clipping().inside(x, y) || this->is_failed()) + return; + + switch (this->rotation_) { + case display::DISPLAY_ROTATION_0_DEGREES: + break; + case display::DISPLAY_ROTATION_90_DEGREES: + std::swap(x, y); + x = this->width_ - x - 1; + break; + case display::DISPLAY_ROTATION_180_DEGREES: + x = this->width_ - x - 1; + y = this->height_ - y - 1; + break; + case display::DISPLAY_ROTATION_270_DEGREES: + std::swap(x, y); + y = this->height_ - y - 1; + break; + } + if (x >= this->get_width_internal() || x < 0 || y >= this->get_height_internal() || y < 0) { + return; + } + if (!this->check_buffer_()) + return; + size_t pos = (y * this->width_) + x; + uint8_t hi_byte = static_cast(color.r & 0xF8) | (color.g >> 5); + uint8_t lo_byte = static_cast((color.g & 0x1C) << 3) | (color.b >> 3); + uint16_t new_color = hi_byte | (lo_byte << 8); // big endian + if (this->buffer_[pos] == new_color) + return; + this->buffer_[pos] = new_color; + // low and high watermark may speed up drawing from buffer + if (x < this->x_low_) + this->x_low_ = x; + if (y < this->y_low_) + this->y_low_ = y; + if (x > this->x_high_) + this->x_high_ = x; + if (y > this->y_high_) + this->y_high_ = y; +} +void MipiRgb::fill(Color color) { + if (!this->check_buffer_()) + return; + auto *ptr_16 = reinterpret_cast(this->buffer_); + uint8_t hi_byte = static_cast(color.r & 0xF8) | (color.g >> 5); + uint8_t lo_byte = static_cast((color.g & 0x1C) << 3) | (color.b >> 3); + uint16_t new_color = lo_byte | (hi_byte << 8); // little endian + std::fill_n(ptr_16, this->width_ * this->height_, new_color); +} + +int MipiRgb::get_width() { + switch (this->rotation_) { + case display::DISPLAY_ROTATION_90_DEGREES: + case display::DISPLAY_ROTATION_270_DEGREES: + return this->get_height_internal(); + case display::DISPLAY_ROTATION_0_DEGREES: + case display::DISPLAY_ROTATION_180_DEGREES: + default: + return this->get_width_internal(); + } +} + +int MipiRgb::get_height() { + switch (this->rotation_) { + case display::DISPLAY_ROTATION_0_DEGREES: + case display::DISPLAY_ROTATION_180_DEGREES: + return this->get_height_internal(); + case display::DISPLAY_ROTATION_90_DEGREES: + case display::DISPLAY_ROTATION_270_DEGREES: + default: + return this->get_width_internal(); + } +} + +static std::string get_pin_name(GPIOPin *pin) { + if (pin == nullptr) + return "None"; + return pin->dump_summary(); +} + +void MipiRgb::dump_pins_(uint8_t start, uint8_t end, const char *name, uint8_t offset) { + for (uint8_t i = start; i != end; i++) { + ESP_LOGCONFIG(TAG, " %s pin %d: %s", name, offset++, this->data_pins_[i]->dump_summary().c_str()); + } +} + +void MipiRgb::dump_config() { + ESP_LOGCONFIG(TAG, + "MIPI_RGB LCD" + "\n Model: %s" + "\n Width: %u" + "\n Height: %u" + "\n Rotation: %d degrees" + "\n HSync Pulse Width: %u" + "\n HSync Back Porch: %u" + "\n HSync Front Porch: %u" + "\n VSync Pulse Width: %u" + "\n VSync Back Porch: %u" + "\n VSync Front Porch: %u" + "\n Invert Colors: %s" + "\n Pixel Clock: %dMHz" + "\n Reset Pin: %s" + "\n DE Pin: %s" + "\n PCLK Pin: %s" + "\n HSYNC Pin: %s" + "\n VSYNC Pin: %s", + this->model_, this->width_, this->height_, this->rotation_, this->hsync_pulse_width_, + this->hsync_back_porch_, this->hsync_front_porch_, this->vsync_pulse_width_, this->vsync_back_porch_, + this->vsync_front_porch_, YESNO(this->invert_colors_), this->pclk_frequency_ / 1000000, + get_pin_name(this->reset_pin_).c_str(), get_pin_name(this->de_pin_).c_str(), + get_pin_name(this->pclk_pin_).c_str(), get_pin_name(this->hsync_pin_).c_str(), + get_pin_name(this->vsync_pin_).c_str()); + + if (this->madctl_ & MADCTL_BGR) { + this->dump_pins_(8, 13, "Blue", 0); + this->dump_pins_(13, 16, "Green", 0); + this->dump_pins_(0, 3, "Green", 3); + this->dump_pins_(3, 8, "Red", 0); + } else { + this->dump_pins_(8, 13, "Red", 0); + this->dump_pins_(13, 16, "Green", 0); + this->dump_pins_(0, 3, "Green", 3); + this->dump_pins_(3, 8, "Blue", 0); + } +} + +} // namespace mipi_rgb +} // namespace esphome +#endif // USE_ESP32_VARIANT_ESP32S3 diff --git a/esphome/components/mipi_rgb/mipi_rgb.h b/esphome/components/mipi_rgb/mipi_rgb.h new file mode 100644 index 0000000000..173e23752d --- /dev/null +++ b/esphome/components/mipi_rgb/mipi_rgb.h @@ -0,0 +1,127 @@ +#pragma once + +#ifdef USE_ESP32_VARIANT_ESP32S3 +#include "esphome/core/gpio.h" +#include "esphome/components/display/display.h" +#include "esp_lcd_panel_ops.h" +#ifdef USE_SPI +#include "esphome/components/spi/spi.h" +#endif + +namespace esphome { +namespace mipi_rgb { + +constexpr static const char *const TAG = "display.mipi_rgb"; +const uint8_t SW_RESET_CMD = 0x01; +const uint8_t SLEEP_OUT = 0x11; +const uint8_t SDIR_CMD = 0xC7; +const uint8_t MADCTL_CMD = 0x36; +const uint8_t INVERT_OFF = 0x20; +const uint8_t INVERT_ON = 0x21; +const uint8_t DISPLAY_ON = 0x29; +const uint8_t CMD2_BKSEL = 0xFF; +const uint8_t CMD2_BK0[5] = {0x77, 0x01, 0x00, 0x00, 0x10}; + +class MipiRgb : public display::Display { + public: + MipiRgb(int width, int height) : width_(width), height_(height) {} + void setup() override; + void loop() override; + void update() override; + void fill(Color color); + void draw_pixels_at(int x_start, int y_start, int w, int h, const uint8_t *ptr, display::ColorOrder order, + display::ColorBitness bitness, bool big_endian, int x_offset, int y_offset, int x_pad) override; + void write_to_display_(int x_start, int y_start, int w, int h, const uint8_t *ptr, int x_offset, int y_offset, + int x_pad); + bool check_buffer_(); + + display::ColorOrder get_color_mode() { return this->color_mode_; } + void set_color_mode(display::ColorOrder color_mode) { this->color_mode_ = color_mode; } + void set_invert_colors(bool invert_colors) { this->invert_colors_ = invert_colors; } + void set_madctl(uint8_t madctl) { this->madctl_ = madctl; } + + void add_data_pin(InternalGPIOPin *data_pin, size_t index) { this->data_pins_[index] = data_pin; }; + void set_de_pin(InternalGPIOPin *de_pin) { this->de_pin_ = de_pin; } + void set_pclk_pin(InternalGPIOPin *pclk_pin) { this->pclk_pin_ = pclk_pin; } + void set_vsync_pin(InternalGPIOPin *vsync_pin) { this->vsync_pin_ = vsync_pin; } + void set_hsync_pin(InternalGPIOPin *hsync_pin) { this->hsync_pin_ = hsync_pin; } + void set_reset_pin(GPIOPin *reset_pin) { this->reset_pin_ = reset_pin; } + void set_width(uint16_t width) { this->width_ = width; } + void set_pclk_frequency(uint32_t pclk_frequency) { this->pclk_frequency_ = pclk_frequency; } + void set_pclk_inverted(bool inverted) { this->pclk_inverted_ = inverted; } + void set_model(const char *model) { this->model_ = model; } + int get_width() override; + int get_height() override; + void set_hsync_back_porch(uint16_t hsync_back_porch) { this->hsync_back_porch_ = hsync_back_porch; } + void set_hsync_front_porch(uint16_t hsync_front_porch) { this->hsync_front_porch_ = hsync_front_porch; } + void set_hsync_pulse_width(uint16_t hsync_pulse_width) { this->hsync_pulse_width_ = hsync_pulse_width; } + void set_vsync_pulse_width(uint16_t vsync_pulse_width) { this->vsync_pulse_width_ = vsync_pulse_width; } + void set_vsync_back_porch(uint16_t vsync_back_porch) { this->vsync_back_porch_ = vsync_back_porch; } + void set_vsync_front_porch(uint16_t vsync_front_porch) { this->vsync_front_porch_ = vsync_front_porch; } + void set_enable_pins(std::vector enable_pins) { this->enable_pins_ = std::move(enable_pins); } + display::DisplayType get_display_type() override { return display::DisplayType::DISPLAY_TYPE_COLOR; } + int get_width_internal() override { return this->width_; } + int get_height_internal() override { return this->height_; } + void dump_pins_(uint8_t start, uint8_t end, const char *name, uint8_t offset); + void dump_config() override; + void draw_pixel_at(int x, int y, Color color) override; + + // this will be horribly slow. + protected: + void setup_enables_(); + void common_setup_(); + InternalGPIOPin *de_pin_{nullptr}; + InternalGPIOPin *pclk_pin_{nullptr}; + InternalGPIOPin *hsync_pin_{nullptr}; + InternalGPIOPin *vsync_pin_{nullptr}; + GPIOPin *reset_pin_{nullptr}; + InternalGPIOPin *data_pins_[16] = {}; + uint16_t hsync_pulse_width_ = 10; + uint16_t hsync_back_porch_ = 10; + uint16_t hsync_front_porch_ = 20; + uint16_t vsync_pulse_width_ = 10; + uint16_t vsync_back_porch_ = 10; + uint16_t vsync_front_porch_ = 10; + uint32_t pclk_frequency_ = 16 * 1000 * 1000; + bool pclk_inverted_{true}; + uint8_t madctl_{}; + const char *model_{"Unknown"}; + bool invert_colors_{}; + display::ColorOrder color_mode_{display::COLOR_ORDER_BGR}; + size_t width_; + size_t height_; + uint16_t *buffer_{nullptr}; + std::vector enable_pins_{}; + uint16_t x_low_{1}; + uint16_t y_low_{1}; + uint16_t x_high_{0}; + uint16_t y_high_{0}; + + esp_lcd_panel_handle_t handle_{}; +}; + +#ifdef USE_SPI +class MipiRgbSpi : public MipiRgb, + public spi::SPIDevice { + public: + MipiRgbSpi(int width, int height) : MipiRgb(width, height) {} + + void set_init_sequence(const std::vector &init_sequence) { this->init_sequence_ = init_sequence; } + void set_dc_pin(GPIOPin *dc_pin) { this->dc_pin_ = dc_pin; } + void setup() override; + + protected: + void write_command_(uint8_t value); + void write_data_(uint8_t value); + void write_init_sequence_(); + void dump_config(); + + GPIOPin *dc_pin_{nullptr}; + std::vector init_sequence_; +}; +#endif + +} // namespace mipi_rgb +} // namespace esphome +#endif diff --git a/esphome/components/mipi_rgb/models/guition.py b/esphome/components/mipi_rgb/models/guition.py new file mode 100644 index 0000000000..da433e686e --- /dev/null +++ b/esphome/components/mipi_rgb/models/guition.py @@ -0,0 +1,24 @@ +from .st7701s import st7701s + +st7701s.extend( + "GUITION-4848S040", + width=480, + height=480, + data_rate="2MHz", + cs_pin=39, + de_pin=18, + hsync_pin=16, + vsync_pin=17, + pclk_pin=21, + pclk_frequency="12MHz", + pixel_mode="18bit", + mirror_x=True, + mirror_y=True, + data_pins={ + "red": [11, 12, 13, 14, 0], + "green": [8, 20, 3, 46, 9, 10], + "blue": [4, 5, 6, 7, 15], + }, + # Additional configuration for Guition 4848S040, 16 bit bus config + add_init_sequence=((0xCD, 0x00),), +) diff --git a/esphome/components/mipi_rgb/models/lilygo.py b/esphome/components/mipi_rgb/models/lilygo.py new file mode 100644 index 0000000000..109dc42af6 --- /dev/null +++ b/esphome/components/mipi_rgb/models/lilygo.py @@ -0,0 +1,228 @@ +from esphome.config_validation import UNDEFINED + +from .st7701s import ST7701S + +# fmt: off +ST7701S( + "T-PANEL-S3", + width=480, + height=480, + color_order="BGR", + invert_colors=False, + swap_xy=UNDEFINED, + spi_mode="MODE3", + cs_pin={"xl9535": None, "number": 17}, + reset_pin={"xl9535": None, "number": 5}, + hsync_pin=39, + vsync_pin=40, + pclk_pin=41, + data_pins={ + "red": [12, 13, 42, 46, 45], + "green": [6, 7, 8, 9, 10, 11], + "blue": [1, 2, 3, 4, 5], + }, + hsync_front_porch=20, + hsync_back_porch=0, + hsync_pulse_width=2, + vsync_front_porch=30, + vsync_back_porch=1, + vsync_pulse_width=8, + pclk_frequency="6MHz", + pclk_inverted=False, + initsequence=( + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x13), (0xEF, 0x08), (0xFF, 0x77, 0x01, 0x00, 0x00, 0x10), + (0xC0, 0x3B, 0x00), (0xC1, 0x0B, 0x02), (0xC2, 0x30, 0x02, 0x37), (0xCC, 0x10), + (0xB0, 0x00, 0x0F, 0x16, 0x0E, 0x11, 0x07, 0x09, 0x09, 0x08, 0x23, 0x05, 0x11, 0x0F, 0x28, 0x2D, 0x18), + (0xB1, 0x00, 0x0F, 0x16, 0x0E, 0x11, 0x07, 0x09, 0x08, 0x09, 0x23, 0x05, 0x11, 0x0F, 0x28, 0x2D, 0x18), + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x11), + (0xB0, 0x4D), (0xB1, 0x33), (0xB2, 0x87), (0xB5, 0x4B), (0xB7, 0x8C), (0xB8, 0x20), (0xC1, 0x78), + (0xC2, 0x78), (0xD0, 0x88), (0xE0, 0x00, 0x00, 0x02), + (0xE1, 0x02, 0xF0, 0x00, 0x00, 0x03, 0xF0, 0x00, 0x00, 0x00, 0x44, 0x44), + (0xE2, 0x10, 0x10, 0x40, 0x40, 0xF2, 0xF0, 0x00, 0x00, 0xF2, 0xF0, 0x00, 0x00), + (0xE3, 0x00, 0x00, 0x11, 0x11), (0xE4, 0x44, 0x44), + (0xE5, 0x07, 0xEF, 0xF0, 0xF0, 0x09, 0xF1, 0xF0, 0xF0, 0x03, 0xF3, 0xF0, 0xF0, 0x05, 0xED, 0xF0, 0xF0), + (0xE6, 0x00, 0x00, 0x11, 0x11), (0xE7, 0x44, 0x44), + (0xE8, 0x08, 0xF0, 0xF0, 0xF0, 0x0A, 0xF2, 0xF0, 0xF0, 0x04, 0xF4, 0xF0, 0xF0, 0x06, 0xEE, 0xF0, 0xF0), + (0xEB, 0x00, 0x00, 0xE4, 0xE4, 0x44, 0x88, 0x40), + (0xEC, 0x78, 0x00), + (0xED, 0x20, 0xF9, 0x87, 0x76, 0x65, 0x54, 0x4F, 0xFF, 0xFF, 0xF4, 0x45, 0x56, 0x67, 0x78, 0x9F, 0x02), + (0xEF, 0x10, 0x0D, 0x04, 0x08, 0x3F, 0x1F), + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x10), + ), +) + + +t_rgb = ST7701S( + "T-RGB-2.1", + width=480, + height=480, + color_order="BGR", + pixel_mode="18bit", + invert_colors=False, + swap_xy=UNDEFINED, + spi_mode="MODE3", + cs_pin={"xl9535": None, "number": 3}, + de_pin=45, + hsync_pin=47, + vsync_pin=41, + pclk_pin=42, + data_pins={ + "red": [7, 6, 5, 3, 2], + "green": [14, 13, 12, 11, 10, 9], + "blue": [21, 18, 17, 16, 15], + }, + hsync_front_porch=50, + hsync_pulse_width=1, + hsync_back_porch=30, + vsync_front_porch=20, + vsync_pulse_width=1, + vsync_back_porch=30, + pclk_frequency="12MHz", + pclk_inverted=False, + initsequence=( + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x10), + + (0xC0, 0x3B, 0x00), + (0xC1, 0x0B, 0x02), + (0xC2, 0x07, 0x02), + (0xCC, 0x10), + (0xCD, 0x08), + + (0xB0, + 0x00, 0x11, 0x16, 0x0e, + 0x11, 0x06, 0x05, 0x09, + 0x08, 0x21, 0x06, 0x13, + 0x10, 0x29, 0x31, 0x18), + + (0xB1, + 0x00, 0x11, 0x16, 0x0e, + 0x11, 0x07, 0x05, 0x09, + 0x09, 0x21, 0x05, 0x13, + 0x11, 0x2a, 0x31, 0x18), + + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x11), + + (0xB0, 0x6D), + (0xB1, 0x37), + (0xB2, 0x81), + (0xB3, 0x80), + (0xB5, 0x43), + (0xB7, 0x85), + (0xB8, 0x20), + + (0xC1, 0x78), + (0xC2, 0x78), + (0xC3, 0x8C), + + (0xD0, 0x88), + + (0xE0, 0x00, 0x00, 0x02), + (0xE1, + 0x03, 0xA0, 0x00, 0x00, + 0x04, 0xA0, 0x00, 0x00, + 0x00, 0x20, 0x20), + + (0xE2, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, + 0x00), + + (0xE3, 0x00, 0x00, 0x11, 0x00), + (0xE4, 0x22, 0x00), + + (0xE5, + 0x05, 0xEC, 0xA0, 0xA0, + 0x07, 0xEE, 0xA0, 0xA0, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00), + + (0xE6, 0x00, 0x00, 0x11, 0x00), + (0xE7, 0x22, 0x00), + + (0xE8, + 0x06, 0xED, 0xA0, 0xA0, + 0x08, 0xEF, 0xA0, 0xA0, + 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00), + + (0xEB, 0x00, 0x00, 0x40, 0x40, 0x00, 0x00, 0x10), + + (0xED, + 0xFF, 0xFF, 0xFF, 0xBA, + 0x0A, 0xBF, 0x45, 0xFF, + 0xFF, 0x54, 0xFB, 0xA0, + 0xAB, 0xFF, 0xFF, 0xFF), + + (0xEF, 0x10, 0x0D, 0x04, 0x08, 0x3F, 0x1F), + + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x13), + (0xEF, 0x08), + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x10) + ) + +) +t_rgb.extend( + "T-RGB-2.8", + initsequence=( + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x13), + (0xEF, 0x08), + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x10), + (0xC0, 0x3B, 0x00), + (0xC1, 0x10, 0x0C), + (0xC2, 0x07, 0x0A), + (0xC7, 0x00), + (0xC7, 0x10), + (0xCD, 0x08), + (0xB0, + 0x05, 0x12, 0x98, 0x0e, 0x0F, + 0x07, 0x07, 0x09, 0x09, 0x23, + 0x05, 0x52, 0x0F, 0x67, 0x2C, 0x11), + (0xB1, + 0x0B, 0x11, 0x97, 0x0C, 0x12, + 0x06, 0x06, 0x08, 0x08, 0x22, + 0x03, 0x51, 0x11, 0x66, 0x2B, 0x0F), + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x11), + (0xB0, 0x5D), + (0xB1, 0x2D), + (0xB2, 0x81), + (0xB3, 0x80), + (0xB5, 0x4E), + (0xB7, 0x85), + (0xB8, 0x20), + (0xC1, 0x78), + (0xC2, 0x78), + (0xD0, 0x88), + (0xE0, 0x00, 0x00, 0x02), + (0xE1, + 0x06, 0x30, 0x08, 0x30, 0x05, + 0x30, 0x07, 0x30, 0x00, 0x33, + 0x33), + (0xE2, + 0x11, 0x11, 0x33, 0x33, 0xf4, + 0x00, 0x00, 0x00, 0xf4, 0x00, + 0x00, 0x00), + (0xE3, 0x00, 0x00, 0x11, 0x11), + (0xE4, 0x44, 0x44), + (0xE5, + 0x0d, 0xf5, 0x30, 0xf0, 0x0f, + 0xf7, 0x30, 0xf0, 0x09, 0xf1, + 0x30, 0xf0, 0x0b, 0xf3, 0x30, 0xf0), + (0xE6, 0x00, 0x00, 0x11, 0x11), + (0xE7, 0x44, 0x44), + (0xE8, + 0x0c, 0xf4, 0x30, 0xf0, + 0x0e, 0xf6, 0x30, 0xf0, + 0x08, 0xf0, 0x30, 0xf0, + 0x0a, 0xf2, 0x30, 0xf0), + (0xe9, 0x36), + (0xEB, 0x00, 0x01, 0xe4, 0xe4, 0x44, 0x88, 0x40), + (0xED, + 0xff, 0x10, 0xaf, 0x76, + 0x54, 0x2b, 0xcf, 0xff, + 0xff, 0xfc, 0xb2, 0x45, + 0x67, 0xfa, 0x01, 0xff), + (0xEF, 0x08, 0x08, 0x08, 0x45, 0x3f, 0x54), + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x10), + ) +) diff --git a/esphome/components/mipi_rgb/models/rpi.py b/esphome/components/mipi_rgb/models/rpi.py new file mode 100644 index 0000000000..076d96b658 --- /dev/null +++ b/esphome/components/mipi_rgb/models/rpi.py @@ -0,0 +1,9 @@ +from esphome.components.mipi import DriverChip +from esphome.config_validation import UNDEFINED + +# A driver chip for Raspberry Pi MIPI RGB displays. These require no init sequence +DriverChip( + "RPI", + swap_xy=UNDEFINED, + initsequence=(), +) diff --git a/esphome/components/mipi_rgb/models/st7701s.py b/esphome/components/mipi_rgb/models/st7701s.py new file mode 100644 index 0000000000..bfd1c9aa3f --- /dev/null +++ b/esphome/components/mipi_rgb/models/st7701s.py @@ -0,0 +1,214 @@ +from esphome.components.mipi import ( + MADCTL, + MADCTL_ML, + MADCTL_XFLIP, + MODE_BGR, + DriverChip, +) +from esphome.config_validation import UNDEFINED +from esphome.const import CONF_COLOR_ORDER, CONF_HEIGHT, CONF_MIRROR_X, CONF_MIRROR_Y + +SDIR_CMD = 0xC7 + + +class ST7701S(DriverChip): + # The ST7701s does not use the standard MADCTL bits for x/y mirroring + def add_madctl(self, sequence: list, config: dict): + transform = self.get_transform(config) + madctl = 0x00 + if config[CONF_COLOR_ORDER] == MODE_BGR: + madctl |= 0x08 + if transform.get(CONF_MIRROR_Y): + madctl |= MADCTL_ML + sequence.append((MADCTL, madctl)) + sdir = 0 + if transform.get(CONF_MIRROR_X): + sdir |= 0x04 + madctl |= MADCTL_XFLIP + sequence.append((SDIR_CMD, sdir)) + return madctl + + @property + def transforms(self) -> set[str]: + """ + The ST7701 never supports axis swapping, and mirroring the y-axis only works for full height. + """ + if self.get_default(CONF_HEIGHT) != 864: + return {CONF_MIRROR_X} + return {CONF_MIRROR_X, CONF_MIRROR_Y} + + +# fmt: off +st7701s = ST7701S( + "ST7701S", + width=480, + height=864, + swap_xy=UNDEFINED, + hsync_front_porch=20, + hsync_back_porch=10, + hsync_pulse_width=10, + vsync_front_porch=10, + vsync_back_porch=10, + vsync_pulse_width=10, + pclk_frequency="16MHz", + pclk_inverted=True, + initsequence=( + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x10), # Page 0 + (0xC0, 0x3B, 0x00), (0xC1, 0x0D, 0x02), (0xC2, 0x31, 0x05), + (0xB0, 0x00, 0x11, 0x18, 0x0E, 0x11, 0x06, 0x07, 0x08, 0x07, 0x22, 0x04, 0x12, 0x0F, 0xAA, 0x31, 0x18,), + (0xB1, 0x00, 0x11, 0x19, 0x0E, 0x12, 0x07, 0x08, 0x08, 0x08, 0x22, 0x04, 0x11, 0x11, 0xA9, 0x32, 0x18,), + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x11), # page 1 + (0xB0, 0x60), (0xB1, 0x32), (0xB2, 0x07), (0xB3, 0x80), (0xB5, 0x49), (0xB7, 0x85), (0xB8, 0x21), (0xC1, 0x78), + (0xC2, 0x78), (0xE0, 0x00, 0x1B, 0x02), + (0xE1, 0x08, 0xA0, 0x00, 0x00, 0x07, 0xA0, 0x00, 0x00, 0x00, 0x44, 0x44), + (0xE2, 0x11, 0x11, 0x44, 0x44, 0xED, 0xA0, 0x00, 0x00, 0xEC, 0xA0, 0x00, 0x00), + (0xE3, 0x00, 0x00, 0x11, 0x11), + (0xE4, 0x44, 0x44), + (0xE5, 0x0A, 0xE9, 0xD8, 0xA0, 0x0C, 0xEB, 0xD8, 0xA0, 0x0E, 0xED, 0xD8, 0xA0, 0x10, 0xEF, 0xD8, 0xA0,), + (0xE6, 0x00, 0x00, 0x11, 0x11), (0xE7, 0x44, 0x44), + (0xE8, 0x09, 0xE8, 0xD8, 0xA0, 0x0B, 0xEA, 0xD8, 0xA0, 0x0D, 0xEC, 0xD8, 0xA0, 0x0F, 0xEE, 0xD8, 0xA0,), + (0xEB, 0x02, 0x00, 0xE4, 0xE4, 0x88, 0x00, 0x40), (0xEC, 0x3C, 0x00), + (0xED, 0xAB, 0x89, 0x76, 0x54, 0x02, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0x20, 0x45, 0x67, 0x98, 0xBA,), + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x13), # Page 3 + (0xE5, 0xE4), + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x10), # Page 0 + (0xCD, 0x08), + ) +) + +st7701s.extend( + "MAKERFABS-4", + width=480, + height=480, + color_order="RGB", + invert_colors=True, + pixel_mode="18bit", + cs_pin=1, + de_pin={ + "number": 45, + "ignore_strapping_warning": True + }, + hsync_pin=5, + vsync_pin=4, + pclk_pin=21, + data_pins={ + "red": [39, 40, 41, 42, 2], + "green": [0, 9, 14, 47, 48, 3], + "blue": [6, 7, 15, 16, 8] + } +) + +st7701s.extend( + "SEEED-INDICATOR-D1", + width=480, + height=480, + mirror_x=True, + mirror_y=True, + invert_colors=True, + pixel_mode="18bit", + spi_mode="MODE3", + data_rate="2MHz", + hsync_front_porch=10, + hsync_pulse_width=8, + hsync_back_porch=50, + vsync_front_porch=10, + vsync_pulse_width=8, + vsync_back_porch=20, + cs_pin={"pca9554": None, "number": 4}, + de_pin=18, + hsync_pin=16, + vsync_pin=17, + pclk_pin=21, + pclk_inverted=False, + data_pins={ + "red": [4, 3, 2, 1, 0], + "green": [10, 9, 8, 7, 6, 5], + "blue": [15, 14, 13, 12, 11] + }, +) + +st7701s.extend( + "UEDX48480021-MD80ET", + width=480, + height=480, + pixel_mode="18bit", + cs_pin=18, + reset_pin=8, + de_pin=17, + vsync_pin={"number": 3, "ignore_strapping_warning": True}, + hsync_pin={"number": 46, "ignore_strapping_warning": True}, + pclk_pin=9, + data_pins={ + "red": [40, 41, 42, 2, 1], + "green": [21, 47, 48, 45, 38, 39], + "blue": [10, 11, {"number": 12, "allow_other_uses": True}, {"number": 13, "allow_other_uses": True}, 14] + }, + initsequence=( + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x13), (0xEF, 0x08), (0xFF, 0x77, 0x01, 0x00, 0x00, 0x10), + (0xC0, 0x3B, 0x00), (0xC1, 0x0B, 0x02), (0xC2, 0x07, 0x02), (0xC7, 0x00), (0xCC, 0x10), (0xCD, 0x08), + (0xB0, 0x00, 0x11, 0x16, 0x0E, 0x11, 0x06, 0x05, 0x09, 0x08, 0x21, 0x06, 0x13, 0x10, 0x29, 0x31, 0x18), + (0xB1, 0x00, 0x11, 0x16, 0x0E, 0x11, 0x07, 0x05, 0x09, 0x09, 0x21, 0x05, 0x13, 0x11, 0x2A, 0x31, 0x18), + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x11), + (0xB0, 0x6D), (0xB1, 0x37), (0xB2, 0x8B), (0xB3, 0x80), (0xB5, 0x43), (0xB7, 0x85), + (0xB8, 0x20), (0xC0, 0x09), (0xC1, 0x78), (0xC2, 0x78), (0xD0, 0x88), + (0xE0, 0x00, 0x00, 0x02), + (0xE1, 0x03, 0xA0, 0x00, 0x00, 0x04, 0xA0, 0x00, 0x00, 0x00, 0x20, 0x20), + (0xE2, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00), + (0xE3, 0x00, 0x00, 0x11, 0x00), + (0xE4, 0x22, 0x00), + (0xE5, 0x05, 0xEC, 0xF6, 0xCA, 0x07, 0xEE, 0xF6, 0xCA, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00), + (0xE6, 0x00, 0x00, 0x11, 0x00), + (0xE7, 0x22, 0x00), + (0xE8, 0x06, 0xED, 0xF6, 0xCA, 0x08, 0xEF, 0xF6, 0xCA, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00), + (0xE9, 0x36, 0x00), + (0xEB, 0x00, 0x00, 0x40, 0x40, 0x00, 0x00, 0x00), + (0xED, 0xFF, 0xFF, 0xFF, 0xBA, 0x0A, 0xFF, 0x45, 0xFF, 0xFF, 0x54, 0xFF, 0xA0, 0xAB, 0xFF, 0xFF, 0xFF), + (0xEF, 0x08, 0x08, 0x08, 0x45, 0x3F, 0x54), + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x13), (0xE8, 0x00, 0x0E), (0xFF, 0x77, 0x01, 0x00, 0x00, 0x00), + (0x11, 0x00), (0xFF, 0x77, 0x01, 0x00, 0x00, 0x13), (0xE8, 0x00, 0x0C), + (0xE8, 0x00, 0x00), (0xFF, 0x77, 0x01, 0x00, 0x00, 0x00) + ) +) + +st7701s.extend( + "ZX2D10GE01R-V4848", + width=480, + height=480, + pixel_mode="18bit", + cs_pin=21, + de_pin=39, + vsync_pin=48, + hsync_pin=40, + pclk_pin={"number": 45, "ignore_strapping_warning": True}, + pclk_frequency="15MHz", + pclk_inverted=True, + hsync_pulse_width=10, + hsync_back_porch=10, + hsync_front_porch=10, + vsync_pulse_width=2, + vsync_back_porch=12, + vsync_front_porch=14, + data_pins={ + "red": [10, 16, 9, 15, 46], + "green": [8, 13, 18, 12, 11, 17], + "blue": [{"number": 47, "allow_other_uses": True}, {"number": 41, "allow_other_uses": True}, 0, 42, 14] + }, + initsequence=( + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x13), (0xEF, 0x08), (0xFF, 0x77, 0x01, 0x00, 0x00, 0x10), (0xC0, 0x3B, 0x00), + (0xC1, 0x0B, 0x02), (0xC2, 0x07, 0x02), (0xCC, 0x10), (0xCD, 0x08), + (0xB0, 0x00, 0x11, 0x16, 0x0e, 0x11, 0x06, 0x05, 0x09, 0x08, 0x21, 0x06, 0x13, 0x10, 0x29, 0x31, 0x18), + (0xB1, 0x00, 0x11, 0x16, 0x0e, 0x11, 0x07, 0x05, 0x09, 0x09, 0x21, 0x05, 0x13, 0x11, 0x2a, 0x31, 0x18), + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x11), (0xB0, 0x6d), (0xB1, 0x37), (0xB2, 0x81), (0xB3, 0x80), (0xB5, 0x43), + (0xB7, 0x85), (0xB8, 0x20), (0xC1, 0x78), (0xC2, 0x78), (0xD0, 0x88), (0xE0, 0x00, 0x00, 0x02), + (0xE1, 0x03, 0xA0, 0x00, 0x00, 0x04, 0xA0, 0x00, 0x00, 0x00, 0x20, 0x20), + (0xE2, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00), + (0xE3, 0x00, 0x00, 0x11, 0x00), (0xE4, 0x22, 0x00), + (0xE5, 0x05, 0xEC, 0xA0, 0xA0, 0x07, 0xEE, 0xA0, 0xA0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00), + (0xE6, 0x00, 0x00, 0x11, 0x00), (0xE7, 0x22, 0x00), + (0xE8, 0x06, 0xED, 0xA0, 0xA0, 0x08, 0xEF, 0xA0, 0xA0, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00), + (0xEB, 0x00, 0x00, 0x40, 0x40, 0x00, 0x00, 0x00), + (0xED, 0xFF, 0xFF, 0xFF, 0xBA, 0x0A, 0xBF, 0x45, 0xFF, 0xFF, 0x54, 0xFB, 0xA0, 0xAB, 0xFF, 0xFF, 0xFF), + (0xEF, 0x10, 0x0D, 0x04, 0x08, 0x3F, 0x1F), + (0xFF, 0x77, 0x01, 0x00, 0x00, 0x13), (0xEF, 0x08), (0xFF, 0x77, 0x01, 0x00, 0x00, 0x00) + ) +) diff --git a/esphome/components/mipi_rgb/models/waveshare.py b/esphome/components/mipi_rgb/models/waveshare.py new file mode 100644 index 0000000000..a38493e816 --- /dev/null +++ b/esphome/components/mipi_rgb/models/waveshare.py @@ -0,0 +1,65 @@ +from esphome.components.mipi import DriverChip +from esphome.config_validation import UNDEFINED + +from .st7701s import st7701s + +wave_4_3 = DriverChip( + "ESP32-S3-TOUCH-LCD-4.3", + swap_xy=UNDEFINED, + initsequence=(), + color_order="RGB", + width=800, + height=480, + pclk_frequency="16MHz", + reset_pin={"ch422g": None, "number": 3}, + enable_pin={"ch422g": None, "number": 2}, + de_pin=5, + hsync_pin={"number": 46, "ignore_strapping_warning": True}, + vsync_pin={"number": 3, "ignore_strapping_warning": True}, + pclk_pin=7, + pclk_inverted=True, + hsync_front_porch=210, + hsync_pulse_width=30, + hsync_back_porch=30, + vsync_front_porch=4, + vsync_pulse_width=4, + vsync_back_porch=4, + data_pins={ + "red": [1, 2, 42, 41, 40], + "green": [39, 0, 45, 48, 47, 21], + "blue": [14, 38, 18, 17, 10], + }, +) +wave_4_3.extend( + "ESP32-S3-TOUCH-LCD-7-800X480", + enable_pin=[{"ch422g": None, "number": 2}, {"ch422g": None, "number": 6}], + hsync_back_porch=8, + hsync_front_porch=8, + hsync_pulse_width=4, + vsync_back_porch=16, + vsync_front_porch=16, + vsync_pulse_width=4, +) + +st7701s.extend( + "WAVESHARE-4-480x480", + data_rate="2MHz", + spi_mode="MODE3", + color_order="BGR", + pixel_mode="18bit", + width=480, + height=480, + invert_colors=True, + cs_pin=42, + de_pin=40, + hsync_pin=38, + vsync_pin=39, + pclk_pin=41, + pclk_frequency="12MHz", + pclk_inverted=False, + data_pins={ + "red": [46, 3, 8, 18, 17], + "green": [14, 13, 12, 11, 10, 9], + "blue": [5, 45, 48, 47, 21], + }, +) diff --git a/esphome/components/mipi_spi/__init__.py b/esphome/components/mipi_spi/__init__.py index 46b0206a1f..f0f02aedd8 100644 --- a/esphome/components/mipi_spi/__init__.py +++ b/esphome/components/mipi_spi/__init__.py @@ -2,14 +2,5 @@ CODEOWNERS = ["@clydebarrow"] DOMAIN = "mipi_spi" -CONF_DRAW_FROM_ORIGIN = "draw_from_origin" CONF_SPI_16 = "spi_16" -CONF_PIXEL_MODE = "pixel_mode" -CONF_COLOR_DEPTH = "color_depth" CONF_BUS_MODE = "bus_mode" -CONF_USE_AXIS_FLIPS = "use_axis_flips" -CONF_NATIVE_WIDTH = "native_width" -CONF_NATIVE_HEIGHT = "native_height" - -MODE_RGB = "RGB" -MODE_BGR = "BGR" diff --git a/esphome/components/mipi_spi/display.py b/esphome/components/mipi_spi/display.py index 061257e859..891c8b42ff 100644 --- a/esphome/components/mipi_spi/display.py +++ b/esphome/components/mipi_spi/display.py @@ -1,20 +1,44 @@ +import importlib import logging +import pkgutil from esphome import pins import esphome.codegen as cg from esphome.components import display, spi +from esphome.components.const import ( + CONF_BYTE_ORDER, + CONF_COLOR_DEPTH, + CONF_DRAW_ROUNDING, +) +from esphome.components.display import CONF_SHOW_TEST_CARD, DISPLAY_ROTATIONS +from esphome.components.mipi import ( + CONF_PIXEL_MODE, + CONF_USE_AXIS_FLIPS, + MADCTL, + MODE_BGR, + MODE_RGB, + PIXFMT, + DriverChip, + dimension_schema, + get_color_depth, + map_sequence, + power_of_two, + requires_buffer, +) +from esphome.components.psram import DOMAIN as PSRAM_DOMAIN from esphome.components.spi import TYPE_OCTAL, TYPE_QUAD, TYPE_SINGLE import esphome.config_validation as cv from esphome.config_validation import ALLOW_EXTRA from esphome.const import ( CONF_BRIGHTNESS, + CONF_BUFFER_SIZE, CONF_COLOR_ORDER, CONF_CS_PIN, CONF_DATA_RATE, CONF_DC_PIN, CONF_DIMENSIONS, + CONF_DISABLED, CONF_ENABLE_PIN, - CONF_HEIGHT, CONF_ID, CONF_INIT_SEQUENCE, CONF_INVERT_COLORS, @@ -22,177 +46,142 @@ from esphome.const import ( CONF_MIRROR_X, CONF_MIRROR_Y, CONF_MODEL, - CONF_OFFSET_HEIGHT, - CONF_OFFSET_WIDTH, CONF_RESET_PIN, CONF_ROTATION, CONF_SWAP_XY, CONF_TRANSFORM, CONF_WIDTH, ) -from esphome.core import TimePeriod +from esphome.core import CORE +from esphome.cpp_generator import TemplateArguments +from esphome.final_validate import full_config -from ..const import CONF_DRAW_ROUNDING -from ..lvgl.defines import CONF_COLOR_DEPTH -from . import ( - CONF_BUS_MODE, - CONF_DRAW_FROM_ORIGIN, - CONF_NATIVE_HEIGHT, - CONF_NATIVE_WIDTH, - CONF_PIXEL_MODE, - CONF_SPI_16, - CONF_USE_AXIS_FLIPS, - DOMAIN, - MODE_BGR, - MODE_RGB, -) -from .models import ( - DELAY_FLAG, - MADCTL_BGR, - MADCTL_MV, - MADCTL_MX, - MADCTL_MY, - MADCTL_XFLIP, - MADCTL_YFLIP, - DriverChip, - amoled, - cyd, - ili, - jc, - lanbon, - lilygo, - waveshare, -) -from .models.commands import BRIGHTNESS, DISPON, INVOFF, INVON, MADCTL, PIXFMT, SLPOUT +from . import CONF_BUS_MODE, CONF_SPI_16, DOMAIN, models DEPENDENCIES = ["spi"] LOGGER = logging.getLogger(DOMAIN) mipi_spi_ns = cg.esphome_ns.namespace("mipi_spi") -MipiSpi = mipi_spi_ns.class_( - "MipiSpi", display.Display, display.DisplayBuffer, cg.Component, spi.SPIDevice +MipiSpi = mipi_spi_ns.class_("MipiSpi", display.Display, cg.Component, spi.SPIDevice) +MipiSpiBuffer = mipi_spi_ns.class_( + "MipiSpiBuffer", MipiSpi, display.Display, cg.Component, spi.SPIDevice ) ColorOrder = display.display_ns.enum("ColorMode") ColorBitness = display.display_ns.enum("ColorBitness") Model = mipi_spi_ns.enum("Model") +PixelMode = mipi_spi_ns.enum("PixelMode") +BusType = mipi_spi_ns.enum("BusType") + COLOR_ORDERS = { MODE_RGB: ColorOrder.COLOR_ORDER_RGB, MODE_BGR: ColorOrder.COLOR_ORDER_BGR, } COLOR_DEPTHS = { - 8: ColorBitness.COLOR_BITNESS_332, - 16: ColorBitness.COLOR_BITNESS_565, + 8: PixelMode.PIXEL_MODE_8, + 16: PixelMode.PIXEL_MODE_16, + 18: PixelMode.PIXEL_MODE_18, } + DATA_PIN_SCHEMA = pins.internal_gpio_output_pin_schema +BusTypes = { + TYPE_SINGLE: BusType.BUS_TYPE_SINGLE, + TYPE_QUAD: BusType.BUS_TYPE_QUAD, + TYPE_OCTAL: BusType.BUS_TYPE_OCTAL, +} -DriverChip("CUSTOM", initsequence={}) +DriverChip("CUSTOM") -MODELS = DriverChip.models -# These statements are noops, but serve to suppress linting of side-effect-only imports -for _ in (ili, jc, amoled, lilygo, lanbon, cyd, waveshare): - pass +# Import all models dynamically from the models package +for module_info in pkgutil.iter_modules(models.__path__): + importlib.import_module(f".models.{module_info.name}", package=__package__) -PixelMode = mipi_spi_ns.enum("PixelMode") +MODELS = DriverChip.get_models() -PIXEL_MODE_18BIT = "18bit" -PIXEL_MODE_16BIT = "16bit" -PIXEL_MODES = { - PIXEL_MODE_16BIT: 0x55, - PIXEL_MODE_18BIT: 0x66, +DISPLAY_18BIT = "18bit" +DISPLAY_16BIT = "16bit" + +DISPLAY_PIXEL_MODES = { + DISPLAY_16BIT: (0x55, PixelMode.PIXEL_MODE_16), + DISPLAY_18BIT: (0x66, PixelMode.PIXEL_MODE_18), } -def validate_dimension(rounding): +def denominator(config): + """ + Calculate the best denominator for a buffer size fraction. + The denominator must be a number between 2 and 16 that divides the display height evenly, + and the fraction represented by the denominator must be less than or equal to the given fraction. + :config: The configuration dictionary containing the buffer size fraction and display dimensions + :return: The denominator to use for the buffer size fraction + """ + model = MODELS[config[CONF_MODEL]] + frac = config.get(CONF_BUFFER_SIZE) + if frac is None or frac > 0.75: + return 1 + height, _width, _offset_width, _offset_height = model.get_dimensions(config) + try: + return next(x for x in range(2, 17) if frac >= 1 / x and height % x == 0) + except StopIteration: + raise cv.Invalid( + f"Buffer size fraction {frac} is not compatible with display height {height}" + ) from StopIteration + + +def swap_xy_schema(model): + uses_swap = model.get_default(CONF_SWAP_XY, None) != cv.UNDEFINED + def validator(value): - value = cv.positive_int(value) - if value % rounding != 0: - raise cv.Invalid(f"Dimensions and offsets must be divisible by {rounding}") - return value + if value: + raise cv.Invalid("Axis swapping not supported by this model") + return cv.boolean(value) - return validator + if uses_swap: + return {cv.Required(CONF_SWAP_XY): cv.boolean} + return {cv.Optional(CONF_SWAP_XY, default=False): validator} -def map_sequence(value): - """ - The format is a repeated sequence of [CMD, ] where is s a sequence of bytes. The length is inferred - from the length of the sequence and should not be explicit. - A delay can be inserted by specifying "- delay N" where N is in ms - """ - if isinstance(value, str) and value.lower().startswith("delay "): - value = value.lower()[6:] - delay = cv.All( - cv.positive_time_period_milliseconds, - cv.Range(TimePeriod(milliseconds=1), TimePeriod(milliseconds=255)), - )(value) - return DELAY_FLAG, delay.total_milliseconds - if isinstance(value, int): - return (value,) - value = cv.All(cv.ensure_list(cv.int_range(0, 255)), cv.Length(1, 254))(value) - return tuple(value) - - -def power_of_two(value): - value = cv.int_range(1, 128)(value) - if value & (value - 1) != 0: - raise cv.Invalid("value must be a power of two") - return value - - -def dimension_schema(rounding): - return cv.Any( - cv.dimensions, +def model_schema(config): + model = MODELS[config[CONF_MODEL]] + bus_mode = config[CONF_BUS_MODE] + transform = cv.Any( cv.Schema( { - cv.Required(CONF_WIDTH): validate_dimension(rounding), - cv.Required(CONF_HEIGHT): validate_dimension(rounding), - cv.Optional(CONF_OFFSET_HEIGHT, default=0): validate_dimension( - rounding - ), - cv.Optional(CONF_OFFSET_WIDTH, default=0): validate_dimension(rounding), + cv.Required(CONF_MIRROR_X): cv.boolean, + cv.Required(CONF_MIRROR_Y): cv.boolean, + **swap_xy_schema(model), } ), + cv.one_of(CONF_DISABLED, lower=True), ) - - -def model_schema(bus_mode, model: DriverChip, swapsies: bool): - transform = cv.Schema( - { - cv.Required(CONF_MIRROR_X): cv.boolean, - cv.Required(CONF_MIRROR_Y): cv.boolean, - } - ) - if model.get_default(CONF_SWAP_XY, False) == cv.UNDEFINED: - transform = transform.extend( - { - cv.Optional(CONF_SWAP_XY): cv.invalid( - "Axis swapping not supported by this model" - ) - } - ) - else: - transform = transform.extend( - { - cv.Required(CONF_SWAP_XY): cv.boolean, - } - ) # CUSTOM model will need to provide a custom init sequence iseqconf = ( cv.Required(CONF_INIT_SEQUENCE) if model.initsequence is None else cv.Optional(CONF_INIT_SEQUENCE) ) - # Dimensions are optional if the model has a default width and the transform is not overridden - cv_dimensions = ( - cv.Optional if model.get_default(CONF_WIDTH) and not swapsies else cv.Required + # Dimensions are optional if the model has a default width and the x-y transform is not overridden + transform_config = config.get(CONF_TRANSFORM, {}) + is_swapped = ( + isinstance(transform_config, dict) + and transform_config.get(CONF_SWAP_XY, False) is True ) - pixel_modes = PIXEL_MODES if bus_mode == TYPE_SINGLE else (PIXEL_MODE_16BIT,) + cv_dimensions = ( + cv.Optional if model.get_default(CONF_WIDTH) and not is_swapped else cv.Required + ) + pixel_modes = DISPLAY_PIXEL_MODES if bus_mode == TYPE_SINGLE else (DISPLAY_16BIT,) color_depth = ( ("16", "8", "16bit", "8bit") if bus_mode == TYPE_SINGLE else ("16", "16bit") ) + other_options = [ + CONF_INVERT_COLORS, + CONF_USE_AXIS_FLIPS, + ] + if bus_mode == TYPE_SINGLE: + other_options.append(CONF_SPI_16) schema = ( display.FULL_DISPLAY_SCHEMA.extend( spi.spi_device_schema( @@ -211,20 +200,20 @@ def model_schema(bus_mode, model: DriverChip, swapsies: bool): .extend( { cv.GenerateID(): cv.declare_id(MipiSpi), - cv_dimensions(CONF_DIMENSIONS): dimension_schema( - model.get_default(CONF_DRAW_ROUNDING, 1) - ), + cv_dimensions(CONF_DIMENSIONS): dimension_schema(1), model.option(CONF_ENABLE_PIN, cv.UNDEFINED): cv.ensure_list( pins.gpio_output_pin_schema ), model.option(CONF_COLOR_ORDER, MODE_BGR): cv.enum( COLOR_ORDERS, upper=True ), + model.option(CONF_BYTE_ORDER, "big_endian"): cv.one_of( + "big_endian", "little_endian", lower=True + ), model.option(CONF_COLOR_DEPTH, 16): cv.one_of(*color_depth, lower=True), model.option(CONF_DRAW_ROUNDING, 2): power_of_two, - model.option(CONF_PIXEL_MODE, PIXEL_MODE_16BIT): cv.Any( - cv.one_of(*pixel_modes, lower=True), - cv.int_range(0, 255, min_included=True, max_included=True), + model.option(CONF_PIXEL_MODE, DISPLAY_16BIT): cv.one_of( + *pixel_modes, lower=True ), cv.Optional(CONF_TRANSFORM): transform, cv.Optional(CONF_BUS_MODE, default=bus_mode): cv.one_of( @@ -232,19 +221,12 @@ def model_schema(bus_mode, model: DriverChip, swapsies: bool): ), cv.Required(CONF_MODEL): cv.one_of(model.name, upper=True), iseqconf: cv.ensure_list(map_sequence), + cv.Optional(CONF_BUFFER_SIZE): cv.All( + cv.percentage, cv.Range(0.12, 1.0) + ), } ) - .extend( - { - model.option(x): cv.boolean - for x in [ - CONF_DRAW_FROM_ORIGIN, - CONF_SPI_16, - CONF_INVERT_COLORS, - CONF_USE_AXIS_FLIPS, - ] - } - ) + .extend({model.option(x): cv.boolean for x in other_options}) ) if brightness := model.get_default(CONF_BRIGHTNESS): schema = schema.extend( @@ -259,18 +241,13 @@ def model_schema(bus_mode, model: DriverChip, swapsies: bool): return schema -def rotation_as_transform(model, config): +def customise_schema(config): """ - Check if a rotation can be implemented in hardware using the MADCTL register. - A rotation of 180 is always possible, 90 and 270 are possible if the model supports swapping X and Y. + Create a customised config schema for a specific model and validate the configuration. + :param config: The configuration dictionary to validate + :return: The validated configuration dictionary + :raises cv.Invalid: If the configuration is invalid """ - rotation = config.get(CONF_ROTATION, 0) - return rotation and ( - model.get_default(CONF_SWAP_XY) != cv.UNDEFINED or rotation == 180 - ) - - -def config_schema(config): # First get the model and bus mode config = cv.Schema( { @@ -279,7 +256,7 @@ def config_schema(config): extra=ALLOW_EXTRA, )(config) model = MODELS[config[CONF_MODEL]] - bus_modes = model.modes + bus_modes = (TYPE_SINGLE, TYPE_QUAD, TYPE_OCTAL) config = cv.Schema( { model.option(CONF_BUS_MODE, TYPE_SINGLE): cv.one_of(*bus_modes, lower=True), @@ -287,30 +264,81 @@ def config_schema(config): }, extra=ALLOW_EXTRA, )(config) - bus_mode = config.get(CONF_BUS_MODE, model.modes[0]) - swapsies = config.get(CONF_TRANSFORM, {}).get(CONF_SWAP_XY) is True - config = model_schema(bus_mode, model, swapsies)(config) + bus_mode = config[CONF_BUS_MODE] + config = model_schema(config)(config) # Check for invalid combinations of MADCTL config if init_sequence := config.get(CONF_INIT_SEQUENCE): - if MADCTL in [x[0] for x in init_sequence] and CONF_TRANSFORM in config: + commands = [x[0] for x in init_sequence] + if MADCTL in commands and CONF_TRANSFORM in config: raise cv.Invalid( f"transform is not supported when MADCTL ({MADCTL:#X}) is in the init sequence" ) + if PIXFMT in commands: + raise cv.Invalid( + f"PIXFMT ({PIXFMT:#X}) should not be in the init sequence, it will be set automatically" + ) if bus_mode == TYPE_QUAD and CONF_DC_PIN in config: raise cv.Invalid("DC pin is not supported in quad mode") - if config[CONF_PIXEL_MODE] == PIXEL_MODE_18BIT and bus_mode != TYPE_SINGLE: - raise cv.Invalid("18-bit pixel mode is not supported on a quad or octal bus") if bus_mode != TYPE_QUAD and CONF_DC_PIN not in config: raise cv.Invalid(f"DC pin is required in {bus_mode} mode") + denominator(config) return config -CONFIG_SCHEMA = config_schema +CONFIG_SCHEMA = customise_schema -def get_transform(model, config): - can_transform = rotation_as_transform(model, config) +def _final_validate(config): + global_config = full_config.get() + model = MODELS[config[CONF_MODEL]] + + from esphome.components.lvgl import DOMAIN as LVGL_DOMAIN + + if not requires_buffer(config) and LVGL_DOMAIN not in global_config: + # If no drawing methods are configured, and LVGL is not enabled, show a test card + config[CONF_SHOW_TEST_CARD] = True + + if PSRAM_DOMAIN not in global_config and CONF_BUFFER_SIZE not in config: + if not requires_buffer(config): + return config # No buffer needed, so no need to set a buffer size + # If PSRAM is not enabled, choose a small buffer size by default + if not requires_buffer(config): + # not our problem. + return config + color_depth = get_color_depth(config) + frac = denominator(config) + height, width, _offset_width, _offset_height = model.get_dimensions(config) + + buffer_size = color_depth // 8 * width * height // frac + # Target a buffer size of 20kB + fraction = 20000.0 / buffer_size + try: + config[CONF_BUFFER_SIZE] = 1.0 / next( + x for x in range(2, 17) if fraction >= 1 / x and height % x == 0 + ) + except StopIteration: + # Either the screen is too big, or the height is not divisible by any of the fractions, so use 1.0 + # PSRAM will be needed. + if CORE.is_esp32: + raise cv.Invalid( + "PSRAM is required for this display" + ) from StopIteration + + return config + + +FINAL_VALIDATE_SCHEMA = _final_validate + + +def get_transform(config): + """ + Get the transformation configuration for the display. + :param config: + :return: + """ + model = MODELS[config[CONF_MODEL]] + can_transform = model.rotation_as_transform(config) transform = config.get( CONF_TRANSFORM, { @@ -336,123 +364,80 @@ def get_transform(model, config): return transform -def get_sequence(model, config): +def get_instance(config): """ - Create the init sequence for the display. - Use the default sequence from the model, if any, and append any custom sequence provided in the config. - Append SLPOUT (if not already in the sequence) and DISPON to the end of the sequence - Pixel format, color order, and orientation will be set. + Get the type of MipiSpi instance to create based on the configuration, + and the template arguments. + :param config: + :return: type, template arguments """ - sequence = list(model.initsequence) - custom_sequence = config.get(CONF_INIT_SEQUENCE, []) - sequence.extend(custom_sequence) - # Ensure each command is a tuple - sequence = [x if isinstance(x, tuple) else (x,) for x in sequence] - commands = [x[0] for x in sequence] - # Set pixel format if not already in the custom sequence - if PIXFMT not in commands: - pixel_mode = config[CONF_PIXEL_MODE] - if not isinstance(pixel_mode, int): - pixel_mode = PIXEL_MODES[pixel_mode] - sequence.append((PIXFMT, pixel_mode)) - # Does the chip use the flipping bits for mirroring rather than the reverse order bits? - use_flip = config[CONF_USE_AXIS_FLIPS] - if MADCTL not in commands: - madctl = 0 - transform = get_transform(model, config) - if transform.get(CONF_TRANSFORM): - LOGGER.info("Using hardware transform to implement rotation") - if transform.get(CONF_MIRROR_X): - madctl |= MADCTL_XFLIP if use_flip else MADCTL_MX - if transform.get(CONF_MIRROR_Y): - madctl |= MADCTL_YFLIP if use_flip else MADCTL_MY - if transform.get(CONF_SWAP_XY) is True: # Exclude Undefined - madctl |= MADCTL_MV - if config[CONF_COLOR_ORDER] == MODE_BGR: - madctl |= MADCTL_BGR - sequence.append((MADCTL, madctl)) - if INVON not in commands and INVOFF not in commands: - if config[CONF_INVERT_COLORS]: - sequence.append((INVON,)) - else: - sequence.append((INVOFF,)) - if BRIGHTNESS not in commands: - if brightness := config.get( - CONF_BRIGHTNESS, model.get_default(CONF_BRIGHTNESS) - ): - sequence.append((BRIGHTNESS, brightness)) - if SLPOUT not in commands: - sequence.append((SLPOUT,)) - sequence.append((DISPON,)) + model = MODELS[config[CONF_MODEL]] + width, height, offset_width, offset_height = model.get_dimensions(config) - # Flatten the sequence into a list of bytes, with the length of each command - # or the delay flag inserted where needed - return sum( - tuple( - (x[1], 0xFF) if x[0] == DELAY_FLAG else (x[0], len(x) - 1) + x[1:] - for x in sequence - ), - (), + color_depth = int(config[CONF_COLOR_DEPTH].removesuffix("bit")) + bufferpixels = COLOR_DEPTHS[color_depth] + + display_pixel_mode = DISPLAY_PIXEL_MODES[config[CONF_PIXEL_MODE]][1] + bus_type = config[CONF_BUS_MODE] + if bus_type == TYPE_SINGLE and config.get(CONF_SPI_16, False): + # If the bus mode is single and spi_16 is set, use single 16-bit mode + bus_type = BusType.BUS_TYPE_SINGLE_16 + else: + bus_type = BusTypes[bus_type] + buffer_type = cg.uint8 if color_depth == 8 else cg.uint16 + frac = denominator(config) + rotation = ( + 0 if model.rotation_as_transform(config) else config.get(CONF_ROTATION, 0) ) + templateargs = [ + buffer_type, + bufferpixels, + config[CONF_BYTE_ORDER] == "big_endian", + display_pixel_mode, + bus_type, + ] + # If a buffer is required, use MipiSpiBuffer, otherwise use MipiSpi + if requires_buffer(config): + templateargs.extend( + [ + width, + height, + offset_width, + offset_height, + DISPLAY_ROTATIONS[rotation], + frac, + config[CONF_DRAW_ROUNDING], + ] + ) + return MipiSpiBuffer, templateargs + # Swap height and width if the display is rotated 90 or 270 degrees in software + if rotation in (90, 270): + width, height = height, width + offset_width, offset_height = offset_height, offset_width + templateargs.extend( + [ + width, + height, + offset_width, + offset_height, + ] + ) + return MipiSpi, templateargs async def to_code(config): model = MODELS[config[CONF_MODEL]] - transform = get_transform(model, config) - if CONF_DIMENSIONS in config: - # Explicit dimensions, just use as is - dimensions = config[CONF_DIMENSIONS] - if isinstance(dimensions, dict): - width = dimensions[CONF_WIDTH] - height = dimensions[CONF_HEIGHT] - offset_width = dimensions[CONF_OFFSET_WIDTH] - offset_height = dimensions[CONF_OFFSET_HEIGHT] - else: - (width, height) = dimensions - offset_width = 0 - offset_height = 0 - else: - # Default dimensions, use model defaults and transform if needed - width = model.get_default(CONF_WIDTH) - height = model.get_default(CONF_HEIGHT) - offset_width = model.get_default(CONF_OFFSET_WIDTH, 0) - offset_height = model.get_default(CONF_OFFSET_HEIGHT, 0) - - # if mirroring axes and there are offsets, also mirror the offsets to cater for situations where - # the offset is asymmetric - if transform[CONF_MIRROR_X]: - native_width = model.get_default( - CONF_NATIVE_WIDTH, width + offset_width * 2 - ) - offset_width = native_width - width - offset_width - if transform[CONF_MIRROR_Y]: - native_height = model.get_default( - CONF_NATIVE_HEIGHT, height + offset_height * 2 - ) - offset_height = native_height - height - offset_height - # Swap default dimensions if swap_xy is set - if transform[CONF_SWAP_XY] is True: - width, height = height, width - offset_height, offset_width = offset_width, offset_height - - color_depth = config[CONF_COLOR_DEPTH] - if color_depth.endswith("bit"): - color_depth = color_depth[:-3] - color_depth = COLOR_DEPTHS[int(color_depth)] - - var = cg.new_Pvariable( - config[CONF_ID], width, height, offset_width, offset_height, color_depth - ) - cg.add(var.set_init_sequence(get_sequence(model, config))) - if rotation_as_transform(model, config): + var_id = config[CONF_ID] + var_id.type, templateargs = get_instance(config) + var = cg.new_Pvariable(var_id, TemplateArguments(*templateargs)) + init_sequence, _madctl = model.get_sequence(config) + cg.add(var.set_init_sequence(init_sequence)) + if model.rotation_as_transform(config): if CONF_TRANSFORM in config: LOGGER.warning("Use of 'transform' with 'rotation' is not recommended") else: config[CONF_ROTATION] = 0 cg.add(var.set_model(config[CONF_MODEL])) - cg.add(var.set_draw_from_origin(config[CONF_DRAW_FROM_ORIGIN])) - cg.add(var.set_draw_rounding(config[CONF_DRAW_ROUNDING])) - cg.add(var.set_spi_16(config[CONF_SPI_16])) if enable_pin := config.get(CONF_ENABLE_PIN): enable = [await cg.gpio_pin_expression(pin) for pin in enable_pin] cg.add(var.set_enable_pins(enable)) @@ -472,4 +457,5 @@ async def to_code(config): cg.add(var.set_writer(lambda_)) await display.register_display(var, config) await spi.register_spi_device(var, config) + # Displays are write-only, set the SPI device to write-only as well cg.add(var.set_write_only(True)) diff --git a/esphome/components/mipi_spi/mipi_spi.cpp b/esphome/components/mipi_spi/mipi_spi.cpp index 962575477d..272915b4e1 100644 --- a/esphome/components/mipi_spi/mipi_spi.cpp +++ b/esphome/components/mipi_spi/mipi_spi.cpp @@ -2,489 +2,5 @@ #include "esphome/core/log.h" namespace esphome { -namespace mipi_spi { - -void MipiSpi::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->spi_setup(); - if (this->dc_pin_ != nullptr) { - this->dc_pin_->setup(); - this->dc_pin_->digital_write(false); - } - for (auto *pin : this->enable_pins_) { - pin->setup(); - pin->digital_write(true); - } - if (this->reset_pin_ != nullptr) { - this->reset_pin_->setup(); - this->reset_pin_->digital_write(true); - delay(5); - this->reset_pin_->digital_write(false); - delay(5); - this->reset_pin_->digital_write(true); - } - this->bus_width_ = this->parent_->get_bus_width(); - - // need to know when the display is ready for SLPOUT command - will be 120ms after reset - auto when = millis() + 120; - delay(10); - size_t index = 0; - auto &vec = this->init_sequence_; - while (index != vec.size()) { - if (vec.size() - index < 2) { - ESP_LOGE(TAG, "Malformed init sequence"); - this->mark_failed(); - return; - } - uint8_t cmd = vec[index++]; - uint8_t x = vec[index++]; - if (x == DELAY_FLAG) { - ESP_LOGD(TAG, "Delay %dms", cmd); - delay(cmd); - } else { - uint8_t num_args = x & 0x7F; - if (vec.size() - index < num_args) { - ESP_LOGE(TAG, "Malformed init sequence"); - this->mark_failed(); - return; - } - auto arg_byte = vec[index]; - switch (cmd) { - case SLEEP_OUT: { - // are we ready, boots? - int duration = when - millis(); - if (duration > 0) { - ESP_LOGD(TAG, "Sleep %dms", duration); - delay(duration); - } - } break; - - case INVERT_ON: - this->invert_colors_ = true; - break; - case MADCTL_CMD: - this->madctl_ = arg_byte; - break; - case PIXFMT: - this->pixel_mode_ = arg_byte & 0x11 ? PIXEL_MODE_16 : PIXEL_MODE_18; - break; - case BRIGHTNESS: - this->brightness_ = arg_byte; - break; - - default: - break; - } - const auto *ptr = vec.data() + index; - ESP_LOGD(TAG, "Command %02X, length %d, byte %02X", cmd, num_args, arg_byte); - this->write_command_(cmd, ptr, num_args); - index += num_args; - if (cmd == SLEEP_OUT) - delay(10); - } - } - this->setup_complete_ = true; - if (this->draw_from_origin_) - check_buffer_(); - ESP_LOGCONFIG(TAG, "MIPI SPI setup complete"); -} - -void MipiSpi::update() { - if (!this->setup_complete_ || this->is_failed()) { - return; - } - this->do_update_(); - if (this->buffer_ == nullptr || this->x_low_ > this->x_high_ || this->y_low_ > this->y_high_) - return; - ESP_LOGV(TAG, "x_low %d, y_low %d, x_high %d, y_high %d", this->x_low_, this->y_low_, this->x_high_, this->y_high_); - // Some chips require that the drawing window be aligned on certain boundaries - auto dr = this->draw_rounding_; - this->x_low_ = this->x_low_ / dr * dr; - this->y_low_ = this->y_low_ / dr * dr; - this->x_high_ = (this->x_high_ + dr) / dr * dr - 1; - this->y_high_ = (this->y_high_ + dr) / dr * dr - 1; - if (this->draw_from_origin_) { - this->x_low_ = 0; - this->y_low_ = 0; - this->x_high_ = this->width_ - 1; - } - int w = this->x_high_ - this->x_low_ + 1; - int h = this->y_high_ - this->y_low_ + 1; - this->write_to_display_(this->x_low_, this->y_low_, w, h, this->buffer_, this->x_low_, this->y_low_, - this->width_ - w - this->x_low_); - // invalidate watermarks - this->x_low_ = this->width_; - this->y_low_ = this->height_; - this->x_high_ = 0; - this->y_high_ = 0; -} - -void MipiSpi::fill(Color color) { - if (!this->check_buffer_()) - return; - this->x_low_ = 0; - this->y_low_ = 0; - this->x_high_ = this->get_width_internal() - 1; - this->y_high_ = this->get_height_internal() - 1; - switch (this->color_depth_) { - case display::COLOR_BITNESS_332: { - auto new_color = display::ColorUtil::color_to_332(color, display::ColorOrder::COLOR_ORDER_RGB); - memset(this->buffer_, (uint8_t) new_color, this->buffer_bytes_); - break; - } - default: { - auto new_color = display::ColorUtil::color_to_565(color); - if (((uint8_t) (new_color >> 8)) == ((uint8_t) new_color)) { - // Upper and lower is equal can use quicker memset operation. Takes ~20ms. - memset(this->buffer_, (uint8_t) new_color, this->buffer_bytes_); - } else { - auto *ptr_16 = reinterpret_cast(this->buffer_); - auto len = this->buffer_bytes_ / 2; - while (len--) { - *ptr_16++ = new_color; - } - } - } - } -} - -void MipiSpi::draw_absolute_pixel_internal(int x, int y, Color color) { - if (x >= this->get_width_internal() || x < 0 || y >= this->get_height_internal() || y < 0) { - return; - } - if (!this->check_buffer_()) - return; - size_t pos = (y * this->width_) + x; - switch (this->color_depth_) { - case display::COLOR_BITNESS_332: { - uint8_t new_color = display::ColorUtil::color_to_332(color); - if (this->buffer_[pos] == new_color) - return; - this->buffer_[pos] = new_color; - break; - } - - case display::COLOR_BITNESS_565: { - auto *ptr_16 = reinterpret_cast(this->buffer_); - uint8_t hi_byte = static_cast(color.r & 0xF8) | (color.g >> 5); - uint8_t lo_byte = static_cast((color.g & 0x1C) << 3) | (color.b >> 3); - uint16_t new_color = hi_byte | (lo_byte << 8); // big endian - if (ptr_16[pos] == new_color) - return; - ptr_16[pos] = new_color; - break; - } - default: - return; - } - // low and high watermark may speed up drawing from buffer - if (x < this->x_low_) - this->x_low_ = x; - if (y < this->y_low_) - this->y_low_ = y; - if (x > this->x_high_) - this->x_high_ = x; - if (y > this->y_high_) - this->y_high_ = y; -} - -void MipiSpi::reset_params_() { - if (!this->is_ready()) - return; - this->write_command_(this->invert_colors_ ? INVERT_ON : INVERT_OFF); - if (this->brightness_.has_value()) - this->write_command_(BRIGHTNESS, this->brightness_.value()); -} - -void MipiSpi::write_init_sequence_() { - size_t index = 0; - auto &vec = this->init_sequence_; - while (index != vec.size()) { - if (vec.size() - index < 2) { - ESP_LOGE(TAG, "Malformed init sequence"); - this->mark_failed(); - return; - } - uint8_t cmd = vec[index++]; - uint8_t x = vec[index++]; - if (x == DELAY_FLAG) { - ESP_LOGV(TAG, "Delay %dms", cmd); - delay(cmd); - } else { - uint8_t num_args = x & 0x7F; - if (vec.size() - index < num_args) { - ESP_LOGE(TAG, "Malformed init sequence"); - this->mark_failed(); - return; - } - const auto *ptr = vec.data() + index; - this->write_command_(cmd, ptr, num_args); - index += num_args; - } - } - this->setup_complete_ = true; - ESP_LOGCONFIG(TAG, "MIPI SPI setup complete"); -} - -void MipiSpi::set_addr_window_(uint16_t x1, uint16_t y1, uint16_t x2, uint16_t y2) { - ESP_LOGVV(TAG, "Set addr %d/%d, %d/%d", x1, y1, x2, y2); - uint8_t buf[4]; - x1 += this->offset_width_; - x2 += this->offset_width_; - y1 += this->offset_height_; - y2 += this->offset_height_; - put16_be(buf, y1); - put16_be(buf + 2, y2); - this->write_command_(RASET, buf, sizeof buf); - put16_be(buf, x1); - put16_be(buf + 2, x2); - this->write_command_(CASET, buf, sizeof buf); -} - -void MipiSpi::draw_pixels_at(int x_start, int y_start, int w, int h, const uint8_t *ptr, display::ColorOrder order, - display::ColorBitness bitness, bool big_endian, int x_offset, int y_offset, int x_pad) { - if (!this->setup_complete_ || this->is_failed()) - return; - if (w <= 0 || h <= 0) - return; - if (bitness != this->color_depth_ || big_endian != (this->bit_order_ == spi::BIT_ORDER_MSB_FIRST)) { - Display::draw_pixels_at(x_start, y_start, w, h, ptr, order, bitness, big_endian, x_offset, y_offset, x_pad); - return; - } - if (this->draw_from_origin_) { - auto stride = x_offset + w + x_pad; - for (int y = 0; y != h; y++) { - memcpy(this->buffer_ + ((y + y_start) * this->width_ + x_start) * 2, - ptr + ((y + y_offset) * stride + x_offset) * 2, w * 2); - } - ptr = this->buffer_; - w = this->width_; - h += y_start; - x_start = 0; - y_start = 0; - x_offset = 0; - y_offset = 0; - } - this->write_to_display_(x_start, y_start, w, h, ptr, x_offset, y_offset, x_pad); -} - -void MipiSpi::write_18_from_16_bit_(const uint16_t *ptr, size_t w, size_t h, size_t stride) { - stride -= w; - uint8_t transfer_buffer[6 * 256]; - size_t idx = 0; // index into transfer_buffer - while (h-- != 0) { - for (auto x = w; x-- != 0;) { - auto color_val = *ptr++; - // deal with byte swapping - transfer_buffer[idx++] = (color_val & 0xF8); // Blue - transfer_buffer[idx++] = ((color_val & 0x7) << 5) | ((color_val & 0xE000) >> 11); // Green - transfer_buffer[idx++] = (color_val >> 5) & 0xF8; // Red - if (idx == sizeof(transfer_buffer)) { - this->write_array(transfer_buffer, idx); - idx = 0; - } - } - ptr += stride; - } - if (idx != 0) - this->write_array(transfer_buffer, idx); -} - -void MipiSpi::write_18_from_8_bit_(const uint8_t *ptr, size_t w, size_t h, size_t stride) { - stride -= w; - uint8_t transfer_buffer[6 * 256]; - size_t idx = 0; // index into transfer_buffer - while (h-- != 0) { - for (auto x = w; x-- != 0;) { - auto color_val = *ptr++; - transfer_buffer[idx++] = color_val & 0xE0; // Red - transfer_buffer[idx++] = (color_val << 3) & 0xE0; // Green - transfer_buffer[idx++] = color_val << 6; // Blue - if (idx == sizeof(transfer_buffer)) { - this->write_array(transfer_buffer, idx); - idx = 0; - } - } - ptr += stride; - } - if (idx != 0) - this->write_array(transfer_buffer, idx); -} - -void MipiSpi::write_16_from_8_bit_(const uint8_t *ptr, size_t w, size_t h, size_t stride) { - stride -= w; - uint8_t transfer_buffer[6 * 256]; - size_t idx = 0; // index into transfer_buffer - while (h-- != 0) { - for (auto x = w; x-- != 0;) { - auto color_val = *ptr++; - transfer_buffer[idx++] = (color_val & 0xE0) | ((color_val & 0x1C) >> 2); - transfer_buffer[idx++] = (color_val & 0x3) << 3; - if (idx == sizeof(transfer_buffer)) { - this->write_array(transfer_buffer, idx); - idx = 0; - } - } - ptr += stride; - } - if (idx != 0) - this->write_array(transfer_buffer, idx); -} - -void MipiSpi::write_to_display_(int x_start, int y_start, int w, int h, const uint8_t *ptr, int x_offset, int y_offset, - int x_pad) { - this->set_addr_window_(x_start, y_start, x_start + w - 1, y_start + h - 1); - auto stride = x_offset + w + x_pad; - const auto *offset_ptr = ptr; - if (this->color_depth_ == display::COLOR_BITNESS_332) { - offset_ptr += y_offset * stride + x_offset; - } else { - stride *= 2; - offset_ptr += y_offset * stride + x_offset * 2; - } - - switch (this->bus_width_) { - case 4: - this->enable(); - if (x_offset == 0 && x_pad == 0 && y_offset == 0) { - // we could deal here with a non-zero y_offset, but if x_offset is zero, y_offset probably will be so don't - // bother - this->write_cmd_addr_data(8, 0x32, 24, WDATA << 8, ptr, w * h * 2, 4); - } else { - this->write_cmd_addr_data(8, 0x32, 24, WDATA << 8, nullptr, 0, 4); - for (int y = 0; y != h; y++) { - this->write_cmd_addr_data(0, 0, 0, 0, offset_ptr, w * 2, 4); - offset_ptr += stride; - } - } - break; - - case 8: - this->write_command_(WDATA); - this->enable(); - if (x_offset == 0 && x_pad == 0 && y_offset == 0) { - this->write_cmd_addr_data(0, 0, 0, 0, ptr, w * h * 2, 8); - } else { - for (int y = 0; y != h; y++) { - this->write_cmd_addr_data(0, 0, 0, 0, offset_ptr, w * 2, 8); - offset_ptr += stride; - } - } - break; - - default: - this->write_command_(WDATA); - this->enable(); - - if (this->color_depth_ == display::COLOR_BITNESS_565) { - // Source buffer is 16-bit RGB565 - if (this->pixel_mode_ == PIXEL_MODE_18) { - // Convert RGB565 to RGB666 - this->write_18_from_16_bit_(reinterpret_cast(offset_ptr), w, h, stride / 2); - } else { - // Direct RGB565 output - if (x_offset == 0 && x_pad == 0 && y_offset == 0) { - this->write_array(ptr, w * h * 2); - } else { - for (int y = 0; y != h; y++) { - this->write_array(offset_ptr, w * 2); - offset_ptr += stride; - } - } - } - } else { - // Source buffer is 8-bit RGB332 - if (this->pixel_mode_ == PIXEL_MODE_18) { - // Convert RGB332 to RGB666 - this->write_18_from_8_bit_(offset_ptr, w, h, stride); - } else { - this->write_16_from_8_bit_(offset_ptr, w, h, stride); - } - break; - } - } - this->disable(); -} - -void MipiSpi::write_command_(uint8_t cmd, const uint8_t *bytes, size_t len) { - ESP_LOGV(TAG, "Command %02X, length %d, bytes %s", cmd, len, format_hex_pretty(bytes, len).c_str()); - if (this->bus_width_ == 4) { - this->enable(); - this->write_cmd_addr_data(8, 0x02, 24, cmd << 8, bytes, len); - this->disable(); - } else if (this->bus_width_ == 8) { - this->dc_pin_->digital_write(false); - this->enable(); - this->write_cmd_addr_data(0, 0, 0, 0, &cmd, 1, 8); - this->disable(); - this->dc_pin_->digital_write(true); - if (len != 0) { - this->enable(); - this->write_cmd_addr_data(0, 0, 0, 0, bytes, len, 8); - this->disable(); - } - } else { - this->dc_pin_->digital_write(false); - this->enable(); - this->write_byte(cmd); - this->disable(); - this->dc_pin_->digital_write(true); - if (len != 0) { - if (this->spi_16_) { - for (size_t i = 0; i != len; i++) { - this->enable(); - this->write_byte(0); - this->write_byte(bytes[i]); - this->disable(); - } - } else { - this->enable(); - this->write_array(bytes, len); - this->disable(); - } - } - } -} - -void MipiSpi::dump_config() { - ESP_LOGCONFIG(TAG, - "MIPI_SPI Display\n" - " Model: %s\n" - " Width: %u\n" - " Height: %u", - this->model_, this->width_, this->height_); - if (this->offset_width_ != 0) - ESP_LOGCONFIG(TAG, " Offset width: %u", this->offset_width_); - if (this->offset_height_ != 0) - ESP_LOGCONFIG(TAG, " Offset height: %u", this->offset_height_); - ESP_LOGCONFIG(TAG, - " Swap X/Y: %s\n" - " Mirror X: %s\n" - " Mirror Y: %s\n" - " Color depth: %d bits\n" - " Invert colors: %s\n" - " Color order: %s\n" - " Pixel mode: %s", - YESNO(this->madctl_ & MADCTL_MV), YESNO(this->madctl_ & (MADCTL_MX | MADCTL_XFLIP)), - YESNO(this->madctl_ & (MADCTL_MY | MADCTL_YFLIP)), - this->color_depth_ == display::COLOR_BITNESS_565 ? 16 : 8, YESNO(this->invert_colors_), - this->madctl_ & MADCTL_BGR ? "BGR" : "RGB", this->pixel_mode_ == PIXEL_MODE_18 ? "18bit" : "16bit"); - if (this->brightness_.has_value()) - ESP_LOGCONFIG(TAG, " Brightness: %u", this->brightness_.value()); - if (this->spi_16_) - ESP_LOGCONFIG(TAG, " SPI 16bit: YES"); - ESP_LOGCONFIG(TAG, " Draw rounding: %u", this->draw_rounding_); - if (this->draw_from_origin_) - ESP_LOGCONFIG(TAG, " Draw from origin: YES"); - LOG_PIN(" CS Pin: ", this->cs_); - LOG_PIN(" Reset Pin: ", this->reset_pin_); - LOG_PIN(" DC Pin: ", this->dc_pin_); - ESP_LOGCONFIG(TAG, - " SPI Mode: %d\n" - " SPI Data rate: %dMHz\n" - " SPI Bus width: %d", - this->mode_, static_cast(this->data_rate_ / 1000000), this->bus_width_); -} - -} // namespace mipi_spi +namespace mipi_spi {} // namespace mipi_spi } // namespace esphome diff --git a/esphome/components/mipi_spi/mipi_spi.h b/esphome/components/mipi_spi/mipi_spi.h index 052ebe3a6b..7e597d1c61 100644 --- a/esphome/components/mipi_spi/mipi_spi.h +++ b/esphome/components/mipi_spi/mipi_spi.h @@ -4,70 +4,85 @@ #include "esphome/components/spi/spi.h" #include "esphome/components/display/display.h" -#include "esphome/components/display/display_buffer.h" #include "esphome/components/display/display_color_utils.h" namespace esphome { namespace mipi_spi { constexpr static const char *const TAG = "display.mipi_spi"; -static const uint8_t SW_RESET_CMD = 0x01; -static const uint8_t SLEEP_OUT = 0x11; -static const uint8_t NORON = 0x13; -static const uint8_t INVERT_OFF = 0x20; -static const uint8_t INVERT_ON = 0x21; -static const uint8_t ALL_ON = 0x23; -static const uint8_t WRAM = 0x24; -static const uint8_t MIPI = 0x26; -static const uint8_t DISPLAY_ON = 0x29; -static const uint8_t RASET = 0x2B; -static const uint8_t CASET = 0x2A; -static const uint8_t WDATA = 0x2C; -static const uint8_t TEON = 0x35; -static const uint8_t MADCTL_CMD = 0x36; -static const uint8_t PIXFMT = 0x3A; -static const uint8_t BRIGHTNESS = 0x51; -static const uint8_t SWIRE1 = 0x5A; -static const uint8_t SWIRE2 = 0x5B; -static const uint8_t PAGESEL = 0xFE; +static constexpr uint8_t SW_RESET_CMD = 0x01; +static constexpr uint8_t SLEEP_OUT = 0x11; +static constexpr uint8_t NORON = 0x13; +static constexpr uint8_t INVERT_OFF = 0x20; +static constexpr uint8_t INVERT_ON = 0x21; +static constexpr uint8_t ALL_ON = 0x23; +static constexpr uint8_t WRAM = 0x24; +static constexpr uint8_t MIPI = 0x26; +static constexpr uint8_t DISPLAY_ON = 0x29; +static constexpr uint8_t RASET = 0x2B; +static constexpr uint8_t CASET = 0x2A; +static constexpr uint8_t WDATA = 0x2C; +static constexpr uint8_t TEON = 0x35; +static constexpr uint8_t MADCTL_CMD = 0x36; +static constexpr uint8_t PIXFMT = 0x3A; +static constexpr uint8_t BRIGHTNESS = 0x51; +static constexpr uint8_t SWIRE1 = 0x5A; +static constexpr uint8_t SWIRE2 = 0x5B; +static constexpr uint8_t PAGESEL = 0xFE; -static const uint8_t MADCTL_MY = 0x80; // Bit 7 Bottom to top -static const uint8_t MADCTL_MX = 0x40; // Bit 6 Right to left -static const uint8_t MADCTL_MV = 0x20; // Bit 5 Swap axes -static const uint8_t MADCTL_RGB = 0x00; // Bit 3 Red-Green-Blue pixel order -static const uint8_t MADCTL_BGR = 0x08; // Bit 3 Blue-Green-Red pixel order -static const uint8_t MADCTL_XFLIP = 0x02; // Mirror the display horizontally -static const uint8_t MADCTL_YFLIP = 0x01; // Mirror the display vertically +static constexpr uint8_t MADCTL_MY = 0x80; // Bit 7 Bottom to top +static constexpr uint8_t MADCTL_MX = 0x40; // Bit 6 Right to left +static constexpr uint8_t MADCTL_MV = 0x20; // Bit 5 Swap axes +static constexpr uint8_t MADCTL_RGB = 0x00; // Bit 3 Red-Green-Blue pixel order +static constexpr uint8_t MADCTL_BGR = 0x08; // Bit 3 Blue-Green-Red pixel order +static constexpr uint8_t MADCTL_XFLIP = 0x02; // Mirror the display horizontally +static constexpr uint8_t MADCTL_YFLIP = 0x01; // Mirror the display vertically -static const uint8_t DELAY_FLAG = 0xFF; +static constexpr uint8_t DELAY_FLAG = 0xFF; // store a 16 bit value in a buffer, big endian. static inline void put16_be(uint8_t *buf, uint16_t value) { buf[0] = value >> 8; buf[1] = value; } +// Buffer mode, conveniently also the number of bytes in a pixel enum PixelMode { - PIXEL_MODE_16, - PIXEL_MODE_18, + PIXEL_MODE_8 = 1, + PIXEL_MODE_16 = 2, + PIXEL_MODE_18 = 3, }; -class MipiSpi : public display::DisplayBuffer, +enum BusType { + BUS_TYPE_SINGLE = 1, + BUS_TYPE_QUAD = 4, + BUS_TYPE_OCTAL = 8, + BUS_TYPE_SINGLE_16 = 16, // Single bit bus, but 16 bits per transfer +}; + +/** + * Base class for MIPI SPI displays. + * All the methods are defined here in the header file, as it is not possible to define templated methods in a cpp file. + * + * @tparam BUFFERTYPE The type of the buffer pixels, e.g. uint8_t or uint16_t + * @tparam BUFFERPIXEL Color depth of the buffer + * @tparam DISPLAYPIXEL Color depth of the display + * @tparam BUS_TYPE The type of the interface bus (single, quad, octal) + * @tparam WIDTH Width of the display in pixels + * @tparam HEIGHT Height of the display in pixels + * @tparam OFFSET_WIDTH The x-offset of the display in pixels + * @tparam OFFSET_HEIGHT The y-offset of the display in pixels + * buffer + */ +template +class MipiSpi : public display::Display, public spi::SPIDevice { public: - MipiSpi(size_t width, size_t height, int16_t offset_width, int16_t offset_height, display::ColorBitness color_depth) - : width_(width), - height_(height), - offset_width_(offset_width), - offset_height_(offset_height), - color_depth_(color_depth) {} + MipiSpi() = default; + void update() override { this->stop_poller(); } + void draw_pixel_at(int x, int y, Color color) override {} void set_model(const char *model) { this->model_ = model; } - void update() override; - void setup() override; - display::ColorOrder get_color_mode() { - return this->madctl_ & MADCTL_BGR ? display::COLOR_ORDER_BGR : display::COLOR_ORDER_RGB; - } - void set_reset_pin(GPIOPin *reset_pin) { this->reset_pin_ = reset_pin; } void set_enable_pins(std::vector enable_pins) { this->enable_pins_ = std::move(enable_pins); } void set_dc_pin(GPIOPin *dc_pin) { this->dc_pin_ = dc_pin; } @@ -79,93 +94,532 @@ class MipiSpi : public display::DisplayBuffer, this->brightness_ = brightness; this->reset_params_(); } - - void set_draw_from_origin(bool draw_from_origin) { this->draw_from_origin_ = draw_from_origin; } display::DisplayType get_display_type() override { return display::DisplayType::DISPLAY_TYPE_COLOR; } - void dump_config() override; - int get_width_internal() override { return this->width_; } - int get_height_internal() override { return this->height_; } - bool can_proceed() override { return this->setup_complete_; } + int get_width_internal() override { return WIDTH; } + int get_height_internal() override { return HEIGHT; } void set_init_sequence(const std::vector &sequence) { this->init_sequence_ = sequence; } - void set_draw_rounding(unsigned rounding) { this->draw_rounding_ = rounding; } - void set_spi_16(bool spi_16) { this->spi_16_ = spi_16; } + + // reset the display, and write the init sequence + void setup() override { + this->spi_setup(); + if (this->dc_pin_ != nullptr) { + this->dc_pin_->setup(); + this->dc_pin_->digital_write(false); + } + for (auto *pin : this->enable_pins_) { + pin->setup(); + pin->digital_write(true); + } + if (this->reset_pin_ != nullptr) { + this->reset_pin_->setup(); + this->reset_pin_->digital_write(true); + delay(5); + this->reset_pin_->digital_write(false); + delay(5); + this->reset_pin_->digital_write(true); + } + + // need to know when the display is ready for SLPOUT command - will be 120ms after reset + auto when = millis() + 120; + delay(10); + size_t index = 0; + auto &vec = this->init_sequence_; + while (index != vec.size()) { + if (vec.size() - index < 2) { + esph_log_e(TAG, "Malformed init sequence"); + this->mark_failed(); + return; + } + uint8_t cmd = vec[index++]; + uint8_t x = vec[index++]; + if (x == DELAY_FLAG) { + esph_log_d(TAG, "Delay %dms", cmd); + delay(cmd); + } else { + uint8_t num_args = x & 0x7F; + if (vec.size() - index < num_args) { + esph_log_e(TAG, "Malformed init sequence"); + this->mark_failed(); + return; + } + auto arg_byte = vec[index]; + switch (cmd) { + case SLEEP_OUT: { + // are we ready, boots? + int duration = when - millis(); + if (duration > 0) { + esph_log_d(TAG, "Sleep %dms", duration); + delay(duration); + } + } break; + + case INVERT_ON: + this->invert_colors_ = true; + break; + case MADCTL_CMD: + this->madctl_ = arg_byte; + break; + case BRIGHTNESS: + this->brightness_ = arg_byte; + break; + + default: + break; + } + const auto *ptr = vec.data() + index; + esph_log_d(TAG, "Command %02X, length %d, byte %02X", cmd, num_args, arg_byte); + this->write_command_(cmd, ptr, num_args); + index += num_args; + if (cmd == SLEEP_OUT) + delay(10); + } + } + // init sequence no longer needed + this->init_sequence_.clear(); + } + + // Drawing operations + + void draw_pixels_at(int x_start, int y_start, int w, int h, const uint8_t *ptr, display::ColorOrder order, + display::ColorBitness bitness, bool big_endian, int x_offset, int y_offset, int x_pad) override { + if (this->is_failed()) + return; + if (w <= 0 || h <= 0) + return; + if (get_pixel_mode(bitness) != BUFFERPIXEL || big_endian != IS_BIG_ENDIAN) { + // note that the usual logging macros are banned in header files, so use their replacement + esph_log_e(TAG, "Unsupported color depth or bit order"); + return; + } + this->write_to_display_(x_start, y_start, w, h, reinterpret_cast(ptr), x_offset, y_offset, + x_pad); + } + + void dump_config() override { + esph_log_config(TAG, + "MIPI_SPI Display\n" + " Model: %s\n" + " Width: %u\n" + " Height: %u", + this->model_, WIDTH, HEIGHT); + if constexpr (OFFSET_WIDTH != 0) + esph_log_config(TAG, " Offset width: %u", OFFSET_WIDTH); + if constexpr (OFFSET_HEIGHT != 0) + esph_log_config(TAG, " Offset height: %u", OFFSET_HEIGHT); + esph_log_config(TAG, + " Swap X/Y: %s\n" + " Mirror X: %s\n" + " Mirror Y: %s\n" + " Invert colors: %s\n" + " Color order: %s\n" + " Display pixels: %d bits\n" + " Endianness: %s\n", + YESNO(this->madctl_ & MADCTL_MV), YESNO(this->madctl_ & (MADCTL_MX | MADCTL_XFLIP)), + YESNO(this->madctl_ & (MADCTL_MY | MADCTL_YFLIP)), YESNO(this->invert_colors_), + this->madctl_ & MADCTL_BGR ? "BGR" : "RGB", DISPLAYPIXEL * 8, IS_BIG_ENDIAN ? "Big" : "Little"); + if (this->brightness_.has_value()) + esph_log_config(TAG, " Brightness: %u", this->brightness_.value()); + if (this->cs_ != nullptr) + esph_log_config(TAG, " CS Pin: %s", this->cs_->dump_summary().c_str()); + if (this->reset_pin_ != nullptr) + esph_log_config(TAG, " Reset Pin: %s", this->reset_pin_->dump_summary().c_str()); + if (this->dc_pin_ != nullptr) + esph_log_config(TAG, " DC Pin: %s", this->dc_pin_->dump_summary().c_str()); + esph_log_config(TAG, + " SPI Mode: %d\n" + " SPI Data rate: %dMHz\n" + " SPI Bus width: %d", + this->mode_, static_cast(this->data_rate_ / 1000000), BUS_TYPE); + } protected: - bool check_buffer_() { - if (this->is_failed()) - return false; - if (this->buffer_ != nullptr) - return true; - auto bytes_per_pixel = this->color_depth_ == display::COLOR_BITNESS_565 ? 2 : 1; - this->init_internal_(this->width_ * this->height_ * bytes_per_pixel); - if (this->buffer_ == nullptr) { - this->mark_failed(); - return false; - } - this->buffer_bytes_ = this->width_ * this->height_ * bytes_per_pixel; - return true; - } - void fill(Color color) override; - void draw_absolute_pixel_internal(int x, int y, Color color) override; - void draw_pixels_at(int x_start, int y_start, int w, int h, const uint8_t *ptr, display::ColorOrder order, - display::ColorBitness bitness, bool big_endian, int x_offset, int y_offset, int x_pad) override; - void write_18_from_16_bit_(const uint16_t *ptr, size_t w, size_t h, size_t stride); - void write_18_from_8_bit_(const uint8_t *ptr, size_t w, size_t h, size_t stride); - void write_16_from_8_bit_(const uint8_t *ptr, size_t w, size_t h, size_t stride); - void write_to_display_(int x_start, int y_start, int w, int h, const uint8_t *ptr, int x_offset, int y_offset, - int x_pad); - /** - * the RM67162 in quad SPI mode seems to work like this (not in the datasheet, this is deduced from the - * sample code.) - * - * Immediately after enabling /CS send 4 bytes in single-dataline SPI mode: - * 0: either 0x2 or 0x32. The first indicates that any subsequent data bytes after the initial 4 will be - * sent in 1-dataline SPI. The second indicates quad mode. - * 1: 0x00 - * 2: The command (register address) byte. - * 3: 0x00 - * - * This is followed by zero or more data bytes in either 1-wire or 4-wire mode, depending on the first byte. - * At the conclusion of the write, de-assert /CS. - * - * @param cmd - * @param bytes - * @param len - */ - void write_command_(uint8_t cmd, const uint8_t *bytes, size_t len); - + /* METHODS */ + // convenience functions to write commands with or without data void write_command_(uint8_t cmd, uint8_t data) { this->write_command_(cmd, &data, 1); } void write_command_(uint8_t cmd) { this->write_command_(cmd, &cmd, 0); } - void reset_params_(); - void write_init_sequence_(); - void set_addr_window_(uint16_t x1, uint16_t y1, uint16_t x2, uint16_t y2); + // Writes a command to the display, with the given bytes. + void write_command_(uint8_t cmd, const uint8_t *bytes, size_t len) { + esph_log_v(TAG, "Command %02X, length %d, bytes %s", cmd, len, format_hex_pretty(bytes, len).c_str()); + if constexpr (BUS_TYPE == BUS_TYPE_QUAD) { + this->enable(); + this->write_cmd_addr_data(8, 0x02, 24, cmd << 8, bytes, len); + this->disable(); + } else if constexpr (BUS_TYPE == BUS_TYPE_OCTAL) { + this->dc_pin_->digital_write(false); + this->enable(); + this->write_cmd_addr_data(0, 0, 0, 0, &cmd, 1, 8); + this->disable(); + this->dc_pin_->digital_write(true); + if (len != 0) { + this->enable(); + this->write_cmd_addr_data(0, 0, 0, 0, bytes, len, 8); + this->disable(); + } + } else if constexpr (BUS_TYPE == BUS_TYPE_SINGLE) { + this->dc_pin_->digital_write(false); + this->enable(); + this->write_byte(cmd); + this->disable(); + this->dc_pin_->digital_write(true); + if (len != 0) { + this->enable(); + this->write_array(bytes, len); + this->disable(); + } + } else if constexpr (BUS_TYPE == BUS_TYPE_SINGLE_16) { + this->dc_pin_->digital_write(false); + this->enable(); + this->write_byte(cmd); + this->disable(); + this->dc_pin_->digital_write(true); + for (size_t i = 0; i != len; i++) { + this->enable(); + this->write_byte(0); + this->write_byte(bytes[i]); + this->disable(); + } + } + } + + // write changed parameters to the display + void reset_params_() { + if (!this->is_ready()) + return; + this->write_command_(this->invert_colors_ ? INVERT_ON : INVERT_OFF); + if (this->brightness_.has_value()) + this->write_command_(BRIGHTNESS, this->brightness_.value()); + } + + // set the address window for the next data write + void set_addr_window_(uint16_t x1, uint16_t y1, uint16_t x2, uint16_t y2) { + esph_log_v(TAG, "Set addr %d/%d, %d/%d", x1, y1, x2, y2); + uint8_t buf[4]; + x1 += OFFSET_WIDTH; + x2 += OFFSET_WIDTH; + y1 += OFFSET_HEIGHT; + y2 += OFFSET_HEIGHT; + put16_be(buf, y1); + put16_be(buf + 2, y2); + this->write_command_(RASET, buf, sizeof buf); + put16_be(buf, x1); + put16_be(buf + 2, x2); + this->write_command_(CASET, buf, sizeof buf); + if constexpr (BUS_TYPE != BUS_TYPE_QUAD) { + this->write_command_(WDATA); + } + } + + // map the display color bitness to the pixel mode + static PixelMode get_pixel_mode(display::ColorBitness bitness) { + switch (bitness) { + case display::COLOR_BITNESS_888: + return PIXEL_MODE_18; // 18 bits per pixel + case display::COLOR_BITNESS_565: + return PIXEL_MODE_16; // 16 bits per pixel + default: + return PIXEL_MODE_8; // Default to 8 bits per pixel + } + } + + /** + * Writes a buffer to the display. + * @param ptr The pointer to the pixel data + * @param w Width of each line in bytes + * @param h Height of the buffer in rows + * @param pad Padding in bytes after each line + */ + void write_display_data_(const uint8_t *ptr, size_t w, size_t h, size_t pad) { + if (pad == 0) { + if constexpr (BUS_TYPE == BUS_TYPE_SINGLE || BUS_TYPE == BUS_TYPE_SINGLE_16) { + this->write_array(ptr, w * h); + } else if constexpr (BUS_TYPE == BUS_TYPE_QUAD) { + this->write_cmd_addr_data(8, 0x32, 24, WDATA << 8, ptr, w * h, 4); + } else if constexpr (BUS_TYPE == BUS_TYPE_OCTAL) { + this->write_cmd_addr_data(0, 0, 0, 0, ptr, w * h, 8); + } + } else { + for (size_t y = 0; y != static_cast(h); y++) { + if constexpr (BUS_TYPE == BUS_TYPE_SINGLE || BUS_TYPE == BUS_TYPE_SINGLE_16) { + this->write_array(ptr, w); + } else if constexpr (BUS_TYPE == BUS_TYPE_QUAD) { + this->write_cmd_addr_data(8, 0x32, 24, WDATA << 8, ptr, w, 4); + } else if constexpr (BUS_TYPE == BUS_TYPE_OCTAL) { + this->write_cmd_addr_data(0, 0, 0, 0, ptr, w, 8); + } + ptr += w + pad; + } + } + } + + /** + * Writes a buffer to the display. + * + * The ptr is a pointer to the pixel data + * The other parameters are all in pixel units. + */ + void write_to_display_(int x_start, int y_start, int w, int h, const BUFFERTYPE *ptr, int x_offset, int y_offset, + int x_pad) { + this->set_addr_window_(x_start, y_start, x_start + w - 1, y_start + h - 1); + this->enable(); + ptr += y_offset * (x_offset + w + x_pad) + x_offset; + if constexpr (BUFFERPIXEL == DISPLAYPIXEL) { + this->write_display_data_(reinterpret_cast(ptr), w * sizeof(BUFFERTYPE), h, + x_pad * sizeof(BUFFERTYPE)); + } else { + // type conversion required, do it in chunks + uint8_t dbuffer[DISPLAYPIXEL * 48]; + uint8_t *dptr = dbuffer; + auto stride = x_offset + w + x_pad; // stride in pixels + for (size_t y = 0; y != static_cast(h); y++) { + for (size_t x = 0; x != static_cast(w); x++) { + auto color_val = ptr[y * stride + x]; + if constexpr (DISPLAYPIXEL == PIXEL_MODE_18 && BUFFERPIXEL == PIXEL_MODE_16) { + // 16 to 18 bit conversion + if constexpr (IS_BIG_ENDIAN) { + *dptr++ = color_val & 0xF8; + *dptr++ = ((color_val & 0x7) << 5) | (color_val & 0xE000) >> 11; + *dptr++ = (color_val >> 5) & 0xF8; + } else { + *dptr++ = (color_val >> 8) & 0xF8; // Blue + *dptr++ = (color_val & 0x7E0) >> 3; + *dptr++ = color_val << 3; + } + } else if constexpr (DISPLAYPIXEL == PIXEL_MODE_18 && BUFFERPIXEL == PIXEL_MODE_8) { + // 8 bit to 18 bit conversion + *dptr++ = color_val << 6; // Blue + *dptr++ = (color_val & 0x1C) << 3; // Green + *dptr++ = (color_val & 0xE0); // Red + } else if constexpr (DISPLAYPIXEL == PIXEL_MODE_16 && BUFFERPIXEL == PIXEL_MODE_8) { + if constexpr (IS_BIG_ENDIAN) { + *dptr++ = (color_val & 0xE0) | ((color_val & 0x1C) >> 2); + *dptr++ = (color_val & 3) << 3; + } else { + *dptr++ = (color_val & 3) << 3; + *dptr++ = (color_val & 0xE0) | ((color_val & 0x1C) >> 2); + } + } + // buffer full? Flush. + if (dptr == dbuffer + sizeof(dbuffer)) { + this->write_display_data_(dbuffer, sizeof(dbuffer), 1, 0); + dptr = dbuffer; + } + } + } + // flush any remaining data + if (dptr != dbuffer) { + this->write_display_data_(dbuffer, dptr - dbuffer, 1, 0); + } + } + this->disable(); + } + + /* PROPERTIES */ + + // GPIO pins GPIOPin *reset_pin_{nullptr}; std::vector enable_pins_{}; GPIOPin *dc_pin_{nullptr}; - uint16_t x_low_{1}; - uint16_t y_low_{1}; - uint16_t x_high_{0}; - uint16_t y_high_{0}; - bool setup_complete_{}; + // other properties set by configuration bool invert_colors_{}; - size_t width_; - size_t height_; - int16_t offset_width_; - int16_t offset_height_; - size_t buffer_bytes_{0}; - display::ColorBitness color_depth_; - PixelMode pixel_mode_{PIXEL_MODE_16}; - uint8_t bus_width_{}; - bool spi_16_{}; - uint8_t madctl_{}; - bool draw_from_origin_{false}; - unsigned draw_rounding_{2}; optional brightness_{}; const char *model_{"Unknown"}; std::vector init_sequence_{}; + uint8_t madctl_{}; }; + +/** + * Class for MIPI SPI displays with a buffer. + * + * @tparam BUFFERTYPE The type of the buffer pixels, e.g. uint8_t or uint16_t + * @tparam BUFFERPIXEL Color depth of the buffer + * @tparam DISPLAYPIXEL Color depth of the display + * @tparam BUS_TYPE The type of the interface bus (single, quad, octal) + * @tparam ROTATION The rotation of the display + * @tparam WIDTH Width of the display in pixels + * @tparam HEIGHT Height of the display in pixels + * @tparam OFFSET_WIDTH The x-offset of the display in pixels + * @tparam OFFSET_HEIGHT The y-offset of the display in pixels + * @tparam FRACTION The fraction of the display size to use for the buffer (e.g. 4 means a 1/4 buffer). + * @tparam ROUNDING The alignment requirement for drawing operations (e.g. 2 means that x coordinates must be even) + */ +template +class MipiSpiBuffer : public MipiSpi { + public: + // these values define the buffer size needed to write in accordance with the chip pixel alignment + // requirements. If the required rounding does not divide the width and height, we round up to the next multiple and + // ignore the extra columns and rows when drawing, but use them to write to the display. + static constexpr unsigned BUFFER_WIDTH = (WIDTH + ROUNDING - 1) / ROUNDING * ROUNDING; + static constexpr unsigned BUFFER_HEIGHT = (HEIGHT + ROUNDING - 1) / ROUNDING * ROUNDING; + + MipiSpiBuffer() { this->rotation_ = ROTATION; } + + void dump_config() override { + MipiSpi::dump_config(); + esph_log_config(TAG, + " Rotation: %d°\n" + " Buffer pixels: %d bits\n" + " Buffer fraction: 1/%d\n" + " Buffer bytes: %zu\n" + " Draw rounding: %u", + this->rotation_, BUFFERPIXEL * 8, FRACTION, + sizeof(BUFFERTYPE) * BUFFER_WIDTH * BUFFER_HEIGHT / FRACTION, ROUNDING); + } + + void setup() override { + MipiSpi::setup(); + RAMAllocator allocator{}; + this->buffer_ = allocator.allocate(BUFFER_WIDTH * BUFFER_HEIGHT / FRACTION); + if (this->buffer_ == nullptr) { + this->mark_failed("Buffer allocation failed"); + } + } + + void update() override { +#if ESPHOME_LOG_LEVEL == ESPHOME_LOG_LEVEL_VERBOSE + auto now = millis(); +#endif + if (this->is_failed()) { + return; + } + // for updates with a small buffer, we repeatedly call the writer_ function, clipping the height to a fraction of + // the display height, + for (this->start_line_ = 0; this->start_line_ < HEIGHT; this->start_line_ += HEIGHT / FRACTION) { +#if ESPHOME_LOG_LEVEL == ESPHOME_LOG_LEVEL_VERBOSE + auto lap = millis(); +#endif + this->end_line_ = this->start_line_ + HEIGHT / FRACTION; + if (this->auto_clear_enabled_) { + this->clear(); + } + if (this->page_ != nullptr) { + this->page_->get_writer()(*this); + } else if (this->writer_.has_value()) { + (*this->writer_)(*this); + } else { + this->test_card(); + } +#if ESPHOME_LOG_LEVEL == ESPHOME_LOG_LEVEL_VERBOSE + esph_log_v(TAG, "Drawing from line %d took %dms", this->start_line_, millis() - lap); + lap = millis(); +#endif + if (this->x_low_ > this->x_high_ || this->y_low_ > this->y_high_) + return; + esph_log_v(TAG, "x_low %d, y_low %d, x_high %d, y_high %d", this->x_low_, this->y_low_, this->x_high_, + this->y_high_); + // Some chips require that the drawing window be aligned on certain boundaries + this->x_low_ = this->x_low_ / ROUNDING * ROUNDING; + this->y_low_ = this->y_low_ / ROUNDING * ROUNDING; + this->x_high_ = (this->x_high_ + ROUNDING) / ROUNDING * ROUNDING - 1; + this->y_high_ = (this->y_high_ + ROUNDING) / ROUNDING * ROUNDING - 1; + int w = this->x_high_ - this->x_low_ + 1; + int h = this->y_high_ - this->y_low_ + 1; + this->write_to_display_(this->x_low_, this->y_low_, w, h, this->buffer_, this->x_low_, + this->y_low_ - this->start_line_, BUFFER_WIDTH - w); + // invalidate watermarks + this->x_low_ = WIDTH; + this->y_low_ = HEIGHT; + this->x_high_ = 0; + this->y_high_ = 0; +#if ESPHOME_LOG_LEVEL == ESPHOME_LOG_LEVEL_VERBOSE + esph_log_v(TAG, "Write to display took %dms", millis() - lap); + lap = millis(); +#endif + } +#if ESPHOME_LOG_LEVEL == ESPHOME_LOG_LEVEL_VERBOSE + esph_log_v(TAG, "Total update took %dms", millis() - now); +#endif + } + + // Draw a pixel at the given coordinates. + void draw_pixel_at(int x, int y, Color color) override { + if (!this->get_clipping().inside(x, y)) + return; + rotate_coordinates(x, y); + if (x < 0 || x >= WIDTH || y < this->start_line_ || y >= this->end_line_) + return; + this->buffer_[(y - this->start_line_) * BUFFER_WIDTH + x] = convert_color(color); + if (x < this->x_low_) { + this->x_low_ = x; + } + if (x > this->x_high_) { + this->x_high_ = x; + } + if (y < this->y_low_) { + this->y_low_ = y; + } + if (y > this->y_high_) { + this->y_high_ = y; + } + } + + // Fills the display with a color. + void fill(Color color) override { + this->x_low_ = 0; + this->y_low_ = this->start_line_; + this->x_high_ = WIDTH - 1; + this->y_high_ = this->end_line_ - 1; + std::fill_n(this->buffer_, HEIGHT * BUFFER_WIDTH / FRACTION, convert_color(color)); + } + + int get_width() override { + if constexpr (ROTATION == display::DISPLAY_ROTATION_90_DEGREES || ROTATION == display::DISPLAY_ROTATION_270_DEGREES) + return HEIGHT; + return WIDTH; + } + + int get_height() override { + if constexpr (ROTATION == display::DISPLAY_ROTATION_90_DEGREES || ROTATION == display::DISPLAY_ROTATION_270_DEGREES) + return WIDTH; + return HEIGHT; + } + + protected: + // Rotate the coordinates to match the display orientation. + static void rotate_coordinates(int &x, int &y) { + if constexpr (ROTATION == display::DISPLAY_ROTATION_180_DEGREES) { + x = WIDTH - x - 1; + y = HEIGHT - y - 1; + } else if constexpr (ROTATION == display::DISPLAY_ROTATION_90_DEGREES) { + auto tmp = x; + x = WIDTH - y - 1; + y = tmp; + } else if constexpr (ROTATION == display::DISPLAY_ROTATION_270_DEGREES) { + auto tmp = y; + y = HEIGHT - x - 1; + x = tmp; + } + } + + // Convert a color to the buffer pixel format. + static BUFFERTYPE convert_color(const Color &color) { + if constexpr (BUFFERPIXEL == PIXEL_MODE_8) { + return (color.red & 0xE0) | (color.g & 0xE0) >> 3 | color.b >> 6; + } else if constexpr (BUFFERPIXEL == PIXEL_MODE_16) { + if constexpr (IS_BIG_ENDIAN) { + return (color.r & 0xF8) | color.g >> 5 | (color.g & 0x1C) << 11 | (color.b & 0xF8) << 5; + } else { + return (color.r & 0xF8) << 8 | (color.g & 0xFC) << 3 | color.b >> 3; + } + } + return static_cast(0); + } + + BUFFERTYPE *buffer_{}; + uint16_t x_low_{WIDTH}; + uint16_t y_low_{HEIGHT}; + uint16_t x_high_{0}; + uint16_t y_high_{0}; + uint16_t start_line_{0}; + uint16_t end_line_{1}; +}; + } // namespace mipi_spi } // namespace esphome diff --git a/esphome/components/mipi_spi/models/__init__.py b/esphome/components/mipi_spi/models/__init__.py index e9726032d4..e69de29bb2 100644 --- a/esphome/components/mipi_spi/models/__init__.py +++ b/esphome/components/mipi_spi/models/__init__.py @@ -1,65 +0,0 @@ -from esphome.components.spi import TYPE_OCTAL, TYPE_QUAD, TYPE_SINGLE -import esphome.config_validation as cv -from esphome.const import CONF_HEIGHT, CONF_OFFSET_HEIGHT, CONF_OFFSET_WIDTH, CONF_WIDTH - -from .. import CONF_NATIVE_HEIGHT, CONF_NATIVE_WIDTH - -MADCTL_MY = 0x80 # Bit 7 Bottom to top -MADCTL_MX = 0x40 # Bit 6 Right to left -MADCTL_MV = 0x20 # Bit 5 Reverse Mode -MADCTL_ML = 0x10 # Bit 4 LCD refresh Bottom to top -MADCTL_RGB = 0x00 # Bit 3 Red-Green-Blue pixel order -MADCTL_BGR = 0x08 # Bit 3 Blue-Green-Red pixel order -MADCTL_MH = 0x04 # Bit 2 LCD refresh right to left - -# These bits are used instead of the above bits on some chips, where using MX and MY results in incorrect -# partial updates. -MADCTL_XFLIP = 0x02 # Mirror the display horizontally -MADCTL_YFLIP = 0x01 # Mirror the display vertically - -DELAY_FLAG = 0xFFF # Special flag to indicate a delay - - -def delay(ms): - return DELAY_FLAG, ms - - -class DriverChip: - models = {} - - def __init__( - self, - name: str, - modes=(TYPE_SINGLE, TYPE_QUAD, TYPE_OCTAL), - initsequence=None, - **defaults, - ): - name = name.upper() - self.name = name - self.modes = modes - self.initsequence = initsequence - self.defaults = defaults - DriverChip.models[name] = self - - def extend(self, name, **kwargs): - defaults = self.defaults.copy() - if ( - CONF_WIDTH in defaults - and CONF_OFFSET_WIDTH in kwargs - and CONF_NATIVE_WIDTH not in defaults - ): - defaults[CONF_NATIVE_WIDTH] = defaults[CONF_WIDTH] - if ( - CONF_HEIGHT in defaults - and CONF_OFFSET_HEIGHT in kwargs - and CONF_NATIVE_HEIGHT not in defaults - ): - defaults[CONF_NATIVE_HEIGHT] = defaults[CONF_HEIGHT] - defaults.update(kwargs) - return DriverChip(name, self.modes, initsequence=self.initsequence, **defaults) - - def get_default(self, key, fallback=False): - return self.defaults.get(key, fallback) - - def option(self, name, fallback=False): - return cv.Optional(name, default=self.get_default(name, fallback)) diff --git a/esphome/components/mipi_spi/models/adafruit.py b/esphome/components/mipi_spi/models/adafruit.py new file mode 100644 index 0000000000..0e91107bee --- /dev/null +++ b/esphome/components/mipi_spi/models/adafruit.py @@ -0,0 +1,30 @@ +from .ili import ST7789V + +ST7789V.extend( + "ADAFRUIT-FUNHOUSE", + height=240, + width=240, + offset_height=0, + offset_width=0, + cs_pin=40, + dc_pin=39, + reset_pin=41, + invert_colors=True, + mirror_x=True, + mirror_y=True, + data_rate="80MHz", +) + +ST7789V.extend( + "ADAFRUIT-S2-TFT-FEATHER", + height=240, + width=135, + offset_height=52, + offset_width=40, + cs_pin=7, + dc_pin=39, + reset_pin=40, + invert_colors=True, +) + +models = {} diff --git a/esphome/components/mipi_spi/models/amoled.py b/esphome/components/mipi_spi/models/amoled.py index 14277b243f..4d6c8da4b0 100644 --- a/esphome/components/mipi_spi/models/amoled.py +++ b/esphome/components/mipi_spi/models/amoled.py @@ -1,9 +1,22 @@ +from esphome.components.mipi import ( + MIPI, + MODE_RGB, + NORON, + PAGESEL, + PIXFMT, + SLPOUT, + SPIMODESEL, + SWIRE1, + SWIRE2, + TEON, + WCE, + WRAM, + WRCTRLD, + DriverChip, + delay, +) from esphome.components.spi import TYPE_QUAD -from .. import MODE_RGB -from . import DriverChip, delay -from .commands import MIPI, NORON, PAGESEL, PIXFMT, SLPOUT, SWIRE1, SWIRE2, TEON, WRAM - DriverChip( "T-DISPLAY-S3-AMOLED", width=240, @@ -14,7 +27,8 @@ DriverChip( bus_mode=TYPE_QUAD, brightness=0xD0, color_order=MODE_RGB, - initsequence=(SLPOUT,), # Requires early SLPOUT + no_slpout=True, # SLPOUT is in the init sequence, early + initsequence=(SLPOUT,), ) DriverChip( @@ -67,6 +81,30 @@ RM690B0 = DriverChip( ), ) -T4_S3_AMOLED = RM690B0.extend("T4-S3", width=450, offset_width=16, bus_mode=TYPE_QUAD) +T4_S3_AMOLED = RM690B0.extend( + "T4-S3", + width=450, + offset_width=16, + cs_pin=11, + reset_pin=13, + enable_pin=9, + bus_mode=TYPE_QUAD, +) + +CO5300 = DriverChip( + "CO5300", + brightness=0xD0, + color_order=MODE_RGB, + bus_mode=TYPE_QUAD, + no_slpout=True, + initsequence=( + (SLPOUT,), # Requires early SLPOUT + (PAGESEL, 0x00), + (SPIMODESEL, 0x80), + (WRCTRLD, 0x20), + (WCE, 0x00), + ), +) + models = {} diff --git a/esphome/components/mipi_spi/models/commands.py b/esphome/components/mipi_spi/models/commands.py deleted file mode 100644 index 032a6e6b2b..0000000000 --- a/esphome/components/mipi_spi/models/commands.py +++ /dev/null @@ -1,82 +0,0 @@ -# MIPI DBI commands - -NOP = 0x00 -SWRESET = 0x01 -RDDID = 0x04 -RDDST = 0x09 -RDMODE = 0x0A -RDMADCTL = 0x0B -RDPIXFMT = 0x0C -RDIMGFMT = 0x0D -RDSELFDIAG = 0x0F -SLEEP_IN = 0x10 -SLPIN = 0x10 -SLEEP_OUT = 0x11 -SLPOUT = 0x11 -PTLON = 0x12 -NORON = 0x13 -INVERT_OFF = 0x20 -INVOFF = 0x20 -INVERT_ON = 0x21 -INVON = 0x21 -ALL_ON = 0x23 -WRAM = 0x24 -GAMMASET = 0x26 -MIPI = 0x26 -DISPOFF = 0x28 -DISPON = 0x29 -CASET = 0x2A -PASET = 0x2B -RASET = 0x2B -RAMWR = 0x2C -WDATA = 0x2C -RAMRD = 0x2E -PTLAR = 0x30 -VSCRDEF = 0x33 -TEON = 0x35 -MADCTL = 0x36 -MADCTL_CMD = 0x36 -VSCRSADD = 0x37 -IDMOFF = 0x38 -IDMON = 0x39 -COLMOD = 0x3A -PIXFMT = 0x3A -GETSCANLINE = 0x45 -BRIGHTNESS = 0x51 -WRDISBV = 0x51 -RDDISBV = 0x52 -WRCTRLD = 0x53 -SWIRE1 = 0x5A -SWIRE2 = 0x5B -IFMODE = 0xB0 -FRMCTR1 = 0xB1 -FRMCTR2 = 0xB2 -FRMCTR3 = 0xB3 -INVCTR = 0xB4 -DFUNCTR = 0xB6 -ETMOD = 0xB7 -PWCTR1 = 0xC0 -PWCTR2 = 0xC1 -PWCTR3 = 0xC2 -PWCTR4 = 0xC3 -PWCTR5 = 0xC4 -VMCTR1 = 0xC5 -IFCTR = 0xC6 -VMCTR2 = 0xC7 -GMCTR = 0xC8 -SETEXTC = 0xC8 -PWSET = 0xD0 -VMCTR = 0xD1 -PWSETN = 0xD2 -RDID4 = 0xD3 -RDINDEX = 0xD9 -RDID1 = 0xDA -RDID2 = 0xDB -RDID3 = 0xDC -RDIDX = 0xDD -GMCTRP1 = 0xE0 -GMCTRN1 = 0xE1 -CSCON = 0xF0 -PWCTR6 = 0xF6 -ADJCTL3 = 0xF7 -PAGESEL = 0xFE diff --git a/esphome/components/mipi_spi/models/ili.py b/esphome/components/mipi_spi/models/ili.py index cc12b38f5d..0102c0f665 100644 --- a/esphome/components/mipi_spi/models/ili.py +++ b/esphome/components/mipi_spi/models/ili.py @@ -1,8 +1,4 @@ -from esphome.components.spi import TYPE_OCTAL - -from .. import MODE_RGB -from . import DriverChip, delay -from .commands import ( +from esphome.components.mipi import ( ADJCTL3, CSCON, DFUNCTR, @@ -18,6 +14,7 @@ from .commands import ( IFCTR, IFMODE, INVCTR, + MODE_RGB, NORON, PWCTR1, PWCTR2, @@ -32,7 +29,10 @@ from .commands import ( VMCTR1, VMCTR2, VSCRSADD, + DriverChip, + delay, ) +from esphome.components.spi import TYPE_OCTAL DriverChip( "M5CORE", diff --git a/esphome/components/mipi_spi/models/jc.py b/esphome/components/mipi_spi/models/jc.py index 449c5b87ae..5dbf049ded 100644 --- a/esphome/components/mipi_spi/models/jc.py +++ b/esphome/components/mipi_spi/models/jc.py @@ -1,10 +1,8 @@ +from esphome.components.mipi import MODE_RGB, DriverChip from esphome.components.spi import TYPE_QUAD import esphome.config_validation as cv from esphome.const import CONF_IGNORE_STRAPPING_WARNING, CONF_NUMBER -from .. import MODE_RGB -from . import DriverChip - AXS15231 = DriverChip( "AXS15231", draw_rounding=8, @@ -257,4 +255,233 @@ DriverChip( ), ) +DriverChip( + "JC3636W518V2", + height=360, + width=360, + offset_height=1, + draw_rounding=1, + cs_pin=10, + reset_pin=47, + invert_colors=True, + color_order=MODE_RGB, + bus_mode=TYPE_QUAD, + data_rate="40MHz", + initsequence=( + (0xF0, 0x28), + (0xF2, 0x28), + (0x73, 0xF0), + (0x7C, 0xD1), + (0x83, 0xE0), + (0x84, 0x61), + (0xF2, 0x82), + (0xF0, 0x00), + (0xF0, 0x01), + (0xF1, 0x01), + (0xB0, 0x56), + (0xB1, 0x4D), + (0xB2, 0x24), + (0xB4, 0x87), + (0xB5, 0x44), + (0xB6, 0x8B), + (0xB7, 0x40), + (0xB8, 0x86), + (0xBA, 0x00), + (0xBB, 0x08), + (0xBC, 0x08), + (0xBD, 0x00), + (0xC0, 0x80), + (0xC1, 0x10), + (0xC2, 0x37), + (0xC3, 0x80), + (0xC4, 0x10), + (0xC5, 0x37), + (0xC6, 0xA9), + (0xC7, 0x41), + (0xC8, 0x01), + (0xC9, 0xA9), + (0xCA, 0x41), + (0xCB, 0x01), + (0xD0, 0x91), + (0xD1, 0x68), + (0xD2, 0x68), + (0xF5, 0x00, 0xA5), + (0xDD, 0x4F), + (0xDE, 0x4F), + (0xF1, 0x10), + (0xF0, 0x00), + (0xF0, 0x02), + ( + 0xE0, + 0xF0, + 0x0A, + 0x10, + 0x09, + 0x09, + 0x36, + 0x35, + 0x33, + 0x4A, + 0x29, + 0x15, + 0x15, + 0x2E, + 0x34, + ), + ( + 0xE1, + 0xF0, + 0x0A, + 0x0F, + 0x08, + 0x08, + 0x05, + 0x34, + 0x33, + 0x4A, + 0x39, + 0x15, + 0x15, + 0x2D, + 0x33, + ), + (0xF0, 0x10), + (0xF3, 0x10), + (0xE0, 0x07), + (0xE1, 0x00), + (0xE2, 0x00), + (0xE3, 0x00), + (0xE4, 0xE0), + (0xE5, 0x06), + (0xE6, 0x21), + (0xE7, 0x01), + (0xE8, 0x05), + (0xE9, 0x02), + (0xEA, 0xDA), + (0xEB, 0x00), + (0xEC, 0x00), + (0xED, 0x0F), + (0xEE, 0x00), + (0xEF, 0x00), + (0xF8, 0x00), + (0xF9, 0x00), + (0xFA, 0x00), + (0xFB, 0x00), + (0xFC, 0x00), + (0xFD, 0x00), + (0xFE, 0x00), + (0xFF, 0x00), + (0x60, 0x40), + (0x61, 0x04), + (0x62, 0x00), + (0x63, 0x42), + (0x64, 0xD9), + (0x65, 0x00), + (0x66, 0x00), + (0x67, 0x00), + (0x68, 0x00), + (0x69, 0x00), + (0x6A, 0x00), + (0x6B, 0x00), + (0x70, 0x40), + (0x71, 0x03), + (0x72, 0x00), + (0x73, 0x42), + (0x74, 0xD8), + (0x75, 0x00), + (0x76, 0x00), + (0x77, 0x00), + (0x78, 0x00), + (0x79, 0x00), + (0x7A, 0x00), + (0x7B, 0x00), + (0x80, 0x48), + (0x81, 0x00), + (0x82, 0x06), + (0x83, 0x02), + (0x84, 0xD6), + (0x85, 0x04), + (0x86, 0x00), + (0x87, 0x00), + (0x88, 0x48), + (0x89, 0x00), + (0x8A, 0x08), + (0x8B, 0x02), + (0x8C, 0xD8), + (0x8D, 0x04), + (0x8E, 0x00), + (0x8F, 0x00), + (0x90, 0x48), + (0x91, 0x00), + (0x92, 0x0A), + (0x93, 0x02), + (0x94, 0xDA), + (0x95, 0x04), + (0x96, 0x00), + (0x97, 0x00), + (0x98, 0x48), + (0x99, 0x00), + (0x9A, 0x0C), + (0x9B, 0x02), + (0x9C, 0xDC), + (0x9D, 0x04), + (0x9E, 0x00), + (0x9F, 0x00), + (0xA0, 0x48), + (0xA1, 0x00), + (0xA2, 0x05), + (0xA3, 0x02), + (0xA4, 0xD5), + (0xA5, 0x04), + (0xA6, 0x00), + (0xA7, 0x00), + (0xA8, 0x48), + (0xA9, 0x00), + (0xAA, 0x07), + (0xAB, 0x02), + (0xAC, 0xD7), + (0xAD, 0x04), + (0xAE, 0x00), + (0xAF, 0x00), + (0xB0, 0x48), + (0xB1, 0x00), + (0xB2, 0x09), + (0xB3, 0x02), + (0xB4, 0xD9), + (0xB5, 0x04), + (0xB6, 0x00), + (0xB7, 0x00), + (0xB8, 0x48), + (0xB9, 0x00), + (0xBA, 0x0B), + (0xBB, 0x02), + (0xBC, 0xDB), + (0xBD, 0x04), + (0xBE, 0x00), + (0xBF, 0x00), + (0xC0, 0x10), + (0xC1, 0x47), + (0xC2, 0x56), + (0xC3, 0x65), + (0xC4, 0x74), + (0xC5, 0x88), + (0xC6, 0x99), + (0xC7, 0x01), + (0xC8, 0xBB), + (0xC9, 0xAA), + (0xD0, 0x10), + (0xD1, 0x47), + (0xD2, 0x56), + (0xD3, 0x65), + (0xD4, 0x74), + (0xD5, 0x88), + (0xD6, 0x99), + (0xD7, 0x01), + (0xD8, 0xBB), + (0xD9, 0xAA), + (0xF3, 0x01), + (0xF0, 0x00), + ), +) + models = {} diff --git a/esphome/components/mipi_spi/models/lilygo.py b/esphome/components/mipi_spi/models/lilygo.py index dd6f9c02f7..13ddc67465 100644 --- a/esphome/components/mipi_spi/models/lilygo.py +++ b/esphome/components/mipi_spi/models/lilygo.py @@ -1,6 +1,6 @@ +from esphome.components.mipi import MODE_BGR from esphome.components.spi import TYPE_OCTAL -from .. import MODE_BGR from .ili import ST7789V, ST7796 ST7789V.extend( diff --git a/esphome/components/mipi_spi/models/waveshare.py b/esphome/components/mipi_spi/models/waveshare.py index 6d14f56fc6..e4e090da2e 100644 --- a/esphome/components/mipi_spi/models/waveshare.py +++ b/esphome/components/mipi_spi/models/waveshare.py @@ -1,5 +1,9 @@ -from . import DriverChip +from esphome.components.mipi import DriverChip +import esphome.config_validation as cv + +from .amoled import CO5300 from .ili import ILI9488_A +from .jc import AXS15231 DriverChip( "WAVESHARE-4-TFT", @@ -128,6 +132,7 @@ DriverChip( ILI9488_A.extend( "PICO-RESTOUCH-LCD-3.5", + swap_xy=cv.UNDEFINED, spi_16=True, pixel_mode="16bit", mirror_x=True, @@ -137,3 +142,23 @@ ILI9488_A.extend( data_rate="20MHz", invert_colors=True, ) + +CO5300.extend( + "WAVESHARE-ESP32-S3-TOUCH-AMOLED-1.75", + width=466, + height=466, + pixel_mode="16bit", + offset_height=0, + offset_width=6, + cs_pin=12, + reset_pin=39, +) + +AXS15231.extend( + "WAVESHARE-ESP32-S3-TOUCH-LCD-3.49", + width=172, + height=640, + data_rate="80MHz", + cs_pin=9, + reset_pin=21, +) diff --git a/esphome/components/mixer/speaker/__init__.py b/esphome/components/mixer/speaker/__init__.py index a451f2b7b4..46729f8eda 100644 --- a/esphome/components/mixer/speaker/__init__.py +++ b/esphome/components/mixer/speaker/__init__.py @@ -124,11 +124,10 @@ async def to_code(config): if task_stack_in_psram := config.get(CONF_TASK_STACK_IN_PSRAM): cg.add(var.set_task_stack_in_psram(task_stack_in_psram)) - if task_stack_in_psram: - if config[CONF_TASK_STACK_IN_PSRAM]: - esp32.add_idf_sdkconfig_option( - "CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY", True - ) + if task_stack_in_psram and config[CONF_TASK_STACK_IN_PSRAM]: + esp32.add_idf_sdkconfig_option( + "CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY", True + ) for speaker_config in config[CONF_SOURCE_SPEAKERS]: source_speaker = cg.new_Pvariable(speaker_config[CONF_ID]) diff --git a/esphome/components/mixer/speaker/mixer_speaker.cpp b/esphome/components/mixer/speaker/mixer_speaker.cpp index fc0517c7be..b0b64f5709 100644 --- a/esphome/components/mixer/speaker/mixer_speaker.cpp +++ b/esphome/components/mixer/speaker/mixer_speaker.cpp @@ -572,7 +572,7 @@ void MixerSpeaker::audio_mixer_task(void *params) { } } else { // Determine how many frames to mix - for (int i = 0; i < transfer_buffers_with_data.size(); ++i) { + for (size_t i = 0; i < transfer_buffers_with_data.size(); ++i) { const uint32_t frames_available_in_buffer = speakers_with_data[i]->get_audio_stream_info().bytes_to_frames(transfer_buffers_with_data[i]->available()); frames_to_mix = std::min(frames_to_mix, frames_available_in_buffer); @@ -581,7 +581,7 @@ void MixerSpeaker::audio_mixer_task(void *params) { audio::AudioStreamInfo primary_stream_info = speakers_with_data[0]->get_audio_stream_info(); // Mix two streams together - for (int i = 1; i < transfer_buffers_with_data.size(); ++i) { + for (size_t i = 1; i < transfer_buffers_with_data.size(); ++i) { mix_audio_samples(primary_buffer, primary_stream_info, reinterpret_cast(transfer_buffers_with_data[i]->get_buffer_start()), speakers_with_data[i]->get_audio_stream_info(), @@ -596,7 +596,7 @@ void MixerSpeaker::audio_mixer_task(void *params) { } // Update source transfer buffer lengths and add new audio durations to the source speaker pending playbacks - for (int i = 0; i < transfer_buffers_with_data.size(); ++i) { + for (size_t i = 0; i < transfer_buffers_with_data.size(); ++i) { transfer_buffers_with_data[i]->decrease_buffer_length( speakers_with_data[i]->get_audio_stream_info().frames_to_bytes(frames_to_mix)); speakers_with_data[i]->pending_playback_frames_ += frames_to_mix; diff --git a/esphome/components/mlx90393/sensor.py b/esphome/components/mlx90393/sensor.py index 372bb05bda..d93c379506 100644 --- a/esphome/components/mlx90393/sensor.py +++ b/esphome/components/mlx90393/sensor.py @@ -63,11 +63,13 @@ def _validate(config): raise cv.Invalid( f"{axis}: {CONF_RESOLUTION} cannot be {res} with {CONF_TEMPERATURE_COMPENSATION} enabled" ) - if config[CONF_HALLCONF] == 0xC: - if (config[CONF_OVERSAMPLING], config[CONF_FILTER]) in [(0, 0), (1, 0), (0, 1)]: - raise cv.Invalid( - f"{CONF_OVERSAMPLING}=={config[CONF_OVERSAMPLING]} and {CONF_FILTER}=={config[CONF_FILTER]} not allowed with {CONF_HALLCONF}=={config[CONF_HALLCONF]:#02x}" - ) + if config[CONF_HALLCONF] == 0xC and ( + config[CONF_OVERSAMPLING], + config[CONF_FILTER], + ) in [(0, 0), (1, 0), (0, 1)]: + raise cv.Invalid( + f"{CONF_OVERSAMPLING}=={config[CONF_OVERSAMPLING]} and {CONF_FILTER}=={config[CONF_FILTER]} not allowed with {CONF_HALLCONF}=={config[CONF_HALLCONF]:#02x}" + ) return config diff --git a/esphome/components/mlx90393/sensor_mlx90393.cpp b/esphome/components/mlx90393/sensor_mlx90393.cpp index 96749cd378..21a5b3a829 100644 --- a/esphome/components/mlx90393/sensor_mlx90393.cpp +++ b/esphome/components/mlx90393/sensor_mlx90393.cpp @@ -103,7 +103,6 @@ bool MLX90393Cls::apply_all_settings_() { } void MLX90393Cls::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // note the two arguments A0 and A1 which are used to construct an i2c address // we can hard-code these because we never actually use the constructed address // see the transceive function above, which uses the address from I2CComponent diff --git a/esphome/components/mlx90614/mlx90614.cpp b/esphome/components/mlx90614/mlx90614.cpp index afc565d38b..8e53b9e3c3 100644 --- a/esphome/components/mlx90614/mlx90614.cpp +++ b/esphome/components/mlx90614/mlx90614.cpp @@ -28,7 +28,6 @@ static const uint8_t MLX90614_ID4 = 0x3F; static const char *const TAG = "mlx90614"; void MLX90614Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (!this->write_emissivity_()) { ESP_LOGE(TAG, ESP_LOG_MSG_COMM_FAIL); this->mark_failed(); @@ -51,28 +50,13 @@ bool MLX90614Component::write_emissivity_() { return true; } -uint8_t MLX90614Component::crc8_pec_(const uint8_t *data, uint8_t len) { - uint8_t crc = 0; - for (uint8_t i = 0; i < len; i++) { - uint8_t in = data[i]; - for (uint8_t j = 0; j < 8; j++) { - bool carry = (crc ^ in) & 0x80; - crc <<= 1; - if (carry) - crc ^= 0x07; - in <<= 1; - } - } - return crc; -} - bool MLX90614Component::write_bytes_(uint8_t reg, uint16_t data) { uint8_t buf[5]; buf[0] = this->address_ << 1; buf[1] = reg; buf[2] = data & 0xFF; buf[3] = data >> 8; - buf[4] = this->crc8_pec_(buf, 4); + buf[4] = crc8(buf, 4, 0x00, 0x07, true); return this->write_bytes(reg, buf + 2, 3); } @@ -91,18 +75,18 @@ float MLX90614Component::get_setup_priority() const { return setup_priority::DAT void MLX90614Component::update() { uint8_t emissivity[3]; - if (this->read_register(MLX90614_EMISSIVITY, emissivity, 3, false) != i2c::ERROR_OK) { + if (this->read_register(MLX90614_EMISSIVITY, emissivity, 3) != i2c::ERROR_OK) { this->status_set_warning(); return; } uint8_t raw_object[3]; - if (this->read_register(MLX90614_TEMPERATURE_OBJECT_1, raw_object, 3, false) != i2c::ERROR_OK) { + if (this->read_register(MLX90614_TEMPERATURE_OBJECT_1, raw_object, 3) != i2c::ERROR_OK) { this->status_set_warning(); return; } uint8_t raw_ambient[3]; - if (this->read_register(MLX90614_TEMPERATURE_AMBIENT, raw_ambient, 3, false) != i2c::ERROR_OK) { + if (this->read_register(MLX90614_TEMPERATURE_AMBIENT, raw_ambient, 3) != i2c::ERROR_OK) { this->status_set_warning(); return; } diff --git a/esphome/components/mlx90614/mlx90614.h b/esphome/components/mlx90614/mlx90614.h index b6bd44172d..fa6fb523bb 100644 --- a/esphome/components/mlx90614/mlx90614.h +++ b/esphome/components/mlx90614/mlx90614.h @@ -22,7 +22,6 @@ class MLX90614Component : public PollingComponent, public i2c::I2CDevice { protected: bool write_emissivity_(); - uint8_t crc8_pec_(const uint8_t *data, uint8_t len); bool write_bytes_(uint8_t reg, uint16_t data); sensor::Sensor *ambient_sensor_{nullptr}; diff --git a/esphome/components/mmc5603/mmc5603.cpp b/esphome/components/mmc5603/mmc5603.cpp index 7f78f9592a..f0d1044f3f 100644 --- a/esphome/components/mmc5603/mmc5603.cpp +++ b/esphome/components/mmc5603/mmc5603.cpp @@ -31,7 +31,6 @@ static const uint8_t MMC56X3_CTRL2_REG = 0x1D; static const uint8_t MMC5603_ODR_REG = 0x1A; void MMC5603Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t id = 0; if (!this->read_byte(MMC56X3_PRODUCT_ID, &id)) { this->error_code_ = COMMUNICATION_FAILED; @@ -129,21 +128,21 @@ void MMC5603Component::update() { raw_x |= buffer[1] << 4; raw_x |= buffer[2] << 0; - const float x = 0.0625 * (raw_x - 524288); + const float x = 0.00625 * (raw_x - 524288); int32_t raw_y = 0; raw_y |= buffer[3] << 12; raw_y |= buffer[4] << 4; raw_y |= buffer[5] << 0; - const float y = 0.0625 * (raw_y - 524288); + const float y = 0.00625 * (raw_y - 524288); int32_t raw_z = 0; raw_z |= buffer[6] << 12; raw_z |= buffer[7] << 4; raw_z |= buffer[8] << 0; - const float z = 0.0625 * (raw_z - 524288); + const float z = 0.00625 * (raw_z - 524288); const float heading = atan2f(0.0f - x, y) * 180.0f / M_PI; ESP_LOGD(TAG, "Got x=%0.02fµT y=%0.02fµT z=%0.02fµT heading=%0.01f°", x, y, z, heading); diff --git a/esphome/components/mmc5983/mmc5983.cpp b/esphome/components/mmc5983/mmc5983.cpp index d5394da618..1e0065020c 100644 --- a/esphome/components/mmc5983/mmc5983.cpp +++ b/esphome/components/mmc5983/mmc5983.cpp @@ -67,8 +67,6 @@ void MMC5983Component::update() { } void MMC5983Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // Verify product id. const uint8_t mmc5983_product_id = 0x30; uint8_t id; diff --git a/esphome/components/modbus/modbus.cpp b/esphome/components/modbus/modbus.cpp index 6350f43ef6..20271b4bdb 100644 --- a/esphome/components/modbus/modbus.cpp +++ b/esphome/components/modbus/modbus.cpp @@ -66,7 +66,10 @@ bool Modbus::parse_modbus_byte_(uint8_t byte) { uint8_t data_offset = 3; // Per https://modbus.org/docs/Modbus_Application_Protocol_V1_1b3.pdf Ch 5 User-Defined function codes - if (((function_code >= 65) && (function_code <= 72)) || ((function_code >= 100) && (function_code <= 110))) { + if (((function_code >= FUNCTION_CODE_USER_DEFINED_SPACE_1_INIT) && + (function_code <= FUNCTION_CODE_USER_DEFINED_SPACE_1_END)) || + ((function_code >= FUNCTION_CODE_USER_DEFINED_SPACE_2_INIT) && + (function_code <= FUNCTION_CODE_USER_DEFINED_SPACE_2_END))) { // Handle user-defined function, since we don't know how big this ought to be, // ideally we should delegate the entire length detection to whatever handler is // installed, but wait, there is the CRC, and if we get a hit there is a good @@ -91,10 +94,14 @@ bool Modbus::parse_modbus_byte_(uint8_t byte) { } else { // data starts at 2 and length is 4 for read registers commands if (this->role == ModbusRole::SERVER) { - if (function_code == 0x1 || function_code == 0x3 || function_code == 0x4 || function_code == 0x6) { + if (function_code == ModbusFunctionCode::READ_COILS || + function_code == ModbusFunctionCode::READ_DISCRETE_INPUTS || + function_code == ModbusFunctionCode::READ_HOLDING_REGISTERS || + function_code == ModbusFunctionCode::READ_INPUT_REGISTERS || + function_code == ModbusFunctionCode::WRITE_SINGLE_REGISTER) { data_offset = 2; data_len = 4; - } else if (function_code == 0x10) { + } else if (function_code == ModbusFunctionCode::WRITE_MULTIPLE_REGISTERS) { if (at < 6) { return true; } @@ -104,7 +111,10 @@ bool Modbus::parse_modbus_byte_(uint8_t byte) { } } else { // the response for write command mirrors the requests and data starts at offset 2 instead of 3 for read commands - if (function_code == 0x5 || function_code == 0x06 || function_code == 0xF || function_code == 0x10) { + if (function_code == ModbusFunctionCode::WRITE_SINGLE_COIL || + function_code == ModbusFunctionCode::WRITE_SINGLE_REGISTER || + function_code == ModbusFunctionCode::WRITE_MULTIPLE_COILS || + function_code == ModbusFunctionCode::WRITE_MULTIPLE_REGISTERS) { data_offset = 2; data_len = 4; } @@ -112,7 +122,7 @@ bool Modbus::parse_modbus_byte_(uint8_t byte) { // Error ( msb indicates error ) // response format: Byte[0] = device address, Byte[1] function code | 0x80 , Byte[2] exception code, Byte[3-4] crc - if ((function_code & 0x80) == 0x80) { + if ((function_code & FUNCTION_CODE_EXCEPTION_MASK) == FUNCTION_CODE_EXCEPTION_MASK) { data_offset = 2; data_len = 1; } @@ -143,10 +153,10 @@ bool Modbus::parse_modbus_byte_(uint8_t byte) { if (device->address_ == address) { found = true; // Is it an error response? - if ((function_code & 0x80) == 0x80) { + if ((function_code & FUNCTION_CODE_EXCEPTION_MASK) == FUNCTION_CODE_EXCEPTION_MASK) { ESP_LOGD(TAG, "Modbus error function code: 0x%X exception: %d", function_code, raw[2]); if (waiting_for_response != 0) { - device->on_modbus_error(function_code & 0x7F, raw[2]); + device->on_modbus_error(function_code & FUNCTION_CODE_MASK, raw[2]); } else { // Ignore modbus exception not related to a pending command ESP_LOGD(TAG, "Ignoring Modbus error - not expecting a response"); @@ -154,12 +164,14 @@ bool Modbus::parse_modbus_byte_(uint8_t byte) { continue; } if (this->role == ModbusRole::SERVER) { - if (function_code == 0x3 || function_code == 0x4) { + if (function_code == ModbusFunctionCode::READ_HOLDING_REGISTERS || + function_code == ModbusFunctionCode::READ_INPUT_REGISTERS) { device->on_modbus_read_registers(function_code, uint16_t(data[1]) | (uint16_t(data[0]) << 8), uint16_t(data[3]) | (uint16_t(data[2]) << 8)); continue; } - if (function_code == 0x6 || function_code == 0x10) { + if (function_code == ModbusFunctionCode::WRITE_SINGLE_REGISTER || + function_code == ModbusFunctionCode::WRITE_MULTIPLE_REGISTERS) { device->on_modbus_write_registers(function_code, data); continue; } @@ -199,7 +211,7 @@ void Modbus::send(uint8_t address, uint8_t function_code, uint16_t start_address // Only check max number of registers for standard function codes // Some devices use non standard codes like 0x43 - if (number_of_entities > MAX_VALUES && function_code <= 0x10) { + if (number_of_entities > MAX_VALUES && function_code <= ModbusFunctionCode::WRITE_MULTIPLE_REGISTERS) { ESP_LOGE(TAG, "send too many values %d max=%zu", number_of_entities, MAX_VALUES); return; } @@ -210,15 +222,17 @@ void Modbus::send(uint8_t address, uint8_t function_code, uint16_t start_address if (this->role == ModbusRole::CLIENT) { data.push_back(start_address >> 8); data.push_back(start_address >> 0); - if (function_code != 0x5 && function_code != 0x6) { + if (function_code != ModbusFunctionCode::WRITE_SINGLE_COIL && + function_code != ModbusFunctionCode::WRITE_SINGLE_REGISTER) { data.push_back(number_of_entities >> 8); data.push_back(number_of_entities >> 0); } } if (payload != nullptr) { - if (this->role == ModbusRole::SERVER || function_code == 0xF || function_code == 0x10) { // Write multiple - data.push_back(payload_len); // Byte count is required for write + if (this->role == ModbusRole::SERVER || function_code == ModbusFunctionCode::WRITE_MULTIPLE_COILS || + function_code == ModbusFunctionCode::WRITE_MULTIPLE_REGISTERS) { // Write multiple + data.push_back(payload_len); // Byte count is required for write } else { payload_len = 2; // Write single register or coil } diff --git a/esphome/components/modbus/modbus.h b/esphome/components/modbus/modbus.h index ec35612690..fac74aaadf 100644 --- a/esphome/components/modbus/modbus.h +++ b/esphome/components/modbus/modbus.h @@ -3,6 +3,8 @@ #include "esphome/core/component.h" #include "esphome/components/uart/uart.h" +#include "esphome/components/modbus/modbus_definitions.h" + #include namespace esphome { @@ -65,12 +67,12 @@ class ModbusDevice { this->parent_->send(this->address_, function, start_address, number_of_entities, payload_len, payload); } void send_raw(const std::vector &payload) { this->parent_->send_raw(payload); } - void send_error(uint8_t function_code, uint8_t exception_code) { + void send_error(uint8_t function_code, ModbusExceptionCode exception_code) { std::vector error_response; error_response.reserve(3); error_response.push_back(this->address_); - error_response.push_back(function_code | 0x80); - error_response.push_back(exception_code); + error_response.push_back(function_code | FUNCTION_CODE_EXCEPTION_MASK); + error_response.push_back(static_cast(exception_code)); this->send_raw(error_response); } // If more than one device is connected block sending a new command before a response is received diff --git a/esphome/components/modbus/modbus_definitions.h b/esphome/components/modbus/modbus_definitions.h new file mode 100644 index 0000000000..07f101ae4c --- /dev/null +++ b/esphome/components/modbus/modbus_definitions.h @@ -0,0 +1,86 @@ +#pragma once + +#include "esphome/core/component.h" + +namespace esphome { +namespace modbus { + +/// Modbus definitions from specs: +/// https://modbus.org/docs/Modbus_Application_Protocol_V1_1b3.pdf +// 5 Function Code Categories +const uint8_t FUNCTION_CODE_USER_DEFINED_SPACE_1_INIT = 65; // 0x41 +const uint8_t FUNCTION_CODE_USER_DEFINED_SPACE_1_END = 72; // 0x48 + +const uint8_t FUNCTION_CODE_USER_DEFINED_SPACE_2_INIT = 100; // 0x64 +const uint8_t FUNCTION_CODE_USER_DEFINED_SPACE_2_END = 110; // 0x6E + +enum class ModbusFunctionCode : uint8_t { + CUSTOM = 0x00, + READ_COILS = 0x01, + READ_DISCRETE_INPUTS = 0x02, + READ_HOLDING_REGISTERS = 0x03, + READ_INPUT_REGISTERS = 0x04, + WRITE_SINGLE_COIL = 0x05, + WRITE_SINGLE_REGISTER = 0x06, + READ_EXCEPTION_STATUS = 0x07, // not implemented + DIAGNOSTICS = 0x08, // not implemented + GET_COMM_EVENT_COUNTER = 0x0B, // not implemented + GET_COMM_EVENT_LOG = 0x0C, // not implemented + WRITE_MULTIPLE_COILS = 0x0F, + WRITE_MULTIPLE_REGISTERS = 0x10, + REPORT_SERVER_ID = 0x11, // not implemented + READ_FILE_RECORD = 0x14, // not implemented + WRITE_FILE_RECORD = 0x15, // not implemented + MASK_WRITE_REGISTER = 0x16, // not implemented + READ_WRITE_MULTIPLE_REGISTERS = 0x17, // not implemented + READ_FIFO_QUEUE = 0x18, // not implemented +}; + +/*Allow comparison operators between ModbusFunctionCode and uint8_t*/ +inline bool operator==(ModbusFunctionCode lhs, uint8_t rhs) { return static_cast(lhs) == rhs; } +inline bool operator==(uint8_t lhs, ModbusFunctionCode rhs) { return lhs == static_cast(rhs); } +inline bool operator!=(ModbusFunctionCode lhs, uint8_t rhs) { return !(static_cast(lhs) == rhs); } +inline bool operator!=(uint8_t lhs, ModbusFunctionCode rhs) { return !(lhs == static_cast(rhs)); } +inline bool operator<(ModbusFunctionCode lhs, uint8_t rhs) { return static_cast(lhs) < rhs; } +inline bool operator<(uint8_t lhs, ModbusFunctionCode rhs) { return lhs < static_cast(rhs); } +inline bool operator<=(ModbusFunctionCode lhs, uint8_t rhs) { return static_cast(lhs) <= rhs; } +inline bool operator<=(uint8_t lhs, ModbusFunctionCode rhs) { return lhs <= static_cast(rhs); } +inline bool operator>(ModbusFunctionCode lhs, uint8_t rhs) { return static_cast(lhs) > rhs; } +inline bool operator>(uint8_t lhs, ModbusFunctionCode rhs) { return lhs > static_cast(rhs); } +inline bool operator>=(ModbusFunctionCode lhs, uint8_t rhs) { return static_cast(lhs) >= rhs; } +inline bool operator>=(uint8_t lhs, ModbusFunctionCode rhs) { return lhs >= static_cast(rhs); } + +// 4.3 MODBUS Data model +enum class ModbusRegisterType : uint8_t { + CUSTOM = 0x00, + COIL = 0x01, + DISCRETE_INPUT = 0x02, + HOLDING = 0x03, + READ = 0x04, +}; + +// 7 MODBUS Exception Responses: +const uint8_t FUNCTION_CODE_MASK = 0x7F; +const uint8_t FUNCTION_CODE_EXCEPTION_MASK = 0x80; + +enum class ModbusExceptionCode : uint8_t { + ILLEGAL_FUNCTION = 0x01, + ILLEGAL_DATA_ADDRESS = 0x02, + ILLEGAL_DATA_VALUE = 0x03, + SERVICE_DEVICE_FAILURE = 0x04, + ACKNOWLEDGE = 0x05, + SERVER_DEVICE_BUSY = 0x06, + MEMORY_PARITY_ERROR = 0x08, + GATEWAY_PATH_UNAVAILABLE = 0x0A, + GATEWAY_TARGET_DEVICE_FAILED_TO_RESPOND = 0x0B, +}; + +// 6.12 16 (0x10) Write Multiple registers: +const uint8_t MAX_NUM_OF_REGISTERS_TO_WRITE = 123; // 0x7B + +// 6.3 03 (0x03) Read Holding Registers +// 6.4 04 (0x04) Read Input Registers +const uint8_t MAX_NUM_OF_REGISTERS_TO_READ = 125; // 0x7D +/// End of Modbus definitions +} // namespace modbus +} // namespace esphome diff --git a/esphome/components/modbus_controller/__init__.py b/esphome/components/modbus_controller/__init__.py index 5ab82f5e17..28f3326c47 100644 --- a/esphome/components/modbus_controller/__init__.py +++ b/esphome/components/modbus_controller/__init__.py @@ -20,6 +20,7 @@ from .const import ( CONF_BYTE_OFFSET, CONF_COMMAND_THROTTLE, CONF_CUSTOM_COMMAND, + CONF_ENABLED, CONF_FORCE_NEW_RANGE, CONF_MAX_CMD_RETRIES, CONF_MODBUS_CONTROLLER_ID, @@ -28,8 +29,11 @@ from .const import ( CONF_ON_OFFLINE, CONF_ON_ONLINE, CONF_REGISTER_COUNT, + CONF_REGISTER_LAST_ADDRESS, CONF_REGISTER_TYPE, + CONF_REGISTER_VALUE, CONF_RESPONSE_SIZE, + CONF_SERVER_COURTESY_RESPONSE, CONF_SKIP_UPDATES, CONF_VALUE_TYPE, ) @@ -49,6 +53,7 @@ ModbusController = modbus_controller_ns.class_( ) SensorItem = modbus_controller_ns.struct("SensorItem") +ServerCourtesyResponse = modbus_controller_ns.struct("ServerCourtesyResponse") ServerRegister = modbus_controller_ns.struct("ServerRegister") ModbusFunctionCode_ns = modbus_controller_ns.namespace("ModbusFunctionCode") @@ -143,6 +148,14 @@ ModbusOfflineTrigger = modbus_controller_ns.class_( _LOGGER = logging.getLogger(__name__) +SERVER_COURTESY_RESPONSE_SCHEMA = cv.Schema( + { + cv.Optional(CONF_ENABLED, default=False): cv.boolean, + cv.Optional(CONF_REGISTER_LAST_ADDRESS, default=0xFFFF): cv.hex_uint16_t, + cv.Optional(CONF_REGISTER_VALUE, default=0): cv.hex_uint16_t, + } +) + ModbusServerRegisterSchema = cv.Schema( { cv.GenerateID(): cv.declare_id(ServerRegister), @@ -162,6 +175,7 @@ CONFIG_SCHEMA = cv.All( cv.Optional( CONF_COMMAND_THROTTLE, default="0ms" ): cv.positive_time_period_milliseconds, + cv.Optional(CONF_SERVER_COURTESY_RESPONSE): SERVER_COURTESY_RESPONSE_SCHEMA, cv.Optional(CONF_MAX_CMD_RETRIES, default=4): cv.positive_int, cv.Optional(CONF_OFFLINE_SKIP_UPDATES, default=0): cv.positive_int, cv.Optional( @@ -232,7 +246,7 @@ def validate_modbus_register(config): def _final_validate(config): - if CONF_SERVER_REGISTERS in config: + if CONF_SERVER_COURTESY_RESPONSE in config or CONF_SERVER_REGISTERS in config: return modbus.final_validate_modbus_device("modbus_controller", role="server")( config ) @@ -299,6 +313,20 @@ async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) cg.add(var.set_allow_duplicate_commands(config[CONF_ALLOW_DUPLICATE_COMMANDS])) cg.add(var.set_command_throttle(config[CONF_COMMAND_THROTTLE])) + if server_courtesy_response := config.get(CONF_SERVER_COURTESY_RESPONSE): + cg.add( + var.set_server_courtesy_response( + cg.StructInitializer( + ServerCourtesyResponse, + ("enabled", server_courtesy_response[CONF_ENABLED]), + ( + "register_last_address", + server_courtesy_response[CONF_REGISTER_LAST_ADDRESS], + ), + ("register_value", server_courtesy_response[CONF_REGISTER_VALUE]), + ) + ) + ) cg.add(var.set_max_cmd_retries(config[CONF_MAX_CMD_RETRIES])) cg.add(var.set_offline_skip_updates(config[CONF_OFFLINE_SKIP_UPDATES])) if CONF_SERVER_REGISTERS in config: diff --git a/esphome/components/modbus_controller/const.py b/esphome/components/modbus_controller/const.py index 4d39e48dcd..ee0b5fc633 100644 --- a/esphome/components/modbus_controller/const.py +++ b/esphome/components/modbus_controller/const.py @@ -2,6 +2,7 @@ CONF_ALLOW_DUPLICATE_COMMANDS = "allow_duplicate_commands" CONF_BITMASK = "bitmask" CONF_BYTE_OFFSET = "byte_offset" CONF_COMMAND_THROTTLE = "command_throttle" +CONF_ENABLED = "enabled" CONF_OFFLINE_SKIP_UPDATES = "offline_skip_updates" CONF_CUSTOM_COMMAND = "custom_command" CONF_FORCE_NEW_RANGE = "force_new_range" @@ -13,8 +14,11 @@ CONF_ON_ONLINE = "on_online" CONF_ON_OFFLINE = "on_offline" CONF_RAW_ENCODE = "raw_encode" CONF_REGISTER_COUNT = "register_count" +CONF_REGISTER_LAST_ADDRESS = "register_last_address" CONF_REGISTER_TYPE = "register_type" +CONF_REGISTER_VALUE = "register_value" CONF_RESPONSE_SIZE = "response_size" +CONF_SERVER_COURTESY_RESPONSE = "server_courtesy_response" CONF_SKIP_UPDATES = "skip_updates" CONF_USE_WRITE_MULTIPLE = "use_write_multiple" CONF_VALUE_TYPE = "value_type" diff --git a/esphome/components/modbus_controller/modbus_controller.cpp b/esphome/components/modbus_controller/modbus_controller.cpp index 0f3ddf920d..50bd9f45cb 100644 --- a/esphome/components/modbus_controller/modbus_controller.cpp +++ b/esphome/components/modbus_controller/modbus_controller.cpp @@ -112,6 +112,12 @@ void ModbusController::on_modbus_read_registers(uint8_t function_code, uint16_t "0x%X.", this->address_, function_code, start_address, number_of_registers); + if (number_of_registers == 0 || number_of_registers > modbus::MAX_NUM_OF_REGISTERS_TO_READ) { + ESP_LOGW(TAG, "Invalid number of registers %d. Sending exception response.", number_of_registers); + this->send_error(function_code, ModbusExceptionCode::ILLEGAL_DATA_ADDRESS); + return; + } + std::vector sixteen_bit_response; for (uint16_t current_address = start_address; current_address < start_address + number_of_registers;) { bool found = false; @@ -136,9 +142,21 @@ void ModbusController::on_modbus_read_registers(uint8_t function_code, uint16_t } if (!found) { - ESP_LOGW(TAG, "Could not match any register to address %02X. Sending exception response.", current_address); - send_error(function_code, 0x02); - return; + if (this->server_courtesy_response_.enabled && + (current_address <= this->server_courtesy_response_.register_last_address)) { + ESP_LOGD(TAG, + "Could not match any register to address 0x%02X, but default allowed. " + "Returning default value: %d.", + current_address, this->server_courtesy_response_.register_value); + sixteen_bit_response.push_back(this->server_courtesy_response_.register_value); + current_address += 1; // Just increment by 1, as the default response is a single register + } else { + ESP_LOGW(TAG, + "Could not match any register to address 0x%02X and default not allowed. Sending exception response.", + current_address); + this->send_error(function_code, ModbusExceptionCode::ILLEGAL_DATA_ADDRESS); + return; + } } } @@ -156,27 +174,27 @@ void ModbusController::on_modbus_write_registers(uint8_t function_code, const st uint16_t number_of_registers; uint16_t payload_offset; - if (function_code == 0x10) { + if (function_code == ModbusFunctionCode::WRITE_MULTIPLE_REGISTERS) { number_of_registers = uint16_t(data[3]) | (uint16_t(data[2]) << 8); - if (number_of_registers == 0 || number_of_registers > 0x7B) { + if (number_of_registers == 0 || number_of_registers > modbus::MAX_NUM_OF_REGISTERS_TO_WRITE) { ESP_LOGW(TAG, "Invalid number of registers %d. Sending exception response.", number_of_registers); - send_error(function_code, 3); + this->send_error(function_code, ModbusExceptionCode::ILLEGAL_DATA_VALUE); return; } uint16_t payload_size = data[4]; if (payload_size != number_of_registers * 2) { ESP_LOGW(TAG, "Payload size of %d bytes is not 2 times the number of registers (%d). Sending exception response.", payload_size, number_of_registers); - send_error(function_code, 3); + this->send_error(function_code, ModbusExceptionCode::ILLEGAL_DATA_VALUE); return; } payload_offset = 5; - } else if (function_code == 0x06) { + } else if (function_code == ModbusFunctionCode::WRITE_SINGLE_REGISTER) { number_of_registers = 1; payload_offset = 2; } else { ESP_LOGW(TAG, "Invalid function code 0x%X. Sending exception response.", function_code); - send_error(function_code, 1); + this->send_error(function_code, ModbusExceptionCode::ILLEGAL_FUNCTION); return; } @@ -211,7 +229,7 @@ void ModbusController::on_modbus_write_registers(uint8_t function_code, const st if (!for_each_register([](ServerRegister *server_register, uint16_t offset) -> bool { return server_register->write_lambda != nullptr; })) { - send_error(function_code, 1); + this->send_error(function_code, ModbusExceptionCode::ILLEGAL_FUNCTION); return; } @@ -220,7 +238,7 @@ void ModbusController::on_modbus_write_registers(uint8_t function_code, const st int64_t number = payload_to_number(data, server_register->value_type, offset, 0xFFFFFFFF); return server_register->write_lambda(number); })) { - send_error(function_code, 4); + this->send_error(function_code, ModbusExceptionCode::SERVICE_DEVICE_FAILURE); return; } @@ -431,8 +449,15 @@ void ModbusController::dump_config() { "ModbusController:\n" " Address: 0x%02X\n" " Max Command Retries: %d\n" - " Offline Skip Updates: %d", - this->address_, this->max_cmd_retries_, this->offline_skip_updates_); + " Offline Skip Updates: %d\n" + " Server Courtesy Response:\n" + " Enabled: %s\n" + " Register Last Address: 0x%02X\n" + " Register Value: %d", + this->address_, this->max_cmd_retries_, this->offline_skip_updates_, + this->server_courtesy_response_.enabled ? "true" : "false", + this->server_courtesy_response_.register_last_address, this->server_courtesy_response_.register_value); + #if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE ESP_LOGCONFIG(TAG, "sensormap"); for (auto &it : this->sensorset_) { diff --git a/esphome/components/modbus_controller/modbus_controller.h b/esphome/components/modbus_controller/modbus_controller.h index a86ad1ccb5..6ed05715cb 100644 --- a/esphome/components/modbus_controller/modbus_controller.h +++ b/esphome/components/modbus_controller/modbus_controller.h @@ -16,35 +16,9 @@ namespace modbus_controller { class ModbusController; -enum class ModbusFunctionCode { - CUSTOM = 0x00, - READ_COILS = 0x01, - READ_DISCRETE_INPUTS = 0x02, - READ_HOLDING_REGISTERS = 0x03, - READ_INPUT_REGISTERS = 0x04, - WRITE_SINGLE_COIL = 0x05, - WRITE_SINGLE_REGISTER = 0x06, - READ_EXCEPTION_STATUS = 0x07, // not implemented - DIAGNOSTICS = 0x08, // not implemented - GET_COMM_EVENT_COUNTER = 0x0B, // not implemented - GET_COMM_EVENT_LOG = 0x0C, // not implemented - WRITE_MULTIPLE_COILS = 0x0F, - WRITE_MULTIPLE_REGISTERS = 0x10, - REPORT_SERVER_ID = 0x11, // not implemented - READ_FILE_RECORD = 0x14, // not implemented - WRITE_FILE_RECORD = 0x15, // not implemented - MASK_WRITE_REGISTER = 0x16, // not implemented - READ_WRITE_MULTIPLE_REGISTERS = 0x17, // not implemented - READ_FIFO_QUEUE = 0x18, // not implemented -}; - -enum class ModbusRegisterType : uint8_t { - CUSTOM = 0x0, - COIL = 0x01, - DISCRETE_INPUT = 0x02, - HOLDING = 0x03, - READ = 0x04, -}; +using modbus::ModbusFunctionCode; +using modbus::ModbusRegisterType; +using modbus::ModbusExceptionCode; enum class SensorValueType : uint8_t { RAW = 0x00, // variable length @@ -256,6 +230,12 @@ class SensorItem { bool force_new_range{false}; }; +struct ServerCourtesyResponse { + bool enabled{false}; + uint16_t register_last_address{0xFFFF}; + uint16_t register_value{0}; +}; + class ServerRegister { using ReadLambda = std::function; using WriteLambda = std::function; @@ -530,6 +510,12 @@ class ModbusController : public PollingComponent, public modbus::ModbusDevice { void set_max_cmd_retries(uint8_t max_cmd_retries) { this->max_cmd_retries_ = max_cmd_retries; } /// get how many times a command will be (re)sent if no response is received uint8_t get_max_cmd_retries() { return this->max_cmd_retries_; } + /// Called by esphome generated code to set the server courtesy response object + void set_server_courtesy_response(const ServerCourtesyResponse &server_courtesy_response) { + this->server_courtesy_response_ = server_courtesy_response; + } + /// Get the server courtesy response object + ServerCourtesyResponse get_server_courtesy_response() const { return this->server_courtesy_response_; } protected: /// parse sensormap_ and create range of sequential addresses @@ -572,6 +558,9 @@ class ModbusController : public PollingComponent, public modbus::ModbusDevice { CallbackManager online_callback_{}; /// Server offline callback CallbackManager offline_callback_{}; + /// Server courtesy response + ServerCourtesyResponse server_courtesy_response_{ + .enabled = false, .register_last_address = 0xFFFF, .register_value = 0}; }; /** Convert vector response payload to float. diff --git a/esphome/components/mpl3115a2/mpl3115a2.cpp b/esphome/components/mpl3115a2/mpl3115a2.cpp index 9b65fb04e4..a689149c89 100644 --- a/esphome/components/mpl3115a2/mpl3115a2.cpp +++ b/esphome/components/mpl3115a2/mpl3115a2.cpp @@ -9,10 +9,8 @@ namespace mpl3115a2 { static const char *const TAG = "mpl3115a2"; void MPL3115A2Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - uint8_t whoami = 0xFF; - if (!this->read_byte(MPL3115A2_WHOAMI, &whoami, false)) { + if (!this->read_byte(MPL3115A2_WHOAMI, &whoami)) { this->error_code_ = COMMUNICATION_FAILED; this->mark_failed(); return; @@ -56,24 +54,24 @@ void MPL3115A2Component::dump_config() { void MPL3115A2Component::update() { uint8_t mode = MPL3115A2_CTRL_REG1_OS128; - this->write_byte(MPL3115A2_CTRL_REG1, mode, true); + this->write_byte(MPL3115A2_CTRL_REG1, mode); // Trigger a new reading mode |= MPL3115A2_CTRL_REG1_OST; if (this->altitude_ != nullptr) mode |= MPL3115A2_CTRL_REG1_ALT; - this->write_byte(MPL3115A2_CTRL_REG1, mode, true); + this->write_byte(MPL3115A2_CTRL_REG1, mode); // Wait until status shows reading available uint8_t status = 0; - if (!this->read_byte(MPL3115A2_REGISTER_STATUS, &status, false) || (status & MPL3115A2_REGISTER_STATUS_PDR) == 0) { + if (!this->read_byte(MPL3115A2_REGISTER_STATUS, &status) || (status & MPL3115A2_REGISTER_STATUS_PDR) == 0) { delay(10); - if (!this->read_byte(MPL3115A2_REGISTER_STATUS, &status, false) || (status & MPL3115A2_REGISTER_STATUS_PDR) == 0) { + if (!this->read_byte(MPL3115A2_REGISTER_STATUS, &status) || (status & MPL3115A2_REGISTER_STATUS_PDR) == 0) { return; } } uint8_t buffer[5] = {0, 0, 0, 0, 0}; - this->read_register(MPL3115A2_REGISTER_PRESSURE_MSB, buffer, 5, false); + this->read_register(MPL3115A2_REGISTER_PRESSURE_MSB, buffer, 5); float altitude = 0, pressure = 0; if (this->altitude_ != nullptr) { diff --git a/esphome/components/mpr121/__init__.py b/esphome/components/mpr121/__init__.py index b736a7e4f0..0bf9377275 100644 --- a/esphome/components/mpr121/__init__.py +++ b/esphome/components/mpr121/__init__.py @@ -56,12 +56,13 @@ def _final_validate(config): for binary_sensor in binary_sensors: if binary_sensor.get(CONF_MPR121_ID) == config[CONF_ID]: max_touch_channel = max(max_touch_channel, binary_sensor[CONF_CHANNEL]) - if max_touch_channel_in_config := config.get(CONF_MAX_TOUCH_CHANNEL): - if max_touch_channel != max_touch_channel_in_config: - raise cv.Invalid( - "Max touch channel must equal the highest binary sensor channel or be removed for auto calculation", - path=[CONF_MAX_TOUCH_CHANNEL], - ) + if ( + max_touch_channel_in_config := config.get(CONF_MAX_TOUCH_CHANNEL) + ) and max_touch_channel != max_touch_channel_in_config: + raise cv.Invalid( + "Max touch channel must equal the highest binary sensor channel or be removed for auto calculation", + path=[CONF_MAX_TOUCH_CHANNEL], + ) path = fconf.get_path_for_id(config[CONF_ID])[:-1] this_config = fconf.get_config_for_path(path) this_config[CONF_MAX_TOUCH_CHANNEL] = max_touch_channel diff --git a/esphome/components/mpr121/mpr121.cpp b/esphome/components/mpr121/mpr121.cpp index 39c45d7a89..5a8a8e7205 100644 --- a/esphome/components/mpr121/mpr121.cpp +++ b/esphome/components/mpr121/mpr121.cpp @@ -11,48 +11,49 @@ namespace mpr121 { static const char *const TAG = "mpr121"; void MPR121Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); + this->disable_loop(); // soft reset device this->write_byte(MPR121_SOFTRESET, 0x63); - delay(100); // NOLINT - if (!this->write_byte(MPR121_ECR, 0x0)) { - this->error_code_ = COMMUNICATION_FAILED; - this->mark_failed(); - return; - } + this->set_timeout(100, [this]() { + if (!this->write_byte(MPR121_ECR, 0x0)) { + this->error_code_ = COMMUNICATION_FAILED; + this->mark_failed(); + return; + } + // set touch sensitivity for all 12 channels + for (auto *channel : this->channels_) { + channel->setup(); + } + this->write_byte(MPR121_MHDR, 0x01); + this->write_byte(MPR121_NHDR, 0x01); + this->write_byte(MPR121_NCLR, 0x0E); + this->write_byte(MPR121_FDLR, 0x00); - // set touch sensitivity for all 12 channels - for (auto *channel : this->channels_) { - channel->setup(); - } - this->write_byte(MPR121_MHDR, 0x01); - this->write_byte(MPR121_NHDR, 0x01); - this->write_byte(MPR121_NCLR, 0x0E); - this->write_byte(MPR121_FDLR, 0x00); + this->write_byte(MPR121_MHDF, 0x01); + this->write_byte(MPR121_NHDF, 0x05); + this->write_byte(MPR121_NCLF, 0x01); + this->write_byte(MPR121_FDLF, 0x00); - this->write_byte(MPR121_MHDF, 0x01); - this->write_byte(MPR121_NHDF, 0x05); - this->write_byte(MPR121_NCLF, 0x01); - this->write_byte(MPR121_FDLF, 0x00); + this->write_byte(MPR121_NHDT, 0x00); + this->write_byte(MPR121_NCLT, 0x00); + this->write_byte(MPR121_FDLT, 0x00); - this->write_byte(MPR121_NHDT, 0x00); - this->write_byte(MPR121_NCLT, 0x00); - this->write_byte(MPR121_FDLT, 0x00); + this->write_byte(MPR121_DEBOUNCE, 0); + // default, 16uA charge current + this->write_byte(MPR121_CONFIG1, 0x10); + // 0.5uS encoding, 1ms period + this->write_byte(MPR121_CONFIG2, 0x20); - this->write_byte(MPR121_DEBOUNCE, 0); - // default, 16uA charge current - this->write_byte(MPR121_CONFIG1, 0x10); - // 0.5uS encoding, 1ms period - this->write_byte(MPR121_CONFIG2, 0x20); + // Write the Electrode Configuration Register + // * Highest 2 bits is "Calibration Lock", which we set to a value corresponding to 5 bits. + // * The 2 bits below is "Proximity Enable" and are left at 0. + // * The 4 least significant bits control how many electrodes are enabled. Electrodes are enabled + // as a range, starting at 0 up to the highest channel index used. + this->write_byte(MPR121_ECR, 0x80 | (this->max_touch_channel_ + 1)); - // Write the Electrode Configuration Register - // * Highest 2 bits is "Calibration Lock", which we set to a value corresponding to 5 bits. - // * The 2 bits below is "Proximity Enable" and are left at 0. - // * The 4 least significant bits control how many electrodes are enabled. Electrodes are enabled - // as a range, starting at 0 up to the highest channel index used. - this->write_byte(MPR121_ECR, 0x80 | (this->max_touch_channel_ + 1)); - - this->flush_gpio_(); + this->flush_gpio_(); + this->enable_loop(); + }); } void MPR121Component::set_touch_debounce(uint8_t debounce) { @@ -74,9 +75,6 @@ void MPR121Component::dump_config() { case COMMUNICATION_FAILED: ESP_LOGE(TAG, ESP_LOG_MSG_COMM_FAIL); break; - case WRONG_CHIP_STATE: - ESP_LOGE(TAG, "MPR121 has wrong default value for CONFIG2?"); - break; case NONE: default: break; diff --git a/esphome/components/mpr121/mpr121.h b/esphome/components/mpr121/mpr121.h index eb2e2edc57..6dd2c38309 100644 --- a/esphome/components/mpr121/mpr121.h +++ b/esphome/components/mpr121/mpr121.h @@ -88,7 +88,6 @@ class MPR121Component : public Component, public i2c::I2CDevice { enum ErrorCode { NONE = 0, COMMUNICATION_FAILED, - WRONG_CHIP_STATE, } error_code_{NONE}; bool flush_gpio_(); diff --git a/esphome/components/mpu6050/mpu6050.cpp b/esphome/components/mpu6050/mpu6050.cpp index 84f0fb4bae..ecbee11c48 100644 --- a/esphome/components/mpu6050/mpu6050.cpp +++ b/esphome/components/mpu6050/mpu6050.cpp @@ -21,7 +21,6 @@ const uint8_t MPU6050_BIT_TEMPERATURE_DISABLED = 3; const float GRAVITY_EARTH = 9.80665f; void MPU6050Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t who_am_i; if (!this->read_byte(MPU6050_REGISTER_WHO_AM_I, &who_am_i) || (who_am_i != 0x68 && who_am_i != 0x70 && who_am_i != 0x98)) { diff --git a/esphome/components/mpu6886/mpu6886.cpp b/esphome/components/mpu6886/mpu6886.cpp index cbd8b601bd..6fdf7b8684 100644 --- a/esphome/components/mpu6886/mpu6886.cpp +++ b/esphome/components/mpu6886/mpu6886.cpp @@ -26,7 +26,6 @@ const float TEMPERATURE_SENSITIVITY = 326.8; const float TEMPERATURE_OFFSET = 25.0; void MPU6886Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t who_am_i; if (!this->read_byte(MPU6886_REGISTER_WHO_AM_I, &who_am_i) || who_am_i != MPU6886_WHO_AM_I_IDENTIFIER) { this->mark_failed(); diff --git a/esphome/components/mqtt/__init__.py b/esphome/components/mqtt/__init__.py index 1a6fcabf42..814fb566d4 100644 --- a/esphome/components/mqtt/__init__.py +++ b/esphome/components/mqtt/__init__.py @@ -57,7 +57,7 @@ from esphome.const import ( PLATFORM_ESP8266, PlatformFramework, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority DEPENDENCIES = ["network"] @@ -312,17 +312,16 @@ CONFIG_SCHEMA = cv.All( def exp_mqtt_message(config): if config is None: return cg.optional(cg.TemplateArguments(MQTTMessage)) - exp = cg.StructInitializer( + return cg.StructInitializer( MQTTMessage, ("topic", config[CONF_TOPIC]), ("payload", config.get(CONF_PAYLOAD, "")), ("qos", config[CONF_QOS]), ("retain", config[CONF_RETAIN]), ) - return exp -@coroutine_with_priority(40.0) +@coroutine_with_priority(CoroPriority.WEB) async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) diff --git a/esphome/components/mqtt/mqtt_button.cpp b/esphome/components/mqtt/mqtt_button.cpp index c619a02344..b3435edf38 100644 --- a/esphome/components/mqtt/mqtt_button.cpp +++ b/esphome/components/mqtt/mqtt_button.cpp @@ -27,7 +27,7 @@ void MQTTButtonComponent::setup() { } void MQTTButtonComponent::dump_config() { ESP_LOGCONFIG(TAG, "MQTT Button '%s': ", this->button_->get_name().c_str()); - LOG_MQTT_COMPONENT(true, true); + LOG_MQTT_COMPONENT(false, true); } void MQTTButtonComponent::send_discovery(JsonObject root, mqtt::SendDiscoveryConfig &config) { diff --git a/esphome/components/mqtt/mqtt_client.cpp b/esphome/components/mqtt/mqtt_client.cpp index 5b93789447..16f54ab8a0 100644 --- a/esphome/components/mqtt/mqtt_client.cpp +++ b/esphome/components/mqtt/mqtt_client.cpp @@ -29,12 +29,12 @@ static const char *const TAG = "mqtt"; MQTTClientComponent::MQTTClientComponent() { global_mqtt_client = this; - this->credentials_.client_id = App.get_name() + "-" + get_mac_address(); + const std::string mac_addr = get_mac_address(); + this->credentials_.client_id = make_name_with_suffix(App.get_name(), '-', mac_addr.c_str(), mac_addr.size()); } // Connection void MQTTClientComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->mqtt_backend_.set_on_message( [this](const char *topic, const char *payload, size_t len, size_t index, size_t total) { if (index == 0) @@ -193,13 +193,17 @@ void MQTTClientComponent::start_dnslookup_() { this->dns_resolve_error_ = false; this->dns_resolved_ = false; ip_addr_t addr; + err_t err; + { + LwIPLock lock; #if USE_NETWORK_IPV6 - err_t err = dns_gethostbyname_addrtype(this->credentials_.address.c_str(), &addr, - MQTTClientComponent::dns_found_callback, this, LWIP_DNS_ADDRTYPE_IPV6_IPV4); + err = dns_gethostbyname_addrtype(this->credentials_.address.c_str(), &addr, MQTTClientComponent::dns_found_callback, + this, LWIP_DNS_ADDRTYPE_IPV6_IPV4); #else - err_t err = dns_gethostbyname_addrtype(this->credentials_.address.c_str(), &addr, - MQTTClientComponent::dns_found_callback, this, LWIP_DNS_ADDRTYPE_IPV4); + err = dns_gethostbyname_addrtype(this->credentials_.address.c_str(), &addr, MQTTClientComponent::dns_found_callback, + this, LWIP_DNS_ADDRTYPE_IPV4); #endif /* USE_NETWORK_IPV6 */ + } switch (err) { case ERR_OK: { // Got IP immediately @@ -488,7 +492,7 @@ bool MQTTClientComponent::publish(const std::string &topic, const std::string &p bool MQTTClientComponent::publish(const std::string &topic, const char *payload, size_t payload_length, uint8_t qos, bool retain) { - return publish({.topic = topic, .payload = payload, .qos = qos, .retain = retain}); + return publish({.topic = topic, .payload = std::string(payload, payload_length), .qos = qos, .retain = retain}); } bool MQTTClientComponent::publish(const MQTTMessage &message) { diff --git a/esphome/components/mqtt/mqtt_climate.cpp b/esphome/components/mqtt/mqtt_climate.cpp index e16f097812..a6f4e0a201 100644 --- a/esphome/components/mqtt/mqtt_climate.cpp +++ b/esphome/components/mqtt/mqtt_climate.cpp @@ -17,11 +17,11 @@ void MQTTClimateComponent::send_discovery(JsonObject root, mqtt::SendDiscoveryCo // NOLINTBEGIN(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson auto traits = this->device_->get_traits(); // current_temperature_topic - if (traits.get_supports_current_temperature()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_TEMPERATURE)) { root[MQTT_CURRENT_TEMPERATURE_TOPIC] = this->get_current_temperature_state_topic(); } // current_humidity_topic - if (traits.get_supports_current_humidity()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_HUMIDITY)) { root[MQTT_CURRENT_HUMIDITY_TOPIC] = this->get_current_humidity_state_topic(); } // mode_command_topic @@ -45,7 +45,8 @@ void MQTTClimateComponent::send_discovery(JsonObject root, mqtt::SendDiscoveryCo if (traits.supports_mode(CLIMATE_MODE_HEAT_COOL)) modes.add("heat_cool"); - if (traits.get_supports_two_point_target_temperature()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | + climate::CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE)) { // temperature_low_command_topic root[MQTT_TEMPERATURE_LOW_COMMAND_TOPIC] = this->get_target_temperature_low_command_topic(); // temperature_low_state_topic @@ -61,7 +62,7 @@ void MQTTClimateComponent::send_discovery(JsonObject root, mqtt::SendDiscoveryCo root[MQTT_TEMPERATURE_STATE_TOPIC] = this->get_target_temperature_state_topic(); } - if (traits.get_supports_target_humidity()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TARGET_HUMIDITY)) { // target_humidity_command_topic root[MQTT_TARGET_HUMIDITY_COMMAND_TOPIC] = this->get_target_humidity_command_topic(); // target_humidity_state_topic @@ -109,7 +110,7 @@ void MQTTClimateComponent::send_discovery(JsonObject root, mqtt::SendDiscoveryCo presets.add(preset); } - if (traits.get_supports_action()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_ACTION)) { // action_topic root[MQTT_ACTION_TOPIC] = this->get_action_state_topic(); } @@ -174,7 +175,8 @@ void MQTTClimateComponent::setup() { call.perform(); }); - if (traits.get_supports_two_point_target_temperature()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | + climate::CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE)) { this->subscribe(this->get_target_temperature_low_command_topic(), [this](const std::string &topic, const std::string &payload) { auto val = parse_number(payload); @@ -211,7 +213,7 @@ void MQTTClimateComponent::setup() { }); } - if (traits.get_supports_target_humidity()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TARGET_HUMIDITY)) { this->subscribe(this->get_target_humidity_command_topic(), [this](const std::string &topic, const std::string &payload) { auto val = parse_number(payload); @@ -290,12 +292,14 @@ bool MQTTClimateComponent::publish_state_() { success = false; int8_t target_accuracy = traits.get_target_temperature_accuracy_decimals(); int8_t current_accuracy = traits.get_current_temperature_accuracy_decimals(); - if (traits.get_supports_current_temperature() && !std::isnan(this->device_->current_temperature)) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_TEMPERATURE) && + !std::isnan(this->device_->current_temperature)) { std::string payload = value_accuracy_to_string(this->device_->current_temperature, current_accuracy); if (!this->publish(this->get_current_temperature_state_topic(), payload)) success = false; } - if (traits.get_supports_two_point_target_temperature()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | + climate::CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE)) { std::string payload = value_accuracy_to_string(this->device_->target_temperature_low, target_accuracy); if (!this->publish(this->get_target_temperature_low_state_topic(), payload)) success = false; @@ -308,12 +312,14 @@ bool MQTTClimateComponent::publish_state_() { success = false; } - if (traits.get_supports_current_humidity() && !std::isnan(this->device_->current_humidity)) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_HUMIDITY) && + !std::isnan(this->device_->current_humidity)) { std::string payload = value_accuracy_to_string(this->device_->current_humidity, 0); if (!this->publish(this->get_current_humidity_state_topic(), payload)) success = false; } - if (traits.get_supports_target_humidity() && !std::isnan(this->device_->target_humidity)) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TARGET_HUMIDITY) && + !std::isnan(this->device_->target_humidity)) { std::string payload = value_accuracy_to_string(this->device_->target_humidity, 0); if (!this->publish(this->get_target_humidity_state_topic(), payload)) success = false; @@ -357,7 +363,7 @@ bool MQTTClimateComponent::publish_state_() { success = false; } - if (traits.get_supports_action()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_ACTION)) { const char *payload; switch (this->device_->action) { case CLIMATE_ACTION_OFF: diff --git a/esphome/components/mqtt/mqtt_component.cpp b/esphome/components/mqtt/mqtt_component.cpp index b51f4d903e..6ceaf219ff 100644 --- a/esphome/components/mqtt/mqtt_component.cpp +++ b/esphome/components/mqtt/mqtt_component.cpp @@ -129,21 +129,16 @@ bool MQTTComponent::send_discovery_() { root[MQTT_PAYLOAD_NOT_AVAILABLE] = this->availability_->payload_not_available; } - std::string unique_id = this->unique_id(); const MQTTDiscoveryInfo &discovery_info = global_mqtt_client->get_discovery_info(); - if (!unique_id.empty()) { - root[MQTT_UNIQUE_ID] = unique_id; + if (discovery_info.unique_id_generator == MQTT_MAC_ADDRESS_UNIQUE_ID_GENERATOR) { + char friendly_name_hash[9]; + sprintf(friendly_name_hash, "%08" PRIx32, fnv1_hash(this->friendly_name())); + friendly_name_hash[8] = 0; // ensure the hash-string ends with null + root[MQTT_UNIQUE_ID] = get_mac_address() + "-" + this->component_type() + "-" + friendly_name_hash; } else { - if (discovery_info.unique_id_generator == MQTT_MAC_ADDRESS_UNIQUE_ID_GENERATOR) { - char friendly_name_hash[9]; - sprintf(friendly_name_hash, "%08" PRIx32, fnv1_hash(this->friendly_name())); - friendly_name_hash[8] = 0; // ensure the hash-string ends with null - root[MQTT_UNIQUE_ID] = get_mac_address() + "-" + this->component_type() + "-" + friendly_name_hash; - } else { - // default to almost-unique ID. It's a hack but the only way to get that - // gorgeous device registry view. - root[MQTT_UNIQUE_ID] = "ESP" + this->component_type() + this->get_default_object_id_(); - } + // default to almost-unique ID. It's a hack but the only way to get that + // gorgeous device registry view. + root[MQTT_UNIQUE_ID] = "ESP" + this->component_type() + this->get_default_object_id_(); } const std::string &node_name = App.get_name(); @@ -286,7 +281,6 @@ void MQTTComponent::call_dump_config() { this->dump_config(); } void MQTTComponent::schedule_resend_state() { this->resend_state_ = true; } -std::string MQTTComponent::unique_id() { return ""; } bool MQTTComponent::is_connected_() const { return global_mqtt_client->is_connected(); } // Pull these properties from EntityBase if not overridden diff --git a/esphome/components/mqtt/mqtt_component.h b/esphome/components/mqtt/mqtt_component.h index 01ba98ad40..851fdd842c 100644 --- a/esphome/components/mqtt/mqtt_component.h +++ b/esphome/components/mqtt/mqtt_component.h @@ -164,13 +164,6 @@ class MQTTComponent : public Component { */ virtual const EntityBase *get_entity() const = 0; - /** A unique ID for this MQTT component, empty for no unique id. See unique ID requirements: - * https://developers.home-assistant.io/docs/en/entity_registry_index.html#unique-id-requirements - * - * @return The unique id as a string. - */ - virtual std::string unique_id(); - /// Get the friendly name of this MQTT component. virtual std::string friendly_name() const; diff --git a/esphome/components/mqtt/mqtt_sensor.cpp b/esphome/components/mqtt/mqtt_sensor.cpp index 9324ea9bb1..9e61f6ef3b 100644 --- a/esphome/components/mqtt/mqtt_sensor.cpp +++ b/esphome/components/mqtt/mqtt_sensor.cpp @@ -58,8 +58,13 @@ void MQTTSensorComponent::send_discovery(JsonObject root, mqtt::SendDiscoveryCon if (this->sensor_->get_force_update()) root[MQTT_FORCE_UPDATE] = true; - if (this->sensor_->get_state_class() != STATE_CLASS_NONE) - root[MQTT_STATE_CLASS] = state_class_to_string(this->sensor_->get_state_class()); + if (this->sensor_->get_state_class() != STATE_CLASS_NONE) { +#ifdef USE_STORE_LOG_STR_IN_FLASH + root[MQTT_STATE_CLASS] = (const __FlashStringHelper *) state_class_to_string(this->sensor_->get_state_class()); +#else + root[MQTT_STATE_CLASS] = LOG_STR_ARG(state_class_to_string(this->sensor_->get_state_class())); +#endif + } config.command_topic = false; } @@ -76,7 +81,6 @@ bool MQTTSensorComponent::publish_state(float value) { int8_t accuracy = this->sensor_->get_accuracy_decimals(); return this->publish(this->get_state_topic_(), value_accuracy_to_string(value, accuracy)); } -std::string MQTTSensorComponent::unique_id() { return this->sensor_->unique_id(); } } // namespace mqtt } // namespace esphome diff --git a/esphome/components/mqtt/mqtt_sensor.h b/esphome/components/mqtt/mqtt_sensor.h index adc201736a..15ea703ad4 100644 --- a/esphome/components/mqtt/mqtt_sensor.h +++ b/esphome/components/mqtt/mqtt_sensor.h @@ -46,7 +46,6 @@ class MQTTSensorComponent : public mqtt::MQTTComponent { /// Override for MQTTComponent, returns "sensor". std::string component_type() const override; const EntityBase *get_entity() const override; - std::string unique_id() override; sensor::Sensor *sensor_; optional expire_after_; // Override the expire after advertised to Home Assistant diff --git a/esphome/components/mqtt/mqtt_text_sensor.cpp b/esphome/components/mqtt/mqtt_text_sensor.cpp index 0cc5de07a3..42260ed2a8 100644 --- a/esphome/components/mqtt/mqtt_text_sensor.cpp +++ b/esphome/components/mqtt/mqtt_text_sensor.cpp @@ -40,7 +40,6 @@ bool MQTTTextSensor::send_initial_state() { } std::string MQTTTextSensor::component_type() const { return "sensor"; } const EntityBase *MQTTTextSensor::get_entity() const { return this->sensor_; } -std::string MQTTTextSensor::unique_id() { return this->sensor_->unique_id(); } } // namespace mqtt } // namespace esphome diff --git a/esphome/components/mqtt/mqtt_text_sensor.h b/esphome/components/mqtt/mqtt_text_sensor.h index fe53a6fefd..9a14efdd16 100644 --- a/esphome/components/mqtt/mqtt_text_sensor.h +++ b/esphome/components/mqtt/mqtt_text_sensor.h @@ -28,7 +28,6 @@ class MQTTTextSensor : public mqtt::MQTTComponent { protected: std::string component_type() const override; const EntityBase *get_entity() const override; - std::string unique_id() override; text_sensor::TextSensor *sensor_; }; diff --git a/esphome/components/ms5611/ms5611.cpp b/esphome/components/ms5611/ms5611.cpp index 7a820f3b5a..5a7622e783 100644 --- a/esphome/components/ms5611/ms5611.cpp +++ b/esphome/components/ms5611/ms5611.cpp @@ -15,18 +15,18 @@ static const uint8_t MS5611_CMD_CONV_D2 = 0x50; static const uint8_t MS5611_CMD_READ_PROM = 0xA2; void MS5611Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (!this->write_bytes(MS5611_CMD_RESET, nullptr, 0)) { this->mark_failed(); return; } - delay(100); // NOLINT - for (uint8_t offset = 0; offset < 6; offset++) { - if (!this->read_byte_16(MS5611_CMD_READ_PROM + (offset * 2), &this->prom_[offset])) { - this->mark_failed(); - return; + this->set_timeout(100, [this]() { + for (uint8_t offset = 0; offset < 6; offset++) { + if (!this->read_byte_16(MS5611_CMD_READ_PROM + (offset * 2), &this->prom_[offset])) { + this->mark_failed(); + return; + } } - } + }); } void MS5611Component::dump_config() { ESP_LOGCONFIG(TAG, "MS5611:"); diff --git a/esphome/components/ms8607/ms8607.cpp b/esphome/components/ms8607/ms8607.cpp index f8ea26bfd9..215131eb8e 100644 --- a/esphome/components/ms8607/ms8607.cpp +++ b/esphome/components/ms8607/ms8607.cpp @@ -67,7 +67,6 @@ static uint8_t crc4(uint16_t *buffer, size_t length); static uint8_t hsensor_crc_check(uint16_t value); void MS8607Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->error_code_ = ErrorCode::NONE; this->setup_status_ = SetupStatus::NEEDS_RESET; diff --git a/esphome/components/msa3xx/msa3xx.cpp b/esphome/components/msa3xx/msa3xx.cpp index 17f0a9c418..56dc919968 100644 --- a/esphome/components/msa3xx/msa3xx.cpp +++ b/esphome/components/msa3xx/msa3xx.cpp @@ -118,8 +118,6 @@ const char *orientation_xy_to_string(OrientationXY orientation) { const char *orientation_z_to_string(bool orientation) { return orientation ? "Downwards looking" : "Upwards looking"; } void MSA3xxComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - uint8_t part_id{0xff}; if (!this->read_byte(static_cast(RegisterMap::PART_ID), &part_id) || (part_id != MSA_3XX_PART_ID)) { ESP_LOGE(TAG, "Part ID is wrong or missing. Got 0x%02X", part_id); diff --git a/esphome/components/my9231/my9231.cpp b/esphome/components/my9231/my9231.cpp index 691c945254..fba7ac2bf3 100644 --- a/esphome/components/my9231/my9231.cpp +++ b/esphome/components/my9231/my9231.cpp @@ -28,7 +28,6 @@ static const uint8_t MY9231_CMD_SCATTER_APDM = 0x0 << 0; static const uint8_t MY9231_CMD_SCATTER_PWM = 0x1 << 0; void MY9231OutputComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->pin_di_->setup(); this->pin_di_->digital_write(false); this->pin_dcki_->setup(); @@ -57,7 +56,6 @@ void MY9231OutputComponent::setup() { this->send_dcki_pulses_(32 * this->num_chips_); this->init_chips_(command); } - ESP_LOGV(TAG, " Chips initialized."); } void MY9231OutputComponent::dump_config() { ESP_LOGCONFIG(TAG, "MY9231:"); diff --git a/esphome/components/nau7802/nau7802.cpp b/esphome/components/nau7802/nau7802.cpp index edcd114852..6a31b754f7 100644 --- a/esphome/components/nau7802/nau7802.cpp +++ b/esphome/components/nau7802/nau7802.cpp @@ -52,7 +52,6 @@ static const uint8_t POWER_PGA_CAP_EN = 0x80; static const uint8_t DEVICE_REV = 0x1F; void NAU7802Sensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); i2c::I2CRegister pu_ctrl = this->reg(PU_CTRL_REG); uint8_t rev; @@ -219,7 +218,7 @@ void NAU7802Sensor::dump_config() { void NAU7802Sensor::write_value_(uint8_t start_reg, size_t size, int32_t value) { uint8_t data[4]; - for (int i = 0; i < size; i++) { + for (size_t i = 0; i < size; i++) { data[i] = 0xFF & (value >> (size - 1 - i) * 8); } this->write_register(start_reg, data, size); @@ -229,7 +228,7 @@ int32_t NAU7802Sensor::read_value_(uint8_t start_reg, size_t size) { uint8_t data[4]; this->read_register(start_reg, data, size); int32_t result = 0; - for (int i = 0; i < size; i++) { + for (size_t i = 0; i < size; i++) { result |= data[i] << (size - 1 - i) * 8; } // extend sign bit diff --git a/esphome/components/neopixelbus/light.py b/esphome/components/neopixelbus/light.py index 3cd1bfd357..0c9604e932 100644 --- a/esphome/components/neopixelbus/light.py +++ b/esphome/components/neopixelbus/light.py @@ -15,6 +15,7 @@ from esphome.const import ( CONF_PIN, CONF_TYPE, CONF_VARIANT, + Framework, ) from esphome.core import CORE @@ -162,7 +163,15 @@ def _validate_method(value): CONFIG_SCHEMA = cv.All( - cv.only_with_arduino, + cv.only_with_framework( + frameworks=Framework.ARDUINO, + suggestions={ + Framework.ESP_IDF: ( + "esp32_rmt_led_strip", + "light/esp32_rmt_led_strip", + ) + }, + ), cv.require_framework_version( esp8266_arduino=cv.Version(2, 4, 0), esp32_arduino=cv.Version(0, 0, 0), @@ -216,6 +225,9 @@ async def to_code(config): # https://github.com/Makuna/NeoPixelBus/blob/master/library.json # Version Listed Here: https://registry.platformio.org/libraries/makuna/NeoPixelBus/versions if CORE.is_esp32: + # disable built in rgb support as it uses the new RMT drivers and will + # conflict with NeoPixelBus which uses the legacy drivers + cg.add_build_flag("-DESP32_ARDUINO_NO_RGB_BUILTIN") cg.add_library("makuna/NeoPixelBus", "2.8.0") else: cg.add_library("makuna/NeoPixelBus", "2.7.3") diff --git a/esphome/components/network/__init__.py b/esphome/components/network/__init__.py index b04fca7a1c..1a74350c4c 100644 --- a/esphome/components/network/__init__.py +++ b/esphome/components/network/__init__.py @@ -2,7 +2,7 @@ import esphome.codegen as cg from esphome.components.esp32 import add_idf_sdkconfig_option import esphome.config_validation as cv from esphome.const import CONF_ENABLE_IPV6, CONF_MIN_IPV6_ADDR_COUNT -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority CODEOWNERS = ["@esphome/core"] AUTO_LOAD = ["mdns"] @@ -36,7 +36,7 @@ CONFIG_SCHEMA = cv.Schema( ) -@coroutine_with_priority(201.0) +@coroutine_with_priority(CoroPriority.NETWORK) async def to_code(config): cg.add_define("USE_NETWORK") if CORE.using_arduino and CORE.is_esp32: @@ -47,9 +47,13 @@ async def to_code(config): cg.add_define( "USE_NETWORK_MIN_IPV6_ADDR_COUNT", config[CONF_MIN_IPV6_ADDR_COUNT] ) - if CORE.using_esp_idf: - add_idf_sdkconfig_option("CONFIG_LWIP_IPV6", enable_ipv6) - add_idf_sdkconfig_option("CONFIG_LWIP_IPV6_AUTOCONFIG", enable_ipv6) + if CORE.is_esp32: + if CORE.using_esp_idf: + add_idf_sdkconfig_option("CONFIG_LWIP_IPV6", enable_ipv6) + add_idf_sdkconfig_option("CONFIG_LWIP_IPV6_AUTOCONFIG", enable_ipv6) + else: + add_idf_sdkconfig_option("CONFIG_LWIP_IPV6", True) + add_idf_sdkconfig_option("CONFIG_LWIP_IPV6_AUTOCONFIG", True) elif enable_ipv6: cg.add_build_flag("-DCONFIG_LWIP_IPV6") cg.add_build_flag("-DCONFIG_LWIP_IPV6_AUTOCONFIG") diff --git a/esphome/components/network/util.cpp b/esphome/components/network/util.cpp index a8e792a2d7..27ad9448a4 100644 --- a/esphome/components/network/util.cpp +++ b/esphome/components/network/util.cpp @@ -13,15 +13,27 @@ #include "esphome/components/openthread/openthread.h" #endif +#ifdef USE_MODEM +#include "esphome/components/modem/modem_component.h" +#endif + namespace esphome { namespace network { +// The order of the components is important: WiFi should come after any possible main interfaces (it may be used as +// an AP that use a previous interface for NAT). + bool is_connected() { #ifdef USE_ETHERNET if (ethernet::global_eth_component != nullptr && ethernet::global_eth_component->is_connected()) return true; #endif +#ifdef USE_MODEM + if (modem::global_modem_component != nullptr) + return modem::global_modem_component->is_connected(); +#endif + #ifdef USE_WIFI if (wifi::global_wifi_component != nullptr) return wifi::global_wifi_component->is_connected(); @@ -39,6 +51,11 @@ bool is_connected() { } bool is_disabled() { +#ifdef USE_MODEM + if (modem::global_modem_component != nullptr) + return modem::global_modem_component->is_disabled(); +#endif + #ifdef USE_WIFI if (wifi::global_wifi_component != nullptr) return wifi::global_wifi_component->is_disabled(); @@ -51,6 +68,12 @@ network::IPAddresses get_ip_addresses() { if (ethernet::global_eth_component != nullptr) return ethernet::global_eth_component->get_ip_addresses(); #endif + +#ifdef USE_MODEM + if (modem::global_modem_component != nullptr) + return modem::global_modem_component->get_ip_addresses(); +#endif + #ifdef USE_WIFI if (wifi::global_wifi_component != nullptr) return wifi::global_wifi_component->get_ip_addresses(); @@ -62,16 +85,25 @@ network::IPAddresses get_ip_addresses() { return {}; } -std::string get_use_address() { +const std::string &get_use_address() { + // Global component pointers are guaranteed to be set by component constructors when USE_* is defined #ifdef USE_ETHERNET - if (ethernet::global_eth_component != nullptr) - return ethernet::global_eth_component->get_use_address(); + return ethernet::global_eth_component->get_use_address(); #endif + +#ifdef USE_MODEM + return modem::global_modem_component->get_use_address(); +#endif + #ifdef USE_WIFI - if (wifi::global_wifi_component != nullptr) - return wifi::global_wifi_component->get_use_address(); + return wifi::global_wifi_component->get_use_address(); +#endif + +#if !defined(USE_ETHERNET) && !defined(USE_MODEM) && !defined(USE_WIFI) + // Fallback when no network component is defined (e.g., host platform) + static const std::string empty; + return empty; #endif - return ""; } } // namespace network diff --git a/esphome/components/network/util.h b/esphome/components/network/util.h index b518696e68..b4a92f8bee 100644 --- a/esphome/components/network/util.h +++ b/esphome/components/network/util.h @@ -12,7 +12,7 @@ bool is_connected(); /// Return whether the network is disabled (only wifi for now) bool is_disabled(); /// Get the active network hostname -std::string get_use_address(); +const std::string &get_use_address(); IPAddresses get_ip_addresses(); } // namespace network diff --git a/esphome/components/nextion/display.py b/esphome/components/nextion/display.py index 4254ae45fe..ed6cd93027 100644 --- a/esphome/components/nextion/display.py +++ b/esphome/components/nextion/display.py @@ -153,10 +153,10 @@ async def to_code(config): if CONF_TFT_URL in config: cg.add_define("USE_NEXTION_TFT_UPLOAD") cg.add(var.set_tft_url(config[CONF_TFT_URL])) - if CORE.is_esp32 and CORE.using_arduino: - cg.add_library("NetworkClientSecure", None) - cg.add_library("HTTPClient", None) - elif CORE.is_esp32 and CORE.using_esp_idf: + if CORE.is_esp32: + if CORE.using_arduino: + cg.add_library("NetworkClientSecure", None) + cg.add_library("HTTPClient", None) esp32.add_idf_sdkconfig_option("CONFIG_ESP_TLS_INSECURE", True) esp32.add_idf_sdkconfig_option( "CONFIG_ESP_TLS_SKIP_SERVER_CERT_VERIFY", True diff --git a/esphome/components/nextion/nextion.cpp b/esphome/components/nextion/nextion.cpp index 66e2d26061..0ce9d02e97 100644 --- a/esphome/components/nextion/nextion.cpp +++ b/esphome/components/nextion/nextion.cpp @@ -77,7 +77,7 @@ bool Nextion::check_connect_() { this->recv_ret_string_(response, 0, false); if (!response.empty() && response[0] == 0x1A) { // Swallow invalid variable name responses that may be caused by the above commands - ESP_LOGD(TAG, "0x1A error ignored (setup)"); + ESP_LOGV(TAG, "0x1A error ignored (setup)"); return false; } if (response.empty() || response.find("comok") == std::string::npos) { @@ -334,7 +334,7 @@ void Nextion::loop() { this->started_ms_ = App.get_loop_component_start_time(); if (this->started_ms_ + this->startup_override_ms_ < App.get_loop_component_start_time()) { - ESP_LOGD(TAG, "Manual ready set"); + ESP_LOGV(TAG, "Manual ready set"); this->connection_state_.nextion_reports_is_setup_ = true; } } @@ -450,7 +450,6 @@ void Nextion::process_nextion_commands_() { this->remove_from_q_(); if (!this->is_setup_) { if (this->nextion_queue_.empty()) { - ESP_LOGD(TAG, "Setup complete"); this->is_setup_ = true; this->setup_callback_.call(); } @@ -545,7 +544,7 @@ void Nextion::process_nextion_commands_() { uint8_t page_id = to_process[0]; uint8_t component_id = to_process[1]; uint8_t touch_event = to_process[2]; // 0 -> release, 1 -> press - ESP_LOGD(TAG, "Touch %s: page %u comp %u", touch_event ? "PRESS" : "RELEASE", page_id, component_id); + ESP_LOGV(TAG, "Touch %s: page %u comp %u", touch_event ? "PRESS" : "RELEASE", page_id, component_id); for (auto *touch : this->touch_) { touch->process_touch(page_id, component_id, touch_event != 0); } @@ -560,7 +559,7 @@ void Nextion::process_nextion_commands_() { } uint8_t page_id = to_process[0]; - ESP_LOGD(TAG, "New page: %u", page_id); + ESP_LOGV(TAG, "New page: %u", page_id); this->page_callback_.call(page_id); break; } @@ -578,7 +577,7 @@ void Nextion::process_nextion_commands_() { const uint16_t x = (uint16_t(to_process[0]) << 8) | to_process[1]; const uint16_t y = (uint16_t(to_process[2]) << 8) | to_process[3]; const uint8_t touch_event = to_process[4]; // 0 -> release, 1 -> press - ESP_LOGD(TAG, "Touch %s at %u,%u", touch_event ? "PRESS" : "RELEASE", x, y); + ESP_LOGV(TAG, "Touch %s at %u,%u", touch_event ? "PRESS" : "RELEASE", x, y); break; } @@ -677,7 +676,7 @@ void Nextion::process_nextion_commands_() { } case 0x88: // system successful start up { - ESP_LOGD(TAG, "System start: %zu", to_process_length); + ESP_LOGV(TAG, "System start: %zu", to_process_length); this->connection_state_.nextion_reports_is_setup_ = true; break; } @@ -765,7 +764,8 @@ void Nextion::process_nextion_commands_() { variable_name = to_process.substr(0, index); ++index; - text_value = to_process.substr(index); + // Get variable value without terminating NUL byte. Length check above ensures substr len >= 0. + text_value = to_process.substr(index, to_process_length - index - 1); ESP_LOGN(TAG, "Text sensor: %s='%s'", variable_name.c_str(), text_value.c_str()); @@ -922,7 +922,7 @@ void Nextion::set_nextion_sensor_state(NextionQueueType queue_type, const std::s } void Nextion::set_nextion_text_state(const std::string &name, const std::string &state) { - ESP_LOGD(TAG, "State: %s='%s'", name.c_str(), state.c_str()); + ESP_LOGV(TAG, "State: %s='%s'", name.c_str(), state.c_str()); for (auto *sensor : this->textsensortype_) { if (name == sensor->get_variable_name()) { @@ -933,7 +933,7 @@ void Nextion::set_nextion_text_state(const std::string &name, const std::string } void Nextion::all_components_send_state_(bool force_update) { - ESP_LOGD(TAG, "Send states"); + ESP_LOGV(TAG, "Send states"); for (auto *binarysensortype : this->binarysensortype_) { if (force_update || binarysensortype->get_needs_to_send_update()) binarysensortype->send_state_to_nextion(); diff --git a/esphome/components/nextion/nextion_upload.cpp b/esphome/components/nextion/nextion_upload.cpp index c47b393f99..7ddd7a2f08 100644 --- a/esphome/components/nextion/nextion_upload.cpp +++ b/esphome/components/nextion/nextion_upload.cpp @@ -11,7 +11,10 @@ static const char *const TAG = "nextion.upload"; bool Nextion::upload_end_(bool successful) { if (successful) { ESP_LOGD(TAG, "Upload successful"); - delay(1500); // NOLINT + for (uint8_t i = 0; i <= 5; i++) { + delay(1000); // NOLINT + App.feed_wdt(); // Feed the watchdog timer. + } App.safe_reboot(); } else { ESP_LOGE(TAG, "Upload failed"); diff --git a/esphome/components/nfc/binary_sensor/binary_sensor.cpp b/esphome/components/nfc/binary_sensor/nfc_binary_sensor.cpp similarity index 99% rename from esphome/components/nfc/binary_sensor/binary_sensor.cpp rename to esphome/components/nfc/binary_sensor/nfc_binary_sensor.cpp index 8f1f6acd51..bc19fa7213 100644 --- a/esphome/components/nfc/binary_sensor/binary_sensor.cpp +++ b/esphome/components/nfc/binary_sensor/nfc_binary_sensor.cpp @@ -1,4 +1,4 @@ -#include "binary_sensor.h" +#include "nfc_binary_sensor.h" #include "../nfc_helpers.h" #include "esphome/core/log.h" diff --git a/esphome/components/nfc/binary_sensor/binary_sensor.h b/esphome/components/nfc/binary_sensor/nfc_binary_sensor.h similarity index 100% rename from esphome/components/nfc/binary_sensor/binary_sensor.h rename to esphome/components/nfc/binary_sensor/nfc_binary_sensor.h diff --git a/esphome/components/npi19/npi19.cpp b/esphome/components/npi19/npi19.cpp index 17ca0ef23e..c531d2ec8f 100644 --- a/esphome/components/npi19/npi19.cpp +++ b/esphome/components/npi19/npi19.cpp @@ -11,8 +11,6 @@ static const char *const TAG = "npi19"; static const uint8_t READ_COMMAND = 0xAC; void NPI19Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - uint16_t raw_temperature(0); uint16_t raw_pressure(0); i2c::ErrorCode err = this->read_(raw_temperature, raw_pressure); @@ -35,7 +33,7 @@ float NPI19Component::get_setup_priority() const { return setup_priority::DATA; i2c::ErrorCode NPI19Component::read_(uint16_t &raw_temperature, uint16_t &raw_pressure) { // initiate data read from device - i2c::ErrorCode w_err = write(&READ_COMMAND, sizeof(READ_COMMAND), true); + i2c::ErrorCode w_err = write(&READ_COMMAND, sizeof(READ_COMMAND)); if (w_err != i2c::ERROR_OK) { return w_err; } diff --git a/esphome/components/nrf52/__init__.py b/esphome/components/nrf52/__init__.py new file mode 100644 index 0000000000..727607933d --- /dev/null +++ b/esphome/components/nrf52/__init__.py @@ -0,0 +1,279 @@ +from __future__ import annotations + +import logging +from pathlib import Path + +from esphome import pins +import esphome.codegen as cg +from esphome.components.zephyr import ( + copy_files as zephyr_copy_files, + zephyr_add_pm_static, + zephyr_add_prj_conf, + zephyr_data, + zephyr_set_core_data, + zephyr_to_code, +) +from esphome.components.zephyr.const import ( + BOOTLOADER_MCUBOOT, + KEY_BOOTLOADER, + KEY_ZEPHYR, +) +import esphome.config_validation as cv +from esphome.const import ( + CONF_BOARD, + CONF_FRAMEWORK, + CONF_ID, + CONF_RESET_PIN, + KEY_CORE, + KEY_FRAMEWORK_VERSION, + KEY_TARGET_FRAMEWORK, + KEY_TARGET_PLATFORM, + PLATFORM_NRF52, + ThreadModel, +) +from esphome.core import CORE, CoroPriority, EsphomeError, coroutine_with_priority +from esphome.storage_json import StorageJSON +from esphome.types import ConfigType + +from .boards import BOARDS_ZEPHYR, BOOTLOADER_CONFIG +from .const import ( + BOOTLOADER_ADAFRUIT, + BOOTLOADER_ADAFRUIT_NRF52_SD132, + BOOTLOADER_ADAFRUIT_NRF52_SD140_V6, + BOOTLOADER_ADAFRUIT_NRF52_SD140_V7, +) + +# force import gpio to register pin schema +from .gpio import nrf52_pin_to_code # noqa + +CODEOWNERS = ["@tomaszduda23"] +AUTO_LOAD = ["zephyr"] +IS_TARGET_PLATFORM = True +_LOGGER = logging.getLogger(__name__) + + +def set_core_data(config: ConfigType) -> ConfigType: + zephyr_set_core_data(config) + CORE.data[KEY_CORE][KEY_TARGET_PLATFORM] = PLATFORM_NRF52 + CORE.data[KEY_CORE][KEY_TARGET_FRAMEWORK] = KEY_ZEPHYR + CORE.data[KEY_CORE][KEY_FRAMEWORK_VERSION] = cv.Version(2, 6, 1) + + if config[KEY_BOOTLOADER] in BOOTLOADER_CONFIG: + zephyr_add_pm_static(BOOTLOADER_CONFIG[config[KEY_BOOTLOADER]]) + + return config + + +BOOTLOADERS = [ + BOOTLOADER_ADAFRUIT, + BOOTLOADER_ADAFRUIT_NRF52_SD132, + BOOTLOADER_ADAFRUIT_NRF52_SD140_V6, + BOOTLOADER_ADAFRUIT_NRF52_SD140_V7, + BOOTLOADER_MCUBOOT, +] + + +def _detect_bootloader(config: ConfigType) -> ConfigType: + """Detect the bootloader for the given board.""" + config = config.copy() + bootloaders: list[str] = [] + board = config[CONF_BOARD] + + if board in BOARDS_ZEPHYR and KEY_BOOTLOADER in BOARDS_ZEPHYR[board]: + # this board have bootloaders config available + bootloaders = BOARDS_ZEPHYR[board][KEY_BOOTLOADER] + + if KEY_BOOTLOADER not in config: + if bootloaders: + # there is no bootloader in config -> take first one + config[KEY_BOOTLOADER] = bootloaders[0] + else: + # make mcuboot as default if there is no configuration for that board + config[KEY_BOOTLOADER] = BOOTLOADER_MCUBOOT + elif bootloaders and config[KEY_BOOTLOADER] not in bootloaders: + raise cv.Invalid( + f"{board} does not support {config[KEY_BOOTLOADER]}, select one of: {', '.join(bootloaders)}" + ) + return config + + +nrf52_ns = cg.esphome_ns.namespace("nrf52") +DeviceFirmwareUpdate = nrf52_ns.class_("DeviceFirmwareUpdate", cg.Component) + +CONF_DFU = "dfu" + +CONFIG_SCHEMA = cv.All( + _detect_bootloader, + set_core_data, + cv.Schema( + { + cv.Required(CONF_BOARD): cv.string_strict, + cv.Optional(KEY_BOOTLOADER): cv.one_of(*BOOTLOADERS, lower=True), + cv.Optional(CONF_DFU): cv.Schema( + { + cv.GenerateID(): cv.declare_id(DeviceFirmwareUpdate), + cv.Required(CONF_RESET_PIN): pins.gpio_output_pin_schema, + } + ), + } + ), +) + + +def _validate_mcumgr(config): + bootloader = zephyr_data()[KEY_BOOTLOADER] + if bootloader == BOOTLOADER_MCUBOOT: + raise cv.Invalid(f"'{bootloader}' bootloader does not support DFU") + + +def _final_validate(config): + if CONF_DFU in config: + _validate_mcumgr(config) + if config[KEY_BOOTLOADER] == BOOTLOADER_ADAFRUIT: + _LOGGER.warning( + "Selected generic Adafruit bootloader. The board might crash. Consider settings `bootloader:`" + ) + + +FINAL_VALIDATE_SCHEMA = _final_validate + + +@coroutine_with_priority(CoroPriority.PLATFORM) +async def to_code(config: ConfigType) -> None: + """Convert the configuration to code.""" + cg.add_platformio_option("board", config[CONF_BOARD]) + cg.add_build_flag("-DUSE_NRF52") + cg.add_define("ESPHOME_BOARD", config[CONF_BOARD]) + cg.add_define("ESPHOME_VARIANT", "NRF52") + # nRF52 processors are single-core + cg.add_define(ThreadModel.SINGLE) + cg.add_platformio_option(CONF_FRAMEWORK, CORE.data[KEY_CORE][KEY_TARGET_FRAMEWORK]) + cg.add_platformio_option( + "platform", + "https://github.com/tomaszduda23/platform-nordicnrf52/archive/refs/tags/v10.3.0-1.zip", + ) + cg.add_platformio_option( + "platform_packages", + [ + "platformio/framework-zephyr@https://github.com/tomaszduda23/framework-sdk-nrf/archive/refs/tags/v2.6.1-7.zip", + "platformio/toolchain-gccarmnoneeabi@https://github.com/tomaszduda23/toolchain-sdk-ng/archive/refs/tags/v0.17.4-0.zip", + ], + ) + + if config[KEY_BOOTLOADER] == BOOTLOADER_MCUBOOT: + cg.add_define("USE_BOOTLOADER_MCUBOOT") + else: + if "_sd" in config[KEY_BOOTLOADER]: + bootloader = config[KEY_BOOTLOADER].split("_") + sd_id = bootloader[2][2:] + cg.add_define("USE_SOFTDEVICE_ID", int(sd_id)) + if (len(bootloader)) > 3: + sd_version = bootloader[3][1:] + cg.add_define("USE_SOFTDEVICE_VERSION", int(sd_version)) + # make sure that firmware.zip is created + # for Adafruit_nRF52_Bootloader + cg.add_platformio_option("board_upload.protocol", "nrfutil") + cg.add_platformio_option("board_upload.use_1200bps_touch", "true") + cg.add_platformio_option("board_upload.require_upload_port", "true") + cg.add_platformio_option("board_upload.wait_for_upload_port", "true") + + zephyr_to_code(config) + + if dfu_config := config.get(CONF_DFU): + CORE.add_job(_dfu_to_code, dfu_config) + + +@coroutine_with_priority(CoroPriority.DIAGNOSTICS) +async def _dfu_to_code(dfu_config): + cg.add_define("USE_NRF52_DFU") + var = cg.new_Pvariable(dfu_config[CONF_ID]) + pin = await cg.gpio_pin_expression(dfu_config[CONF_RESET_PIN]) + cg.add(var.set_reset_pin(pin)) + zephyr_add_prj_conf("CDC_ACM_DTE_RATE_CALLBACK_SUPPORT", True) + await cg.register_component(var, dfu_config) + + +def copy_files() -> None: + """Copy files to the build directory.""" + zephyr_copy_files() + + +def get_download_types(storage_json: StorageJSON) -> list[dict[str, str]]: + """Get the download types for the firmware.""" + types = [] + UF2_PATH = "zephyr/zephyr.uf2" + DFU_PATH = "firmware.zip" + HEX_PATH = "zephyr/zephyr.hex" + HEX_MERGED_PATH = "zephyr/merged.hex" + APP_IMAGE_PATH = "zephyr/app_update.bin" + build_dir = Path(storage_json.firmware_bin_path).parent + if (build_dir / UF2_PATH).is_file(): + types = [ + { + "title": "UF2 package (recommended)", + "description": "For flashing via Adafruit nRF52 Bootloader as a flash drive.", + "file": UF2_PATH, + "download": f"{storage_json.name}.uf2", + }, + { + "title": "DFU package", + "description": "For flashing via adafruit-nrfutil using USB CDC.", + "file": DFU_PATH, + "download": f"dfu-{storage_json.name}.zip", + }, + ] + else: + types = [ + { + "title": "HEX package", + "description": "For flashing via pyocd using SWD.", + "file": ( + HEX_MERGED_PATH + if (build_dir / HEX_MERGED_PATH).is_file() + else HEX_PATH + ), + "download": f"{storage_json.name}.hex", + }, + ] + if (build_dir / APP_IMAGE_PATH).is_file(): + types += [ + { + "title": "App update package", + "description": "For flashing via mcumgr-web using BLE or smpclient using USB CDC.", + "file": APP_IMAGE_PATH, + "download": f"app-{storage_json.name}.img", + }, + ] + + return types + + +def _upload_using_platformio( + config: ConfigType, port: str, upload_args: list[str] +) -> int | str: + from esphome import platformio_api + + if port is not None: + upload_args += ["--upload-port", port] + return platformio_api.run_platformio_cli_run(config, CORE.verbose, *upload_args) + + +def upload_program(config: ConfigType, args, host: str) -> bool: + from esphome.__main__ import check_permissions, get_port_type + + result = 0 + handled = False + + if get_port_type(host) == "SERIAL": + check_permissions(host) + result = _upload_using_platformio(config, host, ["-t", "upload"]) + handled = True + + if host == "PYOCD": + result = _upload_using_platformio(config, host, ["-t", "flash_pyocd"]) + handled = True + + if result != 0: + raise EsphomeError(f"Upload failed with result: {result}") + + return handled diff --git a/esphome/components/nrf52/boards.py b/esphome/components/nrf52/boards.py new file mode 100644 index 0000000000..6064fe844a --- /dev/null +++ b/esphome/components/nrf52/boards.py @@ -0,0 +1,42 @@ +from esphome.components.zephyr import Section +from esphome.components.zephyr.const import KEY_BOOTLOADER + +from .const import ( + BOOTLOADER_ADAFRUIT, + BOOTLOADER_ADAFRUIT_NRF52_SD132, + BOOTLOADER_ADAFRUIT_NRF52_SD140_V6, + BOOTLOADER_ADAFRUIT_NRF52_SD140_V7, +) + +BOARDS_ZEPHYR = { + "adafruit_itsybitsy_nrf52840": { + KEY_BOOTLOADER: [ + BOOTLOADER_ADAFRUIT_NRF52_SD140_V6, + BOOTLOADER_ADAFRUIT, + BOOTLOADER_ADAFRUIT_NRF52_SD132, + BOOTLOADER_ADAFRUIT_NRF52_SD140_V7, + ] + }, + "xiao_ble": { + KEY_BOOTLOADER: [ + BOOTLOADER_ADAFRUIT_NRF52_SD140_V7, + BOOTLOADER_ADAFRUIT, + BOOTLOADER_ADAFRUIT_NRF52_SD132, + BOOTLOADER_ADAFRUIT_NRF52_SD140_V6, + ] + }, +} + +# https://github.com/ffenix113/zigbee_home/blob/17bb7b9e9d375e756da9e38913f53303937fb66a/types/board/known_boards.go +# https://learn.adafruit.com/introducing-the-adafruit-nrf52840-feather?view=all#hathach-memory-map +BOOTLOADER_CONFIG = { + BOOTLOADER_ADAFRUIT_NRF52_SD132: [ + Section("empty_app_offset", 0x0, 0x26000, "flash_primary"), + ], + BOOTLOADER_ADAFRUIT_NRF52_SD140_V6: [ + Section("empty_app_offset", 0x0, 0x26000, "flash_primary"), + ], + BOOTLOADER_ADAFRUIT_NRF52_SD140_V7: [ + Section("empty_app_offset", 0x0, 0x27000, "flash_primary"), + ], +} diff --git a/esphome/components/nrf52/const.py b/esphome/components/nrf52/const.py new file mode 100644 index 0000000000..977ca2252a --- /dev/null +++ b/esphome/components/nrf52/const.py @@ -0,0 +1,19 @@ +BOOTLOADER_ADAFRUIT = "adafruit" +BOOTLOADER_ADAFRUIT_NRF52_SD132 = "adafruit_nrf52_sd132" +BOOTLOADER_ADAFRUIT_NRF52_SD140_V6 = "adafruit_nrf52_sd140_v6" +BOOTLOADER_ADAFRUIT_NRF52_SD140_V7 = "adafruit_nrf52_sd140_v7" + +EXTRA_ADC = [ + "VDD", + "VDDHDIV5", +] +AIN_TO_GPIO = { + "AIN0": 2, + "AIN1": 3, + "AIN2": 4, + "AIN3": 5, + "AIN4": 28, + "AIN5": 29, + "AIN6": 30, + "AIN7": 31, +} diff --git a/esphome/components/nrf52/dfu.cpp b/esphome/components/nrf52/dfu.cpp new file mode 100644 index 0000000000..9e49373467 --- /dev/null +++ b/esphome/components/nrf52/dfu.cpp @@ -0,0 +1,51 @@ +#include "dfu.h" + +#ifdef USE_NRF52_DFU + +#include +#include +#include +#include "esphome/core/log.h" + +namespace esphome { +namespace nrf52 { + +static const char *const TAG = "dfu"; + +volatile bool goto_dfu = false; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) + +static const uint32_t DFU_DBL_RESET_MAGIC = 0x5A1AD5; // SALADS + +#define DEVICE_AND_COMMA(node_id) DEVICE_DT_GET(node_id), + +static void cdc_dte_rate_callback(const struct device * /*unused*/, uint32_t rate) { + if (rate == 1200) { + goto_dfu = true; + } +} +void DeviceFirmwareUpdate::setup() { + this->reset_pin_->setup(); + const struct device *cdc_dev[] = {DT_FOREACH_STATUS_OKAY(zephyr_cdc_acm_uart, DEVICE_AND_COMMA)}; + for (auto &idx : cdc_dev) { + cdc_acm_dte_rate_callback_set(idx, cdc_dte_rate_callback); + } +} + +void DeviceFirmwareUpdate::loop() { + if (goto_dfu) { + goto_dfu = false; + volatile uint32_t *dbl_reset_mem = (volatile uint32_t *) 0x20007F7C; + (*dbl_reset_mem) = DFU_DBL_RESET_MAGIC; + this->reset_pin_->digital_write(true); + } +} + +void DeviceFirmwareUpdate::dump_config() { + ESP_LOGCONFIG(TAG, "DFU:"); + LOG_PIN(" RESET Pin: ", this->reset_pin_); +} + +} // namespace nrf52 +} // namespace esphome + +#endif diff --git a/esphome/components/nrf52/dfu.h b/esphome/components/nrf52/dfu.h new file mode 100644 index 0000000000..979a4567cf --- /dev/null +++ b/esphome/components/nrf52/dfu.h @@ -0,0 +1,24 @@ +#pragma once + +#include "esphome/core/defines.h" +#ifdef USE_NRF52_DFU +#include "esphome/core/component.h" +#include "esphome/core/gpio.h" + +namespace esphome { +namespace nrf52 { +class DeviceFirmwareUpdate : public Component { + public: + void setup() override; + void loop() override; + void set_reset_pin(GPIOPin *reset) { this->reset_pin_ = reset; } + void dump_config() override; + + protected: + GPIOPin *reset_pin_; +}; + +} // namespace nrf52 +} // namespace esphome + +#endif diff --git a/esphome/components/nrf52/gpio.py b/esphome/components/nrf52/gpio.py new file mode 100644 index 0000000000..260114f90e --- /dev/null +++ b/esphome/components/nrf52/gpio.py @@ -0,0 +1,79 @@ +from esphome import pins +import esphome.codegen as cg +from esphome.components.zephyr.const import zephyr_ns +import esphome.config_validation as cv +from esphome.const import ( + CONF_ANALOG, + CONF_ID, + CONF_INVERTED, + CONF_MODE, + CONF_NUMBER, + PLATFORM_NRF52, +) + +from .const import AIN_TO_GPIO, EXTRA_ADC + +ZephyrGPIOPin = zephyr_ns.class_("ZephyrGPIOPin", cg.InternalGPIOPin) + + +def _translate_pin(value): + if value in AIN_TO_GPIO: + return AIN_TO_GPIO[value] + if isinstance(value, dict) or value is None: + raise cv.Invalid( + "This variable only supports pin numbers, not full pin schemas " + "(with inverted and mode)." + ) + if isinstance(value, int): + return value + try: + return int(value) + except ValueError: + pass + # e.g. P0.27 + if len(value) >= len("P0.0") and value[0] == "P" and value[2] == ".": + return cv.int_(value[len("P")].strip()) * 32 + cv.int_( + value[len("P0.") :].strip() + ) + raise cv.Invalid(f"Invalid pin: {value}") + + +def validate_gpio_pin(value): + if value in EXTRA_ADC: + return value + value = _translate_pin(value) + if value < 0 or value > (32 + 16): + raise cv.Invalid(f"NRF52: Invalid pin number: {value}") + return value + + +def validate_supports(value): + num = value[CONF_NUMBER] + mode = value[CONF_MODE] + is_analog = mode[CONF_ANALOG] + if is_analog: + if num in EXTRA_ADC: + return value + if num not in AIN_TO_GPIO.values(): + raise cv.Invalid(f"Cannot use {num} as analog pin") + return value + + +NRF52_PIN_SCHEMA = cv.All( + pins.gpio_base_schema( + ZephyrGPIOPin, + validate_gpio_pin, + modes=pins.GPIO_STANDARD_MODES + (CONF_ANALOG,), + ), + validate_supports, +) + + +@pins.PIN_SCHEMA_REGISTRY.register(PLATFORM_NRF52, NRF52_PIN_SCHEMA) +async def nrf52_pin_to_code(config): + var = cg.new_Pvariable(config[CONF_ID]) + num = config[CONF_NUMBER] + cg.add(var.set_pin(num)) + cg.add(var.set_inverted(config[CONF_INVERTED])) + cg.add(var.set_flags(pins.gpio_flags_expr(config[CONF_MODE]))) + return var diff --git a/esphome/components/ntc/ntc.cpp b/esphome/components/ntc/ntc.cpp index 333dbc5a75..b08f84029b 100644 --- a/esphome/components/ntc/ntc.cpp +++ b/esphome/components/ntc/ntc.cpp @@ -11,7 +11,7 @@ void NTC::setup() { if (this->sensor_->has_state()) this->process_(this->sensor_->state); } -void NTC::dump_config() { LOG_SENSOR("", "NTC Sensor", this) } +void NTC::dump_config() { LOG_SENSOR("", "NTC Sensor", this); } float NTC::get_setup_priority() const { return setup_priority::DATA; } void NTC::process_(float value) { if (std::isnan(value)) { diff --git a/esphome/components/number/__init__.py b/esphome/components/number/__init__.py index 4beed57188..230c3aa0c1 100644 --- a/esphome/components/number/__init__.py +++ b/esphome/components/number/__init__.py @@ -19,6 +19,7 @@ from esphome.const import ( CONF_UNIT_OF_MEASUREMENT, CONF_VALUE, CONF_WEB_SERVER, + DEVICE_CLASS_ABSOLUTE_HUMIDITY, DEVICE_CLASS_APPARENT_POWER, DEVICE_CLASS_AQI, DEVICE_CLASS_AREA, @@ -50,6 +51,7 @@ from esphome.const import ( DEVICE_CLASS_OZONE, DEVICE_CLASS_PH, DEVICE_CLASS_PM1, + DEVICE_CLASS_PM4, DEVICE_CLASS_PM10, DEVICE_CLASS_PM25, DEVICE_CLASS_POWER, @@ -64,6 +66,7 @@ from esphome.const import ( DEVICE_CLASS_SPEED, DEVICE_CLASS_SULPHUR_DIOXIDE, DEVICE_CLASS_TEMPERATURE, + DEVICE_CLASS_TEMPERATURE_DELTA, DEVICE_CLASS_VOLATILE_ORGANIC_COMPOUNDS, DEVICE_CLASS_VOLATILE_ORGANIC_COMPOUNDS_PARTS, DEVICE_CLASS_VOLTAGE, @@ -75,12 +78,13 @@ from esphome.const import ( DEVICE_CLASS_WIND_DIRECTION, DEVICE_CLASS_WIND_SPEED, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass CODEOWNERS = ["@esphome/core"] DEVICE_CLASSES = [ + DEVICE_CLASS_ABSOLUTE_HUMIDITY, DEVICE_CLASS_APPARENT_POWER, DEVICE_CLASS_AQI, DEVICE_CLASS_AREA, @@ -114,6 +118,7 @@ DEVICE_CLASSES = [ DEVICE_CLASS_PM1, DEVICE_CLASS_PM10, DEVICE_CLASS_PM25, + DEVICE_CLASS_PM4, DEVICE_CLASS_POWER, DEVICE_CLASS_POWER_FACTOR, DEVICE_CLASS_PRECIPITATION, @@ -126,6 +131,7 @@ DEVICE_CLASSES = [ DEVICE_CLASS_SPEED, DEVICE_CLASS_SULPHUR_DIOXIDE, DEVICE_CLASS_TEMPERATURE, + DEVICE_CLASS_TEMPERATURE_DELTA, DEVICE_CLASS_VOLATILE_ORGANIC_COMPOUNDS, DEVICE_CLASS_VOLATILE_ORGANIC_COMPOUNDS_PARTS, DEVICE_CLASS_VOLTAGE, @@ -319,9 +325,8 @@ async def number_in_range_to_code(config, condition_id, template_arg, args): return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_NUMBER") cg.add_global(number_ns.using) diff --git a/esphome/components/number/automation.cpp b/esphome/components/number/automation.cpp index cadc6f54f6..bfc59d0465 100644 --- a/esphome/components/number/automation.cpp +++ b/esphome/components/number/automation.cpp @@ -15,7 +15,7 @@ void ValueRangeTrigger::setup() { float local_min = this->min_.value(0.0); float local_max = this->max_.value(0.0); convert hash = {.from = (local_max - local_min)}; - uint32_t myhash = hash.to ^ this->parent_->get_object_id_hash(); + uint32_t myhash = hash.to ^ this->parent_->get_preference_hash(); this->rtc_ = global_preferences->make_preference(myhash); bool initial_state; if (this->rtc_.load(&initial_state)) { diff --git a/esphome/components/number/number.cpp b/esphome/components/number/number.cpp index b6a845b19b..da08faf655 100644 --- a/esphome/components/number/number.cpp +++ b/esphome/components/number/number.cpp @@ -6,6 +6,27 @@ namespace number { static const char *const TAG = "number"; +// Function implementation of LOG_NUMBER macro to reduce code size +void log_number(const char *tag, const char *prefix, const char *type, Number *obj) { + if (obj == nullptr) { + return; + } + + ESP_LOGCONFIG(tag, "%s%s '%s'", prefix, type, obj->get_name().c_str()); + + if (!obj->get_icon_ref().empty()) { + ESP_LOGCONFIG(tag, "%s Icon: '%s'", prefix, obj->get_icon_ref().c_str()); + } + + if (!obj->traits.get_unit_of_measurement_ref().empty()) { + ESP_LOGCONFIG(tag, "%s Unit of Measurement: '%s'", prefix, obj->traits.get_unit_of_measurement_ref().c_str()); + } + + if (!obj->traits.get_device_class_ref().empty()) { + ESP_LOGCONFIG(tag, "%s Device Class: '%s'", prefix, obj->traits.get_device_class_ref().c_str()); + } +} + void Number::publish_state(float state) { this->set_has_state(true); this->state = state; diff --git a/esphome/components/number/number.h b/esphome/components/number/number.h index 49bcbb857c..da91d70d53 100644 --- a/esphome/components/number/number.h +++ b/esphome/components/number/number.h @@ -9,19 +9,10 @@ namespace esphome { namespace number { -#define LOG_NUMBER(prefix, type, obj) \ - if ((obj) != nullptr) { \ - ESP_LOGCONFIG(TAG, "%s%s '%s'", prefix, LOG_STR_LITERAL(type), (obj)->get_name().c_str()); \ - if (!(obj)->get_icon().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon().c_str()); \ - } \ - if (!(obj)->traits.get_unit_of_measurement().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Unit of Measurement: '%s'", prefix, (obj)->traits.get_unit_of_measurement().c_str()); \ - } \ - if (!(obj)->traits.get_device_class().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Device Class: '%s'", prefix, (obj)->traits.get_device_class().c_str()); \ - } \ - } +class Number; +void log_number(const char *tag, const char *prefix, const char *type, Number *obj); + +#define LOG_NUMBER(prefix, type, obj) log_number(TAG, prefix, LOG_STR_LITERAL(type), obj) #define SUB_NUMBER(name) \ protected: \ diff --git a/esphome/components/number/number_call.cpp b/esphome/components/number/number_call.cpp index 4219f85328..669dd65184 100644 --- a/esphome/components/number/number_call.cpp +++ b/esphome/components/number/number_call.cpp @@ -7,6 +7,17 @@ namespace number { static const char *const TAG = "number"; +// Helper functions to reduce code size for logging +void NumberCall::log_perform_warning_(const LogString *message) { + ESP_LOGW(TAG, "'%s': %s", this->parent_->get_name().c_str(), LOG_STR_ARG(message)); +} + +void NumberCall::log_perform_warning_value_range_(const LogString *comparison, const LogString *limit_type, float val, + float limit) { + ESP_LOGW(TAG, "'%s': %f %s %s %f", this->parent_->get_name().c_str(), val, LOG_STR_ARG(comparison), + LOG_STR_ARG(limit_type), limit); +} + NumberCall &NumberCall::set_value(float value) { return this->with_operation(NUMBER_OP_SET).with_value(value); } NumberCall &NumberCall::number_increment(bool cycle) { @@ -42,7 +53,7 @@ void NumberCall::perform() { const auto &traits = parent->traits; if (this->operation_ == NUMBER_OP_NONE) { - ESP_LOGW(TAG, "'%s' - NumberCall performed without selecting an operation", name); + this->log_perform_warning_(LOG_STR("No operation")); return; } @@ -51,28 +62,28 @@ void NumberCall::perform() { float max_value = traits.get_max_value(); if (this->operation_ == NUMBER_OP_SET) { - ESP_LOGD(TAG, "'%s' - Setting number value", name); + ESP_LOGD(TAG, "'%s': Setting value", name); if (!this->value_.has_value() || std::isnan(*this->value_)) { - ESP_LOGW(TAG, "'%s' - No value set for NumberCall", name); + this->log_perform_warning_(LOG_STR("No value")); return; } target_value = this->value_.value(); } else if (this->operation_ == NUMBER_OP_TO_MIN) { if (std::isnan(min_value)) { - ESP_LOGW(TAG, "'%s' - Can't set to min value through NumberCall: no min_value defined", name); + this->log_perform_warning_(LOG_STR("min undefined")); } else { target_value = min_value; } } else if (this->operation_ == NUMBER_OP_TO_MAX) { if (std::isnan(max_value)) { - ESP_LOGW(TAG, "'%s' - Can't set to max value through NumberCall: no max_value defined", name); + this->log_perform_warning_(LOG_STR("max undefined")); } else { target_value = max_value; } } else if (this->operation_ == NUMBER_OP_INCREMENT) { - ESP_LOGD(TAG, "'%s' - Increment number, with%s cycling", name, this->cycle_ ? "" : "out"); + ESP_LOGD(TAG, "'%s': Increment with%s cycling", name, this->cycle_ ? "" : "out"); if (!parent->has_state()) { - ESP_LOGW(TAG, "'%s' - Can't increment number through NumberCall: no active state to modify", name); + this->log_perform_warning_(LOG_STR("Can't increment, no state")); return; } auto step = traits.get_step(); @@ -85,9 +96,9 @@ void NumberCall::perform() { } } } else if (this->operation_ == NUMBER_OP_DECREMENT) { - ESP_LOGD(TAG, "'%s' - Decrement number, with%s cycling", name, this->cycle_ ? "" : "out"); + ESP_LOGD(TAG, "'%s': Decrement with%s cycling", name, this->cycle_ ? "" : "out"); if (!parent->has_state()) { - ESP_LOGW(TAG, "'%s' - Can't decrement number through NumberCall: no active state to modify", name); + this->log_perform_warning_(LOG_STR("Can't decrement, no state")); return; } auto step = traits.get_step(); @@ -102,15 +113,15 @@ void NumberCall::perform() { } if (target_value < min_value) { - ESP_LOGW(TAG, "'%s' - Value %f must not be less than minimum %f", name, target_value, min_value); + this->log_perform_warning_value_range_(LOG_STR("<"), LOG_STR("min"), target_value, min_value); return; } if (target_value > max_value) { - ESP_LOGW(TAG, "'%s' - Value %f must not be greater than maximum %f", name, target_value, max_value); + this->log_perform_warning_value_range_(LOG_STR(">"), LOG_STR("max"), target_value, max_value); return; } - ESP_LOGD(TAG, " New number value: %f", target_value); + ESP_LOGD(TAG, " New value: %f", target_value); this->parent_->control(target_value); } diff --git a/esphome/components/number/number_call.h b/esphome/components/number/number_call.h index bd50170be5..807207f0ec 100644 --- a/esphome/components/number/number_call.h +++ b/esphome/components/number/number_call.h @@ -1,6 +1,7 @@ #pragma once #include "esphome/core/helpers.h" +#include "esphome/core/log.h" #include "number_traits.h" namespace esphome { @@ -33,6 +34,10 @@ class NumberCall { NumberCall &with_cycle(bool cycle); protected: + void log_perform_warning_(const LogString *message); + void log_perform_warning_value_range_(const LogString *comparison, const LogString *limit_type, float val, + float limit); + Number *const parent_; NumberOperation operation_{NUMBER_OP_NONE}; optional value_; diff --git a/esphome/components/one_wire/__init__.py b/esphome/components/one_wire/__init__.py index 99a1ccd1eb..6d95b8fd33 100644 --- a/esphome/components/one_wire/__init__.py +++ b/esphome/components/one_wire/__init__.py @@ -18,13 +18,12 @@ def one_wire_device_schema(): :return: The 1-wire device schema, `extend` this in your config schema. """ - schema = cv.Schema( + return cv.Schema( { cv.GenerateID(CONF_ONE_WIRE_ID): cv.use_id(OneWireBus), cv.Optional(CONF_ADDRESS): cv.hex_uint64_t, } ) - return schema async def register_one_wire_device(var, config): diff --git a/esphome/components/one_wire/one_wire.cpp b/esphome/components/one_wire/one_wire.cpp index 131bc4fbfe..96e6145f63 100644 --- a/esphome/components/one_wire/one_wire.cpp +++ b/esphome/components/one_wire/one_wire.cpp @@ -11,8 +11,6 @@ const std::string &OneWireDevice::get_address_name() { return this->address_name_; } -std::string OneWireDevice::unique_id() { return "dallas-" + str_lower_case(format_hex(this->address_)); } - bool OneWireDevice::send_command_(uint8_t cmd) { if (!this->bus_->select(this->address_)) return false; diff --git a/esphome/components/one_wire/one_wire.h b/esphome/components/one_wire/one_wire.h index bf10e4f82e..e83c6e81e8 100644 --- a/esphome/components/one_wire/one_wire.h +++ b/esphome/components/one_wire/one_wire.h @@ -24,8 +24,6 @@ class OneWireDevice { /// Helper to create (and cache) the name for this sensor. For example "0xfe0000031f1eaf29". const std::string &get_address_name(); - std::string unique_id(); - protected: uint64_t address_{0}; OneWireBus *bus_{nullptr}; ///< pointer to OneWireBus instance diff --git a/esphome/components/online_image/bmp_image.cpp b/esphome/components/online_image/bmp_image.cpp index f55c9f1813..676a2efca9 100644 --- a/esphome/components/online_image/bmp_image.cpp +++ b/esphome/components/online_image/bmp_image.cpp @@ -117,7 +117,8 @@ int HOT BmpDecoder::decode(uint8_t *buffer, size_t size) { this->paint_index_++; this->current_index_ += 3; index += 3; - if (x == this->width_ - 1 && this->padding_bytes_ > 0) { + size_t last_col = static_cast(this->width_) - 1; + if (x == last_col && this->padding_bytes_ > 0) { index += this->padding_bytes_; this->current_index_ += this->padding_bytes_; } diff --git a/esphome/components/online_image/jpeg_image.cpp b/esphome/components/online_image/jpeg_image.cpp index e5ee3dd8bf..10586091d5 100644 --- a/esphome/components/online_image/jpeg_image.cpp +++ b/esphome/components/online_image/jpeg_image.cpp @@ -25,8 +25,10 @@ static int draw_callback(JPEGDRAW *jpeg) { // to avoid crashing. App.feed_wdt(); size_t position = 0; - for (size_t y = 0; y < jpeg->iHeight; y++) { - for (size_t x = 0; x < jpeg->iWidth; x++) { + size_t height = static_cast(jpeg->iHeight); + size_t width = static_cast(jpeg->iWidth); + for (size_t y = 0; y < height; y++) { + for (size_t x = 0; x < width; x++) { auto rg = decode_value(jpeg->pPixels[position++]); auto ba = decode_value(jpeg->pPixels[position++]); Color color(rg[1], rg[0], ba[1], ba[0]); diff --git a/esphome/components/opentherm/hub.cpp b/esphome/components/opentherm/hub.cpp index 0a4ef98507..b23792fc7a 100644 --- a/esphome/components/opentherm/hub.cpp +++ b/esphome/components/opentherm/hub.cpp @@ -145,7 +145,6 @@ void OpenthermHub::process_response(OpenthermData &data) { } void OpenthermHub::setup() { - ESP_LOGD(TAG, "Setting up OpenTherm component"); this->opentherm_ = make_unique(this->in_pin_, this->out_pin_); if (!this->opentherm_->initialize()) { ESP_LOGE(TAG, "Failed to initialize OpenTherm protocol. See previous log messages for details."); diff --git a/esphome/components/opentherm/hub.h b/esphome/components/opentherm/hub.h index 80fd268820..ee0cfd104d 100644 --- a/esphome/components/opentherm/hub.h +++ b/esphome/components/opentherm/hub.h @@ -1,10 +1,10 @@ #pragma once +#include +#include "esphome/core/component.h" #include "esphome/core/defines.h" #include "esphome/core/hal.h" -#include "esphome/core/component.h" #include "esphome/core/log.h" -#include #include "opentherm.h" @@ -17,21 +17,21 @@ #endif #ifdef OPENTHERM_USE_SWITCH -#include "esphome/components/opentherm/switch/switch.h" +#include "esphome/components/opentherm/switch/opentherm_switch.h" #endif #ifdef OPENTHERM_USE_OUTPUT -#include "esphome/components/opentherm/output/output.h" +#include "esphome/components/opentherm/output/opentherm_output.h" #endif #ifdef OPENTHERM_USE_NUMBER -#include "esphome/components/opentherm/number/number.h" +#include "esphome/components/opentherm/number/opentherm_number.h" #endif +#include #include #include #include -#include #include "opentherm_macros.h" diff --git a/esphome/components/opentherm/number/__init__.py b/esphome/components/opentherm/number/__init__.py index a65864647a..6dbc45f49b 100644 --- a/esphome/components/opentherm/number/__init__.py +++ b/esphome/components/opentherm/number/__init__.py @@ -25,9 +25,9 @@ async def new_openthermnumber(config: dict[str, Any]) -> cg.Pvariable: await cg.register_component(var, config) input.generate_setters(var, config) - if (initial_value := config.get(CONF_INITIAL_VALUE, None)) is not None: + if (initial_value := config.get(CONF_INITIAL_VALUE)) is not None: cg.add(var.set_initial_value(initial_value)) - if (restore_value := config.get(CONF_RESTORE_VALUE, None)) is not None: + if (restore_value := config.get(CONF_RESTORE_VALUE)) is not None: cg.add(var.set_restore_value(restore_value)) return var diff --git a/esphome/components/opentherm/number/number.cpp b/esphome/components/opentherm/number/opentherm_number.cpp similarity index 94% rename from esphome/components/opentherm/number/number.cpp rename to esphome/components/opentherm/number/opentherm_number.cpp index 90ab5d6490..f0c69144c8 100644 --- a/esphome/components/opentherm/number/number.cpp +++ b/esphome/components/opentherm/number/opentherm_number.cpp @@ -1,4 +1,4 @@ -#include "number.h" +#include "opentherm_number.h" namespace esphome { namespace opentherm { @@ -17,7 +17,7 @@ void OpenthermNumber::setup() { if (!this->restore_value_) { value = this->initial_value_; } else { - this->pref_ = global_preferences->make_preference(this->get_object_id_hash()); + this->pref_ = global_preferences->make_preference(this->get_preference_hash()); if (!this->pref_.load(&value)) { if (!std::isnan(this->initial_value_)) { value = this->initial_value_; diff --git a/esphome/components/opentherm/number/number.h b/esphome/components/opentherm/number/opentherm_number.h similarity index 100% rename from esphome/components/opentherm/number/number.h rename to esphome/components/opentherm/number/opentherm_number.h diff --git a/esphome/components/opentherm/opentherm.cpp b/esphome/components/opentherm/opentherm.cpp index b2751470b2..d59b9584d1 100644 --- a/esphome/components/opentherm/opentherm.cpp +++ b/esphome/components/opentherm/opentherm.cpp @@ -7,7 +7,7 @@ #include "opentherm.h" #include "esphome/core/helpers.h" -#if defined(ESP32) || defined(USE_ESP_IDF) +#ifdef USE_ESP32 #include "driver/timer.h" #include "esp_err.h" #endif @@ -31,7 +31,7 @@ OpenTherm *OpenTherm::instance = nullptr; OpenTherm::OpenTherm(InternalGPIOPin *in_pin, InternalGPIOPin *out_pin, int32_t device_timeout) : in_pin_(in_pin), out_pin_(out_pin), -#if defined(ESP32) || defined(USE_ESP_IDF) +#ifdef USE_ESP32 timer_group_(TIMER_GROUP_0), timer_idx_(TIMER_0), #endif @@ -57,7 +57,7 @@ bool OpenTherm::initialize() { this->out_pin_->setup(); this->out_pin_->digital_write(true); -#if defined(ESP32) || defined(USE_ESP_IDF) +#ifdef USE_ESP32 return this->init_esp32_timer_(); #else return true; @@ -238,7 +238,7 @@ void IRAM_ATTR OpenTherm::write_bit_(uint8_t high, uint8_t clock) { } } -#if defined(ESP32) || defined(USE_ESP_IDF) +#ifdef USE_ESP32 bool OpenTherm::init_esp32_timer_() { // Search for a free timer. Maybe unstable, we'll see. @@ -365,7 +365,7 @@ void IRAM_ATTR OpenTherm::stop_timer_() { } } -#endif // END ESP32 +#endif // USE_ESP32 #ifdef ESP8266 // 5 kHz timer_ diff --git a/esphome/components/opentherm/opentherm.h b/esphome/components/opentherm/opentherm.h index a5822cdfe1..3996481760 100644 --- a/esphome/components/opentherm/opentherm.h +++ b/esphome/components/opentherm/opentherm.h @@ -12,7 +12,7 @@ #include "esphome/core/helpers.h" #include "esphome/core/log.h" -#if defined(ESP32) || defined(USE_ESP_IDF) +#ifdef USE_ESP32 #include "driver/timer.h" #endif @@ -356,7 +356,7 @@ class OpenTherm { ISRInternalGPIOPin isr_in_pin_; ISRInternalGPIOPin isr_out_pin_; -#if defined(ESP32) || defined(USE_ESP_IDF) +#ifdef USE_ESP32 timer_group_t timer_group_; timer_idx_t timer_idx_; #endif @@ -370,7 +370,7 @@ class OpenTherm { int32_t timeout_counter_; // <0 no timeout int32_t device_timeout_; -#if defined(ESP32) || defined(USE_ESP_IDF) +#ifdef USE_ESP32 esp_err_t timer_error_ = ESP_OK; TimerErrorType timer_error_type_ = TimerErrorType::NO_TIMER_ERROR; diff --git a/esphome/components/opentherm/output/output.cpp b/esphome/components/opentherm/output/opentherm_output.cpp similarity index 95% rename from esphome/components/opentherm/output/output.cpp rename to esphome/components/opentherm/output/opentherm_output.cpp index 486aa0d4e7..ff82ddd72c 100644 --- a/esphome/components/opentherm/output/output.cpp +++ b/esphome/components/opentherm/output/opentherm_output.cpp @@ -1,5 +1,5 @@ #include "esphome/core/helpers.h" // for clamp() and lerp() -#include "output.h" +#include "opentherm_output.h" namespace esphome { namespace opentherm { diff --git a/esphome/components/opentherm/output/output.h b/esphome/components/opentherm/output/opentherm_output.h similarity index 100% rename from esphome/components/opentherm/output/output.h rename to esphome/components/opentherm/output/opentherm_output.h diff --git a/esphome/components/opentherm/switch/switch.cpp b/esphome/components/opentherm/switch/opentherm_switch.cpp similarity index 96% rename from esphome/components/opentherm/switch/switch.cpp rename to esphome/components/opentherm/switch/opentherm_switch.cpp index 228d9ac8f3..5c5d62e68e 100644 --- a/esphome/components/opentherm/switch/switch.cpp +++ b/esphome/components/opentherm/switch/opentherm_switch.cpp @@ -1,4 +1,4 @@ -#include "switch.h" +#include "opentherm_switch.h" namespace esphome { namespace opentherm { diff --git a/esphome/components/opentherm/switch/switch.h b/esphome/components/opentherm/switch/opentherm_switch.h similarity index 100% rename from esphome/components/opentherm/switch/switch.h rename to esphome/components/opentherm/switch/opentherm_switch.h diff --git a/esphome/components/openthread/__init__.py b/esphome/components/openthread/__init__.py index 25e3153d1b..3fac497c3d 100644 --- a/esphome/components/openthread/__init__.py +++ b/esphome/components/openthread/__init__.py @@ -5,7 +5,7 @@ from esphome.components.esp32 import ( add_idf_sdkconfig_option, only_on_variant, ) -from esphome.components.mdns import MDNSComponent +from esphome.components.mdns import MDNSComponent, enable_mdns_storage import esphome.config_validation as cv from esphome.const import CONF_CHANNEL, CONF_ENABLE_IPV6, CONF_ID import esphome.final_validate as fv @@ -79,9 +79,8 @@ def set_sdkconfig_options(config): "CONFIG_OPENTHREAD_NETWORK_PSKC", f"{pskc:X}".lower() ) - if force_dataset := config.get(CONF_FORCE_DATASET): - if force_dataset: - cg.add_define("USE_OPENTHREAD_FORCE_DATASET") + if config.get(CONF_FORCE_DATASET): + cg.add_define("USE_OPENTHREAD_FORCE_DATASET") add_idf_sdkconfig_option("CONFIG_OPENTHREAD_DNS64_CLIENT", True) add_idf_sdkconfig_option("CONFIG_OPENTHREAD_SRP_CLIENT", True) @@ -142,6 +141,9 @@ FINAL_VALIDATE_SCHEMA = _final_validate async def to_code(config): cg.add_define("USE_OPENTHREAD") + # OpenThread SRP needs access to mDNS services after setup + enable_mdns_storage() + ot = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(ot, config) diff --git a/esphome/components/openthread/openthread.cpp b/esphome/components/openthread/openthread.cpp index 24b3c23960..b2c2519c08 100644 --- a/esphome/components/openthread/openthread.cpp +++ b/esphome/components/openthread/openthread.cpp @@ -1,6 +1,9 @@ #include "esphome/core/defines.h" #ifdef USE_OPENTHREAD #include "openthread.h" +#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 5, 0) +#include "esp_openthread.h" +#endif #include @@ -8,8 +11,6 @@ #include #include #include -#include -#include #include #include @@ -28,18 +29,6 @@ OpenThreadComponent *global_openthread_component = // NOLINT(cppcoreguidelines- OpenThreadComponent::OpenThreadComponent() { global_openthread_component = this; } -OpenThreadComponent::~OpenThreadComponent() { - auto lock = InstanceLock::try_acquire(100); - if (!lock) { - ESP_LOGW(TAG, "Failed to acquire OpenThread lock in destructor, leaking memory"); - return; - } - otInstance *instance = lock->get_instance(); - otSrpClientClearHostAndServices(instance); - otSrpClientBuffersFreeAllServices(instance); - global_openthread_component = nullptr; -} - bool OpenThreadComponent::is_connected() { auto lock = InstanceLock::try_acquire(100); if (!lock) { @@ -86,8 +75,14 @@ std::optional OpenThreadComponent::get_omr_address_(InstanceLock & return {}; } -void srp_callback(otError err, const otSrpClientHostInfo *host_info, const otSrpClientService *services, - const otSrpClientService *removed_services, void *context) { +void OpenThreadComponent::defer_factory_reset_external_callback() { + ESP_LOGD(TAG, "Defer factory_reset_external_callback_"); + this->defer([this]() { this->factory_reset_external_callback_(); }); +} + +void OpenThreadSrpComponent::srp_callback(otError err, const otSrpClientHostInfo *host_info, + const otSrpClientService *services, + const otSrpClientService *removed_services, void *context) { if (err != 0) { ESP_LOGW(TAG, "SRP client reported an error: %s", otThreadErrorToString(err)); for (const otSrpClientHostInfo *host = host_info; host; host = nullptr) { @@ -99,16 +94,30 @@ void srp_callback(otError err, const otSrpClientHostInfo *host_info, const otSrp } } -void srp_start_callback(const otSockAddr *server_socket_address, void *context) { +void OpenThreadSrpComponent::srp_start_callback(const otSockAddr *server_socket_address, void *context) { ESP_LOGI(TAG, "SRP client has started"); } +void OpenThreadSrpComponent::srp_factory_reset_callback(otError err, const otSrpClientHostInfo *host_info, + const otSrpClientService *services, + const otSrpClientService *removed_services, void *context) { + OpenThreadComponent *obj = (OpenThreadComponent *) context; + if (err == OT_ERROR_NONE && removed_services != NULL && host_info != NULL && + host_info->mState == OT_SRP_CLIENT_ITEM_STATE_REMOVED) { + ESP_LOGD(TAG, "Successful Removal SRP Host and Services"); + } else if (err != OT_ERROR_NONE) { + // Handle other SRP client events or errors + ESP_LOGW(TAG, "SRP client event/error: %s", otThreadErrorToString(err)); + } + obj->defer_factory_reset_external_callback(); +} + void OpenThreadSrpComponent::setup() { otError error; InstanceLock lock = InstanceLock::acquire(); otInstance *instance = lock.get_instance(); - otSrpClientSetCallback(instance, srp_callback, nullptr); + otSrpClientSetCallback(instance, OpenThreadSrpComponent::srp_callback, nullptr); // set the host name uint16_t size; @@ -134,11 +143,10 @@ void OpenThreadSrpComponent::setup() { return; } - // Copy the mdns services to our local instance so that the c_str pointers remain valid for the lifetime of this - // component - this->mdns_services_ = this->mdns_->get_services(); - ESP_LOGD(TAG, "Setting up SRP services. count = %d\n", this->mdns_services_.size()); - for (const auto &service : this->mdns_services_) { + // Get mdns services and copy their data (strings are copied with strdup below) + const auto &mdns_services = this->mdns_->get_services(); + ESP_LOGD(TAG, "Setting up SRP services. count = %d\n", mdns_services.size()); + for (const auto &service : mdns_services) { otSrpClientBuffersServiceEntry *entry = otSrpClientBuffersAllocateService(instance); if (!entry) { ESP_LOGW(TAG, "Failed to allocate service entry"); @@ -147,7 +155,7 @@ void OpenThreadSrpComponent::setup() { // Set service name char *string = otSrpClientBuffersGetServiceEntryServiceNameString(entry, &size); - std::string full_service = service.service_type + "." + service.proto; + std::string full_service = std::string(MDNS_STR_ARG(service.service_type)) + "." + MDNS_STR_ARG(service.proto); if (full_service.size() > size) { ESP_LOGW(TAG, "Service name too long: %s", full_service.c_str()); continue; @@ -172,10 +180,12 @@ void OpenThreadSrpComponent::setup() { entry->mService.mNumTxtEntries = service.txt_records.size(); for (size_t i = 0; i < service.txt_records.size(); i++) { const auto &txt = service.txt_records[i]; - auto value = const_cast &>(txt.value).value(); - txt_entries[i].mKey = strdup(txt.key.c_str()); - txt_entries[i].mValue = reinterpret_cast(strdup(value.c_str())); - txt_entries[i].mValueLength = value.size(); + // Value is either a compile-time string literal in flash or a pointer to dynamic_txt_values_ + // OpenThread SRP client expects the data to persist, so we strdup it + const char *value_str = MDNS_STR_ARG(txt.value); + txt_entries[i].mKey = MDNS_STR_ARG(txt.key); + txt_entries[i].mValue = reinterpret_cast(strdup(value_str)); + txt_entries[i].mValueLength = strlen(value_str); } entry->mService.mTxtEntries = txt_entries; entry->mService.mNumTxtEntries = service.txt_records.size(); @@ -188,7 +198,7 @@ void OpenThreadSrpComponent::setup() { ESP_LOGD(TAG, "Added service: %s", full_service.c_str()); } - otSrpClientEnableAutoStartMode(instance, srp_start_callback, nullptr); + otSrpClientEnableAutoStartMode(instance, OpenThreadSrpComponent::srp_start_callback, nullptr); ESP_LOGD(TAG, "Finished SRP setup"); } @@ -200,6 +210,48 @@ void *OpenThreadSrpComponent::pool_alloc_(size_t size) { void OpenThreadSrpComponent::set_mdns(esphome::mdns::MDNSComponent *mdns) { this->mdns_ = mdns; } +bool OpenThreadComponent::teardown() { + if (!this->teardown_started_) { + this->teardown_started_ = true; + ESP_LOGD(TAG, "Clear Srp"); + auto lock = InstanceLock::try_acquire(100); + if (!lock) { + ESP_LOGW(TAG, "Failed to acquire OpenThread lock during teardown, leaking memory"); + return true; + } + otInstance *instance = lock->get_instance(); + otSrpClientClearHostAndServices(instance); + otSrpClientBuffersFreeAllServices(instance); + global_openthread_component = nullptr; +#if ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 5, 0) + ESP_LOGD(TAG, "Exit main loop "); + int error = esp_openthread_mainloop_exit(); + if (error != ESP_OK) { + ESP_LOGW(TAG, "Failed attempt to stop main loop %d", error); + this->teardown_complete_ = true; + } +#else + this->teardown_complete_ = true; +#endif + } + return this->teardown_complete_; +} + +void OpenThreadComponent::on_factory_reset(std::function callback) { + factory_reset_external_callback_ = callback; + ESP_LOGD(TAG, "Start Removal SRP Host and Services"); + otError error; + InstanceLock lock = InstanceLock::acquire(); + otInstance *instance = lock.get_instance(); + otSrpClientSetCallback(instance, OpenThreadSrpComponent::srp_factory_reset_callback, this); + error = otSrpClientRemoveHostAndServices(instance, true, true); + if (error != OT_ERROR_NONE) { + ESP_LOGW(TAG, "Failed to Remove SRP Host and Services"); + return; + } + ESP_LOGD(TAG, "Waiting on Confirmation Removal SRP Host and Services"); +} + } // namespace openthread } // namespace esphome diff --git a/esphome/components/openthread/openthread.h b/esphome/components/openthread/openthread.h index 77fd58851a..5d139c633d 100644 --- a/esphome/components/openthread/openthread.h +++ b/esphome/components/openthread/openthread.h @@ -6,6 +6,8 @@ #include "esphome/components/network/ip_address.h" #include "esphome/core/component.h" +#include +#include #include #include @@ -21,15 +23,21 @@ class OpenThreadComponent : public Component { OpenThreadComponent(); ~OpenThreadComponent(); void setup() override; + bool teardown() override; float get_setup_priority() const override { return setup_priority::WIFI; } bool is_connected(); network::IPAddresses get_ip_addresses(); std::optional get_omr_address(); void ot_main(); + void on_factory_reset(std::function callback); + void defer_factory_reset_external_callback(); protected: std::optional get_omr_address_(InstanceLock &lock); + bool teardown_started_{false}; + bool teardown_complete_{false}; + std::function factory_reset_external_callback_; }; extern OpenThreadComponent *global_openthread_component; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) @@ -40,10 +48,15 @@ class OpenThreadSrpComponent : public Component { // This has to run after the mdns component or else no services are available to advertise float get_setup_priority() const override { return this->mdns_->get_setup_priority() - 1.0; } void setup() override; + static void srp_callback(otError err, const otSrpClientHostInfo *host_info, const otSrpClientService *services, + const otSrpClientService *removed_services, void *context); + static void srp_start_callback(const otSockAddr *server_socket_address, void *context); + static void srp_factory_reset_callback(otError err, const otSrpClientHostInfo *host_info, + const otSrpClientService *services, const otSrpClientService *removed_services, + void *context); protected: esphome::mdns::MDNSComponent *mdns_{nullptr}; - std::vector mdns_services_; std::vector> memory_pool_; void *pool_alloc_(size_t size); }; diff --git a/esphome/components/openthread/openthread_esp.cpp b/esphome/components/openthread/openthread_esp.cpp index dc303cef17..b11b7ad34a 100644 --- a/esphome/components/openthread/openthread_esp.cpp +++ b/esphome/components/openthread/openthread_esp.cpp @@ -28,7 +28,6 @@ namespace esphome { namespace openthread { void OpenThreadComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // Used eventfds: // * netif // * ot task queue @@ -144,10 +143,13 @@ void OpenThreadComponent::ot_main() { esp_openthread_launch_mainloop(); // Clean up + esp_openthread_deinit(); esp_openthread_netif_glue_deinit(); esp_netif_destroy(openthread_netif); esp_vfs_eventfd_unregister(); + this->teardown_complete_ = true; + vTaskDelete(NULL); } network::IPAddresses OpenThreadComponent::get_ip_addresses() { diff --git a/esphome/components/opt3001/opt3001.cpp b/esphome/components/opt3001/opt3001.cpp index 2d65f1090d..f5f7ab9412 100644 --- a/esphome/components/opt3001/opt3001.cpp +++ b/esphome/components/opt3001/opt3001.cpp @@ -72,7 +72,7 @@ void OPT3001Sensor::read_lx_(const std::function &f) { } this->set_timeout("read", OPT3001_CONVERSION_TIME_800, [this, f]() { - if (this->write(&OPT3001_REG_CONFIGURATION, 1, true) != i2c::ERROR_OK) { + if (this->write(&OPT3001_REG_CONFIGURATION, 1) != i2c::ERROR_OK) { ESP_LOGW(TAG, "Starting configuration register read failed"); f(NAN); return; diff --git a/esphome/components/ota/__init__.py b/esphome/components/ota/__init__.py index 4d5b8a61e2..eec39668db 100644 --- a/esphome/components/ota/__init__.py +++ b/esphome/components/ota/__init__.py @@ -11,6 +11,7 @@ from esphome.const import ( PlatformFramework, ) from esphome.core import CORE, coroutine_with_priority +from esphome.coroutine import CoroPriority CODEOWNERS = ["@esphome/core"] AUTO_LOAD = ["md5", "safe_mode"] @@ -82,7 +83,7 @@ BASE_OTA_SCHEMA = cv.Schema( ) -@coroutine_with_priority(54.0) +@coroutine_with_priority(CoroPriority.OTA_UPDATES) async def to_code(config): cg.add_define("USE_OTA") diff --git a/esphome/components/ota/ota_backend.h b/esphome/components/ota/ota_backend.h index 372f24df5e..64ee0b9f7c 100644 --- a/esphome/components/ota/ota_backend.h +++ b/esphome/components/ota/ota_backend.h @@ -14,6 +14,7 @@ namespace ota { enum OTAResponseTypes { OTA_RESPONSE_OK = 0x00, OTA_RESPONSE_REQUEST_AUTH = 0x01, + OTA_RESPONSE_REQUEST_SHA256_AUTH = 0x02, OTA_RESPONSE_HEADER_OK = 0x40, OTA_RESPONSE_AUTH_OK = 0x41, diff --git a/esphome/components/output/__init__.py b/esphome/components/output/__init__.py index 78bfa045e1..bde106b085 100644 --- a/esphome/components/output/__init__.py +++ b/esphome/components/output/__init__.py @@ -43,6 +43,8 @@ FloatOutputPtr = FloatOutput.operator("ptr") TurnOffAction = output_ns.class_("TurnOffAction", automation.Action) TurnOnAction = output_ns.class_("TurnOnAction", automation.Action) SetLevelAction = output_ns.class_("SetLevelAction", automation.Action) +SetMinPowerAction = output_ns.class_("SetMinPowerAction", automation.Action) +SetMaxPowerAction = output_ns.class_("SetMaxPowerAction", automation.Action) async def setup_output_platform_(obj, config): @@ -104,6 +106,42 @@ async def output_set_level_to_code(config, action_id, template_arg, args): return var +@automation.register_action( + "output.set_min_power", + SetMinPowerAction, + cv.Schema( + { + cv.Required(CONF_ID): cv.use_id(FloatOutput), + cv.Required(CONF_MIN_POWER): cv.templatable(cv.percentage), + } + ), +) +async def output_set_min_power_to_code(config, action_id, template_arg, args): + paren = await cg.get_variable(config[CONF_ID]) + var = cg.new_Pvariable(action_id, template_arg, paren) + template_ = await cg.templatable(config[CONF_MIN_POWER], args, float) + cg.add(var.set_min_power(template_)) + return var + + +@automation.register_action( + "output.set_max_power", + SetMaxPowerAction, + cv.Schema( + { + cv.Required(CONF_ID): cv.use_id(FloatOutput), + cv.Required(CONF_MAX_POWER): cv.templatable(cv.percentage), + } + ), +) +async def output_set_max_power_to_code(config, action_id, template_arg, args): + paren = await cg.get_variable(config[CONF_ID]) + var = cg.new_Pvariable(action_id, template_arg, paren) + template_ = await cg.templatable(config[CONF_MAX_POWER], args, float) + cg.add(var.set_max_power(template_)) + return var + + async def to_code(config): cg.add_define("USE_OUTPUT") cg.add_global(output_ns.using) diff --git a/esphome/components/output/automation.h b/esphome/components/output/automation.h index 51c2849702..de84bb91ca 100644 --- a/esphome/components/output/automation.h +++ b/esphome/components/output/automation.h @@ -40,5 +40,29 @@ template class SetLevelAction : public Action { FloatOutput *output_; }; +template class SetMinPowerAction : public Action { + public: + SetMinPowerAction(FloatOutput *output) : output_(output) {} + + TEMPLATABLE_VALUE(float, min_power) + + void play(Ts... x) override { this->output_->set_min_power(this->min_power_.value(x...)); } + + protected: + FloatOutput *output_; +}; + +template class SetMaxPowerAction : public Action { + public: + SetMaxPowerAction(FloatOutput *output) : output_(output) {} + + TEMPLATABLE_VALUE(float, max_power) + + void play(Ts... x) override { this->output_->set_max_power(this->max_power_.value(x...)); } + + protected: + FloatOutput *output_; +}; + } // namespace output } // namespace esphome diff --git a/esphome/components/output/switch/output_switch.cpp b/esphome/components/output/switch/output_switch.cpp index c30cfd3f56..54260ba37a 100644 --- a/esphome/components/output/switch/output_switch.cpp +++ b/esphome/components/output/switch/output_switch.cpp @@ -8,8 +8,6 @@ static const char *const TAG = "output.switch"; void OutputSwitch::dump_config() { LOG_SWITCH("", "Output Switch", this); } void OutputSwitch::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); - bool initial_state = this->get_initial_state_with_restore_mode().value_or(false); if (initial_state) { diff --git a/esphome/components/packages/__init__.py b/esphome/components/packages/__init__.py index 0db7841db2..fdc75d995a 100644 --- a/esphome/components/packages/__init__.py +++ b/esphome/components/packages/__init__.py @@ -106,11 +106,13 @@ CONFIG_SCHEMA = cv.Any( ) -def _process_base_package(config: dict) -> dict: +def _process_base_package(config: dict, skip_update: bool = False) -> dict: + # When skip_update is True, use NEVER_REFRESH to prevent updates + actual_refresh = git.NEVER_REFRESH if skip_update else config[CONF_REFRESH] repo_dir, revert = git.clone_or_update( url=config[CONF_URL], ref=config.get(CONF_REF), - refresh=config[CONF_REFRESH], + refresh=actual_refresh, domain=DOMAIN, username=config.get(CONF_USERNAME), password=config.get(CONF_PASSWORD), @@ -180,17 +182,16 @@ def _process_base_package(config: dict) -> dict: return {"packages": packages} -def _process_package(package_config, config): +def _process_package(package_config, config, skip_update: bool = False): recursive_package = package_config if CONF_URL in package_config: - package_config = _process_base_package(package_config) + package_config = _process_base_package(package_config, skip_update) if isinstance(package_config, dict): - recursive_package = do_packages_pass(package_config) - config = merge_config(recursive_package, config) - return config + recursive_package = do_packages_pass(package_config, skip_update) + return merge_config(recursive_package, config) -def do_packages_pass(config: dict): +def do_packages_pass(config: dict, skip_update: bool = False): if CONF_PACKAGES not in config: return config packages = config[CONF_PACKAGES] @@ -199,10 +200,10 @@ def do_packages_pass(config: dict): if isinstance(packages, dict): for package_name, package_config in reversed(packages.items()): with cv.prepend_path(package_name): - config = _process_package(package_config, config) + config = _process_package(package_config, config, skip_update) elif isinstance(packages, list): for package_config in reversed(packages): - config = _process_package(package_config, config) + config = _process_package(package_config, config, skip_update) else: raise cv.Invalid( f"Packages must be a key to value mapping or list, got {type(packages)} instead" diff --git a/esphome/components/packet_transport/__init__.py b/esphome/components/packet_transport/__init__.py index 99c1d824ca..43da7740fe 100644 --- a/esphome/components/packet_transport/__init__.py +++ b/esphome/components/packet_transport/__init__.py @@ -89,9 +89,10 @@ def validate_(config): raise cv.Invalid("No sensors or binary sensors to encrypt") elif config[CONF_ROLLING_CODE_ENABLE]: raise cv.Invalid("Rolling code requires an encryption key") - if config[CONF_PING_PONG_ENABLE]: - if not any(CONF_ENCRYPTION in p for p in config.get(CONF_PROVIDERS) or ()): - raise cv.Invalid("Ping-pong requires at least one encrypted provider") + if config[CONF_PING_PONG_ENABLE] and not any( + CONF_ENCRYPTION in p for p in config.get(CONF_PROVIDERS) or () + ): + raise cv.Invalid("Ping-pong requires at least one encrypted provider") return config @@ -120,15 +121,11 @@ def transport_schema(cls): return TRANSPORT_SCHEMA.extend({cv.GenerateID(): cv.declare_id(cls)}) -# Build a list of sensors for this platform -CORE.data[DOMAIN] = {CONF_SENSORS: []} - - def get_sensors(transport_id): """Return the list of sensors for this platform.""" return ( sensor - for sensor in CORE.data[DOMAIN][CONF_SENSORS] + for sensor in CORE.data.setdefault(DOMAIN, {}).setdefault(CONF_SENSORS, []) if sensor[CONF_TRANSPORT_ID] == transport_id ) @@ -136,7 +133,8 @@ def get_sensors(transport_id): def validate_packet_transport_sensor(config): if CONF_NAME in config and CONF_INTERNAL not in config: raise cv.Invalid("Must provide internal: config when using name:") - CORE.data[DOMAIN][CONF_SENSORS].append(config) + conf_sensors = CORE.data.setdefault(DOMAIN, {}).setdefault(CONF_SENSORS, []) + conf_sensors.append(config) return config diff --git a/esphome/components/packet_transport/packet_transport.cpp b/esphome/components/packet_transport/packet_transport.cpp index b6ce24bc1b..8bde4ee505 100644 --- a/esphome/components/packet_transport/packet_transport.cpp +++ b/esphome/components/packet_transport/packet_transport.cpp @@ -270,6 +270,7 @@ void PacketTransport::add_binary_data_(uint8_t key, const char *id, bool data) { auto len = 1 + 1 + 1 + strlen(id); if (len + this->header_.size() + this->data_.size() > this->get_max_packet_size()) { this->flush_(); + this->init_data_(); } add(this->data_, key); add(this->data_, (uint8_t) data); @@ -284,6 +285,7 @@ void PacketTransport::add_data_(uint8_t key, const char *id, uint32_t data) { auto len = 4 + 1 + 1 + strlen(id); if (len + this->header_.size() + this->data_.size() > this->get_max_packet_size()) { this->flush_(); + this->init_data_(); } add(this->data_, key); add(this->data_, data); diff --git a/esphome/components/pca6416a/__init__.py b/esphome/components/pca6416a/__init__.py index da6c4623c9..e540edb91f 100644 --- a/esphome/components/pca6416a/__init__.py +++ b/esphome/components/pca6416a/__init__.py @@ -14,6 +14,7 @@ from esphome.const import ( CODEOWNERS = ["@Mat931"] DEPENDENCIES = ["i2c"] +AUTO_LOAD = ["gpio_expander"] MULTI_CONF = True pca6416a_ns = cg.esphome_ns.namespace("pca6416a") diff --git a/esphome/components/pca6416a/pca6416a.cpp b/esphome/components/pca6416a/pca6416a.cpp index 3e76df5015..c0056e780b 100644 --- a/esphome/components/pca6416a/pca6416a.cpp +++ b/esphome/components/pca6416a/pca6416a.cpp @@ -24,7 +24,6 @@ enum PCA6416AGPIORegisters { static const char *const TAG = "pca6416a"; void PCA6416AComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // Test to see if device exists uint8_t value; if (!this->read_register_(PCA6416A_INPUT0, &value)) { @@ -34,7 +33,7 @@ void PCA6416AComponent::setup() { } // Test to see if the device supports pull-up resistors - if (this->read_register(PCAL6416A_PULL_EN0, &value, 1, true) == i2c::ERROR_OK) { + if (this->read_register(PCAL6416A_PULL_EN0, &value, 1) == i2c::ERROR_OK) { this->has_pullup_ = true; } @@ -52,6 +51,11 @@ void PCA6416AComponent::setup() { this->status_has_error()); } +void PCA6416AComponent::loop() { + // Invalidate cache at the start of each loop + this->reset_pin_cache_(); +} + void PCA6416AComponent::dump_config() { if (this->has_pullup_) { ESP_LOGCONFIG(TAG, "PCAL6416A:"); @@ -64,15 +68,25 @@ void PCA6416AComponent::dump_config() { } } -bool PCA6416AComponent::digital_read(uint8_t pin) { - uint8_t bit = pin % 8; +bool PCA6416AComponent::digital_read_hw(uint8_t pin) { uint8_t reg_addr = pin < 8 ? PCA6416A_INPUT0 : PCA6416A_INPUT1; uint8_t value = 0; - this->read_register_(reg_addr, &value); - return value & (1 << bit); + if (!this->read_register_(reg_addr, &value)) { + return false; + } + + // Update the appropriate part of input_mask_ + if (pin < 8) { + this->input_mask_ = (this->input_mask_ & 0xFF00) | value; + } else { + this->input_mask_ = (this->input_mask_ & 0x00FF) | (uint16_t(value) << 8); + } + return true; } -void PCA6416AComponent::digital_write(uint8_t pin, bool value) { +bool PCA6416AComponent::digital_read_cache(uint8_t pin) { return this->input_mask_ & (1 << pin); } + +void PCA6416AComponent::digital_write_hw(uint8_t pin, bool value) { uint8_t reg_addr = pin < 8 ? PCA6416A_OUTPUT0 : PCA6416A_OUTPUT1; this->update_register_(pin, value, reg_addr); } @@ -106,7 +120,7 @@ bool PCA6416AComponent::read_register_(uint8_t reg, uint8_t *value) { return false; } - this->last_error_ = this->read_register(reg, value, 1, true); + this->last_error_ = this->read_register(reg, value, 1); if (this->last_error_ != i2c::ERROR_OK) { this->status_set_warning(); ESP_LOGE(TAG, "read_register_(): I2C I/O error: %d", (int) this->last_error_); @@ -123,7 +137,7 @@ bool PCA6416AComponent::write_register_(uint8_t reg, uint8_t value) { return false; } - this->last_error_ = this->write_register(reg, &value, 1, true); + this->last_error_ = this->write_register(reg, &value, 1); if (this->last_error_ != i2c::ERROR_OK) { this->status_set_warning(); ESP_LOGE(TAG, "write_register_(): I2C I/O error: %d", (int) this->last_error_); diff --git a/esphome/components/pca6416a/pca6416a.h b/esphome/components/pca6416a/pca6416a.h index 1e8015c40a..10a4a64e9b 100644 --- a/esphome/components/pca6416a/pca6416a.h +++ b/esphome/components/pca6416a/pca6416a.h @@ -3,20 +3,20 @@ #include "esphome/core/component.h" #include "esphome/core/hal.h" #include "esphome/components/i2c/i2c.h" +#include "esphome/components/gpio_expander/cached_gpio.h" namespace esphome { namespace pca6416a { -class PCA6416AComponent : public Component, public i2c::I2CDevice { +class PCA6416AComponent : public Component, + public i2c::I2CDevice, + public gpio_expander::CachedGpioExpander { public: PCA6416AComponent() = default; /// Check i2c availability and setup masks void setup() override; - /// Helper function to read the value of a pin. - bool digital_read(uint8_t pin); - /// Helper function to write the value of a pin. - void digital_write(uint8_t pin, bool value); + void loop() override; /// Helper function to set the pin mode of a pin. void pin_mode(uint8_t pin, gpio::Flags flags); @@ -25,6 +25,11 @@ class PCA6416AComponent : public Component, public i2c::I2CDevice { void dump_config() override; protected: + // Virtual methods from CachedGpioExpander + bool digital_read_hw(uint8_t pin) override; + bool digital_read_cache(uint8_t pin) override; + void digital_write_hw(uint8_t pin, bool value) override; + bool read_register_(uint8_t reg, uint8_t *value); bool write_register_(uint8_t reg, uint8_t value); void update_register_(uint8_t pin, bool pin_value, uint8_t reg_addr); @@ -32,6 +37,8 @@ class PCA6416AComponent : public Component, public i2c::I2CDevice { /// The mask to write as output state - 1 means HIGH, 0 means LOW uint8_t output_0_{0x00}; uint8_t output_1_{0x00}; + /// Cache for input values (16-bit combined for both banks) + uint16_t input_mask_{0x00}; /// Storage for last I2C error seen esphome::i2c::ErrorCode last_error_; /// Only the PCAL6416A has pull-up resistors diff --git a/esphome/components/pca9554/__init__.py b/esphome/components/pca9554/__init__.py index 05713cccda..626b08a378 100644 --- a/esphome/components/pca9554/__init__.py +++ b/esphome/components/pca9554/__init__.py @@ -11,7 +11,8 @@ from esphome.const import ( CONF_OUTPUT, ) -CODEOWNERS = ["@hwstar", "@clydebarrow"] +CODEOWNERS = ["@hwstar", "@clydebarrow", "@bdraco"] +AUTO_LOAD = ["gpio_expander"] DEPENDENCIES = ["i2c"] MULTI_CONF = True CONF_PIN_COUNT = "pin_count" diff --git a/esphome/components/pca9554/pca9554.cpp b/esphome/components/pca9554/pca9554.cpp index 6b3f2d20af..e8d49f66e2 100644 --- a/esphome/components/pca9554/pca9554.cpp +++ b/esphome/components/pca9554/pca9554.cpp @@ -13,7 +13,6 @@ const uint8_t CONFIG_REG = 3; static const char *const TAG = "pca9554"; void PCA9554Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->reg_width_ = (this->pin_count_ + 7) / 8; // Test to see if device exists if (!this->read_inputs_()) { @@ -38,10 +37,9 @@ void PCA9554Component::setup() { } void PCA9554Component::loop() { - // The read_inputs_() method will cache the input values from the chip. - this->read_inputs_(); - // Clear all the previously read flags. - this->was_previously_read_ = 0x00; + // Invalidate the cache at the start of each loop. + // The actual read will happen on demand when digital_read() is called + this->reset_pin_cache_(); } void PCA9554Component::dump_config() { @@ -55,21 +53,17 @@ void PCA9554Component::dump_config() { } } -bool PCA9554Component::digital_read(uint8_t pin) { - // Note: We want to try and avoid doing any I2C bus read transactions here - // to conserve I2C bus bandwidth. So what we do is check to see if we - // have seen a read during the time esphome is running this loop. If we have, - // we do an I2C bus transaction to get the latest value. If we haven't - // we return a cached value which was read at the time loop() was called. - if (this->was_previously_read_ & (1 << pin)) - this->read_inputs_(); // Force a read of a new value - // Indicate we saw a read request for this pin in case a - // read happens later in the same loop. - this->was_previously_read_ |= (1 << pin); +bool PCA9554Component::digital_read_hw(uint8_t pin) { + // Read all pins from hardware into input_mask_ + return this->read_inputs_(); // Return true if I2C read succeeded, false on error +} + +bool PCA9554Component::digital_read_cache(uint8_t pin) { + // Return the cached pin state from input_mask_ return this->input_mask_ & (1 << pin); } -void PCA9554Component::digital_write(uint8_t pin, bool value) { +void PCA9554Component::digital_write_hw(uint8_t pin, bool value) { if (value) { this->output_mask_ |= (1 << pin); } else { @@ -97,7 +91,7 @@ bool PCA9554Component::read_inputs_() { return false; } - this->last_error_ = this->read_register(INPUT_REG * this->reg_width_, inputs, this->reg_width_, true); + this->last_error_ = this->read_register(INPUT_REG * this->reg_width_, inputs, this->reg_width_); if (this->last_error_ != i2c::ERROR_OK) { this->status_set_warning(); ESP_LOGE(TAG, "read_register_(): I2C I/O error: %d", (int) this->last_error_); @@ -115,7 +109,7 @@ bool PCA9554Component::write_register_(uint8_t reg, uint16_t value) { uint8_t outputs[2]; outputs[0] = (uint8_t) value; outputs[1] = (uint8_t) (value >> 8); - this->last_error_ = this->write_register(reg * this->reg_width_, outputs, this->reg_width_, true); + this->last_error_ = this->write_register(reg * this->reg_width_, outputs, this->reg_width_); if (this->last_error_ != i2c::ERROR_OK) { this->status_set_warning(); ESP_LOGE(TAG, "write_register_(): I2C I/O error: %d", (int) this->last_error_); @@ -128,8 +122,7 @@ bool PCA9554Component::write_register_(uint8_t reg, uint16_t value) { float PCA9554Component::get_setup_priority() const { return setup_priority::IO; } -// Run our loop() method very early in the loop, so that we cache read values before -// before other components call our digital_read() method. +// Run our loop() method early to invalidate cache before any other components access the pins float PCA9554Component::get_loop_priority() const { return 9.0f; } // Just after WIFI void PCA9554GPIOPin::setup() { pin_mode(flags_); } diff --git a/esphome/components/pca9554/pca9554.h b/esphome/components/pca9554/pca9554.h index efeec4d306..7b356b4068 100644 --- a/esphome/components/pca9554/pca9554.h +++ b/esphome/components/pca9554/pca9554.h @@ -3,22 +3,21 @@ #include "esphome/core/component.h" #include "esphome/core/hal.h" #include "esphome/components/i2c/i2c.h" +#include "esphome/components/gpio_expander/cached_gpio.h" namespace esphome { namespace pca9554 { -class PCA9554Component : public Component, public i2c::I2CDevice { +class PCA9554Component : public Component, + public i2c::I2CDevice, + public gpio_expander::CachedGpioExpander { public: PCA9554Component() = default; /// Check i2c availability and setup masks void setup() override; - /// Poll for input changes periodically + /// Invalidate cache at start of each loop void loop() override; - /// Helper function to read the value of a pin. - bool digital_read(uint8_t pin); - /// Helper function to write the value of a pin. - void digital_write(uint8_t pin, bool value); /// Helper function to set the pin mode of a pin. void pin_mode(uint8_t pin, gpio::Flags flags); @@ -32,9 +31,13 @@ class PCA9554Component : public Component, public i2c::I2CDevice { protected: bool read_inputs_(); - bool write_register_(uint8_t reg, uint16_t value); + // Virtual methods from CachedGpioExpander + bool digital_read_hw(uint8_t pin) override; + bool digital_read_cache(uint8_t pin) override; + void digital_write_hw(uint8_t pin, bool value) override; + /// number of bits the expander has size_t pin_count_{8}; /// width of registers @@ -45,8 +48,6 @@ class PCA9554Component : public Component, public i2c::I2CDevice { uint16_t output_mask_{0x00}; /// The state of the actual input pin states - 1 means HIGH, 0 means LOW uint16_t input_mask_{0x00}; - /// Flags to check if read previously during this loop - uint16_t was_previously_read_ = {0x00}; /// Storage for last I2C error seen esphome::i2c::ErrorCode last_error_; }; diff --git a/esphome/components/pca9685/pca9685_output.cpp b/esphome/components/pca9685/pca9685_output.cpp index 2fe22fd1cc..6df708ac84 100644 --- a/esphome/components/pca9685/pca9685_output.cpp +++ b/esphome/components/pca9685/pca9685_output.cpp @@ -26,8 +26,6 @@ static const uint8_t PCA9685_MODE1_AUTOINC = 0b00100000; static const uint8_t PCA9685_MODE1_SLEEP = 0b00010000; void PCA9685Output::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - ESP_LOGV(TAG, " Resetting devices"); if (!this->write_bytes(PCA9685_REGISTER_SOFTWARE_RESET, nullptr, 0)) { this->mark_failed(); diff --git a/esphome/components/pcf85063/pcf85063.cpp b/esphome/components/pcf85063/pcf85063.cpp index d58d35019b..cb987c6129 100644 --- a/esphome/components/pcf85063/pcf85063.cpp +++ b/esphome/components/pcf85063/pcf85063.cpp @@ -10,7 +10,6 @@ namespace pcf85063 { static const char *const TAG = "pcf85063"; void PCF85063Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (!this->read_rtc_()) { this->mark_failed(); } diff --git a/esphome/components/pcf8563/pcf8563.cpp b/esphome/components/pcf8563/pcf8563.cpp index 7dd7a6fea8..27020378a6 100644 --- a/esphome/components/pcf8563/pcf8563.cpp +++ b/esphome/components/pcf8563/pcf8563.cpp @@ -10,7 +10,6 @@ namespace pcf8563 { static const char *const TAG = "PCF8563"; void PCF8563Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (!this->read_rtc_()) { this->mark_failed(); } diff --git a/esphome/components/pcf8574/__init__.py b/esphome/components/pcf8574/__init__.py index ff7c314bcd..f387d0a610 100644 --- a/esphome/components/pcf8574/__init__.py +++ b/esphome/components/pcf8574/__init__.py @@ -11,6 +11,7 @@ from esphome.const import ( CONF_OUTPUT, ) +AUTO_LOAD = ["gpio_expander"] DEPENDENCIES = ["i2c"] MULTI_CONF = True diff --git a/esphome/components/pcf8574/pcf8574.cpp b/esphome/components/pcf8574/pcf8574.cpp index dbab0319d7..72d8865d7f 100644 --- a/esphome/components/pcf8574/pcf8574.cpp +++ b/esphome/components/pcf8574/pcf8574.cpp @@ -7,7 +7,6 @@ namespace pcf8574 { static const char *const TAG = "pcf8574"; void PCF8574Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (!this->read_gpio_()) { ESP_LOGE(TAG, "PCF8574 not available under 0x%02X", this->address_); this->mark_failed(); @@ -17,6 +16,10 @@ void PCF8574Component::setup() { this->write_gpio_(); this->read_gpio_(); } +void PCF8574Component::loop() { + // Invalidate the cache at the start of each loop + this->reset_pin_cache_(); +} void PCF8574Component::dump_config() { ESP_LOGCONFIG(TAG, "PCF8574:"); LOG_I2C_DEVICE(this) @@ -25,17 +28,19 @@ void PCF8574Component::dump_config() { ESP_LOGE(TAG, ESP_LOG_MSG_COMM_FAIL); } } -bool PCF8574Component::digital_read(uint8_t pin) { - this->read_gpio_(); - return this->input_mask_ & (1 << pin); +bool PCF8574Component::digital_read_hw(uint8_t pin) { + // Read all pins from hardware into input_mask_ + return this->read_gpio_(); // Return true if I2C read succeeded, false on error } -void PCF8574Component::digital_write(uint8_t pin, bool value) { + +bool PCF8574Component::digital_read_cache(uint8_t pin) { return this->input_mask_ & (1 << pin); } + +void PCF8574Component::digital_write_hw(uint8_t pin, bool value) { if (value) { this->output_mask_ |= (1 << pin); } else { this->output_mask_ &= ~(1 << pin); } - this->write_gpio_(); } void PCF8574Component::pin_mode(uint8_t pin, gpio::Flags flags) { @@ -92,6 +97,9 @@ bool PCF8574Component::write_gpio_() { } float PCF8574Component::get_setup_priority() const { return setup_priority::IO; } +// Run our loop() method early to invalidate cache before any other components access the pins +float PCF8574Component::get_loop_priority() const { return 9.0f; } // Just after WIFI + void PCF8574GPIOPin::setup() { pin_mode(flags_); } void PCF8574GPIOPin::pin_mode(gpio::Flags flags) { this->parent_->pin_mode(this->pin_, flags); } bool PCF8574GPIOPin::digital_read() { return this->parent_->digital_read(this->pin_) != this->inverted_; } diff --git a/esphome/components/pcf8574/pcf8574.h b/esphome/components/pcf8574/pcf8574.h index 6edc67fc96..fd1ea8af63 100644 --- a/esphome/components/pcf8574/pcf8574.h +++ b/esphome/components/pcf8574/pcf8574.h @@ -3,11 +3,16 @@ #include "esphome/core/component.h" #include "esphome/core/hal.h" #include "esphome/components/i2c/i2c.h" +#include "esphome/components/gpio_expander/cached_gpio.h" namespace esphome { namespace pcf8574 { -class PCF8574Component : public Component, public i2c::I2CDevice { +// PCF8574(8 pins)/PCF8575(16 pins) always read/write all pins in a single I2C transaction +// so we use uint16_t as bank type to ensure all pins are in one bank and cached together +class PCF8574Component : public Component, + public i2c::I2CDevice, + public gpio_expander::CachedGpioExpander { public: PCF8574Component() = default; @@ -15,20 +20,22 @@ class PCF8574Component : public Component, public i2c::I2CDevice { /// Check i2c availability and setup masks void setup() override; - /// Helper function to read the value of a pin. - bool digital_read(uint8_t pin); - /// Helper function to write the value of a pin. - void digital_write(uint8_t pin, bool value); + /// Invalidate cache at start of each loop + void loop() override; /// Helper function to set the pin mode of a pin. void pin_mode(uint8_t pin, gpio::Flags flags); float get_setup_priority() const override; + float get_loop_priority() const override; void dump_config() override; protected: - bool read_gpio_(); + bool digital_read_hw(uint8_t pin) override; + bool digital_read_cache(uint8_t pin) override; + void digital_write_hw(uint8_t pin, bool value) override; + bool read_gpio_(); bool write_gpio_(); /// Mask for the pin mode - 1 means output, 0 means input diff --git a/esphome/components/pi4ioe5v6408/pi4ioe5v6408.cpp b/esphome/components/pi4ioe5v6408/pi4ioe5v6408.cpp index 55b8edffc8..517ca833e6 100644 --- a/esphome/components/pi4ioe5v6408/pi4ioe5v6408.cpp +++ b/esphome/components/pi4ioe5v6408/pi4ioe5v6408.cpp @@ -18,7 +18,6 @@ static const uint8_t PI4IOE5V6408_REGISTER_INTERRUPT_STATUS = 0x13; static const char *const TAG = "pi4ioe5v6408"; void PI4IOE5V6408Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (this->reset_) { this->reg(PI4IOE5V6408_REGISTER_DEVICE_ID) |= 0b00000001; this->reg(PI4IOE5V6408_REGISTER_OUT_HIGH_IMPEDENCE) = 0b00000000; @@ -69,7 +68,7 @@ bool PI4IOE5V6408Component::read_gpio_outputs_() { uint8_t data; if (!this->read_byte(PI4IOE5V6408_REGISTER_OUT_SET, &data)) { - this->status_set_warning("Failed to read output register"); + this->status_set_warning(LOG_STR("Failed to read output register")); return false; } this->output_mask_ = data; @@ -83,7 +82,7 @@ bool PI4IOE5V6408Component::read_gpio_modes_() { uint8_t data; if (!this->read_byte(PI4IOE5V6408_REGISTER_IO_DIR, &data)) { - this->status_set_warning("Failed to read GPIO modes"); + this->status_set_warning(LOG_STR("Failed to read GPIO modes")); return false; } #if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE @@ -100,7 +99,7 @@ bool PI4IOE5V6408Component::digital_read_hw(uint8_t pin) { uint8_t data; if (!this->read_byte(PI4IOE5V6408_REGISTER_IN_STATE, &data)) { - this->status_set_warning("Failed to read GPIO state"); + this->status_set_warning(LOG_STR("Failed to read GPIO state")); return false; } this->input_mask_ = data; @@ -118,7 +117,7 @@ void PI4IOE5V6408Component::digital_write_hw(uint8_t pin, bool value) { this->output_mask_ &= ~(1 << pin); } if (!this->write_byte(PI4IOE5V6408_REGISTER_OUT_SET, this->output_mask_)) { - this->status_set_warning("Failed to write output register"); + this->status_set_warning(LOG_STR("Failed to write output register")); return; } #if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE @@ -132,15 +131,15 @@ bool PI4IOE5V6408Component::write_gpio_modes_() { return false; if (!this->write_byte(PI4IOE5V6408_REGISTER_IO_DIR, this->mode_mask_)) { - this->status_set_warning("Failed to write GPIO modes"); + this->status_set_warning(LOG_STR("Failed to write GPIO modes")); return false; } if (!this->write_byte(PI4IOE5V6408_REGISTER_PULL_SELECT, this->pull_up_down_mask_)) { - this->status_set_warning("Failed to write GPIO pullup/pulldown"); + this->status_set_warning(LOG_STR("Failed to write GPIO pullup/pulldown")); return false; } if (!this->write_byte(PI4IOE5V6408_REGISTER_PULL_ENABLE, this->pull_enable_mask_)) { - this->status_set_warning("Failed to write GPIO pull enable"); + this->status_set_warning(LOG_STR("Failed to write GPIO pull enable")); return false; } #if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE diff --git a/esphome/components/pid/pid_controller.cpp b/esphome/components/pid/pid_controller.cpp index 1a16f14542..5d7aecdb05 100644 --- a/esphome/components/pid/pid_controller.cpp +++ b/esphome/components/pid/pid_controller.cpp @@ -104,7 +104,7 @@ float PIDController::weighted_average_(std::deque &list, float new_value, list.push_front(new_value); // keep only 'samples' readings, by popping off the back of the list - while (list.size() > samples) + while (samples > 0 && list.size() > static_cast(samples)) list.pop_back(); // calculate and return the average of all values in the list diff --git a/esphome/components/pipsolar/__init__.py b/esphome/components/pipsolar/__init__.py index 1e4ea8492b..e3966aa2cc 100644 --- a/esphome/components/pipsolar/__init__.py +++ b/esphome/components/pipsolar/__init__.py @@ -26,7 +26,7 @@ CONFIG_SCHEMA = cv.All( ) -def to_code(config): +async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) - yield cg.register_component(var, config) - yield uart.register_uart_device(var, config) + await cg.register_component(var, config) + await uart.register_uart_device(var, config) diff --git a/esphome/components/pipsolar/output/__init__.py b/esphome/components/pipsolar/output/__init__.py index 1eb7249119..829f8f7037 100644 --- a/esphome/components/pipsolar/output/__init__.py +++ b/esphome/components/pipsolar/output/__init__.py @@ -99,9 +99,9 @@ async def to_code(config): } ), ) -def output_pipsolar_set_level_to_code(config, action_id, template_arg, args): - paren = yield cg.get_variable(config[CONF_ID]) +async def output_pipsolar_set_level_to_code(config, action_id, template_arg, args): + paren = await cg.get_variable(config[CONF_ID]) var = cg.new_Pvariable(action_id, template_arg, paren) - template_ = yield cg.templatable(config[CONF_VALUE], args, float) + template_ = await cg.templatable(config[CONF_VALUE], args, float) cg.add(var.set_level(template_)) - yield var + return var diff --git a/esphome/components/pipsolar/pipsolar.cpp b/esphome/components/pipsolar/pipsolar.cpp index 40405114a4..5751ad59f5 100644 --- a/esphome/components/pipsolar/pipsolar.cpp +++ b/esphome/components/pipsolar/pipsolar.cpp @@ -23,20 +23,18 @@ void Pipsolar::loop() { // Read message if (this->state_ == STATE_IDLE) { this->empty_uart_buffer_(); - switch (this->send_next_command_()) { - case 0: - // no command send (empty queue) time to poll - if (millis() - this->last_poll_ > this->update_interval_) { - this->send_next_poll_(); - this->last_poll_ = millis(); - } - return; - break; - case 1: - // command send - return; - break; + + if (this->send_next_command_()) { + // command sent + return; } + + if (this->send_next_poll_()) { + // poll sent + return; + } + + return; } if (this->state_ == STATE_COMMAND_COMPLETE) { if (this->check_incoming_length_(4)) { @@ -530,7 +528,7 @@ void Pipsolar::loop() { // '(00000000000000000000000000000000' // iterate over all available flag (as not all models have all flags, but at least in the same order) this->value_warnings_present_ = false; - this->value_faults_present_ = true; + this->value_faults_present_ = false; for (size_t i = 1; i < strlen(tmp); i++) { enabled = tmp[i] == '1'; @@ -708,6 +706,7 @@ void Pipsolar::loop() { return; } // crc ok + this->used_polling_commands_[this->last_polling_command_].needs_update = false; this->state_ = STATE_POLL_CHECKED; return; } else { @@ -788,7 +787,7 @@ uint8_t Pipsolar::check_incoming_crc_() { } // send next command used -uint8_t Pipsolar::send_next_command_() { +bool Pipsolar::send_next_command_() { uint16_t crc16; if (!this->command_queue_[this->command_queue_position_].empty()) { const char *command = this->command_queue_[this->command_queue_position_].c_str(); @@ -809,37 +808,43 @@ uint8_t Pipsolar::send_next_command_() { // end Byte this->write(0x0D); ESP_LOGD(TAG, "Sending command from queue: %s with length %d", command, length); - return 1; + return true; } - return 0; + return false; } -void Pipsolar::send_next_poll_() { +bool Pipsolar::send_next_poll_() { uint16_t crc16; - this->last_polling_command_ = (this->last_polling_command_ + 1) % 15; - if (this->used_polling_commands_[this->last_polling_command_].length == 0) { - this->last_polling_command_ = 0; + + for (uint8_t i = 0; i < POLLING_COMMANDS_MAX; i++) { + this->last_polling_command_ = (this->last_polling_command_ + 1) % POLLING_COMMANDS_MAX; + if (this->used_polling_commands_[this->last_polling_command_].length == 0) { + // not enabled + continue; + } + if (!this->used_polling_commands_[this->last_polling_command_].needs_update) { + // no update requested + continue; + } + this->state_ = STATE_POLL; + this->command_start_millis_ = millis(); + this->empty_uart_buffer_(); + this->read_pos_ = 0; + crc16 = this->pipsolar_crc_(this->used_polling_commands_[this->last_polling_command_].command, + this->used_polling_commands_[this->last_polling_command_].length); + this->write_array(this->used_polling_commands_[this->last_polling_command_].command, + this->used_polling_commands_[this->last_polling_command_].length); + // checksum + this->write(((uint8_t) ((crc16) >> 8))); // highbyte + this->write(((uint8_t) ((crc16) &0xff))); // lowbyte + // end Byte + this->write(0x0D); + ESP_LOGD(TAG, "Sending polling command : %s with length %d", + this->used_polling_commands_[this->last_polling_command_].command, + this->used_polling_commands_[this->last_polling_command_].length); + return true; } - if (this->used_polling_commands_[this->last_polling_command_].length == 0) { - // no command specified - return; - } - this->state_ = STATE_POLL; - this->command_start_millis_ = millis(); - this->empty_uart_buffer_(); - this->read_pos_ = 0; - crc16 = this->pipsolar_crc_(this->used_polling_commands_[this->last_polling_command_].command, - this->used_polling_commands_[this->last_polling_command_].length); - this->write_array(this->used_polling_commands_[this->last_polling_command_].command, - this->used_polling_commands_[this->last_polling_command_].length); - // checksum - this->write(((uint8_t) ((crc16) >> 8))); // highbyte - this->write(((uint8_t) ((crc16) &0xff))); // lowbyte - // end Byte - this->write(0x0D); - ESP_LOGD(TAG, "Sending polling command : %s with length %d", - this->used_polling_commands_[this->last_polling_command_].command, - this->used_polling_commands_[this->last_polling_command_].length); + return false; } void Pipsolar::queue_command_(const char *command, uint8_t length) { @@ -869,7 +874,13 @@ void Pipsolar::dump_config() { } } } -void Pipsolar::update() {} +void Pipsolar::update() { + for (auto &used_polling_command : this->used_polling_commands_) { + if (used_polling_command.length != 0) { + used_polling_command.needs_update = true; + } + } +} void Pipsolar::add_polling_command_(const char *command, ENUMPollingCommand polling_command) { for (auto &used_polling_command : this->used_polling_commands_) { @@ -891,6 +902,7 @@ void Pipsolar::add_polling_command_(const char *command, ENUMPollingCommand poll used_polling_command.errors = 0; used_polling_command.identifier = polling_command; used_polling_command.length = length - 1; + used_polling_command.needs_update = true; return; } } diff --git a/esphome/components/pipsolar/pipsolar.h b/esphome/components/pipsolar/pipsolar.h index 373911b2d7..77b18badb9 100644 --- a/esphome/components/pipsolar/pipsolar.h +++ b/esphome/components/pipsolar/pipsolar.h @@ -25,6 +25,7 @@ struct PollingCommand { uint8_t length = 0; uint8_t errors; ENUMPollingCommand identifier; + bool needs_update; }; #define PIPSOLAR_VALUED_ENTITY_(type, name, polling_command, value_type) \ @@ -189,14 +190,14 @@ class Pipsolar : public uart::UARTDevice, public PollingComponent { static const size_t PIPSOLAR_READ_BUFFER_LENGTH = 110; // maximum supported answer length static const size_t COMMAND_QUEUE_LENGTH = 10; static const size_t COMMAND_TIMEOUT = 5000; - uint32_t last_poll_ = 0; + static const size_t POLLING_COMMANDS_MAX = 15; void add_polling_command_(const char *command, ENUMPollingCommand polling_command); void empty_uart_buffer_(); uint8_t check_incoming_crc_(); uint8_t check_incoming_length_(uint8_t length); uint16_t pipsolar_crc_(uint8_t *msg, uint8_t len); - uint8_t send_next_command_(); - void send_next_poll_(); + bool send_next_command_(); + bool send_next_poll_(); void queue_command_(const char *command, uint8_t length); std::string command_queue_[COMMAND_QUEUE_LENGTH]; uint8_t command_queue_position_ = 0; @@ -216,7 +217,7 @@ class Pipsolar : public uart::UARTDevice, public PollingComponent { }; uint8_t last_polling_command_ = 0; - PollingCommand used_polling_commands_[15]; + PollingCommand used_polling_commands_[POLLING_COMMANDS_MAX]; }; } // namespace pipsolar diff --git a/esphome/components/pipsolar/sensor/__init__.py b/esphome/components/pipsolar/sensor/__init__.py index 0d00ba0083..929865b480 100644 --- a/esphome/components/pipsolar/sensor/__init__.py +++ b/esphome/components/pipsolar/sensor/__init__.py @@ -273,7 +273,7 @@ CONFIG_SCHEMA = PIPSOLAR_COMPONENT_SCHEMA.extend( async def to_code(config): paren = await cg.get_variable(config[CONF_PIPSOLAR_ID]) - for type, _ in TYPES.items(): + for type in TYPES: if type in config: conf = config[type] sens = await sensor.new_sensor(conf) diff --git a/esphome/components/pm2005/pm2005.cpp b/esphome/components/pm2005/pm2005.cpp index 57c616c4c6..d8e253a771 100644 --- a/esphome/components/pm2005/pm2005.cpp +++ b/esphome/components/pm2005/pm2005.cpp @@ -39,7 +39,6 @@ static const LogString *pm2005_get_measuring_mode_string(int status) { static inline uint16_t get_sensor_value(const uint8_t *data, uint8_t i) { return data[i] * 0x100 + data[i + 1]; } void PM2005Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (this->sensor_type_ == PM2005) { this->situation_value_index_ = 3; this->pm_1_0_value_index_ = 4; diff --git a/esphome/components/pmsa003i/pmsa003i.cpp b/esphome/components/pmsa003i/pmsa003i.cpp index 4702c0cf5f..4a618586f8 100644 --- a/esphome/components/pmsa003i/pmsa003i.cpp +++ b/esphome/components/pmsa003i/pmsa003i.cpp @@ -19,8 +19,6 @@ static const uint8_t START_CHARACTER_2 = 0x4D; static const uint8_t READ_DATA_RETRY_COUNT = 3; void PMSA003IComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - PM25AQIData data; bool successful_read = this->read_data_(&data); diff --git a/esphome/components/pmwcs3/sensor.py b/esphome/components/pmwcs3/sensor.py index d42338ab6f..075b9b00b5 100644 --- a/esphome/components/pmwcs3/sensor.py +++ b/esphome/components/pmwcs3/sensor.py @@ -114,8 +114,7 @@ PMWCS3_CALIBRATION_SCHEMA = cv.Schema( ) async def pmwcs3_calibration_to_code(config, action_id, template_arg, args): parent = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, parent) - return var + return cg.new_Pvariable(action_id, template_arg, parent) PMWCS3_NEW_I2C_ADDRESS_SCHEMA = cv.maybe_simple_value( diff --git a/esphome/components/pn532/pn532.cpp b/esphome/components/pn532/pn532.cpp index da5598bf10..ef4022db4b 100644 --- a/esphome/components/pn532/pn532.cpp +++ b/esphome/components/pn532/pn532.cpp @@ -15,8 +15,6 @@ namespace pn532 { static const char *const TAG = "pn532"; void PN532::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // Get version data if (!this->write_command_({PN532_COMMAND_VERSION_DATA})) { ESP_LOGW(TAG, "Error sending version command, trying again"); diff --git a/esphome/components/pn532_spi/pn532_spi.cpp b/esphome/components/pn532_spi/pn532_spi.cpp index 2e66d4ed83..0871f7acab 100644 --- a/esphome/components/pn532_spi/pn532_spi.cpp +++ b/esphome/components/pn532_spi/pn532_spi.cpp @@ -12,12 +12,10 @@ namespace pn532_spi { static const char *const TAG = "pn532_spi"; void PN532Spi::setup() { - ESP_LOGI(TAG, "PN532Spi setup started!"); this->spi_setup(); this->cs_->digital_write(false); delay(10); - ESP_LOGI(TAG, "SPI setup finished!"); PN532::setup(); } diff --git a/esphome/components/power_supply/power_supply.cpp b/esphome/components/power_supply/power_supply.cpp index 6fbadc73ae..5db2122412 100644 --- a/esphome/components/power_supply/power_supply.cpp +++ b/esphome/components/power_supply/power_supply.cpp @@ -7,22 +7,19 @@ namespace power_supply { static const char *const TAG = "power_supply"; void PowerSupply::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->pin_->setup(); this->pin_->digital_write(false); if (this->enable_on_boot_) this->request_high_power(); } void PowerSupply::dump_config() { - ESP_LOGCONFIG(TAG, "Power Supply:"); - LOG_PIN(" Pin: ", this->pin_); ESP_LOGCONFIG(TAG, + "Power Supply:\n" " Time to enable: %" PRIu32 " ms\n" - " Keep on time: %.1f s", - this->enable_time_, this->keep_on_time_ / 1000.0f); - if (this->enable_on_boot_) - ESP_LOGCONFIG(TAG, " Enabled at startup: True"); + " Keep on time: %" PRIu32 " s\n" + " Enable at startup: %s", + this->enable_time_, this->keep_on_time_ / 1000u, YESNO(this->enable_on_boot_)); + LOG_PIN(" Pin: ", this->pin_); } float PowerSupply::get_setup_priority() const { return setup_priority::IO; } @@ -32,7 +29,7 @@ bool PowerSupply::is_enabled() const { return this->active_requests_ != 0; } void PowerSupply::request_high_power() { if (this->active_requests_ == 0) { this->cancel_timeout("power-supply-off"); - ESP_LOGD(TAG, "Enabling power supply."); + ESP_LOGV(TAG, "Enabling"); this->pin_->digital_write(true); delay(this->enable_time_); } @@ -47,7 +44,7 @@ void PowerSupply::unrequest_high_power() { this->active_requests_--; if (this->active_requests_ == 0) { this->set_timeout("power-supply-off", this->keep_on_time_, [this]() { - ESP_LOGD(TAG, "Disabling power supply."); + ESP_LOGV(TAG, "Disabling"); this->pin_->digital_write(false); }); } diff --git a/esphome/components/power_supply/power_supply.h b/esphome/components/power_supply/power_supply.h index 3959f6f299..0387074eb8 100644 --- a/esphome/components/power_supply/power_supply.h +++ b/esphome/components/power_supply/power_supply.h @@ -36,10 +36,10 @@ class PowerSupply : public Component { protected: GPIOPin *pin_; - bool enable_on_boot_{false}; uint32_t enable_time_; uint32_t keep_on_time_; int16_t active_requests_{0}; // use signed integer to make catching negative requests easier. + bool enable_on_boot_{false}; }; class PowerSupplyRequester { diff --git a/esphome/components/prometheus/prometheus_handler.cpp b/esphome/components/prometheus/prometheus_handler.cpp index 2677860c7c..6e7ed6f79f 100644 --- a/esphome/components/prometheus/prometheus_handler.cpp +++ b/esphome/components/prometheus/prometheus_handler.cpp @@ -110,21 +110,21 @@ std::string PrometheusHandler::relabel_name_(EntityBase *obj) { void PrometheusHandler::add_area_label_(AsyncResponseStream *stream, std::string &area) { if (!area.empty()) { - stream->print(F("\",area=\"")); + stream->print(ESPHOME_F("\",area=\"")); stream->print(area.c_str()); } } void PrometheusHandler::add_node_label_(AsyncResponseStream *stream, std::string &node) { if (!node.empty()) { - stream->print(F("\",node=\"")); + stream->print(ESPHOME_F("\",node=\"")); stream->print(node.c_str()); } } void PrometheusHandler::add_friendly_name_label_(AsyncResponseStream *stream, std::string &friendly_name) { if (!friendly_name.empty()) { - stream->print(F("\",friendly_name=\"")); + stream->print(ESPHOME_F("\",friendly_name=\"")); stream->print(friendly_name.c_str()); } } @@ -132,8 +132,8 @@ void PrometheusHandler::add_friendly_name_label_(AsyncResponseStream *stream, st // Type-specific implementation #ifdef USE_SENSOR void PrometheusHandler::sensor_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_sensor_value gauge\n")); - stream->print(F("#TYPE esphome_sensor_failed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_sensor_value gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_sensor_failed gauge\n")); } void PrometheusHandler::sensor_row_(AsyncResponseStream *stream, sensor::Sensor *obj, std::string &area, std::string &node, std::string &friendly_name) { @@ -141,37 +141,37 @@ void PrometheusHandler::sensor_row_(AsyncResponseStream *stream, sensor::Sensor return; if (!std::isnan(obj->state)) { // We have a valid value, output this value - stream->print(F("esphome_sensor_failed{id=\"")); + stream->print(ESPHOME_F("esphome_sensor_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 0\n")); + stream->print(ESPHOME_F("\"} 0\n")); // Data itself - stream->print(F("esphome_sensor_value{id=\"")); + stream->print(ESPHOME_F("esphome_sensor_value{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",unit=\"")); + stream->print(ESPHOME_F("\",unit=\"")); stream->print(obj->get_unit_of_measurement().c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(value_accuracy_to_string(obj->state, obj->get_accuracy_decimals()).c_str()); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); } else { // Invalid state - stream->print(F("esphome_sensor_failed{id=\"")); + stream->print(ESPHOME_F("esphome_sensor_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 1\n")); + stream->print(ESPHOME_F("\"} 1\n")); } } #endif @@ -179,8 +179,8 @@ void PrometheusHandler::sensor_row_(AsyncResponseStream *stream, sensor::Sensor // Type-specific implementation #ifdef USE_BINARY_SENSOR void PrometheusHandler::binary_sensor_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_binary_sensor_value gauge\n")); - stream->print(F("#TYPE esphome_binary_sensor_failed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_binary_sensor_value gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_binary_sensor_failed gauge\n")); } void PrometheusHandler::binary_sensor_row_(AsyncResponseStream *stream, binary_sensor::BinarySensor *obj, std::string &area, std::string &node, std::string &friendly_name) { @@ -188,204 +188,204 @@ void PrometheusHandler::binary_sensor_row_(AsyncResponseStream *stream, binary_s return; if (obj->has_state()) { // We have a valid value, output this value - stream->print(F("esphome_binary_sensor_failed{id=\"")); + stream->print(ESPHOME_F("esphome_binary_sensor_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 0\n")); + stream->print(ESPHOME_F("\"} 0\n")); // Data itself - stream->print(F("esphome_binary_sensor_value{id=\"")); + stream->print(ESPHOME_F("esphome_binary_sensor_value{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(obj->state); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); } else { // Invalid state - stream->print(F("esphome_binary_sensor_failed{id=\"")); + stream->print(ESPHOME_F("esphome_binary_sensor_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 1\n")); + stream->print(ESPHOME_F("\"} 1\n")); } } #endif #ifdef USE_FAN void PrometheusHandler::fan_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_fan_value gauge\n")); - stream->print(F("#TYPE esphome_fan_failed gauge\n")); - stream->print(F("#TYPE esphome_fan_speed gauge\n")); - stream->print(F("#TYPE esphome_fan_oscillation gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_fan_value gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_fan_failed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_fan_speed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_fan_oscillation gauge\n")); } void PrometheusHandler::fan_row_(AsyncResponseStream *stream, fan::Fan *obj, std::string &area, std::string &node, std::string &friendly_name) { if (obj->is_internal() && !this->include_internal_) return; - stream->print(F("esphome_fan_failed{id=\"")); + stream->print(ESPHOME_F("esphome_fan_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 0\n")); + stream->print(ESPHOME_F("\"} 0\n")); // Data itself - stream->print(F("esphome_fan_value{id=\"")); + stream->print(ESPHOME_F("esphome_fan_value{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(obj->state); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); // Speed if available if (obj->get_traits().supports_speed()) { - stream->print(F("esphome_fan_speed{id=\"")); + stream->print(ESPHOME_F("esphome_fan_speed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(obj->speed); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); } // Oscillation if available if (obj->get_traits().supports_oscillation()) { - stream->print(F("esphome_fan_oscillation{id=\"")); + stream->print(ESPHOME_F("esphome_fan_oscillation{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(obj->oscillating); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); } } #endif #ifdef USE_LIGHT void PrometheusHandler::light_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_light_state gauge\n")); - stream->print(F("#TYPE esphome_light_color gauge\n")); - stream->print(F("#TYPE esphome_light_effect_active gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_light_state gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_light_color gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_light_effect_active gauge\n")); } void PrometheusHandler::light_row_(AsyncResponseStream *stream, light::LightState *obj, std::string &area, std::string &node, std::string &friendly_name) { if (obj->is_internal() && !this->include_internal_) return; // State - stream->print(F("esphome_light_state{id=\"")); + stream->print(ESPHOME_F("esphome_light_state{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(obj->remote_values.is_on()); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); // Brightness and RGBW light::LightColorValues color = obj->current_values; float brightness, r, g, b, w; color.as_brightness(&brightness); color.as_rgbw(&r, &g, &b, &w); - stream->print(F("esphome_light_color{id=\"")); + stream->print(ESPHOME_F("esphome_light_color{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",channel=\"brightness\"} ")); + stream->print(ESPHOME_F("\",channel=\"brightness\"} ")); stream->print(brightness); - stream->print(F("\n")); - stream->print(F("esphome_light_color{id=\"")); + stream->print(ESPHOME_F("\n")); + stream->print(ESPHOME_F("esphome_light_color{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",channel=\"r\"} ")); + stream->print(ESPHOME_F("\",channel=\"r\"} ")); stream->print(r); - stream->print(F("\n")); - stream->print(F("esphome_light_color{id=\"")); + stream->print(ESPHOME_F("\n")); + stream->print(ESPHOME_F("esphome_light_color{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",channel=\"g\"} ")); + stream->print(ESPHOME_F("\",channel=\"g\"} ")); stream->print(g); - stream->print(F("\n")); - stream->print(F("esphome_light_color{id=\"")); + stream->print(ESPHOME_F("\n")); + stream->print(ESPHOME_F("esphome_light_color{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",channel=\"b\"} ")); + stream->print(ESPHOME_F("\",channel=\"b\"} ")); stream->print(b); - stream->print(F("\n")); - stream->print(F("esphome_light_color{id=\"")); + stream->print(ESPHOME_F("\n")); + stream->print(ESPHOME_F("esphome_light_color{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",channel=\"w\"} ")); + stream->print(ESPHOME_F("\",channel=\"w\"} ")); stream->print(w); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); // Effect std::string effect = obj->get_effect_name(); if (effect == "None") { - stream->print(F("esphome_light_effect_active{id=\"")); + stream->print(ESPHOME_F("esphome_light_effect_active{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",effect=\"None\"} 0\n")); + stream->print(ESPHOME_F("\",effect=\"None\"} 0\n")); } else { - stream->print(F("esphome_light_effect_active{id=\"")); + stream->print(ESPHOME_F("esphome_light_effect_active{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",effect=\"")); + stream->print(ESPHOME_F("\",effect=\"")); stream->print(effect.c_str()); - stream->print(F("\"} 1\n")); + stream->print(ESPHOME_F("\"} 1\n")); } } #endif #ifdef USE_COVER void PrometheusHandler::cover_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_cover_value gauge\n")); - stream->print(F("#TYPE esphome_cover_failed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_cover_value gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_cover_failed gauge\n")); } void PrometheusHandler::cover_row_(AsyncResponseStream *stream, cover::Cover *obj, std::string &area, std::string &node, std::string &friendly_name) { @@ -393,118 +393,118 @@ void PrometheusHandler::cover_row_(AsyncResponseStream *stream, cover::Cover *ob return; if (!std::isnan(obj->position)) { // We have a valid value, output this value - stream->print(F("esphome_cover_failed{id=\"")); + stream->print(ESPHOME_F("esphome_cover_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 0\n")); + stream->print(ESPHOME_F("\"} 0\n")); // Data itself - stream->print(F("esphome_cover_value{id=\"")); + stream->print(ESPHOME_F("esphome_cover_value{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(obj->position); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); if (obj->get_traits().get_supports_tilt()) { - stream->print(F("esphome_cover_tilt{id=\"")); + stream->print(ESPHOME_F("esphome_cover_tilt{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(obj->tilt); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); } } else { // Invalid state - stream->print(F("esphome_cover_failed{id=\"")); + stream->print(ESPHOME_F("esphome_cover_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 1\n")); + stream->print(ESPHOME_F("\"} 1\n")); } } #endif #ifdef USE_SWITCH void PrometheusHandler::switch_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_switch_value gauge\n")); - stream->print(F("#TYPE esphome_switch_failed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_switch_value gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_switch_failed gauge\n")); } void PrometheusHandler::switch_row_(AsyncResponseStream *stream, switch_::Switch *obj, std::string &area, std::string &node, std::string &friendly_name) { if (obj->is_internal() && !this->include_internal_) return; - stream->print(F("esphome_switch_failed{id=\"")); + stream->print(ESPHOME_F("esphome_switch_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 0\n")); + stream->print(ESPHOME_F("\"} 0\n")); // Data itself - stream->print(F("esphome_switch_value{id=\"")); + stream->print(ESPHOME_F("esphome_switch_value{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(obj->state); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); } #endif #ifdef USE_LOCK void PrometheusHandler::lock_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_lock_value gauge\n")); - stream->print(F("#TYPE esphome_lock_failed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_lock_value gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_lock_failed gauge\n")); } void PrometheusHandler::lock_row_(AsyncResponseStream *stream, lock::Lock *obj, std::string &area, std::string &node, std::string &friendly_name) { if (obj->is_internal() && !this->include_internal_) return; - stream->print(F("esphome_lock_failed{id=\"")); + stream->print(ESPHOME_F("esphome_lock_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 0\n")); + stream->print(ESPHOME_F("\"} 0\n")); // Data itself - stream->print(F("esphome_lock_value{id=\"")); + stream->print(ESPHOME_F("esphome_lock_value{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(obj->state); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); } #endif // Type-specific implementation #ifdef USE_TEXT_SENSOR void PrometheusHandler::text_sensor_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_text_sensor_value gauge\n")); - stream->print(F("#TYPE esphome_text_sensor_failed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_text_sensor_value gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_text_sensor_failed gauge\n")); } void PrometheusHandler::text_sensor_row_(AsyncResponseStream *stream, text_sensor::TextSensor *obj, std::string &area, std::string &node, std::string &friendly_name) { @@ -512,37 +512,37 @@ void PrometheusHandler::text_sensor_row_(AsyncResponseStream *stream, text_senso return; if (obj->has_state()) { // We have a valid value, output this value - stream->print(F("esphome_text_sensor_failed{id=\"")); + stream->print(ESPHOME_F("esphome_text_sensor_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 0\n")); + stream->print(ESPHOME_F("\"} 0\n")); // Data itself - stream->print(F("esphome_text_sensor_value{id=\"")); + stream->print(ESPHOME_F("esphome_text_sensor_value{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",value=\"")); + stream->print(ESPHOME_F("\",value=\"")); stream->print(obj->state.c_str()); - stream->print(F("\"} ")); - stream->print(F("1.0")); - stream->print(F("\n")); + stream->print(ESPHOME_F("\"} ")); + stream->print(ESPHOME_F("1.0")); + stream->print(ESPHOME_F("\n")); } else { // Invalid state - stream->print(F("esphome_text_sensor_failed{id=\"")); + stream->print(ESPHOME_F("esphome_text_sensor_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 1\n")); + stream->print(ESPHOME_F("\"} 1\n")); } } #endif @@ -550,8 +550,8 @@ void PrometheusHandler::text_sensor_row_(AsyncResponseStream *stream, text_senso // Type-specific implementation #ifdef USE_NUMBER void PrometheusHandler::number_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_number_value gauge\n")); - stream->print(F("#TYPE esphome_number_failed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_number_value gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_number_failed gauge\n")); } void PrometheusHandler::number_row_(AsyncResponseStream *stream, number::Number *obj, std::string &area, std::string &node, std::string &friendly_name) { @@ -559,43 +559,43 @@ void PrometheusHandler::number_row_(AsyncResponseStream *stream, number::Number return; if (!std::isnan(obj->state)) { // We have a valid value, output this value - stream->print(F("esphome_number_failed{id=\"")); + stream->print(ESPHOME_F("esphome_number_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 0\n")); + stream->print(ESPHOME_F("\"} 0\n")); // Data itself - stream->print(F("esphome_number_value{id=\"")); + stream->print(ESPHOME_F("esphome_number_value{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(obj->state); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); } else { // Invalid state - stream->print(F("esphome_number_failed{id=\"")); + stream->print(ESPHOME_F("esphome_number_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 1\n")); + stream->print(ESPHOME_F("\"} 1\n")); } } #endif #ifdef USE_SELECT void PrometheusHandler::select_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_select_value gauge\n")); - stream->print(F("#TYPE esphome_select_failed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_select_value gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_select_failed gauge\n")); } void PrometheusHandler::select_row_(AsyncResponseStream *stream, select::Select *obj, std::string &area, std::string &node, std::string &friendly_name) { @@ -603,105 +603,105 @@ void PrometheusHandler::select_row_(AsyncResponseStream *stream, select::Select return; if (obj->has_state()) { // We have a valid value, output this value - stream->print(F("esphome_select_failed{id=\"")); + stream->print(ESPHOME_F("esphome_select_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 0\n")); + stream->print(ESPHOME_F("\"} 0\n")); // Data itself - stream->print(F("esphome_select_value{id=\"")); + stream->print(ESPHOME_F("esphome_select_value{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",value=\"")); + stream->print(ESPHOME_F("\",value=\"")); stream->print(obj->state.c_str()); - stream->print(F("\"} ")); - stream->print(F("1.0")); - stream->print(F("\n")); + stream->print(ESPHOME_F("\"} ")); + stream->print(ESPHOME_F("1.0")); + stream->print(ESPHOME_F("\n")); } else { // Invalid state - stream->print(F("esphome_select_failed{id=\"")); + stream->print(ESPHOME_F("esphome_select_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 1\n")); + stream->print(ESPHOME_F("\"} 1\n")); } } #endif #ifdef USE_MEDIA_PLAYER void PrometheusHandler::media_player_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_media_player_state_value gauge\n")); - stream->print(F("#TYPE esphome_media_player_volume gauge\n")); - stream->print(F("#TYPE esphome_media_player_is_muted gauge\n")); - stream->print(F("#TYPE esphome_media_player_failed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_media_player_state_value gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_media_player_volume gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_media_player_is_muted gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_media_player_failed gauge\n")); } void PrometheusHandler::media_player_row_(AsyncResponseStream *stream, media_player::MediaPlayer *obj, std::string &area, std::string &node, std::string &friendly_name) { if (obj->is_internal() && !this->include_internal_) return; - stream->print(F("esphome_media_player_failed{id=\"")); + stream->print(ESPHOME_F("esphome_media_player_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 0\n")); + stream->print(ESPHOME_F("\"} 0\n")); // Data itself - stream->print(F("esphome_media_player_state_value{id=\"")); + stream->print(ESPHOME_F("esphome_media_player_state_value{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",value=\"")); + stream->print(ESPHOME_F("\",value=\"")); stream->print(media_player::media_player_state_to_string(obj->state)); - stream->print(F("\"} ")); - stream->print(F("1.0")); - stream->print(F("\n")); - stream->print(F("esphome_media_player_volume{id=\"")); + stream->print(ESPHOME_F("\"} ")); + stream->print(ESPHOME_F("1.0")); + stream->print(ESPHOME_F("\n")); + stream->print(ESPHOME_F("esphome_media_player_volume{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(obj->volume); - stream->print(F("\n")); - stream->print(F("esphome_media_player_is_muted{id=\"")); + stream->print(ESPHOME_F("\n")); + stream->print(ESPHOME_F("esphome_media_player_is_muted{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); if (obj->is_muted()) { - stream->print(F("1.0")); + stream->print(ESPHOME_F("1.0")); } else { - stream->print(F("0.0")); + stream->print(ESPHOME_F("0.0")); } - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); } #endif #ifdef USE_UPDATE void PrometheusHandler::update_entity_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_update_entity_state gauge\n")); - stream->print(F("#TYPE esphome_update_entity_info gauge\n")); - stream->print(F("#TYPE esphome_update_entity_failed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_update_entity_state gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_update_entity_info gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_update_entity_failed gauge\n")); } void PrometheusHandler::handle_update_state_(AsyncResponseStream *stream, update::UpdateState state) { @@ -730,168 +730,168 @@ void PrometheusHandler::update_entity_row_(AsyncResponseStream *stream, update:: return; if (obj->has_state()) { // We have a valid value, output this value - stream->print(F("esphome_update_entity_failed{id=\"")); + stream->print(ESPHOME_F("esphome_update_entity_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 0\n")); + stream->print(ESPHOME_F("\"} 0\n")); // First update state - stream->print(F("esphome_update_entity_state{id=\"")); + stream->print(ESPHOME_F("esphome_update_entity_state{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",value=\"")); + stream->print(ESPHOME_F("\",value=\"")); handle_update_state_(stream, obj->state); - stream->print(F("\"} ")); - stream->print(F("1.0")); - stream->print(F("\n")); + stream->print(ESPHOME_F("\"} ")); + stream->print(ESPHOME_F("1.0")); + stream->print(ESPHOME_F("\n")); // Next update info - stream->print(F("esphome_update_entity_info{id=\"")); + stream->print(ESPHOME_F("esphome_update_entity_info{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",current_version=\"")); + stream->print(ESPHOME_F("\",current_version=\"")); stream->print(obj->update_info.current_version.c_str()); - stream->print(F("\",latest_version=\"")); + stream->print(ESPHOME_F("\",latest_version=\"")); stream->print(obj->update_info.latest_version.c_str()); - stream->print(F("\",title=\"")); + stream->print(ESPHOME_F("\",title=\"")); stream->print(obj->update_info.title.c_str()); - stream->print(F("\"} ")); - stream->print(F("1.0")); - stream->print(F("\n")); + stream->print(ESPHOME_F("\"} ")); + stream->print(ESPHOME_F("1.0")); + stream->print(ESPHOME_F("\n")); } else { // Invalid state - stream->print(F("esphome_update_entity_failed{id=\"")); + stream->print(ESPHOME_F("esphome_update_entity_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 1\n")); + stream->print(ESPHOME_F("\"} 1\n")); } } #endif #ifdef USE_VALVE void PrometheusHandler::valve_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_valve_operation gauge\n")); - stream->print(F("#TYPE esphome_valve_failed gauge\n")); - stream->print(F("#TYPE esphome_valve_position gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_valve_operation gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_valve_failed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_valve_position gauge\n")); } void PrometheusHandler::valve_row_(AsyncResponseStream *stream, valve::Valve *obj, std::string &area, std::string &node, std::string &friendly_name) { if (obj->is_internal() && !this->include_internal_) return; - stream->print(F("esphome_valve_failed{id=\"")); + stream->print(ESPHOME_F("esphome_valve_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} 0\n")); + stream->print(ESPHOME_F("\"} 0\n")); // Data itself - stream->print(F("esphome_valve_operation{id=\"")); + stream->print(ESPHOME_F("esphome_valve_operation{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",operation=\"")); + stream->print(ESPHOME_F("\",operation=\"")); stream->print(valve::valve_operation_to_str(obj->current_operation)); - stream->print(F("\"} ")); - stream->print(F("1.0")); - stream->print(F("\n")); + stream->print(ESPHOME_F("\"} ")); + stream->print(ESPHOME_F("1.0")); + stream->print(ESPHOME_F("\n")); // Now see if position is supported if (obj->get_traits().get_supports_position()) { - stream->print(F("esphome_valve_position{id=\"")); + stream->print(ESPHOME_F("esphome_valve_position{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(obj->position); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); } } #endif #ifdef USE_CLIMATE void PrometheusHandler::climate_type_(AsyncResponseStream *stream) { - stream->print(F("#TYPE esphome_climate_setting gauge\n")); - stream->print(F("#TYPE esphome_climate_value gauge\n")); - stream->print(F("#TYPE esphome_climate_failed gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_climate_setting gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_climate_value gauge\n")); + stream->print(ESPHOME_F("#TYPE esphome_climate_failed gauge\n")); } void PrometheusHandler::climate_setting_row_(AsyncResponseStream *stream, climate::Climate *obj, std::string &area, std::string &node, std::string &friendly_name, std::string &setting, const LogString *setting_value) { - stream->print(F("esphome_climate_setting{id=\"")); + stream->print(ESPHOME_F("esphome_climate_setting{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",category=\"")); + stream->print(ESPHOME_F("\",category=\"")); stream->print(setting.c_str()); - stream->print(F("\",setting_value=\"")); + stream->print(ESPHOME_F("\",setting_value=\"")); stream->print(LOG_STR_ARG(setting_value)); - stream->print(F("\"} ")); - stream->print(F("1.0")); - stream->print(F("\n")); + stream->print(ESPHOME_F("\"} ")); + stream->print(ESPHOME_F("1.0")); + stream->print(ESPHOME_F("\n")); } void PrometheusHandler::climate_value_row_(AsyncResponseStream *stream, climate::Climate *obj, std::string &area, std::string &node, std::string &friendly_name, std::string &category, std::string &climate_value) { - stream->print(F("esphome_climate_value{id=\"")); + stream->print(ESPHOME_F("esphome_climate_value{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",category=\"")); + stream->print(ESPHOME_F("\",category=\"")); stream->print(category.c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); stream->print(climate_value.c_str()); - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); } void PrometheusHandler::climate_failed_row_(AsyncResponseStream *stream, climate::Climate *obj, std::string &area, std::string &node, std::string &friendly_name, std::string &category, bool is_failed_value) { - stream->print(F("esphome_climate_failed{id=\"")); + stream->print(ESPHOME_F("esphome_climate_failed{id=\"")); stream->print(relabel_id_(obj).c_str()); add_area_label_(stream, area); add_node_label_(stream, node); add_friendly_name_label_(stream, friendly_name); - stream->print(F("\",name=\"")); + stream->print(ESPHOME_F("\",name=\"")); stream->print(relabel_name_(obj).c_str()); - stream->print(F("\",category=\"")); + stream->print(ESPHOME_F("\",category=\"")); stream->print(category.c_str()); - stream->print(F("\"} ")); + stream->print(ESPHOME_F("\"} ")); if (is_failed_value) { - stream->print(F("1.0")); + stream->print(ESPHOME_F("1.0")); } else { - stream->print(F("0.0")); + stream->print(ESPHOME_F("0.0")); } - stream->print(F("\n")); + stream->print(ESPHOME_F("\n")); } void PrometheusHandler::climate_row_(AsyncResponseStream *stream, climate::Climate *obj, std::string &area, @@ -916,7 +916,7 @@ void PrometheusHandler::climate_row_(AsyncResponseStream *stream, climate::Clima auto min_temp_value = value_accuracy_to_string(traits.get_visual_min_temperature(), target_accuracy); climate_value_row_(stream, obj, area, node, friendly_name, min_temp, min_temp_value); // now check optional traits - if (traits.get_supports_current_temperature()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_TEMPERATURE)) { std::string current_temp = "current_temperature"; if (std::isnan(obj->current_temperature)) { climate_failed_row_(stream, obj, area, node, friendly_name, current_temp, true); @@ -927,7 +927,7 @@ void PrometheusHandler::climate_row_(AsyncResponseStream *stream, climate::Clima climate_failed_row_(stream, obj, area, node, friendly_name, current_temp, false); } } - if (traits.get_supports_current_humidity()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_HUMIDITY)) { std::string current_humidity = "current_humidity"; if (std::isnan(obj->current_humidity)) { climate_failed_row_(stream, obj, area, node, friendly_name, current_humidity, true); @@ -938,7 +938,7 @@ void PrometheusHandler::climate_row_(AsyncResponseStream *stream, climate::Clima climate_failed_row_(stream, obj, area, node, friendly_name, current_humidity, false); } } - if (traits.get_supports_target_humidity()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TARGET_HUMIDITY)) { std::string target_humidity = "target_humidity"; if (std::isnan(obj->target_humidity)) { climate_failed_row_(stream, obj, area, node, friendly_name, target_humidity, true); @@ -949,7 +949,8 @@ void PrometheusHandler::climate_row_(AsyncResponseStream *stream, climate::Clima climate_failed_row_(stream, obj, area, node, friendly_name, target_humidity, false); } } - if (traits.get_supports_two_point_target_temperature()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | + climate::CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE)) { std::string target_temp_low = "target_temperature_low"; auto target_temp_low_value = value_accuracy_to_string(obj->target_temperature_low, target_accuracy); climate_value_row_(stream, obj, area, node, friendly_name, target_temp_low, target_temp_low_value); @@ -961,7 +962,7 @@ void PrometheusHandler::climate_row_(AsyncResponseStream *stream, climate::Clima auto target_temp_value = value_accuracy_to_string(obj->target_temperature, target_accuracy); climate_value_row_(stream, obj, area, node, friendly_name, target_temp, target_temp_value); } - if (traits.get_supports_action()) { + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_ACTION)) { std::string climate_trait_category = "action"; const auto *climate_trait_value = climate::climate_action_to_string(obj->action); climate_setting_row_(stream, obj, area, node, friendly_name, climate_trait_category, climate_trait_value); diff --git a/esphome/components/psram/__init__.py b/esphome/components/psram/__init__.py index 9299cdcd0e..8e4f9d7eac 100644 --- a/esphome/components/psram/__init__.py +++ b/esphome/components/psram/__init__.py @@ -16,6 +16,7 @@ from esphome.components.esp32.const import ( import esphome.config_validation as cv from esphome.const import ( CONF_ADVANCED, + CONF_DISABLED, CONF_FRAMEWORK, CONF_ID, CONF_MODE, @@ -28,12 +29,13 @@ from esphome.core import CORE import esphome.final_validate as fv CODEOWNERS = ["@esphome/core"] +DOMAIN = "psram" DEPENDENCIES = [PLATFORM_ESP32] _LOGGER = logging.getLogger(__name__) -psram_ns = cg.esphome_ns.namespace("psram") +psram_ns = cg.esphome_ns.namespace(DOMAIN) PsramComponent = psram_ns.class_("PsramComponent", cg.Component) TYPE_QUAD = "quad" @@ -60,6 +62,13 @@ SPIRAM_SPEEDS = { } +def supported() -> bool: + if not CORE.is_esp32: + return False + variant = get_esp32_variant() + return variant in SPIRAM_MODES + + def validate_psram_mode(config): esp32_config = fv.full_config.get()[PLATFORM_ESP32] if config[CONF_SPEED] == "120MHZ": @@ -93,7 +102,7 @@ def get_config_schema(config): variant = get_esp32_variant() speeds = [f"{s}MHZ" for s in SPIRAM_SPEEDS.get(variant, [])] if not speeds: - return cv.Invalid("PSRAM is not supported on this chip") + raise cv.Invalid("PSRAM is not supported on this chip") modes = SPIRAM_MODES[variant] return cv.Schema( { @@ -101,6 +110,7 @@ def get_config_schema(config): cv.Optional(CONF_MODE, default=modes[0]): cv.one_of(*modes, lower=True), cv.Optional(CONF_ENABLE_ECC, default=False): cv.boolean, cv.Optional(CONF_SPEED, default=speeds[0]): cv.one_of(*speeds, upper=True), + cv.Optional(CONF_DISABLED, default=False): cv.boolean, } )(config) @@ -111,38 +121,37 @@ FINAL_VALIDATE_SCHEMA = validate_psram_mode async def to_code(config): + if config[CONF_DISABLED]: + return if CORE.using_arduino: cg.add_build_flag("-DBOARD_HAS_PSRAM") if config[CONF_MODE] == TYPE_OCTAL: cg.add_platformio_option("board_build.arduino.memory_type", "qio_opi") - if CORE.using_esp_idf: - add_idf_sdkconfig_option( - f"CONFIG_{get_esp32_variant().upper()}_SPIRAM_SUPPORT", True - ) - add_idf_sdkconfig_option("CONFIG_SOC_SPIRAM_SUPPORTED", True) - add_idf_sdkconfig_option("CONFIG_SPIRAM", True) - add_idf_sdkconfig_option("CONFIG_SPIRAM_USE", True) - add_idf_sdkconfig_option("CONFIG_SPIRAM_USE_CAPS_ALLOC", True) - add_idf_sdkconfig_option("CONFIG_SPIRAM_IGNORE_NOTFOUND", True) + add_idf_sdkconfig_option( + f"CONFIG_{get_esp32_variant().upper()}_SPIRAM_SUPPORT", True + ) + add_idf_sdkconfig_option("CONFIG_SOC_SPIRAM_SUPPORTED", True) + add_idf_sdkconfig_option("CONFIG_SPIRAM", True) + add_idf_sdkconfig_option("CONFIG_SPIRAM_USE", True) + add_idf_sdkconfig_option("CONFIG_SPIRAM_USE_CAPS_ALLOC", True) + add_idf_sdkconfig_option("CONFIG_SPIRAM_IGNORE_NOTFOUND", True) - add_idf_sdkconfig_option( - f"CONFIG_SPIRAM_MODE_{SDK_MODES[config[CONF_MODE]]}", True - ) + add_idf_sdkconfig_option(f"CONFIG_SPIRAM_MODE_{SDK_MODES[config[CONF_MODE]]}", True) - # Remove MHz suffix, convert to int - speed = int(config[CONF_SPEED][:-3]) - add_idf_sdkconfig_option(f"CONFIG_SPIRAM_SPEED_{speed}M", True) - add_idf_sdkconfig_option("CONFIG_SPIRAM_SPEED", speed) - if config[CONF_MODE] == TYPE_OCTAL and speed == 120: - add_idf_sdkconfig_option("CONFIG_ESPTOOLPY_FLASHFREQ_120M", True) - add_idf_sdkconfig_option("CONFIG_BOOTLOADER_FLASH_DC_AWARE", True) - if CORE.data[KEY_CORE][KEY_FRAMEWORK_VERSION] >= cv.Version(5, 4, 0): - add_idf_sdkconfig_option( - "CONFIG_SPIRAM_TIMING_TUNING_POINT_VIA_TEMPERATURE_SENSOR", True - ) - if config[CONF_ENABLE_ECC]: - add_idf_sdkconfig_option("CONFIG_SPIRAM_ECC_ENABLE", True) + # Remove MHz suffix, convert to int + speed = int(config[CONF_SPEED][:-3]) + add_idf_sdkconfig_option(f"CONFIG_SPIRAM_SPEED_{speed}M", True) + add_idf_sdkconfig_option("CONFIG_SPIRAM_SPEED", speed) + if config[CONF_MODE] == TYPE_OCTAL and speed == 120: + add_idf_sdkconfig_option("CONFIG_ESPTOOLPY_FLASHFREQ_120M", True) + add_idf_sdkconfig_option("CONFIG_BOOTLOADER_FLASH_DC_AWARE", True) + if CORE.data[KEY_CORE][KEY_FRAMEWORK_VERSION] >= cv.Version(5, 4, 0): + add_idf_sdkconfig_option( + "CONFIG_SPIRAM_TIMING_TUNING_POINT_VIA_TEMPERATURE_SENSOR", True + ) + if config[CONF_ENABLE_ECC]: + add_idf_sdkconfig_option("CONFIG_SPIRAM_ECC_ENABLE", True) cg.add_define("USE_PSRAM") diff --git a/esphome/components/pulse_counter/pulse_counter_sensor.cpp b/esphome/components/pulse_counter/pulse_counter_sensor.cpp index bfca0c6a4e..6300d6fe96 100644 --- a/esphome/components/pulse_counter/pulse_counter_sensor.cpp +++ b/esphome/components/pulse_counter/pulse_counter_sensor.cpp @@ -156,7 +156,6 @@ pulse_counter_t HwPulseCounterStorage::read_raw_value() { #endif // HAS_PCNT void PulseCounterSensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); if (!this->storage_.pulse_counter_setup(this->pin_)) { this->mark_failed(); return; diff --git a/esphome/components/pulse_counter/sensor.py b/esphome/components/pulse_counter/sensor.py index b330758000..dbf67fd2ad 100644 --- a/esphome/components/pulse_counter/sensor.py +++ b/esphome/components/pulse_counter/sensor.py @@ -49,12 +49,15 @@ def validate_internal_filter(value): [CONF_USE_PCNT], ) - if CORE.is_esp32 and use_pcnt: - if value.get(CONF_INTERNAL_FILTER).total_microseconds > 13: - raise cv.Invalid( - "Maximum internal filter value when using ESP32 hardware PCNT is 13us", - [CONF_INTERNAL_FILTER], - ) + if ( + CORE.is_esp32 + and use_pcnt + and value.get(CONF_INTERNAL_FILTER).total_microseconds > 13 + ): + raise cv.Invalid( + "Maximum internal filter value when using ESP32 hardware PCNT is 13us", + [CONF_INTERNAL_FILTER], + ) return value diff --git a/esphome/components/pulse_width/pulse_width.cpp b/esphome/components/pulse_width/pulse_width.cpp index 8d66861049..d083d48b32 100644 --- a/esphome/components/pulse_width/pulse_width.cpp +++ b/esphome/components/pulse_width/pulse_width.cpp @@ -17,8 +17,8 @@ void IRAM_ATTR PulseWidthSensorStore::gpio_intr(PulseWidthSensorStore *arg) { } void PulseWidthSensor::dump_config() { - LOG_SENSOR("", "Pulse Width", this) - LOG_UPDATE_INTERVAL(this) + LOG_SENSOR("", "Pulse Width", this); + LOG_UPDATE_INTERVAL(this); LOG_PIN(" Pin: ", this->pin_); } void PulseWidthSensor::update() { diff --git a/esphome/components/pvvx_mithermometer/display/pvvx_display.cpp b/esphome/components/pvvx_mithermometer/display/pvvx_display.cpp index 4b6c11b332..b6916ad68f 100644 --- a/esphome/components/pvvx_mithermometer/display/pvvx_display.cpp +++ b/esphome/components/pvvx_mithermometer/display/pvvx_display.cpp @@ -46,10 +46,32 @@ void PVVXDisplay::gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t } this->connection_established_ = true; this->char_handle_ = chr->handle; -#ifdef USE_TIME - this->sync_time_(); -#endif - this->display(); + + // Attempt to write immediately + // For devices without security, this will work + // For devices with security that are already paired, this will work + // For devices that need pairing, the write will be retried after auth completes + this->sync_time_and_display_(); + break; + } + default: + break; + } +} + +void PVVXDisplay::gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param) { + switch (event) { + case ESP_GAP_BLE_AUTH_CMPL_EVT: { + if (!this->parent_->check_addr(param->ble_security.auth_cmpl.bd_addr)) + return; + + if (param->ble_security.auth_cmpl.success) { + ESP_LOGD(TAG, "[%s] Authentication successful, performing writes.", this->parent_->address_str().c_str()); + // Now that pairing is complete, perform the pending writes + this->sync_time_and_display_(); + } else { + ESP_LOGW(TAG, "[%s] Authentication failed.", this->parent_->address_str().c_str()); + } break; } default: @@ -127,6 +149,13 @@ void PVVXDisplay::delayed_disconnect_() { this->set_timeout("disconnect", this->disconnect_delay_ms_, [this]() { this->parent_->set_enabled(false); }); } +void PVVXDisplay::sync_time_and_display_() { +#ifdef USE_TIME + this->sync_time_(); +#endif + this->display(); +} + #ifdef USE_TIME void PVVXDisplay::sync_time_() { if (this->time_ == nullptr) diff --git a/esphome/components/pvvx_mithermometer/display/pvvx_display.h b/esphome/components/pvvx_mithermometer/display/pvvx_display.h index 9739362024..c7fc523420 100644 --- a/esphome/components/pvvx_mithermometer/display/pvvx_display.h +++ b/esphome/components/pvvx_mithermometer/display/pvvx_display.h @@ -43,6 +43,7 @@ class PVVXDisplay : public ble_client::BLEClientNode, public PollingComponent { void gattc_event_handler(esp_gattc_cb_event_t event, esp_gatt_if_t gattc_if, esp_ble_gattc_cb_param_t *param) override; + void gap_event_handler(esp_gap_ble_cb_event_t event, esp_ble_gap_cb_param_t *param) override; /// Set validity period of the display information in seconds (1..65535) void set_validity_period(uint16_t validity_period) { this->validity_period_ = validity_period; } @@ -112,6 +113,7 @@ class PVVXDisplay : public ble_client::BLEClientNode, public PollingComponent { void setcfgbit_(uint8_t bit, bool value); void send_to_setup_char_(uint8_t *blk, size_t size); void delayed_disconnect_(); + void sync_time_and_display_(); #ifdef USE_TIME void sync_time_(); time::RealTimeClock *time_{nullptr}; diff --git a/esphome/components/pylontech/pylontech.cpp b/esphome/components/pylontech/pylontech.cpp index ef3de069ca..74b7caefb2 100644 --- a/esphome/components/pylontech/pylontech.cpp +++ b/esphome/components/pylontech/pylontech.cpp @@ -26,7 +26,6 @@ void PylontechComponent::dump_config() { } void PylontechComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); while (this->available() != 0) { this->read(); } diff --git a/esphome/components/qmc5883l/qmc5883l.cpp b/esphome/components/qmc5883l/qmc5883l.cpp index e41d7de644..d2041a2d52 100644 --- a/esphome/components/qmc5883l/qmc5883l.cpp +++ b/esphome/components/qmc5883l/qmc5883l.cpp @@ -8,6 +8,7 @@ namespace esphome { namespace qmc5883l { static const char *const TAG = "qmc5883l"; + static const uint8_t QMC5883L_ADDRESS = 0x0D; static const uint8_t QMC5883L_REGISTER_DATA_X_LSB = 0x00; @@ -24,7 +25,6 @@ static const uint8_t QMC5883L_REGISTER_CONTROL_2 = 0x0A; static const uint8_t QMC5883L_REGISTER_PERIOD = 0x0B; void QMC5883LComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // Soft Reset if (!this->write_byte(QMC5883L_REGISTER_CONTROL_2, 1 << 7)) { this->error_code_ = COMMUNICATION_FAILED; @@ -33,6 +33,10 @@ void QMC5883LComponent::setup() { } delay(10); + if (this->drdy_pin_) { + this->drdy_pin_->setup(); + } + uint8_t control_1 = 0; control_1 |= 0b01 << 0; // MODE (Mode) -> 0b00=standby, 0b01=continuous control_1 |= this->datarate_ << 2; @@ -65,6 +69,7 @@ void QMC5883LComponent::setup() { high_freq_.start(); } } + void QMC5883LComponent::dump_config() { ESP_LOGCONFIG(TAG, "QMC5883L:"); LOG_I2C_DEVICE(this); @@ -78,11 +83,20 @@ void QMC5883LComponent::dump_config() { LOG_SENSOR(" ", "Z Axis", this->z_sensor_); LOG_SENSOR(" ", "Heading", this->heading_sensor_); LOG_SENSOR(" ", "Temperature", this->temperature_sensor_); + LOG_PIN(" DRDY Pin: ", this->drdy_pin_); } + float QMC5883LComponent::get_setup_priority() const { return setup_priority::DATA; } + void QMC5883LComponent::update() { i2c::ErrorCode err; uint8_t status = false; + + // If DRDY pin is configured and the data is not ready return. + if (this->drdy_pin_ && !this->drdy_pin_->digital_read()) { + return; + } + // Status byte gets cleared when data is read, so we have to read this first. // If status and two axes are desired, it's possible to save one byte of traffic by enabling // ROL_PNT in setup and reading 7 bytes starting at the status register. diff --git a/esphome/components/qmc5883l/qmc5883l.h b/esphome/components/qmc5883l/qmc5883l.h index 3202e37780..5ba7180e23 100644 --- a/esphome/components/qmc5883l/qmc5883l.h +++ b/esphome/components/qmc5883l/qmc5883l.h @@ -3,6 +3,7 @@ #include "esphome/core/component.h" #include "esphome/components/sensor/sensor.h" #include "esphome/components/i2c/i2c.h" +#include "esphome/core/hal.h" namespace esphome { namespace qmc5883l { @@ -33,6 +34,7 @@ class QMC5883LComponent : public PollingComponent, public i2c::I2CDevice { float get_setup_priority() const override; void update() override; + void set_drdy_pin(GPIOPin *pin) { drdy_pin_ = pin; } void set_datarate(QMC5883LDatarate datarate) { datarate_ = datarate; } void set_range(QMC5883LRange range) { range_ = range; } void set_oversampling(QMC5883LOversampling oversampling) { oversampling_ = oversampling; } @@ -51,6 +53,7 @@ class QMC5883LComponent : public PollingComponent, public i2c::I2CDevice { sensor::Sensor *z_sensor_{nullptr}; sensor::Sensor *heading_sensor_{nullptr}; sensor::Sensor *temperature_sensor_{nullptr}; + GPIOPin *drdy_pin_{nullptr}; enum ErrorCode { NONE = 0, COMMUNICATION_FAILED, diff --git a/esphome/components/qmc5883l/sensor.py b/esphome/components/qmc5883l/sensor.py index ade286cb9e..b79e370a05 100644 --- a/esphome/components/qmc5883l/sensor.py +++ b/esphome/components/qmc5883l/sensor.py @@ -1,8 +1,12 @@ +import logging + +from esphome import pins import esphome.codegen as cg from esphome.components import i2c, sensor import esphome.config_validation as cv from esphome.const import ( CONF_ADDRESS, + CONF_DATA_RATE, CONF_FIELD_STRENGTH_X, CONF_FIELD_STRENGTH_Y, CONF_FIELD_STRENGTH_Z, @@ -21,6 +25,10 @@ from esphome.const import ( UNIT_MICROTESLA, ) +_LOGGER = logging.getLogger(__name__) + +CONF_DRDY_PIN = "drdy_pin" + DEPENDENCIES = ["i2c"] qmc5883l_ns = cg.esphome_ns.namespace("qmc5883l") @@ -52,6 +60,18 @@ QMC5883LOversamplings = { } +def validate_config(config): + if ( + config[CONF_UPDATE_INTERVAL].total_milliseconds < 15 + and CONF_DRDY_PIN not in config + ): + _LOGGER.warning( + "[qmc5883l] 'update_interval' is less than 15ms and 'drdy_pin' is " + "not configured, this may result in I2C errors" + ) + return config + + def validate_enum(enum_values, units=None, int=True): _units = [] if units is not None: @@ -88,7 +108,7 @@ temperature_schema = sensor.sensor_schema( state_class=STATE_CLASS_MEASUREMENT, ) -CONFIG_SCHEMA = ( +CONFIG_SCHEMA = cv.All( cv.Schema( { cv.GenerateID(): cv.declare_id(QMC5883LComponent), @@ -104,29 +124,25 @@ CONFIG_SCHEMA = ( cv.Optional(CONF_FIELD_STRENGTH_Z): field_strength_schema, cv.Optional(CONF_HEADING): heading_schema, cv.Optional(CONF_TEMPERATURE): temperature_schema, + cv.Optional(CONF_DRDY_PIN): pins.gpio_input_pin_schema, + cv.Optional(CONF_DATA_RATE, default="200hz"): validate_enum( + QMC5883LDatarates, units=["hz", "Hz"] + ), } ) .extend(cv.polling_component_schema("60s")) - .extend(i2c.i2c_device_schema(0x0D)) + .extend(i2c.i2c_device_schema(0x0D)), + validate_config, ) -def auto_data_rate(config): - interval_sec = config[CONF_UPDATE_INTERVAL].total_milliseconds / 1000 - interval_hz = 1.0 / interval_sec - for datarate in sorted(QMC5883LDatarates.keys()): - if float(datarate) >= interval_hz: - return QMC5883LDatarates[datarate] - return QMC5883LDatarates[200] - - async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) await i2c.register_i2c_device(var, config) cg.add(var.set_oversampling(config[CONF_OVERSAMPLING])) - cg.add(var.set_datarate(auto_data_rate(config))) + cg.add(var.set_datarate(config[CONF_DATA_RATE])) cg.add(var.set_range(config[CONF_RANGE])) if CONF_FIELD_STRENGTH_X in config: sens = await sensor.new_sensor(config[CONF_FIELD_STRENGTH_X]) @@ -143,3 +159,6 @@ async def to_code(config): if CONF_TEMPERATURE in config: sens = await sensor.new_sensor(config[CONF_TEMPERATURE]) cg.add(var.set_temperature_sensor(sens)) + if CONF_DRDY_PIN in config: + pin = await cg.gpio_pin_expression(config[CONF_DRDY_PIN]) + cg.add(var.set_drdy_pin(pin)) diff --git a/esphome/components/qmp6988/qmp6988.cpp b/esphome/components/qmp6988/qmp6988.cpp index 4c81e124ba..61fde186d7 100644 --- a/esphome/components/qmp6988/qmp6988.cpp +++ b/esphome/components/qmp6988/qmp6988.cpp @@ -18,14 +18,6 @@ static const uint8_t QMP6988_TEMPERATURE_MSB_REG = 0xFA; /* Temperature MSB Reg static const uint8_t QMP6988_CALIBRATION_DATA_START = 0xA0; /* QMP6988 compensation coefficients */ static const uint8_t QMP6988_CALIBRATION_DATA_LENGTH = 25; -static const uint8_t SHIFT_RIGHT_4_POSITION = 4; -static const uint8_t SHIFT_LEFT_2_POSITION = 2; -static const uint8_t SHIFT_LEFT_4_POSITION = 4; -static const uint8_t SHIFT_LEFT_5_POSITION = 5; -static const uint8_t SHIFT_LEFT_8_POSITION = 8; -static const uint8_t SHIFT_LEFT_12_POSITION = 12; -static const uint8_t SHIFT_LEFT_16_POSITION = 16; - /* power mode */ static const uint8_t QMP6988_SLEEP_MODE = 0x00; static const uint8_t QMP6988_FORCED_MODE = 0x01; @@ -95,64 +87,45 @@ static const char *iir_filter_to_str(QMP6988IIRFilter filter) { } bool QMP6988Component::device_check_() { - uint8_t ret = 0; - - ret = this->read_register(QMP6988_CHIP_ID_REG, &(qmp6988_data_.chip_id), 1); - if (ret != i2c::ERROR_OK) { - ESP_LOGE(TAG, "%s: read chip ID (0xD1) failed", __func__); + if (this->read_register(QMP6988_CHIP_ID_REG, &(qmp6988_data_.chip_id), 1) != i2c::ERROR_OK) { + ESP_LOGE(TAG, "Read chip ID (0xD1) failed"); + return false; } - ESP_LOGD(TAG, "qmp6988 read chip id = 0x%x", qmp6988_data_.chip_id); + ESP_LOGV(TAG, "Read chip ID = 0x%x", qmp6988_data_.chip_id); return qmp6988_data_.chip_id == QMP6988_CHIP_ID; } bool QMP6988Component::get_calibration_data_() { - uint8_t status = 0; // BITFIELDS temp_COE; uint8_t a_data_uint8_tr[QMP6988_CALIBRATION_DATA_LENGTH] = {0}; - int len; - for (len = 0; len < QMP6988_CALIBRATION_DATA_LENGTH; len += 1) { - status = this->read_register(QMP6988_CALIBRATION_DATA_START + len, &a_data_uint8_tr[len], 1); - if (status != i2c::ERROR_OK) { - ESP_LOGE(TAG, "qmp6988 read calibration data (0xA0) error!"); + for (uint8_t len = 0; len < QMP6988_CALIBRATION_DATA_LENGTH; len += 1) { + if (this->read_register(QMP6988_CALIBRATION_DATA_START + len, &a_data_uint8_tr[len], 1) != i2c::ERROR_OK) { + ESP_LOGE(TAG, "Read calibration data (0xA0) error"); return false; } } qmp6988_data_.qmp6988_cali.COE_a0 = - (QMP6988_S32_t) (((a_data_uint8_tr[18] << SHIFT_LEFT_12_POSITION) | - (a_data_uint8_tr[19] << SHIFT_LEFT_4_POSITION) | (a_data_uint8_tr[24] & 0x0f)) - << 12); + (int32_t) encode_uint32(a_data_uint8_tr[18], a_data_uint8_tr[19], (a_data_uint8_tr[24] & 0x0f) << 4, 0); qmp6988_data_.qmp6988_cali.COE_a0 = qmp6988_data_.qmp6988_cali.COE_a0 >> 12; - qmp6988_data_.qmp6988_cali.COE_a1 = - (QMP6988_S16_t) (((a_data_uint8_tr[20]) << SHIFT_LEFT_8_POSITION) | a_data_uint8_tr[21]); - qmp6988_data_.qmp6988_cali.COE_a2 = - (QMP6988_S16_t) (((a_data_uint8_tr[22]) << SHIFT_LEFT_8_POSITION) | a_data_uint8_tr[23]); + qmp6988_data_.qmp6988_cali.COE_a1 = (int16_t) encode_uint16(a_data_uint8_tr[20], a_data_uint8_tr[21]); + qmp6988_data_.qmp6988_cali.COE_a2 = (int16_t) encode_uint16(a_data_uint8_tr[22], a_data_uint8_tr[23]); qmp6988_data_.qmp6988_cali.COE_b00 = - (QMP6988_S32_t) (((a_data_uint8_tr[0] << SHIFT_LEFT_12_POSITION) | (a_data_uint8_tr[1] << SHIFT_LEFT_4_POSITION) | - ((a_data_uint8_tr[24] & 0xf0) >> SHIFT_RIGHT_4_POSITION)) - << 12); + (int32_t) encode_uint32(a_data_uint8_tr[0], a_data_uint8_tr[1], a_data_uint8_tr[24] & 0xf0, 0); qmp6988_data_.qmp6988_cali.COE_b00 = qmp6988_data_.qmp6988_cali.COE_b00 >> 12; - qmp6988_data_.qmp6988_cali.COE_bt1 = - (QMP6988_S16_t) (((a_data_uint8_tr[2]) << SHIFT_LEFT_8_POSITION) | a_data_uint8_tr[3]); - qmp6988_data_.qmp6988_cali.COE_bt2 = - (QMP6988_S16_t) (((a_data_uint8_tr[4]) << SHIFT_LEFT_8_POSITION) | a_data_uint8_tr[5]); - qmp6988_data_.qmp6988_cali.COE_bp1 = - (QMP6988_S16_t) (((a_data_uint8_tr[6]) << SHIFT_LEFT_8_POSITION) | a_data_uint8_tr[7]); - qmp6988_data_.qmp6988_cali.COE_b11 = - (QMP6988_S16_t) (((a_data_uint8_tr[8]) << SHIFT_LEFT_8_POSITION) | a_data_uint8_tr[9]); - qmp6988_data_.qmp6988_cali.COE_bp2 = - (QMP6988_S16_t) (((a_data_uint8_tr[10]) << SHIFT_LEFT_8_POSITION) | a_data_uint8_tr[11]); - qmp6988_data_.qmp6988_cali.COE_b12 = - (QMP6988_S16_t) (((a_data_uint8_tr[12]) << SHIFT_LEFT_8_POSITION) | a_data_uint8_tr[13]); - qmp6988_data_.qmp6988_cali.COE_b21 = - (QMP6988_S16_t) (((a_data_uint8_tr[14]) << SHIFT_LEFT_8_POSITION) | a_data_uint8_tr[15]); - qmp6988_data_.qmp6988_cali.COE_bp3 = - (QMP6988_S16_t) (((a_data_uint8_tr[16]) << SHIFT_LEFT_8_POSITION) | a_data_uint8_tr[17]); + qmp6988_data_.qmp6988_cali.COE_bt1 = (int16_t) encode_uint16(a_data_uint8_tr[2], a_data_uint8_tr[3]); + qmp6988_data_.qmp6988_cali.COE_bt2 = (int16_t) encode_uint16(a_data_uint8_tr[4], a_data_uint8_tr[5]); + qmp6988_data_.qmp6988_cali.COE_bp1 = (int16_t) encode_uint16(a_data_uint8_tr[6], a_data_uint8_tr[7]); + qmp6988_data_.qmp6988_cali.COE_b11 = (int16_t) encode_uint16(a_data_uint8_tr[8], a_data_uint8_tr[9]); + qmp6988_data_.qmp6988_cali.COE_bp2 = (int16_t) encode_uint16(a_data_uint8_tr[10], a_data_uint8_tr[11]); + qmp6988_data_.qmp6988_cali.COE_b12 = (int16_t) encode_uint16(a_data_uint8_tr[12], a_data_uint8_tr[13]); + qmp6988_data_.qmp6988_cali.COE_b21 = (int16_t) encode_uint16(a_data_uint8_tr[14], a_data_uint8_tr[15]); + qmp6988_data_.qmp6988_cali.COE_bp3 = (int16_t) encode_uint16(a_data_uint8_tr[16], a_data_uint8_tr[17]); ESP_LOGV(TAG, "<-----------calibration data-------------->\r\n"); ESP_LOGV(TAG, "COE_a0[%d] COE_a1[%d] COE_a2[%d] COE_b00[%d]\r\n", qmp6988_data_.qmp6988_cali.COE_a0, @@ -166,17 +139,17 @@ bool QMP6988Component::get_calibration_data_() { qmp6988_data_.ik.a0 = qmp6988_data_.qmp6988_cali.COE_a0; // 20Q4 qmp6988_data_.ik.b00 = qmp6988_data_.qmp6988_cali.COE_b00; // 20Q4 - qmp6988_data_.ik.a1 = 3608L * (QMP6988_S32_t) qmp6988_data_.qmp6988_cali.COE_a1 - 1731677965L; // 31Q23 - qmp6988_data_.ik.a2 = 16889L * (QMP6988_S32_t) qmp6988_data_.qmp6988_cali.COE_a2 - 87619360L; // 30Q47 + qmp6988_data_.ik.a1 = 3608L * (int32_t) qmp6988_data_.qmp6988_cali.COE_a1 - 1731677965L; // 31Q23 + qmp6988_data_.ik.a2 = 16889L * (int32_t) qmp6988_data_.qmp6988_cali.COE_a2 - 87619360L; // 30Q47 - qmp6988_data_.ik.bt1 = 2982L * (QMP6988_S64_t) qmp6988_data_.qmp6988_cali.COE_bt1 + 107370906L; // 28Q15 - qmp6988_data_.ik.bt2 = 329854L * (QMP6988_S64_t) qmp6988_data_.qmp6988_cali.COE_bt2 + 108083093L; // 34Q38 - qmp6988_data_.ik.bp1 = 19923L * (QMP6988_S64_t) qmp6988_data_.qmp6988_cali.COE_bp1 + 1133836764L; // 31Q20 - qmp6988_data_.ik.b11 = 2406L * (QMP6988_S64_t) qmp6988_data_.qmp6988_cali.COE_b11 + 118215883L; // 28Q34 - qmp6988_data_.ik.bp2 = 3079L * (QMP6988_S64_t) qmp6988_data_.qmp6988_cali.COE_bp2 - 181579595L; // 29Q43 - qmp6988_data_.ik.b12 = 6846L * (QMP6988_S64_t) qmp6988_data_.qmp6988_cali.COE_b12 + 85590281L; // 29Q53 - qmp6988_data_.ik.b21 = 13836L * (QMP6988_S64_t) qmp6988_data_.qmp6988_cali.COE_b21 + 79333336L; // 29Q60 - qmp6988_data_.ik.bp3 = 2915L * (QMP6988_S64_t) qmp6988_data_.qmp6988_cali.COE_bp3 + 157155561L; // 28Q65 + qmp6988_data_.ik.bt1 = 2982L * (int64_t) qmp6988_data_.qmp6988_cali.COE_bt1 + 107370906L; // 28Q15 + qmp6988_data_.ik.bt2 = 329854L * (int64_t) qmp6988_data_.qmp6988_cali.COE_bt2 + 108083093L; // 34Q38 + qmp6988_data_.ik.bp1 = 19923L * (int64_t) qmp6988_data_.qmp6988_cali.COE_bp1 + 1133836764L; // 31Q20 + qmp6988_data_.ik.b11 = 2406L * (int64_t) qmp6988_data_.qmp6988_cali.COE_b11 + 118215883L; // 28Q34 + qmp6988_data_.ik.bp2 = 3079L * (int64_t) qmp6988_data_.qmp6988_cali.COE_bp2 - 181579595L; // 29Q43 + qmp6988_data_.ik.b12 = 6846L * (int64_t) qmp6988_data_.qmp6988_cali.COE_b12 + 85590281L; // 29Q53 + qmp6988_data_.ik.b21 = 13836L * (int64_t) qmp6988_data_.qmp6988_cali.COE_b21 + 79333336L; // 29Q60 + qmp6988_data_.ik.bp3 = 2915L * (int64_t) qmp6988_data_.qmp6988_cali.COE_bp3 + 157155561L; // 28Q65 ESP_LOGV(TAG, "<----------- int calibration data -------------->\r\n"); ESP_LOGV(TAG, "a0[%d] a1[%d] a2[%d] b00[%d]\r\n", qmp6988_data_.ik.a0, qmp6988_data_.ik.a1, qmp6988_data_.ik.a2, qmp6988_data_.ik.b00); @@ -188,55 +161,55 @@ bool QMP6988Component::get_calibration_data_() { return true; } -QMP6988_S16_t QMP6988Component::get_compensated_temperature_(qmp6988_ik_data_t *ik, QMP6988_S32_t dt) { - QMP6988_S16_t ret; - QMP6988_S64_t wk1, wk2; +int16_t QMP6988Component::get_compensated_temperature_(qmp6988_ik_data_t *ik, int32_t dt) { + int16_t ret; + int64_t wk1, wk2; // wk1: 60Q4 // bit size - wk1 = ((QMP6988_S64_t) ik->a1 * (QMP6988_S64_t) dt); // 31Q23+24-1=54 (54Q23) - wk2 = ((QMP6988_S64_t) ik->a2 * (QMP6988_S64_t) dt) >> 14; // 30Q47+24-1=53 (39Q33) - wk2 = (wk2 * (QMP6988_S64_t) dt) >> 10; // 39Q33+24-1=62 (52Q23) - wk2 = ((wk1 + wk2) / 32767) >> 19; // 54,52->55Q23 (20Q04) - ret = (QMP6988_S16_t) ((ik->a0 + wk2) >> 4); // 21Q4 -> 17Q0 + wk1 = ((int64_t) ik->a1 * (int64_t) dt); // 31Q23+24-1=54 (54Q23) + wk2 = ((int64_t) ik->a2 * (int64_t) dt) >> 14; // 30Q47+24-1=53 (39Q33) + wk2 = (wk2 * (int64_t) dt) >> 10; // 39Q33+24-1=62 (52Q23) + wk2 = ((wk1 + wk2) / 32767) >> 19; // 54,52->55Q23 (20Q04) + ret = (int16_t) ((ik->a0 + wk2) >> 4); // 21Q4 -> 17Q0 return ret; } -QMP6988_S32_t QMP6988Component::get_compensated_pressure_(qmp6988_ik_data_t *ik, QMP6988_S32_t dp, QMP6988_S16_t tx) { - QMP6988_S32_t ret; - QMP6988_S64_t wk1, wk2, wk3; +int32_t QMP6988Component::get_compensated_pressure_(qmp6988_ik_data_t *ik, int32_t dp, int16_t tx) { + int32_t ret; + int64_t wk1, wk2, wk3; // wk1 = 48Q16 // bit size - wk1 = ((QMP6988_S64_t) ik->bt1 * (QMP6988_S64_t) tx); // 28Q15+16-1=43 (43Q15) - wk2 = ((QMP6988_S64_t) ik->bp1 * (QMP6988_S64_t) dp) >> 5; // 31Q20+24-1=54 (49Q15) - wk1 += wk2; // 43,49->50Q15 - wk2 = ((QMP6988_S64_t) ik->bt2 * (QMP6988_S64_t) tx) >> 1; // 34Q38+16-1=49 (48Q37) - wk2 = (wk2 * (QMP6988_S64_t) tx) >> 8; // 48Q37+16-1=63 (55Q29) - wk3 = wk2; // 55Q29 - wk2 = ((QMP6988_S64_t) ik->b11 * (QMP6988_S64_t) tx) >> 4; // 28Q34+16-1=43 (39Q30) - wk2 = (wk2 * (QMP6988_S64_t) dp) >> 1; // 39Q30+24-1=62 (61Q29) - wk3 += wk2; // 55,61->62Q29 - wk2 = ((QMP6988_S64_t) ik->bp2 * (QMP6988_S64_t) dp) >> 13; // 29Q43+24-1=52 (39Q30) - wk2 = (wk2 * (QMP6988_S64_t) dp) >> 1; // 39Q30+24-1=62 (61Q29) - wk3 += wk2; // 62,61->63Q29 - wk1 += wk3 >> 14; // Q29 >> 14 -> Q15 - wk2 = ((QMP6988_S64_t) ik->b12 * (QMP6988_S64_t) tx); // 29Q53+16-1=45 (45Q53) - wk2 = (wk2 * (QMP6988_S64_t) tx) >> 22; // 45Q53+16-1=61 (39Q31) - wk2 = (wk2 * (QMP6988_S64_t) dp) >> 1; // 39Q31+24-1=62 (61Q30) - wk3 = wk2; // 61Q30 - wk2 = ((QMP6988_S64_t) ik->b21 * (QMP6988_S64_t) tx) >> 6; // 29Q60+16-1=45 (39Q54) - wk2 = (wk2 * (QMP6988_S64_t) dp) >> 23; // 39Q54+24-1=62 (39Q31) - wk2 = (wk2 * (QMP6988_S64_t) dp) >> 1; // 39Q31+24-1=62 (61Q20) - wk3 += wk2; // 61,61->62Q30 - wk2 = ((QMP6988_S64_t) ik->bp3 * (QMP6988_S64_t) dp) >> 12; // 28Q65+24-1=51 (39Q53) - wk2 = (wk2 * (QMP6988_S64_t) dp) >> 23; // 39Q53+24-1=62 (39Q30) - wk2 = (wk2 * (QMP6988_S64_t) dp); // 39Q30+24-1=62 (62Q30) - wk3 += wk2; // 62,62->63Q30 - wk1 += wk3 >> 15; // Q30 >> 15 = Q15 + wk1 = ((int64_t) ik->bt1 * (int64_t) tx); // 28Q15+16-1=43 (43Q15) + wk2 = ((int64_t) ik->bp1 * (int64_t) dp) >> 5; // 31Q20+24-1=54 (49Q15) + wk1 += wk2; // 43,49->50Q15 + wk2 = ((int64_t) ik->bt2 * (int64_t) tx) >> 1; // 34Q38+16-1=49 (48Q37) + wk2 = (wk2 * (int64_t) tx) >> 8; // 48Q37+16-1=63 (55Q29) + wk3 = wk2; // 55Q29 + wk2 = ((int64_t) ik->b11 * (int64_t) tx) >> 4; // 28Q34+16-1=43 (39Q30) + wk2 = (wk2 * (int64_t) dp) >> 1; // 39Q30+24-1=62 (61Q29) + wk3 += wk2; // 55,61->62Q29 + wk2 = ((int64_t) ik->bp2 * (int64_t) dp) >> 13; // 29Q43+24-1=52 (39Q30) + wk2 = (wk2 * (int64_t) dp) >> 1; // 39Q30+24-1=62 (61Q29) + wk3 += wk2; // 62,61->63Q29 + wk1 += wk3 >> 14; // Q29 >> 14 -> Q15 + wk2 = ((int64_t) ik->b12 * (int64_t) tx); // 29Q53+16-1=45 (45Q53) + wk2 = (wk2 * (int64_t) tx) >> 22; // 45Q53+16-1=61 (39Q31) + wk2 = (wk2 * (int64_t) dp) >> 1; // 39Q31+24-1=62 (61Q30) + wk3 = wk2; // 61Q30 + wk2 = ((int64_t) ik->b21 * (int64_t) tx) >> 6; // 29Q60+16-1=45 (39Q54) + wk2 = (wk2 * (int64_t) dp) >> 23; // 39Q54+24-1=62 (39Q31) + wk2 = (wk2 * (int64_t) dp) >> 1; // 39Q31+24-1=62 (61Q20) + wk3 += wk2; // 61,61->62Q30 + wk2 = ((int64_t) ik->bp3 * (int64_t) dp) >> 12; // 28Q65+24-1=51 (39Q53) + wk2 = (wk2 * (int64_t) dp) >> 23; // 39Q53+24-1=62 (39Q30) + wk2 = (wk2 * (int64_t) dp); // 39Q30+24-1=62 (62Q30) + wk3 += wk2; // 62,62->63Q30 + wk1 += wk3 >> 15; // Q30 >> 15 = Q15 wk1 /= 32767L; wk1 >>= 11; // Q15 >> 7 = Q4 wk1 += ik->b00; // Q4 + 20Q4 // wk1 >>= 4; // 28Q4 -> 24Q0 - ret = (QMP6988_S32_t) wk1; + ret = (int32_t) wk1; return ret; } @@ -274,7 +247,7 @@ void QMP6988Component::set_power_mode_(uint8_t power_mode) { delay(10); } -void QMP6988Component::write_filter_(unsigned char filter) { +void QMP6988Component::write_filter_(QMP6988IIRFilter filter) { uint8_t data; data = (filter & 0x03); @@ -282,7 +255,7 @@ void QMP6988Component::write_filter_(unsigned char filter) { delay(10); } -void QMP6988Component::write_oversampling_pressure_(unsigned char oversampling_p) { +void QMP6988Component::write_oversampling_pressure_(QMP6988Oversampling oversampling_p) { uint8_t data; this->read_register(QMP6988_CTRLMEAS_REG, &data, 1); @@ -292,7 +265,7 @@ void QMP6988Component::write_oversampling_pressure_(unsigned char oversampling_p delay(10); } -void QMP6988Component::write_oversampling_temperature_(unsigned char oversampling_t) { +void QMP6988Component::write_oversampling_temperature_(QMP6988Oversampling oversampling_t) { uint8_t data; this->read_register(QMP6988_CTRLMEAS_REG, &data, 1); @@ -302,16 +275,6 @@ void QMP6988Component::write_oversampling_temperature_(unsigned char oversamplin delay(10); } -void QMP6988Component::set_temperature_oversampling(QMP6988Oversampling oversampling_t) { - this->temperature_oversampling_ = oversampling_t; -} - -void QMP6988Component::set_pressure_oversampling(QMP6988Oversampling oversampling_p) { - this->pressure_oversampling_ = oversampling_p; -} - -void QMP6988Component::set_iir_filter(QMP6988IIRFilter iirfilter) { this->iir_filter_ = iirfilter; } - void QMP6988Component::calculate_altitude_(float pressure, float temp) { float altitude; altitude = (pow((101325 / pressure), 1 / 5.257) - 1) * (temp + 273.15) / 0.0065; @@ -320,10 +283,10 @@ void QMP6988Component::calculate_altitude_(float pressure, float temp) { void QMP6988Component::calculate_pressure_() { uint8_t err = 0; - QMP6988_U32_t p_read, t_read; - QMP6988_S32_t p_raw, t_raw; + uint32_t p_read, t_read; + int32_t p_raw, t_raw; uint8_t a_data_uint8_tr[6] = {0}; - QMP6988_S32_t t_int, p_int; + int32_t t_int, p_int; this->qmp6988_data_.temperature = 0; this->qmp6988_data_.pressure = 0; @@ -332,13 +295,11 @@ void QMP6988Component::calculate_pressure_() { ESP_LOGE(TAG, "Error reading raw pressure/temp values"); return; } - p_read = (QMP6988_U32_t) ((((QMP6988_U32_t) (a_data_uint8_tr[0])) << SHIFT_LEFT_16_POSITION) | - (((QMP6988_U16_t) (a_data_uint8_tr[1])) << SHIFT_LEFT_8_POSITION) | (a_data_uint8_tr[2])); - p_raw = (QMP6988_S32_t) (p_read - SUBTRACTOR); + p_read = encode_uint24(a_data_uint8_tr[0], a_data_uint8_tr[1], a_data_uint8_tr[2]); + p_raw = (int32_t) (p_read - SUBTRACTOR); - t_read = (QMP6988_U32_t) ((((QMP6988_U32_t) (a_data_uint8_tr[3])) << SHIFT_LEFT_16_POSITION) | - (((QMP6988_U16_t) (a_data_uint8_tr[4])) << SHIFT_LEFT_8_POSITION) | (a_data_uint8_tr[5])); - t_raw = (QMP6988_S32_t) (t_read - SUBTRACTOR); + t_read = encode_uint24(a_data_uint8_tr[3], a_data_uint8_tr[4], a_data_uint8_tr[5]); + t_raw = (int32_t) (t_read - SUBTRACTOR); t_int = this->get_compensated_temperature_(&(qmp6988_data_.ik), t_raw); p_int = this->get_compensated_pressure_(&(qmp6988_data_.ik), p_raw, t_int); @@ -348,12 +309,9 @@ void QMP6988Component::calculate_pressure_() { } void QMP6988Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - - bool ret; - ret = this->device_check_(); - if (!ret) { - ESP_LOGCONFIG(TAG, "Setup failed - device not found"); + if (!this->device_check_()) { + this->mark_failed(ESP_LOG_MSG_COMM_FAIL); + return; } this->software_reset_(); @@ -367,9 +325,6 @@ void QMP6988Component::setup() { void QMP6988Component::dump_config() { ESP_LOGCONFIG(TAG, "QMP6988:"); LOG_I2C_DEVICE(this); - if (this->is_failed()) { - ESP_LOGE(TAG, ESP_LOG_MSG_COMM_FAIL); - } LOG_UPDATE_INTERVAL(this); LOG_SENSOR(" ", "Temperature", this->temperature_sensor_); @@ -379,8 +334,6 @@ void QMP6988Component::dump_config() { ESP_LOGCONFIG(TAG, " IIR Filter: %s", iir_filter_to_str(this->iir_filter_)); } -float QMP6988Component::get_setup_priority() const { return setup_priority::DATA; } - void QMP6988Component::update() { this->calculate_pressure_(); float pressurehectopascals = this->qmp6988_data_.pressure / 100; diff --git a/esphome/components/qmp6988/qmp6988.h b/esphome/components/qmp6988/qmp6988.h index 61b46a4189..5b0f80c77e 100644 --- a/esphome/components/qmp6988/qmp6988.h +++ b/esphome/components/qmp6988/qmp6988.h @@ -1,24 +1,17 @@ #pragma once +#include "esphome/components/i2c/i2c.h" +#include "esphome/components/sensor/sensor.h" #include "esphome/core/component.h" #include "esphome/core/hal.h" #include "esphome/core/helpers.h" #include "esphome/core/log.h" -#include "esphome/components/sensor/sensor.h" -#include "esphome/components/i2c/i2c.h" namespace esphome { namespace qmp6988 { -#define QMP6988_U16_t unsigned short -#define QMP6988_S16_t short -#define QMP6988_U32_t unsigned int -#define QMP6988_S32_t int -#define QMP6988_U64_t unsigned long long -#define QMP6988_S64_t long long - /* oversampling */ -enum QMP6988Oversampling { +enum QMP6988Oversampling : uint8_t { QMP6988_OVERSAMPLING_SKIPPED = 0x00, QMP6988_OVERSAMPLING_1X = 0x01, QMP6988_OVERSAMPLING_2X = 0x02, @@ -30,7 +23,7 @@ enum QMP6988Oversampling { }; /* filter */ -enum QMP6988IIRFilter { +enum QMP6988IIRFilter : uint8_t { QMP6988_IIR_FILTER_OFF = 0x00, QMP6988_IIR_FILTER_2X = 0x01, QMP6988_IIR_FILTER_4X = 0x02, @@ -40,18 +33,18 @@ enum QMP6988IIRFilter { }; using qmp6988_cali_data_t = struct Qmp6988CaliData { - QMP6988_S32_t COE_a0; - QMP6988_S16_t COE_a1; - QMP6988_S16_t COE_a2; - QMP6988_S32_t COE_b00; - QMP6988_S16_t COE_bt1; - QMP6988_S16_t COE_bt2; - QMP6988_S16_t COE_bp1; - QMP6988_S16_t COE_b11; - QMP6988_S16_t COE_bp2; - QMP6988_S16_t COE_b12; - QMP6988_S16_t COE_b21; - QMP6988_S16_t COE_bp3; + int32_t COE_a0; + int16_t COE_a1; + int16_t COE_a2; + int32_t COE_b00; + int16_t COE_bt1; + int16_t COE_bt2; + int16_t COE_bp1; + int16_t COE_b11; + int16_t COE_bp2; + int16_t COE_b12; + int16_t COE_b21; + int16_t COE_bp3; }; using qmp6988_fk_data_t = struct Qmp6988FkData { @@ -60,9 +53,9 @@ using qmp6988_fk_data_t = struct Qmp6988FkData { }; using qmp6988_ik_data_t = struct Qmp6988IkData { - QMP6988_S32_t a0, b00; - QMP6988_S32_t a1, a2; - QMP6988_S64_t bt1, bt2, bp1, b11, bp2, b12, b21, bp3; + int32_t a0, b00; + int32_t a1, a2; + int64_t bt1, bt2, bp1, b11, bp2, b12, b21, bp3; }; using qmp6988_data_t = struct Qmp6988Data { @@ -77,17 +70,18 @@ using qmp6988_data_t = struct Qmp6988Data { class QMP6988Component : public PollingComponent, public i2c::I2CDevice { public: - void set_temperature_sensor(sensor::Sensor *temperature_sensor) { temperature_sensor_ = temperature_sensor; } - void set_pressure_sensor(sensor::Sensor *pressure_sensor) { pressure_sensor_ = pressure_sensor; } + void set_temperature_sensor(sensor::Sensor *temperature_sensor) { this->temperature_sensor_ = temperature_sensor; } + void set_pressure_sensor(sensor::Sensor *pressure_sensor) { this->pressure_sensor_ = pressure_sensor; } void setup() override; void dump_config() override; - float get_setup_priority() const override; void update() override; - void set_iir_filter(QMP6988IIRFilter iirfilter); - void set_temperature_oversampling(QMP6988Oversampling oversampling_t); - void set_pressure_oversampling(QMP6988Oversampling oversampling_p); + void set_iir_filter(QMP6988IIRFilter iirfilter) { this->iir_filter_ = iirfilter; } + void set_temperature_oversampling(QMP6988Oversampling oversampling_t) { + this->temperature_oversampling_ = oversampling_t; + } + void set_pressure_oversampling(QMP6988Oversampling oversampling_p) { this->pressure_oversampling_ = oversampling_p; } protected: qmp6988_data_t qmp6988_data_; @@ -102,14 +96,14 @@ class QMP6988Component : public PollingComponent, public i2c::I2CDevice { bool get_calibration_data_(); bool device_check_(); void set_power_mode_(uint8_t power_mode); - void write_oversampling_temperature_(unsigned char oversampling_t); - void write_oversampling_pressure_(unsigned char oversampling_p); - void write_filter_(unsigned char filter); + void write_oversampling_temperature_(QMP6988Oversampling oversampling_t); + void write_oversampling_pressure_(QMP6988Oversampling oversampling_p); + void write_filter_(QMP6988IIRFilter filter); void calculate_pressure_(); void calculate_altitude_(float pressure, float temp); - QMP6988_S32_t get_compensated_pressure_(qmp6988_ik_data_t *ik, QMP6988_S32_t dp, QMP6988_S16_t tx); - QMP6988_S16_t get_compensated_temperature_(qmp6988_ik_data_t *ik, QMP6988_S32_t dt); + int32_t get_compensated_pressure_(qmp6988_ik_data_t *ik, int32_t dp, int16_t tx); + int16_t get_compensated_temperature_(qmp6988_ik_data_t *ik, int32_t dt); }; } // namespace qmp6988 diff --git a/esphome/components/qspi_dbi/display.py b/esphome/components/qspi_dbi/display.py index 5b01bcc6ca..74d837a794 100644 --- a/esphome/components/qspi_dbi/display.py +++ b/esphome/components/qspi_dbi/display.py @@ -73,9 +73,8 @@ def map_sequence(value): def _validate(config): chip = DriverChip.chips[config[CONF_MODEL]] - if not chip.initsequence: - if CONF_INIT_SEQUENCE not in config: - raise cv.Invalid(f"{chip.name} model requires init_sequence") + if not chip.initsequence and CONF_INIT_SEQUENCE not in config: + raise cv.Invalid(f"{chip.name} model requires init_sequence") return config diff --git a/esphome/components/qspi_dbi/qspi_dbi.cpp b/esphome/components/qspi_dbi/qspi_dbi.cpp index 2901d40268..6c95bb7cf2 100644 --- a/esphome/components/qspi_dbi/qspi_dbi.cpp +++ b/esphome/components/qspi_dbi/qspi_dbi.cpp @@ -6,7 +6,6 @@ namespace esphome { namespace qspi_dbi { void QspiDbi::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); if (this->enable_pin_ != nullptr) { this->enable_pin_->setup(); @@ -113,7 +112,6 @@ void QspiDbi::write_init_sequence_() { } this->reset_params_(true); this->setup_complete_ = true; - ESP_LOGCONFIG(TAG, "QSPI_DBI setup complete"); } void QspiDbi::set_addr_window_(uint16_t x1, uint16_t y1, uint16_t x2, uint16_t y2) { diff --git a/esphome/components/qwiic_pir/binary_sensor.py b/esphome/components/qwiic_pir/binary_sensor.py index 0a549ccb32..cd3eda5ac8 100644 --- a/esphome/components/qwiic_pir/binary_sensor.py +++ b/esphome/components/qwiic_pir/binary_sensor.py @@ -24,9 +24,8 @@ QwiicPIRComponent = qwiic_pir_ns.class_( def validate_no_debounce_unless_native(config): - if CONF_DEBOUNCE in config: - if config[CONF_DEBOUNCE_MODE] != "NATIVE": - raise cv.Invalid("debounce can only be set if debounce_mode is NATIVE") + if CONF_DEBOUNCE in config and config[CONF_DEBOUNCE_MODE] != "NATIVE": + raise cv.Invalid("debounce can only be set if debounce_mode is NATIVE") return config diff --git a/esphome/components/qwiic_pir/qwiic_pir.cpp b/esphome/components/qwiic_pir/qwiic_pir.cpp index 6a5196f831..c04c0fcc18 100644 --- a/esphome/components/qwiic_pir/qwiic_pir.cpp +++ b/esphome/components/qwiic_pir/qwiic_pir.cpp @@ -7,8 +7,6 @@ namespace qwiic_pir { static const char *const TAG = "qwiic_pir"; void QwiicPIRComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // Verify I2C communcation by reading and verifying the chip ID uint8_t chip_id; if (!this->read_byte(QWIIC_PIR_CHIP_ID, &chip_id)) { diff --git a/esphome/components/radon_eye_ble/__init__.py b/esphome/components/radon_eye_ble/__init__.py index 01910c81a8..99daef30e5 100644 --- a/esphome/components/radon_eye_ble/__init__.py +++ b/esphome/components/radon_eye_ble/__init__.py @@ -18,6 +18,6 @@ CONFIG_SCHEMA = cv.Schema( ).extend(esp32_ble_tracker.ESP_BLE_DEVICE_SCHEMA) -def to_code(config): +async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) - yield esp32_ble_tracker.register_ble_device(var, config) + await esp32_ble_tracker.register_ble_device(var, config) diff --git a/esphome/components/rc522/rc522.cpp b/esphome/components/rc522/rc522.cpp index 018b714190..fa8564f614 100644 --- a/esphome/components/rc522/rc522.cpp +++ b/esphome/components/rc522/rc522.cpp @@ -1,4 +1,5 @@ #include "rc522.h" +#include "esphome/core/helpers.h" #include "esphome/core/log.h" // Based on: @@ -13,30 +14,6 @@ static const char *const TAG = "rc522"; static const uint8_t RESET_COUNT = 5; -std::string format_buffer(uint8_t *b, uint8_t len) { - char buf[32]; - int offset = 0; - for (uint8_t i = 0; i < len; i++) { - const char *format = "%02X"; - if (i + 1 < len) - format = "%02X-"; - offset += sprintf(buf + offset, format, b[i]); - } - return std::string(buf); -} - -std::string format_uid(std::vector &uid) { - char buf[32]; - int offset = 0; - for (size_t i = 0; i < uid.size(); i++) { - const char *format = "%02X"; - if (i + 1 < uid.size()) - format = "%02X-"; - offset += sprintf(buf + offset, format, uid[i]); - } - return std::string(buf); -} - void RC522::setup() { state_ = STATE_SETUP; // Pull device out of power down / reset state. @@ -215,7 +192,7 @@ void RC522::loop() { ESP_LOGV(TAG, "STATE_READ_SERIAL_DONE -> TIMEOUT (no tag present) %d", status); } else { ESP_LOGW(TAG, "Unexpected response. Read status is %d. Read bytes: %d (%s)", status, back_length_, - format_buffer(buffer_, 9).c_str()); + format_hex_pretty(buffer_, back_length_, '-', false).c_str()); } state_ = STATE_DONE; @@ -239,7 +216,7 @@ void RC522::loop() { std::vector rfid_uid(std::begin(uid_buffer_), std::begin(uid_buffer_) + uid_idx_); uid_idx_ = 0; - // ESP_LOGD(TAG, "Processing '%s'", format_uid(rfid_uid).c_str()); + // ESP_LOGD(TAG, "Processing '%s'", format_hex_pretty(rfid_uid, '-', false).c_str()); pcd_antenna_off_(); state_ = STATE_INIT; // scan again on next update bool report = true; @@ -260,13 +237,13 @@ void RC522::loop() { trigger->process(rfid_uid); if (report) { - ESP_LOGD(TAG, "Found new tag '%s'", format_uid(rfid_uid).c_str()); + ESP_LOGD(TAG, "Found new tag '%s'", format_hex_pretty(rfid_uid, '-', false).c_str()); } break; } case STATE_DONE: { if (!this->current_uid_.empty()) { - ESP_LOGV(TAG, "Tag '%s' removed", format_uid(this->current_uid_).c_str()); + ESP_LOGV(TAG, "Tag '%s' removed", format_hex_pretty(this->current_uid_, '-', false).c_str()); for (auto *trigger : this->triggers_ontagremoved_) trigger->process(this->current_uid_); } @@ -361,7 +338,7 @@ void RC522::pcd_clear_register_bit_mask_(PcdRegister reg, ///< The register to * @return STATUS_OK on success, STATUS_??? otherwise. */ void RC522::pcd_transceive_data_(uint8_t send_len) { - ESP_LOGV(TAG, "PCD TRANSCEIVE: RX: %s", format_buffer(buffer_, send_len).c_str()); + ESP_LOGV(TAG, "PCD TRANSCEIVE: RX: %s", format_hex_pretty(buffer_, send_len, '-', false).c_str()); delayMicroseconds(1000); // we need 1 ms delay between antenna on and those communication commands send_len_ = send_len; // Prepare values for BitFramingReg @@ -435,7 +412,8 @@ RC522::StatusCode RC522::await_transceive_() { error_reg_value); // TODO: is this always due to collissions? return STATUS_ERROR; } - ESP_LOGV(TAG, "received %d bytes: %s", back_length_, format_buffer(buffer_ + send_len_, back_length_).c_str()); + ESP_LOGV(TAG, "received %d bytes: %s", back_length_, + format_hex_pretty(buffer_ + send_len_, back_length_, '-', false).c_str()); return STATUS_OK; } @@ -499,7 +477,7 @@ bool RC522BinarySensor::process(std::vector &data) { this->found_ = result; return result; } -void RC522Trigger::process(std::vector &data) { this->trigger(format_uid(data)); } +void RC522Trigger::process(std::vector &data) { this->trigger(format_hex_pretty(data, '-', false)); } } // namespace rc522 } // namespace esphome diff --git a/esphome/components/rc522_spi/rc522_spi.cpp b/esphome/components/rc522_spi/rc522_spi.cpp index fe1f6097e2..23e92be65a 100644 --- a/esphome/components/rc522_spi/rc522_spi.cpp +++ b/esphome/components/rc522_spi/rc522_spi.cpp @@ -10,7 +10,6 @@ namespace rc522_spi { static const char *const TAG = "rc522_spi"; void RC522Spi::setup() { - ESP_LOGI(TAG, "SPI Setup"); this->spi_setup(); RC522::setup(); diff --git a/esphome/components/remote_base/__init__.py b/esphome/components/remote_base/__init__.py index fc824ef704..ccf16a8beb 100644 --- a/esphome/components/remote_base/__init__.py +++ b/esphome/components/remote_base/__init__.py @@ -1056,18 +1056,63 @@ async def sony_action(var, config, args): cg.add(var.set_nbits(template_)) +# Symphony +SymphonyData, SymphonyBinarySensor, SymphonyTrigger, SymphonyAction, SymphonyDumper = ( + declare_protocol("Symphony") +) +SYMPHONY_SCHEMA = cv.Schema( + { + cv.Required(CONF_DATA): cv.hex_uint32_t, + cv.Required(CONF_NBITS): cv.int_range(min=1, max=32), + cv.Optional(CONF_COMMAND_REPEATS, default=2): cv.uint8_t, + } +) + + +@register_binary_sensor("symphony", SymphonyBinarySensor, SYMPHONY_SCHEMA) +def symphony_binary_sensor(var, config): + cg.add( + var.set_data( + cg.StructInitializer( + SymphonyData, + ("data", config[CONF_DATA]), + ("nbits", config[CONF_NBITS]), + ) + ) + ) + + +@register_trigger("symphony", SymphonyTrigger, SymphonyData) +def symphony_trigger(var, config): + pass + + +@register_dumper("symphony", SymphonyDumper) +def symphony_dumper(var, config): + pass + + +@register_action("symphony", SymphonyAction, SYMPHONY_SCHEMA) +async def symphony_action(var, config, args): + template_ = await cg.templatable(config[CONF_DATA], args, cg.uint32) + cg.add(var.set_data(template_)) + template_ = await cg.templatable(config[CONF_NBITS], args, cg.uint32) + cg.add(var.set_nbits(template_)) + template_ = await cg.templatable(config[CONF_COMMAND_REPEATS], args, cg.uint8) + cg.add(var.set_repeats(template_)) + + # Raw def validate_raw_alternating(value): assert isinstance(value, list) last_negative = None for i, val in enumerate(value): this_negative = val < 0 - if i != 0: - if this_negative == last_negative: - raise cv.Invalid( - f"Values must alternate between being positive and negative, please see index {i} and {i + 1}", - [i], - ) + if i != 0 and this_negative == last_negative: + raise cv.Invalid( + f"Values must alternate between being positive and negative, please see index {i} and {i + 1}", + [i], + ) last_negative = this_negative return value @@ -1783,14 +1828,12 @@ def nexa_dumper(var, config): @register_action("nexa", NexaAction, NEXA_SCHEMA) -def nexa_action(var, config, args): - cg.add(var.set_device((yield cg.templatable(config[CONF_DEVICE], args, cg.uint32)))) - cg.add(var.set_group((yield cg.templatable(config[CONF_GROUP], args, cg.uint8)))) - cg.add(var.set_state((yield cg.templatable(config[CONF_STATE], args, cg.uint8)))) - cg.add( - var.set_channel((yield cg.templatable(config[CONF_CHANNEL], args, cg.uint8))) - ) - cg.add(var.set_level((yield cg.templatable(config[CONF_LEVEL], args, cg.uint8)))) +async def nexa_action(var, config, args): + cg.add(var.set_device(await cg.templatable(config[CONF_DEVICE], args, cg.uint32))) + cg.add(var.set_group(await cg.templatable(config[CONF_GROUP], args, cg.uint8))) + cg.add(var.set_state(await cg.templatable(config[CONF_STATE], args, cg.uint8))) + cg.add(var.set_channel(await cg.templatable(config[CONF_CHANNEL], args, cg.uint8))) + cg.add(var.set_level(await cg.templatable(config[CONF_LEVEL], args, cg.uint8))) # Midea diff --git a/esphome/components/remote_base/gobox_protocol.cpp b/esphome/components/remote_base/gobox_protocol.cpp index 54e0dff663..4f6de5e59e 100644 --- a/esphome/components/remote_base/gobox_protocol.cpp +++ b/esphome/components/remote_base/gobox_protocol.cpp @@ -10,8 +10,8 @@ constexpr uint32_t BIT_MARK_US = 580; // 70us seems like a safe time delta for constexpr uint32_t BIT_ONE_SPACE_US = 1640; constexpr uint32_t BIT_ZERO_SPACE_US = 545; constexpr uint64_t HEADER = 0b011001001100010uL; // 15 bits -constexpr uint64_t HEADER_SIZE = 15; -constexpr uint64_t CODE_SIZE = 17; +constexpr size_t HEADER_SIZE = 15; +constexpr size_t CODE_SIZE = 17; void GoboxProtocol::dump_timings_(const RawTimings &timings) const { ESP_LOGD(TAG, "Gobox: size=%u", timings.size()); @@ -39,7 +39,7 @@ void GoboxProtocol::encode(RemoteTransmitData *dst, const GoboxData &data) { } optional GoboxProtocol::decode(RemoteReceiveData src) { - if (src.size() < ((HEADER_SIZE + CODE_SIZE) * 2 + 1)) { + if (static_cast(src.size()) < ((HEADER_SIZE + CODE_SIZE) * 2 + 1)) { return {}; } diff --git a/esphome/components/remote_base/symphony_protocol.cpp b/esphome/components/remote_base/symphony_protocol.cpp new file mode 100644 index 0000000000..34b5dba07f --- /dev/null +++ b/esphome/components/remote_base/symphony_protocol.cpp @@ -0,0 +1,120 @@ +#include "symphony_protocol.h" +#include "esphome/core/log.h" + +namespace esphome { +namespace remote_base { + +static const char *const TAG = "remote.symphony"; + +// Reference implementation and timing details: +// IRremoteESP8266 ir_Symphony.cpp +// https://github.com/crankyoldgit/IRremoteESP8266/blob/master/src/ir_Symphony.cpp +// The implementation below mirrors the constant bit-time mapping and +// footer-gap handling used there. + +// Symphony protocol timing specifications (tuned to handset captures) +static const uint32_t BIT_ZERO_HIGH_US = 460; // short +static const uint32_t BIT_ZERO_LOW_US = 1260; // long +static const uint32_t BIT_ONE_HIGH_US = 1260; // long +static const uint32_t BIT_ONE_LOW_US = 460; // short +static const uint32_t CARRIER_FREQUENCY = 38000; + +// IRremoteESP8266 reference: kSymphonyFooterGap = 4 * (mark + space) +static const uint32_t FOOTER_GAP_US = 4 * (BIT_ZERO_HIGH_US + BIT_ZERO_LOW_US); +// Typical inter-frame gap (~34.8 ms observed) +static const uint32_t INTER_FRAME_GAP_US = 34760; + +void SymphonyProtocol::encode(RemoteTransmitData *dst, const SymphonyData &data) { + dst->set_carrier_frequency(CARRIER_FREQUENCY); + ESP_LOGD(TAG, "Sending Symphony: data=0x%0*X nbits=%u repeats=%u", (data.nbits + 3) / 4, (uint32_t) data.data, + data.nbits, data.repeats); + // Each bit produces a mark+space (2 entries). We fold the inter-frame/footer gap + // into the last bit's space of each frame to avoid over-length gaps. + dst->reserve(data.nbits * 2u * data.repeats); + + for (uint8_t repeats = 0; repeats < data.repeats; repeats++) { + // Data bits (MSB first) + for (uint32_t mask = 1UL << (data.nbits - 1); mask != 0; mask >>= 1) { + const bool is_last_bit = (mask == 1); + const bool is_last_frame = (repeats == (data.repeats - 1)); + if (is_last_bit) { + // Emit last bit's mark; replace its space with the proper gap + if (data.data & mask) { + dst->mark(BIT_ONE_HIGH_US); + } else { + dst->mark(BIT_ZERO_HIGH_US); + } + dst->space(is_last_frame ? FOOTER_GAP_US : INTER_FRAME_GAP_US); + } else { + if (data.data & mask) { + dst->item(BIT_ONE_HIGH_US, BIT_ONE_LOW_US); + } else { + dst->item(BIT_ZERO_HIGH_US, BIT_ZERO_LOW_US); + } + } + } + } +} + +optional SymphonyProtocol::decode(RemoteReceiveData src) { + auto is_valid_len = [](uint8_t nbits) -> bool { return nbits == 8 || nbits == 12 || nbits == 16; }; + + RemoteReceiveData s = src; // copy + SymphonyData out{0, 0, 1}; + + for (; out.nbits < 32; out.nbits++) { + if (s.expect_mark(BIT_ONE_HIGH_US)) { + if (!s.expect_space(BIT_ONE_LOW_US)) { + // Allow footer gap immediately after the last mark + if (s.peek_space_at_least(FOOTER_GAP_US)) { + uint8_t bits_with_this = out.nbits + 1; + if (is_valid_len(bits_with_this)) { + out.data = (out.data << 1UL) | 1UL; + out.nbits = bits_with_this; + return out; + } + } + return {}; + } + // Successfully consumed a '1' bit (mark + space) + out.data = (out.data << 1UL) | 1UL; + continue; + } else if (s.expect_mark(BIT_ZERO_HIGH_US)) { + if (!s.expect_space(BIT_ZERO_LOW_US)) { + // Allow footer gap immediately after the last mark + if (s.peek_space_at_least(FOOTER_GAP_US)) { + uint8_t bits_with_this = out.nbits + 1; + if (is_valid_len(bits_with_this)) { + out.data = (out.data << 1UL) | 0UL; + out.nbits = bits_with_this; + return out; + } + } + return {}; + } + // Successfully consumed a '0' bit (mark + space) + out.data = (out.data << 1UL) | 0UL; + continue; + } else { + // Completed a valid-length frame followed by a footer gap + if (is_valid_len(out.nbits) && s.peek_space_at_least(FOOTER_GAP_US)) { + return out; + } + return {}; + } + } + + if (is_valid_len(out.nbits) && s.peek_space_at_least(FOOTER_GAP_US)) { + return out; + } + + return {}; +} + +void SymphonyProtocol::dump(const SymphonyData &data) { + const int32_t hex_width = (data.nbits + 3) / 4; // pad to nibble width + ESP_LOGI(TAG, "Received Symphony: data=0x%0*X, nbits=%d", hex_width, (uint32_t) data.data, data.nbits); +} + +} // namespace remote_base +} // namespace esphome diff --git a/esphome/components/remote_base/symphony_protocol.h b/esphome/components/remote_base/symphony_protocol.h new file mode 100644 index 0000000000..7e77a268ba --- /dev/null +++ b/esphome/components/remote_base/symphony_protocol.h @@ -0,0 +1,44 @@ +#pragma once + +#include "esphome/core/component.h" +#include "remote_base.h" + +#include + +namespace esphome { +namespace remote_base { + +struct SymphonyData { + uint32_t data; + uint8_t nbits; + uint8_t repeats{1}; + + bool operator==(const SymphonyData &rhs) const { return data == rhs.data && nbits == rhs.nbits; } +}; + +class SymphonyProtocol : public RemoteProtocol { + public: + void encode(RemoteTransmitData *dst, const SymphonyData &data) override; + optional decode(RemoteReceiveData src) override; + void dump(const SymphonyData &data) override; +}; + +DECLARE_REMOTE_PROTOCOL(Symphony) + +template class SymphonyAction : public RemoteTransmitterActionBase { + public: + TEMPLATABLE_VALUE(uint32_t, data) + TEMPLATABLE_VALUE(uint8_t, nbits) + TEMPLATABLE_VALUE(uint8_t, repeats) + + void encode(RemoteTransmitData *dst, Ts... x) override { + SymphonyData data{}; + data.data = this->data_.value(x...); + data.nbits = this->nbits_.value(x...); + data.repeats = this->repeats_.value(x...); + SymphonyProtocol().encode(dst, data); + } +}; + +} // namespace remote_base +} // namespace esphome diff --git a/esphome/components/remote_receiver/__init__.py b/esphome/components/remote_receiver/__init__.py index dffc088085..cd2b440645 100644 --- a/esphome/components/remote_receiver/__init__.py +++ b/esphome/components/remote_receiver/__init__.py @@ -5,6 +5,8 @@ from esphome.config_helpers import filter_source_files_from_platform import esphome.config_validation as cv from esphome.const import ( CONF_BUFFER_SIZE, + CONF_CARRIER_DUTY_PERCENT, + CONF_CARRIER_FREQUENCY, CONF_CLOCK_RESOLUTION, CONF_DUMP, CONF_FILTER, @@ -60,6 +62,20 @@ RemoteReceiverComponent = remote_receiver_ns.class_( ) +def validate_config(config): + if CORE.is_esp32: + variant = esp32.get_esp32_variant() + if variant in (esp32.const.VARIANT_ESP32, esp32.const.VARIANT_ESP32S2): + max_idle = 65535 + else: + max_idle = 32767 + if CONF_CLOCK_RESOLUTION in config: + max_idle = int(max_idle * 1000000 / config[CONF_CLOCK_RESOLUTION]) + if config[CONF_IDLE].total_microseconds > max_idle: + raise cv.Invalid(f"config 'idle' exceeds the maximum value of {max_idle}us") + return config + + def validate_tolerance(value): if isinstance(value, dict): return TOLERANCE_SCHEMA(value) @@ -135,8 +151,18 @@ CONFIG_SCHEMA = remote_base.validate_triggers( ), cv.boolean, ), + cv.SplitDefault(CONF_CARRIER_DUTY_PERCENT, esp32=100): cv.All( + cv.only_on_esp32, + cv.percentage_int, + cv.Range(min=1, max=100), + ), + cv.SplitDefault(CONF_CARRIER_FREQUENCY, esp32="0Hz"): cv.All( + cv.only_on_esp32, cv.frequency, cv.int_ + ), } - ).extend(cv.COMPONENT_SCHEMA) + ) + .extend(cv.COMPONENT_SCHEMA) + .add_extra(validate_config) ) @@ -152,6 +178,8 @@ async def to_code(config): cg.add(var.set_clock_resolution(config[CONF_CLOCK_RESOLUTION])) if CONF_FILTER_SYMBOLS in config: cg.add(var.set_filter_symbols(config[CONF_FILTER_SYMBOLS])) + cg.add(var.set_carrier_duty_percent(config[CONF_CARRIER_DUTY_PERCENT])) + cg.add(var.set_carrier_frequency(config[CONF_CARRIER_FREQUENCY])) else: var = cg.new_Pvariable(config[CONF_ID], pin) @@ -180,8 +208,8 @@ FILTER_SOURCE_FILES = filter_source_files_from_platform( PlatformFramework.ESP32_ARDUINO, PlatformFramework.ESP32_IDF, }, - "remote_receiver_esp8266.cpp": {PlatformFramework.ESP8266_ARDUINO}, - "remote_receiver_libretiny.cpp": { + "remote_receiver.cpp": { + PlatformFramework.ESP8266_ARDUINO, PlatformFramework.BK72XX_ARDUINO, PlatformFramework.RTL87XX_ARDUINO, PlatformFramework.LN882X_ARDUINO, diff --git a/esphome/components/remote_receiver/remote_receiver_esp8266.cpp b/esphome/components/remote_receiver/remote_receiver.cpp similarity index 97% rename from esphome/components/remote_receiver/remote_receiver_esp8266.cpp rename to esphome/components/remote_receiver/remote_receiver.cpp index fe935ba227..a8438e20d7 100644 --- a/esphome/components/remote_receiver/remote_receiver_esp8266.cpp +++ b/esphome/components/remote_receiver/remote_receiver.cpp @@ -3,12 +3,12 @@ #include "esphome/core/helpers.h" #include "esphome/core/log.h" -#ifdef USE_ESP8266 +#if defined(USE_LIBRETINY) || defined(USE_ESP8266) namespace esphome { namespace remote_receiver { -static const char *const TAG = "remote_receiver.esp8266"; +static const char *const TAG = "remote_receiver"; void IRAM_ATTR HOT RemoteReceiverComponentStore::gpio_intr(RemoteReceiverComponentStore *arg) { const uint32_t now = micros(); @@ -31,7 +31,6 @@ void IRAM_ATTR HOT RemoteReceiverComponentStore::gpio_intr(RemoteReceiverCompone } void RemoteReceiverComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->pin_->setup(); auto &s = this->store_; s.filter_us = this->filter_us_; diff --git a/esphome/components/remote_receiver/remote_receiver.h b/esphome/components/remote_receiver/remote_receiver.h index 45e06e664a..3ddcf353c7 100644 --- a/esphome/components/remote_receiver/remote_receiver.h +++ b/esphome/components/remote_receiver/remote_receiver.h @@ -64,6 +64,8 @@ class RemoteReceiverComponent : public remote_base::RemoteReceiverBase, void set_filter_symbols(uint32_t filter_symbols) { this->filter_symbols_ = filter_symbols; } void set_receive_symbols(uint32_t receive_symbols) { this->receive_symbols_ = receive_symbols; } void set_with_dma(bool with_dma) { this->with_dma_ = with_dma; } + void set_carrier_duty_percent(uint8_t carrier_duty_percent) { this->carrier_duty_percent_ = carrier_duty_percent; } + void set_carrier_frequency(uint32_t carrier_frequency) { this->carrier_frequency_ = carrier_frequency; } #endif void set_buffer_size(uint32_t buffer_size) { this->buffer_size_ = buffer_size; } void set_filter_us(uint32_t filter_us) { this->filter_us_ = filter_us; } @@ -76,6 +78,8 @@ class RemoteReceiverComponent : public remote_base::RemoteReceiverBase, uint32_t filter_symbols_{0}; uint32_t receive_symbols_{0}; bool with_dma_{false}; + uint32_t carrier_frequency_{0}; + uint8_t carrier_duty_percent_{100}; esp_err_t error_code_{ESP_OK}; std::string error_string_{""}; #endif diff --git a/esphome/components/remote_receiver/remote_receiver_esp32.cpp b/esphome/components/remote_receiver/remote_receiver_esp32.cpp index 3d6346baec..49358eef3f 100644 --- a/esphome/components/remote_receiver/remote_receiver_esp32.cpp +++ b/esphome/components/remote_receiver/remote_receiver_esp32.cpp @@ -38,7 +38,6 @@ static bool IRAM_ATTR HOT rmt_callback(rmt_channel_handle_t channel, const rmt_r } void RemoteReceiverComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); rmt_rx_channel_config_t channel; memset(&channel, 0, sizeof(channel)); channel.clk_src = RMT_CLK_SRC_DEFAULT; @@ -73,6 +72,21 @@ void RemoteReceiverComponent::setup() { return; } + if (this->carrier_frequency_ > 0 && 0 < this->carrier_duty_percent_ && this->carrier_duty_percent_ < 100) { + rmt_carrier_config_t carrier; + memset(&carrier, 0, sizeof(carrier)); + carrier.frequency_hz = this->carrier_frequency_; + carrier.duty_cycle = (float) this->carrier_duty_percent_ / 100.0f; + carrier.flags.polarity_active_low = this->pin_->is_inverted(); + error = rmt_apply_carrier(this->channel_, &carrier); + if (error != ESP_OK) { + this->error_code_ = error; + this->error_string_ = "in rmt_apply_carrier"; + this->mark_failed(); + return; + } + } + rmt_rx_event_callbacks_t callbacks; memset(&callbacks, 0, sizeof(callbacks)); callbacks.on_recv_done = rmt_callback; @@ -86,10 +100,9 @@ void RemoteReceiverComponent::setup() { uint32_t event_size = sizeof(rmt_rx_done_event_data_t); uint32_t max_filter_ns = 255u * 1000 / (RMT_CLK_FREQ / 1000000); - uint32_t max_idle_ns = 65535u * 1000; memset(&this->store_.config, 0, sizeof(this->store_.config)); this->store_.config.signal_range_min_ns = std::min(this->filter_us_ * 1000, max_filter_ns); - this->store_.config.signal_range_max_ns = std::min(this->idle_us_ * 1000, max_idle_ns); + this->store_.config.signal_range_max_ns = this->idle_us_ * 1000; this->store_.filter_symbols = this->filter_symbols_; this->store_.receive_size = this->receive_symbols_ * sizeof(rmt_symbol_word_t); this->store_.buffer_size = std::max((event_size + this->store_.receive_size) * 2, this->buffer_size_); @@ -113,11 +126,13 @@ void RemoteReceiverComponent::dump_config() { " Filter symbols: %" PRIu32 "\n" " Receive symbols: %" PRIu32 "\n" " Tolerance: %" PRIu32 "%s\n" + " Carrier frequency: %" PRIu32 " hz\n" + " Carrier duty: %u%%\n" " Filter out pulses shorter than: %" PRIu32 " us\n" " Signal is done after %" PRIu32 " us of no changes", this->clock_resolution_, this->rmt_symbols_, this->filter_symbols_, this->receive_symbols_, this->tolerance_, (this->tolerance_mode_ == remote_base::TOLERANCE_MODE_TIME) ? " us" : "%", - this->filter_us_, this->idle_us_); + this->carrier_frequency_, this->carrier_duty_percent_, this->filter_us_, this->idle_us_); if (this->is_failed()) { ESP_LOGE(TAG, "Configuring RMT driver failed: %s (%s)", esp_err_to_name(this->error_code_), this->error_string_.c_str()); diff --git a/esphome/components/remote_receiver/remote_receiver_libretiny.cpp b/esphome/components/remote_receiver/remote_receiver_libretiny.cpp deleted file mode 100644 index 7a6054737e..0000000000 --- a/esphome/components/remote_receiver/remote_receiver_libretiny.cpp +++ /dev/null @@ -1,126 +0,0 @@ -#include "remote_receiver.h" -#include "esphome/core/hal.h" -#include "esphome/core/helpers.h" -#include "esphome/core/log.h" - -#ifdef USE_LIBRETINY - -namespace esphome { -namespace remote_receiver { - -static const char *const TAG = "remote_receiver.libretiny"; - -void IRAM_ATTR HOT RemoteReceiverComponentStore::gpio_intr(RemoteReceiverComponentStore *arg) { - const uint32_t now = micros(); - // If the lhs is 1 (rising edge) we should write to an uneven index and vice versa - const uint32_t next = (arg->buffer_write_at + 1) % arg->buffer_size; - const bool level = arg->pin.digital_read(); - if (level != next % 2) - return; - - // If next is buffer_read, we have hit an overflow - if (next == arg->buffer_read_at) - return; - - const uint32_t last_change = arg->buffer[arg->buffer_write_at]; - const uint32_t time_since_change = now - last_change; - if (time_since_change <= arg->filter_us) - return; - - arg->buffer[arg->buffer_write_at = next] = now; -} - -void RemoteReceiverComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->pin_->setup(); - auto &s = this->store_; - s.filter_us = this->filter_us_; - s.pin = this->pin_->to_isr(); - s.buffer_size = this->buffer_size_; - - this->high_freq_.start(); - if (s.buffer_size % 2 != 0) { - // Make sure divisible by two. This way, we know that every 0bxxx0 index is a space and every 0bxxx1 index is a mark - s.buffer_size++; - } - - s.buffer = new uint32_t[s.buffer_size]; - void *buf = (void *) s.buffer; - memset(buf, 0, s.buffer_size * sizeof(uint32_t)); - - // First index is a space. - if (this->pin_->digital_read()) { - s.buffer_write_at = s.buffer_read_at = 1; - } else { - s.buffer_write_at = s.buffer_read_at = 0; - } - this->pin_->attach_interrupt(RemoteReceiverComponentStore::gpio_intr, &this->store_, gpio::INTERRUPT_ANY_EDGE); -} -void RemoteReceiverComponent::dump_config() { - ESP_LOGCONFIG(TAG, "Remote Receiver:"); - LOG_PIN(" Pin: ", this->pin_); - if (this->pin_->digital_read()) { - ESP_LOGW(TAG, "Remote Receiver Signal starts with a HIGH value. Usually this means you have to " - "invert the signal using 'inverted: True' in the pin schema!"); - } - ESP_LOGCONFIG(TAG, - " Buffer Size: %u\n" - " Tolerance: %u%s\n" - " Filter out pulses shorter than: %u us\n" - " Signal is done after %u us of no changes", - this->buffer_size_, this->tolerance_, - (this->tolerance_mode_ == remote_base::TOLERANCE_MODE_TIME) ? " us" : "%", this->filter_us_, - this->idle_us_); -} - -void RemoteReceiverComponent::loop() { - auto &s = this->store_; - - // copy write at to local variables, as it's volatile - const uint32_t write_at = s.buffer_write_at; - const uint32_t dist = (s.buffer_size + write_at - s.buffer_read_at) % s.buffer_size; - // signals must at least one rising and one leading edge - if (dist <= 1) - return; - const uint32_t now = micros(); - if (now - s.buffer[write_at] < this->idle_us_) { - // The last change was fewer than the configured idle time ago. - return; - } - - ESP_LOGVV(TAG, "read_at=%u write_at=%u dist=%u now=%u end=%u", s.buffer_read_at, write_at, dist, now, - s.buffer[write_at]); - - // Skip first value, it's from the previous idle level - s.buffer_read_at = (s.buffer_read_at + 1) % s.buffer_size; - uint32_t prev = s.buffer_read_at; - s.buffer_read_at = (s.buffer_read_at + 1) % s.buffer_size; - const uint32_t reserve_size = 1 + (s.buffer_size + write_at - s.buffer_read_at) % s.buffer_size; - this->temp_.clear(); - this->temp_.reserve(reserve_size); - int32_t multiplier = s.buffer_read_at % 2 == 0 ? 1 : -1; - - for (uint32_t i = 0; prev != write_at; i++) { - int32_t delta = s.buffer[s.buffer_read_at] - s.buffer[prev]; - if (uint32_t(delta) >= this->idle_us_) { - // already found a space longer than idle. There must have been two pulses - break; - } - - ESP_LOGVV(TAG, " i=%u buffer[%u]=%u - buffer[%u]=%u -> %d", i, s.buffer_read_at, s.buffer[s.buffer_read_at], prev, - s.buffer[prev], multiplier * delta); - this->temp_.push_back(multiplier * delta); - prev = s.buffer_read_at; - s.buffer_read_at = (s.buffer_read_at + 1) % s.buffer_size; - multiplier *= -1; - } - s.buffer_read_at = (s.buffer_size + s.buffer_read_at - 1) % s.buffer_size; - this->temp_.push_back(this->idle_us_ * multiplier); - - this->call_listeners_dumpers_(); -} - -} // namespace remote_receiver -} // namespace esphome - -#endif diff --git a/esphome/components/remote_transmitter/__init__.py b/esphome/components/remote_transmitter/__init__.py index 47a46ff56b..cb98c017f1 100644 --- a/esphome/components/remote_transmitter/__init__.py +++ b/esphome/components/remote_transmitter/__init__.py @@ -13,6 +13,7 @@ from esphome.const import ( CONF_PIN, CONF_RMT_SYMBOLS, CONF_USE_DMA, + CONF_VALUE, PlatformFramework, ) from esphome.core import CORE @@ -22,11 +23,17 @@ AUTO_LOAD = ["remote_base"] CONF_EOT_LEVEL = "eot_level" CONF_ON_TRANSMIT = "on_transmit" CONF_ON_COMPLETE = "on_complete" +CONF_TRANSMITTER_ID = remote_base.CONF_TRANSMITTER_ID remote_transmitter_ns = cg.esphome_ns.namespace("remote_transmitter") RemoteTransmitterComponent = remote_transmitter_ns.class_( "RemoteTransmitterComponent", remote_base.RemoteTransmitterBase, cg.Component ) +DigitalWriteAction = remote_transmitter_ns.class_( + "DigitalWriteAction", + automation.Action, + cg.Parented.template(RemoteTransmitterComponent), +) MULTI_CONF = True CONFIG_SCHEMA = cv.Schema( @@ -63,6 +70,25 @@ CONFIG_SCHEMA = cv.Schema( } ).extend(cv.COMPONENT_SCHEMA) +DIGITAL_WRITE_ACTION_SCHEMA = cv.maybe_simple_value( + { + cv.GenerateID(CONF_TRANSMITTER_ID): cv.use_id(RemoteTransmitterComponent), + cv.Required(CONF_VALUE): cv.templatable(cv.boolean), + }, + key=CONF_VALUE, +) + + +@automation.register_action( + "remote_transmitter.digital_write", DigitalWriteAction, DIGITAL_WRITE_ACTION_SCHEMA +) +async def digital_write_action_to_code(config, action_id, template_arg, args): + var = cg.new_Pvariable(action_id, template_arg) + await cg.register_parented(var, config[CONF_TRANSMITTER_ID]) + template_ = await cg.templatable(config[CONF_VALUE], args, bool) + cg.add(var.set_value(template_)) + return var + async def to_code(config): pin = await cg.gpio_pin_expression(config[CONF_PIN]) @@ -105,8 +131,8 @@ FILTER_SOURCE_FILES = filter_source_files_from_platform( PlatformFramework.ESP32_ARDUINO, PlatformFramework.ESP32_IDF, }, - "remote_transmitter_esp8266.cpp": {PlatformFramework.ESP8266_ARDUINO}, - "remote_transmitter_libretiny.cpp": { + "remote_transmitter.cpp": { + PlatformFramework.ESP8266_ARDUINO, PlatformFramework.BK72XX_ARDUINO, PlatformFramework.RTL87XX_ARDUINO, PlatformFramework.LN882X_ARDUINO, diff --git a/esphome/components/remote_transmitter/automation.h b/esphome/components/remote_transmitter/automation.h new file mode 100644 index 0000000000..75b017ec61 --- /dev/null +++ b/esphome/components/remote_transmitter/automation.h @@ -0,0 +1,18 @@ +#pragma once + +#include "esphome/components/remote_transmitter/remote_transmitter.h" +#include "esphome/core/automation.h" +#include "esphome/core/component.h" +#include "esphome/core/helpers.h" + +namespace esphome { +namespace remote_transmitter { + +template class DigitalWriteAction : public Action, public Parented { + public: + TEMPLATABLE_VALUE(bool, value) + void play(Ts... x) override { this->parent_->digital_write(this->value_.value(x...)); } +}; + +} // namespace remote_transmitter +} // namespace esphome diff --git a/esphome/components/remote_transmitter/remote_transmitter.cpp b/esphome/components/remote_transmitter/remote_transmitter.cpp index 425418ff39..347e9d9d33 100644 --- a/esphome/components/remote_transmitter/remote_transmitter.cpp +++ b/esphome/components/remote_transmitter/remote_transmitter.cpp @@ -2,10 +2,113 @@ #include "esphome/core/log.h" #include "esphome/core/application.h" +#if defined(USE_LIBRETINY) || defined(USE_ESP8266) + namespace esphome { namespace remote_transmitter { static const char *const TAG = "remote_transmitter"; +void RemoteTransmitterComponent::setup() { + this->pin_->setup(); + this->pin_->digital_write(false); +} + +void RemoteTransmitterComponent::dump_config() { + ESP_LOGCONFIG(TAG, + "Remote Transmitter:\n" + " Carrier Duty: %u%%", + this->carrier_duty_percent_); + LOG_PIN(" Pin: ", this->pin_); +} + +void RemoteTransmitterComponent::calculate_on_off_time_(uint32_t carrier_frequency, uint32_t *on_time_period, + uint32_t *off_time_period) { + if (carrier_frequency == 0) { + *on_time_period = 0; + *off_time_period = 0; + return; + } + uint32_t period = (1000000UL + carrier_frequency / 2) / carrier_frequency; // round(1000000/freq) + period = std::max(uint32_t(1), period); + *on_time_period = (period * this->carrier_duty_percent_) / 100; + *off_time_period = period - *on_time_period; +} + +void RemoteTransmitterComponent::await_target_time_() { + const uint32_t current_time = micros(); + if (this->target_time_ == 0) { + this->target_time_ = current_time; + } else if ((int32_t) (this->target_time_ - current_time) > 0) { +#if defined(USE_LIBRETINY) + // busy loop for libretiny is required (see the comment inside micros() in wiring.c) + while ((int32_t) (this->target_time_ - micros()) > 0) + ; +#else + delayMicroseconds(this->target_time_ - current_time); +#endif + } +} + +void RemoteTransmitterComponent::mark_(uint32_t on_time, uint32_t off_time, uint32_t usec) { + this->await_target_time_(); + this->pin_->digital_write(true); + + const uint32_t target = this->target_time_ + usec; + if (this->carrier_duty_percent_ < 100 && (on_time > 0 || off_time > 0)) { + while (true) { // Modulate with carrier frequency + this->target_time_ += on_time; + if ((int32_t) (this->target_time_ - target) >= 0) + break; + this->await_target_time_(); + this->pin_->digital_write(false); + + this->target_time_ += off_time; + if ((int32_t) (this->target_time_ - target) >= 0) + break; + this->await_target_time_(); + this->pin_->digital_write(true); + } + } + this->target_time_ = target; +} + +void RemoteTransmitterComponent::space_(uint32_t usec) { + this->await_target_time_(); + this->pin_->digital_write(false); + this->target_time_ += usec; +} + +void RemoteTransmitterComponent::digital_write(bool value) { this->pin_->digital_write(value); } + +void RemoteTransmitterComponent::send_internal(uint32_t send_times, uint32_t send_wait) { + ESP_LOGD(TAG, "Sending remote code"); + uint32_t on_time, off_time; + this->calculate_on_off_time_(this->temp_.get_carrier_frequency(), &on_time, &off_time); + this->target_time_ = 0; + this->transmit_trigger_->trigger(); + for (uint32_t i = 0; i < send_times; i++) { + InterruptLock lock; + for (int32_t item : this->temp_.get_data()) { + if (item > 0) { + const auto length = uint32_t(item); + this->mark_(on_time, off_time, length); + } else { + const auto length = uint32_t(-item); + this->space_(length); + } + App.feed_wdt(); + } + this->await_target_time_(); // wait for duration of last pulse + this->pin_->digital_write(false); + + if (i + 1 < send_times) + this->target_time_ += send_wait; + } + this->complete_trigger_->trigger(); +} + } // namespace remote_transmitter } // namespace esphome + +#endif diff --git a/esphome/components/remote_transmitter/remote_transmitter.h b/esphome/components/remote_transmitter/remote_transmitter.h index f0dab2aaf8..aa1f54911d 100644 --- a/esphome/components/remote_transmitter/remote_transmitter.h +++ b/esphome/components/remote_transmitter/remote_transmitter.h @@ -30,10 +30,11 @@ class RemoteTransmitterComponent : public remote_base::RemoteTransmitterBase, void set_carrier_duty_percent(uint8_t carrier_duty_percent) { this->carrier_duty_percent_ = carrier_duty_percent; } + void digital_write(bool value); + #if defined(USE_ESP32) void set_with_dma(bool with_dma) { this->with_dma_ = with_dma; } void set_eot_level(bool eot_level) { this->eot_level_ = eot_level; } - void digital_write(bool value); #endif Trigger<> *get_transmit_trigger() const { return this->transmit_trigger_; }; diff --git a/esphome/components/remote_transmitter/remote_transmitter_esp32.cpp b/esphome/components/remote_transmitter/remote_transmitter_esp32.cpp index 411e380670..119aa81e7e 100644 --- a/esphome/components/remote_transmitter/remote_transmitter_esp32.cpp +++ b/esphome/components/remote_transmitter/remote_transmitter_esp32.cpp @@ -11,7 +11,6 @@ namespace remote_transmitter { static const char *const TAG = "remote_transmitter"; void RemoteTransmitterComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->inverted_ = this->pin_->is_inverted(); this->configure_rmt_(); } diff --git a/esphome/components/remote_transmitter/remote_transmitter_esp8266.cpp b/esphome/components/remote_transmitter/remote_transmitter_esp8266.cpp deleted file mode 100644 index 73a1a7754f..0000000000 --- a/esphome/components/remote_transmitter/remote_transmitter_esp8266.cpp +++ /dev/null @@ -1,105 +0,0 @@ -#include "remote_transmitter.h" -#include "esphome/core/log.h" -#include "esphome/core/application.h" - -#ifdef USE_ESP8266 - -namespace esphome { -namespace remote_transmitter { - -static const char *const TAG = "remote_transmitter"; - -void RemoteTransmitterComponent::setup() { - this->pin_->setup(); - this->pin_->digital_write(false); -} - -void RemoteTransmitterComponent::dump_config() { - ESP_LOGCONFIG(TAG, - "Remote Transmitter:\n" - " Carrier Duty: %u%%", - this->carrier_duty_percent_); - LOG_PIN(" Pin: ", this->pin_); -} - -void RemoteTransmitterComponent::calculate_on_off_time_(uint32_t carrier_frequency, uint32_t *on_time_period, - uint32_t *off_time_period) { - if (carrier_frequency == 0) { - *on_time_period = 0; - *off_time_period = 0; - return; - } - uint32_t period = (1000000UL + carrier_frequency / 2) / carrier_frequency; // round(1000000/freq) - period = std::max(uint32_t(1), period); - *on_time_period = (period * this->carrier_duty_percent_) / 100; - *off_time_period = period - *on_time_period; -} - -void RemoteTransmitterComponent::await_target_time_() { - const uint32_t current_time = micros(); - if (this->target_time_ == 0) { - this->target_time_ = current_time; - } else if ((int32_t) (this->target_time_ - current_time) > 0) { - delayMicroseconds(this->target_time_ - current_time); - } -} - -void RemoteTransmitterComponent::mark_(uint32_t on_time, uint32_t off_time, uint32_t usec) { - this->await_target_time_(); - this->pin_->digital_write(true); - - const uint32_t target = this->target_time_ + usec; - if (this->carrier_duty_percent_ < 100 && (on_time > 0 || off_time > 0)) { - while (true) { // Modulate with carrier frequency - this->target_time_ += on_time; - if ((int32_t) (this->target_time_ - target) >= 0) - break; - this->await_target_time_(); - this->pin_->digital_write(false); - - this->target_time_ += off_time; - if ((int32_t) (this->target_time_ - target) >= 0) - break; - this->await_target_time_(); - this->pin_->digital_write(true); - } - } - this->target_time_ = target; -} - -void RemoteTransmitterComponent::space_(uint32_t usec) { - this->await_target_time_(); - this->pin_->digital_write(false); - this->target_time_ += usec; -} - -void RemoteTransmitterComponent::send_internal(uint32_t send_times, uint32_t send_wait) { - ESP_LOGD(TAG, "Sending remote code"); - uint32_t on_time, off_time; - this->calculate_on_off_time_(this->temp_.get_carrier_frequency(), &on_time, &off_time); - this->target_time_ = 0; - this->transmit_trigger_->trigger(); - for (uint32_t i = 0; i < send_times; i++) { - for (int32_t item : this->temp_.get_data()) { - if (item > 0) { - const auto length = uint32_t(item); - this->mark_(on_time, off_time, length); - } else { - const auto length = uint32_t(-item); - this->space_(length); - } - App.feed_wdt(); - } - this->await_target_time_(); // wait for duration of last pulse - this->pin_->digital_write(false); - - if (i + 1 < send_times) - this->target_time_ += send_wait; - } - this->complete_trigger_->trigger(); -} - -} // namespace remote_transmitter -} // namespace esphome - -#endif diff --git a/esphome/components/remote_transmitter/remote_transmitter_libretiny.cpp b/esphome/components/remote_transmitter/remote_transmitter_libretiny.cpp deleted file mode 100644 index 42bf5bd95b..0000000000 --- a/esphome/components/remote_transmitter/remote_transmitter_libretiny.cpp +++ /dev/null @@ -1,108 +0,0 @@ -#include "remote_transmitter.h" -#include "esphome/core/log.h" -#include "esphome/core/application.h" - -#ifdef USE_LIBRETINY - -namespace esphome { -namespace remote_transmitter { - -static const char *const TAG = "remote_transmitter"; - -void RemoteTransmitterComponent::setup() { - this->pin_->setup(); - this->pin_->digital_write(false); -} - -void RemoteTransmitterComponent::dump_config() { - ESP_LOGCONFIG(TAG, - "Remote Transmitter:\n" - " Carrier Duty: %u%%", - this->carrier_duty_percent_); - LOG_PIN(" Pin: ", this->pin_); -} - -void RemoteTransmitterComponent::calculate_on_off_time_(uint32_t carrier_frequency, uint32_t *on_time_period, - uint32_t *off_time_period) { - if (carrier_frequency == 0) { - *on_time_period = 0; - *off_time_period = 0; - return; - } - uint32_t period = (1000000UL + carrier_frequency / 2) / carrier_frequency; // round(1000000/freq) - period = std::max(uint32_t(1), period); - *on_time_period = (period * this->carrier_duty_percent_) / 100; - *off_time_period = period - *on_time_period; -} - -void RemoteTransmitterComponent::await_target_time_() { - const uint32_t current_time = micros(); - if (this->target_time_ == 0) { - this->target_time_ = current_time; - } else { - while ((int32_t) (this->target_time_ - micros()) > 0) { - // busy loop that ensures micros is constantly called - } - } -} - -void RemoteTransmitterComponent::mark_(uint32_t on_time, uint32_t off_time, uint32_t usec) { - this->await_target_time_(); - this->pin_->digital_write(true); - - const uint32_t target = this->target_time_ + usec; - if (this->carrier_duty_percent_ < 100 && (on_time > 0 || off_time > 0)) { - while (true) { // Modulate with carrier frequency - this->target_time_ += on_time; - if ((int32_t) (this->target_time_ - target) >= 0) - break; - this->await_target_time_(); - this->pin_->digital_write(false); - - this->target_time_ += off_time; - if ((int32_t) (this->target_time_ - target) >= 0) - break; - this->await_target_time_(); - this->pin_->digital_write(true); - } - } - this->target_time_ = target; -} - -void RemoteTransmitterComponent::space_(uint32_t usec) { - this->await_target_time_(); - this->pin_->digital_write(false); - this->target_time_ += usec; -} - -void RemoteTransmitterComponent::send_internal(uint32_t send_times, uint32_t send_wait) { - ESP_LOGD(TAG, "Sending remote code"); - uint32_t on_time, off_time; - this->calculate_on_off_time_(this->temp_.get_carrier_frequency(), &on_time, &off_time); - this->target_time_ = 0; - this->transmit_trigger_->trigger(); - for (uint32_t i = 0; i < send_times; i++) { - InterruptLock lock; - for (int32_t item : this->temp_.get_data()) { - if (item > 0) { - const auto length = uint32_t(item); - this->mark_(on_time, off_time, length); - } else { - const auto length = uint32_t(-item); - this->space_(length); - } - App.feed_wdt(); - } - this->await_target_time_(); // wait for duration of last pulse - this->pin_->digital_write(false); - - if (i + 1 < send_times) - this->target_time_ += send_wait; - } - this->complete_trigger_->trigger(); -} - -} // namespace remote_transmitter -} // namespace esphome - -#endif diff --git a/esphome/components/resampler/speaker/__init__.py b/esphome/components/resampler/speaker/__init__.py index 9e9b32476f..def62547b2 100644 --- a/esphome/components/resampler/speaker/__init__.py +++ b/esphome/components/resampler/speaker/__init__.py @@ -90,11 +90,10 @@ async def to_code(config): if task_stack_in_psram := config.get(CONF_TASK_STACK_IN_PSRAM): cg.add(var.set_task_stack_in_psram(task_stack_in_psram)) - if task_stack_in_psram: - if config[CONF_TASK_STACK_IN_PSRAM]: - esp32.add_idf_sdkconfig_option( - "CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY", True - ) + if task_stack_in_psram and config[CONF_TASK_STACK_IN_PSRAM]: + esp32.add_idf_sdkconfig_option( + "CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY", True + ) cg.add(var.set_target_bits_per_sample(config[CONF_BITS_PER_SAMPLE])) cg.add(var.set_target_sample_rate(config[CONF_SAMPLE_RATE])) diff --git a/esphome/components/rf_bridge/__init__.py b/esphome/components/rf_bridge/__init__.py index 5ccca823de..b4770726b4 100644 --- a/esphome/components/rf_bridge/__init__.py +++ b/esphome/components/rf_bridge/__init__.py @@ -136,8 +136,7 @@ RFBRIDGE_ID_SCHEMA = cv.Schema({cv.GenerateID(): cv.use_id(RFBridgeComponent)}) @automation.register_action("rf_bridge.learn", RFBridgeLearnAction, RFBRIDGE_ID_SCHEMA) async def rf_bridge_learnx_to_code(config, action_id, template_args, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_args, paren) - return var + return cg.new_Pvariable(action_id, template_args, paren) @automation.register_action( @@ -149,8 +148,7 @@ async def rf_bridge_start_advanced_sniffing_to_code( config, action_id, template_args, args ): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_args, paren) - return var + return cg.new_Pvariable(action_id, template_args, paren) @automation.register_action( @@ -162,8 +160,7 @@ async def rf_bridge_stop_advanced_sniffing_to_code( config, action_id, template_args, args ): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_args, paren) - return var + return cg.new_Pvariable(action_id, template_args, paren) @automation.register_action( @@ -175,8 +172,7 @@ async def rf_bridge_start_bucket_sniffing_to_code( config, action_id, template_args, args ): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_args, paren) - return var + return cg.new_Pvariable(action_id, template_args, paren) RFBRIDGE_SEND_ADVANCED_CODE_SCHEMA = cv.Schema( diff --git a/esphome/components/rotary_encoder/rotary_encoder.cpp b/esphome/components/rotary_encoder/rotary_encoder.cpp index 79bc123597..26e20664f2 100644 --- a/esphome/components/rotary_encoder/rotary_encoder.cpp +++ b/esphome/components/rotary_encoder/rotary_encoder.cpp @@ -129,12 +129,10 @@ void IRAM_ATTR HOT RotaryEncoderSensorStore::gpio_intr(RotaryEncoderSensorStore } void RotaryEncoderSensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); - int32_t initial_value = 0; switch (this->restore_mode_) { case ROTARY_ENCODER_RESTORE_DEFAULT_ZERO: - this->rtc_ = global_preferences->make_preference(this->get_object_id_hash()); + this->rtc_ = global_preferences->make_preference(this->get_preference_hash()); if (!this->rtc_.load(&initial_value)) { initial_value = 0; } diff --git a/esphome/components/rp2040/__init__.py b/esphome/components/rp2040/__init__.py index 11ed97831e..3a1ea16fa3 100644 --- a/esphome/components/rp2040/__init__.py +++ b/esphome/components/rp2040/__init__.py @@ -1,5 +1,5 @@ import logging -import os +from pathlib import Path from string import ascii_letters, digits import esphome.codegen as cg @@ -16,9 +16,10 @@ from esphome.const import ( KEY_TARGET_FRAMEWORK, KEY_TARGET_PLATFORM, PLATFORM_RP2040, + ThreadModel, ) -from esphome.core import CORE, EsphomeError, coroutine_with_priority -from esphome.helpers import copy_file_if_changed, mkdir_p, read_file, write_file +from esphome.core import CORE, CoroPriority, EsphomeError, coroutine_with_priority +from esphome.helpers import copy_file_if_changed, read_file, write_file_if_changed from .const import KEY_BOARD, KEY_PIO_FILES, KEY_RP2040, rp2040_ns @@ -158,7 +159,7 @@ CONFIG_SCHEMA = cv.All( ) -@coroutine_with_priority(1000) +@coroutine_with_priority(CoroPriority.PLATFORM) async def to_code(config): cg.add(rp2040_ns.setup_preferences()) @@ -171,6 +172,7 @@ async def to_code(config): cg.set_cpp_standard("gnu++20") cg.add_define("ESPHOME_BOARD", config[CONF_BOARD]) cg.add_define("ESPHOME_VARIANT", "RP2040") + cg.add_define(ThreadModel.SINGLE) cg.add_platformio_option("extra_scripts", ["post:post_build.py"]) @@ -204,7 +206,7 @@ def add_pio_file(component: str, key: str, data: str): cv.validate_id_name(key) except cv.Invalid as e: raise EsphomeError( - f"[{component}] Invalid PIO key: {key}. Allowed characters: [{ascii_letters}{digits}_]\nPlease report an issue https://github.com/esphome/issues" + f"[{component}] Invalid PIO key: {key}. Allowed characters: [{ascii_letters}{digits}_]\nPlease report an issue https://github.com/esphome/esphome/issues" ) from e CORE.data[KEY_RP2040][KEY_PIO_FILES][key] = data @@ -219,18 +221,18 @@ def generate_pio_files() -> bool: if not files: return False for key, data in files.items(): - pio_path = CORE.relative_build_path(f"src/pio/{key}.pio") - mkdir_p(os.path.dirname(pio_path)) - write_file(pio_path, data) + pio_path = CORE.build_path / "src" / "pio" / f"{key}.pio" + pio_path.parent.mkdir(parents=True, exist_ok=True) + write_file_if_changed(pio_path, data) includes.append(f"pio/{key}.pio.h") - write_file( + write_file_if_changed( CORE.relative_build_path("src/pio_includes.h"), "#pragma once\n" + "\n".join([f'#include "{include}"' for include in includes]), ) - dir = os.path.dirname(__file__) - build_pio_file = os.path.join(dir, "build_pio.py.script") + dir = Path(__file__).parent + build_pio_file = dir / "build_pio.py.script" copy_file_if_changed( build_pio_file, CORE.relative_build_path("build_pio.py"), @@ -241,8 +243,8 @@ def generate_pio_files() -> bool: # Called by writer.py def copy_files(): - dir = os.path.dirname(__file__) - post_build_file = os.path.join(dir, "post_build.py.script") + dir = Path(__file__).parent + post_build_file = dir / "post_build.py.script" copy_file_if_changed( post_build_file, CORE.relative_build_path("post_build.py"), @@ -250,4 +252,4 @@ def copy_files(): if generate_pio_files(): path = CORE.relative_src_path("esphome.h") content = read_file(path).rstrip("\n") - write_file(path, content + '\n#include "pio_includes.h"\n') + write_file_if_changed(path, content + '\n#include "pio_includes.h"\n') diff --git a/esphome/components/rp2040/helpers.cpp b/esphome/components/rp2040/helpers.cpp index a6eac58dc6..30b40a723a 100644 --- a/esphome/components/rp2040/helpers.cpp +++ b/esphome/components/rp2040/helpers.cpp @@ -44,6 +44,10 @@ void Mutex::unlock() {} IRAM_ATTR InterruptLock::InterruptLock() { state_ = save_and_disable_interrupts(); } IRAM_ATTR InterruptLock::~InterruptLock() { restore_interrupts(state_); } +// RP2040 doesn't support lwIP core locking, so this is a no-op +LwIPLock::LwIPLock() {} +LwIPLock::~LwIPLock() {} + void get_mac_address_raw(uint8_t *mac) { // NOLINT(readability-non-const-parameter) #ifdef USE_WIFI WiFi.macAddress(mac); diff --git a/esphome/components/rp2040_pio_led_strip/led_strip.cpp b/esphome/components/rp2040_pio_led_strip/led_strip.cpp index 42f7e9cf52..dc0d3c315a 100644 --- a/esphome/components/rp2040_pio_led_strip/led_strip.cpp +++ b/esphome/components/rp2040_pio_led_strip/led_strip.cpp @@ -40,8 +40,6 @@ void RP2040PIOLEDStripLightOutput::dma_write_complete_handler_() { } void RP2040PIOLEDStripLightOutput::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - size_t buffer_size = this->get_buffer_size_(); RAMAllocator allocator; diff --git a/esphome/components/rp2040_pio_led_strip/light.py b/esphome/components/rp2040_pio_led_strip/light.py index 9107db9b7f..62f7fffdc9 100644 --- a/esphome/components/rp2040_pio_led_strip/light.py +++ b/esphome/components/rp2040_pio_led_strip/light.py @@ -125,8 +125,7 @@ writezero: def time_to_cycles(time_us): cycles_per_us = 57.5 - cycles = round(float(time_us) * cycles_per_us) - return cycles + return round(float(time_us) * cycles_per_us) CONF_PIO = "pio" diff --git a/esphome/components/rp2040_pwm/rp2040_pwm.cpp b/esphome/components/rp2040_pwm/rp2040_pwm.cpp index 40920f9351..ec164b3c05 100644 --- a/esphome/components/rp2040_pwm/rp2040_pwm.cpp +++ b/esphome/components/rp2040_pwm/rp2040_pwm.cpp @@ -16,11 +16,7 @@ namespace rp2040_pwm { static const char *const TAG = "rp2040_pwm"; -void RP2040PWM::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - - this->setup_pwm_(); -} +void RP2040PWM::setup() { this->setup_pwm_(); } void RP2040PWM::setup_pwm_() { pwm_config config = pwm_get_default_config(); diff --git a/esphome/components/rpi_dpi_rgb/display.py b/esphome/components/rpi_dpi_rgb/display.py index c26143d63e..513ed8eb58 100644 --- a/esphome/components/rpi_dpi_rgb/display.py +++ b/esphome/components/rpi_dpi_rgb/display.py @@ -2,6 +2,18 @@ from esphome import pins import esphome.codegen as cg from esphome.components import display from esphome.components.esp32 import const, only_on_variant +from esphome.components.mipi import ( + CONF_DE_PIN, + CONF_HSYNC_BACK_PORCH, + CONF_HSYNC_FRONT_PORCH, + CONF_HSYNC_PULSE_WIDTH, + CONF_PCLK_FREQUENCY, + CONF_PCLK_INVERTED, + CONF_PCLK_PIN, + CONF_VSYNC_BACK_PORCH, + CONF_VSYNC_FRONT_PORCH, + CONF_VSYNC_PULSE_WIDTH, +) import esphome.config_validation as cv from esphome.const import ( CONF_BLUE, @@ -27,18 +39,6 @@ from esphome.const import ( DEPENDENCIES = ["esp32"] -CONF_DE_PIN = "de_pin" -CONF_PCLK_PIN = "pclk_pin" - -CONF_HSYNC_FRONT_PORCH = "hsync_front_porch" -CONF_HSYNC_PULSE_WIDTH = "hsync_pulse_width" -CONF_HSYNC_BACK_PORCH = "hsync_back_porch" -CONF_VSYNC_FRONT_PORCH = "vsync_front_porch" -CONF_VSYNC_PULSE_WIDTH = "vsync_pulse_width" -CONF_VSYNC_BACK_PORCH = "vsync_back_porch" -CONF_PCLK_FREQUENCY = "pclk_frequency" -CONF_PCLK_INVERTED = "pclk_inverted" - rpi_dpi_rgb_ns = cg.esphome_ns.namespace("rpi_dpi_rgb") RPI_DPI_RGB = rpi_dpi_rgb_ns.class_("RpiDpiRgb", display.Display, cg.Component) ColorOrder = display.display_ns.enum("ColorMode") @@ -140,7 +140,6 @@ async def to_code(config): cg.add(var.set_vsync_front_porch(config[CONF_VSYNC_FRONT_PORCH])) cg.add(var.set_pclk_inverted(config[CONF_PCLK_INVERTED])) cg.add(var.set_pclk_frequency(config[CONF_PCLK_FREQUENCY])) - index = 0 dpins = [] if CONF_RED in config[CONF_DATA_PINS]: red_pins = config[CONF_DATA_PINS][CONF_RED] @@ -158,10 +157,9 @@ async def to_code(config): dpins = dpins[8:16] + dpins[0:8] else: dpins = config[CONF_DATA_PINS] - for pin in dpins: + for index, pin in enumerate(dpins): data_pin = await cg.gpio_pin_expression(pin) cg.add(var.add_data_pin(data_pin, index)) - index += 1 if enable_pin := config.get(CONF_ENABLE_PIN): enable = await cg.gpio_pin_expression(enable_pin) diff --git a/esphome/components/rpi_dpi_rgb/rpi_dpi_rgb.cpp b/esphome/components/rpi_dpi_rgb/rpi_dpi_rgb.cpp index 91eb947a3e..042b8877e6 100644 --- a/esphome/components/rpi_dpi_rgb/rpi_dpi_rgb.cpp +++ b/esphome/components/rpi_dpi_rgb/rpi_dpi_rgb.cpp @@ -6,7 +6,6 @@ namespace esphome { namespace rpi_dpi_rgb { void RpiDpiRgb::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->reset_display_(); esp_lcd_rgb_panel_config_t config{}; config.flags.fb_in_psram = 1; @@ -23,7 +22,6 @@ void RpiDpiRgb::setup() { config.timings.flags.pclk_active_neg = this->pclk_inverted_; config.timings.pclk_hz = this->pclk_frequency_; config.clk_src = LCD_CLK_SRC_PLL160M; - config.psram_trans_align = 64; size_t data_pin_count = sizeof(this->data_pins_) / sizeof(this->data_pins_[0]); for (size_t i = 0; i != data_pin_count; i++) { config.data_gpio_nums[i] = this->data_pins_[i]->get_pin(); @@ -42,7 +40,6 @@ void RpiDpiRgb::setup() { } ESP_ERROR_CHECK(esp_lcd_panel_reset(this->handle_)); ESP_ERROR_CHECK(esp_lcd_panel_init(this->handle_)); - ESP_LOGCONFIG(TAG, "RPI_DPI_RGB setup complete"); } void RpiDpiRgb::loop() { if (this->handle_ != nullptr) diff --git a/esphome/components/rtttl/rtttl.cpp b/esphome/components/rtttl/rtttl.cpp index 65a3af1bbc..b79f27e2e5 100644 --- a/esphome/components/rtttl/rtttl.cpp +++ b/esphome/components/rtttl/rtttl.cpp @@ -138,11 +138,37 @@ void Rtttl::stop() { this->set_state_(STATE_STOPPING); } #endif + this->position_ = this->rtttl_.length(); + this->note_duration_ = 0; +} + +void Rtttl::finish_() { + ESP_LOGV(TAG, "Rtttl::finish_()"); +#ifdef USE_OUTPUT + if (this->output_ != nullptr) { + this->output_->set_level(0.0); + this->set_state_(State::STATE_STOPPED); + } +#endif +#ifdef USE_SPEAKER + if (this->speaker_ != nullptr) { + SpeakerSample sample[2]; + sample[0].left = 0; + sample[0].right = 0; + sample[1].left = 0; + sample[1].right = 0; + this->speaker_->play((uint8_t *) (&sample), 8); + this->speaker_->finish(); + this->set_state_(State::STATE_STOPPING); + } +#endif + // Ensure no more notes are played in case finish_() is called for an error. + this->position_ = this->rtttl_.length(); this->note_duration_ = 0; } void Rtttl::loop() { - if (this->note_duration_ == 0 || this->state_ == State::STATE_STOPPED) { + if (this->state_ == State::STATE_STOPPED) { this->disable_loop(); return; } @@ -152,6 +178,8 @@ void Rtttl::loop() { if (this->state_ == State::STATE_STOPPING) { if (this->speaker_->is_stopped()) { this->set_state_(State::STATE_STOPPED); + } else { + return; } } else if (this->state_ == State::STATE_INIT) { if (this->speaker_->is_stopped()) { @@ -187,7 +215,7 @@ void Rtttl::loop() { sample[x].right = 0; } - if (x >= SAMPLE_BUFFER_SIZE || this->samples_sent_ >= this->samples_count_) { + if (static_cast(x) >= SAMPLE_BUFFER_SIZE || this->samples_sent_ >= this->samples_count_) { break; } this->samples_sent_++; @@ -207,7 +235,7 @@ void Rtttl::loop() { if (this->output_ != nullptr && millis() - this->last_note_ < this->note_duration_) return; #endif - if (!this->rtttl_[this->position_]) { + if (this->position_ >= this->rtttl_.length()) { this->finish_(); return; } @@ -346,32 +374,7 @@ void Rtttl::loop() { this->last_note_ = millis(); } -void Rtttl::finish_() { -#ifdef USE_OUTPUT - if (this->output_ != nullptr) { - this->output_->set_level(0.0); - this->set_state_(State::STATE_STOPPED); - } -#endif -#ifdef USE_SPEAKER - if (this->speaker_ != nullptr) { - SpeakerSample sample[2]; - sample[0].left = 0; - sample[0].right = 0; - sample[1].left = 0; - sample[1].right = 0; - this->speaker_->play((uint8_t *) (&sample), 8); - - this->speaker_->finish(); - this->set_state_(State::STATE_STOPPING); - } -#endif - this->note_duration_ = 0; - this->on_finished_playback_callback_.call(); - ESP_LOGD(TAG, "Playback finished"); -} - -#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_DEBUG +#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE static const LogString *state_to_string(State state) { switch (state) { case STATE_STOPPED: @@ -397,7 +400,11 @@ void Rtttl::set_state_(State state) { LOG_STR_ARG(state_to_string(state))); // Clear loop_done when transitioning from STOPPED to any other state - if (old_state == State::STATE_STOPPED && state != State::STATE_STOPPED) { + if (state == State::STATE_STOPPED) { + this->disable_loop(); + this->on_finished_playback_callback_.call(); + ESP_LOGD(TAG, "Playback finished"); + } else if (old_state == State::STATE_STOPPED) { this->enable_loop(); } } diff --git a/esphome/components/rtttl/rtttl.h b/esphome/components/rtttl/rtttl.h index 420948bfbf..d536c6c08e 100644 --- a/esphome/components/rtttl/rtttl.h +++ b/esphome/components/rtttl/rtttl.h @@ -60,35 +60,60 @@ class Rtttl : public Component { } return ret; } + /** + * @brief Finalizes the playback of the RTTTL string. + * + * This method is called internally when the end of the RTTTL string is reached + * or when a parsing error occurs. It stops the output, sets the component state, + * and triggers the on_finished_playback_callback_. + */ void finish_(); void set_state_(State state); + /// The RTTTL string to play. std::string rtttl_{""}; + /// The current position in the RTTTL string. size_t position_{0}; + /// The duration of a whole note in milliseconds. uint16_t wholenote_; + /// The default duration of a note (e.g. 4 for a quarter note). uint16_t default_duration_; + /// The default octave for a note. uint16_t default_octave_; + /// The time the last note was started. uint32_t last_note_; + /// The duration of the current note in milliseconds. uint16_t note_duration_; + /// The frequency of the current note in Hz. uint32_t output_freq_; + /// The gain of the output. float gain_{0.6f}; + /// The current state of the RTTTL player. State state_{State::STATE_STOPPED}; #ifdef USE_OUTPUT + /// The output to write the sound to. output::FloatOutput *output_; #endif #ifdef USE_SPEAKER + /// The speaker to write the sound to. speaker::Speaker *speaker_{nullptr}; + /// The sample rate of the speaker. int sample_rate_{16000}; + /// The number of samples for one full cycle of a note's waveform, in Q10 fixed-point format. int samples_per_wave_{0}; + /// The number of samples sent. int samples_sent_{0}; + /// The total number of samples to send. int samples_count_{0}; + /// The number of samples for the gap between notes. int samples_gap_{0}; #endif + /// The callback to call when playback is finished. CallbackManager on_finished_playback_callback_; }; diff --git a/esphome/components/runtime_stats/__init__.py b/esphome/components/runtime_stats/__init__.py new file mode 100644 index 0000000000..a36e8bfd28 --- /dev/null +++ b/esphome/components/runtime_stats/__init__.py @@ -0,0 +1,34 @@ +""" +Runtime statistics component for ESPHome. +""" + +import esphome.codegen as cg +import esphome.config_validation as cv +from esphome.const import CONF_ID + +CODEOWNERS = ["@bdraco"] + +CONF_LOG_INTERVAL = "log_interval" + +runtime_stats_ns = cg.esphome_ns.namespace("runtime_stats") +RuntimeStatsCollector = runtime_stats_ns.class_("RuntimeStatsCollector") + +CONFIG_SCHEMA = cv.Schema( + { + cv.GenerateID(): cv.declare_id(RuntimeStatsCollector), + cv.Optional( + CONF_LOG_INTERVAL, default="60s" + ): cv.positive_time_period_milliseconds, + } +) + + +async def to_code(config): + """Generate code for the runtime statistics component.""" + # Define USE_RUNTIME_STATS when this component is used + cg.add_define("USE_RUNTIME_STATS") + + # Create the runtime stats instance (constructor sets global_runtime_stats) + var = cg.new_Pvariable(config[CONF_ID]) + + cg.add(var.set_log_interval(config[CONF_LOG_INTERVAL])) diff --git a/esphome/components/runtime_stats/runtime_stats.cpp b/esphome/components/runtime_stats/runtime_stats.cpp new file mode 100644 index 0000000000..f95be5291f --- /dev/null +++ b/esphome/components/runtime_stats/runtime_stats.cpp @@ -0,0 +1,89 @@ +#include "runtime_stats.h" + +#ifdef USE_RUNTIME_STATS + +#include "esphome/core/component.h" +#include + +namespace esphome { + +namespace runtime_stats { + +RuntimeStatsCollector::RuntimeStatsCollector() : log_interval_(60000), next_log_time_(0) { + global_runtime_stats = this; +} + +void RuntimeStatsCollector::record_component_time(Component *component, uint32_t duration_ms, uint32_t current_time) { + if (component == nullptr) + return; + + // Record stats using component pointer as key + this->component_stats_[component].record_time(duration_ms); + + if (this->next_log_time_ == 0) { + this->next_log_time_ = current_time + this->log_interval_; + return; + } +} + +void RuntimeStatsCollector::log_stats_() { + ESP_LOGI(TAG, "Component Runtime Statistics"); + ESP_LOGI(TAG, "Period stats (last %" PRIu32 "ms):", this->log_interval_); + + // First collect stats we want to display + std::vector stats_to_display; + + for (const auto &it : this->component_stats_) { + Component *component = it.first; + const ComponentRuntimeStats &stats = it.second; + if (stats.get_period_count() > 0) { + ComponentStatPair pair = {component, &stats}; + stats_to_display.push_back(pair); + } + } + + // Sort by period runtime (descending) + std::sort(stats_to_display.begin(), stats_to_display.end(), std::greater()); + + // Log top components by period runtime + for (const auto &it : stats_to_display) { + ESP_LOGI(TAG, " %s: count=%" PRIu32 ", avg=%.2fms, max=%" PRIu32 "ms, total=%" PRIu32 "ms", + LOG_STR_ARG(it.component->get_component_log_str()), it.stats->get_period_count(), + it.stats->get_period_avg_time_ms(), it.stats->get_period_max_time_ms(), it.stats->get_period_time_ms()); + } + + // Log total stats since boot + ESP_LOGI(TAG, "Total stats (since boot):"); + + // Re-sort by total runtime for all-time stats + std::sort(stats_to_display.begin(), stats_to_display.end(), + [](const ComponentStatPair &a, const ComponentStatPair &b) { + return a.stats->get_total_time_ms() > b.stats->get_total_time_ms(); + }); + + for (const auto &it : stats_to_display) { + ESP_LOGI(TAG, " %s: count=%" PRIu32 ", avg=%.2fms, max=%" PRIu32 "ms, total=%" PRIu32 "ms", + LOG_STR_ARG(it.component->get_component_log_str()), it.stats->get_total_count(), + it.stats->get_total_avg_time_ms(), it.stats->get_total_max_time_ms(), it.stats->get_total_time_ms()); + } +} + +void RuntimeStatsCollector::process_pending_stats(uint32_t current_time) { + if (this->next_log_time_ == 0) + return; + + if (current_time >= this->next_log_time_) { + this->log_stats_(); + this->reset_stats_(); + this->next_log_time_ = current_time + this->log_interval_; + } +} + +} // namespace runtime_stats + +runtime_stats::RuntimeStatsCollector *global_runtime_stats = + nullptr; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) + +} // namespace esphome + +#endif // USE_RUNTIME_STATS diff --git a/esphome/components/runtime_stats/runtime_stats.h b/esphome/components/runtime_stats/runtime_stats.h new file mode 100644 index 0000000000..56122364c2 --- /dev/null +++ b/esphome/components/runtime_stats/runtime_stats.h @@ -0,0 +1,126 @@ +#pragma once + +#include "esphome/core/defines.h" + +#ifdef USE_RUNTIME_STATS + +#include +#include +#include +#include +#include "esphome/core/helpers.h" +#include "esphome/core/log.h" + +namespace esphome { + +class Component; // Forward declaration + +namespace runtime_stats { + +static const char *const TAG = "runtime_stats"; + +class ComponentRuntimeStats { + public: + ComponentRuntimeStats() + : period_count_(0), + period_time_ms_(0), + period_max_time_ms_(0), + total_count_(0), + total_time_ms_(0), + total_max_time_ms_(0) {} + + void record_time(uint32_t duration_ms) { + // Update period counters + this->period_count_++; + this->period_time_ms_ += duration_ms; + if (duration_ms > this->period_max_time_ms_) + this->period_max_time_ms_ = duration_ms; + + // Update total counters + this->total_count_++; + this->total_time_ms_ += duration_ms; + if (duration_ms > this->total_max_time_ms_) + this->total_max_time_ms_ = duration_ms; + } + + void reset_period_stats() { + this->period_count_ = 0; + this->period_time_ms_ = 0; + this->period_max_time_ms_ = 0; + } + + // Period stats (reset each logging interval) + uint32_t get_period_count() const { return this->period_count_; } + uint32_t get_period_time_ms() const { return this->period_time_ms_; } + uint32_t get_period_max_time_ms() const { return this->period_max_time_ms_; } + float get_period_avg_time_ms() const { + return this->period_count_ > 0 ? this->period_time_ms_ / static_cast(this->period_count_) : 0.0f; + } + + // Total stats (persistent until reboot) + uint32_t get_total_count() const { return this->total_count_; } + uint32_t get_total_time_ms() const { return this->total_time_ms_; } + uint32_t get_total_max_time_ms() const { return this->total_max_time_ms_; } + float get_total_avg_time_ms() const { + return this->total_count_ > 0 ? this->total_time_ms_ / static_cast(this->total_count_) : 0.0f; + } + + protected: + // Period stats (reset each logging interval) + uint32_t period_count_; + uint32_t period_time_ms_; + uint32_t period_max_time_ms_; + + // Total stats (persistent until reboot) + uint32_t total_count_; + uint32_t total_time_ms_; + uint32_t total_max_time_ms_; +}; + +// For sorting components by run time +struct ComponentStatPair { + Component *component; + const ComponentRuntimeStats *stats; + + bool operator>(const ComponentStatPair &other) const { + // Sort by period time as that's what we're displaying in the logs + return stats->get_period_time_ms() > other.stats->get_period_time_ms(); + } +}; + +class RuntimeStatsCollector { + public: + RuntimeStatsCollector(); + + void set_log_interval(uint32_t log_interval) { this->log_interval_ = log_interval; } + uint32_t get_log_interval() const { return this->log_interval_; } + + void record_component_time(Component *component, uint32_t duration_ms, uint32_t current_time); + + // Process any pending stats printing (should be called after component loop) + void process_pending_stats(uint32_t current_time); + + protected: + void log_stats_(); + + void reset_stats_() { + for (auto &it : this->component_stats_) { + it.second.reset_period_stats(); + } + } + + // Map from component to its stats + // We use Component* as the key since each component is unique + std::map component_stats_; + uint32_t log_interval_; + uint32_t next_log_time_; +}; + +} // namespace runtime_stats + +extern runtime_stats::RuntimeStatsCollector + *global_runtime_stats; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) + +} // namespace esphome + +#endif // USE_RUNTIME_STATS diff --git a/esphome/components/safe_mode/__init__.py b/esphome/components/safe_mode/__init__.py index 991747b089..9944d71722 100644 --- a/esphome/components/safe_mode/__init__.py +++ b/esphome/components/safe_mode/__init__.py @@ -10,7 +10,7 @@ from esphome.const import ( CONF_TRIGGER_ID, KEY_PAST_SAFE_MODE, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.cpp_generator import RawExpression CODEOWNERS = ["@paulmonigatti", "@jsuanet", "@kbx81"] @@ -53,7 +53,7 @@ CONFIG_SCHEMA = cv.All( ) -@coroutine_with_priority(50.0) +@coroutine_with_priority(CoroPriority.APPLICATION) async def to_code(config): if not config[CONF_DISABLED]: var = cg.new_Pvariable(config[CONF_ID]) diff --git a/esphome/components/safe_mode/safe_mode.cpp b/esphome/components/safe_mode/safe_mode.cpp index 5a62604269..62bbca4fb1 100644 --- a/esphome/components/safe_mode/safe_mode.cpp +++ b/esphome/components/safe_mode/safe_mode.cpp @@ -15,11 +15,11 @@ namespace safe_mode { static const char *const TAG = "safe_mode"; void SafeModeComponent::dump_config() { - ESP_LOGCONFIG(TAG, "Safe Mode:"); ESP_LOGCONFIG(TAG, - " Boot considered successful after %" PRIu32 " seconds\n" - " Invoke after %u boot attempts\n" - " Remain for %" PRIu32 " seconds", + "Safe Mode:\n" + " Successful after: %" PRIu32 "s\n" + " Invoke after: %u attempts\n" + " Duration: %" PRIu32 "s", this->safe_mode_boot_is_good_after_ / 1000, // because milliseconds this->safe_mode_num_attempts_, this->safe_mode_enable_time_ / 1000); // because milliseconds @@ -27,7 +27,7 @@ void SafeModeComponent::dump_config() { if (this->safe_mode_rtc_value_ > 1 && this->safe_mode_rtc_value_ != SafeModeComponent::ENTER_SAFE_MODE_MAGIC) { auto remaining_restarts = this->safe_mode_num_attempts_ - this->safe_mode_rtc_value_; if (remaining_restarts) { - ESP_LOGW(TAG, "Last reset occurred too quickly; will be invoked in %" PRIu32 " restarts", remaining_restarts); + ESP_LOGW(TAG, "Last reset too quick; invoke in %" PRIu32 " restarts", remaining_restarts); } else { ESP_LOGW(TAG, "SAFE MODE IS ACTIVE"); } @@ -72,43 +72,45 @@ bool SafeModeComponent::should_enter_safe_mode(uint8_t num_attempts, uint32_t en this->safe_mode_boot_is_good_after_ = boot_is_good_after; this->safe_mode_num_attempts_ = num_attempts; this->rtc_ = global_preferences->make_preference(233825507UL, false); - this->safe_mode_rtc_value_ = this->read_rtc_(); - bool is_manual_safe_mode = this->safe_mode_rtc_value_ == SafeModeComponent::ENTER_SAFE_MODE_MAGIC; + uint32_t rtc_val = this->read_rtc_(); + this->safe_mode_rtc_value_ = rtc_val; - if (is_manual_safe_mode) { - ESP_LOGI(TAG, "Safe mode invoked manually"); + bool is_manual = rtc_val == SafeModeComponent::ENTER_SAFE_MODE_MAGIC; + + if (is_manual) { + ESP_LOGI(TAG, "Manual mode"); } else { - ESP_LOGCONFIG(TAG, "There have been %" PRIu32 " suspected unsuccessful boot attempts", this->safe_mode_rtc_value_); + ESP_LOGCONFIG(TAG, "Unsuccessful boot attempts: %" PRIu32, rtc_val); } - if (this->safe_mode_rtc_value_ >= num_attempts || is_manual_safe_mode) { - this->clean_rtc(); - - if (!is_manual_safe_mode) { - ESP_LOGE(TAG, "Boot loop detected. Proceeding"); - } - - this->status_set_error(); - this->set_timeout(enable_time, []() { - ESP_LOGW(TAG, "Safe mode enable time has elapsed -- restarting"); - App.reboot(); - }); - - // Delay here to allow power to stabilize before Wi-Fi/Ethernet is initialised - delay(300); // NOLINT - App.setup(); - - ESP_LOGW(TAG, "SAFE MODE IS ACTIVE"); - - this->safe_mode_callback_.call(); - - return true; - } else { + if (rtc_val < num_attempts && !is_manual) { // increment counter - this->write_rtc_(this->safe_mode_rtc_value_ + 1); + this->write_rtc_(rtc_val + 1); return false; } + + this->clean_rtc(); + + if (!is_manual) { + ESP_LOGE(TAG, "Boot loop detected"); + } + + this->status_set_error(); + this->set_timeout(enable_time, []() { + ESP_LOGW(TAG, "Timeout, restarting"); + App.reboot(); + }); + + // Delay here to allow power to stabilize before Wi-Fi/Ethernet is initialised + delay(300); // NOLINT + App.setup(); + + ESP_LOGW(TAG, "SAFE MODE IS ACTIVE"); + + this->safe_mode_callback_.call(); + + return true; } void SafeModeComponent::write_rtc_(uint32_t val) { diff --git a/esphome/components/scd30/scd30.cpp b/esphome/components/scd30/scd30.cpp index 8561732d8b..3c2c06fd68 100644 --- a/esphome/components/scd30/scd30.cpp +++ b/esphome/components/scd30/scd30.cpp @@ -26,8 +26,6 @@ static const uint16_t SCD30_CMD_TEMPERATURE_OFFSET = 0x5403; static const uint16_t SCD30_CMD_SOFT_RESET = 0xD304; void SCD30Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - #ifdef USE_ESP8266 Wire.setClockStretchLimit(150000); #endif diff --git a/esphome/components/scd30/sensor.py b/esphome/components/scd30/sensor.py index f341d2a47b..194df8ec4f 100644 --- a/esphome/components/scd30/sensor.py +++ b/esphome/components/scd30/sensor.py @@ -3,6 +3,7 @@ import esphome.codegen as cg from esphome.components import i2c, sensirion_common, sensor import esphome.config_validation as cv from esphome.const import ( + CONF_ALTITUDE_COMPENSATION, CONF_AMBIENT_PRESSURE_COMPENSATION, CONF_AUTOMATIC_SELF_CALIBRATION, CONF_CO2, @@ -35,8 +36,6 @@ ForceRecalibrationWithReference = scd30_ns.class_( "ForceRecalibrationWithReference", automation.Action ) -CONF_ALTITUDE_COMPENSATION = "altitude_compensation" - CONFIG_SCHEMA = ( cv.Schema( { @@ -67,7 +66,7 @@ CONFIG_SCHEMA = ( ), cv.Optional(CONF_AMBIENT_PRESSURE_COMPENSATION, default=0): cv.pressure, cv.Optional(CONF_TEMPERATURE_OFFSET): cv.All( - cv.temperature, + cv.temperature_delta, cv.float_range(min=0, max=655.35), ), cv.Optional(CONF_UPDATE_INTERVAL, default="60s"): cv.All( diff --git a/esphome/components/scd4x/scd4x.cpp b/esphome/components/scd4x/scd4x.cpp index 06db70e3f3..a265386cc2 100644 --- a/esphome/components/scd4x/scd4x.cpp +++ b/esphome/components/scd4x/scd4x.cpp @@ -27,7 +27,6 @@ static const uint16_t SCD4X_CMD_GET_FEATURESET = 0x202f; static const float SCD4X_TEMPERATURE_OFFSET_MULTIPLIER = (1 << 16) / 175.0f; void SCD4XComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // the sensor needs 1000 ms to enter the idle state this->set_timeout(1000, [this]() { this->status_clear_error(); diff --git a/esphome/components/scd4x/sensor.py b/esphome/components/scd4x/sensor.py index fc859d63b8..ec90234ac3 100644 --- a/esphome/components/scd4x/sensor.py +++ b/esphome/components/scd4x/sensor.py @@ -4,6 +4,7 @@ import esphome.codegen as cg from esphome.components import i2c, sensirion_common, sensor import esphome.config_validation as cv from esphome.const import ( + CONF_ALTITUDE_COMPENSATION, CONF_AMBIENT_PRESSURE_COMPENSATION, CONF_AMBIENT_PRESSURE_COMPENSATION_SOURCE, CONF_AUTOMATIC_SELF_CALIBRATION, @@ -49,9 +50,6 @@ PerformForcedCalibrationAction = scd4x_ns.class_( ) FactoryResetAction = scd4x_ns.class_("FactoryResetAction", automation.Action) - -CONF_ALTITUDE_COMPENSATION = "altitude_compensation" - CONFIG_SCHEMA = ( cv.Schema( { @@ -83,7 +81,7 @@ CONFIG_SCHEMA = ( cv.int_range(min=0, max=0xFFFF, max_included=False), ), cv.Optional(CONF_AMBIENT_PRESSURE_COMPENSATION): cv.pressure, - cv.Optional(CONF_TEMPERATURE_OFFSET, default="4°C"): cv.temperature, + cv.Optional(CONF_TEMPERATURE_OFFSET, default="4°C"): cv.temperature_delta, cv.Optional(CONF_AMBIENT_PRESSURE_COMPENSATION_SOURCE): cv.use_id( sensor.Sensor ), diff --git a/esphome/components/script/__init__.py b/esphome/components/script/__init__.py index ee1f6a4ad0..e8a8aa5671 100644 --- a/esphome/components/script/__init__.py +++ b/esphome/components/script/__init__.py @@ -124,7 +124,7 @@ async def to_code(config): template, func_args = parameters_to_template(conf[CONF_PARAMETERS]) trigger = cg.new_Pvariable(conf[CONF_ID], template) # Add a human-readable name to the script - cg.add(trigger.set_name(conf[CONF_ID].id)) + cg.add(trigger.set_name(cg.LogStringLiteral(conf[CONF_ID].id))) if CONF_MAX_RUNS in conf: cg.add(trigger.set_max_runs(conf[CONF_MAX_RUNS])) diff --git a/esphome/components/script/script.cpp b/esphome/components/script/script.cpp index 331f7dcd65..81f652d26a 100644 --- a/esphome/components/script/script.cpp +++ b/esphome/components/script/script.cpp @@ -6,9 +6,15 @@ namespace script { static const char *const TAG = "script"; +#ifdef USE_STORE_LOG_STR_IN_FLASH +void ScriptLogger::esp_log_(int level, int line, const __FlashStringHelper *format, const char *param) { + esp_log_printf_(level, TAG, line, format, param); +} +#else void ScriptLogger::esp_log_(int level, int line, const char *format, const char *param) { esp_log_printf_(level, TAG, line, format, param); } +#endif } // namespace script } // namespace esphome diff --git a/esphome/components/script/script.h b/esphome/components/script/script.h index 60175ec933..b87402f52e 100644 --- a/esphome/components/script/script.h +++ b/esphome/components/script/script.h @@ -10,6 +10,15 @@ namespace script { class ScriptLogger { protected: +#ifdef USE_STORE_LOG_STR_IN_FLASH + void esp_logw_(int line, const __FlashStringHelper *format, const char *param) { + esp_log_(ESPHOME_LOG_LEVEL_WARN, line, format, param); + } + void esp_logd_(int line, const __FlashStringHelper *format, const char *param) { + esp_log_(ESPHOME_LOG_LEVEL_DEBUG, line, format, param); + } + void esp_log_(int level, int line, const __FlashStringHelper *format, const char *param); +#else void esp_logw_(int line, const char *format, const char *param) { esp_log_(ESPHOME_LOG_LEVEL_WARN, line, format, param); } @@ -17,6 +26,7 @@ class ScriptLogger { esp_log_(ESPHOME_LOG_LEVEL_DEBUG, line, format, param); } void esp_log_(int level, int line, const char *format, const char *param); +#endif }; /// The abstract base class for all script types. @@ -38,14 +48,14 @@ template class Script : public ScriptLogger, public Trigger void execute_tuple_(const std::tuple &tuple, seq /*unused*/) { this->execute(std::get(tuple)...); } - std::string name_; + const LogString *name_{nullptr}; }; /** A script type for which only a single instance at a time is allowed. @@ -57,7 +67,8 @@ template class SingleScript : public Script { public: void execute(Ts... x) override { if (this->is_action_running()) { - this->esp_logw_(__LINE__, "Script '%s' is already running! (mode: single)", this->name_.c_str()); + this->esp_logw_(__LINE__, ESPHOME_LOG_FORMAT("Script '%s' is already running! (mode: single)"), + LOG_STR_ARG(this->name_)); return; } @@ -74,7 +85,7 @@ template class RestartScript : public Script { public: void execute(Ts... x) override { if (this->is_action_running()) { - this->esp_logd_(__LINE__, "Script '%s' restarting (mode: restart)", this->name_.c_str()); + this->esp_logd_(__LINE__, ESPHOME_LOG_FORMAT("Script '%s' restarting (mode: restart)"), LOG_STR_ARG(this->name_)); this->stop_action(); } @@ -93,11 +104,13 @@ template class QueueingScript : public Script, public Com // num_runs_ is the number of *queued* instances, so total number of instances is // num_runs_ + 1 if (this->max_runs_ != 0 && this->num_runs_ + 1 >= this->max_runs_) { - this->esp_logw_(__LINE__, "Script '%s' maximum number of queued runs exceeded!", this->name_.c_str()); + this->esp_logw_(__LINE__, ESPHOME_LOG_FORMAT("Script '%s' maximum number of queued runs exceeded!"), + LOG_STR_ARG(this->name_)); return; } - this->esp_logd_(__LINE__, "Script '%s' queueing new instance (mode: queued)", this->name_.c_str()); + this->esp_logd_(__LINE__, ESPHOME_LOG_FORMAT("Script '%s' queueing new instance (mode: queued)"), + LOG_STR_ARG(this->name_)); this->num_runs_++; this->var_queue_.push(std::make_tuple(x...)); return; @@ -143,7 +156,8 @@ template class ParallelScript : public Script { public: void execute(Ts... x) override { if (this->max_runs_ != 0 && this->automation_parent_->num_running() >= this->max_runs_) { - this->esp_logw_(__LINE__, "Script '%s' maximum number of parallel runs exceeded!", this->name_.c_str()); + this->esp_logw_(__LINE__, ESPHOME_LOG_FORMAT("Script '%s' maximum number of parallel runs exceeded!"), + LOG_STR_ARG(this->name_)); return; } this->trigger(x...); diff --git a/esphome/components/sdl/display.py b/esphome/components/sdl/display.py index ae8b0fd43a..78c180aa65 100644 --- a/esphome/components/sdl/display.py +++ b/esphome/components/sdl/display.py @@ -36,7 +36,9 @@ def get_sdl_options(value): if value != "": return value try: - return subprocess.check_output(["sdl2-config", "--cflags", "--libs"]).decode() + return subprocess.check_output( + ["sdl2-config", "--cflags", "--libs"], close_fds=False + ).decode() except Exception as e: raise cv.Invalid("Unable to run sdl2-config - have you installed sdl2?") from e diff --git a/esphome/components/sdl/sdl_esphome.cpp b/esphome/components/sdl/sdl_esphome.cpp index e55bff58fe..f235e4e68c 100644 --- a/esphome/components/sdl/sdl_esphome.cpp +++ b/esphome/components/sdl/sdl_esphome.cpp @@ -6,7 +6,6 @@ namespace esphome { namespace sdl { void Sdl::setup() { - ESP_LOGD(TAG, "Starting setup"); SDL_Init(SDL_INIT_VIDEO); this->window_ = SDL_CreateWindow(App.get_name().c_str(), this->pos_x_, this->pos_y_, this->width_, this->height_, this->window_options_); @@ -15,7 +14,6 @@ void Sdl::setup() { this->texture_ = SDL_CreateTexture(this->renderer_, SDL_PIXELFORMAT_RGB565, SDL_TEXTUREACCESS_STATIC, this->width_, this->height_); SDL_SetTextureBlendMode(this->texture_, SDL_BLENDMODE_BLEND); - ESP_LOGD(TAG, "Setup Complete"); } void Sdl::update() { this->do_update_(); @@ -48,6 +46,9 @@ void Sdl::draw_pixels_at(int x_start, int y_start, int w, int h, const uint8_t * } void Sdl::draw_pixel_at(int x, int y, Color color) { + if (!this->get_clipping().inside(x, y)) + return; + SDL_Rect rect{x, y, 1, 1}; auto data = (display::ColorUtil::color_to_565(color, display::COLOR_ORDER_RGB)); SDL_UpdateTexture(this->texture_, &rect, &data, 2); diff --git a/esphome/components/sdm_meter/sensor.py b/esphome/components/sdm_meter/sensor.py index 24ae32c7bc..affbc0409e 100644 --- a/esphome/components/sdm_meter/sensor.py +++ b/esphome/components/sdm_meter/sensor.py @@ -1,6 +1,5 @@ import esphome.codegen as cg from esphome.components import modbus, sensor -from esphome.components.atm90e32.sensor import CONF_PHASE_A, CONF_PHASE_B, CONF_PHASE_C import esphome.config_validation as cv from esphome.const import ( CONF_ACTIVE_POWER, @@ -12,7 +11,10 @@ from esphome.const import ( CONF_ID, CONF_IMPORT_ACTIVE_ENERGY, CONF_IMPORT_REACTIVE_ENERGY, + CONF_PHASE_A, CONF_PHASE_ANGLE, + CONF_PHASE_B, + CONF_PHASE_C, CONF_POWER_FACTOR, CONF_REACTIVE_POWER, CONF_TOTAL_POWER, diff --git a/esphome/components/sdp3x/sdp3x.cpp b/esphome/components/sdp3x/sdp3x.cpp index 58aefe09d7..d4ab04e7cd 100644 --- a/esphome/components/sdp3x/sdp3x.cpp +++ b/esphome/components/sdp3x/sdp3x.cpp @@ -17,8 +17,6 @@ static const uint16_t SDP3X_STOP_MEAS = 0x3FF9; void SDP3XComponent::update() { this->read_pressure_(); } void SDP3XComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - if (!this->write_command(SDP3X_STOP_MEAS)) { ESP_LOGW(TAG, "Stop failed"); // This sometimes fails for no good reason } diff --git a/esphome/components/seeed_mr24hpc1/seeed_mr24hpc1.cpp b/esphome/components/seeed_mr24hpc1/seeed_mr24hpc1.cpp index 8683d6cad7..76523ce5c0 100644 --- a/esphome/components/seeed_mr24hpc1/seeed_mr24hpc1.cpp +++ b/esphome/components/seeed_mr24hpc1/seeed_mr24hpc1.cpp @@ -62,7 +62,6 @@ void MR24HPC1Component::dump_config() { // Initialisation functions void MR24HPC1Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->check_uart_settings(115200); if (this->custom_mode_number_ != nullptr) { @@ -91,7 +90,6 @@ void MR24HPC1Component::setup() { memset(this->sg_frame_buf_, 0, FRAME_BUF_MAX_SIZE); this->set_interval(8000, [this]() { this->update_(); }); - ESP_LOGCONFIG(TAG, "Set up MR24HPC1 complete"); } // Timed polling of radar data diff --git a/esphome/components/seeed_mr60fda2/seeed_mr60fda2.cpp b/esphome/components/seeed_mr60fda2/seeed_mr60fda2.cpp index e40cd9c0c7..dea7976578 100644 --- a/esphome/components/seeed_mr60fda2/seeed_mr60fda2.cpp +++ b/esphome/components/seeed_mr60fda2/seeed_mr60fda2.cpp @@ -31,7 +31,6 @@ void MR60FDA2Component::dump_config() { // Initialisation functions void MR60FDA2Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->check_uart_settings(115200); this->current_frame_locate_ = LOCATE_FRAME_HEADER; @@ -43,8 +42,6 @@ void MR60FDA2Component::setup() { memset(this->current_frame_buf_, 0, FRAME_BUF_MAX_SIZE); memset(this->current_data_buf_, 0, DATA_BUF_MAX_SIZE); - - ESP_LOGCONFIG(TAG, "Set up MR60FDA2 complete"); } // main loop diff --git a/esphome/components/select/__init__.py b/esphome/components/select/__init__.py index ed1f6c020d..c7146df9fb 100644 --- a/esphome/components/select/__init__.py +++ b/esphome/components/select/__init__.py @@ -16,7 +16,7 @@ from esphome.const import ( CONF_TRIGGER_ID, CONF_WEB_SERVER, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass @@ -118,15 +118,14 @@ async def register_select(var, config, *, options: list[str]): await setup_select_core_(var, config, options=options) -async def new_select(config, *, options: list[str]): - var = cg.new_Pvariable(config[CONF_ID]) +async def new_select(config, *args, options: list[str]): + var = cg.new_Pvariable(config[CONF_ID], *args) await register_select(var, config, options=options) return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_SELECT") cg.add_global(select_ns.using) diff --git a/esphome/components/select/select.cpp b/esphome/components/select/select.cpp index 37887da27c..16e8288ca1 100644 --- a/esphome/components/select/select.cpp +++ b/esphome/components/select/select.cpp @@ -28,17 +28,18 @@ bool Select::has_option(const std::string &option) const { return this->index_of bool Select::has_index(size_t index) const { return index < this->size(); } size_t Select::size() const { - auto options = traits.get_options(); + const auto &options = traits.get_options(); return options.size(); } optional Select::index_of(const std::string &option) const { - auto options = traits.get_options(); - auto it = std::find(options.begin(), options.end(), option); - if (it == options.end()) { - return {}; + const auto &options = traits.get_options(); + for (size_t i = 0; i < options.size(); i++) { + if (options[i] == option) { + return i; + } } - return std::distance(options.begin(), it); + return {}; } optional Select::active_index() const { @@ -51,7 +52,7 @@ optional Select::active_index() const { optional Select::at(size_t index) const { if (this->has_index(index)) { - auto options = traits.get_options(); + const auto &options = traits.get_options(); return options.at(index); } else { return {}; diff --git a/esphome/components/select/select.h b/esphome/components/select/select.h index 3ab651b241..902b8a78ce 100644 --- a/esphome/components/select/select.h +++ b/esphome/components/select/select.h @@ -12,8 +12,8 @@ namespace select { #define LOG_SELECT(prefix, type, obj) \ if ((obj) != nullptr) { \ ESP_LOGCONFIG(TAG, "%s%s '%s'", prefix, LOG_STR_LITERAL(type), (obj)->get_name().c_str()); \ - if (!(obj)->get_icon().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon().c_str()); \ + if (!(obj)->get_icon_ref().empty()) { \ + ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon_ref().c_str()); \ } \ } diff --git a/esphome/components/select/select_call.cpp b/esphome/components/select/select_call.cpp index 85f755645c..dd398b4052 100644 --- a/esphome/components/select/select_call.cpp +++ b/esphome/components/select/select_call.cpp @@ -45,7 +45,7 @@ void SelectCall::perform() { auto *parent = this->parent_; const auto *name = parent->get_name().c_str(); const auto &traits = parent->traits; - auto options = traits.get_options(); + const auto &options = traits.get_options(); if (this->operation_ == SELECT_OP_NONE) { ESP_LOGW(TAG, "'%s' - SelectCall performed without selecting an operation", name); @@ -107,7 +107,7 @@ void SelectCall::perform() { } } - if (std::find(options.begin(), options.end(), target_value) == options.end()) { + if (!parent->has_option(target_value)) { ESP_LOGW(TAG, "'%s' - Option %s is not a valid option", name, target_value.c_str()); return; } diff --git a/esphome/components/select/select_traits.cpp b/esphome/components/select/select_traits.cpp index 89da30c405..a8cd4290c8 100644 --- a/esphome/components/select/select_traits.cpp +++ b/esphome/components/select/select_traits.cpp @@ -5,7 +5,7 @@ namespace select { void SelectTraits::set_options(std::vector options) { this->options_ = std::move(options); } -std::vector SelectTraits::get_options() const { return this->options_; } +const std::vector &SelectTraits::get_options() const { return this->options_; } } // namespace select } // namespace esphome diff --git a/esphome/components/select/select_traits.h b/esphome/components/select/select_traits.h index ccf23dc6d0..128066dd6b 100644 --- a/esphome/components/select/select_traits.h +++ b/esphome/components/select/select_traits.h @@ -9,7 +9,7 @@ namespace select { class SelectTraits { public: void set_options(std::vector options); - std::vector get_options() const; + const std::vector &get_options() const; protected: std::vector options_; diff --git a/esphome/components/sen0321/sen0321.cpp b/esphome/components/sen0321/sen0321.cpp index c727dda0b1..6a5931272d 100644 --- a/esphome/components/sen0321/sen0321.cpp +++ b/esphome/components/sen0321/sen0321.cpp @@ -8,7 +8,6 @@ namespace sen0321_sensor { static const char *const TAG = "sen0321_sensor.sensor"; void Sen0321Sensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (!this->write_byte(SENSOR_MODE_REGISTER, SENSOR_MODE_AUTO)) { ESP_LOGW(TAG, "Error setting measurement mode."); this->mark_failed(); diff --git a/esphome/components/sen5x/sen5x.cpp b/esphome/components/sen5x/sen5x.cpp index c7fd997b0c..3298a5b8db 100644 --- a/esphome/components/sen5x/sen5x.cpp +++ b/esphome/components/sen5x/sen5x.cpp @@ -29,9 +29,20 @@ static const int8_t SEN5X_INDEX_SCALE_FACTOR = 10; // static const int8_t SEN5X_MIN_INDEX_VALUE = 1 * SEN5X_INDEX_SCALE_FACTOR; // must be adjusted by the scale factor static const int16_t SEN5X_MAX_INDEX_VALUE = 500 * SEN5X_INDEX_SCALE_FACTOR; // must be adjusted by the scale factor -void SEN5XComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); +static const LogString *rht_accel_mode_to_string(RhtAccelerationMode mode) { + switch (mode) { + case LOW_ACCELERATION: + return LOG_STR("LOW"); + case MEDIUM_ACCELERATION: + return LOG_STR("MEDIUM"); + case HIGH_ACCELERATION: + return LOG_STR("HIGH"); + default: + return LOG_STR("UNKNOWN"); + } +} +void SEN5XComponent::setup() { // the sensor needs 1000 ms to enter the idle state this->set_timeout(1000, [this]() { // Check if measurement is ready before reading the value @@ -40,6 +51,7 @@ void SEN5XComponent::setup() { this->mark_failed(); return; } + delay(20); // per datasheet uint16_t raw_read_status; if (!this->read_data(raw_read_status)) { @@ -51,7 +63,7 @@ void SEN5XComponent::setup() { uint32_t stop_measurement_delay = 0; // In order to query the device periodic measurement must be ceased if (raw_read_status) { - ESP_LOGD(TAG, "Sensor has data available, stopping periodic measurement"); + ESP_LOGD(TAG, "Data is available; stopping periodic measurement"); if (!this->write_command(SEN5X_CMD_STOP_MEASUREMENTS)) { ESP_LOGE(TAG, "Failed to stop measurements"); this->mark_failed(); @@ -72,7 +84,8 @@ void SEN5XComponent::setup() { this->serial_number_[0] = static_cast(uint16_t(raw_serial_number[0]) & 0xFF); this->serial_number_[1] = static_cast(raw_serial_number[0] & 0xFF); this->serial_number_[2] = static_cast(raw_serial_number[1] >> 8); - ESP_LOGD(TAG, "Serial number %02d.%02d.%02d", serial_number_[0], serial_number_[1], serial_number_[2]); + ESP_LOGV(TAG, "Serial number %02d.%02d.%02d", this->serial_number_[0], this->serial_number_[1], + this->serial_number_[2]); uint16_t raw_product_name[16]; if (!this->get_register(SEN5X_CMD_GET_PRODUCT_NAME, raw_product_name, 16, 20)) { @@ -89,45 +102,43 @@ void SEN5XComponent::setup() { // first char current_char = *current_int >> 8; if (current_char) { - product_name_.push_back(current_char); + this->product_name_.push_back(current_char); // second char current_char = *current_int & 0xFF; if (current_char) { - product_name_.push_back(current_char); + this->product_name_.push_back(current_char); } } current_int++; } while (current_char && --max); Sen5xType sen5x_type = UNKNOWN; - if (product_name_ == "SEN50") { + if (this->product_name_ == "SEN50") { sen5x_type = SEN50; } else { - if (product_name_ == "SEN54") { + if (this->product_name_ == "SEN54") { sen5x_type = SEN54; } else { - if (product_name_ == "SEN55") { + if (this->product_name_ == "SEN55") { sen5x_type = SEN55; } } - ESP_LOGD(TAG, "Productname %s", product_name_.c_str()); + ESP_LOGD(TAG, "Product name: %s", this->product_name_.c_str()); } if (this->humidity_sensor_ && sen5x_type == SEN50) { - ESP_LOGE(TAG, "For Relative humidity a SEN54 OR SEN55 is required. You are using a <%s> sensor", - this->product_name_.c_str()); + ESP_LOGE(TAG, "Relative humidity requires a SEN54 or SEN55"); this->humidity_sensor_ = nullptr; // mark as not used } if (this->temperature_sensor_ && sen5x_type == SEN50) { - ESP_LOGE(TAG, "For Temperature a SEN54 OR SEN55 is required. You are using a <%s> sensor", - this->product_name_.c_str()); + ESP_LOGE(TAG, "Temperature requires a SEN54 or SEN55"); this->temperature_sensor_ = nullptr; // mark as not used } if (this->voc_sensor_ && sen5x_type == SEN50) { - ESP_LOGE(TAG, "For VOC a SEN54 OR SEN55 is required. You are using a <%s> sensor", this->product_name_.c_str()); + ESP_LOGE(TAG, "VOC requires a SEN54 or SEN55"); this->voc_sensor_ = nullptr; // mark as not used } if (this->nox_sensor_ && sen5x_type != SEN55) { - ESP_LOGE(TAG, "For NOx a SEN55 is required. You are using a <%s> sensor", this->product_name_.c_str()); + ESP_LOGE(TAG, "NOx requires a SEN55"); this->nox_sensor_ = nullptr; // mark as not used } @@ -138,7 +149,7 @@ void SEN5XComponent::setup() { return; } this->firmware_version_ >>= 8; - ESP_LOGD(TAG, "Firmware version %d", this->firmware_version_); + ESP_LOGV(TAG, "Firmware version %d", this->firmware_version_); if (this->voc_sensor_ && this->store_baseline_) { uint32_t combined_serial = @@ -151,7 +162,7 @@ void SEN5XComponent::setup() { if (this->pref_.load(&this->voc_baselines_storage_)) { ESP_LOGI(TAG, "Loaded VOC baseline state0: 0x%04" PRIX32 ", state1: 0x%04" PRIX32, - this->voc_baselines_storage_.state0, voc_baselines_storage_.state1); + this->voc_baselines_storage_.state0, this->voc_baselines_storage_.state1); } // Initialize storage timestamp @@ -159,13 +170,13 @@ void SEN5XComponent::setup() { if (this->voc_baselines_storage_.state0 > 0 && this->voc_baselines_storage_.state1 > 0) { ESP_LOGI(TAG, "Setting VOC baseline from save state0: 0x%04" PRIX32 ", state1: 0x%04" PRIX32, - this->voc_baselines_storage_.state0, voc_baselines_storage_.state1); + this->voc_baselines_storage_.state0, this->voc_baselines_storage_.state1); uint16_t states[4]; - states[0] = voc_baselines_storage_.state0 >> 16; - states[1] = voc_baselines_storage_.state0 & 0xFFFF; - states[2] = voc_baselines_storage_.state1 >> 16; - states[3] = voc_baselines_storage_.state1 & 0xFFFF; + states[0] = this->voc_baselines_storage_.state0 >> 16; + states[1] = this->voc_baselines_storage_.state0 & 0xFFFF; + states[2] = this->voc_baselines_storage_.state1 >> 16; + states[3] = this->voc_baselines_storage_.state1 & 0xFFFF; if (!this->write_command(SEN5X_CMD_VOC_ALGORITHM_STATE, states, 4)) { ESP_LOGE(TAG, "Failed to set VOC baseline from saved state"); @@ -183,11 +194,11 @@ void SEN5XComponent::setup() { delay(20); uint16_t secs[2]; if (this->read_data(secs, 2)) { - auto_cleaning_interval_ = secs[0] << 16 | secs[1]; + this->auto_cleaning_interval_ = secs[0] << 16 | secs[1]; } } - if (acceleration_mode_.has_value()) { - result = this->write_command(SEN5X_CMD_RHT_ACCELERATION_MODE, acceleration_mode_.value()); + if (this->acceleration_mode_.has_value()) { + result = this->write_command(SEN5X_CMD_RHT_ACCELERATION_MODE, this->acceleration_mode_.value()); } else { result = this->write_command(SEN5X_CMD_RHT_ACCELERATION_MODE); } @@ -198,7 +209,7 @@ void SEN5XComponent::setup() { return; } delay(20); - if (!acceleration_mode_.has_value()) { + if (!this->acceleration_mode_.has_value()) { uint16_t mode; if (this->read_data(mode)) { this->acceleration_mode_ = RhtAccelerationMode(mode); @@ -228,19 +239,18 @@ void SEN5XComponent::setup() { } if (!this->write_command(cmd)) { - ESP_LOGE(TAG, "Error starting continuous measurements."); + ESP_LOGE(TAG, "Error starting continuous measurements"); this->error_code_ = MEASUREMENT_INIT_FAILED; this->mark_failed(); return; } - initialized_ = true; - ESP_LOGD(TAG, "Sensor initialized"); + this->initialized_ = true; }); }); } void SEN5XComponent::dump_config() { - ESP_LOGCONFIG(TAG, "sen5x:"); + ESP_LOGCONFIG(TAG, "SEN5X:"); LOG_I2C_DEVICE(this); if (this->is_failed()) { switch (this->error_code_) { @@ -248,16 +258,16 @@ void SEN5XComponent::dump_config() { ESP_LOGW(TAG, ESP_LOG_MSG_COMM_FAIL); break; case MEASUREMENT_INIT_FAILED: - ESP_LOGW(TAG, "Measurement Initialization failed"); + ESP_LOGW(TAG, "Measurement initialization failed"); break; case SERIAL_NUMBER_IDENTIFICATION_FAILED: - ESP_LOGW(TAG, "Unable to read sensor serial id"); + ESP_LOGW(TAG, "Unable to read serial ID"); break; case PRODUCT_NAME_FAILED: ESP_LOGW(TAG, "Unable to read product name"); break; case FIRMWARE_FAILED: - ESP_LOGW(TAG, "Unable to read sensor firmware version"); + ESP_LOGW(TAG, "Unable to read firmware version"); break; default: ESP_LOGW(TAG, "Unknown setup error"); @@ -265,26 +275,17 @@ void SEN5XComponent::dump_config() { } } ESP_LOGCONFIG(TAG, - " Productname: %s\n" + " Product name: %s\n" " Firmware version: %d\n" " Serial number %02d.%02d.%02d", - this->product_name_.c_str(), this->firmware_version_, serial_number_[0], serial_number_[1], - serial_number_[2]); + this->product_name_.c_str(), this->firmware_version_, this->serial_number_[0], this->serial_number_[1], + this->serial_number_[2]); if (this->auto_cleaning_interval_.has_value()) { - ESP_LOGCONFIG(TAG, " Auto cleaning interval %" PRId32 " seconds", auto_cleaning_interval_.value()); + ESP_LOGCONFIG(TAG, " Auto cleaning interval: %" PRId32 "s", this->auto_cleaning_interval_.value()); } if (this->acceleration_mode_.has_value()) { - switch (this->acceleration_mode_.value()) { - case LOW_ACCELERATION: - ESP_LOGCONFIG(TAG, " Low RH/T acceleration mode"); - break; - case MEDIUM_ACCELERATION: - ESP_LOGCONFIG(TAG, " Medium RH/T acceleration mode"); - break; - case HIGH_ACCELERATION: - ESP_LOGCONFIG(TAG, " High RH/T acceleration mode"); - break; - } + ESP_LOGCONFIG(TAG, " RH/T acceleration mode: %s", + LOG_STR_ARG(rht_accel_mode_to_string(this->acceleration_mode_.value()))); } LOG_UPDATE_INTERVAL(this); LOG_SENSOR(" ", "PM 1.0", this->pm_1_0_sensor_); @@ -298,7 +299,7 @@ void SEN5XComponent::dump_config() { } void SEN5XComponent::update() { - if (!initialized_) { + if (!this->initialized_) { return; } @@ -321,8 +322,8 @@ void SEN5XComponent::update() { this->voc_baselines_storage_.state1 = state1; if (this->pref_.save(&this->voc_baselines_storage_)) { - ESP_LOGI(TAG, "Stored VOC baseline state0: 0x%04" PRIX32 " ,state1: 0x%04" PRIX32, - this->voc_baselines_storage_.state0, voc_baselines_storage_.state1); + ESP_LOGI(TAG, "Stored VOC baseline state0: 0x%04" PRIX32 ", state1: 0x%04" PRIX32, + this->voc_baselines_storage_.state0, this->voc_baselines_storage_.state1); } else { ESP_LOGW(TAG, "Could not store VOC baselines"); } @@ -334,7 +335,7 @@ void SEN5XComponent::update() { if (!this->write_command(SEN5X_CMD_READ_MEASUREMENT)) { this->status_set_warning(); - ESP_LOGD(TAG, "write error read measurement (%d)", this->last_error_); + ESP_LOGD(TAG, "Write error: read measurement (%d)", this->last_error_); return; } this->set_timeout(20, [this]() { @@ -342,7 +343,7 @@ void SEN5XComponent::update() { if (!this->read_data(measurements, 8)) { this->status_set_warning(); - ESP_LOGD(TAG, "read data error (%d)", this->last_error_); + ESP_LOGD(TAG, "Read data error (%d)", this->last_error_); return; } @@ -414,7 +415,7 @@ bool SEN5XComponent::write_tuning_parameters_(uint16_t i2c_command, const GasTun params[5] = tuning.gain_factor; auto result = write_command(i2c_command, params, 6); if (!result) { - ESP_LOGE(TAG, "set tuning parameters failed. i2c command=%0xX, err=%d", i2c_command, this->last_error_); + ESP_LOGE(TAG, "Set tuning parameters failed (command=%0xX, err=%d)", i2c_command, this->last_error_); } return result; } @@ -425,7 +426,7 @@ bool SEN5XComponent::write_temperature_compensation_(const TemperatureCompensati params[1] = compensation.normalized_offset_slope; params[2] = compensation.time_constant; if (!write_command(SEN5X_CMD_TEMPERATURE_COMPENSATION, params, 3)) { - ESP_LOGE(TAG, "set temperature_compensation failed. Err=%d", this->last_error_); + ESP_LOGE(TAG, "Set temperature_compensation failed (%d)", this->last_error_); return false; } return true; @@ -434,7 +435,7 @@ bool SEN5XComponent::write_temperature_compensation_(const TemperatureCompensati bool SEN5XComponent::start_fan_cleaning() { if (!write_command(SEN5X_CMD_START_CLEANING_FAN)) { this->status_set_warning(); - ESP_LOGE(TAG, "write error start fan (%d)", this->last_error_); + ESP_LOGE(TAG, "Start fan cleaning failed (%d)", this->last_error_); return false; } else { ESP_LOGD(TAG, "Fan auto clean started"); diff --git a/esphome/components/sen5x/sen5x.h b/esphome/components/sen5x/sen5x.h index 0fa31605e6..9e5b6bf231 100644 --- a/esphome/components/sen5x/sen5x.h +++ b/esphome/components/sen5x/sen5x.h @@ -9,7 +9,7 @@ namespace esphome { namespace sen5x { -enum ERRORCODE { +enum ERRORCODE : uint8_t { COMMUNICATION_FAILED, SERIAL_NUMBER_IDENTIFICATION_FAILED, MEASUREMENT_INIT_FAILED, @@ -18,19 +18,17 @@ enum ERRORCODE { UNKNOWN }; -// Shortest time interval of 3H for storing baseline values. -// Prevents wear of the flash because of too many write operations -const uint32_t SHORTEST_BASELINE_STORE_INTERVAL = 10800; -// Store anyway if the baseline difference exceeds the max storage diff value -const uint32_t MAXIMUM_STORAGE_DIFF = 50; +enum RhtAccelerationMode : uint16_t { + LOW_ACCELERATION = 0, + MEDIUM_ACCELERATION = 1, + HIGH_ACCELERATION = 2, +}; struct Sen5xBaselines { int32_t state0; int32_t state1; } PACKED; // NOLINT -enum RhtAccelerationMode : uint16_t { LOW_ACCELERATION = 0, MEDIUM_ACCELERATION = 1, HIGH_ACCELERATION = 2 }; - struct GasTuning { uint16_t index_offset; uint16_t learning_time_offset_hours; @@ -46,6 +44,12 @@ struct TemperatureCompensation { uint16_t time_constant; }; +// Shortest time interval of 3H for storing baseline values. +// Prevents wear of the flash because of too many write operations +static const uint32_t SHORTEST_BASELINE_STORE_INTERVAL = 10800; +// Store anyway if the baseline difference exceeds the max storage diff value +static const uint32_t MAXIMUM_STORAGE_DIFF = 50; + class SEN5XComponent : public PollingComponent, public sensirion_common::SensirionI2CDevice { public: void setup() override; @@ -102,8 +106,14 @@ class SEN5XComponent : public PollingComponent, public sensirion_common::Sensiri protected: bool write_tuning_parameters_(uint16_t i2c_command, const GasTuning &tuning); bool write_temperature_compensation_(const TemperatureCompensation &compensation); + + uint32_t seconds_since_last_store_; + uint16_t firmware_version_; ERRORCODE error_code_; + uint8_t serial_number_[4]; bool initialized_{false}; + bool store_baseline_; + sensor::Sensor *pm_1_0_sensor_{nullptr}; sensor::Sensor *pm_2_5_sensor_{nullptr}; sensor::Sensor *pm_4_0_sensor_{nullptr}; @@ -115,18 +125,14 @@ class SEN5XComponent : public PollingComponent, public sensirion_common::Sensiri // SEN55 only sensor::Sensor *nox_sensor_{nullptr}; - std::string product_name_; - uint8_t serial_number_[4]; - uint16_t firmware_version_; - Sen5xBaselines voc_baselines_storage_; - bool store_baseline_; - uint32_t seconds_since_last_store_; - ESPPreferenceObject pref_; optional acceleration_mode_; optional auto_cleaning_interval_; optional voc_tuning_params_; optional nox_tuning_params_; optional temperature_compensation_; + ESPPreferenceObject pref_; + std::string product_name_; + Sen5xBaselines voc_baselines_storage_; }; } // namespace sen5x diff --git a/esphome/components/sen5x/sensor.py b/esphome/components/sen5x/sensor.py index f52de5fe85..9668a253c0 100644 --- a/esphome/components/sen5x/sensor.py +++ b/esphome/components/sen5x/sensor.py @@ -65,26 +65,47 @@ ACCELERATION_MODES = { "high": RhtAccelerationMode.HIGH_ACCELERATION, } -GAS_SENSOR = cv.Schema( - { - cv.Optional(CONF_ALGORITHM_TUNING): cv.Schema( - { - cv.Optional(CONF_INDEX_OFFSET, default=100): cv.int_range(1, 250), - cv.Optional(CONF_LEARNING_TIME_OFFSET_HOURS, default=12): cv.int_range( - 1, 1000 - ), - cv.Optional(CONF_LEARNING_TIME_GAIN_HOURS, default=12): cv.int_range( - 1, 1000 - ), - cv.Optional( - CONF_GATING_MAX_DURATION_MINUTES, default=720 - ): cv.int_range(0, 3000), - cv.Optional(CONF_STD_INITIAL, default=50): cv.int_, - cv.Optional(CONF_GAIN_FACTOR, default=230): cv.int_range(1, 1000), - } - ) - } -) + +def _gas_sensor( + *, + index_offset: int, + learning_time_offset: int, + learning_time_gain: int, + gating_max_duration: int, + std_initial: int, + gain_factor: int, +) -> cv.Schema: + return sensor.sensor_schema( + icon=ICON_RADIATOR, + accuracy_decimals=0, + device_class=DEVICE_CLASS_AQI, + state_class=STATE_CLASS_MEASUREMENT, + ).extend( + { + cv.Optional(CONF_ALGORITHM_TUNING): cv.Schema( + { + cv.Optional(CONF_INDEX_OFFSET, default=index_offset): cv.int_range( + 1, 250 + ), + cv.Optional( + CONF_LEARNING_TIME_OFFSET_HOURS, default=learning_time_offset + ): cv.int_range(1, 1000), + cv.Optional( + CONF_LEARNING_TIME_GAIN_HOURS, default=learning_time_gain + ): cv.int_range(1, 1000), + cv.Optional( + CONF_GATING_MAX_DURATION_MINUTES, default=gating_max_duration + ): cv.int_range(0, 3000), + cv.Optional(CONF_STD_INITIAL, default=std_initial): cv.int_range( + 10, 5000 + ), + cv.Optional(CONF_GAIN_FACTOR, default=gain_factor): cv.int_range( + 1, 1000 + ), + } + ) + } + ) def float_previously_pct(value): @@ -127,18 +148,22 @@ CONFIG_SCHEMA = ( state_class=STATE_CLASS_MEASUREMENT, ), cv.Optional(CONF_AUTO_CLEANING_INTERVAL): cv.update_interval, - cv.Optional(CONF_VOC): sensor.sensor_schema( - icon=ICON_RADIATOR, - accuracy_decimals=0, - device_class=DEVICE_CLASS_AQI, - state_class=STATE_CLASS_MEASUREMENT, - ).extend(GAS_SENSOR), - cv.Optional(CONF_NOX): sensor.sensor_schema( - icon=ICON_RADIATOR, - accuracy_decimals=0, - device_class=DEVICE_CLASS_AQI, - state_class=STATE_CLASS_MEASUREMENT, - ).extend(GAS_SENSOR), + cv.Optional(CONF_VOC): _gas_sensor( + index_offset=100, + learning_time_offset=12, + learning_time_gain=12, + gating_max_duration=180, + std_initial=50, + gain_factor=230, + ), + cv.Optional(CONF_NOX): _gas_sensor( + index_offset=1, + learning_time_offset=12, + learning_time_gain=12, + gating_max_duration=720, + std_initial=50, + gain_factor=230, + ), cv.Optional(CONF_STORE_BASELINE, default=True): cv.boolean, cv.Optional(CONF_VOC_BASELINE): cv.hex_uint16_t, cv.Optional(CONF_TEMPERATURE): sensor.sensor_schema( @@ -194,16 +219,15 @@ async def to_code(config): await i2c.register_i2c_device(var, config) for key, funcName in SETTING_MAP.items(): - if key in config: - cg.add(getattr(var, funcName)(config[key])) + if cfg := config.get(key): + cg.add(getattr(var, funcName)(cfg)) for key, funcName in SENSOR_MAP.items(): - if key in config: - sens = await sensor.new_sensor(config[key]) + if cfg := config.get(key): + sens = await sensor.new_sensor(cfg) cg.add(getattr(var, funcName)(sens)) - if CONF_VOC in config and CONF_ALGORITHM_TUNING in config[CONF_VOC]: - cfg = config[CONF_VOC][CONF_ALGORITHM_TUNING] + if cfg := config.get(CONF_VOC, {}).get(CONF_ALGORITHM_TUNING): cg.add( var.set_voc_algorithm_tuning( cfg[CONF_INDEX_OFFSET], @@ -214,8 +238,7 @@ async def to_code(config): cfg[CONF_GAIN_FACTOR], ) ) - if CONF_NOX in config and CONF_ALGORITHM_TUNING in config[CONF_NOX]: - cfg = config[CONF_NOX][CONF_ALGORITHM_TUNING] + if cfg := config.get(CONF_NOX, {}).get(CONF_ALGORITHM_TUNING): cg.add( var.set_nox_algorithm_tuning( cfg[CONF_INDEX_OFFSET], @@ -225,12 +248,12 @@ async def to_code(config): cfg[CONF_GAIN_FACTOR], ) ) - if CONF_TEMPERATURE_COMPENSATION in config: + if cfg := config.get(CONF_TEMPERATURE_COMPENSATION): cg.add( var.set_temperature_compensation( - config[CONF_TEMPERATURE_COMPENSATION][CONF_OFFSET], - config[CONF_TEMPERATURE_COMPENSATION][CONF_NORMALIZED_OFFSET_SLOPE], - config[CONF_TEMPERATURE_COMPENSATION][CONF_TIME_CONSTANT], + cfg[CONF_OFFSET], + cfg[CONF_NORMALIZED_OFFSET_SLOPE], + cfg[CONF_TIME_CONSTANT], ) ) diff --git a/esphome/components/senseair/senseair.cpp b/esphome/components/senseair/senseair.cpp index e58ee157f7..84520d407d 100644 --- a/esphome/components/senseair/senseair.cpp +++ b/esphome/components/senseair/senseair.cpp @@ -53,10 +53,14 @@ void SenseAirComponent::update() { this->status_clear_warning(); const uint8_t length = response[2]; - const uint16_t status = (uint16_t(response[3]) << 8) | response[4]; - const int16_t ppm = int16_t((response[length + 1] << 8) | response[length + 2]); + const uint16_t status = encode_uint16(response[3], response[4]); + const uint16_t ppm = encode_uint16(response[length + 1], response[length + 2]); - ESP_LOGD(TAG, "SenseAir Received CO₂=%dppm Status=0x%02X", ppm, status); + ESP_LOGD(TAG, "SenseAir Received CO₂=%uppm Status=0x%02X", ppm, status); + if (ppm == 0 && (status & SenseAirStatus::OUT_OF_RANGE_ERROR) != 0) { + ESP_LOGD(TAG, "Discarding 0 ppm reading with out-of-range status."); + return; + } if (this->co2_sensor_ != nullptr) this->co2_sensor_->publish_state(ppm); } diff --git a/esphome/components/senseair/senseair.h b/esphome/components/senseair/senseair.h index 9f939d5b07..5b66860f1a 100644 --- a/esphome/components/senseair/senseair.h +++ b/esphome/components/senseair/senseair.h @@ -8,6 +8,17 @@ namespace esphome { namespace senseair { +enum SenseAirStatus : uint8_t { + FATAL_ERROR = 1 << 0, + OFFSET_ERROR = 1 << 1, + ALGORITHM_ERROR = 1 << 2, + OUTPUT_ERROR = 1 << 3, + SELF_DIAGNOSTIC_ERROR = 1 << 4, + OUT_OF_RANGE_ERROR = 1 << 5, + MEMORY_ERROR = 1 << 6, + RESERVED = 1 << 7 +}; + class SenseAirComponent : public PollingComponent, public uart::UARTDevice { public: void set_co2_sensor(sensor::Sensor *co2_sensor) { co2_sensor_ = co2_sensor; } diff --git a/esphome/components/sensirion_common/i2c_sensirion.cpp b/esphome/components/sensirion_common/i2c_sensirion.cpp index f71b3c14cb..9eac6b4525 100644 --- a/esphome/components/sensirion_common/i2c_sensirion.cpp +++ b/esphome/components/sensirion_common/i2c_sensirion.cpp @@ -11,21 +11,22 @@ static const char *const TAG = "sensirion_i2c"; // To avoid memory allocations for small writes a stack buffer is used static const size_t BUFFER_STACK_SIZE = 16; -bool SensirionI2CDevice::read_data(uint16_t *data, uint8_t len) { +bool SensirionI2CDevice::read_data(uint16_t *data, const uint8_t len) { const uint8_t num_bytes = len * 3; - std::vector buf(num_bytes); + uint8_t buf[num_bytes]; - last_error_ = this->read(buf.data(), num_bytes); - if (last_error_ != i2c::ERROR_OK) { + this->last_error_ = this->read(buf, num_bytes); + if (this->last_error_ != i2c::ERROR_OK) { return false; } for (uint8_t i = 0; i < len; i++) { const uint8_t j = 3 * i; - uint8_t crc = sht_crc_(buf[j], buf[j + 1]); + // Use MSB first since Sensirion devices use CRC-8 with MSB first + uint8_t crc = crc8(&buf[j], 2, 0xFF, CRC_POLYNOMIAL, true); if (crc != buf[j + 2]) { - ESP_LOGE(TAG, "CRC8 Checksum invalid at pos %d! 0x%02X != 0x%02X", i, buf[j + 2], crc); - last_error_ = i2c::ERROR_CRC; + ESP_LOGE(TAG, "CRC invalid @ %d! 0x%02X != 0x%02X", i, buf[j + 2], crc); + this->last_error_ = i2c::ERROR_CRC; return false; } data[i] = encode_uint16(buf[j], buf[j + 1]); @@ -34,10 +35,10 @@ bool SensirionI2CDevice::read_data(uint16_t *data, uint8_t len) { } /*** * write command with parameters and insert crc - * use stack array for less than 4 parameters. Most sensirion i2c commands have less parameters + * use stack array for less than 4 parameters. Most Sensirion I2C commands have less parameters */ bool SensirionI2CDevice::write_command_(uint16_t command, CommandLen command_len, const uint16_t *data, - uint8_t data_len) { + const uint8_t data_len) { uint8_t temp_stack[BUFFER_STACK_SIZE]; std::unique_ptr temp_heap; uint8_t *temp; @@ -74,56 +75,27 @@ bool SensirionI2CDevice::write_command_(uint16_t command, CommandLen command_len temp[raw_idx++] = data[i] & 0xFF; temp[raw_idx++] = data[i] >> 8; #endif - temp[raw_idx++] = sht_crc_(data[i]); + // Use MSB first since Sensirion devices use CRC-8 with MSB first + uint8_t crc = crc8(&temp[raw_idx - 2], 2, 0xFF, CRC_POLYNOMIAL, true); + temp[raw_idx++] = crc; } - last_error_ = this->write(temp, raw_idx); - return last_error_ == i2c::ERROR_OK; + this->last_error_ = this->write(temp, raw_idx); + return this->last_error_ == i2c::ERROR_OK; } -bool SensirionI2CDevice::get_register_(uint16_t reg, CommandLen command_len, uint16_t *data, uint8_t len, - uint8_t delay_ms) { +bool SensirionI2CDevice::get_register_(uint16_t reg, CommandLen command_len, uint16_t *data, const uint8_t len, + const uint8_t delay_ms) { if (!this->write_command_(reg, command_len, nullptr, 0)) { - ESP_LOGE(TAG, "Failed to write i2c register=0x%X (%d) err=%d,", reg, command_len, this->last_error_); + ESP_LOGE(TAG, "Write failed: reg=0x%X (%d) err=%d,", reg, command_len, this->last_error_); return false; } delay(delay_ms); bool result = this->read_data(data, len); if (!result) { - ESP_LOGE(TAG, "Failed to read data from register=0x%X err=%d,", reg, this->last_error_); + ESP_LOGE(TAG, "Read failed: reg=0x%X err=%d,", reg, this->last_error_); } return result; } -// The 8-bit CRC checksum is transmitted after each data word -uint8_t SensirionI2CDevice::sht_crc_(uint16_t data) { - uint8_t bit; - uint8_t crc = 0xFF; -#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ - crc ^= data >> 8; -#else - crc ^= data & 0xFF; -#endif - for (bit = 8; bit > 0; --bit) { - if (crc & 0x80) { - crc = (crc << 1) ^ crc_polynomial_; - } else { - crc = (crc << 1); - } - } -#if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ - crc ^= data & 0xFF; -#else - crc ^= data >> 8; -#endif - for (bit = 8; bit > 0; --bit) { - if (crc & 0x80) { - crc = (crc << 1) ^ crc_polynomial_; - } else { - crc = (crc << 1); - } - } - return crc; -} - } // namespace sensirion_common } // namespace esphome diff --git a/esphome/components/sensirion_common/i2c_sensirion.h b/esphome/components/sensirion_common/i2c_sensirion.h index aba93d6cc3..f3eb3761f6 100644 --- a/esphome/components/sensirion_common/i2c_sensirion.h +++ b/esphome/components/sensirion_common/i2c_sensirion.h @@ -8,90 +8,92 @@ namespace esphome { namespace sensirion_common { /** - * Implementation of a i2c functions for Sensirion sensors - * Sensirion data requires crc checking. + * Implementation of I2C functions for Sensirion sensors + * Sensirion data requires CRC checking. * Each 16 bit word is/must be followed 8 bit CRC code - * (Applies to read and write - note the i2c command code doesn't need a CRC) + * (Applies to read and write - note the I2C command code doesn't need a CRC) * Format: * | 16 Bit Command Code | 16 bit Data word 1 | CRC of DW 1 | 16 bit Data word 1 | CRC of DW 2 | .. */ +static const uint8_t CRC_POLYNOMIAL = 0x31; // default for Sensirion + class SensirionI2CDevice : public i2c::I2CDevice { public: enum CommandLen : uint8_t { ADDR_8_BIT = 1, ADDR_16_BIT = 2 }; - /** Read data words from i2c device. - * handles crc check used by Sensirion sensors + /** Read data words from I2C device. + * handles CRC check used by Sensirion sensors * @param data pointer to raw result * @param len number of words to read * @return true if reading succeeded */ bool read_data(uint16_t *data, uint8_t len); - /** Read 1 data word from i2c device. + /** Read 1 data word from I2C device. * @param data reference to raw result * @return true if reading succeeded */ bool read_data(uint16_t &data) { return this->read_data(&data, 1); } - /** get data words from i2c register. - * handles crc check used by Sensirion sensors - * @param i2c register + /** get data words from I2C register. + * handles CRC check used by Sensirion sensors + * @param I2C register * @param data pointer to raw result * @param len number of words to read - * @param delay milliseconds to to wait between sending the i2c command and reading the result + * @param delay milliseconds to to wait between sending the I2C command and reading the result * @return true if reading succeeded */ bool get_register(uint16_t command, uint16_t *data, uint8_t len, uint8_t delay = 0) { return get_register_(command, ADDR_16_BIT, data, len, delay); } - /** Read 1 data word from 16 bit i2c register. - * @param i2c register + /** Read 1 data word from 16 bit I2C register. + * @param I2C register * @param data reference to raw result - * @param delay milliseconds to to wait between sending the i2c command and reading the result + * @param delay milliseconds to to wait between sending the I2C command and reading the result * @return true if reading succeeded */ bool get_register(uint16_t i2c_register, uint16_t &data, uint8_t delay = 0) { return this->get_register_(i2c_register, ADDR_16_BIT, &data, 1, delay); } - /** get data words from i2c register. - * handles crc check used by Sensirion sensors - * @param i2c register + /** get data words from I2C register. + * handles CRC check used by Sensirion sensors + * @param I2C register * @param data pointer to raw result * @param len number of words to read - * @param delay milliseconds to to wait between sending the i2c command and reading the result + * @param delay milliseconds to to wait between sending the I2C command and reading the result * @return true if reading succeeded */ bool get_8bit_register(uint8_t i2c_register, uint16_t *data, uint8_t len, uint8_t delay = 0) { return get_register_(i2c_register, ADDR_8_BIT, data, len, delay); } - /** Read 1 data word from 8 bit i2c register. - * @param i2c register + /** Read 1 data word from 8 bit I2C register. + * @param I2C register * @param data reference to raw result - * @param delay milliseconds to to wait between sending the i2c command and reading the result + * @param delay milliseconds to to wait between sending the I2C command and reading the result * @return true if reading succeeded */ bool get_8bit_register(uint8_t i2c_register, uint16_t &data, uint8_t delay = 0) { return this->get_register_(i2c_register, ADDR_8_BIT, &data, 1, delay); } - /** Write a command to the i2c device. - * @param command i2c command to send + /** Write a command to the I2C device. + * @param command I2C command to send * @return true if reading succeeded */ template bool write_command(T i2c_register) { return write_command(i2c_register, nullptr, 0); } - /** Write a command and one data word to the i2c device . - * @param command i2c command to send - * @param data argument for the i2c command + /** Write a command and one data word to the I2C device . + * @param command I2C command to send + * @param data argument for the I2C command * @return true if reading succeeded */ template bool write_command(T i2c_register, uint16_t data) { return write_command(i2c_register, &data, 1); } /** Write a command with arguments as words - * @param i2c_register i2c command to send - an be uint8_t or uint16_t - * @param data vector arguments for the i2c command + * @param i2c_register I2C command to send - an be uint8_t or uint16_t + * @param data vector arguments for the I2C command * @return true if reading succeeded */ template bool write_command(T i2c_register, const std::vector &data) { @@ -99,57 +101,39 @@ class SensirionI2CDevice : public i2c::I2CDevice { } /** Write a command with arguments as words - * @param i2c_register i2c command to send - an be uint8_t or uint16_t - * @param data arguments for the i2c command + * @param i2c_register I2C command to send - an be uint8_t or uint16_t + * @param data arguments for the I2C command * @param len number of arguments (words) * @return true if reading succeeded */ template bool write_command(T i2c_register, const uint16_t *data, uint8_t len) { // limit to 8 or 16 bit only - static_assert(sizeof(i2c_register) == 1 || sizeof(i2c_register) == 2, - "only 8 or 16 bit command types are supported."); + static_assert(sizeof(i2c_register) == 1 || sizeof(i2c_register) == 2, "Only 8 or 16 bit command types supported"); return write_command_(i2c_register, CommandLen(sizeof(T)), data, len); } protected: - uint8_t crc_polynomial_{0x31u}; // default for sensirion /** Write a command with arguments as words - * @param command i2c command to send can be uint8_t or uint16_t + * @param command I2C command to send can be uint8_t or uint16_t * @param command_len either 1 for short 8 bit command or 2 for 16 bit command codes - * @param data arguments for the i2c command + * @param data arguments for the I2C command * @param data_len number of arguments (words) * @return true if reading succeeded */ bool write_command_(uint16_t command, CommandLen command_len, const uint16_t *data, uint8_t data_len); - /** get data words from i2c register. - * handles crc check used by Sensirion sensors - * @param i2c register + /** get data words from I2C register. + * handles CRC check used by Sensirion sensors + * @param I2C register * @param command_len either 1 for short 8 bit command or 2 for 16 bit command codes * @param data pointer to raw result * @param len number of words to read - * @param delay milliseconds to to wait between sending the i2c command and reading the result + * @param delay milliseconds to to wait between sending the I2C command and reading the result * @return true if reading succeeded */ bool get_register_(uint16_t reg, CommandLen command_len, uint16_t *data, uint8_t len, uint8_t delay); - /** 8-bit CRC checksum that is transmitted after each data word for read and write operation - * @param command i2c command to send - * @param data data word for which the crc8 checksum is calculated - * @param len number of arguments (words) - * @return 8 Bit CRC - */ - uint8_t sht_crc_(uint16_t data); - - /** 8-bit CRC checksum that is transmitted after each data word for read and write operation - * @param command i2c command to send - * @param data1 high byte of data word - * @param data2 low byte of data word - * @return 8 Bit CRC - */ - uint8_t sht_crc_(uint8_t data1, uint8_t data2) { return sht_crc_(encode_uint16(data1, data2)); } - - /** last error code from i2c operation + /** last error code from I2C operation */ i2c::ErrorCode last_error_; }; diff --git a/esphome/components/sensor/__init__.py b/esphome/components/sensor/__init__.py index ea74361d51..bf13217787 100644 --- a/esphome/components/sensor/__init__.py +++ b/esphome/components/sensor/__init__.py @@ -41,6 +41,7 @@ from esphome.const import ( CONF_VALUE, CONF_WEB_SERVER, CONF_WINDOW_SIZE, + DEVICE_CLASS_ABSOLUTE_HUMIDITY, DEVICE_CLASS_APPARENT_POWER, DEVICE_CLASS_AQI, DEVICE_CLASS_AREA, @@ -73,6 +74,7 @@ from esphome.const import ( DEVICE_CLASS_OZONE, DEVICE_CLASS_PH, DEVICE_CLASS_PM1, + DEVICE_CLASS_PM4, DEVICE_CLASS_PM10, DEVICE_CLASS_PM25, DEVICE_CLASS_POWER, @@ -87,6 +89,7 @@ from esphome.const import ( DEVICE_CLASS_SPEED, DEVICE_CLASS_SULPHUR_DIOXIDE, DEVICE_CLASS_TEMPERATURE, + DEVICE_CLASS_TEMPERATURE_DELTA, DEVICE_CLASS_TIMESTAMP, DEVICE_CLASS_VOLATILE_ORGANIC_COMPOUNDS, DEVICE_CLASS_VOLATILE_ORGANIC_COMPOUNDS_PARTS, @@ -100,13 +103,14 @@ from esphome.const import ( DEVICE_CLASS_WIND_SPEED, ENTITY_CATEGORY_CONFIG, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass from esphome.util import Registry CODEOWNERS = ["@esphome/core"] DEVICE_CLASSES = [ + DEVICE_CLASS_ABSOLUTE_HUMIDITY, DEVICE_CLASS_APPARENT_POWER, DEVICE_CLASS_AQI, DEVICE_CLASS_AREA, @@ -141,6 +145,7 @@ DEVICE_CLASSES = [ DEVICE_CLASS_PM1, DEVICE_CLASS_PM10, DEVICE_CLASS_PM25, + DEVICE_CLASS_PM4, DEVICE_CLASS_POWER, DEVICE_CLASS_POWER_FACTOR, DEVICE_CLASS_PRECIPITATION, @@ -153,6 +158,7 @@ DEVICE_CLASSES = [ DEVICE_CLASS_SPEED, DEVICE_CLASS_SULPHUR_DIOXIDE, DEVICE_CLASS_TEMPERATURE, + DEVICE_CLASS_TEMPERATURE_DELTA, DEVICE_CLASS_TIMESTAMP, DEVICE_CLASS_VOLATILE_ORGANIC_COMPOUNDS, DEVICE_CLASS_VOLATILE_ORGANIC_COMPOUNDS_PARTS, @@ -254,6 +260,7 @@ OffsetFilter = sensor_ns.class_("OffsetFilter", Filter) MultiplyFilter = sensor_ns.class_("MultiplyFilter", Filter) FilterOutValueFilter = sensor_ns.class_("FilterOutValueFilter", Filter) ThrottleFilter = sensor_ns.class_("ThrottleFilter", Filter) +ThrottleWithPriorityFilter = sensor_ns.class_("ThrottleWithPriorityFilter", Filter) TimeoutFilter = sensor_ns.class_("TimeoutFilter", Filter, cg.Component) DebounceFilter = sensor_ns.class_("DebounceFilter", Filter, cg.Component) HeartbeatFilter = sensor_ns.class_("HeartbeatFilter", Filter, cg.Component) @@ -330,6 +337,7 @@ def sensor_schema( device_class: str = cv.UNDEFINED, state_class: str = cv.UNDEFINED, entity_category: str = cv.UNDEFINED, + filters: list = cv.UNDEFINED, ) -> cv.Schema: schema = {} @@ -344,6 +352,7 @@ def sensor_schema( (CONF_DEVICE_CLASS, device_class, validate_device_class), (CONF_STATE_CLASS, state_class, validate_state_class), (CONF_ENTITY_CATEGORY, entity_category, sensor_entity_category), + (CONF_FILTERS, filters, validate_filters), ]: if default is not cv.UNDEFINED: schema[cv.Optional(key, default=default)] = validator @@ -591,6 +600,29 @@ async def throttle_filter_to_code(config, filter_id): return cg.new_Pvariable(filter_id, config) +THROTTLE_WITH_PRIORITY_SCHEMA = cv.maybe_simple_value( + { + cv.Required(CONF_TIMEOUT): cv.positive_time_period_milliseconds, + cv.Optional(CONF_VALUE, default="nan"): cv.Any( + cv.templatable(cv.float_), [cv.templatable(cv.float_)] + ), + }, + key=CONF_TIMEOUT, +) + + +@FILTER_REGISTRY.register( + "throttle_with_priority", + ThrottleWithPriorityFilter, + THROTTLE_WITH_PRIORITY_SCHEMA, +) +async def throttle_with_priority_filter_to_code(config, filter_id): + if not isinstance(config[CONF_VALUE], list): + config[CONF_VALUE] = [config[CONF_VALUE]] + template_ = [await cg.templatable(x, [], float) for x in config[CONF_VALUE]] + return cg.new_Pvariable(filter_id, config[CONF_TIMEOUT], template_) + + @FILTER_REGISTRY.register( "heartbeat", HeartbeatFilter, cv.positive_time_period_milliseconds ) @@ -603,7 +635,9 @@ async def heartbeat_filter_to_code(config, filter_id): TIMEOUT_SCHEMA = cv.maybe_simple_value( { cv.Required(CONF_TIMEOUT): cv.positive_time_period_milliseconds, - cv.Optional(CONF_VALUE, default="nan"): cv.templatable(cv.float_), + cv.Optional(CONF_VALUE, default="nan"): cv.Any( + "last", cv.templatable(cv.float_) + ), }, key=CONF_TIMEOUT, ) @@ -611,8 +645,11 @@ TIMEOUT_SCHEMA = cv.maybe_simple_value( @FILTER_REGISTRY.register("timeout", TimeoutFilter, TIMEOUT_SCHEMA) async def timeout_filter_to_code(config, filter_id): - template_ = await cg.templatable(config[CONF_VALUE], [], float) - var = cg.new_Pvariable(filter_id, config[CONF_TIMEOUT], template_) + if config[CONF_VALUE] == "last": + var = cg.new_Pvariable(filter_id, config[CONF_TIMEOUT]) + else: + template_ = await cg.templatable(config[CONF_VALUE], [], float) + var = cg.new_Pvariable(filter_id, config[CONF_TIMEOUT], template_) await cg.register_component(var, {}) return var @@ -1109,7 +1146,6 @@ def _lstsq(a, b): return _mat_dot(_mat_dot(x, a_t), b) -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_SENSOR") cg.add_global(sensor_ns.using) diff --git a/esphome/components/sensor/automation.h b/esphome/components/sensor/automation.h index 8cd0adbeb2..4f34c35023 100644 --- a/esphome/components/sensor/automation.h +++ b/esphome/components/sensor/automation.h @@ -40,7 +40,7 @@ class ValueRangeTrigger : public Trigger, public Component { template void set_max(V max) { this->max_ = max; } void setup() override { - this->rtc_ = global_preferences->make_preference(this->parent_->get_object_id_hash()); + this->rtc_ = global_preferences->make_preference(this->parent_->get_preference_hash()); bool initial_state; if (this->rtc_.load(&initial_state)) { this->previous_in_range_ = initial_state; diff --git a/esphome/components/sensor/filter.cpp b/esphome/components/sensor/filter.cpp index dd8635f0c0..3241ae28af 100644 --- a/esphome/components/sensor/filter.cpp +++ b/esphome/components/sensor/filter.cpp @@ -1,5 +1,6 @@ #include "filter.h" #include +#include "esphome/core/application.h" #include "esphome/core/hal.h" #include "esphome/core/log.h" #include "sensor.h" @@ -50,6 +51,7 @@ optional MedianFilter::new_value(float value) { if (!this->queue_.empty()) { // Copy queue without NaN values std::vector median_queue; + median_queue.reserve(this->queue_.size()); for (auto v : this->queue_) { if (!std::isnan(v)) { median_queue.push_back(v); @@ -223,7 +225,7 @@ optional SlidingWindowMovingAverageFilter::new_value(float value) { // ExponentialMovingAverageFilter ExponentialMovingAverageFilter::ExponentialMovingAverageFilter(float alpha, size_t send_every, size_t send_first_at) - : send_every_(send_every), send_at_(send_every - send_first_at), alpha_(alpha) {} + : alpha_(alpha), send_every_(send_every), send_at_(send_every - send_first_at) {} optional ExponentialMovingAverageFilter::new_value(float value) { if (!std::isnan(value)) { if (this->first_value_) { @@ -323,7 +325,7 @@ optional FilterOutValueFilter::new_value(float value) { // ThrottleFilter ThrottleFilter::ThrottleFilter(uint32_t min_time_between_inputs) : min_time_between_inputs_(min_time_between_inputs) {} optional ThrottleFilter::new_value(float value) { - const uint32_t now = millis(); + const uint32_t now = App.get_loop_component_start_time(); if (this->last_input_ == 0 || now - this->last_input_ >= min_time_between_inputs_) { this->last_input_ = now; return value; @@ -331,21 +333,53 @@ optional ThrottleFilter::new_value(float value) { return {}; } +// ThrottleWithPriorityFilter +ThrottleWithPriorityFilter::ThrottleWithPriorityFilter(uint32_t min_time_between_inputs, + std::vector> prioritized_values) + : min_time_between_inputs_(min_time_between_inputs), prioritized_values_(std::move(prioritized_values)) {} + +optional ThrottleWithPriorityFilter::new_value(float value) { + bool is_prioritized_value = false; + int8_t accuracy = this->parent_->get_accuracy_decimals(); + float accuracy_mult = powf(10.0f, accuracy); + const uint32_t now = App.get_loop_component_start_time(); + // First, determine if the new value is one of the prioritized values + for (auto prioritized_value : this->prioritized_values_) { + if (std::isnan(prioritized_value.value())) { + if (std::isnan(value)) { + is_prioritized_value = true; + break; + } + continue; + } + float rounded_prioritized_value = roundf(accuracy_mult * prioritized_value.value()); + float rounded_value = roundf(accuracy_mult * value); + if (rounded_prioritized_value == rounded_value) { + is_prioritized_value = true; + break; + } + } + // Finally, determine if the new value should be throttled and pass it through if not + if (this->last_input_ == 0 || now - this->last_input_ >= min_time_between_inputs_ || is_prioritized_value) { + this->last_input_ = now; + return value; + } + return {}; +} + // DeltaFilter DeltaFilter::DeltaFilter(float delta, bool percentage_mode) - : delta_(delta), current_delta_(delta), percentage_mode_(percentage_mode), last_value_(NAN) {} + : delta_(delta), current_delta_(delta), last_value_(NAN), percentage_mode_(percentage_mode) {} optional DeltaFilter::new_value(float value) { if (std::isnan(value)) { if (std::isnan(this->last_value_)) { return {}; } else { - if (this->percentage_mode_) { - this->current_delta_ = fabsf(value * this->delta_); - } return this->last_value_ = value; } } - if (std::isnan(this->last_value_) || fabsf(value - this->last_value_) >= this->current_delta_) { + float diff = fabsf(value - this->last_value_); + if (std::isnan(this->last_value_) || (diff > 0.0f && diff >= this->current_delta_)) { if (this->percentage_mode_) { this->current_delta_ = fabsf(value * this->delta_); } @@ -383,12 +417,17 @@ void OrFilter::initialize(Sensor *parent, Filter *next) { // TimeoutFilter optional TimeoutFilter::new_value(float value) { - this->set_timeout("timeout", this->time_period_, [this]() { this->output(this->value_.value()); }); + if (this->value_.has_value()) { + this->set_timeout("timeout", this->time_period_, [this]() { this->output(this->value_.value().value()); }); + } else { + this->set_timeout("timeout", this->time_period_, [this, value]() { this->output(value); }); + } return value; } -TimeoutFilter::TimeoutFilter(uint32_t time_period, TemplatableValue new_value) - : time_period_(time_period), value_(std::move(new_value)) {} +TimeoutFilter::TimeoutFilter(uint32_t time_period) : time_period_(time_period) {} +TimeoutFilter::TimeoutFilter(uint32_t time_period, const TemplatableValue &new_value) + : time_period_(time_period), value_(new_value) {} float TimeoutFilter::get_setup_priority() const { return setup_priority::HARDWARE; } // DebounceFilter diff --git a/esphome/components/sensor/filter.h b/esphome/components/sensor/filter.h index 94fec8208b..49d83e5b4b 100644 --- a/esphome/components/sensor/filter.h +++ b/esphome/components/sensor/filter.h @@ -221,11 +221,11 @@ class ExponentialMovingAverageFilter : public Filter { void set_alpha(float alpha); protected: - bool first_value_{true}; float accumulator_{NAN}; + float alpha_; size_t send_every_; size_t send_at_; - float alpha_; + bool first_value_{true}; }; /** Simple throttle average filter. @@ -243,9 +243,9 @@ class ThrottleAverageFilter : public Filter, public Component { float get_setup_priority() const override; protected: - uint32_t time_period_; float sum_{0.0f}; unsigned int n_{0}; + uint32_t time_period_; bool have_nan_{false}; }; @@ -314,9 +314,24 @@ class ThrottleFilter : public Filter { uint32_t min_time_between_inputs_; }; +/// Same as 'throttle' but will immediately publish values contained in `value_to_prioritize`. +class ThrottleWithPriorityFilter : public Filter { + public: + explicit ThrottleWithPriorityFilter(uint32_t min_time_between_inputs, + std::vector> prioritized_values); + + optional new_value(float value) override; + + protected: + uint32_t last_input_{0}; + uint32_t min_time_between_inputs_; + std::vector> prioritized_values_; +}; + class TimeoutFilter : public Filter, public Component { public: - explicit TimeoutFilter(uint32_t time_period, TemplatableValue new_value); + explicit TimeoutFilter(uint32_t time_period); + explicit TimeoutFilter(uint32_t time_period, const TemplatableValue &new_value); optional new_value(float value) override; @@ -324,7 +339,7 @@ class TimeoutFilter : public Filter, public Component { protected: uint32_t time_period_; - TemplatableValue value_; + optional> value_; }; class DebounceFilter : public Filter, public Component { @@ -364,8 +379,8 @@ class DeltaFilter : public Filter { protected: float delta_; float current_delta_; - bool percentage_mode_; float last_value_{NAN}; + bool percentage_mode_; }; class OrFilter : public Filter { @@ -387,8 +402,8 @@ class OrFilter : public Filter { }; std::vector filters_; - bool has_value_{false}; PhiNode phi_; + bool has_value_{false}; }; class CalibrateLinearFilter : public Filter { diff --git a/esphome/components/sensor/sensor.cpp b/esphome/components/sensor/sensor.cpp index 7dab63b026..4292b8c0bc 100644 --- a/esphome/components/sensor/sensor.cpp +++ b/esphome/components/sensor/sensor.cpp @@ -6,17 +6,45 @@ namespace sensor { static const char *const TAG = "sensor"; -std::string state_class_to_string(StateClass state_class) { +// Function implementation of LOG_SENSOR macro to reduce code size +void log_sensor(const char *tag, const char *prefix, const char *type, Sensor *obj) { + if (obj == nullptr) { + return; + } + + ESP_LOGCONFIG(tag, + "%s%s '%s'\n" + "%s State Class: '%s'\n" + "%s Unit of Measurement: '%s'\n" + "%s Accuracy Decimals: %d", + prefix, type, obj->get_name().c_str(), prefix, + LOG_STR_ARG(state_class_to_string(obj->get_state_class())), prefix, + obj->get_unit_of_measurement_ref().c_str(), prefix, obj->get_accuracy_decimals()); + + if (!obj->get_device_class_ref().empty()) { + ESP_LOGCONFIG(tag, "%s Device Class: '%s'", prefix, obj->get_device_class_ref().c_str()); + } + + if (!obj->get_icon_ref().empty()) { + ESP_LOGCONFIG(tag, "%s Icon: '%s'", prefix, obj->get_icon_ref().c_str()); + } + + if (obj->get_force_update()) { + ESP_LOGV(tag, "%s Force Update: YES", prefix); + } +} + +const LogString *state_class_to_string(StateClass state_class) { switch (state_class) { case STATE_CLASS_MEASUREMENT: - return "measurement"; + return LOG_STR("measurement"); case STATE_CLASS_TOTAL_INCREASING: - return "total_increasing"; + return LOG_STR("total_increasing"); case STATE_CLASS_TOTAL: - return "total"; + return LOG_STR("total"); case STATE_CLASS_NONE: default: - return ""; + return LOG_STR(""); } } @@ -96,13 +124,12 @@ void Sensor::clear_filters() { } float Sensor::get_state() const { return this->state; } float Sensor::get_raw_state() const { return this->raw_state; } -std::string Sensor::unique_id() { return ""; } void Sensor::internal_send_state_to_frontend(float state) { this->set_has_state(true); this->state = state; ESP_LOGD(TAG, "'%s': Sending state %.5f %s with %d decimals of accuracy", this->get_name().c_str(), state, - this->get_unit_of_measurement().c_str(), this->get_accuracy_decimals()); + this->get_unit_of_measurement_ref().c_str(), this->get_accuracy_decimals()); this->callback_.call(state); } diff --git a/esphome/components/sensor/sensor.h b/esphome/components/sensor/sensor.h index 3fb6e5522b..f3fa601a5e 100644 --- a/esphome/components/sensor/sensor.h +++ b/esphome/components/sensor/sensor.h @@ -12,29 +12,9 @@ namespace esphome { namespace sensor { -#define LOG_SENSOR(prefix, type, obj) \ - if ((obj) != nullptr) { \ - ESP_LOGCONFIG(TAG, \ - "%s%s '%s'\n" \ - "%s State Class: '%s'\n" \ - "%s Unit of Measurement: '%s'\n" \ - "%s Accuracy Decimals: %d", \ - prefix, LOG_STR_LITERAL(type), (obj)->get_name().c_str(), prefix, \ - state_class_to_string((obj)->get_state_class()).c_str(), prefix, \ - (obj)->get_unit_of_measurement().c_str(), prefix, (obj)->get_accuracy_decimals()); \ - if (!(obj)->get_device_class().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Device Class: '%s'", prefix, (obj)->get_device_class().c_str()); \ - } \ - if (!(obj)->get_icon().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon().c_str()); \ - } \ - if (!(obj)->unique_id().empty()) { \ - ESP_LOGV(TAG, "%s Unique ID: '%s'", prefix, (obj)->unique_id().c_str()); \ - } \ - if ((obj)->get_force_update()) { \ - ESP_LOGV(TAG, "%s Force Update: YES", prefix); \ - } \ - } +void log_sensor(const char *tag, const char *prefix, const char *type, Sensor *obj); + +#define LOG_SENSOR(prefix, type, obj) log_sensor(TAG, prefix, LOG_STR_LITERAL(type), obj) #define SUB_SENSOR(name) \ protected: \ @@ -53,7 +33,7 @@ enum StateClass : uint8_t { STATE_CLASS_TOTAL = 3, }; -std::string state_class_to_string(StateClass state_class); +const LogString *state_class_to_string(StateClass state_class); /** Base-class for all sensors. * @@ -141,12 +121,6 @@ class Sensor : public EntityBase, public EntityBase_DeviceClass, public EntityBa */ float raw_state; - /** Override this method to set the unique ID of this sensor. - * - * @deprecated Do not use for new sensors, a suitable unique ID is automatically generated (2023.4). - */ - virtual std::string unique_id(); - void internal_send_state_to_frontend(float state); protected: diff --git a/esphome/components/sfa30/sfa30.cpp b/esphome/components/sfa30/sfa30.cpp index c521b3aa02..99709d5fbb 100644 --- a/esphome/components/sfa30/sfa30.cpp +++ b/esphome/components/sfa30/sfa30.cpp @@ -11,8 +11,6 @@ static const uint16_t SFA30_CMD_START_CONTINUOUS_MEASUREMENTS = 0x0006; static const uint16_t SFA30_CMD_READ_MEASUREMENT = 0x0327; void SFA30Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // Serial Number identification uint16_t raw_device_marking[16]; if (!this->get_register(SFA30_CMD_GET_DEVICE_MARKING, raw_device_marking, 16, 5)) { @@ -34,8 +32,6 @@ void SFA30Component::setup() { this->mark_failed(); return; } - - ESP_LOGD(TAG, "Sensor initialized"); } void SFA30Component::dump_config() { diff --git a/esphome/components/sgp30/sgp30.cpp b/esphome/components/sgp30/sgp30.cpp index 0c7f25b699..0e3aeff812 100644 --- a/esphome/components/sgp30/sgp30.cpp +++ b/esphome/components/sgp30/sgp30.cpp @@ -1,9 +1,11 @@ #include "sgp30.h" -#include #include "esphome/core/application.h" #include "esphome/core/hal.h" +#include "esphome/core/helpers.h" #include "esphome/core/log.h" +#include + namespace esphome { namespace sgp30 { @@ -33,17 +35,14 @@ const uint32_t SHORTEST_BASELINE_STORE_INTERVAL = 3600; const uint32_t MAXIMUM_STORAGE_DIFF = 50; void SGP30Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // Serial Number identification uint16_t raw_serial_number[3]; if (!this->get_register(SGP30_CMD_GET_SERIAL_ID, raw_serial_number, 3)) { this->mark_failed(); return; } - this->serial_number_ = (uint64_t(raw_serial_number[0]) << 24) | (uint64_t(raw_serial_number[1]) << 16) | - (uint64_t(raw_serial_number[2])); - ESP_LOGD(TAG, "Serial Number: %" PRIu64, this->serial_number_); + this->serial_number_ = encode_uint24(raw_serial_number[0], raw_serial_number[1], raw_serial_number[2]); + ESP_LOGD(TAG, "Serial number: %" PRIu64, this->serial_number_); // Featureset identification for future use uint16_t raw_featureset; @@ -63,11 +62,11 @@ void SGP30Component::setup() { this->mark_failed(); return; } - ESP_LOGD(TAG, "Product version: 0x%0X", uint16_t(this->featureset_ & 0x1FF)); + ESP_LOGV(TAG, "Product version: 0x%0X", uint16_t(this->featureset_ & 0x1FF)); // Sensor initialization if (!this->write_command(SGP30_CMD_IAQ_INIT)) { - ESP_LOGE(TAG, "Sensor sgp30_iaq_init failed."); + ESP_LOGE(TAG, "sgp30_iaq_init failed"); this->error_code_ = MEASUREMENT_INIT_FAILED; this->mark_failed(); return; @@ -125,7 +124,7 @@ void SGP30Component::read_iaq_baseline_() { uint16_t eco2baseline = (raw_data[0]); uint16_t tvocbaseline = (raw_data[1]); - ESP_LOGI(TAG, "Current eCO2 baseline: 0x%04X, TVOC baseline: 0x%04X", eco2baseline, tvocbaseline); + ESP_LOGI(TAG, "Baselines: eCO2: 0x%04X, TVOC: 0x%04X", eco2baseline, tvocbaseline); if (eco2baseline != this->eco2_baseline_ || tvocbaseline != this->tvoc_baseline_) { this->eco2_baseline_ = eco2baseline; this->tvoc_baseline_ = tvocbaseline; @@ -144,7 +143,7 @@ void SGP30Component::read_iaq_baseline_() { this->baselines_storage_.eco2 = this->eco2_baseline_; this->baselines_storage_.tvoc = this->tvoc_baseline_; if (this->pref_.save(&this->baselines_storage_)) { - ESP_LOGI(TAG, "Store eCO2 baseline: 0x%04X, TVOC baseline: 0x%04X", this->baselines_storage_.eco2, + ESP_LOGI(TAG, "Store baselines: eCO2: 0x%04X, TVOC: 0x%04X", this->baselines_storage_.eco2, this->baselines_storage_.tvoc); } else { ESP_LOGW(TAG, "Could not store eCO2 and TVOC baselines"); @@ -166,7 +165,7 @@ void SGP30Component::send_env_data_() { if (this->humidity_sensor_ != nullptr) humidity = this->humidity_sensor_->state; if (std::isnan(humidity) || humidity < 0.0f || humidity > 100.0f) { - ESP_LOGW(TAG, "Compensation not possible yet: bad humidity data."); + ESP_LOGW(TAG, "Compensation not possible yet: bad humidity data"); return; } else { ESP_LOGD(TAG, "External compensation data received: Humidity %0.2f%%", humidity); @@ -176,7 +175,7 @@ void SGP30Component::send_env_data_() { temperature = float(this->temperature_sensor_->state); } if (std::isnan(temperature) || temperature < -40.0f || temperature > 85.0f) { - ESP_LOGW(TAG, "Compensation not possible yet: bad temperature value data."); + ESP_LOGW(TAG, "Compensation not possible yet: bad temperature value"); return; } else { ESP_LOGD(TAG, "External compensation data received: Temperature %0.2f°C", temperature); @@ -194,18 +193,17 @@ void SGP30Component::send_env_data_() { ((humidity * 0.061121f * std::exp((18.678f - temperature / 234.5f) * (temperature / (257.14f + temperature)))) / (273.15f + temperature)); } - uint8_t humidity_full = uint8_t(std::floor(absolute_humidity)); - uint8_t humidity_dec = uint8_t(std::floor((absolute_humidity - std::floor(absolute_humidity)) * 256)); - ESP_LOGD(TAG, "Calculated Absolute humidity: %0.3f g/m³ (0x%04X)", absolute_humidity, - uint16_t(uint16_t(humidity_full) << 8 | uint16_t(humidity_dec))); - uint8_t crc = sht_crc_(humidity_full, humidity_dec); - uint8_t data[4]; - data[0] = SGP30_CMD_SET_ABSOLUTE_HUMIDITY & 0xFF; - data[1] = humidity_full; - data[2] = humidity_dec; - data[3] = crc; + uint8_t data[4] = { + SGP30_CMD_SET_ABSOLUTE_HUMIDITY & 0xFF, + uint8_t(std::floor(absolute_humidity)), // humidity_full + uint8_t(std::floor((absolute_humidity - std::floor(absolute_humidity)) * 256)), // humidity_dec + 0, + }; + data[3] = crc8(&data[1], 2, 0xFF, sensirion_common::CRC_POLYNOMIAL, true); + ESP_LOGD(TAG, "Calculated absolute humidity: %0.3f g/m³ (0x%04X)", absolute_humidity, + encode_uint16(data[1], data[2])); if (!this->write_bytes(SGP30_CMD_SET_ABSOLUTE_HUMIDITY >> 8, data, 4)) { - ESP_LOGE(TAG, "Error sending compensation data."); + ESP_LOGE(TAG, "Error sending compensation data"); } } @@ -214,15 +212,14 @@ void SGP30Component::write_iaq_baseline_(uint16_t eco2_baseline, uint16_t tvoc_b data[0] = SGP30_CMD_SET_IAQ_BASELINE & 0xFF; data[1] = tvoc_baseline >> 8; data[2] = tvoc_baseline & 0xFF; - data[3] = sht_crc_(data[1], data[2]); + data[3] = crc8(&data[1], 2, 0xFF, sensirion_common::CRC_POLYNOMIAL, true); data[4] = eco2_baseline >> 8; data[5] = eco2_baseline & 0xFF; - data[6] = sht_crc_(data[4], data[5]); + data[6] = crc8(&data[4], 2, 0xFF, sensirion_common::CRC_POLYNOMIAL, true); if (!this->write_bytes(SGP30_CMD_SET_IAQ_BASELINE >> 8, data, 7)) { - ESP_LOGE(TAG, "Error applying eCO2 baseline: 0x%04X, TVOC baseline: 0x%04X", eco2_baseline, tvoc_baseline); + ESP_LOGE(TAG, "Error applying baselines: eCO2: 0x%04X, TVOC: 0x%04X", eco2_baseline, tvoc_baseline); } else { - ESP_LOGI(TAG, "Initial baselines applied successfully! eCO2 baseline: 0x%04X, TVOC baseline: 0x%04X", eco2_baseline, - tvoc_baseline); + ESP_LOGI(TAG, "Initial baselines applied: eCO2: 0x%04X, TVOC: 0x%04X", eco2_baseline, tvoc_baseline); } } @@ -238,10 +235,10 @@ void SGP30Component::dump_config() { ESP_LOGW(TAG, "Measurement Initialization failed"); break; case INVALID_ID: - ESP_LOGW(TAG, "Sensor reported an invalid ID. Is this an SGP30?"); + ESP_LOGW(TAG, "Invalid ID"); break; case UNSUPPORTED_ID: - ESP_LOGW(TAG, "Sensor reported an unsupported ID (SGPC3)"); + ESP_LOGW(TAG, "Unsupported ID"); break; default: ESP_LOGW(TAG, "Unknown setup error"); @@ -251,12 +248,12 @@ void SGP30Component::dump_config() { ESP_LOGCONFIG(TAG, " Serial number: %" PRIu64, this->serial_number_); if (this->eco2_baseline_ != 0x0000 && this->tvoc_baseline_ != 0x0000) { ESP_LOGCONFIG(TAG, - " Baseline:\n" - " eCO2 Baseline: 0x%04X\n" - " TVOC Baseline: 0x%04X", + " Baselines:\n" + " eCO2: 0x%04X\n" + " TVOC: 0x%04X", this->eco2_baseline_, this->tvoc_baseline_); } else { - ESP_LOGCONFIG(TAG, " Baseline: No baseline configured"); + ESP_LOGCONFIG(TAG, " Baselines not configured"); } ESP_LOGCONFIG(TAG, " Warm up time: %" PRIu32 "s", this->required_warm_up_time_); } @@ -268,8 +265,8 @@ void SGP30Component::dump_config() { ESP_LOGCONFIG(TAG, "Store baseline: %s", YESNO(this->store_baseline_)); if (this->humidity_sensor_ != nullptr && this->temperature_sensor_ != nullptr) { ESP_LOGCONFIG(TAG, " Compensation:"); - LOG_SENSOR(" ", "Temperature Source:", this->temperature_sensor_); - LOG_SENSOR(" ", "Humidity Source:", this->humidity_sensor_); + LOG_SENSOR(" ", "Temperature source:", this->temperature_sensor_); + LOG_SENSOR(" ", "Humidity source:", this->humidity_sensor_); } else { ESP_LOGCONFIG(TAG, " Compensation: No source configured"); } @@ -291,7 +288,7 @@ void SGP30Component::update() { float eco2 = (raw_data[0]); float tvoc = (raw_data[1]); - ESP_LOGD(TAG, "Got eCO2=%.1fppm TVOC=%.1fppb", eco2, tvoc); + ESP_LOGV(TAG, "eCO2=%.1fppm TVOC=%.1fppb", eco2, tvoc); if (this->eco2_sensor_ != nullptr) this->eco2_sensor_->publish_state(eco2); if (this->tvoc_sensor_ != nullptr) diff --git a/esphome/components/sgp30/sgp30.h b/esphome/components/sgp30/sgp30.h index e6429a7bfa..4648a33e15 100644 --- a/esphome/components/sgp30/sgp30.h +++ b/esphome/components/sgp30/sgp30.h @@ -1,8 +1,8 @@ #pragma once -#include "esphome/core/component.h" -#include "esphome/components/sensor/sensor.h" #include "esphome/components/sensirion_common/i2c_sensirion.h" +#include "esphome/components/sensor/sensor.h" +#include "esphome/core/component.h" #include "esphome/core/preferences.h" #include @@ -38,14 +38,16 @@ class SGP30Component : public PollingComponent, public sensirion_common::Sensiri void read_iaq_baseline_(); bool is_sensor_baseline_reliable_(); void write_iaq_baseline_(uint16_t eco2_baseline, uint16_t tvoc_baseline); + uint64_t serial_number_; - uint16_t featureset_; uint32_t required_warm_up_time_; uint32_t seconds_since_last_store_; - SGP30Baselines baselines_storage_; - ESPPreferenceObject pref_; + uint16_t featureset_; + uint16_t eco2_baseline_{0x0000}; + uint16_t tvoc_baseline_{0x0000}; + bool store_baseline_; - enum ErrorCode { + enum ErrorCode : uint8_t { COMMUNICATION_FAILED, MEASUREMENT_INIT_FAILED, INVALID_ID, @@ -53,14 +55,13 @@ class SGP30Component : public PollingComponent, public sensirion_common::Sensiri UNKNOWN } error_code_{UNKNOWN}; + ESPPreferenceObject pref_; + SGP30Baselines baselines_storage_; + sensor::Sensor *eco2_sensor_{nullptr}; sensor::Sensor *tvoc_sensor_{nullptr}; sensor::Sensor *eco2_sensor_baseline_{nullptr}; sensor::Sensor *tvoc_sensor_baseline_{nullptr}; - uint16_t eco2_baseline_{0x0000}; - uint16_t tvoc_baseline_{0x0000}; - bool store_baseline_; - /// Input sensor for humidity and temperature compensation. sensor::Sensor *humidity_sensor_{nullptr}; sensor::Sensor *temperature_sensor_{nullptr}; diff --git a/esphome/components/sgp4x/sensor.py b/esphome/components/sgp4x/sensor.py index 4f29248881..7c6fe580b2 100644 --- a/esphome/components/sgp4x/sensor.py +++ b/esphome/components/sgp4x/sensor.py @@ -139,7 +139,7 @@ async def to_code(config): ) ) cg.add_library( - None, + "Sensirion Gas Index Algorithm", None, "https://github.com/Sensirion/arduino-gas-index-algorithm.git#3.2.1", ) diff --git a/esphome/components/sgp4x/sgp4x.cpp b/esphome/components/sgp4x/sgp4x.cpp index bd84ae97f3..99d88006f7 100644 --- a/esphome/components/sgp4x/sgp4x.cpp +++ b/esphome/components/sgp4x/sgp4x.cpp @@ -9,8 +9,6 @@ namespace sgp4x { static const char *const TAG = "sgp4x"; void SGP4xComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // Serial Number identification uint16_t raw_serial_number[3]; if (!this->get_register(SGP4X_CMD_GET_SERIAL_ID, raw_serial_number, 3, 1)) { @@ -213,7 +211,7 @@ void SGP4xComponent::measure_raw_() { if (!this->write_command(command, data, 2)) { ESP_LOGD(TAG, "write error (%d)", this->last_error_); - this->status_set_warning("measurement request failed"); + this->status_set_warning(LOG_STR("measurement request failed")); return; } @@ -222,7 +220,7 @@ void SGP4xComponent::measure_raw_() { raw_data[1] = 0; if (!this->read_data(raw_data, response_words)) { ESP_LOGD(TAG, "read error (%d)", this->last_error_); - this->status_set_warning("measurement read failed"); + this->status_set_warning(LOG_STR("measurement read failed")); this->voc_index_ = this->nox_index_ = UINT16_MAX; return; } diff --git a/esphome/components/sha256/__init__.py b/esphome/components/sha256/__init__.py new file mode 100644 index 0000000000..f07157416d --- /dev/null +++ b/esphome/components/sha256/__init__.py @@ -0,0 +1,22 @@ +import esphome.codegen as cg +import esphome.config_validation as cv +from esphome.core import CORE +from esphome.helpers import IS_MACOS +from esphome.types import ConfigType + +CODEOWNERS = ["@esphome/core"] + +sha256_ns = cg.esphome_ns.namespace("sha256") + +CONFIG_SCHEMA = cv.Schema({}) + + +async def to_code(config: ConfigType) -> None: + # Add OpenSSL library for host platform + if not CORE.is_host: + return + if IS_MACOS: + # macOS needs special handling for Homebrew OpenSSL + cg.add_build_flag("-I/opt/homebrew/opt/openssl/include") + cg.add_build_flag("-L/opt/homebrew/opt/openssl/lib") + cg.add_build_flag("-lcrypto") diff --git a/esphome/components/sha256/sha256.cpp b/esphome/components/sha256/sha256.cpp new file mode 100644 index 0000000000..32abbd739d --- /dev/null +++ b/esphome/components/sha256/sha256.cpp @@ -0,0 +1,116 @@ +#include "sha256.h" + +// Only compile SHA256 implementation on platforms that support it +#if defined(USE_ESP32) || defined(USE_ESP8266) || defined(USE_RP2040) || defined(USE_LIBRETINY) || defined(USE_HOST) + +#include "esphome/core/helpers.h" +#include + +namespace esphome::sha256 { + +#if defined(USE_ESP32) || defined(USE_LIBRETINY) + +// CRITICAL ESP32-S3 HARDWARE SHA ACCELERATION REQUIREMENTS: +// +// The ESP32-S3 uses hardware DMA for SHA acceleration. The mbedtls_sha256_context structure contains +// internal state that the DMA engine references. This imposes two critical constraints: +// +// 1. NO VARIABLE LENGTH ARRAYS (VLAs): VLAs corrupt the stack layout, causing the DMA engine to +// write to incorrect memory locations. This results in null pointer dereferences and crashes. +// ALWAYS use fixed-size arrays (e.g., char buf[65], not char buf[size+1]). +// +// 2. SAME STACK FRAME ONLY: The SHA256 object must be created and used entirely within the same +// function. NEVER pass the SHA256 object or HashBase pointer to another function. When the stack +// frame changes (function call/return), the DMA references become invalid and will produce +// truncated hash output (20 bytes instead of 32) or corrupt memory. +// +// CORRECT USAGE: +// void my_function() { +// sha256::SHA256 hasher; // Created locally +// hasher.init(); +// hasher.add(data, len); // Any size, no chunking needed +// hasher.calculate(); +// bool ok = hasher.equals_hex(expected); +// // hasher destroyed when function returns +// } +// +// INCORRECT USAGE (WILL FAIL ON ESP32-S3): +// void my_function() { +// sha256::SHA256 hasher; +// helper(&hasher); // WRONG: Passed to different stack frame +// } +// void helper(HashBase *h) { +// h->init(); // WRONG: Will produce truncated/corrupted output +// } + +SHA256::~SHA256() { mbedtls_sha256_free(&this->ctx_); } + +void SHA256::init() { + mbedtls_sha256_init(&this->ctx_); + mbedtls_sha256_starts(&this->ctx_, 0); // 0 = SHA256, not SHA224 +} + +void SHA256::add(const uint8_t *data, size_t len) { mbedtls_sha256_update(&this->ctx_, data, len); } + +void SHA256::calculate() { mbedtls_sha256_finish(&this->ctx_, this->digest_); } + +#elif defined(USE_ESP8266) || defined(USE_RP2040) + +SHA256::~SHA256() = default; + +void SHA256::init() { + br_sha256_init(&this->ctx_); + this->calculated_ = false; +} + +void SHA256::add(const uint8_t *data, size_t len) { br_sha256_update(&this->ctx_, data, len); } + +void SHA256::calculate() { + if (!this->calculated_) { + br_sha256_out(&this->ctx_, this->digest_); + this->calculated_ = true; + } +} + +#elif defined(USE_HOST) + +SHA256::~SHA256() { + if (this->ctx_) { + EVP_MD_CTX_free(this->ctx_); + } +} + +void SHA256::init() { + if (this->ctx_) { + EVP_MD_CTX_free(this->ctx_); + } + this->ctx_ = EVP_MD_CTX_new(); + EVP_DigestInit_ex(this->ctx_, EVP_sha256(), nullptr); + this->calculated_ = false; +} + +void SHA256::add(const uint8_t *data, size_t len) { + if (!this->ctx_) { + this->init(); + } + EVP_DigestUpdate(this->ctx_, data, len); +} + +void SHA256::calculate() { + if (!this->ctx_) { + this->init(); + } + if (!this->calculated_) { + unsigned int len = 32; + EVP_DigestFinal_ex(this->ctx_, this->digest_, &len); + this->calculated_ = true; + } +} + +#else +#error "SHA256 not supported on this platform" +#endif + +} // namespace esphome::sha256 + +#endif // Platform check diff --git a/esphome/components/sha256/sha256.h b/esphome/components/sha256/sha256.h new file mode 100644 index 0000000000..a2b62799e1 --- /dev/null +++ b/esphome/components/sha256/sha256.h @@ -0,0 +1,60 @@ +#pragma once + +#include "esphome/core/defines.h" + +// Only define SHA256 on platforms that support it +#if defined(USE_ESP32) || defined(USE_ESP8266) || defined(USE_RP2040) || defined(USE_LIBRETINY) || defined(USE_HOST) + +#include +#include +#include +#include "esphome/core/hash_base.h" + +#if defined(USE_ESP32) || defined(USE_LIBRETINY) +#include "mbedtls/sha256.h" +#elif defined(USE_ESP8266) || defined(USE_RP2040) +#include +#elif defined(USE_HOST) +#include +#else +#error "SHA256 not supported on this platform" +#endif + +namespace esphome::sha256 { + +class SHA256 : public esphome::HashBase { + public: + SHA256() = default; + ~SHA256() override; + + void init() override; + void add(const uint8_t *data, size_t len) override; + using HashBase::add; // Bring base class overload into scope + void add(const std::string &data) { this->add((const uint8_t *) data.c_str(), data.length()); } + + void calculate() override; + + /// Get the size of the hash in bytes (32 for SHA256) + size_t get_size() const override { return 32; } + + protected: +#if defined(USE_ESP32) || defined(USE_LIBRETINY) + // CRITICAL: The mbedtls context MUST be stack-allocated (not a pointer) for ESP32-S3 hardware SHA acceleration. + // The ESP32-S3 DMA engine references this structure's memory addresses. If the context is passed to another + // function (crossing stack frames) or if VLAs are present, the DMA operations will corrupt memory and produce + // truncated/incorrect hash results. + mbedtls_sha256_context ctx_{}; +#elif defined(USE_ESP8266) || defined(USE_RP2040) + br_sha256_context ctx_{}; + bool calculated_{false}; +#elif defined(USE_HOST) + EVP_MD_CTX *ctx_{nullptr}; + bool calculated_{false}; +#else +#error "SHA256 not supported on this platform" +#endif +}; + +} // namespace esphome::sha256 + +#endif // Platform check diff --git a/esphome/components/shelly_dimmer/light.py b/esphome/components/shelly_dimmer/light.py index bb2c3ceee8..c96bc380d7 100644 --- a/esphome/components/shelly_dimmer/light.py +++ b/esphome/components/shelly_dimmer/light.py @@ -183,7 +183,7 @@ CONFIG_SCHEMA = ( ) -def to_code(config): +async def to_code(config): fw_hex = get_firmware(config[CONF_FIRMWARE]) fw_major, fw_minor = parse_firmware_version(config[CONF_FIRMWARE][CONF_VERSION]) @@ -193,17 +193,17 @@ def to_code(config): cg.add_define("USE_SHD_FIRMWARE_MINOR_VERSION", fw_minor) var = cg.new_Pvariable(config[CONF_OUTPUT_ID]) - yield cg.register_component(var, config) + await cg.register_component(var, config) config.pop( CONF_UPDATE_INTERVAL ) # drop UPDATE_INTERVAL as it does not apply to the light component - yield light.register_light(var, config) - yield uart.register_uart_device(var, config) + await light.register_light(var, config) + await uart.register_uart_device(var, config) - nrst_pin = yield cg.gpio_pin_expression(config[CONF_NRST_PIN]) + nrst_pin = await cg.gpio_pin_expression(config[CONF_NRST_PIN]) cg.add(var.set_nrst_pin(nrst_pin)) - boot0_pin = yield cg.gpio_pin_expression(config[CONF_BOOT0_PIN]) + boot0_pin = await cg.gpio_pin_expression(config[CONF_BOOT0_PIN]) cg.add(var.set_boot0_pin(boot0_pin)) cg.add(var.set_leading_edge(config[CONF_LEADING_EDGE])) @@ -217,5 +217,5 @@ def to_code(config): continue conf = config[key] - sens = yield sensor.new_sensor(conf) + sens = await sensor.new_sensor(conf) cg.add(getattr(var, f"set_{key}_sensor")(sens)) diff --git a/esphome/components/shelly_dimmer/shelly_dimmer.cpp b/esphome/components/shelly_dimmer/shelly_dimmer.cpp index 6b4ab13c48..b336bbcb65 100644 --- a/esphome/components/shelly_dimmer/shelly_dimmer.cpp +++ b/esphome/components/shelly_dimmer/shelly_dimmer.cpp @@ -101,8 +101,6 @@ void ShellyDimmer::setup() { this->pin_nrst_->setup(); this->pin_boot0_->setup(); - ESP_LOGI(TAG, "Initializing"); - this->handle_firmware(); this->send_settings_(); diff --git a/esphome/components/sht3xd/sht3xd.cpp b/esphome/components/sht3xd/sht3xd.cpp index 9dc866ddc3..79f1674020 100644 --- a/esphome/components/sht3xd/sht3xd.cpp +++ b/esphome/components/sht3xd/sht3xd.cpp @@ -25,7 +25,6 @@ static const uint16_t SHT3XD_COMMAND_POLLING_H = 0x2400; static const uint16_t SHT3XD_COMMAND_FETCH_DATA = 0xE000; void SHT3XDComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint16_t raw_serial_number[2]; if (!this->get_register(SHT3XD_COMMAND_READ_SERIAL_NUMBER_CLOCK_STRETCHING, raw_serial_number, 2)) { this->error_code_ = READ_SERIAL_STRETCHED_FAILED; @@ -61,7 +60,6 @@ void SHT3XDComponent::dump_config() { ESP_LOGE(TAG, " Communication with SHT3xD failed!"); return; } - ESP_LOGD(TAG, " Setup successful"); ESP_LOGD(TAG, " Serial Number: 0x%08" PRIX32, this->serial_number_); ESP_LOGD(TAG, " Heater Enabled: %s", this->heater_enabled_ ? "true" : "false"); diff --git a/esphome/components/sht4x/sht4x.cpp b/esphome/components/sht4x/sht4x.cpp index 944b13023e..62b8717ded 100644 --- a/esphome/components/sht4x/sht4x.cpp +++ b/esphome/components/sht4x/sht4x.cpp @@ -18,8 +18,6 @@ void SHT4XComponent::start_heater_() { } void SHT4XComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - auto err = this->write(nullptr, 0); if (err != i2c::ERROR_OK) { this->mark_failed(); @@ -67,7 +65,7 @@ void SHT4XComponent::update() { // Send command if (!this->write_command(MEASURECOMMANDS[this->precision_])) { // Warning will be printed only if warning status is not set yet - this->status_set_warning("Failed to send measurement command"); + this->status_set_warning(LOG_STR("Failed to send measurement command")); return; } diff --git a/esphome/components/shtcx/shtcx.cpp b/esphome/components/shtcx/shtcx.cpp index 5420119bd6..d532bd7f44 100644 --- a/esphome/components/shtcx/shtcx.cpp +++ b/esphome/components/shtcx/shtcx.cpp @@ -25,7 +25,6 @@ inline const char *to_string(SHTCXType type) { } void SHTCXComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->wake_up(); this->soft_reset(); diff --git a/esphome/components/sim800l/__init__.py b/esphome/components/sim800l/__init__.py index 2ca9127d3f..c48a3c63c4 100644 --- a/esphome/components/sim800l/__init__.py +++ b/esphome/components/sim800l/__init__.py @@ -171,8 +171,7 @@ async def sim800l_dial_to_code(config, action_id, template_arg, args): ) async def sim800l_connect_to_code(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + return cg.new_Pvariable(action_id, template_arg, paren) SIM800L_SEND_USSD_SCHEMA = cv.Schema( @@ -201,5 +200,4 @@ async def sim800l_send_ussd_to_code(config, action_id, template_arg, args): ) async def sim800l_disconnect_to_code(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + return cg.new_Pvariable(action_id, template_arg, paren) diff --git a/esphome/components/sim800l/sim800l.cpp b/esphome/components/sim800l/sim800l.cpp index d97b0ae364..55cadcf182 100644 --- a/esphome/components/sim800l/sim800l.cpp +++ b/esphome/components/sim800l/sim800l.cpp @@ -288,11 +288,15 @@ void Sim800LComponent::parse_cmd_(std::string message) { if (item == 3) { // stat uint8_t current_call_state = parse_number(message.substr(start, end - start)).value_or(6); if (current_call_state != this->call_state_) { - ESP_LOGD(TAG, "Call state is now: %d", current_call_state); - if (current_call_state == 0) - this->call_connected_callback_.call(); + if (current_call_state == 4) { + ESP_LOGV(TAG, "Premature call state '4'. Ignoring, waiting for RING"); + } else { + this->call_state_ = current_call_state; + ESP_LOGD(TAG, "Call state is now: %d", current_call_state); + if (current_call_state == 0) + this->call_connected_callback_.call(); + } } - this->call_state_ = current_call_state; break; } // item 4 = "" diff --git a/esphome/components/sm16716/sm16716.cpp b/esphome/components/sm16716/sm16716.cpp index b25f935eba..aa33b7b679 100644 --- a/esphome/components/sm16716/sm16716.cpp +++ b/esphome/components/sm16716/sm16716.cpp @@ -7,7 +7,6 @@ namespace sm16716 { static const char *const TAG = "sm16716"; void SM16716::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->data_pin_->setup(); this->data_pin_->digital_write(false); this->clock_pin_->setup(); diff --git a/esphome/components/sm2135/sm2135.cpp b/esphome/components/sm2135/sm2135.cpp index cd647ef3b9..e55f836929 100644 --- a/esphome/components/sm2135/sm2135.cpp +++ b/esphome/components/sm2135/sm2135.cpp @@ -20,7 +20,6 @@ static const uint8_t SM2135_RGB = 0x00; // RGB channel static const uint8_t SM2135_CW = 0x80; // CW channel (Chip default) void SM2135::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->data_pin_->setup(); this->data_pin_->digital_write(false); this->data_pin_->pin_mode(gpio::FLAG_OUTPUT); diff --git a/esphome/components/sm2235/sm2235.cpp b/esphome/components/sm2235/sm2235.cpp index e9f84773e2..820fcb521a 100644 --- a/esphome/components/sm2235/sm2235.cpp +++ b/esphome/components/sm2235/sm2235.cpp @@ -7,7 +7,6 @@ namespace sm2235 { static const char *const TAG = "sm2235"; void SM2235::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->data_pin_->setup(); this->data_pin_->digital_write(true); this->clock_pin_->setup(); diff --git a/esphome/components/sm2335/sm2335.cpp b/esphome/components/sm2335/sm2335.cpp index 99b722a639..0580a782f5 100644 --- a/esphome/components/sm2335/sm2335.cpp +++ b/esphome/components/sm2335/sm2335.cpp @@ -7,7 +7,6 @@ namespace sm2335 { static const char *const TAG = "sm2335"; void SM2335::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->data_pin_->setup(); this->data_pin_->digital_write(true); this->clock_pin_->setup(); diff --git a/esphome/components/sn74hc165/sn74hc165.cpp b/esphome/components/sn74hc165/sn74hc165.cpp index 69e0df5785..416d9db293 100644 --- a/esphome/components/sn74hc165/sn74hc165.cpp +++ b/esphome/components/sn74hc165/sn74hc165.cpp @@ -7,7 +7,6 @@ namespace sn74hc165 { static const char *const TAG = "sn74hc165"; void SN74HC165Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // initialize pins this->clock_pin_->setup(); this->data_pin_->setup(); diff --git a/esphome/components/sn74hc595/sn74hc595.cpp b/esphome/components/sn74hc595/sn74hc595.cpp index d8e33eec22..fc47a6dc5e 100644 --- a/esphome/components/sn74hc595/sn74hc595.cpp +++ b/esphome/components/sn74hc595/sn74hc595.cpp @@ -8,7 +8,6 @@ namespace sn74hc595 { static const char *const TAG = "sn74hc595"; void SN74HC595Component::pre_setup_() { - ESP_LOGCONFIG(TAG, "Running setup"); if (this->have_oe_pin_) { // disable output this->oe_pin_->setup(); this->oe_pin_->digital_write(true); diff --git a/esphome/components/sntp/sntp_component.cpp b/esphome/components/sntp/sntp_component.cpp index d5839c1a2b..1cca5e8043 100644 --- a/esphome/components/sntp/sntp_component.cpp +++ b/esphome/components/sntp/sntp_component.cpp @@ -14,9 +14,13 @@ namespace sntp { static const char *const TAG = "sntp"; -void SNTPComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); #if defined(USE_ESP32) +SNTPComponent *SNTPComponent::instance = nullptr; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables) +#endif + +void SNTPComponent::setup() { +#if defined(USE_ESP32) + SNTPComponent::instance = this; if (esp_sntp_enabled()) { esp_sntp_stop(); } @@ -26,6 +30,11 @@ void SNTPComponent::setup() { esp_sntp_setservername(i++, server.c_str()); } esp_sntp_set_sync_interval(this->get_update_interval()); + esp_sntp_set_time_sync_notification_cb([](struct timeval *tv) { + if (SNTPComponent::instance != nullptr) { + SNTPComponent::instance->defer([]() { SNTPComponent::instance->time_synced(); }); + } + }); esp_sntp_init(); #else sntp_stop(); @@ -35,6 +44,14 @@ void SNTPComponent::setup() { for (auto &server : this->servers_) { sntp_setservername(i++, server.c_str()); } + +#if defined(USE_ESP8266) + settimeofday_cb([this](bool from_sntp) { + if (from_sntp) + this->time_synced(); + }); +#endif + sntp_init(); #endif } @@ -47,7 +64,8 @@ void SNTPComponent::dump_config() { } void SNTPComponent::update() { #if !defined(USE_ESP32) - // force resync + // Some platforms currently cannot set the sync interval at runtime so we need + // to do the re-sync by hand for now. if (sntp_enabled()) { sntp_stop(); this->has_time_ = false; @@ -56,23 +74,31 @@ void SNTPComponent::update() { #endif } void SNTPComponent::loop() { +// The loop is used to infer whether we have valid time on platforms where we +// cannot tell whether SNTP has succeeded. +// One limitation of this approach is that we cannot tell if it was the SNTP +// component that set the time. +// ESP-IDF and ESP8266 use callbacks from the SNTP task to trigger the +// `on_time_sync` trigger on successful sync events. +#if defined(USE_ESP32) || defined(USE_ESP8266) + this->disable_loop(); +#endif + if (this->has_time_) return; + this->time_synced(); +} + +void SNTPComponent::time_synced() { auto time = this->now(); - if (!time.is_valid()) + this->has_time_ = time.is_valid(); + if (!this->has_time_) return; ESP_LOGD(TAG, "Synchronized time: %04d-%02d-%02d %02d:%02d:%02d", time.year, time.month, time.day_of_month, time.hour, time.minute, time.second); this->time_sync_callback_.call(); - this->has_time_ = true; - -#ifdef USE_ESP_IDF - // On ESP-IDF, time sync is permanent and update() doesn't force resync - // Time is now synchronized, no need to check anymore - this->disable_loop(); -#endif } } // namespace sntp diff --git a/esphome/components/sntp/sntp_component.h b/esphome/components/sntp/sntp_component.h index a4e8267383..dd4c71e082 100644 --- a/esphome/components/sntp/sntp_component.h +++ b/esphome/components/sntp/sntp_component.h @@ -26,9 +26,16 @@ class SNTPComponent : public time::RealTimeClock { void update() override; void loop() override; + void time_synced(); + protected: std::vector servers_; bool has_time_{false}; + +#if defined(USE_ESP32) + private: + static SNTPComponent *instance; +#endif }; } // namespace sntp diff --git a/esphome/components/socket/bsd_sockets_impl.cpp b/esphome/components/socket/bsd_sockets_impl.cpp index e056696bcf..c7cca62027 100644 --- a/esphome/components/socket/bsd_sockets_impl.cpp +++ b/esphome/components/socket/bsd_sockets_impl.cpp @@ -145,7 +145,7 @@ class BSDSocketImpl : public Socket { } ssize_t sendto(const void *buf, size_t len, int flags, const struct sockaddr *to, socklen_t tolen) override { - return ::sendto(fd_, buf, len, flags, to, tolen); + return ::sendto(fd_, buf, len, flags, to, tolen); // NOLINT(readability-suspicious-call-argument) } int setblocking(bool blocking) override { diff --git a/esphome/components/socket/lwip_raw_tcp_impl.cpp b/esphome/components/socket/lwip_raw_tcp_impl.cpp index 2d64a275df..4dedeffb6a 100644 --- a/esphome/components/socket/lwip_raw_tcp_impl.cpp +++ b/esphome/components/socket/lwip_raw_tcp_impl.cpp @@ -9,7 +9,7 @@ #include "lwip/tcp.h" #include #include -#include +#include #include "esphome/core/helpers.h" #include "esphome/core/log.h" @@ -40,27 +40,14 @@ class LWIPRawImpl : public Socket { void init() { LWIP_LOG("init(%p)", pcb_); tcp_arg(pcb_, this); - tcp_accept(pcb_, LWIPRawImpl::s_accept_fn); tcp_recv(pcb_, LWIPRawImpl::s_recv_fn); tcp_err(pcb_, LWIPRawImpl::s_err_fn); } std::unique_ptr accept(struct sockaddr *addr, socklen_t *addrlen) override { - if (pcb_ == nullptr) { - errno = EBADF; - return nullptr; - } - if (accepted_sockets_.empty()) { - errno = EWOULDBLOCK; - return nullptr; - } - std::unique_ptr sock = std::move(accepted_sockets_.front()); - accepted_sockets_.pop(); - if (addr != nullptr) { - sock->getpeername(addr, addrlen); - } - LWIP_LOG("accept(%p)", sock.get()); - return std::unique_ptr(std::move(sock)); + // Non-listening sockets return error + errno = EINVAL; + return nullptr; } int bind(const struct sockaddr *name, socklen_t addrlen) override { if (pcb_ == nullptr) { @@ -286,25 +273,10 @@ class LWIPRawImpl : public Socket { return -1; } int listen(int backlog) override { - if (pcb_ == nullptr) { - errno = EBADF; - return -1; - } - LWIP_LOG("tcp_listen_with_backlog(%p backlog=%d)", pcb_, backlog); - struct tcp_pcb *listen_pcb = tcp_listen_with_backlog(pcb_, backlog); - if (listen_pcb == nullptr) { - tcp_abort(pcb_); - pcb_ = nullptr; - errno = EOPNOTSUPP; - return -1; - } - // tcp_listen reallocates the pcb, replace ours - pcb_ = listen_pcb; - // set callbacks on new pcb - LWIP_LOG("tcp_arg(%p)", pcb_); - tcp_arg(pcb_, this); - tcp_accept(pcb_, LWIPRawImpl::s_accept_fn); - return 0; + // Regular sockets can't be converted to listening - this shouldn't happen + // as listen() should only be called on sockets created for listening + errno = EOPNOTSUPP; + return -1; } ssize_t read(void *buf, size_t len) override { if (pcb_ == nullptr) { @@ -485,20 +457,6 @@ class LWIPRawImpl : public Socket { return 0; } - err_t accept_fn(struct tcp_pcb *newpcb, err_t err) { - LWIP_LOG("accept(newpcb=%p err=%d)", newpcb, err); - if (err != ERR_OK || newpcb == nullptr) { - // "An error code if there has been an error accepting. Only return ERR_ABRT if you have - // called tcp_abort from within the callback function!" - // https://www.nongnu.org/lwip/2_1_x/tcp_8h.html#a00517abce6856d6c82f0efebdafb734d - // nothing to do here, we just don't push it to the queue - return ERR_OK; - } - auto sock = make_unique(family_, newpcb); - sock->init(); - accepted_sockets_.push(std::move(sock)); - return ERR_OK; - } void err_fn(err_t err) { LWIP_LOG("err(err=%d)", err); // "If a connection is aborted because of an error, the application is alerted of this event by @@ -530,11 +488,6 @@ class LWIPRawImpl : public Socket { return ERR_OK; } - static err_t s_accept_fn(void *arg, struct tcp_pcb *newpcb, err_t err) { - LWIPRawImpl *arg_this = reinterpret_cast(arg); - return arg_this->accept_fn(newpcb, err); - } - static void s_err_fn(void *arg, err_t err) { LWIPRawImpl *arg_this = reinterpret_cast(arg); arg_this->err_fn(err); @@ -586,22 +539,133 @@ class LWIPRawImpl : public Socket { return -1; } + // Member ordering optimized to minimize padding on 32-bit systems + // Largest members first (4 bytes), then smaller members (1 byte each) struct tcp_pcb *pcb_; - std::queue> accepted_sockets_; - bool rx_closed_ = false; pbuf *rx_buf_ = nullptr; size_t rx_buf_offset_ = 0; + bool rx_closed_ = false; // don't use lwip nodelay flag, it sometimes causes reconnect // instead use it for determining whether to call lwip_output bool nodelay_ = false; sa_family_t family_ = 0; }; +// Listening socket class - only allocates accept queue when needed (for bind+listen sockets) +// This saves 16 bytes (12 bytes array + 1 byte count + 3 bytes padding) for regular connected sockets on ESP8266/RP2040 +class LWIPRawListenImpl : public LWIPRawImpl { + public: + LWIPRawListenImpl(sa_family_t family, struct tcp_pcb *pcb) : LWIPRawImpl(family, pcb) {} + + void init() { + LWIP_LOG("init(%p)", pcb_); + tcp_arg(pcb_, this); + tcp_accept(pcb_, LWIPRawListenImpl::s_accept_fn); + tcp_err(pcb_, LWIPRawImpl::s_err_fn); // Use base class error handler + } + + std::unique_ptr accept(struct sockaddr *addr, socklen_t *addrlen) override { + if (pcb_ == nullptr) { + errno = EBADF; + return nullptr; + } + if (accepted_socket_count_ == 0) { + errno = EWOULDBLOCK; + return nullptr; + } + // Take from front for FIFO ordering + std::unique_ptr sock = std::move(accepted_sockets_[0]); + // Shift remaining sockets forward + for (uint8_t i = 1; i < accepted_socket_count_; i++) { + accepted_sockets_[i - 1] = std::move(accepted_sockets_[i]); + } + accepted_socket_count_--; + LWIP_LOG("Connection accepted by application, queue size: %d", accepted_socket_count_); + if (addr != nullptr) { + sock->getpeername(addr, addrlen); + } + LWIP_LOG("accept(%p)", sock.get()); + return std::unique_ptr(std::move(sock)); + } + + int listen(int backlog) override { + if (pcb_ == nullptr) { + errno = EBADF; + return -1; + } + LWIP_LOG("tcp_listen_with_backlog(%p backlog=%d)", pcb_, backlog); + struct tcp_pcb *listen_pcb = tcp_listen_with_backlog(pcb_, backlog); + if (listen_pcb == nullptr) { + tcp_abort(pcb_); + pcb_ = nullptr; + errno = EOPNOTSUPP; + return -1; + } + // tcp_listen reallocates the pcb, replace ours + pcb_ = listen_pcb; + // set callbacks on new pcb + LWIP_LOG("tcp_arg(%p)", pcb_); + tcp_arg(pcb_, this); + tcp_accept(pcb_, LWIPRawListenImpl::s_accept_fn); + return 0; + } + + private: + err_t accept_fn(struct tcp_pcb *newpcb, err_t err) { + LWIP_LOG("accept(newpcb=%p err=%d)", newpcb, err); + if (err != ERR_OK || newpcb == nullptr) { + // "An error code if there has been an error accepting. Only return ERR_ABRT if you have + // called tcp_abort from within the callback function!" + // https://www.nongnu.org/lwip/2_1_x/tcp_8h.html#a00517abce6856d6c82f0efebdafb734d + // nothing to do here, we just don't push it to the queue + return ERR_OK; + } + // Check if we've reached the maximum accept queue size + if (accepted_socket_count_ >= MAX_ACCEPTED_SOCKETS) { + LWIP_LOG("Rejecting connection, queue full (%d)", accepted_socket_count_); + // Abort the connection when queue is full + tcp_abort(newpcb); + // Must return ERR_ABRT since we called tcp_abort() + return ERR_ABRT; + } + auto sock = make_unique(family_, newpcb); + sock->init(); + accepted_sockets_[accepted_socket_count_++] = std::move(sock); + LWIP_LOG("Accepted connection, queue size: %d", accepted_socket_count_); + return ERR_OK; + } + + static err_t s_accept_fn(void *arg, struct tcp_pcb *newpcb, err_t err) { + LWIPRawListenImpl *arg_this = reinterpret_cast(arg); + return arg_this->accept_fn(newpcb, err); + } + + // Accept queue - holds incoming connections briefly until the event loop calls accept() + // This is NOT a connection pool - just a temporary queue between LWIP callbacks and the main loop + // 3 slots is plenty since connections are pulled out quickly by the event loop + // + // Memory analysis: std::array<3> vs original std::queue implementation: + // - std::queue uses std::deque internally which on 32-bit systems needs: + // 24 bytes (deque object) + 32+ bytes (map array) + heap allocations + // Total: ~56+ bytes minimum, plus heap fragmentation + // - std::array<3>: 12 bytes fixed (3 pointers × 4 bytes) + // Saves ~44+ bytes RAM per listening socket + avoids ALL heap allocations + // Used on ESP8266 and RP2040 (platforms using LWIP_TCP implementation) + // + // By using a separate listening socket class, regular connected sockets save + // 16 bytes (12 bytes array + 1 byte count + 3 bytes padding) of memory overhead on 32-bit systems + static constexpr size_t MAX_ACCEPTED_SOCKETS = 3; + std::array, MAX_ACCEPTED_SOCKETS> accepted_sockets_; + uint8_t accepted_socket_count_ = 0; // Number of sockets currently in queue +}; + std::unique_ptr socket(int domain, int type, int protocol) { auto *pcb = tcp_new(); if (pcb == nullptr) return nullptr; - auto *sock = new LWIPRawImpl((sa_family_t) domain, pcb); // NOLINT(cppcoreguidelines-owning-memory) + // Create listening socket implementation since user sockets typically bind+listen + // Accepted connections are created directly as LWIPRawImpl in the accept callback + auto *sock = new LWIPRawListenImpl((sa_family_t) domain, pcb); // NOLINT(cppcoreguidelines-owning-memory) sock->init(); return std::unique_ptr{sock}; } diff --git a/esphome/components/sonoff_d1/sonoff_d1.cpp b/esphome/components/sonoff_d1/sonoff_d1.cpp index e3d55681c5..cd09f31dd7 100644 --- a/esphome/components/sonoff_d1/sonoff_d1.cpp +++ b/esphome/components/sonoff_d1/sonoff_d1.cpp @@ -50,7 +50,7 @@ static const char *const TAG = "sonoff_d1"; uint8_t SonoffD1Output::calc_checksum_(const uint8_t *cmd, const size_t len) { uint8_t crc = 0; - for (int i = 2; i < len - 1; i++) { + for (size_t i = 2; i < len - 1; i++) { crc += cmd[i]; } return crc; diff --git a/esphome/components/sound_level/sensor.py b/esphome/components/sound_level/sensor.py index 292efadab8..8ca0feccc0 100644 --- a/esphome/components/sound_level/sensor.py +++ b/esphome/components/sound_level/sensor.py @@ -12,7 +12,7 @@ from esphome.const import ( UNIT_DECIBEL, ) -AUTOLOAD = ["audio"] +AUTO_LOAD = ["audio"] CODEOWNERS = ["@kahrendt"] DEPENDENCIES = ["microphone"] diff --git a/esphome/components/sound_level/sound_level.cpp b/esphome/components/sound_level/sound_level.cpp index decf630aba..db6b168bbc 100644 --- a/esphome/components/sound_level/sound_level.cpp +++ b/esphome/components/sound_level/sound_level.cpp @@ -56,7 +56,7 @@ void SoundLevelComponent::loop() { } } else { if (!this->status_has_warning()) { - this->status_set_warning("Microphone isn't running, can't compute statistics"); + this->status_set_warning(LOG_STR("Microphone isn't running, can't compute statistics")); // Deallocate buffers, if necessary this->stop_(); diff --git a/esphome/components/speaker/__init__.py b/esphome/components/speaker/__init__.py index 2ac1ca0cb9..5f1ba94ee6 100644 --- a/esphome/components/speaker/__init__.py +++ b/esphome/components/speaker/__init__.py @@ -4,7 +4,7 @@ from esphome.components import audio, audio_dac import esphome.config_validation as cv from esphome.const import CONF_DATA, CONF_ID, CONF_VOLUME from esphome.core import CORE -from esphome.coroutine import coroutine_with_priority +from esphome.coroutine import CoroPriority, coroutine_with_priority AUTO_LOAD = ["audio"] CODEOWNERS = ["@jesserockz", "@kahrendt"] @@ -138,7 +138,7 @@ async def speaker_mute_action_to_code(config, action_id, template_arg, args): return cg.new_Pvariable(action_id, template_arg, paren) -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): cg.add_global(speaker_ns.using) cg.add_define("USE_SPEAKER") diff --git a/esphome/components/speaker/media_player/__init__.py b/esphome/components/speaker/media_player/__init__.py index dc2dae2ef1..7537a61e4e 100644 --- a/esphome/components/speaker/media_player/__init__.py +++ b/esphome/components/speaker/media_player/__init__.py @@ -6,7 +6,7 @@ from pathlib import Path from esphome import automation, external_files import esphome.codegen as cg -from esphome.components import audio, esp32, media_player, speaker +from esphome.components import audio, esp32, media_player, psram, speaker import esphome.config_validation as cv from esphome.const import ( CONF_BUFFER_SIZE, @@ -26,10 +26,21 @@ from esphome.const import ( from esphome.core import CORE, HexInt from esphome.core.entity_helpers import inherit_property_from from esphome.external_files import download_content +from esphome.types import ConfigType _LOGGER = logging.getLogger(__name__) -AUTO_LOAD = ["audio", "psram"] + +def AUTO_LOAD(config: ConfigType) -> list[str]: + load = ["audio"] + if ( + not config + or config.get(CONF_TASK_STACK_IN_PSRAM) + or config.get(CONF_CODEC_SUPPORT_ENABLED) + ): + return load + ["psram"] + return load + CODEOWNERS = ["@kahrendt", "@synesthesiam"] DOMAIN = "media_player" @@ -147,7 +158,7 @@ def _read_audio_file_and_type(file_config): elif file_source == TYPE_WEB: path = _compute_local_file_path(conf_file) else: - raise cv.Invalid("Unsupported file source.") + raise cv.Invalid("Unsupported file source") with open(path, "rb") as f: data = f.read() @@ -155,8 +166,7 @@ def _read_audio_file_and_type(file_config): import puremagic file_type: str = puremagic.from_string(data) - if file_type.startswith("."): - file_type = file_type[1:] + file_type = file_type.removeprefix(".") media_file_type = audio.AUDIO_FILE_TYPE_ENUM["NONE"] if file_type in ("wav"): @@ -204,13 +214,14 @@ def _validate_pipeline(config): def _validate_repeated_speaker(config): - if (announcement_config := config.get(CONF_ANNOUNCEMENT_PIPELINE)) and ( - media_config := config.get(CONF_MEDIA_PIPELINE) + if ( + (announcement_config := config.get(CONF_ANNOUNCEMENT_PIPELINE)) + and (media_config := config.get(CONF_MEDIA_PIPELINE)) + and announcement_config[CONF_SPEAKER] == media_config[CONF_SPEAKER] ): - if announcement_config[CONF_SPEAKER] == media_config[CONF_SPEAKER]: - raise cv.Invalid( - "The announcement and media pipelines cannot use the same speaker. Use the `mixer` speaker component to create two source speakers." - ) + raise cv.Invalid( + "The announcement and media pipelines cannot use the same speaker. Use the `mixer` speaker component to create two source speakers." + ) return config @@ -219,7 +230,7 @@ def _validate_supported_local_file(config): for file_config in config.get(CONF_FILES, []): _, media_file_type = _read_audio_file_and_type(file_config) if str(media_file_type) == str(audio.AUDIO_FILE_TYPE_ENUM["NONE"]): - raise cv.Invalid("Unsupported local media file.") + raise cv.Invalid("Unsupported local media file") if not config[CONF_CODEC_SUPPORT_ENABLED] and str(media_file_type) != str( audio.AUDIO_FILE_TYPE_ENUM["WAV"] ): @@ -279,7 +290,9 @@ CONFIG_SCHEMA = cv.All( cv.Optional(CONF_BUFFER_SIZE, default=1000000): cv.int_range( min=4000, max=4000000 ), - cv.Optional(CONF_CODEC_SUPPORT_ENABLED, default=True): cv.boolean, + cv.Optional( + CONF_CODEC_SUPPORT_ENABLED, default=psram.supported() + ): cv.boolean, cv.Optional(CONF_FILES): cv.ensure_list(MEDIA_FILE_TYPE_SCHEMA), cv.Optional(CONF_TASK_STACK_IN_PSRAM, default=False): cv.boolean, cv.Optional(CONF_VOLUME_INCREMENT, default=0.05): cv.percentage, @@ -315,31 +328,19 @@ async def to_code(config): cg.add_define("USE_AUDIO_FLAC_SUPPORT", True) cg.add_define("USE_AUDIO_MP3_SUPPORT", True) - # Wifi settings based on https://github.com/espressif/esp-adf/issues/297#issuecomment-783811702 - esp32.add_idf_sdkconfig_option("CONFIG_ESP32_WIFI_STATIC_RX_BUFFER_NUM", 16) - esp32.add_idf_sdkconfig_option("CONFIG_ESP32_WIFI_DYNAMIC_RX_BUFFER_NUM", 512) - esp32.add_idf_sdkconfig_option("CONFIG_ESP32_WIFI_STATIC_TX_BUFFER", True) - esp32.add_idf_sdkconfig_option("CONFIG_ESP32_WIFI_TX_BUFFER_TYPE", 0) - esp32.add_idf_sdkconfig_option("CONFIG_ESP32_WIFI_STATIC_TX_BUFFER_NUM", 8) - esp32.add_idf_sdkconfig_option("CONFIG_ESP32_WIFI_CACHE_TX_BUFFER_NUM", 32) - esp32.add_idf_sdkconfig_option("CONFIG_ESP32_WIFI_AMPDU_TX_ENABLED", True) - esp32.add_idf_sdkconfig_option("CONFIG_ESP32_WIFI_TX_BA_WIN", 16) - esp32.add_idf_sdkconfig_option("CONFIG_ESP32_WIFI_AMPDU_RX_ENABLED", True) - esp32.add_idf_sdkconfig_option("CONFIG_ESP32_WIFI_RX_BA_WIN", 32) - esp32.add_idf_sdkconfig_option("CONFIG_LWIP_MAX_ACTIVE_TCP", 16) - esp32.add_idf_sdkconfig_option("CONFIG_LWIP_MAX_LISTENING_TCP", 16) - esp32.add_idf_sdkconfig_option("CONFIG_TCP_MAXRTX", 12) - esp32.add_idf_sdkconfig_option("CONFIG_TCP_SYNMAXRTX", 6) - esp32.add_idf_sdkconfig_option("CONFIG_TCP_MSS", 1436) - esp32.add_idf_sdkconfig_option("CONFIG_TCP_MSL", 60000) - esp32.add_idf_sdkconfig_option("CONFIG_TCP_SND_BUF_DEFAULT", 65535) - esp32.add_idf_sdkconfig_option("CONFIG_TCP_WND_DEFAULT", 512000) - esp32.add_idf_sdkconfig_option("CONFIG_TCP_RECVMBOX_SIZE", 512) - esp32.add_idf_sdkconfig_option("CONFIG_TCP_QUEUE_OOSEQ", True) - esp32.add_idf_sdkconfig_option("CONFIG_TCP_OVERSIZE_MSS", True) - esp32.add_idf_sdkconfig_option("CONFIG_LWIP_WND_SCALE", True) - esp32.add_idf_sdkconfig_option("CONFIG_LWIP_TCP_RCV_SCALE", 3) - esp32.add_idf_sdkconfig_option("CONFIG_LWIP_TCPIP_RECVMBOX_SIZE", 512) + # Based on https://github.com/espressif/esp-idf/blob/release/v5.4/examples/wifi/iperf/sdkconfig.defaults.esp32 + esp32.add_idf_sdkconfig_option("CONFIG_ESP_WIFI_STATIC_RX_BUFFER_NUM", 16) + esp32.add_idf_sdkconfig_option("CONFIG_ESP_WIFI_DYNAMIC_RX_BUFFER_NUM", 64) + esp32.add_idf_sdkconfig_option("CONFIG_ESP_WIFI_DYNAMIC_TX_BUFFER_NUM", 64) + esp32.add_idf_sdkconfig_option("CONFIG_ESP_WIFI_AMPDU_TX_ENABLED", True) + esp32.add_idf_sdkconfig_option("CONFIG_ESP_WIFI_TX_BA_WIN", 32) + esp32.add_idf_sdkconfig_option("CONFIG_ESP_WIFI_AMPDU_RX_ENABLED", True) + esp32.add_idf_sdkconfig_option("CONFIG_ESP_WIFI_RX_BA_WIN", 32) + + esp32.add_idf_sdkconfig_option("CONFIG_LWIP_TCP_SND_BUF_DEFAULT", 65534) + esp32.add_idf_sdkconfig_option("CONFIG_LWIP_TCP_WND_DEFAULT", 65534) + esp32.add_idf_sdkconfig_option("CONFIG_LWIP_TCP_RECVMBOX_SIZE", 64) + esp32.add_idf_sdkconfig_option("CONFIG_LWIP_TCPIP_RECVMBOX_SIZE", 64) # Allocate wifi buffers in PSRAM esp32.add_idf_sdkconfig_option("CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP", True) diff --git a/esphome/components/speaker/media_player/audio_pipeline.cpp b/esphome/components/speaker/media_player/audio_pipeline.cpp index 333a076bec..dc8572ae43 100644 --- a/esphome/components/speaker/media_player/audio_pipeline.cpp +++ b/esphome/components/speaker/media_player/audio_pipeline.cpp @@ -200,7 +200,7 @@ AudioPipelineState AudioPipeline::process_state() { if ((this->read_task_handle_ != nullptr) || (this->decode_task_handle_ != nullptr)) { this->delete_tasks_(); if (this->hard_stop_) { - // Stop command was sent, so immediately end of the playback + // Stop command was sent, so immediately end the playback this->speaker_->stop(); this->hard_stop_ = false; } else { @@ -210,13 +210,25 @@ AudioPipelineState AudioPipeline::process_state() { } } this->is_playing_ = false; - return AudioPipelineState::STOPPED; + if (!this->speaker_->is_running()) { + return AudioPipelineState::STOPPED; + } else { + this->is_finishing_ = true; + } } if (this->pause_state_) { return AudioPipelineState::PAUSED; } + if (this->is_finishing_) { + if (!this->speaker_->is_running()) { + this->is_finishing_ = false; + } else { + return AudioPipelineState::PLAYING; + } + } + if ((this->read_task_handle_ == nullptr) && (this->decode_task_handle_ == nullptr)) { // No tasks are running, so the pipeline is stopped. xEventGroupClearBits(this->event_group_, EventGroupBits::PIPELINE_COMMAND_STOP); @@ -247,13 +259,10 @@ esp_err_t AudioPipeline::allocate_communications_() { esp_err_t AudioPipeline::start_tasks_() { if (this->read_task_handle_ == nullptr) { if (this->read_task_stack_buffer_ == nullptr) { - if (this->task_stack_in_psram_) { - RAMAllocator stack_allocator(RAMAllocator::ALLOC_EXTERNAL); - this->read_task_stack_buffer_ = stack_allocator.allocate(READ_TASK_STACK_SIZE); - } else { - RAMAllocator stack_allocator(RAMAllocator::ALLOC_INTERNAL); - this->read_task_stack_buffer_ = stack_allocator.allocate(READ_TASK_STACK_SIZE); - } + // Reader task uses the AudioReader class which uses esp_http_client. This crashes on IDF 5.4 if the task stack is + // in PSRAM. As a workaround, always allocate the read task in internal memory. + RAMAllocator stack_allocator(RAMAllocator::ALLOC_INTERNAL); + this->read_task_stack_buffer_ = stack_allocator.allocate(READ_TASK_STACK_SIZE); } if (this->read_task_stack_buffer_ == nullptr) { diff --git a/esphome/components/speaker/media_player/audio_pipeline.h b/esphome/components/speaker/media_player/audio_pipeline.h index 722d9cbb2a..98f43fda6e 100644 --- a/esphome/components/speaker/media_player/audio_pipeline.h +++ b/esphome/components/speaker/media_player/audio_pipeline.h @@ -114,6 +114,7 @@ class AudioPipeline { bool hard_stop_{false}; bool is_playing_{false}; + bool is_finishing_{false}; bool pause_state_{false}; bool task_stack_in_psram_; diff --git a/esphome/components/speaker/media_player/speaker_media_player.cpp b/esphome/components/speaker/media_player/speaker_media_player.cpp index 2c30f17c78..b45a78010a 100644 --- a/esphome/components/speaker/media_player/speaker_media_player.cpp +++ b/esphome/components/speaker/media_player/speaker_media_player.cpp @@ -55,7 +55,7 @@ void SpeakerMediaPlayer::setup() { this->media_control_command_queue_ = xQueueCreate(MEDIA_CONTROLS_QUEUE_LENGTH, sizeof(MediaCallCommand)); - this->pref_ = global_preferences->make_preference(this->get_object_id_hash()); + this->pref_ = global_preferences->make_preference(this->get_preference_hash()); VolumeRestoreState volume_restore_state; if (this->pref_.load(&volume_restore_state)) { diff --git a/esphome/components/spi/__init__.py b/esphome/components/spi/__init__.py index 58bfc3f411..d803ee66dc 100644 --- a/esphome/components/spi/__init__.py +++ b/esphome/components/spi/__init__.py @@ -1,4 +1,5 @@ import re +from typing import Any from esphome import pins import esphome.codegen as cg @@ -34,7 +35,7 @@ from esphome.const import ( PLATFORM_RP2040, PlatformFramework, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority import esphome.final_validate as fv CODEOWNERS = ["@esphome/core", "@clydebarrow"] @@ -114,9 +115,7 @@ def get_target_platform(): def get_target_variant(): - return ( - CORE.data[KEY_ESP32][KEY_VARIANT] if KEY_VARIANT in CORE.data[KEY_ESP32] else "" - ) + return CORE.data[KEY_ESP32].get(KEY_VARIANT, "") # Get a list of available hardware interfaces based on target and variant. @@ -139,6 +138,27 @@ def get_hw_interface_list(): return [] +def one_of_interface_validator(additional_values: list[str] | None = None) -> Any: + """Helper to create a one_of validator for SPI interfaces. + + This delays evaluation of get_hw_interface_list() until validation time, + avoiding access to CORE.data during module import. + + Args: + additional_values: List of additional valid values to include + """ + if additional_values is None: + additional_values = [] + + def validator(value: str) -> str: + return cv.one_of( + *sum(get_hw_interface_list(), additional_values), + lower=True, + )(value) + + return cv.All(cv.string, validator) + + # Given an SPI name, return the index of it in the available list def get_spi_index(name): for i, ilist in enumerate(get_hw_interface_list()): @@ -191,9 +211,7 @@ def validate_hw_pins(spi, index=-1): return False if sdo_pin_no not in pin_set[CONF_MOSI_PIN]: return False - if sdi_pin_no not in pin_set[CONF_MISO_PIN]: - return False - return True + return sdi_pin_no in pin_set[CONF_MISO_PIN] return False @@ -258,9 +276,6 @@ def get_spi_interface(index): return ["&SPI", "&SPI1"][index] if index == 0: return "&SPI" - # Following code can't apply to C2, H2 or 8266 since they have only one SPI - if get_target_variant() in (VARIANT_ESP32S3, VARIANT_ESP32S2): - return "new SPIClass(FSPI)" return "new SPIClass(HSPI)" @@ -274,9 +289,8 @@ SPI_SINGLE_SCHEMA = cv.All( cv.Optional(CONF_FORCE_SW): cv.invalid( "force_sw is deprecated - use interface: software" ), - cv.Optional(CONF_INTERFACE, default="any"): cv.one_of( - *sum(get_hw_interface_list(), ["software", "hardware", "any"]), - lower=True, + cv.Optional(CONF_INTERFACE, default="any"): one_of_interface_validator( + ["software", "hardware", "any"] ), cv.Optional(CONF_DATA_PINS): cv.invalid( "'data_pins' should be used with 'type: quad or octal' only" @@ -309,10 +323,9 @@ def spi_mode_schema(mode): cv.ensure_list(pins.internal_gpio_output_pin_number), cv.Length(min=pin_count, max=pin_count), ), - cv.Optional(CONF_INTERFACE, default="hardware"): cv.one_of( - *sum(get_hw_interface_list(), ["hardware"]), - lower=True, - ), + cv.Optional( + CONF_INTERFACE, default="hardware" + ): one_of_interface_validator(["hardware"]), cv.Optional(CONF_MISO_PIN): cv.invalid( f"'miso_pin' should not be used with {mode} SPI" ), @@ -335,7 +348,7 @@ CONFIG_SCHEMA = cv.All( ) -@coroutine_with_priority(1.0) +@coroutine_with_priority(CoroPriority.BUS) async def to_code(configs): cg.add_define("USE_SPI") cg.add_global(spi_ns.using) diff --git a/esphome/components/spi/spi.cpp b/esphome/components/spi/spi.cpp index 805a774ceb..00e9845a03 100644 --- a/esphome/components/spi/spi.cpp +++ b/esphome/components/spi/spi.cpp @@ -37,8 +37,6 @@ void SPIComponent::unregister_device(SPIClient *device) { } void SPIComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - if (this->sdo_pin_ == nullptr) this->sdo_pin_ = NullPin::NULL_PIN; if (this->sdi_pin_ == nullptr) diff --git a/esphome/components/spi_device/spi_device.cpp b/esphome/components/spi_device/spi_device.cpp index 872b3054e6..dbfbc9eccb 100644 --- a/esphome/components/spi_device/spi_device.cpp +++ b/esphome/components/spi_device/spi_device.cpp @@ -8,10 +8,7 @@ namespace spi_device { static const char *const TAG = "spi_device"; -void SPIDeviceComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->spi_setup(); -} +void SPIDeviceComponent::setup() { this->spi_setup(); } void SPIDeviceComponent::dump_config() { ESP_LOGCONFIG(TAG, "SPIDevice"); diff --git a/esphome/components/split_buffer/__init__.py b/esphome/components/split_buffer/__init__.py new file mode 100644 index 0000000000..be7472936f --- /dev/null +++ b/esphome/components/split_buffer/__init__.py @@ -0,0 +1,5 @@ +CODEOWNERS = ["@jesserockz"] + +# Allows split_buffer to be configured in yaml, to allow use of the C++ api. + +CONFIG_SCHEMA = {} diff --git a/esphome/components/split_buffer/split_buffer.cpp b/esphome/components/split_buffer/split_buffer.cpp new file mode 100644 index 0000000000..a710670a5d --- /dev/null +++ b/esphome/components/split_buffer/split_buffer.cpp @@ -0,0 +1,133 @@ +#include "split_buffer.h" + +#include "esphome/core/helpers.h" +#include "esphome/core/log.h" + +namespace esphome::split_buffer { + +static constexpr const char *const TAG = "split_buffer"; + +SplitBuffer::~SplitBuffer() { this->free(); } + +bool SplitBuffer::init(size_t total_length) { + this->free(); // Clean up any existing allocation + + if (total_length == 0) { + return false; + } + + this->total_length_ = total_length; + size_t current_buffer_size = total_length; + + RAMAllocator ptr_allocator; + RAMAllocator allocator; + + // Try to allocate the entire buffer first + while (current_buffer_size > 0) { + // Calculate how many buffers we need of this size + size_t needed_buffers = (total_length + current_buffer_size - 1) / current_buffer_size; + + // Try to allocate array of buffer pointers + uint8_t **temp_buffers = ptr_allocator.allocate(needed_buffers); + if (temp_buffers == nullptr) { + // If we can't even allocate the pointer array, don't need to continue + ESP_LOGE(TAG, "Failed to allocate pointers"); + return false; + } + + // Initialize all pointers to null + for (size_t i = 0; i < needed_buffers; i++) { + temp_buffers[i] = nullptr; + } + + // Try to allocate all the buffers + bool allocation_success = true; + for (size_t i = 0; i < needed_buffers; i++) { + size_t this_buffer_size = current_buffer_size; + // Last buffer might be smaller if total_length is not divisible by current_buffer_size + if (i == needed_buffers - 1 && total_length % current_buffer_size != 0) { + this_buffer_size = total_length % current_buffer_size; + } + + temp_buffers[i] = allocator.allocate(this_buffer_size); + if (temp_buffers[i] == nullptr) { + allocation_success = false; + break; + } + + // Initialize buffer to zero + memset(temp_buffers[i], 0, this_buffer_size); + } + + if (allocation_success) { + // Success! Store the result + this->buffers_ = temp_buffers; + this->buffer_count_ = needed_buffers; + this->buffer_size_ = current_buffer_size; + ESP_LOGD(TAG, "Allocated %zu * %zu bytes - %zu bytes", this->buffer_count_, this->buffer_size_, + this->total_length_); + return true; + } + + // Allocation failed, clean up and try smaller buffers + for (size_t i = 0; i < needed_buffers; i++) { + if (temp_buffers[i] != nullptr) { + allocator.deallocate(temp_buffers[i], 0); + } + } + ptr_allocator.deallocate(temp_buffers, 0); + + // Halve the buffer size and try again + current_buffer_size = current_buffer_size / 2; + } + + ESP_LOGE(TAG, "Failed to allocate %zu bytes", total_length); + return false; +} + +void SplitBuffer::free() { + if (this->buffers_ != nullptr) { + RAMAllocator allocator; + for (size_t i = 0; i < this->buffer_count_; i++) { + if (this->buffers_[i] != nullptr) { + allocator.deallocate(this->buffers_[i], 0); + } + } + RAMAllocator ptr_allocator; + ptr_allocator.deallocate(this->buffers_, 0); + this->buffers_ = nullptr; + } + this->buffer_count_ = 0; + this->buffer_size_ = 0; + this->total_length_ = 0; +} + +uint8_t &SplitBuffer::operator[](size_t index) { + if (index >= this->total_length_) { + ESP_LOGE(TAG, "Out of bounds - %zu >= %zu", index, this->total_length_); + // Return reference to a static dummy byte to avoid crash + static uint8_t dummy = 0; + return dummy; + } + + size_t buffer_index = index / this->buffer_size_; + size_t offset_in_buffer = index - this->buffer_size_ * buffer_index; + + return this->buffers_[buffer_index][offset_in_buffer]; +} + +const uint8_t &SplitBuffer::operator[](size_t index) const { + if (index >= this->total_length_) { + ESP_LOGE(TAG, "Out of bounds - %zu >= %zu", index, this->total_length_); + // Return reference to a static dummy byte to avoid crash + static const uint8_t DUMMY = 0; + return DUMMY; + } + + size_t buffer_index = index / this->buffer_size_; + size_t offset_in_buffer = index - this->buffer_size_ * buffer_index; + + return this->buffers_[buffer_index][offset_in_buffer]; +} + +} // namespace esphome::split_buffer diff --git a/esphome/components/split_buffer/split_buffer.h b/esphome/components/split_buffer/split_buffer.h new file mode 100644 index 0000000000..c3490f3d6e --- /dev/null +++ b/esphome/components/split_buffer/split_buffer.h @@ -0,0 +1,40 @@ +#pragma once + +#include +#include + +namespace esphome::split_buffer { + +class SplitBuffer { + public: + SplitBuffer() = default; + ~SplitBuffer(); + + // Initialize the buffer with the desired total length + bool init(size_t total_length); + + // Free all allocated buffers + void free(); + + // Access operators + uint8_t &operator[](size_t index); + const uint8_t &operator[](size_t index) const; + + // Get the total length + size_t size() const { return this->total_length_; } + + // Get buffer information + size_t get_buffer_count() const { return this->buffer_count_; } + size_t get_buffer_size() const { return this->buffer_size_; } + + // Check if successfully initialized + bool is_valid() const { return this->buffers_ != nullptr && this->buffer_count_ > 0; } + + private: + uint8_t **buffers_{nullptr}; + size_t buffer_count_{0}; + size_t buffer_size_{0}; + size_t total_length_{0}; +}; + +} // namespace esphome::split_buffer diff --git a/esphome/components/sprinkler/__init__.py b/esphome/components/sprinkler/__init__.py index 3c94d97739..2dccb6896a 100644 --- a/esphome/components/sprinkler/__init__.py +++ b/esphome/components/sprinkler/__init__.py @@ -130,11 +130,11 @@ def validate_sprinkler(config): if ( CONF_PUMP_SWITCH_OFF_DURING_VALVE_OPEN_DELAY in sprinkler_controller and CONF_VALVE_OPEN_DELAY not in sprinkler_controller + and sprinkler_controller[CONF_PUMP_SWITCH_OFF_DURING_VALVE_OPEN_DELAY] ): - if sprinkler_controller[CONF_PUMP_SWITCH_OFF_DURING_VALVE_OPEN_DELAY]: - raise cv.Invalid( - f"{CONF_VALVE_OPEN_DELAY} must be defined when {CONF_PUMP_SWITCH_OFF_DURING_VALVE_OPEN_DELAY} is enabled" - ) + raise cv.Invalid( + f"{CONF_VALVE_OPEN_DELAY} must be defined when {CONF_PUMP_SWITCH_OFF_DURING_VALVE_OPEN_DELAY} is enabled" + ) if ( CONF_REPEAT in sprinkler_controller diff --git a/esphome/components/sprinkler/sprinkler.cpp b/esphome/components/sprinkler/sprinkler.cpp index e191498857..7676e17468 100644 --- a/esphome/components/sprinkler/sprinkler.cpp +++ b/esphome/components/sprinkler/sprinkler.cpp @@ -81,7 +81,7 @@ void SprinklerControllerNumber::setup() { if (!this->restore_value_) { value = this->initial_value_; } else { - this->pref_ = global_preferences->make_preference(this->get_object_id_hash()); + this->pref_ = global_preferences->make_preference(this->get_preference_hash()); if (!this->pref_.load(&value)) { if (!std::isnan(this->initial_value_)) { value = this->initial_value_; diff --git a/esphome/components/sps30/sps30.cpp b/esphome/components/sps30/sps30.cpp index c0df539867..21a782e49a 100644 --- a/esphome/components/sps30/sps30.cpp +++ b/esphome/components/sps30/sps30.cpp @@ -22,7 +22,6 @@ static const size_t SERIAL_NUMBER_LENGTH = 8; static const uint8_t MAX_SKIPPED_DATA_CYCLES_BEFORE_ERROR = 5; void SPS30Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->write_command(SPS30_CMD_SOFT_RESET); /// Deferred Sensor initialization this->set_timeout(500, [this]() { @@ -44,31 +43,33 @@ void SPS30Component::setup() { this->serial_number_[i * 2] = static_cast(raw_serial_number[i] >> 8); this->serial_number_[i * 2 + 1] = uint16_t(uint16_t(raw_serial_number[i] & 0xFF)); } - ESP_LOGD(TAG, " Serial Number: '%s'", this->serial_number_); + ESP_LOGV(TAG, " Serial number: %s", this->serial_number_); bool result; if (this->fan_interval_.has_value()) { // override default value - result = write_command(SPS30_CMD_SET_AUTOMATIC_CLEANING_INTERVAL_SECONDS, this->fan_interval_.value()); + result = this->write_command(SPS30_CMD_SET_AUTOMATIC_CLEANING_INTERVAL_SECONDS, this->fan_interval_.value()); } else { - result = write_command(SPS30_CMD_SET_AUTOMATIC_CLEANING_INTERVAL_SECONDS); - } - if (result) { - delay(20); - uint16_t secs[2]; - if (this->read_data(secs, 2)) { - fan_interval_ = secs[0] << 16 | secs[1]; - } + result = this->write_command(SPS30_CMD_SET_AUTOMATIC_CLEANING_INTERVAL_SECONDS); } - this->status_clear_warning(); - this->skipped_data_read_cycles_ = 0; - this->start_continuous_measurement_(); + this->set_timeout(20, [this, result]() { + if (result) { + uint16_t secs[2]; + if (this->read_data(secs, 2)) { + this->fan_interval_ = secs[0] << 16 | secs[1]; + } + } + this->status_clear_warning(); + this->skipped_data_read_cycles_ = 0; + this->start_continuous_measurement_(); + this->setup_complete_ = true; + }); }); } void SPS30Component::dump_config() { - ESP_LOGCONFIG(TAG, "sps30:"); + ESP_LOGCONFIG(TAG, "SPS30:"); LOG_I2C_DEVICE(this); if (this->is_failed()) { switch (this->error_code_) { @@ -79,16 +80,16 @@ void SPS30Component::dump_config() { ESP_LOGW(TAG, "Measurement Initialization failed"); break; case SERIAL_NUMBER_REQUEST_FAILED: - ESP_LOGW(TAG, "Unable to request sensor serial number"); + ESP_LOGW(TAG, "Unable to request serial number"); break; case SERIAL_NUMBER_READ_FAILED: - ESP_LOGW(TAG, "Unable to read sensor serial number"); + ESP_LOGW(TAG, "Unable to read serial number"); break; case FIRMWARE_VERSION_REQUEST_FAILED: - ESP_LOGW(TAG, "Unable to request sensor firmware version"); + ESP_LOGW(TAG, "Unable to request firmware version"); break; case FIRMWARE_VERSION_READ_FAILED: - ESP_LOGW(TAG, "Unable to read sensor firmware version"); + ESP_LOGW(TAG, "Unable to read firmware version"); break; default: ESP_LOGW(TAG, "Unknown setup error"); @@ -97,9 +98,9 @@ void SPS30Component::dump_config() { } LOG_UPDATE_INTERVAL(this); ESP_LOGCONFIG(TAG, - " Serial Number: '%s'\n" + " Serial number: %s\n" " Firmware version v%0d.%0d", - this->serial_number_, (raw_firmware_version_ >> 8), uint16_t(raw_firmware_version_ & 0xFF)); + this->serial_number_, this->raw_firmware_version_ >> 8, this->raw_firmware_version_ & 0xFF); LOG_SENSOR(" ", "PM1.0 Weight Concentration", this->pm_1_0_sensor_); LOG_SENSOR(" ", "PM2.5 Weight Concentration", this->pm_2_5_sensor_); LOG_SENSOR(" ", "PM4 Weight Concentration", this->pm_4_0_sensor_); @@ -112,17 +113,19 @@ void SPS30Component::dump_config() { } void SPS30Component::update() { + if (!this->setup_complete_) + return; /// Check if warning flag active (sensor reconnected?) if (this->status_has_warning()) { - ESP_LOGD(TAG, "Trying to reconnect"); + ESP_LOGD(TAG, "Reconnecting"); if (this->write_command(SPS30_CMD_SOFT_RESET)) { - ESP_LOGD(TAG, "Soft-reset successful. Waiting for reconnection in 500 ms"); + ESP_LOGD(TAG, "Soft-reset successful; waiting 500 ms"); this->set_timeout(500, [this]() { this->start_continuous_measurement_(); /// Sensor restarted and reading attempt made next cycle this->status_clear_warning(); this->skipped_data_read_cycles_ = 0; - ESP_LOGD(TAG, "Reconnect successful. Resuming continuous measurement"); + ESP_LOGD(TAG, "Reconnected; resuming continuous measurement"); }); } else { ESP_LOGD(TAG, "Soft-reset failed"); @@ -137,12 +140,12 @@ void SPS30Component::update() { uint16_t raw_read_status; if (!this->read_data(&raw_read_status, 1) || raw_read_status == 0x00) { - ESP_LOGD(TAG, "Not ready yet"); + ESP_LOGD(TAG, "Not ready"); this->skipped_data_read_cycles_++; /// The following logic is required to address the cases when a sensor is quickly replaced before it's marked /// as failed so that new sensor is eventually forced to be reinitialized for continuous measurement. if (this->skipped_data_read_cycles_ > MAX_SKIPPED_DATA_CYCLES_BEFORE_ERROR) { - ESP_LOGD(TAG, "Exceeded max allowed attempts; communication will be reinitialized"); + ESP_LOGD(TAG, "Exceeded max attempts; will reinitialize"); this->status_set_warning(); } return; @@ -212,11 +215,6 @@ void SPS30Component::update() { } bool SPS30Component::start_continuous_measurement_() { - uint8_t data[4]; - data[0] = SPS30_CMD_START_CONTINUOUS_MEASUREMENTS & 0xFF; - data[1] = 0x03; - data[2] = 0x00; - data[3] = sht_crc_(0x03, 0x00); if (!this->write_command(SPS30_CMD_START_CONTINUOUS_MEASUREMENTS, SPS30_CMD_START_CONTINUOUS_MEASUREMENTS_ARG)) { ESP_LOGE(TAG, "Error initiating measurements"); return false; @@ -225,9 +223,9 @@ bool SPS30Component::start_continuous_measurement_() { } bool SPS30Component::start_fan_cleaning() { - if (!write_command(SPS30_CMD_START_FAN_CLEANING)) { + if (!this->write_command(SPS30_CMD_START_FAN_CLEANING)) { this->status_set_warning(); - ESP_LOGE(TAG, "write error start fan (%d)", this->last_error_); + ESP_LOGE(TAG, "Start fan cleaning failed (%d)", this->last_error_); return false; } else { ESP_LOGD(TAG, "Fan auto clean started"); diff --git a/esphome/components/sps30/sps30.h b/esphome/components/sps30/sps30.h index 04189247e8..18847e16d9 100644 --- a/esphome/components/sps30/sps30.h +++ b/esphome/components/sps30/sps30.h @@ -30,12 +30,14 @@ class SPS30Component : public PollingComponent, public sensirion_common::Sensiri bool start_fan_cleaning(); protected: - char serial_number_[17] = {0}; /// Terminating NULL character + bool setup_complete_{false}; uint16_t raw_firmware_version_; - bool start_continuous_measurement_(); + char serial_number_[17] = {0}; /// Terminating NULL character uint8_t skipped_data_read_cycles_ = 0; - enum ErrorCode { + bool start_continuous_measurement_(); + + enum ErrorCode : uint8_t { COMMUNICATION_FAILED, FIRMWARE_VERSION_REQUEST_FAILED, FIRMWARE_VERSION_READ_FAILED, diff --git a/esphome/components/ssd1306_base/__init__.py b/esphome/components/ssd1306_base/__init__.py index ab2c7a5496..9d397e396b 100644 --- a/esphome/components/ssd1306_base/__init__.py +++ b/esphome/components/ssd1306_base/__init__.py @@ -6,6 +6,8 @@ from esphome.const import ( CONF_BRIGHTNESS, CONF_CONTRAST, CONF_EXTERNAL_VCC, + CONF_FLIP_X, + CONF_FLIP_Y, CONF_INVERT, CONF_LAMBDA, CONF_MODEL, @@ -18,9 +20,6 @@ ssd1306_base_ns = cg.esphome_ns.namespace("ssd1306_base") SSD1306 = ssd1306_base_ns.class_("SSD1306", cg.PollingComponent, display.DisplayBuffer) SSD1306Model = ssd1306_base_ns.enum("SSD1306Model") -CONF_FLIP_X = "flip_x" -CONF_FLIP_Y = "flip_y" - MODELS = { "SSD1306_128X32": SSD1306Model.SSD1306_MODEL_128_32, "SSD1306_128X64": SSD1306Model.SSD1306_MODEL_128_64, @@ -43,14 +42,15 @@ SSD1306_MODEL = cv.enum(MODELS, upper=True, space="_") def _validate(value): model = value[CONF_MODEL] - if model not in ("SSD1305_128X32", "SSD1305_128X64"): - # Contrast is default value (1.0) while brightness is not - # Indicates user is using old `brightness` option - if value[CONF_BRIGHTNESS] != 1.0 and value[CONF_CONTRAST] == 1.0: - raise cv.Invalid( - "SSD1306/SH1106 no longer accepts brightness option, " - 'please use "contrast" instead.' - ) + if ( + model not in ("SSD1305_128X32", "SSD1305_128X64") + and value[CONF_BRIGHTNESS] != 1.0 + and value[CONF_CONTRAST] == 1.0 + ): + raise cv.Invalid( + "SSD1306/SH1106 no longer accepts brightness option, " + 'please use "contrast" instead.' + ) return value diff --git a/esphome/components/ssd1306_base/ssd1306_base.cpp b/esphome/components/ssd1306_base/ssd1306_base.cpp index 1f039cff78..00425b853f 100644 --- a/esphome/components/ssd1306_base/ssd1306_base.cpp +++ b/esphome/components/ssd1306_base/ssd1306_base.cpp @@ -176,7 +176,7 @@ void SSD1306::setup() { // Disable scrolling mode (0x2E) this->command(SSD1306_COMMAND_DEACTIVATE_SCROLL); - // Contrast and brighrness + // Contrast and brightness // SSD1306 does not have brightness setting set_contrast(this->contrast_); if (this->is_ssd1305_()) diff --git a/esphome/components/ssd1306_i2c/ssd1306_i2c.cpp b/esphome/components/ssd1306_i2c/ssd1306_i2c.cpp index f9a2609948..8e490834bc 100644 --- a/esphome/components/ssd1306_i2c/ssd1306_i2c.cpp +++ b/esphome/components/ssd1306_i2c/ssd1306_i2c.cpp @@ -7,7 +7,6 @@ namespace ssd1306_i2c { static const char *const TAG = "ssd1306_i2c"; void I2CSSD1306::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->init_reset_(); auto err = this->write(nullptr, 0); diff --git a/esphome/components/ssd1306_spi/ssd1306_spi.cpp b/esphome/components/ssd1306_spi/ssd1306_spi.cpp index 249e6593ae..d93742c0e5 100644 --- a/esphome/components/ssd1306_spi/ssd1306_spi.cpp +++ b/esphome/components/ssd1306_spi/ssd1306_spi.cpp @@ -8,7 +8,6 @@ namespace ssd1306_spi { static const char *const TAG = "ssd1306_spi"; void SPISSD1306::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); this->dc_pin_->setup(); // OUTPUT diff --git a/esphome/components/ssd1322_spi/ssd1322_spi.cpp b/esphome/components/ssd1322_spi/ssd1322_spi.cpp index fb2d8afe1c..6a8918353b 100644 --- a/esphome/components/ssd1322_spi/ssd1322_spi.cpp +++ b/esphome/components/ssd1322_spi/ssd1322_spi.cpp @@ -8,7 +8,6 @@ namespace ssd1322_spi { static const char *const TAG = "ssd1322_spi"; void SPISSD1322::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); this->dc_pin_->setup(); // OUTPUT if (this->cs_) diff --git a/esphome/components/ssd1325_spi/ssd1325_spi.cpp b/esphome/components/ssd1325_spi/ssd1325_spi.cpp index d2a365326f..3c9dfd3324 100644 --- a/esphome/components/ssd1325_spi/ssd1325_spi.cpp +++ b/esphome/components/ssd1325_spi/ssd1325_spi.cpp @@ -8,7 +8,6 @@ namespace ssd1325_spi { static const char *const TAG = "ssd1325_spi"; void SPISSD1325::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); this->dc_pin_->setup(); // OUTPUT if (this->cs_) diff --git a/esphome/components/ssd1327_i2c/ssd1327_i2c.cpp b/esphome/components/ssd1327_i2c/ssd1327_i2c.cpp index 4e1c5e4ea0..3597a38c44 100644 --- a/esphome/components/ssd1327_i2c/ssd1327_i2c.cpp +++ b/esphome/components/ssd1327_i2c/ssd1327_i2c.cpp @@ -7,7 +7,6 @@ namespace ssd1327_i2c { static const char *const TAG = "ssd1327_i2c"; void I2CSSD1327::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->init_reset_(); auto err = this->write(nullptr, 0); diff --git a/esphome/components/ssd1327_spi/ssd1327_spi.cpp b/esphome/components/ssd1327_spi/ssd1327_spi.cpp index a5eaf252c4..c26238ae19 100644 --- a/esphome/components/ssd1327_spi/ssd1327_spi.cpp +++ b/esphome/components/ssd1327_spi/ssd1327_spi.cpp @@ -8,7 +8,6 @@ namespace ssd1327_spi { static const char *const TAG = "ssd1327_spi"; void SPISSD1327::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); this->dc_pin_->setup(); // OUTPUT if (this->cs_) diff --git a/esphome/components/ssd1331_spi/ssd1331_spi.cpp b/esphome/components/ssd1331_spi/ssd1331_spi.cpp index aeff2bbbfd..232822d192 100644 --- a/esphome/components/ssd1331_spi/ssd1331_spi.cpp +++ b/esphome/components/ssd1331_spi/ssd1331_spi.cpp @@ -8,7 +8,6 @@ namespace ssd1331_spi { static const char *const TAG = "ssd1331_spi"; void SPISSD1331::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); this->dc_pin_->setup(); // OUTPUT if (this->cs_) diff --git a/esphome/components/ssd1351_spi/ssd1351_spi.cpp b/esphome/components/ssd1351_spi/ssd1351_spi.cpp index 5ae7c308d4..ffac07b82b 100644 --- a/esphome/components/ssd1351_spi/ssd1351_spi.cpp +++ b/esphome/components/ssd1351_spi/ssd1351_spi.cpp @@ -8,7 +8,6 @@ namespace ssd1351_spi { static const char *const TAG = "ssd1351_spi"; void SPISSD1351::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); this->dc_pin_->setup(); // OUTPUT if (this->cs_) diff --git a/esphome/components/st7567_i2c/st7567_i2c.cpp b/esphome/components/st7567_i2c/st7567_i2c.cpp index 0640d3be8d..14c21d5148 100644 --- a/esphome/components/st7567_i2c/st7567_i2c.cpp +++ b/esphome/components/st7567_i2c/st7567_i2c.cpp @@ -7,7 +7,6 @@ namespace st7567_i2c { static const char *const TAG = "st7567_i2c"; void I2CST7567::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->init_reset_(); auto err = this->write(nullptr, 0); @@ -51,9 +50,10 @@ void HOT I2CST7567::write_display_data() { static const size_t BLOCK_SIZE = 64; for (uint8_t x = 0; x < (uint8_t) this->get_width_internal(); x += BLOCK_SIZE) { + size_t remaining = static_cast(this->get_width_internal()) - x; + size_t chunk = remaining > BLOCK_SIZE ? BLOCK_SIZE : remaining; this->write_register(esphome::st7567_base::ST7567_SET_START_LINE, &buffer_[y * this->get_width_internal() + x], - this->get_width_internal() - x > BLOCK_SIZE ? BLOCK_SIZE : this->get_width_internal() - x, - true); + chunk); } } } diff --git a/esphome/components/st7567_spi/st7567_spi.cpp b/esphome/components/st7567_spi/st7567_spi.cpp index c5c5836200..813afcf682 100644 --- a/esphome/components/st7567_spi/st7567_spi.cpp +++ b/esphome/components/st7567_spi/st7567_spi.cpp @@ -7,7 +7,6 @@ namespace st7567_spi { static const char *const TAG = "st7567_spi"; void SPIST7567::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); this->dc_pin_->setup(); if (this->cs_) diff --git a/esphome/components/st7701s/display.py b/esphome/components/st7701s/display.py index c6ad43c14c..497740b8d2 100644 --- a/esphome/components/st7701s/display.py +++ b/esphome/components/st7701s/display.py @@ -2,9 +2,17 @@ from esphome import pins import esphome.codegen as cg from esphome.components import display, spi from esphome.components.esp32 import const, only_on_variant -from esphome.components.rpi_dpi_rgb.display import ( +from esphome.components.mipi import ( + CONF_DE_PIN, + CONF_HSYNC_BACK_PORCH, + CONF_HSYNC_FRONT_PORCH, + CONF_HSYNC_PULSE_WIDTH, CONF_PCLK_FREQUENCY, CONF_PCLK_INVERTED, + CONF_PCLK_PIN, + CONF_VSYNC_BACK_PORCH, + CONF_VSYNC_FRONT_PORCH, + CONF_VSYNC_PULSE_WIDTH, ) import esphome.config_validation as cv from esphome.const import ( @@ -36,16 +44,6 @@ from esphome.core import TimePeriod from .init_sequences import ST7701S_INITS, cmd -CONF_DE_PIN = "de_pin" -CONF_PCLK_PIN = "pclk_pin" - -CONF_HSYNC_PULSE_WIDTH = "hsync_pulse_width" -CONF_HSYNC_BACK_PORCH = "hsync_back_porch" -CONF_HSYNC_FRONT_PORCH = "hsync_front_porch" -CONF_VSYNC_PULSE_WIDTH = "vsync_pulse_width" -CONF_VSYNC_BACK_PORCH = "vsync_back_porch" -CONF_VSYNC_FRONT_PORCH = "vsync_front_porch" - DEPENDENCIES = ["spi", "esp32"] st7701s_ns = cg.esphome_ns.namespace("st7701s") @@ -191,7 +189,6 @@ async def to_code(config): cg.add(var.set_vsync_front_porch(config[CONF_VSYNC_FRONT_PORCH])) cg.add(var.set_pclk_inverted(config[CONF_PCLK_INVERTED])) cg.add(var.set_pclk_frequency(config[CONF_PCLK_FREQUENCY])) - index = 0 dpins = [] if CONF_RED in config[CONF_DATA_PINS]: red_pins = config[CONF_DATA_PINS][CONF_RED] @@ -209,10 +206,9 @@ async def to_code(config): dpins = dpins[8:16] + dpins[0:8] else: dpins = config[CONF_DATA_PINS] - for pin in dpins: + for index, pin in enumerate(dpins): data_pin = await cg.gpio_pin_expression(pin) cg.add(var.add_data_pin(data_pin, index)) - index += 1 if dc_pin := config.get(CONF_DC_PIN): dc = await cg.gpio_pin_expression(dc_pin) diff --git a/esphome/components/st7701s/st7701s.cpp b/esphome/components/st7701s/st7701s.cpp index 46509a7f9f..6314c99fb0 100644 --- a/esphome/components/st7701s/st7701s.cpp +++ b/esphome/components/st7701s/st7701s.cpp @@ -6,7 +6,6 @@ namespace esphome { namespace st7701s { void ST7701S::setup() { - esph_log_config(TAG, "Setting up ST7701S"); this->spi_setup(); this->write_init_sequence_(); @@ -25,7 +24,6 @@ void ST7701S::setup() { config.timings.flags.pclk_active_neg = this->pclk_inverted_; config.timings.pclk_hz = this->pclk_frequency_; config.clk_src = LCD_CLK_SRC_PLL160M; - config.psram_trans_align = 64; size_t data_pin_count = sizeof(this->data_pins_) / sizeof(this->data_pins_[0]); for (size_t i = 0; i != data_pin_count; i++) { config.data_gpio_nums[i] = this->data_pins_[i]->get_pin(); @@ -42,7 +40,6 @@ void ST7701S::setup() { if (err != ESP_OK) { esph_log_e(TAG, "lcd_new_rgb_panel failed: %s", esp_err_to_name(err)); } - esph_log_config(TAG, "ST7701S setup complete"); } void ST7701S::loop() { diff --git a/esphome/components/st7735/st7735.cpp b/esphome/components/st7735/st7735.cpp index 9c9c0a3df5..160ba151f7 100644 --- a/esphome/components/st7735/st7735.cpp +++ b/esphome/components/st7735/st7735.cpp @@ -233,7 +233,6 @@ ST7735::ST7735(ST7735Model model, int width, int height, int colstart, int rowst height_(height) {} void ST7735::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->spi_setup(); this->dc_pin_->setup(); // OUTPUT diff --git a/esphome/components/st7789v/st7789v.cpp b/esphome/components/st7789v/st7789v.cpp index 1f3cd50d6c..ade9c1126f 100644 --- a/esphome/components/st7789v/st7789v.cpp +++ b/esphome/components/st7789v/st7789v.cpp @@ -8,7 +8,6 @@ static const char *const TAG = "st7789v"; static const size_t TEMP_BUFFER_SIZE = 128; void ST7789V::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); #ifdef USE_POWER_SUPPLY this->power_.request(); // the PowerSupply component takes care of post turn-on delay @@ -177,8 +176,9 @@ void ST7789V::write_display_data() { if (this->eightbitcolor_) { uint8_t temp_buffer[TEMP_BUFFER_SIZE]; size_t temp_index = 0; - for (int line = 0; line < this->get_buffer_length_(); line = line + this->get_width_internal()) { - for (int index = 0; index < this->get_width_internal(); ++index) { + size_t width = static_cast(this->get_width_internal()); + for (size_t line = 0; line < this->get_buffer_length_(); line += width) { + for (size_t index = 0; index < width; ++index) { auto color = display::ColorUtil::color_to_565( display::ColorUtil::to_color(this->buffer_[index + line], display::ColorOrder::COLOR_ORDER_RGB, display::ColorBitness::COLOR_BITNESS_332, true)); diff --git a/esphome/components/st7920/st7920.cpp b/esphome/components/st7920/st7920.cpp index 54ac6d2efd..c7ce7140e3 100644 --- a/esphome/components/st7920/st7920.cpp +++ b/esphome/components/st7920/st7920.cpp @@ -32,7 +32,6 @@ static const uint8_t LCD_LINE2 = 0x88; static const uint8_t LCD_LINE3 = 0x98; void ST7920::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->dump_config(); this->spi_setup(); this->init_internal_(this->get_buffer_length_()); diff --git a/esphome/components/statsd/statsd.cpp b/esphome/components/statsd/statsd.cpp index 05f71c7b24..7729f36858 100644 --- a/esphome/components/statsd/statsd.cpp +++ b/esphome/components/statsd/statsd.cpp @@ -151,7 +151,7 @@ void StatsdComponent::send_(std::string *out) { int n_bytes = this->sock_->sendto(out->c_str(), out->length(), 0, reinterpret_cast(&this->destination_), sizeof(this->destination_)); - if (n_bytes != out->length()) { + if (n_bytes != static_cast(out->length())) { ESP_LOGE(TAG, "Failed to send UDP packed (%d of %d)", n_bytes, out->length()); } #endif diff --git a/esphome/components/statsd/statsd.h b/esphome/components/statsd/statsd.h index 34f84cbe00..eab77a7a6e 100644 --- a/esphome/components/statsd/statsd.h +++ b/esphome/components/statsd/statsd.h @@ -28,21 +28,6 @@ namespace esphome { namespace statsd { -using sensor_type_t = enum { TYPE_SENSOR, TYPE_BINARY_SENSOR }; - -using sensors_t = struct { - const char *name; - sensor_type_t type; - union { -#ifdef USE_SENSOR - esphome::sensor::Sensor *sensor; -#endif -#ifdef USE_BINARY_SENSOR - esphome::binary_sensor::BinarySensor *binary_sensor; -#endif - }; -}; - class StatsdComponent : public PollingComponent { public: ~StatsdComponent(); @@ -71,6 +56,20 @@ class StatsdComponent : public PollingComponent { const char *prefix_; uint16_t port_; + using sensor_type_t = enum { TYPE_SENSOR, TYPE_BINARY_SENSOR }; + using sensors_t = struct { + const char *name; + sensor_type_t type; + union { +#ifdef USE_SENSOR + esphome::sensor::Sensor *sensor; +#endif +#ifdef USE_BINARY_SENSOR + esphome::binary_sensor::BinarySensor *binary_sensor; +#endif + }; + }; + std::vector sensors_; #ifdef USE_ESP8266 diff --git a/esphome/components/status_led/__init__.py b/esphome/components/status_led/__init__.py index b299ae7ff7..b0fce37126 100644 --- a/esphome/components/status_led/__init__.py +++ b/esphome/components/status_led/__init__.py @@ -2,7 +2,7 @@ from esphome import pins import esphome.codegen as cg import esphome.config_validation as cv from esphome.const import CONF_ID, CONF_PIN -from esphome.core import coroutine_with_priority +from esphome.core import CoroPriority, coroutine_with_priority status_led_ns = cg.esphome_ns.namespace("status_led") StatusLED = status_led_ns.class_("StatusLED", cg.Component) @@ -15,7 +15,7 @@ CONFIG_SCHEMA = cv.Schema( ).extend(cv.COMPONENT_SCHEMA) -@coroutine_with_priority(80.0) +@coroutine_with_priority(CoroPriority.STATUS) async def to_code(config): pin = await cg.gpio_pin_expression(config[CONF_PIN]) rhs = StatusLED.new(pin) diff --git a/esphome/components/status_led/light/status_led_light.cpp b/esphome/components/status_led/light/status_led_light.cpp index dc4820f6da..ec7bf2dae1 100644 --- a/esphome/components/status_led/light/status_led_light.cpp +++ b/esphome/components/status_led/light/status_led_light.cpp @@ -53,8 +53,6 @@ void StatusLEDLightOutput::write_state(light::LightState *state) { } void StatusLEDLightOutput::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - if (this->pin_ != nullptr) { this->pin_->setup(); this->pin_->digital_write(false); diff --git a/esphome/components/status_led/status_led.cpp b/esphome/components/status_led/status_led.cpp index a17d4398fd..344c1e3070 100644 --- a/esphome/components/status_led/status_led.cpp +++ b/esphome/components/status_led/status_led.cpp @@ -11,7 +11,6 @@ StatusLED *global_status_led = nullptr; // NOLINT(cppcoreguidelines-avoid-non-c StatusLED::StatusLED(GPIOPin *pin) : pin_(pin) { global_status_led = this; } void StatusLED::pre_setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->pin_->setup(); this->pin_->digital_write(false); } diff --git a/esphome/components/stepper/__init__.py b/esphome/components/stepper/__init__.py index 66fe88c6c4..62bc71f2d1 100644 --- a/esphome/components/stepper/__init__.py +++ b/esphome/components/stepper/__init__.py @@ -10,7 +10,7 @@ from esphome.const import ( CONF_SPEED, CONF_TARGET, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority IS_PLATFORM_COMPONENT = True @@ -27,8 +27,7 @@ SetDecelerationAction = stepper_ns.class_("SetDecelerationAction", automation.Ac def validate_acceleration(value): value = cv.string(value) for suffix in ("steps/s^2", "steps/s*s", "steps/s/s", "steps/ss", "steps/(s*s)"): - if value.endswith(suffix): - value = value[: -len(suffix)] + value = value.removesuffix(suffix) if value == "inf": return 1e6 @@ -48,8 +47,7 @@ def validate_acceleration(value): def validate_speed(value): value = cv.string(value) for suffix in ("steps/s", "steps/s"): - if value.endswith(suffix): - value = value[: -len(suffix)] + value = value.removesuffix(suffix) if value == "inf": return 1e6 @@ -180,6 +178,6 @@ async def stepper_set_deceleration_to_code(config, action_id, template_arg, args return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): cg.add_global(stepper_ns.using) diff --git a/esphome/components/sts3x/sts3x.cpp b/esphome/components/sts3x/sts3x.cpp index 29aac24e90..eee2aca73e 100644 --- a/esphome/components/sts3x/sts3x.cpp +++ b/esphome/components/sts3x/sts3x.cpp @@ -18,7 +18,6 @@ static const uint16_t STS3X_COMMAND_HEATER_DISABLE = 0x3066; static const uint16_t STS3X_COMMAND_FETCH_DATA = 0xE000; void STS3XComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (!this->write_command(STS3X_COMMAND_READ_SERIAL_NUMBER)) { this->mark_failed(); return; diff --git a/esphome/components/substitutions/__init__.py b/esphome/components/substitutions/__init__.py index e494529b9e..e6bcdc063a 100644 --- a/esphome/components/substitutions/__init__.py +++ b/esphome/components/substitutions/__init__.py @@ -1,10 +1,10 @@ import logging from esphome import core -from esphome.config_helpers import Extend, Remove, merge_config +from esphome.config_helpers import Extend, Remove, merge_config, merge_dicts_ordered import esphome.config_validation as cv from esphome.const import CONF_SUBSTITUTIONS, VALID_SUBSTITUTIONS_CHARACTERS -from esphome.yaml_util import ESPHomeDataBase, make_data_base +from esphome.yaml_util import ESPHomeDataBase, ESPLiteralValue, make_data_base from .jinja import Jinja, JinjaStr, TemplateError, TemplateRuntimeError, has_jinja @@ -49,15 +49,14 @@ def _expand_jinja(value, orig_value, path, jinja, ignore_missing): try: # Invoke the jinja engine to evaluate the expression. value, err = jinja.expand(value) - if err is not None: - if not ignore_missing and "password" not in path: - _LOGGER.warning( - "Found '%s' (see %s) which looks like an expression," - " but could not resolve all the variables: %s", - value, - "->".join(str(x) for x in path), - err.message, - ) + if err is not None and not ignore_missing and "password" not in path: + _LOGGER.warning( + "Found '%s' (see %s) which looks like an expression," + " but could not resolve all the variables: %s", + value, + "->".join(str(x) for x in path), + err.message, + ) except ( TemplateError, TemplateRuntimeError, @@ -128,6 +127,8 @@ def _expand_substitutions(substitutions, value, path, jinja, ignore_missing): def _substitute_item(substitutions, item, path, jinja, ignore_missing): + if isinstance(item, ESPLiteralValue): + return None # do not substitute inside literal blocks if isinstance(item, list): for i, it in enumerate(item): sub = _substitute_item(substitutions, it, path + [i], jinja, ignore_missing) @@ -169,10 +170,10 @@ def do_substitution_pass(config, command_line_substitutions, ignore_missing=Fals return # Merge substitutions in config, overriding with substitutions coming from command line: - substitutions = { - **config.get(CONF_SUBSTITUTIONS, {}), - **(command_line_substitutions or {}), - } + # Use merge_dicts_ordered to preserve OrderedDict type for move_to_end() + substitutions = merge_dicts_ordered( + config.get(CONF_SUBSTITUTIONS, {}), command_line_substitutions or {} + ) with cv.prepend_path("substitutions"): if not isinstance(substitutions, dict): raise cv.Invalid( diff --git a/esphome/components/substitutions/jinja.py b/esphome/components/substitutions/jinja.py index cf393d2a5d..e7164d8fff 100644 --- a/esphome/components/substitutions/jinja.py +++ b/esphome/components/substitutions/jinja.py @@ -1,9 +1,10 @@ +from ast import literal_eval import logging import math import re import jinja2 as jinja -from jinja2.nativetypes import NativeEnvironment +from jinja2.sandbox import SandboxedEnvironment TemplateError = jinja.TemplateError TemplateSyntaxError = jinja.TemplateSyntaxError @@ -25,6 +26,24 @@ def has_jinja(st): return detect_jinja_re.search(st) is not None +# SAFE_GLOBAL_FUNCTIONS defines a allowlist of built-in functions that are considered safe to expose +# in Jinja templates or other sandboxed evaluation contexts. Only functions that do not allow +# arbitrary code execution, file access, or other security risks are included. +# +# The following functions are considered safe: +# - ord: Converts a character to its Unicode code point integer. +# - chr: Converts an integer to its corresponding Unicode character. +# - len: Returns the length of a sequence or collection. +# +# These functions were chosen because they are pure, have no side effects, and do not provide access +# to the file system, environment, or other potentially sensitive resources. +SAFE_GLOBAL_FUNCTIONS = { + "ord": ord, + "chr": chr, + "len": len, +} + + class JinjaStr(str): """ Wraps a string containing an unresolved Jinja expression, @@ -52,7 +71,7 @@ class Jinja: """ def __init__(self, context_vars): - self.env = NativeEnvironment( + self.env = SandboxedEnvironment( trim_blocks=True, lstrip_blocks=True, block_start_string="<%", @@ -66,7 +85,20 @@ class Jinja: self.env.add_extension("jinja2.ext.do") self.env.globals["math"] = math # Inject entire math module self.context_vars = {**context_vars} - self.env.globals = {**self.env.globals, **self.context_vars} + self.env.globals = { + **self.env.globals, + **self.context_vars, + **SAFE_GLOBAL_FUNCTIONS, + } + + def safe_eval(self, expr): + try: + result = literal_eval(expr) + if not isinstance(result, str): + return result + except (ValueError, SyntaxError, MemoryError, TypeError): + pass + return expr def expand(self, content_str): """ @@ -84,7 +116,7 @@ class Jinja: override_vars = content_str.upvalues try: template = self.env.from_string(content_str) - result = template.render(override_vars) + result = self.safe_eval(template.render(override_vars)) if isinstance(result, Undefined): # This happens when the expression is simply an undefined variable. Jinja does not # raise an exception, instead we get "Undefined". diff --git a/esphome/components/sun/__init__.py b/esphome/components/sun/__init__.py index 5a2a39c427..c065a82958 100644 --- a/esphome/components/sun/__init__.py +++ b/esphome/components/sun/__init__.py @@ -1,3 +1,4 @@ +import contextlib import re from esphome import automation @@ -41,12 +42,10 @@ ELEVATION_MAP = { def elevation(value): if isinstance(value, str): - try: + with contextlib.suppress(cv.Invalid): value = ELEVATION_MAP[ cv.one_of(*ELEVATION_MAP, lower=True, space="_")(value) ] - except cv.Invalid: - pass value = cv.angle(value) return cv.float_range(min=-180, max=180)(value) diff --git a/esphome/components/switch/__init__.py b/esphome/components/switch/__init__.py index c09675069f..0e7b35b373 100644 --- a/esphome/components/switch/__init__.py +++ b/esphome/components/switch/__init__.py @@ -10,16 +10,18 @@ from esphome.const import ( CONF_ID, CONF_INVERTED, CONF_MQTT_ID, + CONF_ON_STATE, CONF_ON_TURN_OFF, CONF_ON_TURN_ON, CONF_RESTORE_MODE, + CONF_STATE, CONF_TRIGGER_ID, CONF_WEB_SERVER, DEVICE_CLASS_EMPTY, DEVICE_CLASS_OUTLET, DEVICE_CLASS_SWITCH, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass @@ -48,12 +50,16 @@ RESTORE_MODES = { } +ControlAction = switch_ns.class_("ControlAction", automation.Action) ToggleAction = switch_ns.class_("ToggleAction", automation.Action) TurnOffAction = switch_ns.class_("TurnOffAction", automation.Action) TurnOnAction = switch_ns.class_("TurnOnAction", automation.Action) SwitchPublishAction = switch_ns.class_("SwitchPublishAction", automation.Action) SwitchCondition = switch_ns.class_("SwitchCondition", Condition) +SwitchStateTrigger = switch_ns.class_( + "SwitchStateTrigger", automation.Trigger.template(bool) +) SwitchTurnOnTrigger = switch_ns.class_( "SwitchTurnOnTrigger", automation.Trigger.template() ) @@ -75,6 +81,11 @@ _SWITCH_SCHEMA = ( cv.Optional(CONF_RESTORE_MODE, default="ALWAYS_OFF"): cv.enum( RESTORE_MODES, upper=True, space="_" ), + cv.Optional(CONF_ON_STATE): automation.validate_automation( + { + cv.GenerateID(CONF_TRIGGER_ID): cv.declare_id(SwitchStateTrigger), + } + ), cv.Optional(CONF_ON_TURN_ON): automation.validate_automation( { cv.GenerateID(CONF_TRIGGER_ID): cv.declare_id(SwitchTurnOnTrigger), @@ -138,6 +149,9 @@ async def setup_switch_core_(var, config): if (inverted := config.get(CONF_INVERTED)) is not None: cg.add(var.set_inverted(inverted)) + for conf in config.get(CONF_ON_STATE, []): + trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var) + await automation.build_automation(trigger, [(bool, "x")], conf) for conf in config.get(CONF_ON_TURN_ON, []): trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], var) await automation.build_automation(trigger, [], conf) @@ -177,6 +191,23 @@ SWITCH_ACTION_SCHEMA = maybe_simple_id( cv.Required(CONF_ID): cv.use_id(Switch), } ) +SWITCH_CONTROL_ACTION_SCHEMA = automation.maybe_simple_id( + { + cv.Required(CONF_ID): cv.use_id(Switch), + cv.Required(CONF_STATE): cv.templatable(cv.boolean), + } +) + + +@automation.register_action( + "switch.control", ControlAction, SWITCH_CONTROL_ACTION_SCHEMA +) +async def switch_control_to_code(config, action_id, template_arg, args): + paren = await cg.get_variable(config[CONF_ID]) + var = cg.new_Pvariable(action_id, template_arg, paren) + template_ = await cg.templatable(config[CONF_STATE], args, bool) + cg.add(var.set_state(template_)) + return var @automation.register_action("switch.toggle", ToggleAction, SWITCH_ACTION_SCHEMA) @@ -199,7 +230,6 @@ async def switch_is_off_to_code(config, condition_id, template_arg, args): return cg.new_Pvariable(condition_id, template_arg, paren, False) -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): cg.add_global(switch_ns.using) - cg.add_define("USE_SWITCH") diff --git a/esphome/components/switch/automation.h b/esphome/components/switch/automation.h index 579daf4d24..b8cbc9b976 100644 --- a/esphome/components/switch/automation.h +++ b/esphome/components/switch/automation.h @@ -37,6 +37,23 @@ template class ToggleAction : public Action { Switch *switch_; }; +template class ControlAction : public Action { + public: + explicit ControlAction(Switch *a_switch) : switch_(a_switch) {} + + TEMPLATABLE_VALUE(bool, state) + + void play(Ts... x) override { + auto state = this->state_.optional_value(x...); + if (state.has_value()) { + this->switch_->control(*state); + } + } + + protected: + Switch *switch_; +}; + template class SwitchCondition : public Condition { public: SwitchCondition(Switch *parent, bool state) : parent_(parent), state_(state) {} @@ -47,6 +64,13 @@ template class SwitchCondition : public Condition { bool state_; }; +class SwitchStateTrigger : public Trigger { + public: + SwitchStateTrigger(Switch *a_switch) { + a_switch->add_on_state_callback([this](bool state) { this->trigger(state); }); + } +}; + class SwitchTurnOnTrigger : public Trigger<> { public: SwitchTurnOnTrigger(Switch *a_switch) { diff --git a/esphome/components/switch/switch.cpp b/esphome/components/switch/switch.cpp index c204895755..02cee91a76 100644 --- a/esphome/components/switch/switch.cpp +++ b/esphome/components/switch/switch.cpp @@ -8,6 +8,14 @@ static const char *const TAG = "switch"; Switch::Switch() : state(false) {} +void Switch::control(bool target_state) { + ESP_LOGV(TAG, "'%s' Control: %s", this->get_name().c_str(), ONOFF(target_state)); + if (target_state) { + this->turn_on(); + } else { + this->turn_off(); + } +} void Switch::turn_on() { ESP_LOGD(TAG, "'%s' Turning ON.", this->get_name().c_str()); this->write_state(!this->inverted_); @@ -24,7 +32,7 @@ optional Switch::get_initial_state() { if (!(restore_mode & RESTORE_MODE_PERSISTENT_MASK)) return {}; - this->rtc_ = global_preferences->make_preference(this->get_object_id_hash()); + this->rtc_ = global_preferences->make_preference(this->get_preference_hash()); bool initial_state; if (!this->rtc_.load(&initial_state)) return {}; @@ -83,8 +91,8 @@ void log_switch(const char *tag, const char *prefix, const char *type, Switch *o LOG_STR_ARG(onoff)); // Add optional fields separately - if (!obj->get_icon().empty()) { - ESP_LOGCONFIG(tag, "%s Icon: '%s'", prefix, obj->get_icon().c_str()); + if (!obj->get_icon_ref().empty()) { + ESP_LOGCONFIG(tag, "%s Icon: '%s'", prefix, obj->get_icon_ref().c_str()); } if (obj->assumed_state()) { ESP_LOGCONFIG(tag, "%s Assumed State: YES", prefix); @@ -92,8 +100,8 @@ void log_switch(const char *tag, const char *prefix, const char *type, Switch *o if (obj->is_inverted()) { ESP_LOGCONFIG(tag, "%s Inverted: YES", prefix); } - if (!obj->get_device_class().empty()) { - ESP_LOGCONFIG(tag, "%s Device Class: '%s'", prefix, obj->get_device_class().c_str()); + if (!obj->get_device_class_ref().empty()) { + ESP_LOGCONFIG(tag, "%s Device Class: '%s'", prefix, obj->get_device_class_ref().c_str()); } } } diff --git a/esphome/components/switch/switch.h b/esphome/components/switch/switch.h index b999296564..6371e35292 100644 --- a/esphome/components/switch/switch.h +++ b/esphome/components/switch/switch.h @@ -55,6 +55,14 @@ class Switch : public EntityBase, public EntityBase_DeviceClass { /// The current reported state of the binary sensor. bool state; + /** Control this switch using a boolean state value. + * + * This method provides a unified interface for setting the switch state based on a boolean parameter. + * It automatically calls turn_on() when state is true or turn_off() when state is false. + * + * @param target_state The desired state: true to turn the switch ON, false to turn it OFF. + */ + void control(bool target_state); /** Turn this switch on. This is called by the front-end. * * For implementing switches, please override write_state. diff --git a/esphome/components/sx126x/__init__.py b/esphome/components/sx126x/__init__.py index b6aeaf072c..370cd102d4 100644 --- a/esphome/components/sx126x/__init__.py +++ b/esphome/components/sx126x/__init__.py @@ -15,6 +15,10 @@ CONF_BANDWIDTH = "bandwidth" CONF_BITRATE = "bitrate" CONF_CODING_RATE = "coding_rate" CONF_CRC_ENABLE = "crc_enable" +CONF_CRC_INVERTED = "crc_inverted" +CONF_CRC_SIZE = "crc_size" +CONF_CRC_POLYNOMIAL = "crc_polynomial" +CONF_CRC_INITIAL = "crc_initial" CONF_DEVIATION = "deviation" CONF_DIO1_PIN = "dio1_pin" CONF_HW_VERSION = "hw_version" @@ -188,6 +192,14 @@ CONFIG_SCHEMA = ( cv.Required(CONF_BUSY_PIN): pins.internal_gpio_input_pin_schema, cv.Optional(CONF_CODING_RATE, default="CR_4_5"): cv.enum(CODING_RATE), cv.Optional(CONF_CRC_ENABLE, default=False): cv.boolean, + cv.Optional(CONF_CRC_INVERTED, default=True): cv.boolean, + cv.Optional(CONF_CRC_SIZE, default=2): cv.int_range(min=1, max=2), + cv.Optional(CONF_CRC_POLYNOMIAL, default=0x1021): cv.All( + cv.hex_int, cv.Range(min=0, max=0xFFFF) + ), + cv.Optional(CONF_CRC_INITIAL, default=0x1D0F): cv.All( + cv.hex_int, cv.Range(min=0, max=0xFFFF) + ), cv.Optional(CONF_DEVIATION, default=5000): cv.int_range(min=0, max=100000), cv.Required(CONF_DIO1_PIN): pins.internal_gpio_input_pin_schema, cv.Required(CONF_FREQUENCY): cv.int_range(min=137000000, max=1020000000), @@ -251,6 +263,10 @@ async def to_code(config): cg.add(var.set_shaping(config[CONF_SHAPING])) cg.add(var.set_bitrate(config[CONF_BITRATE])) cg.add(var.set_crc_enable(config[CONF_CRC_ENABLE])) + cg.add(var.set_crc_inverted(config[CONF_CRC_INVERTED])) + cg.add(var.set_crc_size(config[CONF_CRC_SIZE])) + cg.add(var.set_crc_polynomial(config[CONF_CRC_POLYNOMIAL])) + cg.add(var.set_crc_initial(config[CONF_CRC_INITIAL])) cg.add(var.set_payload_length(config[CONF_PAYLOAD_LENGTH])) cg.add(var.set_preamble_size(config[CONF_PREAMBLE_SIZE])) cg.add(var.set_preamble_detect(config[CONF_PREAMBLE_DETECT])) diff --git a/esphome/components/sx126x/sx126x.cpp b/esphome/components/sx126x/sx126x.cpp index b1c81b324a..bb59f26b79 100644 --- a/esphome/components/sx126x/sx126x.cpp +++ b/esphome/components/sx126x/sx126x.cpp @@ -105,8 +105,6 @@ void SX126x::write_register_(uint16_t reg, uint8_t *data, uint8_t size) { } void SX126x::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // setup pins this->busy_pin_->setup(); this->rst_pin_->setup(); @@ -219,7 +217,7 @@ void SX126x::configure() { this->write_opcode_(RADIO_SET_MODULATIONPARAMS, buf, 4); // set packet params and sync word - this->set_packet_params_(this->payload_length_); + this->set_packet_params_(this->get_max_packet_size()); if (this->sync_value_.size() == 2) { this->write_register_(REG_LORA_SYNCWORD, this->sync_value_.data(), this->sync_value_.size()); } @@ -237,8 +235,18 @@ void SX126x::configure() { buf[7] = (fdev >> 0) & 0xFF; this->write_opcode_(RADIO_SET_MODULATIONPARAMS, buf, 8); + // set crc params + if (this->crc_enable_) { + buf[0] = this->crc_initial_ >> 8; + buf[1] = this->crc_initial_ & 0xFF; + this->write_register_(REG_CRC_INITIAL, buf, 2); + buf[0] = this->crc_polynomial_ >> 8; + buf[1] = this->crc_polynomial_ & 0xFF; + this->write_register_(REG_CRC_POLYNOMIAL, buf, 2); + } + // set packet params and sync word - this->set_packet_params_(this->payload_length_); + this->set_packet_params_(this->get_max_packet_size()); if (!this->sync_value_.empty()) { this->write_register_(REG_GFSK_SYNCWORD, this->sync_value_.data(), this->sync_value_.size()); } @@ -276,9 +284,13 @@ void SX126x::set_packet_params_(uint8_t payload_length) { buf[2] = (this->preamble_detect_ > 0) ? ((this->preamble_detect_ - 1) | 0x04) : 0x00; buf[3] = this->sync_value_.size() * 8; buf[4] = 0x00; - buf[5] = 0x00; + buf[5] = (this->payload_length_ > 0) ? 0x00 : 0x01; buf[6] = payload_length; - buf[7] = this->crc_enable_ ? 0x06 : 0x01; + if (this->crc_enable_) { + buf[7] = (this->crc_inverted_ ? 0x04 : 0x00) + (this->crc_size_ & 0x02); + } else { + buf[7] = 0x01; + } buf[8] = 0x00; this->write_opcode_(RADIO_SET_PACKETPARAMS, buf, 9); } @@ -316,6 +328,9 @@ SX126xError SX126x::transmit_packet(const std::vector &packet) { buf[0] = 0xFF; buf[1] = 0xFF; this->write_opcode_(RADIO_CLR_IRQSTATUS, buf, 2); + if (this->payload_length_ == 0) { + this->set_packet_params_(this->get_max_packet_size()); + } if (this->rx_start_) { this->set_mode_rx(); } else { diff --git a/esphome/components/sx126x/sx126x.h b/esphome/components/sx126x/sx126x.h index fd5c37942d..47d6449738 100644 --- a/esphome/components/sx126x/sx126x.h +++ b/esphome/components/sx126x/sx126x.h @@ -67,6 +67,10 @@ class SX126x : public Component, void set_busy_pin(InternalGPIOPin *busy_pin) { this->busy_pin_ = busy_pin; } void set_coding_rate(uint8_t coding_rate) { this->coding_rate_ = coding_rate; } void set_crc_enable(bool crc_enable) { this->crc_enable_ = crc_enable; } + void set_crc_inverted(bool crc_inverted) { this->crc_inverted_ = crc_inverted; } + void set_crc_size(uint8_t crc_size) { this->crc_size_ = crc_size; } + void set_crc_polynomial(uint16_t crc_polynomial) { this->crc_polynomial_ = crc_polynomial; } + void set_crc_initial(uint16_t crc_initial) { this->crc_initial_ = crc_initial; } void set_deviation(uint32_t deviation) { this->deviation_ = deviation; } void set_dio1_pin(InternalGPIOPin *dio1_pin) { this->dio1_pin_ = dio1_pin; } void set_frequency(uint32_t frequency) { this->frequency_ = frequency; } @@ -118,6 +122,11 @@ class SX126x : public Component, char version_[16]; SX126xBw bandwidth_{SX126X_BW_125000}; uint32_t bitrate_{0}; + bool crc_enable_{false}; + bool crc_inverted_{false}; + uint8_t crc_size_{0}; + uint16_t crc_polynomial_{0}; + uint16_t crc_initial_{0}; uint32_t deviation_{0}; uint32_t frequency_{0}; uint32_t payload_length_{0}; @@ -131,7 +140,6 @@ class SX126x : public Component, uint8_t shaping_{0}; uint8_t spreading_factor_{0}; int8_t pa_power_{0}; - bool crc_enable_{false}; bool rx_start_{false}; bool rf_switch_{false}; }; diff --git a/esphome/components/sx126x/sx126x_reg.h b/esphome/components/sx126x/sx126x_reg.h index 3b12d822b5..143f4a05da 100644 --- a/esphome/components/sx126x/sx126x_reg.h +++ b/esphome/components/sx126x/sx126x_reg.h @@ -53,6 +53,8 @@ enum SX126xOpCode : uint8_t { enum SX126xRegister : uint16_t { REG_VERSION_STRING = 0x0320, + REG_CRC_INITIAL = 0x06BC, + REG_CRC_POLYNOMIAL = 0x06BE, REG_GFSK_SYNCWORD = 0x06C0, REG_LORA_SYNCWORD = 0x0740, REG_OCP = 0x08E7, diff --git a/esphome/components/sx127x/sx127x.cpp b/esphome/components/sx127x/sx127x.cpp index 2d2326549b..8e6db5dc9e 100644 --- a/esphome/components/sx127x/sx127x.cpp +++ b/esphome/components/sx127x/sx127x.cpp @@ -50,8 +50,6 @@ void SX127x::write_fifo_(const std::vector &packet) { } void SX127x::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // setup reset this->rst_pin_->setup(); diff --git a/esphome/components/sx1509/__init__.py b/esphome/components/sx1509/__init__.py index f1b08a505a..b61b92fd1e 100644 --- a/esphome/components/sx1509/__init__.py +++ b/esphome/components/sx1509/__init__.py @@ -25,7 +25,7 @@ CONF_SCAN_TIME = "scan_time" CONF_DEBOUNCE_TIME = "debounce_time" CONF_SX1509_ID = "sx1509_id" -AUTO_LOAD = ["key_provider"] +AUTO_LOAD = ["key_provider", "gpio_expander"] DEPENDENCIES = ["i2c"] MULTI_CONF = True @@ -41,11 +41,13 @@ SX1509KeyTrigger = sx1509_ns.class_( def check_keys(config): - if CONF_KEYS in config: - if len(config[CONF_KEYS]) != config[CONF_KEY_ROWS] * config[CONF_KEY_COLUMNS]: - raise cv.Invalid( - "The number of key codes must equal the number of rows * columns" - ) + if ( + CONF_KEYS in config + and len(config[CONF_KEYS]) != config[CONF_KEY_ROWS] * config[CONF_KEY_COLUMNS] + ): + raise cv.Invalid( + "The number of key codes must equal the number of rows * columns" + ) return config diff --git a/esphome/components/sx1509/output/sx1509_float_output.cpp b/esphome/components/sx1509/output/sx1509_float_output.cpp index e9c401eeed..1d2541bb46 100644 --- a/esphome/components/sx1509/output/sx1509_float_output.cpp +++ b/esphome/components/sx1509/output/sx1509_float_output.cpp @@ -15,7 +15,7 @@ void SX1509FloatOutputChannel::write_state(float state) { } void SX1509FloatOutputChannel::setup() { - ESP_LOGD(TAG, "setup pin %d", this->pin_); + ESP_LOGD(TAG, "Pin %d", this->pin_); this->parent_->pin_mode(this->pin_, gpio::FLAG_OUTPUT); this->parent_->setup_led_driver(this->pin_); this->turn_off(); diff --git a/esphome/components/sx1509/sx1509.cpp b/esphome/components/sx1509/sx1509.cpp index d323c9a92c..746ec9cda3 100644 --- a/esphome/components/sx1509/sx1509.cpp +++ b/esphome/components/sx1509/sx1509.cpp @@ -8,8 +8,6 @@ namespace sx1509 { static const char *const TAG = "sx1509"; void SX1509Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - ESP_LOGV(TAG, " Resetting devices"); if (!this->write_byte(REG_RESET, 0x12)) { this->mark_failed(); @@ -41,6 +39,9 @@ void SX1509Component::dump_config() { } void SX1509Component::loop() { + // Reset cache at the start of each loop + this->reset_pin_cache_(); + if (this->has_keypad_) { if (millis() - this->last_loop_timestamp_ < min_loop_period_) return; @@ -75,18 +76,20 @@ void SX1509Component::loop() { } } -bool SX1509Component::digital_read(uint8_t pin) { +bool SX1509Component::digital_read_hw(uint8_t pin) { + // Always read all pins when any input pin is accessed + return this->read_byte_16(REG_DATA_B, &this->input_mask_); +} + +bool SX1509Component::digital_read_cache(uint8_t pin) { + // Return cached value for input pins, false for output pins if (this->ddr_mask_ & (1 << pin)) { - uint16_t temp_reg_data; - if (!this->read_byte_16(REG_DATA_B, &temp_reg_data)) - return false; - if (temp_reg_data & (1 << pin)) - return true; + return (this->input_mask_ & (1 << pin)) != 0; } return false; } -void SX1509Component::digital_write(uint8_t pin, bool bit_value) { +void SX1509Component::digital_write_hw(uint8_t pin, bool bit_value) { if ((~this->ddr_mask_) & (1 << pin)) { // If the pin is an output, write high/low uint16_t temp_reg_data = 0; diff --git a/esphome/components/sx1509/sx1509.h b/esphome/components/sx1509/sx1509.h index c0e86aa8a1..2afd0d0e4e 100644 --- a/esphome/components/sx1509/sx1509.h +++ b/esphome/components/sx1509/sx1509.h @@ -2,6 +2,7 @@ #include "esphome/components/i2c/i2c.h" #include "esphome/components/key_provider/key_provider.h" +#include "esphome/components/gpio_expander/cached_gpio.h" #include "esphome/core/component.h" #include "esphome/core/hal.h" #include "sx1509_gpio_pin.h" @@ -30,7 +31,10 @@ class SX1509Processor { class SX1509KeyTrigger : public Trigger {}; -class SX1509Component : public Component, public i2c::I2CDevice, public key_provider::KeyProvider { +class SX1509Component : public Component, + public i2c::I2CDevice, + public gpio_expander::CachedGpioExpander, + public key_provider::KeyProvider { public: SX1509Component() = default; @@ -39,11 +43,9 @@ class SX1509Component : public Component, public i2c::I2CDevice, public key_prov float get_setup_priority() const override { return setup_priority::HARDWARE; } void loop() override; - bool digital_read(uint8_t pin); uint16_t read_key_data(); void set_pin_value(uint8_t pin, uint8_t i_on) { this->write_byte(REG_I_ON[pin], i_on); }; void pin_mode(uint8_t pin, gpio::Flags flags); - void digital_write(uint8_t pin, bool bit_value); uint32_t get_clock() { return this->clk_x_; }; void set_rows_cols(uint8_t rows, uint8_t cols) { this->rows_ = rows; @@ -61,10 +63,15 @@ class SX1509Component : public Component, public i2c::I2CDevice, public key_prov void setup_led_driver(uint8_t pin); protected: + // Virtual methods from CachedGpioExpander + bool digital_read_hw(uint8_t pin) override; + bool digital_read_cache(uint8_t pin) override; + void digital_write_hw(uint8_t pin, bool value) override; + uint32_t clk_x_ = 2000000; uint8_t frequency_ = 0; uint16_t ddr_mask_ = 0x00; - uint16_t input_mask_ = 0x00; + uint16_t input_mask_ = 0x00; // Cache for input values (16-bit for all pins) uint16_t port_mask_ = 0x00; uint16_t output_state_ = 0x00; bool has_keypad_ = false; diff --git a/esphome/components/syslog/esphome_syslog.cpp b/esphome/components/syslog/esphome_syslog.cpp index e322a6951d..71468fa932 100644 --- a/esphome/components/syslog/esphome_syslog.cpp +++ b/esphome/components/syslog/esphome_syslog.cpp @@ -35,7 +35,7 @@ void Syslog::log_(const int level, const char *tag, const char *message, size_t severity = LOG_LEVEL_TO_SYSLOG_SEVERITY[level]; } int pri = this->facility_ * 8 + severity; - auto timestamp = this->time_->now().strftime("%b %d %H:%M:%S"); + auto timestamp = this->time_->now().strftime("%b %e %H:%M:%S"); size_t len = message_len; // remove color formatting if (this->strip_ && message[0] == 0x1B && len > 11) { diff --git a/esphome/components/tc74/tc74.cpp b/esphome/components/tc74/tc74.cpp index b79bcb5592..abf3839e00 100644 --- a/esphome/components/tc74/tc74.cpp +++ b/esphome/components/tc74/tc74.cpp @@ -15,7 +15,6 @@ static const uint8_t TC74_DATA_READY_MASK = 0x40; // It is possible the "Data Ready" bit will not be set if the TC74 has not been powered on for at least 250ms, so it not // being set does not constitute a failure. void TC74Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t config_reg; if (this->read_register(TC74_REGISTER_CONFIGURATION, &config_reg, 1) != i2c::ERROR_OK) { this->mark_failed(); diff --git a/esphome/components/tca9548a/tca9548a.cpp b/esphome/components/tca9548a/tca9548a.cpp index cdeb94ceca..1de3c49108 100644 --- a/esphome/components/tca9548a/tca9548a.cpp +++ b/esphome/components/tca9548a/tca9548a.cpp @@ -6,25 +6,16 @@ namespace tca9548a { static const char *const TAG = "tca9548a"; -i2c::ErrorCode TCA9548AChannel::readv(uint8_t address, i2c::ReadBuffer *buffers, size_t cnt) { +i2c::ErrorCode TCA9548AChannel::write_readv(uint8_t address, const uint8_t *write_buffer, size_t write_count, + uint8_t *read_buffer, size_t read_count) { auto err = this->parent_->switch_to_channel(channel_); if (err != i2c::ERROR_OK) return err; - err = this->parent_->bus_->readv(address, buffers, cnt); + err = this->parent_->bus_->write_readv(address, write_buffer, write_count, read_buffer, read_count); this->parent_->disable_all_channels(); return err; } -i2c::ErrorCode TCA9548AChannel::writev(uint8_t address, i2c::WriteBuffer *buffers, size_t cnt, bool stop) { - auto err = this->parent_->switch_to_channel(channel_); - if (err != i2c::ERROR_OK) - return err; - err = this->parent_->bus_->writev(address, buffers, cnt, stop); - this->parent_->disable_all_channels(); - return err; -} - void TCA9548AComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t status = 0; if (this->read(&status, 1) != i2c::ERROR_OK) { ESP_LOGE(TAG, "TCA9548A failed"); diff --git a/esphome/components/tca9548a/tca9548a.h b/esphome/components/tca9548a/tca9548a.h index 08f1674d11..0fb9ada99a 100644 --- a/esphome/components/tca9548a/tca9548a.h +++ b/esphome/components/tca9548a/tca9548a.h @@ -14,8 +14,8 @@ class TCA9548AChannel : public i2c::I2CBus { void set_channel(uint8_t channel) { channel_ = channel; } void set_parent(TCA9548AComponent *parent) { parent_ = parent; } - i2c::ErrorCode readv(uint8_t address, i2c::ReadBuffer *buffers, size_t cnt) override; - i2c::ErrorCode writev(uint8_t address, i2c::WriteBuffer *buffers, size_t cnt, bool stop) override; + i2c::ErrorCode write_readv(uint8_t address, const uint8_t *write_buffer, size_t write_count, uint8_t *read_buffer, + size_t read_count) override; protected: uint8_t channel_; diff --git a/esphome/components/tca9555/tca9555.cpp b/esphome/components/tca9555/tca9555.cpp index 7bd2f44918..c3449ce254 100644 --- a/esphome/components/tca9555/tca9555.cpp +++ b/esphome/components/tca9555/tca9555.cpp @@ -16,7 +16,6 @@ namespace tca9555 { static const char *const TAG = "tca9555"; void TCA9555Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); if (!this->read_gpio_modes_()) { this->mark_failed(); return; @@ -51,7 +50,7 @@ bool TCA9555Component::read_gpio_outputs_() { return false; uint8_t data[2]; if (!this->read_bytes(TCA9555_OUTPUT_PORT_REGISTER_0, data, 2)) { - this->status_set_warning("Failed to read output register"); + this->status_set_warning(LOG_STR("Failed to read output register")); return false; } this->output_mask_ = (uint16_t(data[1]) << 8) | (uint16_t(data[0]) << 0); @@ -65,7 +64,7 @@ bool TCA9555Component::read_gpio_modes_() { uint8_t data[2]; bool success = this->read_bytes(TCA9555_CONFIGURATION_PORT_0, data, 2); if (!success) { - this->status_set_warning("Failed to read mode register"); + this->status_set_warning(LOG_STR("Failed to read mode register")); return false; } this->mode_mask_ = (uint16_t(data[1]) << 8) | (uint16_t(data[0]) << 0); @@ -80,7 +79,7 @@ bool TCA9555Component::digital_read_hw(uint8_t pin) { uint8_t bank_number = pin < 8 ? 0 : 1; uint8_t register_to_read = bank_number ? TCA9555_INPUT_PORT_REGISTER_1 : TCA9555_INPUT_PORT_REGISTER_0; if (!this->read_bytes(register_to_read, &data, 1)) { - this->status_set_warning("Failed to read input register"); + this->status_set_warning(LOG_STR("Failed to read input register")); return false; } uint8_t second_half = this->input_mask_ >> 8; @@ -109,7 +108,7 @@ void TCA9555Component::digital_write_hw(uint8_t pin, bool value) { data[0] = this->output_mask_; data[1] = this->output_mask_ >> 8; if (!this->write_bytes(TCA9555_OUTPUT_PORT_REGISTER_0, data, 2)) { - this->status_set_warning("Failed to write output register"); + this->status_set_warning(LOG_STR("Failed to write output register")); return; } @@ -124,7 +123,7 @@ bool TCA9555Component::write_gpio_modes_() { data[0] = this->mode_mask_; data[1] = this->mode_mask_ >> 8; if (!this->write_bytes(TCA9555_CONFIGURATION_PORT_0, data, 2)) { - this->status_set_warning("Failed to write mode register"); + this->status_set_warning(LOG_STR("Failed to write mode register")); return false; } this->status_clear_warning(); diff --git a/esphome/components/tcs34725/tcs34725.cpp b/esphome/components/tcs34725/tcs34725.cpp index 9926ebc553..e4e5547595 100644 --- a/esphome/components/tcs34725/tcs34725.cpp +++ b/esphome/components/tcs34725/tcs34725.cpp @@ -18,7 +18,6 @@ static const uint8_t TCS34725_REGISTER_ENABLE = TCS34725_COMMAND_BIT | 0x00; static const uint8_t TCS34725_REGISTER_CRGBDATAL = TCS34725_COMMAND_BIT | 0x14; void TCS34725Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t id; if (this->read_register(TCS34725_REGISTER_ID, &id, 1) != i2c::ERROR_OK) { this->mark_failed(); diff --git a/esphome/components/tee501/tee501.cpp b/esphome/components/tee501/tee501.cpp index 45241627f9..d6513dbbe0 100644 --- a/esphome/components/tee501/tee501.cpp +++ b/esphome/components/tee501/tee501.cpp @@ -8,12 +8,11 @@ namespace tee501 { static const char *const TAG = "tee501"; void TEE501Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t address[] = {0x70, 0x29}; - this->write(address, 2, false); uint8_t identification[9]; this->read(identification, 9); - if (identification[8] != calc_crc8_(identification, 0, 7)) { + this->write_read(address, sizeof address, identification, sizeof identification); + if (identification[8] != crc8(identification, 8, 0xFF, 0x31, true)) { this->error_code_ = CRC_CHECK_FAILED; this->mark_failed(); return; @@ -42,11 +41,11 @@ void TEE501Component::dump_config() { float TEE501Component::get_setup_priority() const { return setup_priority::DATA; } void TEE501Component::update() { uint8_t address_1[] = {0x2C, 0x1B}; - this->write(address_1, 2, true); + this->write(address_1, 2); this->set_timeout(50, [this]() { uint8_t i2c_response[3]; this->read(i2c_response, 3); - if (i2c_response[2] != calc_crc8_(i2c_response, 0, 1)) { + if (i2c_response[2] != crc8(i2c_response, 2, 0xFF, 0x31, true)) { this->error_code_ = CRC_CHECK_FAILED; this->status_set_warning(); return; @@ -63,24 +62,5 @@ void TEE501Component::update() { }); } -unsigned char TEE501Component::calc_crc8_(const unsigned char buf[], unsigned char from, unsigned char to) { - unsigned char crc_val = 0xFF; - unsigned char i = 0; - unsigned char j = 0; - for (i = from; i <= to; i++) { - int cur_val = buf[i]; - for (j = 0; j < 8; j++) { - if (((crc_val ^ cur_val) & 0x80) != 0) // If MSBs are not equal - { - crc_val = ((crc_val << 1) ^ 0x31); - } else { - crc_val = (crc_val << 1); - } - cur_val = cur_val << 1; - } - } - return crc_val; -} - } // namespace tee501 } // namespace esphome diff --git a/esphome/components/tee501/tee501.h b/esphome/components/tee501/tee501.h index fc655e58c9..2437ac92eb 100644 --- a/esphome/components/tee501/tee501.h +++ b/esphome/components/tee501/tee501.h @@ -1,8 +1,8 @@ #pragma once -#include "esphome/core/component.h" -#include "esphome/components/sensor/sensor.h" #include "esphome/components/i2c/i2c.h" +#include "esphome/components/sensor/sensor.h" +#include "esphome/core/component.h" namespace esphome { namespace tee501 { @@ -16,8 +16,6 @@ class TEE501Component : public sensor::Sensor, public PollingComponent, public i void update() override; protected: - unsigned char calc_crc8_(const unsigned char buf[], unsigned char from, unsigned char to); - enum ErrorCode { NONE = 0, COMMUNICATION_FAILED, CRC_CHECK_FAILED } error_code_{NONE}; }; diff --git a/esphome/components/tem3200/tem3200.cpp b/esphome/components/tem3200/tem3200.cpp index c0655d02b8..b31496142c 100644 --- a/esphome/components/tem3200/tem3200.cpp +++ b/esphome/components/tem3200/tem3200.cpp @@ -16,8 +16,6 @@ enum ErrorCode { }; void TEM3200Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - uint8_t status(NONE); uint16_t raw_temperature(0); uint16_t raw_pressure(0); diff --git a/esphome/components/template/alarm_control_panel/template_alarm_control_panel.cpp b/esphome/components/template/alarm_control_panel/template_alarm_control_panel.cpp index 6f743a77ef..eac0629480 100644 --- a/esphome/components/template/alarm_control_panel/template_alarm_control_panel.cpp +++ b/esphome/components/template/alarm_control_panel/template_alarm_control_panel.cpp @@ -80,14 +80,13 @@ void TemplateAlarmControlPanel::dump_config() { } void TemplateAlarmControlPanel::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); switch (this->restore_mode_) { case ALARM_CONTROL_PANEL_ALWAYS_DISARMED: this->current_state_ = ACP_STATE_DISARMED; break; case ALARM_CONTROL_PANEL_RESTORE_DEFAULT_DISARMED: { uint8_t value; - this->pref_ = global_preferences->make_preference(this->get_object_id_hash()); + this->pref_ = global_preferences->make_preference(this->get_preference_hash()); if (this->pref_.load(&value)) { this->current_state_ = static_cast(value); } else { diff --git a/esphome/components/template/cover/template_cover.cpp b/esphome/components/template/cover/template_cover.cpp index d32c6ac546..84c687536e 100644 --- a/esphome/components/template/cover/template_cover.cpp +++ b/esphome/components/template/cover/template_cover.cpp @@ -16,7 +16,6 @@ TemplateCover::TemplateCover() position_trigger_(new Trigger()), tilt_trigger_(new Trigger()) {} void TemplateCover::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); switch (this->restore_mode_) { case COVER_NO_RESTORE: break; diff --git a/esphome/components/template/datetime/template_date.cpp b/esphome/components/template/datetime/template_date.cpp index 01e15e532e..2fa8016802 100644 --- a/esphome/components/template/datetime/template_date.cpp +++ b/esphome/components/template/datetime/template_date.cpp @@ -20,7 +20,7 @@ void TemplateDate::setup() { } else { datetime::DateEntityRestoreState temp; this->pref_ = - global_preferences->make_preference(194434030U ^ this->get_object_id_hash()); + global_preferences->make_preference(194434030U ^ this->get_preference_hash()); if (this->pref_.load(&temp)) { temp.apply(this); return; diff --git a/esphome/components/template/datetime/template_datetime.cpp b/esphome/components/template/datetime/template_datetime.cpp index 3ab74e197f..a4a4e47d65 100644 --- a/esphome/components/template/datetime/template_datetime.cpp +++ b/esphome/components/template/datetime/template_datetime.cpp @@ -19,8 +19,8 @@ void TemplateDateTime::setup() { state = this->initial_value_; } else { datetime::DateTimeEntityRestoreState temp; - this->pref_ = global_preferences->make_preference(194434090U ^ - this->get_object_id_hash()); + this->pref_ = global_preferences->make_preference( + 194434090U ^ this->get_preference_hash()); if (this->pref_.load(&temp)) { temp.apply(this); return; diff --git a/esphome/components/template/datetime/template_time.cpp b/esphome/components/template/datetime/template_time.cpp index 0e4d734d16..349700f187 100644 --- a/esphome/components/template/datetime/template_time.cpp +++ b/esphome/components/template/datetime/template_time.cpp @@ -20,7 +20,7 @@ void TemplateTime::setup() { } else { datetime::TimeEntityRestoreState temp; this->pref_ = - global_preferences->make_preference(194434060U ^ this->get_object_id_hash()); + global_preferences->make_preference(194434060U ^ this->get_preference_hash()); if (this->pref_.load(&temp)) { temp.apply(this); return; diff --git a/esphome/components/template/number/template_number.cpp b/esphome/components/template/number/template_number.cpp index aaf5b27a71..187f426273 100644 --- a/esphome/components/template/number/template_number.cpp +++ b/esphome/components/template/number/template_number.cpp @@ -14,7 +14,7 @@ void TemplateNumber::setup() { if (!this->restore_value_) { value = this->initial_value_; } else { - this->pref_ = global_preferences->make_preference(this->get_object_id_hash()); + this->pref_ = global_preferences->make_preference(this->get_preference_hash()); if (!this->pref_.load(&value)) { if (!std::isnan(this->initial_value_)) { value = this->initial_value_; diff --git a/esphome/components/template/select/template_select.cpp b/esphome/components/template/select/template_select.cpp index 0160fab04b..95b0ee0d2b 100644 --- a/esphome/components/template/select/template_select.cpp +++ b/esphome/components/template/select/template_select.cpp @@ -11,13 +11,12 @@ void TemplateSelect::setup() { return; std::string value; - ESP_LOGD(TAG, "Setting up"); if (!this->restore_value_) { value = this->initial_option_; ESP_LOGD(TAG, "State from initial: %s", value.c_str()); } else { size_t index; - this->pref_ = global_preferences->make_preference(this->get_object_id_hash()); + this->pref_ = global_preferences->make_preference(this->get_preference_hash()); if (!this->pref_.load(&index)) { value = this->initial_option_; ESP_LOGD(TAG, "State from initial (could not load stored index): %s", value.c_str()); diff --git a/esphome/components/template/text/template_text.cpp b/esphome/components/template/text/template_text.cpp index f8d883e848..d8e840ba7e 100644 --- a/esphome/components/template/text/template_text.cpp +++ b/esphome/components/template/text/template_text.cpp @@ -11,13 +11,11 @@ void TemplateText::setup() { if (this->f_.has_value()) return; } - - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); std::string value = this->initial_value_; if (!this->pref_) { ESP_LOGD(TAG, "State from initial: %s", value.c_str()); } else { - uint32_t key = this->get_object_id_hash(); + uint32_t key = this->get_preference_hash(); key += this->traits.get_min_length() << 2; key += this->traits.get_max_length() << 4; key += fnv1_hash(this->traits.get_pattern()) << 6; diff --git a/esphome/components/template/valve/template_valve.cpp b/esphome/components/template/valve/template_valve.cpp index 8421f5e06f..5fa14a2de7 100644 --- a/esphome/components/template/valve/template_valve.cpp +++ b/esphome/components/template/valve/template_valve.cpp @@ -16,7 +16,6 @@ TemplateValve::TemplateValve() position_trigger_(new Trigger()) {} void TemplateValve::setup() { - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); switch (this->restore_mode_) { case VALVE_NO_RESTORE: break; diff --git a/esphome/components/text/__init__.py b/esphome/components/text/__init__.py index 8362e09ac0..1baacc239f 100644 --- a/esphome/components/text/__init__.py +++ b/esphome/components/text/__init__.py @@ -13,7 +13,7 @@ from esphome.const import ( CONF_VALUE, CONF_WEB_SERVER, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass @@ -149,9 +149,8 @@ async def new_text( return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_TEXT") cg.add_global(text_ns.using) diff --git a/esphome/components/text/text.h b/esphome/components/text/text.h index 3cc0cefc3e..74d08eda8a 100644 --- a/esphome/components/text/text.h +++ b/esphome/components/text/text.h @@ -12,8 +12,8 @@ namespace text { #define LOG_TEXT(prefix, type, obj) \ if ((obj) != nullptr) { \ ESP_LOGCONFIG(TAG, "%s%s '%s'", prefix, LOG_STR_LITERAL(type), (obj)->get_name().c_str()); \ - if (!(obj)->get_icon().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon().c_str()); \ + if (!(obj)->get_icon_ref().empty()) { \ + ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon_ref().c_str()); \ } \ } diff --git a/esphome/components/text/text_traits.h b/esphome/components/text/text_traits.h index 952afa70c7..ceaba2dead 100644 --- a/esphome/components/text/text_traits.h +++ b/esphome/components/text/text_traits.h @@ -3,6 +3,7 @@ #include #include "esphome/core/helpers.h" +#include "esphome/core/string_ref.h" namespace esphome { namespace text { @@ -23,6 +24,7 @@ class TextTraits { // Set/get the pattern. void set_pattern(std::string pattern) { this->pattern_ = std::move(pattern); } std::string get_pattern() const { return this->pattern_; } + StringRef get_pattern_ref() const { return StringRef(this->pattern_); } // Set/get the frontend mode. void set_mode(TextMode mode) { this->mode_ = mode; } diff --git a/esphome/components/text_sensor/__init__.py b/esphome/components/text_sensor/__init__.py index abb2dcae6c..f7b3b5c55e 100644 --- a/esphome/components/text_sensor/__init__.py +++ b/esphome/components/text_sensor/__init__.py @@ -20,7 +20,7 @@ from esphome.const import ( DEVICE_CLASS_EMPTY, DEVICE_CLASS_TIMESTAMP, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass from esphome.util import Registry @@ -162,6 +162,7 @@ def text_sensor_schema( device_class: str = cv.UNDEFINED, entity_category: str = cv.UNDEFINED, icon: str = cv.UNDEFINED, + filters: list = cv.UNDEFINED, ) -> cv.Schema: schema = {} @@ -172,6 +173,7 @@ def text_sensor_schema( (CONF_ICON, icon, cv.icon), (CONF_DEVICE_CLASS, device_class, validate_device_class), (CONF_ENTITY_CATEGORY, entity_category, cv.entity_category), + (CONF_FILTERS, filters, validate_filters), ]: if default is not cv.UNDEFINED: schema[cv.Optional(key, default=default)] = validator @@ -228,9 +230,8 @@ async def new_text_sensor(config, *args): return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_TEXT_SENSOR") cg.add_global(text_sensor_ns.using) diff --git a/esphome/components/text_sensor/text_sensor.cpp b/esphome/components/text_sensor/text_sensor.cpp index c57e0ffefb..17bf20466e 100644 --- a/esphome/components/text_sensor/text_sensor.cpp +++ b/esphome/components/text_sensor/text_sensor.cpp @@ -6,6 +6,22 @@ namespace text_sensor { static const char *const TAG = "text_sensor"; +void log_text_sensor(const char *tag, const char *prefix, const char *type, TextSensor *obj) { + if (obj == nullptr) { + return; + } + + ESP_LOGCONFIG(tag, "%s%s '%s'", prefix, type, obj->get_name().c_str()); + + if (!obj->get_device_class_ref().empty()) { + ESP_LOGCONFIG(tag, "%s Device Class: '%s'", prefix, obj->get_device_class_ref().c_str()); + } + + if (!obj->get_icon_ref().empty()) { + ESP_LOGCONFIG(tag, "%s Icon: '%s'", prefix, obj->get_icon_ref().c_str()); + } +} + void TextSensor::publish_state(const std::string &state) { this->raw_state = state; if (this->raw_callback_) { @@ -70,7 +86,5 @@ void TextSensor::internal_send_state_to_frontend(const std::string &state) { this->callback_.call(state); } -std::string TextSensor::unique_id() { return ""; } - } // namespace text_sensor } // namespace esphome diff --git a/esphome/components/text_sensor/text_sensor.h b/esphome/components/text_sensor/text_sensor.h index b27145aa18..abbea27b59 100644 --- a/esphome/components/text_sensor/text_sensor.h +++ b/esphome/components/text_sensor/text_sensor.h @@ -11,19 +11,9 @@ namespace esphome { namespace text_sensor { -#define LOG_TEXT_SENSOR(prefix, type, obj) \ - if ((obj) != nullptr) { \ - ESP_LOGCONFIG(TAG, "%s%s '%s'", prefix, LOG_STR_LITERAL(type), (obj)->get_name().c_str()); \ - if (!(obj)->get_device_class().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Device Class: '%s'", prefix, (obj)->get_device_class().c_str()); \ - } \ - if (!(obj)->get_icon().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Icon: '%s'", prefix, (obj)->get_icon().c_str()); \ - } \ - if (!(obj)->unique_id().empty()) { \ - ESP_LOGV(TAG, "%s Unique ID: '%s'", prefix, (obj)->unique_id().c_str()); \ - } \ - } +void log_text_sensor(const char *tag, const char *prefix, const char *type, TextSensor *obj); + +#define LOG_TEXT_SENSOR(prefix, type, obj) log_text_sensor(TAG, prefix, LOG_STR_LITERAL(type), obj) #define SUB_TEXT_SENSOR(name) \ protected: \ @@ -64,11 +54,6 @@ class TextSensor : public EntityBase, public EntityBase_DeviceClass { // ========== INTERNAL METHODS ========== // (In most use cases you won't need these) - /** Override this method to set the unique ID of this sensor. - * - * @deprecated Do not use for new sensors, a suitable unique ID is automatically generated (2023.4). - */ - virtual std::string unique_id(); void internal_send_state_to_frontend(const std::string &state); diff --git a/esphome/components/thermostat/climate.py b/esphome/components/thermostat/climate.py index 0314d877a3..a928d208f3 100644 --- a/esphome/components/thermostat/climate.py +++ b/esphome/components/thermostat/climate.py @@ -32,6 +32,7 @@ from esphome.const import ( CONF_FAN_WITH_COOLING, CONF_FAN_WITH_HEATING, CONF_HEAT_ACTION, + CONF_HEAT_COOL_MODE, CONF_HEAT_DEADBAND, CONF_HEAT_MODE, CONF_HEAT_OVERRUN, @@ -70,9 +71,14 @@ from esphome.const import ( CONF_VISUAL, ) -CONF_PRESET_CHANGE = "preset_change" CONF_DEFAULT_PRESET = "default_preset" +CONF_HUMIDITY_CONTROL_DEHUMIDIFY_ACTION = "humidity_control_dehumidify_action" +CONF_HUMIDITY_CONTROL_HUMIDIFY_ACTION = "humidity_control_humidify_action" +CONF_HUMIDITY_CONTROL_OFF_ACTION = "humidity_control_off_action" +CONF_HUMIDITY_HYSTERESIS = "humidity_hysteresis" CONF_ON_BOOT_RESTORE_FROM = "on_boot_restore_from" +CONF_PRESET_CHANGE = "preset_change" +CONF_TARGET_HUMIDITY_CHANGE_ACTION = "target_humidity_change_action" CODEOWNERS = ["@kbx81"] @@ -150,7 +156,7 @@ def generate_comparable_preset(config, name): def validate_thermostat(config): # verify corresponding action(s) exist(s) for any defined climate mode or action requirements = { - CONF_AUTO_MODE: [ + CONF_HEAT_COOL_MODE: [ CONF_COOL_ACTION, CONF_HEAT_ACTION, CONF_MIN_COOLING_OFF_TIME, @@ -240,6 +246,14 @@ def validate_thermostat(config): CONF_MAX_HEATING_RUN_TIME, CONF_SUPPLEMENTAL_HEATING_ACTION, ], + CONF_HUMIDITY_CONTROL_DEHUMIDIFY_ACTION: [ + CONF_HUMIDITY_CONTROL_OFF_ACTION, + CONF_HUMIDITY_SENSOR, + ], + CONF_HUMIDITY_CONTROL_HUMIDIFY_ACTION: [ + CONF_HUMIDITY_CONTROL_OFF_ACTION, + CONF_HUMIDITY_SENSOR, + ], } for config_trigger, req_triggers in requirements.items(): for req_trigger in req_triggers: @@ -337,7 +351,7 @@ def validate_thermostat(config): # Warn about using the removed CONF_DEFAULT_MODE and advise users if CONF_DEFAULT_MODE in config and config[CONF_DEFAULT_MODE] is not None: raise cv.Invalid( - f"{CONF_DEFAULT_MODE} is no longer valid. Please switch to using presets and specify a {CONF_DEFAULT_PRESET}." + f"{CONF_DEFAULT_MODE} is no longer valid. Please switch to using presets and specify a {CONF_DEFAULT_PRESET}" ) default_mode = config[CONF_DEFAULT_MODE] @@ -477,11 +491,11 @@ def validate_thermostat(config): if ( CONF_ON_BOOT_RESTORE_FROM in config and config[CONF_ON_BOOT_RESTORE_FROM] is OnBootRestoreFrom.DEFAULT_PRESET + and CONF_DEFAULT_PRESET not in config ): - if CONF_DEFAULT_PRESET not in config: - raise cv.Invalid( - f"{CONF_DEFAULT_PRESET} must be defined to use {CONF_ON_BOOT_RESTORE_FROM} in DEFAULT_PRESET mode" - ) + raise cv.Invalid( + f"{CONF_DEFAULT_PRESET} must be defined to use {CONF_ON_BOOT_RESTORE_FROM} in DEFAULT_PRESET mode" + ) if config[CONF_FAN_WITH_COOLING] is True and CONF_FAN_ONLY_ACTION not in config: raise cv.Invalid( @@ -540,6 +554,9 @@ CONFIG_SCHEMA = cv.All( cv.Optional(CONF_FAN_ONLY_MODE): automation.validate_automation( single=True ), + cv.Optional(CONF_HEAT_COOL_MODE): automation.validate_automation( + single=True + ), cv.Optional(CONF_HEAT_MODE): automation.validate_automation(single=True), cv.Optional(CONF_OFF_MODE): automation.validate_automation(single=True), cv.Optional(CONF_FAN_MODE_ON_ACTION): automation.validate_automation( @@ -584,9 +601,24 @@ CONFIG_SCHEMA = cv.All( cv.Optional(CONF_SWING_VERTICAL_ACTION): automation.validate_automation( single=True ), + cv.Optional( + CONF_TARGET_HUMIDITY_CHANGE_ACTION + ): automation.validate_automation(single=True), cv.Optional( CONF_TARGET_TEMPERATURE_CHANGE_ACTION ): automation.validate_automation(single=True), + cv.Exclusive( + CONF_HUMIDITY_CONTROL_DEHUMIDIFY_ACTION, + group_of_exclusion="humidity_control", + ): automation.validate_automation(single=True), + cv.Exclusive( + CONF_HUMIDITY_CONTROL_HUMIDIFY_ACTION, + group_of_exclusion="humidity_control", + ): automation.validate_automation(single=True), + cv.Optional( + CONF_HUMIDITY_CONTROL_OFF_ACTION + ): automation.validate_automation(single=True), + cv.Optional(CONF_HUMIDITY_HYSTERESIS, default=1.0): cv.percentage, cv.Optional(CONF_DEFAULT_MODE, default=None): cv.valid, cv.Optional(CONF_DEFAULT_PRESET): cv.templatable(cv.string), cv.Optional(CONF_DEFAULT_TARGET_TEMPERATURE_HIGH): cv.temperature, @@ -644,7 +676,6 @@ async def to_code(config): var = await climate.new_climate(config) await cg.register_component(var, config) - heat_cool_mode_available = CONF_HEAT_ACTION in config and CONF_COOL_ACTION in config two_points_available = CONF_HEAT_ACTION in config and ( CONF_COOL_ACTION in config or (config[CONF_FAN_ONLY_COOLING] and CONF_FAN_ONLY_ACTION in config) @@ -739,11 +770,6 @@ async def to_code(config): var.get_idle_action_trigger(), [], config[CONF_IDLE_ACTION] ) - if heat_cool_mode_available is True: - cg.add(var.set_supports_heat_cool(True)) - else: - cg.add(var.set_supports_heat_cool(False)) - if CONF_COOL_ACTION in config: await automation.build_automation( var.get_cool_action_trigger(), [], config[CONF_COOL_ACTION] @@ -780,6 +806,7 @@ async def to_code(config): await automation.build_automation( var.get_auto_mode_trigger(), [], config[CONF_AUTO_MODE] ) + cg.add(var.set_supports_auto(True)) if CONF_COOL_MODE in config: await automation.build_automation( var.get_cool_mode_trigger(), [], config[CONF_COOL_MODE] @@ -800,6 +827,11 @@ async def to_code(config): var.get_heat_mode_trigger(), [], config[CONF_HEAT_MODE] ) cg.add(var.set_supports_heat(True)) + if CONF_HEAT_COOL_MODE in config: + await automation.build_automation( + var.get_heat_cool_mode_trigger(), [], config[CONF_HEAT_COOL_MODE] + ) + cg.add(var.set_supports_heat_cool(True)) if CONF_OFF_MODE in config: await automation.build_automation( var.get_off_mode_trigger(), [], config[CONF_OFF_MODE] @@ -878,12 +910,39 @@ async def to_code(config): config[CONF_SWING_VERTICAL_ACTION], ) cg.add(var.set_supports_swing_mode_vertical(True)) + if CONF_TARGET_HUMIDITY_CHANGE_ACTION in config: + await automation.build_automation( + var.get_humidity_change_trigger(), + [], + config[CONF_TARGET_HUMIDITY_CHANGE_ACTION], + ) if CONF_TARGET_TEMPERATURE_CHANGE_ACTION in config: await automation.build_automation( var.get_temperature_change_trigger(), [], config[CONF_TARGET_TEMPERATURE_CHANGE_ACTION], ) + if CONF_HUMIDITY_CONTROL_DEHUMIDIFY_ACTION in config: + cg.add(var.set_supports_dehumidification(True)) + await automation.build_automation( + var.get_humidity_control_dehumidify_action_trigger(), + [], + config[CONF_HUMIDITY_CONTROL_DEHUMIDIFY_ACTION], + ) + if CONF_HUMIDITY_CONTROL_HUMIDIFY_ACTION in config: + cg.add(var.set_supports_humidification(True)) + await automation.build_automation( + var.get_humidity_control_humidify_action_trigger(), + [], + config[CONF_HUMIDITY_CONTROL_HUMIDIFY_ACTION], + ) + if CONF_HUMIDITY_CONTROL_OFF_ACTION in config: + await automation.build_automation( + var.get_humidity_control_off_action_trigger(), + [], + config[CONF_HUMIDITY_CONTROL_OFF_ACTION], + ) + cg.add(var.set_humidity_hysteresis(config[CONF_HUMIDITY_HYSTERESIS])) if CONF_PRESET in config: for preset_config in config[CONF_PRESET]: diff --git a/esphome/components/thermostat/thermostat_climate.cpp b/esphome/components/thermostat/thermostat_climate.cpp index 404e585aff..18efe3984e 100644 --- a/esphome/components/thermostat/thermostat_climate.cpp +++ b/esphome/components/thermostat/thermostat_climate.cpp @@ -1,4 +1,6 @@ #include "thermostat_climate.h" +#include "esphome/core/application.h" +#include "esphome/core/helpers.h" #include "esphome/core/log.h" namespace esphome { @@ -9,11 +11,11 @@ static const char *const TAG = "thermostat.climate"; void ThermostatClimate::setup() { if (this->use_startup_delay_) { // start timers so that no actions are called for a moment - this->start_timer_(thermostat::TIMER_COOLING_OFF); - this->start_timer_(thermostat::TIMER_FANNING_OFF); - this->start_timer_(thermostat::TIMER_HEATING_OFF); + this->start_timer_(thermostat::THERMOSTAT_TIMER_COOLING_OFF); + this->start_timer_(thermostat::THERMOSTAT_TIMER_FANNING_OFF); + this->start_timer_(thermostat::THERMOSTAT_TIMER_HEATING_OFF); if (this->supports_fan_only_action_uses_fan_mode_timer_) - this->start_timer_(thermostat::TIMER_FAN_MODE); + this->start_timer_(thermostat::THERMOSTAT_TIMER_FAN_MODE); } // add a callback so that whenever the sensor state changes we can take action this->sensor_->add_on_state_callback([this](float state) { @@ -30,6 +32,7 @@ void ThermostatClimate::setup() { if (this->humidity_sensor_ != nullptr) { this->humidity_sensor_->add_on_state_callback([this](float state) { this->current_humidity = state; + this->switch_to_humidity_control_action_(this->compute_humidity_control_action_()); this->publish_state(); }); this->current_humidity = this->humidity_sensor_->state; @@ -64,7 +67,7 @@ void ThermostatClimate::setup() { void ThermostatClimate::loop() { for (auto &timer : this->timer_) { - if (timer.active && (timer.started + timer.time < millis())) { + if (timer.active && (timer.started + timer.time < App.get_loop_component_start_time())) { timer.active = false; timer.func(); } @@ -82,6 +85,8 @@ void ThermostatClimate::refresh() { this->switch_to_supplemental_action_(this->compute_supplemental_action_()); this->switch_to_fan_mode_(this->fan_mode.value(), false); this->switch_to_swing_mode_(this->swing_mode, false); + this->switch_to_humidity_control_action_(this->compute_humidity_control_action_()); + this->check_humidity_change_trigger_(); this->check_temperature_change_trigger_(); this->publish_state(); } @@ -127,26 +132,40 @@ bool ThermostatClimate::hysteresis_valid() { return true; } +bool ThermostatClimate::humidity_hysteresis_valid() { + return !std::isnan(this->humidity_hysteresis_) && this->humidity_hysteresis_ >= 0.0f && + this->humidity_hysteresis_ < 100.0f; +} + +bool ThermostatClimate::limit_setpoints_for_heat_cool() { + return this->mode == climate::CLIMATE_MODE_HEAT_COOL || + (this->mode == climate::CLIMATE_MODE_AUTO && this->supports_heat_cool_); +} + void ThermostatClimate::validate_target_temperature() { if (std::isnan(this->target_temperature)) { + // default to the midpoint between visual min and max this->target_temperature = ((this->get_traits().get_visual_max_temperature() - this->get_traits().get_visual_min_temperature()) / 2) + this->get_traits().get_visual_min_temperature(); } else { // target_temperature must be between the visual minimum and the visual maximum - if (this->target_temperature < this->get_traits().get_visual_min_temperature()) - this->target_temperature = this->get_traits().get_visual_min_temperature(); - if (this->target_temperature > this->get_traits().get_visual_max_temperature()) - this->target_temperature = this->get_traits().get_visual_max_temperature(); + this->target_temperature = clamp(this->target_temperature, this->get_traits().get_visual_min_temperature(), + this->get_traits().get_visual_max_temperature()); } } -void ThermostatClimate::validate_target_temperatures() { - if (this->supports_two_points_) { +void ThermostatClimate::validate_target_temperatures(const bool pin_target_temperature_high) { + if (!this->supports_two_points_) { + this->validate_target_temperature(); + } else if (pin_target_temperature_high) { + // if target_temperature_high is set less than target_temperature_low, move down target_temperature_low this->validate_target_temperature_low(); this->validate_target_temperature_high(); } else { - this->validate_target_temperature(); + // if target_temperature_low is set greater than target_temperature_high, move up target_temperature_high + this->validate_target_temperature_high(); + this->validate_target_temperature_low(); } } @@ -154,18 +173,13 @@ void ThermostatClimate::validate_target_temperature_low() { if (std::isnan(this->target_temperature_low)) { this->target_temperature_low = this->get_traits().get_visual_min_temperature(); } else { - // target_temperature_low must not be lower than the visual minimum - if (this->target_temperature_low < this->get_traits().get_visual_min_temperature()) - this->target_temperature_low = this->get_traits().get_visual_min_temperature(); - // target_temperature_low must not be greater than the visual maximum minus set_point_minimum_differential_ - if (this->target_temperature_low > - this->get_traits().get_visual_max_temperature() - this->set_point_minimum_differential_) { - this->target_temperature_low = - this->get_traits().get_visual_max_temperature() - this->set_point_minimum_differential_; - } - // if target_temperature_low is set greater than target_temperature_high, move up target_temperature_high - if (this->target_temperature_low > this->target_temperature_high - this->set_point_minimum_differential_) - this->target_temperature_high = this->target_temperature_low + this->set_point_minimum_differential_; + float target_temperature_low_upper_limit = + this->limit_setpoints_for_heat_cool() + ? clamp(this->target_temperature_high - this->set_point_minimum_differential_, + this->get_traits().get_visual_min_temperature(), this->get_traits().get_visual_max_temperature()) + : this->get_traits().get_visual_max_temperature(); + this->target_temperature_low = clamp(this->target_temperature_low, this->get_traits().get_visual_min_temperature(), + target_temperature_low_upper_limit); } } @@ -173,111 +187,135 @@ void ThermostatClimate::validate_target_temperature_high() { if (std::isnan(this->target_temperature_high)) { this->target_temperature_high = this->get_traits().get_visual_max_temperature(); } else { - // target_temperature_high must not be lower than the visual maximum - if (this->target_temperature_high > this->get_traits().get_visual_max_temperature()) - this->target_temperature_high = this->get_traits().get_visual_max_temperature(); - // target_temperature_high must not be lower than the visual minimum plus set_point_minimum_differential_ - if (this->target_temperature_high < - this->get_traits().get_visual_min_temperature() + this->set_point_minimum_differential_) { - this->target_temperature_high = - this->get_traits().get_visual_min_temperature() + this->set_point_minimum_differential_; - } - // if target_temperature_high is set less than target_temperature_low, move down target_temperature_low - if (this->target_temperature_high < this->target_temperature_low + this->set_point_minimum_differential_) - this->target_temperature_low = this->target_temperature_high - this->set_point_minimum_differential_; + float target_temperature_high_lower_limit = + this->limit_setpoints_for_heat_cool() + ? clamp(this->target_temperature_low + this->set_point_minimum_differential_, + this->get_traits().get_visual_min_temperature(), this->get_traits().get_visual_max_temperature()) + : this->get_traits().get_visual_min_temperature(); + this->target_temperature_high = clamp(this->target_temperature_high, target_temperature_high_lower_limit, + this->get_traits().get_visual_max_temperature()); + } +} + +void ThermostatClimate::validate_target_humidity() { + if (std::isnan(this->target_humidity)) { + this->target_humidity = + (this->get_traits().get_visual_max_humidity() - this->get_traits().get_visual_min_humidity()) / 2.0f; + } else { + this->target_humidity = clamp(this->target_humidity, this->get_traits().get_visual_min_humidity(), + this->get_traits().get_visual_max_humidity()); } } void ThermostatClimate::control(const climate::ClimateCall &call) { + bool target_temperature_high_changed = false; + if (call.get_preset().has_value()) { // setup_complete_ blocks modifying/resetting the temps immediately after boot if (this->setup_complete_) { - this->change_preset_(*call.get_preset()); + this->change_preset_(call.get_preset().value()); } else { - this->preset = *call.get_preset(); + this->preset = call.get_preset().value(); } } if (call.get_custom_preset().has_value()) { // setup_complete_ blocks modifying/resetting the temps immediately after boot if (this->setup_complete_) { - this->change_custom_preset_(*call.get_custom_preset()); + this->change_custom_preset_(call.get_custom_preset().value()); } else { - this->custom_preset = *call.get_custom_preset(); + this->custom_preset = call.get_custom_preset().value(); } } - if (call.get_mode().has_value()) - this->mode = *call.get_mode(); - if (call.get_fan_mode().has_value()) - this->fan_mode = *call.get_fan_mode(); - if (call.get_swing_mode().has_value()) - this->swing_mode = *call.get_swing_mode(); + if (call.get_mode().has_value()) { + this->mode = call.get_mode().value(); + } + if (call.get_fan_mode().has_value()) { + this->fan_mode = call.get_fan_mode().value(); + } + if (call.get_swing_mode().has_value()) { + this->swing_mode = call.get_swing_mode().value(); + } if (this->supports_two_points_) { if (call.get_target_temperature_low().has_value()) { - this->target_temperature_low = *call.get_target_temperature_low(); - validate_target_temperature_low(); + this->target_temperature_low = call.get_target_temperature_low().value(); } if (call.get_target_temperature_high().has_value()) { - this->target_temperature_high = *call.get_target_temperature_high(); - validate_target_temperature_high(); + target_temperature_high_changed = this->target_temperature_high != call.get_target_temperature_high().value(); + this->target_temperature_high = call.get_target_temperature_high().value(); } + // ensure the two set points are valid and adjust one of them if necessary + this->validate_target_temperatures(target_temperature_high_changed || + (this->prev_mode_ == climate::CLIMATE_MODE_COOL)); } else { if (call.get_target_temperature().has_value()) { - this->target_temperature = *call.get_target_temperature(); - validate_target_temperature(); + this->target_temperature = call.get_target_temperature().value(); + this->validate_target_temperature(); } } + if (call.get_target_humidity().has_value()) { + this->target_humidity = call.get_target_humidity().value(); + this->validate_target_humidity(); + } // make any changes happen - refresh(); + this->refresh(); } climate::ClimateTraits ThermostatClimate::traits() { auto traits = climate::ClimateTraits(); - traits.set_supports_current_temperature(true); - if (this->humidity_sensor_ != nullptr) - traits.set_supports_current_humidity(true); - if (supports_auto_) + traits.add_feature_flags(climate::CLIMATE_SUPPORTS_ACTION | climate::CLIMATE_SUPPORTS_CURRENT_TEMPERATURE); + + if (this->supports_two_points_) + traits.add_feature_flags(climate::CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE); + + if (this->humidity_sensor_ != nullptr) + traits.add_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_HUMIDITY); + + if (this->supports_humidification_ || this->supports_dehumidification_) + traits.add_feature_flags(climate::CLIMATE_SUPPORTS_TARGET_HUMIDITY); + + if (this->supports_auto_) traits.add_supported_mode(climate::CLIMATE_MODE_AUTO); - if (supports_heat_cool_) + if (this->supports_heat_cool_) traits.add_supported_mode(climate::CLIMATE_MODE_HEAT_COOL); - if (supports_cool_) + if (this->supports_cool_) traits.add_supported_mode(climate::CLIMATE_MODE_COOL); - if (supports_dry_) + if (this->supports_dry_) traits.add_supported_mode(climate::CLIMATE_MODE_DRY); - if (supports_fan_only_) + if (this->supports_fan_only_) traits.add_supported_mode(climate::CLIMATE_MODE_FAN_ONLY); - if (supports_heat_) + if (this->supports_heat_) traits.add_supported_mode(climate::CLIMATE_MODE_HEAT); - if (supports_fan_mode_on_) + if (this->supports_fan_mode_on_) traits.add_supported_fan_mode(climate::CLIMATE_FAN_ON); - if (supports_fan_mode_off_) + if (this->supports_fan_mode_off_) traits.add_supported_fan_mode(climate::CLIMATE_FAN_OFF); - if (supports_fan_mode_auto_) + if (this->supports_fan_mode_auto_) traits.add_supported_fan_mode(climate::CLIMATE_FAN_AUTO); - if (supports_fan_mode_low_) + if (this->supports_fan_mode_low_) traits.add_supported_fan_mode(climate::CLIMATE_FAN_LOW); - if (supports_fan_mode_medium_) + if (this->supports_fan_mode_medium_) traits.add_supported_fan_mode(climate::CLIMATE_FAN_MEDIUM); - if (supports_fan_mode_high_) + if (this->supports_fan_mode_high_) traits.add_supported_fan_mode(climate::CLIMATE_FAN_HIGH); - if (supports_fan_mode_middle_) + if (this->supports_fan_mode_middle_) traits.add_supported_fan_mode(climate::CLIMATE_FAN_MIDDLE); - if (supports_fan_mode_focus_) + if (this->supports_fan_mode_focus_) traits.add_supported_fan_mode(climate::CLIMATE_FAN_FOCUS); - if (supports_fan_mode_diffuse_) + if (this->supports_fan_mode_diffuse_) traits.add_supported_fan_mode(climate::CLIMATE_FAN_DIFFUSE); - if (supports_fan_mode_quiet_) + if (this->supports_fan_mode_quiet_) traits.add_supported_fan_mode(climate::CLIMATE_FAN_QUIET); - if (supports_swing_mode_both_) + if (this->supports_swing_mode_both_) traits.add_supported_swing_mode(climate::CLIMATE_SWING_BOTH); - if (supports_swing_mode_horizontal_) + if (this->supports_swing_mode_horizontal_) traits.add_supported_swing_mode(climate::CLIMATE_SWING_HORIZONTAL); - if (supports_swing_mode_off_) + if (this->supports_swing_mode_off_) traits.add_supported_swing_mode(climate::CLIMATE_SWING_OFF); - if (supports_swing_mode_vertical_) + if (this->supports_swing_mode_vertical_) traits.add_supported_swing_mode(climate::CLIMATE_SWING_VERTICAL); for (auto &it : this->preset_config_) { @@ -286,9 +324,6 @@ climate::ClimateTraits ThermostatClimate::traits() { for (auto &it : this->custom_preset_config_) { traits.add_supported_custom_preset(it.first); } - - traits.set_supports_two_point_target_temperature(this->supports_two_points_); - traits.set_supports_action(true); return traits; } @@ -299,14 +334,15 @@ climate::ClimateAction ThermostatClimate::compute_action_(const bool ignore_time return climate::CLIMATE_ACTION_OFF; } // do not change the action if an "ON" timer is running - if ((!ignore_timers) && - (timer_active_(thermostat::TIMER_IDLE_ON) || timer_active_(thermostat::TIMER_COOLING_ON) || - timer_active_(thermostat::TIMER_FANNING_ON) || timer_active_(thermostat::TIMER_HEATING_ON))) { + if ((!ignore_timers) && (this->timer_active_(thermostat::THERMOSTAT_TIMER_IDLE_ON) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_COOLING_ON) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_FANNING_ON) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_HEATING_ON))) { return this->action; } // ensure set point(s) is/are valid before computing the action - this->validate_target_temperatures(); + this->validate_target_temperatures(this->prev_mode_ == climate::CLIMATE_MODE_COOL); // everything has been validated so we can now safely compute the action switch (this->mode) { // if the climate mode is OFF then the climate action must be OFF @@ -340,6 +376,22 @@ climate::ClimateAction ThermostatClimate::compute_action_(const bool ignore_time target_action = climate::CLIMATE_ACTION_HEATING; } break; + case climate::CLIMATE_MODE_AUTO: + if (this->supports_two_points_) { + if (this->cooling_required_() && this->heating_required_()) { + // this is bad and should never happen, so just stop. + // target_action = climate::CLIMATE_ACTION_IDLE; + } else if (this->cooling_required_()) { + target_action = climate::CLIMATE_ACTION_COOLING; + } else if (this->heating_required_()) { + target_action = climate::CLIMATE_ACTION_HEATING; + } + } else if (this->supports_cool_ && this->cooling_required_()) { + target_action = climate::CLIMATE_ACTION_COOLING; + } else if (this->supports_heat_ && this->heating_required_()) { + target_action = climate::CLIMATE_ACTION_HEATING; + } + break; default: break; } @@ -362,7 +414,7 @@ climate::ClimateAction ThermostatClimate::compute_supplemental_action_() { } // ensure set point(s) is/are valid before computing the action - this->validate_target_temperatures(); + this->validate_target_temperatures(this->prev_mode_ == climate::CLIMATE_MODE_COOL); // everything has been validated so we can now safely compute the action switch (this->mode) { // if the climate mode is OFF then the climate action must be OFF @@ -396,6 +448,28 @@ climate::ClimateAction ThermostatClimate::compute_supplemental_action_() { return target_action; } +HumidificationAction ThermostatClimate::compute_humidity_control_action_() { + auto target_action = THERMOSTAT_HUMIDITY_CONTROL_ACTION_OFF; + // if hysteresis value or current_humidity is not valid, we go to OFF + if (std::isnan(this->current_humidity) || !this->humidity_hysteresis_valid()) { + return THERMOSTAT_HUMIDITY_CONTROL_ACTION_OFF; + } + + // ensure set point is valid before computing the action + this->validate_target_humidity(); + // everything has been validated so we can now safely compute the action + if (this->dehumidification_required_() && this->humidification_required_()) { + // this is bad and should never happen, so just stop. + // target_action = THERMOSTAT_HUMIDITY_CONTROL_ACTION_OFF; + } else if (this->supports_dehumidification_ && this->dehumidification_required_()) { + target_action = THERMOSTAT_HUMIDITY_CONTROL_ACTION_DEHUMIDIFY; + } else if (this->supports_humidification_ && this->humidification_required_()) { + target_action = THERMOSTAT_HUMIDITY_CONTROL_ACTION_HUMIDIFY; + } + + return target_action; +} + void ThermostatClimate::switch_to_action_(climate::ClimateAction action, bool publish_state) { // setup_complete_ helps us ensure an action is called immediately after boot if ((action == this->action) && this->setup_complete_) { @@ -420,18 +494,18 @@ void ThermostatClimate::switch_to_action_(climate::ClimateAction action, bool pu case climate::CLIMATE_ACTION_OFF: case climate::CLIMATE_ACTION_IDLE: if (this->idle_action_ready_()) { - this->start_timer_(thermostat::TIMER_IDLE_ON); + this->start_timer_(thermostat::THERMOSTAT_TIMER_IDLE_ON); if (this->action == climate::CLIMATE_ACTION_COOLING) - this->start_timer_(thermostat::TIMER_COOLING_OFF); + this->start_timer_(thermostat::THERMOSTAT_TIMER_COOLING_OFF); if (this->action == climate::CLIMATE_ACTION_FAN) { if (this->supports_fan_only_action_uses_fan_mode_timer_) { - this->start_timer_(thermostat::TIMER_FAN_MODE); + this->start_timer_(thermostat::THERMOSTAT_TIMER_FAN_MODE); } else { - this->start_timer_(thermostat::TIMER_FANNING_OFF); + this->start_timer_(thermostat::THERMOSTAT_TIMER_FANNING_OFF); } } if (this->action == climate::CLIMATE_ACTION_HEATING) - this->start_timer_(thermostat::TIMER_HEATING_OFF); + this->start_timer_(thermostat::THERMOSTAT_TIMER_HEATING_OFF); // trig = this->idle_action_trigger_; ESP_LOGVV(TAG, "Switching to IDLE/OFF action"); this->cooling_max_runtime_exceeded_ = false; @@ -441,10 +515,10 @@ void ThermostatClimate::switch_to_action_(climate::ClimateAction action, bool pu break; case climate::CLIMATE_ACTION_COOLING: if (this->cooling_action_ready_()) { - this->start_timer_(thermostat::TIMER_COOLING_ON); - this->start_timer_(thermostat::TIMER_COOLING_MAX_RUN_TIME); + this->start_timer_(thermostat::THERMOSTAT_TIMER_COOLING_ON); + this->start_timer_(thermostat::THERMOSTAT_TIMER_COOLING_MAX_RUN_TIME); if (this->supports_fan_with_cooling_) { - this->start_timer_(thermostat::TIMER_FANNING_ON); + this->start_timer_(thermostat::THERMOSTAT_TIMER_FANNING_ON); trig_fan = this->fan_only_action_trigger_; } this->cooling_max_runtime_exceeded_ = false; @@ -455,10 +529,10 @@ void ThermostatClimate::switch_to_action_(climate::ClimateAction action, bool pu break; case climate::CLIMATE_ACTION_HEATING: if (this->heating_action_ready_()) { - this->start_timer_(thermostat::TIMER_HEATING_ON); - this->start_timer_(thermostat::TIMER_HEATING_MAX_RUN_TIME); + this->start_timer_(thermostat::THERMOSTAT_TIMER_HEATING_ON); + this->start_timer_(thermostat::THERMOSTAT_TIMER_HEATING_MAX_RUN_TIME); if (this->supports_fan_with_heating_) { - this->start_timer_(thermostat::TIMER_FANNING_ON); + this->start_timer_(thermostat::THERMOSTAT_TIMER_FANNING_ON); trig_fan = this->fan_only_action_trigger_; } this->heating_max_runtime_exceeded_ = false; @@ -470,9 +544,9 @@ void ThermostatClimate::switch_to_action_(climate::ClimateAction action, bool pu case climate::CLIMATE_ACTION_FAN: if (this->fanning_action_ready_()) { if (this->supports_fan_only_action_uses_fan_mode_timer_) { - this->start_timer_(thermostat::TIMER_FAN_MODE); + this->start_timer_(thermostat::THERMOSTAT_TIMER_FAN_MODE); } else { - this->start_timer_(thermostat::TIMER_FANNING_ON); + this->start_timer_(thermostat::THERMOSTAT_TIMER_FANNING_ON); } trig = this->fan_only_action_trigger_; ESP_LOGVV(TAG, "Switching to FAN_ONLY action"); @@ -481,8 +555,8 @@ void ThermostatClimate::switch_to_action_(climate::ClimateAction action, bool pu break; case climate::CLIMATE_ACTION_DRYING: if (this->drying_action_ready_()) { - this->start_timer_(thermostat::TIMER_COOLING_ON); - this->start_timer_(thermostat::TIMER_FANNING_ON); + this->start_timer_(thermostat::THERMOSTAT_TIMER_COOLING_ON); + this->start_timer_(thermostat::THERMOSTAT_TIMER_FANNING_ON); trig = this->dry_action_trigger_; ESP_LOGVV(TAG, "Switching to DRYING action"); action_ready = true; @@ -525,14 +599,14 @@ void ThermostatClimate::switch_to_supplemental_action_(climate::ClimateAction ac switch (action) { case climate::CLIMATE_ACTION_OFF: case climate::CLIMATE_ACTION_IDLE: - this->cancel_timer_(thermostat::TIMER_COOLING_MAX_RUN_TIME); - this->cancel_timer_(thermostat::TIMER_HEATING_MAX_RUN_TIME); + this->cancel_timer_(thermostat::THERMOSTAT_TIMER_COOLING_MAX_RUN_TIME); + this->cancel_timer_(thermostat::THERMOSTAT_TIMER_HEATING_MAX_RUN_TIME); break; case climate::CLIMATE_ACTION_COOLING: - this->cancel_timer_(thermostat::TIMER_COOLING_MAX_RUN_TIME); + this->cancel_timer_(thermostat::THERMOSTAT_TIMER_COOLING_MAX_RUN_TIME); break; case climate::CLIMATE_ACTION_HEATING: - this->cancel_timer_(thermostat::TIMER_HEATING_MAX_RUN_TIME); + this->cancel_timer_(thermostat::THERMOSTAT_TIMER_HEATING_MAX_RUN_TIME); break; default: return; @@ -547,15 +621,15 @@ void ThermostatClimate::trigger_supplemental_action_() { switch (this->supplemental_action_) { case climate::CLIMATE_ACTION_COOLING: - if (!this->timer_active_(thermostat::TIMER_COOLING_MAX_RUN_TIME)) { - this->start_timer_(thermostat::TIMER_COOLING_MAX_RUN_TIME); + if (!this->timer_active_(thermostat::THERMOSTAT_TIMER_COOLING_MAX_RUN_TIME)) { + this->start_timer_(thermostat::THERMOSTAT_TIMER_COOLING_MAX_RUN_TIME); } trig = this->supplemental_cool_action_trigger_; ESP_LOGVV(TAG, "Calling supplemental COOLING action"); break; case climate::CLIMATE_ACTION_HEATING: - if (!this->timer_active_(thermostat::TIMER_HEATING_MAX_RUN_TIME)) { - this->start_timer_(thermostat::TIMER_HEATING_MAX_RUN_TIME); + if (!this->timer_active_(thermostat::THERMOSTAT_TIMER_HEATING_MAX_RUN_TIME)) { + this->start_timer_(thermostat::THERMOSTAT_TIMER_HEATING_MAX_RUN_TIME); } trig = this->supplemental_heat_action_trigger_; ESP_LOGVV(TAG, "Calling supplemental HEATING action"); @@ -569,6 +643,44 @@ void ThermostatClimate::trigger_supplemental_action_() { } } +void ThermostatClimate::switch_to_humidity_control_action_(HumidificationAction action) { + // setup_complete_ helps us ensure an action is called immediately after boot + if ((action == this->humidification_action_) && this->setup_complete_) { + // already in target mode + return; + } + + Trigger<> *trig = this->humidity_control_off_action_trigger_; + switch (action) { + case THERMOSTAT_HUMIDITY_CONTROL_ACTION_OFF: + // trig = this->humidity_control_off_action_trigger_; + ESP_LOGVV(TAG, "Switching to HUMIDIFICATION_OFF action"); + break; + case THERMOSTAT_HUMIDITY_CONTROL_ACTION_DEHUMIDIFY: + trig = this->humidity_control_dehumidify_action_trigger_; + ESP_LOGVV(TAG, "Switching to DEHUMIDIFY action"); + break; + case THERMOSTAT_HUMIDITY_CONTROL_ACTION_HUMIDIFY: + trig = this->humidity_control_humidify_action_trigger_; + ESP_LOGVV(TAG, "Switching to HUMIDIFY action"); + break; + case THERMOSTAT_HUMIDITY_CONTROL_ACTION_NONE: + default: + action = THERMOSTAT_HUMIDITY_CONTROL_ACTION_OFF; + // trig = this->humidity_control_off_action_trigger_; + } + + if (this->prev_humidity_control_trigger_ != nullptr) { + this->prev_humidity_control_trigger_->stop_action(); + this->prev_humidity_control_trigger_ = nullptr; + } + this->humidification_action_ = action; + this->prev_humidity_control_trigger_ = trig; + if (trig != nullptr) { + trig->trigger(); + } +} + void ThermostatClimate::switch_to_fan_mode_(climate::ClimateFanMode fan_mode, bool publish_state) { // setup_complete_ helps us ensure an action is called immediately after boot if ((fan_mode == this->prev_fan_mode_) && this->setup_complete_) { @@ -633,7 +745,7 @@ void ThermostatClimate::switch_to_fan_mode_(climate::ClimateFanMode fan_mode, bo this->prev_fan_mode_trigger_->stop_action(); this->prev_fan_mode_trigger_ = nullptr; } - this->start_timer_(thermostat::TIMER_FAN_MODE); + this->start_timer_(thermostat::THERMOSTAT_TIMER_FAN_MODE); if (trig != nullptr) { trig->trigger(); } @@ -653,13 +765,13 @@ void ThermostatClimate::switch_to_mode_(climate::ClimateMode mode, bool publish_ this->prev_mode_trigger_->stop_action(); this->prev_mode_trigger_ = nullptr; } - Trigger<> *trig = this->auto_mode_trigger_; + Trigger<> *trig = this->off_mode_trigger_; switch (mode) { - case climate::CLIMATE_MODE_OFF: - trig = this->off_mode_trigger_; + case climate::CLIMATE_MODE_AUTO: + trig = this->auto_mode_trigger_; break; case climate::CLIMATE_MODE_HEAT_COOL: - // trig = this->auto_mode_trigger_; + trig = this->heat_cool_mode_trigger_; break; case climate::CLIMATE_MODE_COOL: trig = this->cool_mode_trigger_; @@ -673,11 +785,12 @@ void ThermostatClimate::switch_to_mode_(climate::ClimateMode mode, bool publish_ case climate::CLIMATE_MODE_DRY: trig = this->dry_mode_trigger_; break; + case climate::CLIMATE_MODE_OFF: default: // we cannot report an invalid mode back to HA (even if it asked for one) // and must assume some valid value - mode = climate::CLIMATE_MODE_HEAT_COOL; - // trig = this->auto_mode_trigger_; + mode = climate::CLIMATE_MODE_OFF; + // trig = this->off_mode_trigger_; } if (trig != nullptr) { trig->trigger(); @@ -685,8 +798,9 @@ void ThermostatClimate::switch_to_mode_(climate::ClimateMode mode, bool publish_ this->mode = mode; this->prev_mode_ = mode; this->prev_mode_trigger_ = trig; - if (publish_state) + if (publish_state) { this->publish_state(); + } } void ThermostatClimate::switch_to_swing_mode_(climate::ClimateSwingMode swing_mode, bool publish_state) { @@ -732,35 +846,44 @@ void ThermostatClimate::switch_to_swing_mode_(climate::ClimateSwingMode swing_mo bool ThermostatClimate::idle_action_ready_() { if (this->supports_fan_only_action_uses_fan_mode_timer_) { - return !(this->timer_active_(thermostat::TIMER_COOLING_ON) || this->timer_active_(thermostat::TIMER_FAN_MODE) || - this->timer_active_(thermostat::TIMER_HEATING_ON)); + return !(this->timer_active_(thermostat::THERMOSTAT_TIMER_COOLING_ON) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_FAN_MODE) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_HEATING_ON)); } - return !(this->timer_active_(thermostat::TIMER_COOLING_ON) || this->timer_active_(thermostat::TIMER_FANNING_ON) || - this->timer_active_(thermostat::TIMER_HEATING_ON)); + return !(this->timer_active_(thermostat::THERMOSTAT_TIMER_COOLING_ON) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_FANNING_ON) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_HEATING_ON)); } bool ThermostatClimate::cooling_action_ready_() { - return !(this->timer_active_(thermostat::TIMER_IDLE_ON) || this->timer_active_(thermostat::TIMER_FANNING_OFF) || - this->timer_active_(thermostat::TIMER_COOLING_OFF) || this->timer_active_(thermostat::TIMER_HEATING_ON)); + return !(this->timer_active_(thermostat::THERMOSTAT_TIMER_IDLE_ON) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_FANNING_OFF) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_COOLING_OFF) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_HEATING_ON)); } bool ThermostatClimate::drying_action_ready_() { - return !(this->timer_active_(thermostat::TIMER_IDLE_ON) || this->timer_active_(thermostat::TIMER_FANNING_OFF) || - this->timer_active_(thermostat::TIMER_COOLING_OFF) || this->timer_active_(thermostat::TIMER_HEATING_ON)); + return !(this->timer_active_(thermostat::THERMOSTAT_TIMER_IDLE_ON) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_FANNING_OFF) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_COOLING_OFF) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_HEATING_ON)); } -bool ThermostatClimate::fan_mode_ready_() { return !(this->timer_active_(thermostat::TIMER_FAN_MODE)); } +bool ThermostatClimate::fan_mode_ready_() { return !(this->timer_active_(thermostat::THERMOSTAT_TIMER_FAN_MODE)); } bool ThermostatClimate::fanning_action_ready_() { if (this->supports_fan_only_action_uses_fan_mode_timer_) { - return !(this->timer_active_(thermostat::TIMER_FAN_MODE)); + return !(this->timer_active_(thermostat::THERMOSTAT_TIMER_FAN_MODE)); } - return !(this->timer_active_(thermostat::TIMER_IDLE_ON) || this->timer_active_(thermostat::TIMER_FANNING_OFF)); + return !(this->timer_active_(thermostat::THERMOSTAT_TIMER_IDLE_ON) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_FANNING_OFF)); } bool ThermostatClimate::heating_action_ready_() { - return !(this->timer_active_(thermostat::TIMER_IDLE_ON) || this->timer_active_(thermostat::TIMER_COOLING_ON) || - this->timer_active_(thermostat::TIMER_FANNING_OFF) || this->timer_active_(thermostat::TIMER_HEATING_OFF)); + return !(this->timer_active_(thermostat::THERMOSTAT_TIMER_IDLE_ON) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_COOLING_ON) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_FANNING_OFF) || + this->timer_active_(thermostat::THERMOSTAT_TIMER_HEATING_OFF)); } void ThermostatClimate::start_timer_(const ThermostatClimateTimerIndex timer_index) { @@ -849,6 +972,20 @@ void ThermostatClimate::idle_on_timer_callback_() { this->switch_to_supplemental_action_(this->compute_supplemental_action_()); } +void ThermostatClimate::check_humidity_change_trigger_() { + if ((this->prev_target_humidity_ == this->target_humidity) && this->setup_complete_) { + return; // nothing changed, no reason to trigger + } else { + // save the new temperature so we can check it again later; the trigger will fire below + this->prev_target_humidity_ = this->target_humidity; + } + // trigger the action + Trigger<> *trig = this->humidity_change_trigger_; + if (trig != nullptr) { + trig->trigger(); + } +} + void ThermostatClimate::check_temperature_change_trigger_() { if (this->supports_two_points_) { // setup_complete_ helps us ensure an action is called immediately after boot @@ -958,37 +1095,51 @@ bool ThermostatClimate::supplemental_heating_required_() { (this->supplemental_action_ == climate::CLIMATE_ACTION_HEATING)); } -void ThermostatClimate::dump_preset_config_(const char *preset_name, const ThermostatClimateTargetTempConfig &config, - bool is_default_preset) { - ESP_LOGCONFIG(TAG, " %s Is Default: %s", preset_name, YESNO(is_default_preset)); +bool ThermostatClimate::dehumidification_required_() { + if (this->current_humidity > this->target_humidity + this->humidity_hysteresis_) { + // if the current humidity exceeds the target + hysteresis, dehumidification is required + return true; + } else if (this->current_humidity < this->target_humidity - this->humidity_hysteresis_) { + // if the current humidity is less than the target - hysteresis, dehumidification should stop + return false; + } + // if we get here, the current humidity is between target + hysteresis and target - hysteresis, + // so the action should not change + return this->humidification_action_ == THERMOSTAT_HUMIDITY_CONTROL_ACTION_DEHUMIDIFY; +} +bool ThermostatClimate::humidification_required_() { + if (this->current_humidity < this->target_humidity - this->humidity_hysteresis_) { + // if the current humidity is below the target - hysteresis, humidification is required + return true; + } else if (this->current_humidity > this->target_humidity + this->humidity_hysteresis_) { + // if the current humidity is above the target + hysteresis, humidification should stop + return false; + } + // if we get here, the current humidity is between target - hysteresis and target + hysteresis, + // so the action should not change + return this->humidification_action_ == THERMOSTAT_HUMIDITY_CONTROL_ACTION_HUMIDIFY; +} + +void ThermostatClimate::dump_preset_config_(const char *preset_name, const ThermostatClimateTargetTempConfig &config) { if (this->supports_heat_) { - if (this->supports_two_points_) { - ESP_LOGCONFIG(TAG, " %s Default Target Temperature Low: %.1f°C", preset_name, - config.default_temperature_low); - } else { - ESP_LOGCONFIG(TAG, " %s Default Target Temperature Low: %.1f°C", preset_name, config.default_temperature); - } + ESP_LOGCONFIG(TAG, " Default Target Temperature Low: %.1f°C", + this->supports_two_points_ ? config.default_temperature_low : config.default_temperature); } if ((this->supports_cool_) || (this->supports_fan_only_)) { - if (this->supports_two_points_) { - ESP_LOGCONFIG(TAG, " %s Default Target Temperature High: %.1f°C", preset_name, - config.default_temperature_high); - } else { - ESP_LOGCONFIG(TAG, " %s Default Target Temperature High: %.1f°C", preset_name, config.default_temperature); - } + ESP_LOGCONFIG(TAG, " Default Target Temperature High: %.1f°C", + this->supports_two_points_ ? config.default_temperature_high : config.default_temperature); } if (config.mode_.has_value()) { - ESP_LOGCONFIG(TAG, " %s Default Mode: %s", preset_name, - LOG_STR_ARG(climate::climate_mode_to_string(*config.mode_))); + ESP_LOGCONFIG(TAG, " Default Mode: %s", LOG_STR_ARG(climate::climate_mode_to_string(*config.mode_))); } if (config.fan_mode_.has_value()) { - ESP_LOGCONFIG(TAG, " %s Default Fan Mode: %s", preset_name, + ESP_LOGCONFIG(TAG, " Default Fan Mode: %s", LOG_STR_ARG(climate::climate_fan_mode_to_string(*config.fan_mode_))); } if (config.swing_mode_.has_value()) { - ESP_LOGCONFIG(TAG, " %s Default Swing Mode: %s", preset_name, + ESP_LOGCONFIG(TAG, " Default Swing Mode: %s", LOG_STR_ARG(climate::climate_swing_mode_to_string(*config.swing_mode_))); } } @@ -1106,6 +1257,7 @@ ThermostatClimate::ThermostatClimate() heat_action_trigger_(new Trigger<>()), supplemental_heat_action_trigger_(new Trigger<>()), heat_mode_trigger_(new Trigger<>()), + heat_cool_mode_trigger_(new Trigger<>()), auto_mode_trigger_(new Trigger<>()), idle_action_trigger_(new Trigger<>()), off_mode_trigger_(new Trigger<>()), @@ -1125,8 +1277,12 @@ ThermostatClimate::ThermostatClimate() swing_mode_off_trigger_(new Trigger<>()), swing_mode_horizontal_trigger_(new Trigger<>()), swing_mode_vertical_trigger_(new Trigger<>()), + humidity_change_trigger_(new Trigger<>()), temperature_change_trigger_(new Trigger<>()), - preset_change_trigger_(new Trigger<>()) {} + preset_change_trigger_(new Trigger<>()), + humidity_control_dehumidify_action_trigger_(new Trigger<>()), + humidity_control_humidify_action_trigger_(new Trigger<>()), + humidity_control_off_action_trigger_(new Trigger<>()) {} void ThermostatClimate::set_default_preset(const std::string &custom_preset) { this->default_custom_preset_ = custom_preset; @@ -1147,49 +1303,52 @@ void ThermostatClimate::set_heat_overrun(float overrun) { this->heating_overrun_ void ThermostatClimate::set_supplemental_cool_delta(float delta) { this->supplemental_cool_delta_ = delta; } void ThermostatClimate::set_supplemental_heat_delta(float delta) { this->supplemental_heat_delta_ = delta; } void ThermostatClimate::set_cooling_maximum_run_time_in_sec(uint32_t time) { - this->timer_[thermostat::TIMER_COOLING_MAX_RUN_TIME].time = + this->timer_[thermostat::THERMOSTAT_TIMER_COOLING_MAX_RUN_TIME].time = 1000 * (time < this->min_timer_duration_ ? this->min_timer_duration_ : time); } void ThermostatClimate::set_cooling_minimum_off_time_in_sec(uint32_t time) { - this->timer_[thermostat::TIMER_COOLING_OFF].time = + this->timer_[thermostat::THERMOSTAT_TIMER_COOLING_OFF].time = 1000 * (time < this->min_timer_duration_ ? this->min_timer_duration_ : time); } void ThermostatClimate::set_cooling_minimum_run_time_in_sec(uint32_t time) { - this->timer_[thermostat::TIMER_COOLING_ON].time = + this->timer_[thermostat::THERMOSTAT_TIMER_COOLING_ON].time = 1000 * (time < this->min_timer_duration_ ? this->min_timer_duration_ : time); } void ThermostatClimate::set_fan_mode_minimum_switching_time_in_sec(uint32_t time) { - this->timer_[thermostat::TIMER_FAN_MODE].time = + this->timer_[thermostat::THERMOSTAT_TIMER_FAN_MODE].time = 1000 * (time < this->min_timer_duration_ ? this->min_timer_duration_ : time); } void ThermostatClimate::set_fanning_minimum_off_time_in_sec(uint32_t time) { - this->timer_[thermostat::TIMER_FANNING_OFF].time = + this->timer_[thermostat::THERMOSTAT_TIMER_FANNING_OFF].time = 1000 * (time < this->min_timer_duration_ ? this->min_timer_duration_ : time); } void ThermostatClimate::set_fanning_minimum_run_time_in_sec(uint32_t time) { - this->timer_[thermostat::TIMER_FANNING_ON].time = + this->timer_[thermostat::THERMOSTAT_TIMER_FANNING_ON].time = 1000 * (time < this->min_timer_duration_ ? this->min_timer_duration_ : time); } void ThermostatClimate::set_heating_maximum_run_time_in_sec(uint32_t time) { - this->timer_[thermostat::TIMER_HEATING_MAX_RUN_TIME].time = + this->timer_[thermostat::THERMOSTAT_TIMER_HEATING_MAX_RUN_TIME].time = 1000 * (time < this->min_timer_duration_ ? this->min_timer_duration_ : time); } void ThermostatClimate::set_heating_minimum_off_time_in_sec(uint32_t time) { - this->timer_[thermostat::TIMER_HEATING_OFF].time = + this->timer_[thermostat::THERMOSTAT_TIMER_HEATING_OFF].time = 1000 * (time < this->min_timer_duration_ ? this->min_timer_duration_ : time); } void ThermostatClimate::set_heating_minimum_run_time_in_sec(uint32_t time) { - this->timer_[thermostat::TIMER_HEATING_ON].time = + this->timer_[thermostat::THERMOSTAT_TIMER_HEATING_ON].time = 1000 * (time < this->min_timer_duration_ ? this->min_timer_duration_ : time); } void ThermostatClimate::set_idle_minimum_time_in_sec(uint32_t time) { - this->timer_[thermostat::TIMER_IDLE_ON].time = + this->timer_[thermostat::THERMOSTAT_TIMER_IDLE_ON].time = 1000 * (time < this->min_timer_duration_ ? this->min_timer_duration_ : time); } void ThermostatClimate::set_sensor(sensor::Sensor *sensor) { this->sensor_ = sensor; } void ThermostatClimate::set_humidity_sensor(sensor::Sensor *humidity_sensor) { this->humidity_sensor_ = humidity_sensor; } +void ThermostatClimate::set_humidity_hysteresis(float humidity_hysteresis) { + this->humidity_hysteresis_ = std::clamp(humidity_hysteresis, 0.0f, 100.0f); +} void ThermostatClimate::set_use_startup_delay(bool use_startup_delay) { this->use_startup_delay_ = use_startup_delay; } void ThermostatClimate::set_supports_heat_cool(bool supports_heat_cool) { this->supports_heat_cool_ = supports_heat_cool; @@ -1257,6 +1416,18 @@ void ThermostatClimate::set_supports_swing_mode_vertical(bool supports_swing_mod void ThermostatClimate::set_supports_two_points(bool supports_two_points) { this->supports_two_points_ = supports_two_points; } +void ThermostatClimate::set_supports_dehumidification(bool supports_dehumidification) { + this->supports_dehumidification_ = supports_dehumidification; + if (supports_dehumidification) { + this->supports_humidification_ = false; + } +} +void ThermostatClimate::set_supports_humidification(bool supports_humidification) { + this->supports_humidification_ = supports_humidification; + if (supports_humidification) { + this->supports_dehumidification_ = false; + } +} Trigger<> *ThermostatClimate::get_cool_action_trigger() const { return this->cool_action_trigger_; } Trigger<> *ThermostatClimate::get_supplemental_cool_action_trigger() const { @@ -1274,6 +1445,7 @@ Trigger<> *ThermostatClimate::get_cool_mode_trigger() const { return this->cool_ Trigger<> *ThermostatClimate::get_dry_mode_trigger() const { return this->dry_mode_trigger_; } Trigger<> *ThermostatClimate::get_fan_only_mode_trigger() const { return this->fan_only_mode_trigger_; } Trigger<> *ThermostatClimate::get_heat_mode_trigger() const { return this->heat_mode_trigger_; } +Trigger<> *ThermostatClimate::get_heat_cool_mode_trigger() const { return this->heat_cool_mode_trigger_; } Trigger<> *ThermostatClimate::get_off_mode_trigger() const { return this->off_mode_trigger_; } Trigger<> *ThermostatClimate::get_fan_mode_on_trigger() const { return this->fan_mode_on_trigger_; } Trigger<> *ThermostatClimate::get_fan_mode_off_trigger() const { return this->fan_mode_off_trigger_; } @@ -1289,70 +1461,85 @@ Trigger<> *ThermostatClimate::get_swing_mode_both_trigger() const { return this- Trigger<> *ThermostatClimate::get_swing_mode_off_trigger() const { return this->swing_mode_off_trigger_; } Trigger<> *ThermostatClimate::get_swing_mode_horizontal_trigger() const { return this->swing_mode_horizontal_trigger_; } Trigger<> *ThermostatClimate::get_swing_mode_vertical_trigger() const { return this->swing_mode_vertical_trigger_; } +Trigger<> *ThermostatClimate::get_humidity_change_trigger() const { return this->humidity_change_trigger_; } Trigger<> *ThermostatClimate::get_temperature_change_trigger() const { return this->temperature_change_trigger_; } Trigger<> *ThermostatClimate::get_preset_change_trigger() const { return this->preset_change_trigger_; } +Trigger<> *ThermostatClimate::get_humidity_control_dehumidify_action_trigger() const { + return this->humidity_control_dehumidify_action_trigger_; +} +Trigger<> *ThermostatClimate::get_humidity_control_humidify_action_trigger() const { + return this->humidity_control_humidify_action_trigger_; +} +Trigger<> *ThermostatClimate::get_humidity_control_off_action_trigger() const { + return this->humidity_control_off_action_trigger_; +} void ThermostatClimate::dump_config() { LOG_CLIMATE("", "Thermostat", this); + ESP_LOGCONFIG(TAG, + " On boot, restore from: %s\n" + " Use Start-up Delay: %s", + this->on_boot_restore_from_ == thermostat::DEFAULT_PRESET ? "DEFAULT_PRESET" : "MEMORY", + YESNO(this->use_startup_delay_)); if (this->supports_two_points_) { ESP_LOGCONFIG(TAG, " Minimum Set Point Differential: %.1f°C", this->set_point_minimum_differential_); } - ESP_LOGCONFIG(TAG, " Use Start-up Delay: %s", YESNO(this->use_startup_delay_)); if (this->supports_cool_) { ESP_LOGCONFIG(TAG, " Cooling Parameters:\n" " Deadband: %.1f°C\n" - " Overrun: %.1f°C", - this->cooling_deadband_, this->cooling_overrun_); - if ((this->supplemental_cool_delta_ > 0) || (this->timer_duration_(thermostat::TIMER_COOLING_MAX_RUN_TIME) > 0)) { - ESP_LOGCONFIG(TAG, - " Supplemental Delta: %.1f°C\n" - " Maximum Run Time: %" PRIu32 "s", - this->supplemental_cool_delta_, - this->timer_duration_(thermostat::TIMER_COOLING_MAX_RUN_TIME) / 1000); - } - ESP_LOGCONFIG(TAG, + " Overrun: %.1f°C\n" " Minimum Off Time: %" PRIu32 "s\n" " Minimum Run Time: %" PRIu32 "s", - this->timer_duration_(thermostat::TIMER_COOLING_OFF) / 1000, - this->timer_duration_(thermostat::TIMER_COOLING_ON) / 1000); + this->cooling_deadband_, this->cooling_overrun_, + this->timer_duration_(thermostat::THERMOSTAT_TIMER_COOLING_OFF) / 1000, + this->timer_duration_(thermostat::THERMOSTAT_TIMER_COOLING_ON) / 1000); + if ((this->supplemental_cool_delta_ > 0) || + (this->timer_duration_(thermostat::THERMOSTAT_TIMER_COOLING_MAX_RUN_TIME) > 0)) { + ESP_LOGCONFIG(TAG, + " Maximum Run Time: %" PRIu32 "s\n" + " Supplemental Delta: %.1f°C", + this->timer_duration_(thermostat::THERMOSTAT_TIMER_COOLING_MAX_RUN_TIME) / 1000, + this->supplemental_cool_delta_); + } } if (this->supports_heat_) { ESP_LOGCONFIG(TAG, " Heating Parameters:\n" " Deadband: %.1f°C\n" - " Overrun: %.1f°C", - this->heating_deadband_, this->heating_overrun_); - if ((this->supplemental_heat_delta_ > 0) || (this->timer_duration_(thermostat::TIMER_HEATING_MAX_RUN_TIME) > 0)) { - ESP_LOGCONFIG(TAG, - " Supplemental Delta: %.1f°C\n" - " Maximum Run Time: %" PRIu32 "s", - this->supplemental_heat_delta_, - this->timer_duration_(thermostat::TIMER_HEATING_MAX_RUN_TIME) / 1000); - } - ESP_LOGCONFIG(TAG, + " Overrun: %.1f°C\n" " Minimum Off Time: %" PRIu32 "s\n" " Minimum Run Time: %" PRIu32 "s", - this->timer_duration_(thermostat::TIMER_HEATING_OFF) / 1000, - this->timer_duration_(thermostat::TIMER_HEATING_ON) / 1000); + this->heating_deadband_, this->heating_overrun_, + this->timer_duration_(thermostat::THERMOSTAT_TIMER_HEATING_OFF) / 1000, + this->timer_duration_(thermostat::THERMOSTAT_TIMER_HEATING_ON) / 1000); + if ((this->supplemental_heat_delta_ > 0) || + (this->timer_duration_(thermostat::THERMOSTAT_TIMER_HEATING_MAX_RUN_TIME) > 0)) { + ESP_LOGCONFIG(TAG, + " Maximum Run Time: %" PRIu32 "s\n" + " Supplemental Delta: %.1f°C", + this->timer_duration_(thermostat::THERMOSTAT_TIMER_HEATING_MAX_RUN_TIME) / 1000, + this->supplemental_heat_delta_); + } } if (this->supports_fan_only_) { ESP_LOGCONFIG(TAG, - " Fanning Minimum Off Time: %" PRIu32 "s\n" - " Fanning Minimum Run Time: %" PRIu32 "s", - this->timer_duration_(thermostat::TIMER_FANNING_OFF) / 1000, - this->timer_duration_(thermostat::TIMER_FANNING_ON) / 1000); + " Fan Parameters:\n" + " Minimum Off Time: %" PRIu32 "s\n" + " Minimum Run Time: %" PRIu32 "s", + this->timer_duration_(thermostat::THERMOSTAT_TIMER_FANNING_OFF) / 1000, + this->timer_duration_(thermostat::THERMOSTAT_TIMER_FANNING_ON) / 1000); } if (this->supports_fan_mode_on_ || this->supports_fan_mode_off_ || this->supports_fan_mode_auto_ || this->supports_fan_mode_low_ || this->supports_fan_mode_medium_ || this->supports_fan_mode_high_ || this->supports_fan_mode_middle_ || this->supports_fan_mode_focus_ || this->supports_fan_mode_diffuse_ || this->supports_fan_mode_quiet_) { ESP_LOGCONFIG(TAG, " Minimum Fan Mode Switching Time: %" PRIu32 "s", - this->timer_duration_(thermostat::TIMER_FAN_MODE) / 1000); + this->timer_duration_(thermostat::THERMOSTAT_TIMER_FAN_MODE) / 1000); } - ESP_LOGCONFIG(TAG, " Minimum Idle Time: %" PRIu32 "s", this->timer_[thermostat::TIMER_IDLE_ON].time / 1000); ESP_LOGCONFIG(TAG, + " Minimum Idle Time: %" PRIu32 "s\n" " Supported MODES:\n" " AUTO: %s\n" " HEAT/COOL: %s\n" @@ -1362,8 +1549,9 @@ void ThermostatClimate::dump_config() { " FAN_ONLY: %s\n" " FAN_ONLY_ACTION_USES_FAN_MODE_TIMER: %s\n" " FAN_ONLY_COOLING: %s", - YESNO(this->supports_auto_), YESNO(this->supports_heat_cool_), YESNO(this->supports_heat_), - YESNO(this->supports_cool_), YESNO(this->supports_dry_), YESNO(this->supports_fan_only_), + this->timer_[thermostat::THERMOSTAT_TIMER_IDLE_ON].time / 1000, YESNO(this->supports_auto_), + YESNO(this->supports_heat_cool_), YESNO(this->supports_heat_), YESNO(this->supports_cool_), + YESNO(this->supports_dry_), YESNO(this->supports_fan_only_), YESNO(this->supports_fan_only_action_uses_fan_mode_timer_), YESNO(this->supports_fan_only_cooling_)); if (this->supports_cool_) { ESP_LOGCONFIG(TAG, " FAN_WITH_COOLING: %s", YESNO(this->supports_fan_with_cooling_)); @@ -1382,40 +1570,47 @@ void ThermostatClimate::dump_config() { " MIDDLE: %s\n" " FOCUS: %s\n" " DIFFUSE: %s\n" - " QUIET: %s", - YESNO(this->supports_fan_mode_on_), YESNO(this->supports_fan_mode_off_), - YESNO(this->supports_fan_mode_auto_), YESNO(this->supports_fan_mode_low_), - YESNO(this->supports_fan_mode_medium_), YESNO(this->supports_fan_mode_high_), - YESNO(this->supports_fan_mode_middle_), YESNO(this->supports_fan_mode_focus_), - YESNO(this->supports_fan_mode_diffuse_), YESNO(this->supports_fan_mode_quiet_)); - ESP_LOGCONFIG(TAG, + " QUIET: %s\n" " Supported SWING MODES:\n" " BOTH: %s\n" " OFF: %s\n" " HORIZONTAL: %s\n" " VERTICAL: %s\n" - " Supports TWO SET POINTS: %s", + " Supports TWO SET POINTS: %s\n" + " Supported Humidity Parameters:\n" + " CURRENT: %s\n" + " TARGET: %s\n" + " DEHUMIDIFICATION: %s\n" + " HUMIDIFICATION: %s", + YESNO(this->supports_fan_mode_on_), YESNO(this->supports_fan_mode_off_), + YESNO(this->supports_fan_mode_auto_), YESNO(this->supports_fan_mode_low_), + YESNO(this->supports_fan_mode_medium_), YESNO(this->supports_fan_mode_high_), + YESNO(this->supports_fan_mode_middle_), YESNO(this->supports_fan_mode_focus_), + YESNO(this->supports_fan_mode_diffuse_), YESNO(this->supports_fan_mode_quiet_), YESNO(this->supports_swing_mode_both_), YESNO(this->supports_swing_mode_off_), YESNO(this->supports_swing_mode_horizontal_), YESNO(this->supports_swing_mode_vertical_), - YESNO(this->supports_two_points_)); + YESNO(this->supports_two_points_), + YESNO(this->get_traits().has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_HUMIDITY)), + YESNO(this->supports_dehumidification_ || this->supports_humidification_), + YESNO(this->supports_dehumidification_), YESNO(this->supports_humidification_)); - ESP_LOGCONFIG(TAG, " Supported PRESETS: "); - for (auto &it : this->preset_config_) { - const auto *preset_name = LOG_STR_ARG(climate::climate_preset_to_string(it.first)); - - ESP_LOGCONFIG(TAG, " Supports %s: %s", preset_name, YESNO(true)); - this->dump_preset_config_(preset_name, it.second, it.first == this->default_preset_); + if (!this->preset_config_.empty()) { + ESP_LOGCONFIG(TAG, " Supported PRESETS:"); + for (auto &it : this->preset_config_) { + const auto *preset_name = LOG_STR_ARG(climate::climate_preset_to_string(it.first)); + ESP_LOGCONFIG(TAG, " %s:%s", preset_name, it.first == this->default_preset_ ? " (default)" : ""); + this->dump_preset_config_(preset_name, it.second); + } } - ESP_LOGCONFIG(TAG, " Supported CUSTOM PRESETS: "); - for (auto &it : this->custom_preset_config_) { - const auto *preset_name = it.first.c_str(); - - ESP_LOGCONFIG(TAG, " Supports %s: %s", preset_name, YESNO(true)); - this->dump_preset_config_(preset_name, it.second, it.first == this->default_custom_preset_); + if (!this->custom_preset_config_.empty()) { + ESP_LOGCONFIG(TAG, " Supported CUSTOM PRESETS:"); + for (auto &it : this->custom_preset_config_) { + const auto *preset_name = it.first.c_str(); + ESP_LOGCONFIG(TAG, " %s:%s", preset_name, it.first == this->default_custom_preset_ ? " (default)" : ""); + this->dump_preset_config_(preset_name, it.second); + } } - ESP_LOGCONFIG(TAG, " On boot, restore from: %s", - this->on_boot_restore_from_ == thermostat::DEFAULT_PRESET ? "DEFAULT_PRESET" : "MEMORY"); } ThermostatClimateTargetTempConfig::ThermostatClimateTargetTempConfig() = default; diff --git a/esphome/components/thermostat/thermostat_climate.h b/esphome/components/thermostat/thermostat_climate.h index 007d7297d5..363d2b09fc 100644 --- a/esphome/components/thermostat/thermostat_climate.h +++ b/esphome/components/thermostat/thermostat_climate.h @@ -6,24 +6,32 @@ #include "esphome/components/climate/climate.h" #include "esphome/components/sensor/sensor.h" +#include #include #include -#include namespace esphome { namespace thermostat { +enum HumidificationAction : uint8_t { + THERMOSTAT_HUMIDITY_CONTROL_ACTION_OFF = 0, + THERMOSTAT_HUMIDITY_CONTROL_ACTION_DEHUMIDIFY = 1, + THERMOSTAT_HUMIDITY_CONTROL_ACTION_HUMIDIFY = 2, + THERMOSTAT_HUMIDITY_CONTROL_ACTION_NONE, +}; + enum ThermostatClimateTimerIndex : uint8_t { - TIMER_COOLING_MAX_RUN_TIME = 0, - TIMER_COOLING_OFF = 1, - TIMER_COOLING_ON = 2, - TIMER_FAN_MODE = 3, - TIMER_FANNING_OFF = 4, - TIMER_FANNING_ON = 5, - TIMER_HEATING_MAX_RUN_TIME = 6, - TIMER_HEATING_OFF = 7, - TIMER_HEATING_ON = 8, - TIMER_IDLE_ON = 9, + THERMOSTAT_TIMER_COOLING_MAX_RUN_TIME = 0, + THERMOSTAT_TIMER_COOLING_OFF = 1, + THERMOSTAT_TIMER_COOLING_ON = 2, + THERMOSTAT_TIMER_FAN_MODE = 3, + THERMOSTAT_TIMER_FANNING_OFF = 4, + THERMOSTAT_TIMER_FANNING_ON = 5, + THERMOSTAT_TIMER_HEATING_MAX_RUN_TIME = 6, + THERMOSTAT_TIMER_HEATING_OFF = 7, + THERMOSTAT_TIMER_HEATING_ON = 8, + THERMOSTAT_TIMER_IDLE_ON = 9, + THERMOSTAT_TIMER_COUNT = 10, }; enum OnBootRestoreFrom : uint8_t { @@ -89,6 +97,7 @@ class ThermostatClimate : public climate::Climate, public Component { void set_idle_minimum_time_in_sec(uint32_t time); void set_sensor(sensor::Sensor *sensor); void set_humidity_sensor(sensor::Sensor *humidity_sensor); + void set_humidity_hysteresis(float humidity_hysteresis); void set_use_startup_delay(bool use_startup_delay); void set_supports_auto(bool supports_auto); void set_supports_heat_cool(bool supports_heat_cool); @@ -114,6 +123,8 @@ class ThermostatClimate : public climate::Climate, public Component { void set_supports_swing_mode_horizontal(bool supports_swing_mode_horizontal); void set_supports_swing_mode_off(bool supports_swing_mode_off); void set_supports_swing_mode_vertical(bool supports_swing_mode_vertical); + void set_supports_dehumidification(bool supports_dehumidification); + void set_supports_humidification(bool supports_humidification); void set_supports_two_points(bool supports_two_points); void set_preset_config(climate::ClimatePreset preset, const ThermostatClimateTargetTempConfig &config); @@ -131,6 +142,7 @@ class ThermostatClimate : public climate::Climate, public Component { Trigger<> *get_dry_mode_trigger() const; Trigger<> *get_fan_only_mode_trigger() const; Trigger<> *get_heat_mode_trigger() const; + Trigger<> *get_heat_cool_mode_trigger() const; Trigger<> *get_off_mode_trigger() const; Trigger<> *get_fan_mode_on_trigger() const; Trigger<> *get_fan_mode_off_trigger() const; @@ -146,8 +158,12 @@ class ThermostatClimate : public climate::Climate, public Component { Trigger<> *get_swing_mode_horizontal_trigger() const; Trigger<> *get_swing_mode_off_trigger() const; Trigger<> *get_swing_mode_vertical_trigger() const; + Trigger<> *get_humidity_change_trigger() const; Trigger<> *get_temperature_change_trigger() const; Trigger<> *get_preset_change_trigger() const; + Trigger<> *get_humidity_control_dehumidify_action_trigger() const; + Trigger<> *get_humidity_control_humidify_action_trigger() const; + Trigger<> *get_humidity_control_off_action_trigger() const; /// Get current hysteresis values float cool_deadband(); float cool_overrun(); @@ -163,11 +179,14 @@ class ThermostatClimate : public climate::Climate, public Component { /// Returns the fan mode that is locked in (check fan_mode_change_delayed(), first!) climate::ClimateFanMode locked_fan_mode(); /// Set point and hysteresis validation - bool hysteresis_valid(); // returns true if valid + bool hysteresis_valid(); // returns true if valid + bool humidity_hysteresis_valid(); // returns true if valid + bool limit_setpoints_for_heat_cool(); // returns true if set points should be further limited within visual range void validate_target_temperature(); - void validate_target_temperatures(); + void validate_target_temperatures(bool pin_target_temperature_high); void validate_target_temperature_low(); void validate_target_temperature_high(); + void validate_target_humidity(); protected: /// Override control to change settings of the climate device. @@ -189,11 +208,13 @@ class ThermostatClimate : public climate::Climate, public Component { /// Re-compute the required action of this climate controller. climate::ClimateAction compute_action_(bool ignore_timers = false); climate::ClimateAction compute_supplemental_action_(); + HumidificationAction compute_humidity_control_action_(); /// Switch the climate device to the given climate action. void switch_to_action_(climate::ClimateAction action, bool publish_state = true); void switch_to_supplemental_action_(climate::ClimateAction action); void trigger_supplemental_action_(); + void switch_to_humidity_control_action_(HumidificationAction action); /// Switch the climate device to the given climate fan mode. void switch_to_fan_mode_(climate::ClimateFanMode fan_mode, bool publish_state = true); @@ -204,6 +225,9 @@ class ThermostatClimate : public climate::Climate, public Component { /// Switch the climate device to the given climate swing mode. void switch_to_swing_mode_(climate::ClimateSwingMode swing_mode, bool publish_state = true); + /// Check if the humidity change trigger should be called. + void check_humidity_change_trigger_(); + /// Check if the temperature change trigger should be called. void check_temperature_change_trigger_(); @@ -240,13 +264,34 @@ class ThermostatClimate : public climate::Climate, public Component { bool heating_required_(); bool supplemental_cooling_required_(); bool supplemental_heating_required_(); + bool dehumidification_required_(); + bool humidification_required_(); - void dump_preset_config_(const char *preset_name, const ThermostatClimateTargetTempConfig &config, - bool is_default_preset); + void dump_preset_config_(const char *preset_name, const ThermostatClimateTargetTempConfig &config); /// Minimum allowable duration in seconds for action timers const uint8_t min_timer_duration_{1}; + /// Store previously-known states + /// + /// These are used to determine when a trigger/action needs to be called + climate::ClimateFanMode prev_fan_mode_{climate::CLIMATE_FAN_ON}; + climate::ClimateMode prev_mode_{climate::CLIMATE_MODE_OFF}; + climate::ClimateSwingMode prev_swing_mode_{climate::CLIMATE_SWING_OFF}; + + /// The current supplemental action + climate::ClimateAction supplemental_action_{climate::CLIMATE_ACTION_OFF}; + + /// The current humidification action + HumidificationAction humidification_action_{THERMOSTAT_HUMIDITY_CONTROL_ACTION_NONE}; + + /// Default standard preset to use on start up + climate::ClimatePreset default_preset_{}; + + /// If set to DEFAULT_PRESET then the default preset is always used. When MEMORY prior + /// state will attempt to be restored if possible + OnBootRestoreFrom on_boot_restore_from_{OnBootRestoreFrom::MEMORY}; + /// Whether the controller supports auto/cooling/drying/fanning/heating. /// /// A false value for any given attribute means that the controller has no such action @@ -302,6 +347,12 @@ class ThermostatClimate : public climate::Climate, public Component { /// A false value means that the controller has no such support. bool supports_two_points_{false}; + /// Whether the controller supports dehumidification and/or humidification + /// + /// A false value means that the controller has no such support. + bool supports_dehumidification_{false}; + bool supports_humidification_{false}; + /// Flags indicating if maximum allowable run time was exceeded bool cooling_max_runtime_exceeded_{false}; bool heating_max_runtime_exceeded_{false}; @@ -312,9 +363,10 @@ class ThermostatClimate : public climate::Climate, public Component { /// setup_complete_ blocks modifying/resetting the temps immediately after boot bool setup_complete_{false}; - /// Store previously-known temperatures + /// Store previously-known humidity and temperatures /// - /// These are used to determine when the temperature change trigger/action needs to be called + /// These are used to determine when a temperature/humidity has changed + float prev_target_humidity_{NAN}; float prev_target_temperature_{NAN}; float prev_target_temperature_low_{NAN}; float prev_target_temperature_high_{NAN}; @@ -328,6 +380,9 @@ class ThermostatClimate : public climate::Climate, public Component { float heating_deadband_{0}; float heating_overrun_{0}; + /// Hysteresis values used for computing humidification action + float humidity_hysteresis_{0}; + /// Maximum allowable temperature deltas before engaging supplemental cooling/heating actions float supplemental_cool_delta_{0}; float supplemental_heat_delta_{0}; @@ -362,9 +417,15 @@ class ThermostatClimate : public climate::Climate, public Component { Trigger<> *supplemental_heat_action_trigger_{nullptr}; Trigger<> *heat_mode_trigger_{nullptr}; + /// The trigger to call when the controller should switch to heat/cool mode. + /// + /// In heat/cool mode, the controller will enable heating/cooling as necessary and switch + /// to idle when the temperature is within the thresholds/set points. + Trigger<> *heat_cool_mode_trigger_{nullptr}; + /// The trigger to call when the controller should switch to auto mode. /// - /// In auto mode, the controller will enable heating/cooling as necessary and switch + /// In auto mode, the controller will enable heating/cooling as supported/necessary and switch /// to idle when the temperature is within the thresholds/set points. Trigger<> *auto_mode_trigger_{nullptr}; @@ -423,12 +484,24 @@ class ThermostatClimate : public climate::Climate, public Component { /// The trigger to call when the controller should switch the swing mode to "vertical". Trigger<> *swing_mode_vertical_trigger_{nullptr}; + /// The trigger to call when the target humidity changes. + Trigger<> *humidity_change_trigger_{nullptr}; + /// The trigger to call when the target temperature(s) change(es). Trigger<> *temperature_change_trigger_{nullptr}; /// The trigger to call when the preset mode changes Trigger<> *preset_change_trigger_{nullptr}; + /// The trigger to call when dehumidification is required + Trigger<> *humidity_control_dehumidify_action_trigger_{nullptr}; + + /// The trigger to call when humidification is required + Trigger<> *humidity_control_humidify_action_trigger_{nullptr}; + + /// The trigger to call when (de)humidification should stop + Trigger<> *humidity_control_off_action_trigger_{nullptr}; + /// A reference to the trigger that was previously active. /// /// This is so that the previous trigger can be stopped before enabling a new one @@ -437,36 +510,23 @@ class ThermostatClimate : public climate::Climate, public Component { Trigger<> *prev_fan_mode_trigger_{nullptr}; Trigger<> *prev_mode_trigger_{nullptr}; Trigger<> *prev_swing_mode_trigger_{nullptr}; + Trigger<> *prev_humidity_control_trigger_{nullptr}; - /// If set to DEFAULT_PRESET then the default preset is always used. When MEMORY prior - /// state will attempt to be restored if possible - OnBootRestoreFrom on_boot_restore_from_{OnBootRestoreFrom::MEMORY}; - - /// Store previously-known states - /// - /// These are used to determine when a trigger/action needs to be called - climate::ClimateAction supplemental_action_{climate::CLIMATE_ACTION_OFF}; - climate::ClimateFanMode prev_fan_mode_{climate::CLIMATE_FAN_ON}; - climate::ClimateMode prev_mode_{climate::CLIMATE_MODE_OFF}; - climate::ClimateSwingMode prev_swing_mode_{climate::CLIMATE_SWING_OFF}; - - /// Default standard preset to use on start up - climate::ClimatePreset default_preset_{}; /// Default custom preset to use on start up std::string default_custom_preset_{}; /// Climate action timers - std::vector timer_{ - {false, 0, 0, std::bind(&ThermostatClimate::cooling_max_run_time_timer_callback_, this)}, - {false, 0, 0, std::bind(&ThermostatClimate::cooling_off_timer_callback_, this)}, - {false, 0, 0, std::bind(&ThermostatClimate::cooling_on_timer_callback_, this)}, - {false, 0, 0, std::bind(&ThermostatClimate::fan_mode_timer_callback_, this)}, - {false, 0, 0, std::bind(&ThermostatClimate::fanning_off_timer_callback_, this)}, - {false, 0, 0, std::bind(&ThermostatClimate::fanning_on_timer_callback_, this)}, - {false, 0, 0, std::bind(&ThermostatClimate::heating_max_run_time_timer_callback_, this)}, - {false, 0, 0, std::bind(&ThermostatClimate::heating_off_timer_callback_, this)}, - {false, 0, 0, std::bind(&ThermostatClimate::heating_on_timer_callback_, this)}, - {false, 0, 0, std::bind(&ThermostatClimate::idle_on_timer_callback_, this)}, + std::array timer_{ + ThermostatClimateTimer(false, 0, 0, std::bind(&ThermostatClimate::cooling_max_run_time_timer_callback_, this)), + ThermostatClimateTimer(false, 0, 0, std::bind(&ThermostatClimate::cooling_off_timer_callback_, this)), + ThermostatClimateTimer(false, 0, 0, std::bind(&ThermostatClimate::cooling_on_timer_callback_, this)), + ThermostatClimateTimer(false, 0, 0, std::bind(&ThermostatClimate::fan_mode_timer_callback_, this)), + ThermostatClimateTimer(false, 0, 0, std::bind(&ThermostatClimate::fanning_off_timer_callback_, this)), + ThermostatClimateTimer(false, 0, 0, std::bind(&ThermostatClimate::fanning_on_timer_callback_, this)), + ThermostatClimateTimer(false, 0, 0, std::bind(&ThermostatClimate::heating_max_run_time_timer_callback_, this)), + ThermostatClimateTimer(false, 0, 0, std::bind(&ThermostatClimate::heating_off_timer_callback_, this)), + ThermostatClimateTimer(false, 0, 0, std::bind(&ThermostatClimate::heating_on_timer_callback_, this)), + ThermostatClimateTimer(false, 0, 0, std::bind(&ThermostatClimate::idle_on_timer_callback_, this)), }; /// The set of standard preset configurations this thermostat supports (Eg. AWAY, ECO, etc) diff --git a/esphome/components/time/__init__.py b/esphome/components/time/__init__.py index ab821d457b..a20d79b857 100644 --- a/esphome/components/time/__init__.py +++ b/esphome/components/time/__init__.py @@ -6,6 +6,7 @@ import tzlocal from esphome import automation from esphome.automation import Condition import esphome.codegen as cg +from esphome.components.zephyr import zephyr_add_prj_conf import esphome.config_validation as cv from esphome.const import ( CONF_AT, @@ -25,11 +26,11 @@ from esphome.const import ( CONF_TIMEZONE, CONF_TRIGGER_ID, ) -from esphome.core import coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority _LOGGER = logging.getLogger(__name__) -CODEOWNERS = ["@OttoWinter"] +CODEOWNERS = ["@esphome/core"] IS_PLATFORM_COMPONENT = True time_ns = cg.esphome_ns.namespace("time") @@ -235,7 +236,7 @@ def validate_time_at(value): def validate_cron_keys(value): if CONF_CRON in value: - for key in value.keys(): + for key in value: if key in CRON_KEYS: raise cv.Invalid(f"Cannot use option {key} when cron: is specified.") if CONF_AT in value: @@ -245,7 +246,7 @@ def validate_cron_keys(value): value.update(cron_) return value if CONF_AT in value: - for key in value.keys(): + for key in value: if key in CRON_KEYS: raise cv.Invalid(f"Cannot use option {key} when at: is specified.") at_ = value[CONF_AT] @@ -339,8 +340,10 @@ async def register_time(time_var, config): await setup_time_core_(time_var, config) -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): + if CORE.using_zephyr: + zephyr_add_prj_conf("POSIX_CLOCK", True) cg.add_define("USE_TIME") cg.add_global(time_ns.using) diff --git a/esphome/components/time/real_time_clock.cpp b/esphome/components/time/real_time_clock.cpp index 61391d2c6b..42c564659f 100644 --- a/esphome/components/time/real_time_clock.cpp +++ b/esphome/components/time/real_time_clock.cpp @@ -2,13 +2,15 @@ #include "esphome/core/log.h" #ifdef USE_HOST #include +#elif defined(USE_ZEPHYR) +#include #else #include "lwip/opt.h" #endif #ifdef USE_ESP8266 #include "sys/time.h" #endif -#ifdef USE_RP2040 +#if defined(USE_RP2040) || defined(USE_ZEPHYR) #include #endif #include @@ -22,11 +24,22 @@ static const char *const TAG = "time"; RealTimeClock::RealTimeClock() = default; void RealTimeClock::synchronize_epoch_(uint32_t epoch) { + ESP_LOGVV(TAG, "Got epoch %" PRIu32, epoch); // Update UTC epoch time. +#ifdef USE_ZEPHYR + struct timespec ts; + ts.tv_nsec = 0; + ts.tv_sec = static_cast(epoch); + + int ret = clock_settime(CLOCK_REALTIME, &ts); + + if (ret != 0) { + ESP_LOGW(TAG, "clock_settime() failed with code %d", ret); + } +#else struct timeval timev { .tv_sec = static_cast(epoch), .tv_usec = 0, }; - ESP_LOGVV(TAG, "Got epoch %" PRIu32, epoch); struct timezone tz = {0, 0}; int ret = settimeofday(&timev, &tz); if (ret == EINVAL) { @@ -43,7 +56,7 @@ void RealTimeClock::synchronize_epoch_(uint32_t epoch) { if (ret != 0) { ESP_LOGW(TAG, "setimeofday() failed with code %d", ret); } - +#endif auto time = this->now(); ESP_LOGD(TAG, "Synchronized time: %04d-%02d-%02d %02d:%02d:%02d", time.year, time.month, time.day_of_month, time.hour, time.minute, time.second); diff --git a/esphome/components/tlc59208f/tlc59208f_output.cpp b/esphome/components/tlc59208f/tlc59208f_output.cpp index b1aad42bd7..85311a877c 100644 --- a/esphome/components/tlc59208f/tlc59208f_output.cpp +++ b/esphome/components/tlc59208f/tlc59208f_output.cpp @@ -71,12 +71,11 @@ static const uint8_t LDR_PWM = 0x02; static const uint8_t LDR_GRPPWM = 0x03; void TLC59208FOutput::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - ESP_LOGV(TAG, " Resetting all devices on the bus"); // Reset all devices on the bus - if (this->bus_->write(TLC59208F_SWRST_ADDR >> 1, TLC59208F_SWRST_SEQ, 2) != i2c::ERROR_OK) { + if (this->bus_->write_readv(TLC59208F_SWRST_ADDR >> 1, TLC59208F_SWRST_SEQ, sizeof TLC59208F_SWRST_SEQ, nullptr, 0) != + i2c::ERROR_OK) { ESP_LOGE(TAG, "RESET failed"); this->mark_failed(); return; diff --git a/esphome/components/tlc5947/tlc5947.cpp b/esphome/components/tlc5947/tlc5947.cpp index 5a5c0c17c0..6d4e099f7a 100644 --- a/esphome/components/tlc5947/tlc5947.cpp +++ b/esphome/components/tlc5947/tlc5947.cpp @@ -19,8 +19,6 @@ void TLC5947::setup() { } this->pwm_amounts_.resize(this->num_chips_ * N_CHANNELS_PER_CHIP, 0); - - ESP_LOGCONFIG(TAG, "Done setting up TLC5947 output component."); } void TLC5947::dump_config() { ESP_LOGCONFIG(TAG, "TLC5947:"); diff --git a/esphome/components/tlc5971/tlc5971.cpp b/esphome/components/tlc5971/tlc5971.cpp index 05ff0a0080..719ab7c2b3 100644 --- a/esphome/components/tlc5971/tlc5971.cpp +++ b/esphome/components/tlc5971/tlc5971.cpp @@ -13,8 +13,6 @@ void TLC5971::setup() { this->clock_pin_->digital_write(true); this->pwm_amounts_.resize(this->num_chips_ * N_CHANNELS_PER_CHIP, 0); - - ESP_LOGCONFIG(TAG, "Done setting up TLC5971 output component."); } void TLC5971::dump_config() { ESP_LOGCONFIG(TAG, "TLC5971:"); diff --git a/esphome/components/tm1621/tm1621.cpp b/esphome/components/tm1621/tm1621.cpp index 502e45b35e..6859973857 100644 --- a/esphome/components/tm1621/tm1621.cpp +++ b/esphome/components/tm1621/tm1621.cpp @@ -29,8 +29,6 @@ const uint8_t TM1621_DIGIT_ROW[2][12] = {{0x5F, 0x50, 0x3D, 0x79, 0x72, 0x6B, 0x {0xF5, 0x05, 0xB6, 0x97, 0x47, 0xD3, 0xF3, 0x85, 0xF7, 0xD7, 0x02, 0x00}}; void TM1621Display::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->cs_pin_->setup(); // OUTPUT this->cs_pin_->digital_write(true); this->data_pin_->setup(); // OUTPUT diff --git a/esphome/components/tm1637/tm1637.cpp b/esphome/components/tm1637/tm1637.cpp index 358a683efb..49da01472f 100644 --- a/esphome/components/tm1637/tm1637.cpp +++ b/esphome/components/tm1637/tm1637.cpp @@ -125,8 +125,6 @@ const uint8_t TM1637_ASCII_TO_RAW[] PROGMEM = { 0b01100011, // '~', ord 0x7E (degree symbol) }; void TM1637Display::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->clk_pin_->setup(); // OUTPUT this->clk_pin_->digital_write(false); // LOW this->dio_pin_->setup(); // OUTPUT diff --git a/esphome/components/tm1638/tm1638.cpp b/esphome/components/tm1638/tm1638.cpp index f43b496b35..7ba63fe218 100644 --- a/esphome/components/tm1638/tm1638.cpp +++ b/esphome/components/tm1638/tm1638.cpp @@ -20,8 +20,6 @@ static const uint8_t TM1638_UNKNOWN_CHAR = 0b11111111; static const uint8_t TM1638_SHIFT_DELAY = 4; // clock pause between commands, default 4ms void TM1638Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - this->clk_pin_->setup(); // OUTPUT this->dio_pin_->setup(); // OUTPUT this->stb_pin_->setup(); // OUTPUT diff --git a/esphome/components/tm1651/__init__.py b/esphome/components/tm1651/__init__.py index 153cc690e7..49796d9b42 100644 --- a/esphome/components/tm1651/__init__.py +++ b/esphome/components/tm1651/__init__.py @@ -10,26 +10,28 @@ from esphome.const import ( CONF_LEVEL, ) -CODEOWNERS = ["@freekode"] +CODEOWNERS = ["@mrtoy-me"] + +CONF_LEVEL_PERCENT = "level_percent" tm1651_ns = cg.esphome_ns.namespace("tm1651") TM1651Brightness = tm1651_ns.enum("TM1651Brightness") TM1651Display = tm1651_ns.class_("TM1651Display", cg.Component) -SetLevelPercentAction = tm1651_ns.class_("SetLevelPercentAction", automation.Action) -SetLevelAction = tm1651_ns.class_("SetLevelAction", automation.Action) SetBrightnessAction = tm1651_ns.class_("SetBrightnessAction", automation.Action) -TurnOnAction = tm1651_ns.class_("SetLevelPercentAction", automation.Action) -TurnOffAction = tm1651_ns.class_("SetLevelPercentAction", automation.Action) - -CONF_LEVEL_PERCENT = "level_percent" +SetLevelAction = tm1651_ns.class_("SetLevelAction", automation.Action) +SetLevelPercentAction = tm1651_ns.class_("SetLevelPercentAction", automation.Action) +TurnOnAction = tm1651_ns.class_("TurnOnAction", automation.Action) +TurnOffAction = tm1651_ns.class_("TurnOffAction", automation.Action) TM1651_BRIGHTNESS_OPTIONS = { - 1: TM1651Brightness.TM1651_BRIGHTNESS_LOW, - 2: TM1651Brightness.TM1651_BRIGHTNESS_MEDIUM, - 3: TM1651Brightness.TM1651_BRIGHTNESS_HIGH, + 1: TM1651Brightness.TM1651_DARKEST, + 2: TM1651Brightness.TM1651_TYPICAL, + 3: TM1651Brightness.TM1651_BRIGHTEST, } +MULTI_CONF = True + CONFIG_SCHEMA = cv.All( cv.Schema( { @@ -38,26 +40,21 @@ CONFIG_SCHEMA = cv.All( cv.Required(CONF_DIO_PIN): pins.internal_gpio_output_pin_schema, } ), - cv.only_with_arduino, ) -validate_level_percent = cv.All(cv.int_range(min=0, max=100)) -validate_level = cv.All(cv.int_range(min=0, max=7)) -validate_brightness = cv.enum(TM1651_BRIGHTNESS_OPTIONS, int=True) - async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) - clk_pin = await cg.gpio_pin_expression(config[CONF_CLK_PIN]) cg.add(var.set_clk_pin(clk_pin)) dio_pin = await cg.gpio_pin_expression(config[CONF_DIO_PIN]) cg.add(var.set_dio_pin(dio_pin)) - # https://platformio.org/lib/show/6865/TM1651 - cg.add_library("freekode/TM1651", "1.0.1") +validate_brightness = cv.enum(TM1651_BRIGHTNESS_OPTIONS, int=True) +validate_level = cv.All(cv.int_range(min=0, max=7)) +validate_level_percent = cv.All(cv.int_range(min=0, max=100)) BINARY_OUTPUT_ACTION_SCHEMA = maybe_simple_id( { @@ -66,38 +63,22 @@ BINARY_OUTPUT_ACTION_SCHEMA = maybe_simple_id( ) -@automation.register_action("tm1651.turn_on", TurnOnAction, BINARY_OUTPUT_ACTION_SCHEMA) -async def output_turn_on_to_code(config, action_id, template_arg, args): - var = cg.new_Pvariable(action_id, template_arg) - await cg.register_parented(var, config[CONF_ID]) - return var - - @automation.register_action( - "tm1651.turn_off", TurnOffAction, BINARY_OUTPUT_ACTION_SCHEMA -) -async def output_turn_off_to_code(config, action_id, template_arg, args): - var = cg.new_Pvariable(action_id, template_arg) - await cg.register_parented(var, config[CONF_ID]) - return var - - -@automation.register_action( - "tm1651.set_level_percent", - SetLevelPercentAction, + "tm1651.set_brightness", + SetBrightnessAction, cv.maybe_simple_value( { cv.GenerateID(): cv.use_id(TM1651Display), - cv.Required(CONF_LEVEL_PERCENT): cv.templatable(validate_level_percent), + cv.Required(CONF_BRIGHTNESS): cv.templatable(validate_brightness), }, - key=CONF_LEVEL_PERCENT, + key=CONF_BRIGHTNESS, ), ) -async def tm1651_set_level_percent_to_code(config, action_id, template_arg, args): +async def tm1651_set_brightness_to_code(config, action_id, template_arg, args): var = cg.new_Pvariable(action_id, template_arg) await cg.register_parented(var, config[CONF_ID]) - template_ = await cg.templatable(config[CONF_LEVEL_PERCENT], args, cg.uint8) - cg.add(var.set_level_percent(template_)) + template_ = await cg.templatable(config[CONF_BRIGHTNESS], args, cg.uint8) + cg.add(var.set_brightness(template_)) return var @@ -121,19 +102,35 @@ async def tm1651_set_level_to_code(config, action_id, template_arg, args): @automation.register_action( - "tm1651.set_brightness", - SetBrightnessAction, + "tm1651.set_level_percent", + SetLevelPercentAction, cv.maybe_simple_value( { cv.GenerateID(): cv.use_id(TM1651Display), - cv.Required(CONF_BRIGHTNESS): cv.templatable(validate_brightness), + cv.Required(CONF_LEVEL_PERCENT): cv.templatable(validate_level_percent), }, - key=CONF_BRIGHTNESS, + key=CONF_LEVEL_PERCENT, ), ) -async def tm1651_set_brightness_to_code(config, action_id, template_arg, args): +async def tm1651_set_level_percent_to_code(config, action_id, template_arg, args): + var = cg.new_Pvariable(action_id, template_arg) + await cg.register_parented(var, config[CONF_ID]) + template_ = await cg.templatable(config[CONF_LEVEL_PERCENT], args, cg.uint8) + cg.add(var.set_level_percent(template_)) + return var + + +@automation.register_action( + "tm1651.turn_off", TurnOffAction, BINARY_OUTPUT_ACTION_SCHEMA +) +async def output_turn_off_to_code(config, action_id, template_arg, args): + var = cg.new_Pvariable(action_id, template_arg) + await cg.register_parented(var, config[CONF_ID]) + return var + + +@automation.register_action("tm1651.turn_on", TurnOnAction, BINARY_OUTPUT_ACTION_SCHEMA) +async def output_turn_on_to_code(config, action_id, template_arg, args): var = cg.new_Pvariable(action_id, template_arg) await cg.register_parented(var, config[CONF_ID]) - template_ = await cg.templatable(config[CONF_BRIGHTNESS], args, cg.uint8) - cg.add(var.set_brightness(template_)) return var diff --git a/esphome/components/tm1651/tm1651.cpp b/esphome/components/tm1651/tm1651.cpp index 64c3e62b32..15ada0f8ff 100644 --- a/esphome/components/tm1651/tm1651.cpp +++ b/esphome/components/tm1651/tm1651.cpp @@ -1,7 +1,54 @@ -#ifdef USE_ARDUINO +// This Esphome TM1651 component for use with Mini Battery Displays (7 LED levels) +// and removes the Esphome dependency on the TM1651 Arduino library. +// It was largely based on the work of others as set out below. +// @mrtoy-me July 2025 +// ============================================================================================== +// Original Arduino TM1651 library: +// Author:Fred.Chu +// Date:14 August, 2014 +// Applicable Module: Battery Display v1.0 +// This library is free software; you can redistribute it and/or +// modify it under the terms of the GNU Lesser General Public +// License as published by the Free Software Foundation; either +// version 2.1 of the License, or (at your option) any later version. +// This library is distributed in the hope that it will be useful, +// but WITHOUT ANY WARRANTY; without even the implied warranty of +// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.See the GNU +// Lesser General Public License for more details. +// Modified record: +// Author: Detlef Giessmann Germany +// Mail: mydiyp@web.de +// Demo for the new 7 LED Battery-Display 2017 +// IDE: Arduino-1.6.5 +// Type: OPEN-SMART CX10*4RY68 4Color +// Date: 01.05.2017 +// ============================================================================================== +// Esphome component using arduino TM1651 library: +// MIT License +// Copyright (c) 2019 freekode +// ============================================================================================== +// Library and command-line (python) program to control mini battery displays on Raspberry Pi: +// MIT License +// Copyright (c) 2020 Koen Vervloese +// ============================================================================================== +// MIT License +// Permission is hereby granted, free of charge, to any person obtaining a copy +// of this software and associated documentation files (the "Software"), to deal +// in the Software without restriction, including without limitation the rights +// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +// copies of the Software, and to permit persons to whom the Software is +// furnished to do so, subject to the following conditions: +// The above copyright notice and this permission notice shall be included in all +// copies or substantial portions of the Software. +// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +// SOFTWARE. #include "tm1651.h" -#include "esphome/core/helpers.h" #include "esphome/core/log.h" namespace esphome { @@ -9,86 +56,205 @@ namespace tm1651 { static const char *const TAG = "tm1651.display"; -static const uint8_t MAX_INPUT_LEVEL_PERCENT = 100; -static const uint8_t TM1651_MAX_LEVEL = 7; +static const bool LINE_HIGH = true; +static const bool LINE_LOW = false; -static const uint8_t TM1651_BRIGHTNESS_LOW_HW = 0; -static const uint8_t TM1651_BRIGHTNESS_MEDIUM_HW = 2; -static const uint8_t TM1651_BRIGHTNESS_HIGH_HW = 7; +// TM1651 maximum frequency is 500 kHz (duty ratio 50%) = 2 microseconds / cycle +static const uint8_t CLOCK_CYCLE = 8; + +static const uint8_t HALF_CLOCK_CYCLE = CLOCK_CYCLE / 2; +static const uint8_t QUARTER_CLOCK_CYCLE = CLOCK_CYCLE / 4; + +static const uint8_t ADDR_FIXED = 0x44; // fixed address mode +static const uint8_t ADDR_START = 0xC0; // address of the display register + +static const uint8_t DISPLAY_OFF = 0x80; +static const uint8_t DISPLAY_ON = 0x88; + +static const uint8_t MAX_DISPLAY_LEVELS = 7; + +static const uint8_t PERCENT100 = 100; +static const uint8_t PERCENT50 = 50; + +static const uint8_t TM1651_BRIGHTNESS_DARKEST = 0; +static const uint8_t TM1651_BRIGHTNESS_TYPICAL = 2; +static const uint8_t TM1651_BRIGHTNESS_BRIGHTEST = 7; + +static const uint8_t TM1651_LEVEL_TAB[] = {0b00000000, 0b00000001, 0b00000011, 0b00000111, + 0b00001111, 0b00011111, 0b00111111, 0b01111111}; + +// public void TM1651Display::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); + this->clk_pin_->setup(); + this->clk_pin_->pin_mode(gpio::FLAG_OUTPUT); - uint8_t clk = clk_pin_->get_pin(); - uint8_t dio = dio_pin_->get_pin(); + this->dio_pin_->setup(); + this->dio_pin_->pin_mode(gpio::FLAG_OUTPUT); - battery_display_ = make_unique(clk, dio); - battery_display_->init(); - battery_display_->clearDisplay(); + this->brightness_ = TM1651_BRIGHTNESS_TYPICAL; + + // clear display + this->display_level_(); + this->update_brightness_(DISPLAY_ON); } void TM1651Display::dump_config() { - ESP_LOGCONFIG(TAG, "TM1651 Battery Display"); + ESP_LOGCONFIG(TAG, "Battery Display"); LOG_PIN(" CLK: ", clk_pin_); LOG_PIN(" DIO: ", dio_pin_); } -void TM1651Display::set_level_percent(uint8_t new_level) { - this->level_ = calculate_level_(new_level); - this->repaint_(); +void TM1651Display::set_brightness(uint8_t new_brightness) { + this->brightness_ = this->remap_brightness_(new_brightness); + if (this->display_on_) { + this->update_brightness_(DISPLAY_ON); + } } void TM1651Display::set_level(uint8_t new_level) { + if (new_level > MAX_DISPLAY_LEVELS) + new_level = MAX_DISPLAY_LEVELS; this->level_ = new_level; - this->repaint_(); + if (this->display_on_) { + this->display_level_(); + } } -void TM1651Display::set_brightness(uint8_t new_brightness) { - this->brightness_ = calculate_brightness_(new_brightness); - this->repaint_(); -} - -void TM1651Display::turn_on() { - this->is_on_ = true; - this->repaint_(); +void TM1651Display::set_level_percent(uint8_t percentage) { + this->level_ = this->calculate_level_(percentage); + if (this->display_on_) { + this->display_level_(); + } } void TM1651Display::turn_off() { - this->is_on_ = false; - battery_display_->displayLevel(0); + this->display_on_ = false; + this->update_brightness_(DISPLAY_OFF); } -void TM1651Display::repaint_() { - if (!this->is_on_) { - return; - } - - battery_display_->set(this->brightness_); - battery_display_->displayLevel(this->level_); +void TM1651Display::turn_on() { + this->display_on_ = true; + // display level as it could have been changed when display turned off + this->display_level_(); + this->update_brightness_(DISPLAY_ON); } -uint8_t TM1651Display::calculate_level_(uint8_t new_level) { - if (new_level == 0) { - return 0; - } +// protected - float calculated_level = TM1651_MAX_LEVEL / (float) (MAX_INPUT_LEVEL_PERCENT / (float) new_level); - return (uint8_t) roundf(calculated_level); +uint8_t TM1651Display::calculate_level_(uint8_t percentage) { + if (percentage > PERCENT100) + percentage = PERCENT100; + // scale 0-100% to 0-7 display levels + // use integer arithmetic with rounding + uint16_t initial_scaling = (percentage * MAX_DISPLAY_LEVELS) + PERCENT50; + return (uint8_t) (initial_scaling / PERCENT100); } -uint8_t TM1651Display::calculate_brightness_(uint8_t new_brightness) { - if (new_brightness <= 1) { - return TM1651_BRIGHTNESS_LOW_HW; - } else if (new_brightness == 2) { - return TM1651_BRIGHTNESS_MEDIUM_HW; - } else if (new_brightness >= 3) { - return TM1651_BRIGHTNESS_HIGH_HW; +void TM1651Display::display_level_() { + this->start_(); + this->write_byte_(ADDR_FIXED); + this->stop_(); + + this->start_(); + this->write_byte_(ADDR_START); + this->write_byte_(TM1651_LEVEL_TAB[this->level_]); + this->stop_(); +} + +uint8_t TM1651Display::remap_brightness_(uint8_t new_brightness) { + if (new_brightness <= 1) + return TM1651_BRIGHTNESS_DARKEST; + if (new_brightness == 2) + return TM1651_BRIGHTNESS_TYPICAL; + + // new_brightness >= 3 + return TM1651_BRIGHTNESS_BRIGHTEST; +} + +void TM1651Display::update_brightness_(uint8_t on_off_control) { + this->start_(); + this->write_byte_(on_off_control | this->brightness_); + this->stop_(); +} + +// low level functions + +bool TM1651Display::write_byte_(uint8_t data) { + // data bit written to DIO when CLK is low + for (uint8_t i = 0; i < 8; i++) { + this->half_cycle_clock_low_((bool) (data & 0x01)); + this->half_cycle_clock_high_(); + data >>= 1; } - return TM1651_BRIGHTNESS_LOW_HW; + // start 9th cycle, setting DIO high and look for ack + this->half_cycle_clock_low_(LINE_HIGH); + return this->half_cycle_clock_high_ack_(); +} + +void TM1651Display::half_cycle_clock_low_(bool data_bit) { + // first half cycle, clock low and write data bit + this->clk_pin_->digital_write(LINE_LOW); + delayMicroseconds(QUARTER_CLOCK_CYCLE); + + this->dio_pin_->digital_write(data_bit); + delayMicroseconds(QUARTER_CLOCK_CYCLE); +} + +void TM1651Display::half_cycle_clock_high_() { + // second half cycle, clock high + this->clk_pin_->digital_write(LINE_HIGH); + delayMicroseconds(HALF_CLOCK_CYCLE); +} + +bool TM1651Display::half_cycle_clock_high_ack_() { + // second half cycle, clock high and check for ack + this->clk_pin_->digital_write(LINE_HIGH); + delayMicroseconds(QUARTER_CLOCK_CYCLE); + + this->dio_pin_->pin_mode(gpio::FLAG_INPUT); + // valid ack on DIO is low + bool ack = (!this->dio_pin_->digital_read()); + + this->dio_pin_->pin_mode(gpio::FLAG_OUTPUT); + + // ack should be set DIO low by now + // if its not, set DIO low before the next cycle + if (!ack) { + this->dio_pin_->digital_write(LINE_LOW); + } + delayMicroseconds(QUARTER_CLOCK_CYCLE); + + // begin next cycle + this->clk_pin_->digital_write(LINE_LOW); + + return ack; +} + +void TM1651Display::start_() { + // start data transmission + this->delineate_transmission_(LINE_HIGH); +} + +void TM1651Display::stop_() { + // stop data transmission + this->delineate_transmission_(LINE_LOW); +} + +void TM1651Display::delineate_transmission_(bool dio_state) { + // delineate data transmission + // DIO changes its value while CLK is high + + this->dio_pin_->digital_write(dio_state); + delayMicroseconds(HALF_CLOCK_CYCLE); + + this->clk_pin_->digital_write(LINE_HIGH); + delayMicroseconds(QUARTER_CLOCK_CYCLE); + + this->dio_pin_->digital_write(!dio_state); + delayMicroseconds(QUARTER_CLOCK_CYCLE); } } // namespace tm1651 } // namespace esphome - -#endif // USE_ARDUINO diff --git a/esphome/components/tm1651/tm1651.h b/esphome/components/tm1651/tm1651.h index fe7b7d9c6f..7079910adf 100644 --- a/esphome/components/tm1651/tm1651.h +++ b/esphome/components/tm1651/tm1651.h @@ -1,22 +1,16 @@ #pragma once -#ifdef USE_ARDUINO - -#include - +#include "esphome/core/automation.h" #include "esphome/core/component.h" #include "esphome/core/hal.h" -#include "esphome/core/automation.h" - -#include namespace esphome { namespace tm1651 { enum TM1651Brightness : uint8_t { - TM1651_BRIGHTNESS_LOW = 1, - TM1651_BRIGHTNESS_MEDIUM = 2, - TM1651_BRIGHTNESS_HIGH = 3, + TM1651_DARKEST = 1, + TM1651_TYPICAL = 2, + TM1651_BRIGHTEST = 3, }; class TM1651Display : public Component { @@ -27,36 +21,49 @@ class TM1651Display : public Component { void setup() override; void dump_config() override; - void set_level_percent(uint8_t new_level); - void set_level(uint8_t new_level); void set_brightness(uint8_t new_brightness); void set_brightness(TM1651Brightness new_brightness) { this->set_brightness(static_cast(new_brightness)); } - void turn_on(); + void set_level(uint8_t new_level); + void set_level_percent(uint8_t percentage); + void turn_off(); + void turn_on(); protected: - std::unique_ptr battery_display_; + uint8_t calculate_level_(uint8_t percentage); + void display_level_(); + + uint8_t remap_brightness_(uint8_t new_brightness); + void update_brightness_(uint8_t on_off_control); + + // low level functions + bool write_byte_(uint8_t data); + + void half_cycle_clock_low_(bool data_bit); + void half_cycle_clock_high_(); + bool half_cycle_clock_high_ack_(); + + void start_(); + void stop_(); + + void delineate_transmission_(bool dio_state); + InternalGPIOPin *clk_pin_; InternalGPIOPin *dio_pin_; - bool is_on_ = true; - uint8_t brightness_; - uint8_t level_; - - void repaint_(); - - uint8_t calculate_level_(uint8_t new_level); - uint8_t calculate_brightness_(uint8_t new_brightness); + bool display_on_{true}; + uint8_t brightness_{}; + uint8_t level_{0}; }; -template class SetLevelPercentAction : public Action, public Parented { +template class SetBrightnessAction : public Action, public Parented { public: - TEMPLATABLE_VALUE(uint8_t, level_percent) + TEMPLATABLE_VALUE(uint8_t, brightness) void play(Ts... x) override { - auto level_percent = this->level_percent_.value(x...); - this->parent_->set_level_percent(level_percent); + auto brightness = this->brightness_.value(x...); + this->parent_->set_brightness(brightness); } }; @@ -70,13 +77,13 @@ template class SetLevelAction : public Action, public Par } }; -template class SetBrightnessAction : public Action, public Parented { +template class SetLevelPercentAction : public Action, public Parented { public: - TEMPLATABLE_VALUE(uint8_t, brightness) + TEMPLATABLE_VALUE(uint8_t, level_percent) void play(Ts... x) override { - auto brightness = this->brightness_.value(x...); - this->parent_->set_brightness(brightness); + auto level_percent = this->level_percent_.value(x...); + this->parent_->set_level_percent(level_percent); } }; @@ -92,5 +99,3 @@ template class TurnOffAction : public Action, public Pare } // namespace tm1651 } // namespace esphome - -#endif // USE_ARDUINO diff --git a/esphome/components/tmp1075/tmp1075.cpp b/esphome/components/tmp1075/tmp1075.cpp index 831f905bd2..1d9b384c66 100644 --- a/esphome/components/tmp1075/tmp1075.cpp +++ b/esphome/components/tmp1075/tmp1075.cpp @@ -32,7 +32,7 @@ void TMP1075Sensor::update() { uint16_t regvalue; if (!read_byte_16(REG_TEMP, ®value)) { ESP_LOGW(TAG, "'%s' - unable to read temperature register", this->name_.c_str()); - this->status_set_warning("can't read"); + this->status_set_warning(LOG_STR("can't read")); return; } this->status_clear_warning(); diff --git a/esphome/components/tmp117/tmp117.cpp b/esphome/components/tmp117/tmp117.cpp index 5fe8f51414..c9eff41399 100644 --- a/esphome/components/tmp117/tmp117.cpp +++ b/esphome/components/tmp117/tmp117.cpp @@ -26,8 +26,6 @@ void TMP117Component::update() { } } void TMP117Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - if (!this->write_config_(this->config_)) { this->mark_failed(); return; diff --git a/esphome/components/tormatic/tormatic_cover.cpp b/esphome/components/tormatic/tormatic_cover.cpp index be412d62a8..ef93964a28 100644 --- a/esphome/components/tormatic/tormatic_cover.cpp +++ b/esphome/components/tormatic/tormatic_cover.cpp @@ -251,7 +251,7 @@ void Tormatic::stop_at_target_() { // Read a GateStatus from the unit. The unit only sends messages in response to // status requests or commands, so a message needs to be sent first. optional Tormatic::read_gate_status_() { - if (this->available() < sizeof(MessageHeader)) { + if (this->available() < static_cast(sizeof(MessageHeader))) { return {}; } diff --git a/esphome/components/toshiba/climate.py b/esphome/components/toshiba/climate.py index b8e390dd66..bdb17923fa 100644 --- a/esphome/components/toshiba/climate.py +++ b/esphome/components/toshiba/climate.py @@ -14,6 +14,7 @@ MODELS = { "GENERIC": Model.MODEL_GENERIC, "RAC-PT1411HWRU-C": Model.MODEL_RAC_PT1411HWRU_C, "RAC-PT1411HWRU-F": Model.MODEL_RAC_PT1411HWRU_F, + "RAS-2819T": Model.MODEL_RAS_2819T, } CONFIG_SCHEMA = climate_ir.climate_ir_with_receiver_schema(ToshibaClimate).extend( diff --git a/esphome/components/toshiba/toshiba.cpp b/esphome/components/toshiba/toshiba.cpp index ff4241a81f..36e5a21ffa 100644 --- a/esphome/components/toshiba/toshiba.cpp +++ b/esphome/components/toshiba/toshiba.cpp @@ -1,4 +1,5 @@ #include "toshiba.h" +#include "esphome/components/remote_base/toshiba_ac_protocol.h" #include @@ -97,6 +98,282 @@ const std::vector RAC_PT1411HWRU_TEMPERATURE_F{0x10, 0x30, 0x00, 0x20, 0x22, 0x06, 0x26, 0x07, 0x05, 0x25, 0x04, 0x24, 0x0C, 0x2C, 0x0D, 0x2D, 0x09, 0x08, 0x28, 0x0A, 0x2A, 0x0B}; +// RAS-2819T protocol constants +const uint16_t RAS_2819T_HEADER1 = 0xC23D; +const uint8_t RAS_2819T_HEADER2 = 0xD5; +const uint8_t RAS_2819T_MESSAGE_LENGTH = 6; + +// RAS-2819T fan speed codes for rc_code_1 (bytes 2-3) +const uint16_t RAS_2819T_FAN_AUTO = 0xBF40; +const uint16_t RAS_2819T_FAN_QUIET = 0xFF00; +const uint16_t RAS_2819T_FAN_LOW = 0x9F60; +const uint16_t RAS_2819T_FAN_MEDIUM = 0x5FA0; +const uint16_t RAS_2819T_FAN_HIGH = 0x3FC0; + +// RAS-2819T fan speed codes for rc_code_2 (byte 1) +const uint8_t RAS_2819T_FAN2_AUTO = 0x66; +const uint8_t RAS_2819T_FAN2_QUIET = 0x01; +const uint8_t RAS_2819T_FAN2_LOW = 0x28; +const uint8_t RAS_2819T_FAN2_MEDIUM = 0x3C; +const uint8_t RAS_2819T_FAN2_HIGH = 0x50; + +// RAS-2819T second packet suffix bytes for rc_code_2 (bytes 3-5) +// These are fixed patterns, not actual checksums +struct Ras2819tPacketSuffix { + uint8_t byte3; + uint8_t byte4; + uint8_t byte5; +}; +const Ras2819tPacketSuffix RAS_2819T_SUFFIX_AUTO{0x00, 0x02, 0x3D}; +const Ras2819tPacketSuffix RAS_2819T_SUFFIX_QUIET{0x00, 0x02, 0xD8}; +const Ras2819tPacketSuffix RAS_2819T_SUFFIX_LOW{0x00, 0x02, 0xFF}; +const Ras2819tPacketSuffix RAS_2819T_SUFFIX_MEDIUM{0x00, 0x02, 0x13}; +const Ras2819tPacketSuffix RAS_2819T_SUFFIX_HIGH{0x00, 0x02, 0x27}; + +// RAS-2819T swing toggle command +const uint64_t RAS_2819T_SWING_TOGGLE = 0xC23D6B94E01F; + +// RAS-2819T single-packet commands +const uint64_t RAS_2819T_POWER_OFF_COMMAND = 0xC23D7B84E01F; + +// RAS-2819T known valid command patterns for validation +const std::array RAS_2819T_VALID_SINGLE_COMMANDS = { + RAS_2819T_POWER_OFF_COMMAND, // Power off + RAS_2819T_SWING_TOGGLE, // Swing toggle +}; + +const uint16_t RAS_2819T_VALID_HEADER1 = 0xC23D; +const uint8_t RAS_2819T_VALID_HEADER2 = 0xD5; + +const uint8_t RAS_2819T_DRY_BYTE2 = 0x1F; +const uint8_t RAS_2819T_DRY_BYTE3 = 0xE0; +const uint8_t RAS_2819T_DRY_TEMP_OFFSET = 0x24; + +const uint8_t RAS_2819T_AUTO_BYTE2 = 0x1F; +const uint8_t RAS_2819T_AUTO_BYTE3 = 0xE0; +const uint8_t RAS_2819T_AUTO_TEMP_OFFSET = 0x08; + +const uint8_t RAS_2819T_FAN_ONLY_TEMP = 0xE4; +const uint8_t RAS_2819T_FAN_ONLY_TEMP_INV = 0x1B; + +const uint8_t RAS_2819T_HEAT_TEMP_OFFSET = 0x0C; + +// RAS-2819T second packet fixed values +const uint8_t RAS_2819T_AUTO_DRY_FAN_BYTE = 0x65; +const uint8_t RAS_2819T_AUTO_DRY_SUFFIX = 0x3A; +const uint8_t RAS_2819T_HEAT_SUFFIX = 0x3B; + +// RAS-2819T temperature codes for 18-30°C +static const uint8_t RAS_2819T_TEMP_CODES[] = { + 0x10, // 18°C + 0x30, // 19°C + 0x20, // 20°C + 0x60, // 21°C + 0x70, // 22°C + 0x50, // 23°C + 0x40, // 24°C + 0xC0, // 25°C + 0xD0, // 26°C + 0x90, // 27°C + 0x80, // 28°C + 0xA0, // 29°C + 0xB0 // 30°C +}; + +// Helper functions for RAS-2819T protocol +// +// ===== RAS-2819T PROTOCOL DOCUMENTATION ===== +// +// The RAS-2819T uses a two-packet IR protocol with some exceptions for simple commands. +// +// PACKET STRUCTURE: +// All packets are 6 bytes (48 bits) transmitted with standard Toshiba timing. +// +// TWO-PACKET COMMANDS (Mode/Temperature/Fan changes): +// +// First Packet (rc_code_1): [C2 3D] [FAN_HI FAN_LO] [TEMP] [~TEMP] +// Byte 0-1: Header (always 0xC23D) +// Byte 2-3: Fan speed encoding (varies by mode, see fan tables below) +// Byte 4: Temperature + mode encoding +// Byte 5: Bitwise complement of temperature byte +// +// Second Packet (rc_code_2): [D5] [FAN2] [00] [SUF1] [SUF2] [SUF3] +// Byte 0: Header (always 0xD5) +// Byte 1: Fan speed secondary encoding +// Byte 2: Always 0x00 +// Byte 3-5: Fixed suffix pattern (depends on fan speed and mode) +// +// TEMPERATURE ENCODING: +// Base temp codes: 18°C=0x10, 19°C=0x30, 20°C=0x20, 21°C=0x60, 22°C=0x70, +// 23°C=0x50, 24°C=0x40, 25°C=0xC0, 26°C=0xD0, 27°C=0x90, +// 28°C=0x80, 29°C=0xA0, 30°C=0xB0 +// Mode offsets added to base temp: +// COOL: No offset +// HEAT: +0x0C (e.g., 24°C heat = 0x40 | 0x0C = 0x4C) +// AUTO: +0x08 (e.g., 24°C auto = 0x40 | 0x08 = 0x48) +// DRY: +0x24 (e.g., 24°C dry = 0x40 | 0x24 = 0x64) +// +// FAN SPEED ENCODING (First packet bytes 2-3): +// AUTO: 0xBF40, QUIET: 0xFF00, LOW: 0x9F60, MEDIUM: 0x5FA0, HIGH: 0x3FC0 +// Special cases: AUTO/DRY modes use 0x1FE0 instead +// +// SINGLE-PACKET COMMANDS: +// Power Off: 0xC23D7B84E01F (6 bytes, no second packet) +// Swing Toggle: 0xC23D6B94E01F (6 bytes, no second packet) +// +// MODE DETECTION (from first packet): +// - Check bytes 2-3: if 0x7B84 → OFF mode +// - Check bytes 2-3: if 0x1FE0 → AUTO/DRY/low-temp-COOL (distinguish by temp code) +// - Otherwise: COOL/HEAT/FAN_ONLY (distinguish by temp code and byte 5) + +/** + * Get fan speed encoding for RAS-2819T first packet (rc_code_1, bytes 2-3) + */ +static uint16_t get_ras_2819t_fan_code(climate::ClimateFanMode fan_mode) { + switch (fan_mode) { + case climate::CLIMATE_FAN_QUIET: + return RAS_2819T_FAN_QUIET; + case climate::CLIMATE_FAN_LOW: + return RAS_2819T_FAN_LOW; + case climate::CLIMATE_FAN_MEDIUM: + return RAS_2819T_FAN_MEDIUM; + case climate::CLIMATE_FAN_HIGH: + return RAS_2819T_FAN_HIGH; + case climate::CLIMATE_FAN_AUTO: + default: + return RAS_2819T_FAN_AUTO; + } +} + +/** + * Get fan speed encoding for RAS-2819T rc_code_2 packet (second packet) + */ +struct Ras2819tSecondPacketCodes { + uint8_t fan_byte; + Ras2819tPacketSuffix suffix; +}; + +static Ras2819tSecondPacketCodes get_ras_2819t_second_packet_codes(climate::ClimateFanMode fan_mode) { + switch (fan_mode) { + case climate::CLIMATE_FAN_QUIET: + return {RAS_2819T_FAN2_QUIET, RAS_2819T_SUFFIX_QUIET}; + case climate::CLIMATE_FAN_LOW: + return {RAS_2819T_FAN2_LOW, RAS_2819T_SUFFIX_LOW}; + case climate::CLIMATE_FAN_MEDIUM: + return {RAS_2819T_FAN2_MEDIUM, RAS_2819T_SUFFIX_MEDIUM}; + case climate::CLIMATE_FAN_HIGH: + return {RAS_2819T_FAN2_HIGH, RAS_2819T_SUFFIX_HIGH}; + case climate::CLIMATE_FAN_AUTO: + default: + return {RAS_2819T_FAN2_AUTO, RAS_2819T_SUFFIX_AUTO}; + } +} + +/** + * Get temperature code for RAS-2819T protocol + */ +static uint8_t get_ras_2819t_temp_code(float temperature) { + int temp_index = static_cast(temperature) - 18; + if (temp_index < 0 || temp_index >= static_cast(sizeof(RAS_2819T_TEMP_CODES))) { + ESP_LOGW(TAG, "Temperature %.1f°C out of range [18-30°C], defaulting to 24°C", temperature); + return 0x40; // Default to 24°C + } + + return RAS_2819T_TEMP_CODES[temp_index]; +} + +/** + * Decode temperature from RAS-2819T temp code + */ +static float decode_ras_2819t_temperature(uint8_t temp_code) { + uint8_t base_temp_code = temp_code & 0xF0; + + // Find the code in the temperature array + for (size_t temp_index = 0; temp_index < sizeof(RAS_2819T_TEMP_CODES); temp_index++) { + if (RAS_2819T_TEMP_CODES[temp_index] == base_temp_code) { + return static_cast(temp_index + 18); // 18°C is the minimum + } + } + + ESP_LOGW(TAG, "Unknown temp code: 0x%02X, defaulting to 24°C", base_temp_code); + return 24.0f; // Default to 24°C +} + +/** + * Decode fan speed from RAS-2819T IR codes + */ +static climate::ClimateFanMode decode_ras_2819t_fan_mode(uint16_t fan_code) { + switch (fan_code) { + case RAS_2819T_FAN_QUIET: + return climate::CLIMATE_FAN_QUIET; + case RAS_2819T_FAN_LOW: + return climate::CLIMATE_FAN_LOW; + case RAS_2819T_FAN_MEDIUM: + return climate::CLIMATE_FAN_MEDIUM; + case RAS_2819T_FAN_HIGH: + return climate::CLIMATE_FAN_HIGH; + case RAS_2819T_FAN_AUTO: + default: + return climate::CLIMATE_FAN_AUTO; + } +} + +/** + * Validate RAS-2819T IR command structure and content + */ +static bool is_valid_ras_2819t_command(uint64_t rc_code_1, uint64_t rc_code_2 = 0) { + // Check header of first packet + uint16_t header1 = (rc_code_1 >> 32) & 0xFFFF; + if (header1 != RAS_2819T_VALID_HEADER1) { + return false; + } + + // Single packet commands + if (rc_code_2 == 0) { + for (uint64_t valid_cmd : RAS_2819T_VALID_SINGLE_COMMANDS) { + if (rc_code_1 == valid_cmd) { + return true; + } + } + // Additional validation for unknown single packets + return false; + } + + // Two-packet commands - validate second packet header + uint8_t header2 = (rc_code_2 >> 40) & 0xFF; + if (header2 != RAS_2819T_VALID_HEADER2) { + return false; + } + + // Validate temperature complement in first packet (byte 4 should be ~byte 5) + uint8_t temp_byte = (rc_code_1 >> 8) & 0xFF; + uint8_t temp_complement = rc_code_1 & 0xFF; + if (temp_byte != static_cast(~temp_complement)) { + return false; + } + + // Validate fan speed combinations make sense + uint16_t fan_code = (rc_code_1 >> 16) & 0xFFFF; + uint8_t fan2_byte = (rc_code_2 >> 32) & 0xFF; + + // Check if fan codes are from known valid patterns + bool valid_fan_combo = false; + if (fan_code == RAS_2819T_FAN_AUTO && fan2_byte == RAS_2819T_FAN2_AUTO) + valid_fan_combo = true; + if (fan_code == RAS_2819T_FAN_QUIET && fan2_byte == RAS_2819T_FAN2_QUIET) + valid_fan_combo = true; + if (fan_code == RAS_2819T_FAN_LOW && fan2_byte == RAS_2819T_FAN2_LOW) + valid_fan_combo = true; + if (fan_code == RAS_2819T_FAN_MEDIUM && fan2_byte == RAS_2819T_FAN2_MEDIUM) + valid_fan_combo = true; + if (fan_code == RAS_2819T_FAN_HIGH && fan2_byte == RAS_2819T_FAN2_HIGH) + valid_fan_combo = true; + if (fan_code == 0x1FE0 && fan2_byte == RAS_2819T_AUTO_DRY_FAN_BYTE) + valid_fan_combo = true; // AUTO/DRY + + return valid_fan_combo; +} + void ToshibaClimate::setup() { if (this->sensor_) { this->sensor_->add_on_state_callback([this](float state) { @@ -126,16 +403,43 @@ void ToshibaClimate::setup() { this->minimum_temperature_ = this->temperature_min_(); this->maximum_temperature_ = this->temperature_max_(); this->swing_modes_ = this->toshiba_swing_modes_(); + + // Ensure swing mode is always initialized to a valid value + if (this->swing_modes_.empty() || this->swing_modes_.find(this->swing_mode) == this->swing_modes_.end()) { + // No swing support for this model or current swing mode not supported, reset to OFF + this->swing_mode = climate::CLIMATE_SWING_OFF; + } + + // Ensure mode is valid - ESPHome should only use standard climate modes + if (this->mode != climate::CLIMATE_MODE_OFF && this->mode != climate::CLIMATE_MODE_HEAT && + this->mode != climate::CLIMATE_MODE_COOL && this->mode != climate::CLIMATE_MODE_HEAT_COOL && + this->mode != climate::CLIMATE_MODE_DRY && this->mode != climate::CLIMATE_MODE_FAN_ONLY) { + ESP_LOGW(TAG, "Invalid mode detected during setup, resetting to OFF"); + this->mode = climate::CLIMATE_MODE_OFF; + } + + // Ensure fan mode is valid + if (!this->fan_mode.has_value()) { + ESP_LOGW(TAG, "Fan mode not set during setup, defaulting to AUTO"); + this->fan_mode = climate::CLIMATE_FAN_AUTO; + } + // Never send nan to HA if (std::isnan(this->target_temperature)) this->target_temperature = 24; + // Log final state for debugging HA errors + ESP_LOGV(TAG, "Setup complete - Mode: %d, Fan: %s, Swing: %d, Temp: %.1f", static_cast(this->mode), + this->fan_mode.has_value() ? std::to_string(static_cast(this->fan_mode.value())).c_str() : "NONE", + static_cast(this->swing_mode), this->target_temperature); } void ToshibaClimate::transmit_state() { if (this->model_ == MODEL_RAC_PT1411HWRU_C || this->model_ == MODEL_RAC_PT1411HWRU_F) { - transmit_rac_pt1411hwru_(); + this->transmit_rac_pt1411hwru_(); + } else if (this->model_ == MODEL_RAS_2819T) { + this->transmit_ras_2819t_(); } else { - transmit_generic_(); + this->transmit_generic_(); } } @@ -230,7 +534,7 @@ void ToshibaClimate::transmit_generic_() { auto transmit = this->transmitter_->transmit(); auto *data = transmit.get_data(); - encode_(data, message, message_length, 1); + this->encode_(data, message, message_length, 1); transmit.perform(); } @@ -348,15 +652,12 @@ void ToshibaClimate::transmit_rac_pt1411hwru_() { message[11] += message[index]; } } - ESP_LOGV(TAG, "*** Generated codes: 0x%.2X%.2X%.2X%.2X%.2X%.2X 0x%.2X%.2X%.2X%.2X%.2X%.2X", message[0], message[1], - message[2], message[3], message[4], message[5], message[6], message[7], message[8], message[9], message[10], - message[11]); // load first block of IR code and repeat it once - encode_(data, &message[0], RAC_PT1411HWRU_MESSAGE_LENGTH, 1); + this->encode_(data, &message[0], RAC_PT1411HWRU_MESSAGE_LENGTH, 1); // load second block of IR code, if present if (message[6] != 0) { - encode_(data, &message[6], RAC_PT1411HWRU_MESSAGE_LENGTH, 0); + this->encode_(data, &message[6], RAC_PT1411HWRU_MESSAGE_LENGTH, 0); } transmit.perform(); @@ -366,19 +667,19 @@ void ToshibaClimate::transmit_rac_pt1411hwru_() { data->space(TOSHIBA_PACKET_SPACE); switch (this->swing_mode) { case climate::CLIMATE_SWING_VERTICAL: - encode_(data, &RAC_PT1411HWRU_SWING_VERTICAL[0], RAC_PT1411HWRU_MESSAGE_LENGTH, 1); + this->encode_(data, &RAC_PT1411HWRU_SWING_VERTICAL[0], RAC_PT1411HWRU_MESSAGE_LENGTH, 1); break; case climate::CLIMATE_SWING_OFF: default: - encode_(data, &RAC_PT1411HWRU_SWING_OFF[0], RAC_PT1411HWRU_MESSAGE_LENGTH, 1); + this->encode_(data, &RAC_PT1411HWRU_SWING_OFF[0], RAC_PT1411HWRU_MESSAGE_LENGTH, 1); } data->space(TOSHIBA_PACKET_SPACE); transmit.perform(); if (this->sensor_) { - transmit_rac_pt1411hwru_temp_(true, false); + this->transmit_rac_pt1411hwru_temp_(true, false); } } @@ -430,15 +731,217 @@ void ToshibaClimate::transmit_rac_pt1411hwru_temp_(const bool cs_state, const bo // Byte 5: Footer lower/bitwise complement of byte 4 message[5] = ~message[4]; - ESP_LOGV(TAG, "*** Generated code: 0x%.2X%.2X%.2X%.2X%.2X%.2X", message[0], message[1], message[2], message[3], - message[4], message[5]); // load IR code and repeat it once - encode_(data, message, RAC_PT1411HWRU_MESSAGE_LENGTH, 1); + this->encode_(data, message, RAC_PT1411HWRU_MESSAGE_LENGTH, 1); transmit.perform(); } } +void ToshibaClimate::transmit_ras_2819t_() { + // Handle swing mode transmission for RAS-2819T + // Note: RAS-2819T uses a toggle command, so we need to track state changes + + // Check if ONLY swing mode changed (and no other climate parameters) + bool swing_changed = (this->swing_mode != this->last_swing_mode_); + bool mode_changed = (this->mode != this->last_mode_); + bool fan_changed = (this->fan_mode != this->last_fan_mode_); + bool temp_changed = (abs(this->target_temperature - this->last_target_temperature_) > 0.1f); + + bool only_swing_changed = swing_changed && !mode_changed && !fan_changed && !temp_changed; + + if (only_swing_changed) { + // Send ONLY swing toggle command (like the physical remote does) + auto swing_transmit = this->transmitter_->transmit(); + auto *swing_data = swing_transmit.get_data(); + + // Convert toggle command to bytes for transmission + uint8_t swing_message[RAS_2819T_MESSAGE_LENGTH]; + swing_message[0] = (RAS_2819T_SWING_TOGGLE >> 40) & 0xFF; + swing_message[1] = (RAS_2819T_SWING_TOGGLE >> 32) & 0xFF; + swing_message[2] = (RAS_2819T_SWING_TOGGLE >> 24) & 0xFF; + swing_message[3] = (RAS_2819T_SWING_TOGGLE >> 16) & 0xFF; + swing_message[4] = (RAS_2819T_SWING_TOGGLE >> 8) & 0xFF; + swing_message[5] = RAS_2819T_SWING_TOGGLE & 0xFF; + + // Use single packet transmission WITH repeat (like regular commands) + this->encode_(swing_data, swing_message, RAS_2819T_MESSAGE_LENGTH, 1); + swing_transmit.perform(); + + // Update all state tracking + this->last_swing_mode_ = this->swing_mode; + this->last_mode_ = this->mode; + this->last_fan_mode_ = this->fan_mode; + this->last_target_temperature_ = this->target_temperature; + + // Immediately publish the state change to Home Assistant + this->publish_state(); + + return; // Exit early - don't send climate command + } + + // If we get here, send the regular climate command (temperature/mode/fan) + uint8_t message1[RAS_2819T_MESSAGE_LENGTH] = {0}; + uint8_t message2[RAS_2819T_MESSAGE_LENGTH] = {0}; + float temperature = + clamp(this->target_temperature, TOSHIBA_RAS_2819T_TEMP_C_MIN, TOSHIBA_RAS_2819T_TEMP_C_MAX); + + // Build first packet (RAS_2819T_HEADER1 + 4 bytes) + message1[0] = (RAS_2819T_HEADER1 >> 8) & 0xFF; + message1[1] = RAS_2819T_HEADER1 & 0xFF; + + // Handle OFF mode + if (this->mode == climate::CLIMATE_MODE_OFF) { + // Extract bytes from power off command constant + message1[2] = (RAS_2819T_POWER_OFF_COMMAND >> 24) & 0xFF; + message1[3] = (RAS_2819T_POWER_OFF_COMMAND >> 16) & 0xFF; + message1[4] = (RAS_2819T_POWER_OFF_COMMAND >> 8) & 0xFF; + message1[5] = RAS_2819T_POWER_OFF_COMMAND & 0xFF; + // No second packet for OFF + } else { + // Get temperature and fan encoding + uint8_t temp_code = get_ras_2819t_temp_code(temperature); + + // Get fan speed encoding for rc_code_1 + climate::ClimateFanMode effective_fan_mode = this->fan_mode.value(); + + // Dry mode only supports AUTO fan speed + if (this->mode == climate::CLIMATE_MODE_DRY) { + effective_fan_mode = climate::CLIMATE_FAN_AUTO; + if (this->fan_mode.value() != climate::CLIMATE_FAN_AUTO) { + ESP_LOGW(TAG, "Dry mode only supports AUTO fan speed, forcing AUTO"); + } + } + + uint16_t fan_code = get_ras_2819t_fan_code(effective_fan_mode); + + // Mode and temperature encoding + switch (this->mode) { + case climate::CLIMATE_MODE_COOL: + // All cooling temperatures support fan speed control + message1[2] = (fan_code >> 8) & 0xFF; + message1[3] = fan_code & 0xFF; + message1[4] = temp_code; + message1[5] = ~temp_code; + break; + + case climate::CLIMATE_MODE_HEAT: + // Heating supports fan speed control + message1[2] = (fan_code >> 8) & 0xFF; + message1[3] = fan_code & 0xFF; + // Heat mode adds offset to temperature code + message1[4] = temp_code | RAS_2819T_HEAT_TEMP_OFFSET; + message1[5] = ~(temp_code | RAS_2819T_HEAT_TEMP_OFFSET); + break; + + case climate::CLIMATE_MODE_HEAT_COOL: + // Auto mode uses fixed encoding + message1[2] = RAS_2819T_AUTO_BYTE2; + message1[3] = RAS_2819T_AUTO_BYTE3; + message1[4] = temp_code | RAS_2819T_AUTO_TEMP_OFFSET; + message1[5] = ~(temp_code | RAS_2819T_AUTO_TEMP_OFFSET); + break; + + case climate::CLIMATE_MODE_DRY: + // Dry mode uses fixed encoding and forces AUTO fan + message1[2] = RAS_2819T_DRY_BYTE2; + message1[3] = RAS_2819T_DRY_BYTE3; + message1[4] = temp_code | RAS_2819T_DRY_TEMP_OFFSET; + message1[5] = ~message1[4]; + break; + + case climate::CLIMATE_MODE_FAN_ONLY: + // Fan only mode supports fan speed control + message1[2] = (fan_code >> 8) & 0xFF; + message1[3] = fan_code & 0xFF; + message1[4] = RAS_2819T_FAN_ONLY_TEMP; + message1[5] = RAS_2819T_FAN_ONLY_TEMP_INV; + break; + + default: + // Default case supports fan speed control + message1[2] = (fan_code >> 8) & 0xFF; + message1[3] = fan_code & 0xFF; + message1[4] = temp_code; + message1[5] = ~temp_code; + break; + } + + // Build second packet (RAS_2819T_HEADER2 + 4 bytes) + message2[0] = RAS_2819T_HEADER2; + + // Get fan speed encoding for rc_code_2 + Ras2819tSecondPacketCodes second_packet_codes = get_ras_2819t_second_packet_codes(effective_fan_mode); + + // Determine header byte 2 and fan encoding based on mode + switch (this->mode) { + case climate::CLIMATE_MODE_COOL: + message2[1] = second_packet_codes.fan_byte; + message2[2] = 0x00; + message2[3] = second_packet_codes.suffix.byte3; + message2[4] = second_packet_codes.suffix.byte4; + message2[5] = second_packet_codes.suffix.byte5; + break; + + case climate::CLIMATE_MODE_HEAT: + message2[1] = second_packet_codes.fan_byte; + message2[2] = 0x00; + message2[3] = second_packet_codes.suffix.byte3; + message2[4] = 0x00; + message2[5] = RAS_2819T_HEAT_SUFFIX; + break; + + case climate::CLIMATE_MODE_HEAT_COOL: + case climate::CLIMATE_MODE_DRY: + // Auto/Dry modes use fixed values regardless of fan setting + message2[1] = RAS_2819T_AUTO_DRY_FAN_BYTE; + message2[2] = 0x00; + message2[3] = 0x00; + message2[4] = 0x00; + message2[5] = RAS_2819T_AUTO_DRY_SUFFIX; + break; + + case climate::CLIMATE_MODE_FAN_ONLY: + message2[1] = second_packet_codes.fan_byte; + message2[2] = 0x00; + message2[3] = second_packet_codes.suffix.byte3; + message2[4] = 0x00; + message2[5] = RAS_2819T_HEAT_SUFFIX; + break; + + default: + message2[1] = second_packet_codes.fan_byte; + message2[2] = 0x00; + message2[3] = second_packet_codes.suffix.byte3; + message2[4] = second_packet_codes.suffix.byte4; + message2[5] = second_packet_codes.suffix.byte5; + break; + } + } + + // Log final messages being transmitted + + // Transmit using proper Toshiba protocol timing + auto transmit = this->transmitter_->transmit(); + auto *data = transmit.get_data(); + + // Use existing Toshiba encode function for proper timing + this->encode_(data, message1, RAS_2819T_MESSAGE_LENGTH, 1); + + if (this->mode != climate::CLIMATE_MODE_OFF) { + // Send second packet with gap + this->encode_(data, message2, RAS_2819T_MESSAGE_LENGTH, 0); + } + + transmit.perform(); + + // Update all state tracking after successful transmission + this->last_swing_mode_ = this->swing_mode; + this->last_mode_ = this->mode; + this->last_fan_mode_ = this->fan_mode; + this->last_target_temperature_ = this->target_temperature; +} + uint8_t ToshibaClimate::is_valid_rac_pt1411hwru_header_(const uint8_t *message) { const std::vector header{RAC_PT1411HWRU_MESSAGE_HEADER0, RAC_PT1411HWRU_CS_HEADER, RAC_PT1411HWRU_SWING_HEADER}; @@ -464,11 +967,11 @@ bool ToshibaClimate::compare_rac_pt1411hwru_packets_(const uint8_t *message1, co bool ToshibaClimate::is_valid_rac_pt1411hwru_message_(const uint8_t *message) { uint8_t checksum = 0; - switch (is_valid_rac_pt1411hwru_header_(message)) { + switch (this->is_valid_rac_pt1411hwru_header_(message)) { case RAC_PT1411HWRU_MESSAGE_HEADER0: case RAC_PT1411HWRU_CS_HEADER: case RAC_PT1411HWRU_SWING_HEADER: - if (is_valid_rac_pt1411hwru_header_(message) && (message[2] == static_cast(~message[3])) && + if (this->is_valid_rac_pt1411hwru_header_(message) && (message[2] == static_cast(~message[3])) && (message[4] == static_cast(~message[5]))) { return true; } @@ -490,7 +993,103 @@ bool ToshibaClimate::is_valid_rac_pt1411hwru_message_(const uint8_t *message) { return false; } +bool ToshibaClimate::process_ras_2819t_command_(const remote_base::ToshibaAcData &toshiba_data) { + // Check for power-off command (single packet) + if (toshiba_data.rc_code_2 == 0 && toshiba_data.rc_code_1 == RAS_2819T_POWER_OFF_COMMAND) { + this->mode = climate::CLIMATE_MODE_OFF; + ESP_LOGI(TAG, "Mode: OFF"); + this->publish_state(); + return true; + } + + // Check for swing toggle command (single packet) + if (toshiba_data.rc_code_2 == 0 && toshiba_data.rc_code_1 == RAS_2819T_SWING_TOGGLE) { + // Toggle swing mode + if (this->swing_mode == climate::CLIMATE_SWING_VERTICAL) { + this->swing_mode = climate::CLIMATE_SWING_OFF; + ESP_LOGI(TAG, "Swing: OFF"); + } else { + this->swing_mode = climate::CLIMATE_SWING_VERTICAL; + ESP_LOGI(TAG, "Swing: VERTICAL"); + } + this->publish_state(); + return true; + } + + // Handle regular two-packet commands (mode/temperature/fan changes) + if (toshiba_data.rc_code_2 != 0) { + // Convert to byte array for easier processing + uint8_t message1[6], message2[6]; + for (uint8_t i = 0; i < 6; i++) { + message1[i] = (toshiba_data.rc_code_1 >> (40 - i * 8)) & 0xFF; + message2[i] = (toshiba_data.rc_code_2 >> (40 - i * 8)) & 0xFF; + } + + // Decode the protocol using message1 (rc_code_1) + uint8_t temp_code = message1[4]; + + // Decode mode - check bytes 2-3 pattern and temperature code + if ((message1[2] == 0x7B) && (message1[3] == 0x84)) { + // OFF mode has specific pattern + this->mode = climate::CLIMATE_MODE_OFF; + ESP_LOGI(TAG, "Mode: OFF"); + } else if ((message1[2] == 0x1F) && (message1[3] == 0xE0)) { + // 0x1FE0 pattern is used for AUTO, DRY, and low-temp COOL + if ((temp_code & 0x0F) == 0x08) { + this->mode = climate::CLIMATE_MODE_HEAT_COOL; + ESP_LOGI(TAG, "Mode: AUTO"); + } else if ((temp_code & 0x0F) == 0x04) { + this->mode = climate::CLIMATE_MODE_DRY; + ESP_LOGI(TAG, "Mode: DRY"); + } else { + this->mode = climate::CLIMATE_MODE_COOL; + ESP_LOGI(TAG, "Mode: COOL (low temp)"); + } + } else { + // Variable fan speed patterns - decode by temperature code + if ((temp_code & 0x0F) == 0x0C) { + this->mode = climate::CLIMATE_MODE_HEAT; + ESP_LOGI(TAG, "Mode: HEAT"); + } else if (message1[5] == 0x1B) { + this->mode = climate::CLIMATE_MODE_FAN_ONLY; + ESP_LOGI(TAG, "Mode: FAN_ONLY"); + } else { + this->mode = climate::CLIMATE_MODE_COOL; + ESP_LOGI(TAG, "Mode: COOL"); + } + } + + // Decode fan speed from rc_code_1 + uint16_t fan_code = (message1[2] << 8) | message1[3]; + this->fan_mode = decode_ras_2819t_fan_mode(fan_code); + + // Decode temperature + if (this->mode != climate::CLIMATE_MODE_OFF && this->mode != climate::CLIMATE_MODE_FAN_ONLY) { + this->target_temperature = decode_ras_2819t_temperature(temp_code); + } + + this->publish_state(); + return true; + } else { + ESP_LOGD(TAG, "Unknown single-packet RAS-2819T command: 0x%" PRIX64, toshiba_data.rc_code_1); + return false; + } +} + bool ToshibaClimate::on_receive(remote_base::RemoteReceiveData data) { + // Try modern ToshibaAcProtocol decoder first (handles RAS-2819T and potentially others) + remote_base::ToshibaAcProtocol toshiba_protocol; + auto decode_result = toshiba_protocol.decode(data); + + if (decode_result.has_value()) { + auto toshiba_data = decode_result.value(); + // Validate and process RAS-2819T commands + if (is_valid_ras_2819t_command(toshiba_data.rc_code_1, toshiba_data.rc_code_2)) { + return this->process_ras_2819t_command_(toshiba_data); + } + } + + // Fall back to generic processing for older protocols uint8_t message[18] = {0}; uint8_t message_length = TOSHIBA_HEADER_LENGTH, temperature_code = 0; @@ -499,11 +1098,11 @@ bool ToshibaClimate::on_receive(remote_base::RemoteReceiveData data) { return false; } // Read incoming bits into buffer - if (!decode_(&data, message, message_length)) { + if (!this->decode_(&data, message, message_length)) { return false; } // Determine incoming message protocol version and/or length - if (is_valid_rac_pt1411hwru_header_(message)) { + if (this->is_valid_rac_pt1411hwru_header_(message)) { // We already received four bytes message_length = RAC_PT1411HWRU_MESSAGE_LENGTH - 4; } else if ((message[0] ^ message[1] ^ message[2]) != message[3]) { @@ -514,11 +1113,11 @@ bool ToshibaClimate::on_receive(remote_base::RemoteReceiveData data) { message_length = message[2] + 2; } // Decode the remaining bytes - if (!decode_(&data, &message[4], message_length)) { + if (!this->decode_(&data, &message[4], message_length)) { return false; } // If this is a RAC-PT1411HWRU message, we expect the first packet a second time and also possibly a third packet - if (is_valid_rac_pt1411hwru_header_(message)) { + if (this->is_valid_rac_pt1411hwru_header_(message)) { // There is always a space between packets if (!data.expect_item(TOSHIBA_BIT_MARK, TOSHIBA_GAP_SPACE)) { return false; @@ -527,7 +1126,7 @@ bool ToshibaClimate::on_receive(remote_base::RemoteReceiveData data) { if (!data.expect_item(TOSHIBA_HEADER_MARK, TOSHIBA_HEADER_SPACE)) { return false; } - if (!decode_(&data, &message[6], RAC_PT1411HWRU_MESSAGE_LENGTH)) { + if (!this->decode_(&data, &message[6], RAC_PT1411HWRU_MESSAGE_LENGTH)) { return false; } // If this is a RAC-PT1411HWRU message, there may also be a third packet. @@ -535,25 +1134,25 @@ bool ToshibaClimate::on_receive(remote_base::RemoteReceiveData data) { if (data.expect_item(TOSHIBA_BIT_MARK, TOSHIBA_GAP_SPACE)) { // Validate header 3 data.expect_item(TOSHIBA_HEADER_MARK, TOSHIBA_HEADER_SPACE); - if (decode_(&data, &message[12], RAC_PT1411HWRU_MESSAGE_LENGTH)) { - if (!is_valid_rac_pt1411hwru_message_(&message[12])) { + if (this->decode_(&data, &message[12], RAC_PT1411HWRU_MESSAGE_LENGTH)) { + if (!this->is_valid_rac_pt1411hwru_message_(&message[12])) { // If a third packet was received but the checksum is not valid, fail return false; } } } - if (!compare_rac_pt1411hwru_packets_(&message[0], &message[6])) { + if (!this->compare_rac_pt1411hwru_packets_(&message[0], &message[6])) { // If the first two packets don't match each other, fail return false; } - if (!is_valid_rac_pt1411hwru_message_(&message[0])) { + if (!this->is_valid_rac_pt1411hwru_message_(&message[0])) { // If the first packet isn't valid, fail return false; } } // Header has been verified, now determine protocol version and set the climate component properties - switch (is_valid_rac_pt1411hwru_header_(message)) { + switch (this->is_valid_rac_pt1411hwru_header_(message)) { // Power, temperature, mode, fan speed case RAC_PT1411HWRU_MESSAGE_HEADER0: // Get the mode @@ -608,7 +1207,7 @@ bool ToshibaClimate::on_receive(remote_base::RemoteReceiveData data) { break; } // Get the target temperature - if (is_valid_rac_pt1411hwru_message_(&message[12])) { + if (this->is_valid_rac_pt1411hwru_message_(&message[12])) { temperature_code = (message[4] >> 4) | (message[14] & RAC_PT1411HWRU_FLAG_FRAC) | (message[15] & RAC_PT1411HWRU_FLAG_NEG); if (message[15] & RAC_PT1411HWRU_FLAG_FAH) { diff --git a/esphome/components/toshiba/toshiba.h b/esphome/components/toshiba/toshiba.h index 83e85c34db..d76833f406 100644 --- a/esphome/components/toshiba/toshiba.h +++ b/esphome/components/toshiba/toshiba.h @@ -1,6 +1,7 @@ #pragma once #include "esphome/components/climate_ir/climate_ir.h" +#include "esphome/components/remote_base/toshiba_ac_protocol.h" namespace esphome { namespace toshiba { @@ -10,6 +11,7 @@ enum Model { MODEL_GENERIC = 0, // Temperature range is from 17 to 30 MODEL_RAC_PT1411HWRU_C = 1, // Temperature range is from 16 to 30 MODEL_RAC_PT1411HWRU_F = 2, // Temperature range is from 16 to 30 + MODEL_RAS_2819T = 3, // RAS-2819T protocol variant, temperature range 18 to 30 }; // Supported temperature ranges @@ -19,6 +21,8 @@ const float TOSHIBA_RAC_PT1411HWRU_TEMP_C_MIN = 16.0; const float TOSHIBA_RAC_PT1411HWRU_TEMP_C_MAX = 30.0; const float TOSHIBA_RAC_PT1411HWRU_TEMP_F_MIN = 60.0; const float TOSHIBA_RAC_PT1411HWRU_TEMP_F_MAX = 86.0; +const float TOSHIBA_RAS_2819T_TEMP_C_MIN = 18.0; +const float TOSHIBA_RAS_2819T_TEMP_C_MAX = 30.0; class ToshibaClimate : public climate_ir::ClimateIR { public: @@ -35,6 +39,9 @@ class ToshibaClimate : public climate_ir::ClimateIR { void transmit_generic_(); void transmit_rac_pt1411hwru_(); void transmit_rac_pt1411hwru_temp_(bool cs_state = true, bool cs_send_update = true); + void transmit_ras_2819t_(); + // Process RAS-2819T IR command data + bool process_ras_2819t_command_(const remote_base::ToshibaAcData &toshiba_data); // Returns the header if valid, else returns zero uint8_t is_valid_rac_pt1411hwru_header_(const uint8_t *message); // Returns true if message is a valid RAC-PT1411HWRU IR message, regardless if first or second packet @@ -43,11 +50,26 @@ class ToshibaClimate : public climate_ir::ClimateIR { bool compare_rac_pt1411hwru_packets_(const uint8_t *message1, const uint8_t *message2); bool on_receive(remote_base::RemoteReceiveData data) override; + private: + // RAS-2819T state tracking for swing mode optimization + climate::ClimateSwingMode last_swing_mode_{climate::CLIMATE_SWING_OFF}; + climate::ClimateMode last_mode_{climate::CLIMATE_MODE_OFF}; + optional last_fan_mode_{}; + float last_target_temperature_{24.0f}; + float temperature_min_() { - return (this->model_ == MODEL_GENERIC) ? TOSHIBA_GENERIC_TEMP_C_MIN : TOSHIBA_RAC_PT1411HWRU_TEMP_C_MIN; + if (this->model_ == MODEL_RAC_PT1411HWRU_C || this->model_ == MODEL_RAC_PT1411HWRU_F) + return TOSHIBA_RAC_PT1411HWRU_TEMP_C_MIN; + if (this->model_ == MODEL_RAS_2819T) + return TOSHIBA_RAS_2819T_TEMP_C_MIN; + return TOSHIBA_GENERIC_TEMP_C_MIN; // Default to GENERIC for unknown models } float temperature_max_() { - return (this->model_ == MODEL_GENERIC) ? TOSHIBA_GENERIC_TEMP_C_MAX : TOSHIBA_RAC_PT1411HWRU_TEMP_C_MAX; + if (this->model_ == MODEL_RAC_PT1411HWRU_C || this->model_ == MODEL_RAC_PT1411HWRU_F) + return TOSHIBA_RAC_PT1411HWRU_TEMP_C_MAX; + if (this->model_ == MODEL_RAS_2819T) + return TOSHIBA_RAS_2819T_TEMP_C_MAX; + return TOSHIBA_GENERIC_TEMP_C_MAX; // Default to GENERIC for unknown models } std::set toshiba_swing_modes_() { return (this->model_ == MODEL_GENERIC) diff --git a/esphome/components/total_daily_energy/total_daily_energy.cpp b/esphome/components/total_daily_energy/total_daily_energy.cpp index 7c316c495d..818696f99b 100644 --- a/esphome/components/total_daily_energy/total_daily_energy.cpp +++ b/esphome/components/total_daily_energy/total_daily_energy.cpp @@ -10,7 +10,7 @@ void TotalDailyEnergy::setup() { float initial_value = 0; if (this->restore_) { - this->pref_ = global_preferences->make_preference(this->get_object_id_hash()); + this->pref_ = global_preferences->make_preference(this->get_preference_hash()); this->pref_.load(&initial_value); } this->publish_state_and_save(initial_value); diff --git a/esphome/components/touchscreen/__init__.py b/esphome/components/touchscreen/__init__.py index 01a271a34e..4a5c03ace4 100644 --- a/esphome/components/touchscreen/__init__.py +++ b/esphome/components/touchscreen/__init__.py @@ -13,7 +13,7 @@ from esphome.const import ( CONF_SWAP_XY, CONF_TRANSFORM, ) -from esphome.core import coroutine_with_priority +from esphome.core import CoroPriority, coroutine_with_priority CODEOWNERS = ["@jesserockz", "@nielsnl68"] DEPENDENCIES = ["display"] @@ -152,7 +152,7 @@ async def register_touchscreen(var, config): ) -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): cg.add_global(touchscreen_ns.using) cg.add_define("USE_TOUCHSCREEN") diff --git a/esphome/components/tsl2561/tsl2561.cpp b/esphome/components/tsl2561/tsl2561.cpp index 1b5c9f2635..1442dd176c 100644 --- a/esphome/components/tsl2561/tsl2561.cpp +++ b/esphome/components/tsl2561/tsl2561.cpp @@ -15,7 +15,6 @@ static const uint8_t TSL2561_REGISTER_DATA_0 = 0x0C; static const uint8_t TSL2561_REGISTER_DATA_1 = 0x0E; void TSL2561Sensor::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t id; if (!this->tsl2561_read_byte(TSL2561_REGISTER_ID, &id)) { this->mark_failed(); diff --git a/esphome/components/tsl2591/tsl2591.cpp b/esphome/components/tsl2591/tsl2591.cpp index c7622b116a..999e42e949 100644 --- a/esphome/components/tsl2591/tsl2591.cpp +++ b/esphome/components/tsl2591/tsl2591.cpp @@ -43,7 +43,6 @@ void TSL2591Component::disable_if_power_saving_() { } void TSL2591Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup for address 0x%02X", this->address_); switch (this->component_gain_) { case TSL2591_CGAIN_LOW: this->gain_ = TSL2591_GAIN_LOW; diff --git a/esphome/components/tt21100/touchscreen/tt21100.cpp b/esphome/components/tt21100/touchscreen/tt21100.cpp index d4dd1c195f..b4735fe6d7 100644 --- a/esphome/components/tt21100/touchscreen/tt21100.cpp +++ b/esphome/components/tt21100/touchscreen/tt21100.cpp @@ -47,8 +47,6 @@ struct TT21100TouchReport { float TT21100Touchscreen::get_setup_priority() const { return setup_priority::HARDWARE - 1.0f; } void TT21100Touchscreen::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // Register interrupt pin if (this->interrupt_pin_ != nullptr) { this->interrupt_pin_->pin_mode(gpio::FLAG_INPUT | gpio::FLAG_PULLUP); diff --git a/esphome/components/ttp229_bsf/ttp229_bsf.cpp b/esphome/components/ttp229_bsf/ttp229_bsf.cpp index 8b58795ebb..8d1ed45bb0 100644 --- a/esphome/components/ttp229_bsf/ttp229_bsf.cpp +++ b/esphome/components/ttp229_bsf/ttp229_bsf.cpp @@ -7,7 +7,6 @@ namespace ttp229_bsf { static const char *const TAG = "ttp229_bsf"; void TTP229BSFComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->sdo_pin_->setup(); this->scl_pin_->setup(); this->scl_pin_->digital_write(true); diff --git a/esphome/components/ttp229_lsf/ttp229_lsf.cpp b/esphome/components/ttp229_lsf/ttp229_lsf.cpp index 8e976da4ef..7bdb57ebec 100644 --- a/esphome/components/ttp229_lsf/ttp229_lsf.cpp +++ b/esphome/components/ttp229_lsf/ttp229_lsf.cpp @@ -7,7 +7,6 @@ namespace ttp229_lsf { static const char *const TAG = "ttp229_lsf"; void TTP229LSFComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); uint8_t data[2]; if (this->read(data, 2) != i2c::ERROR_OK) { this->error_code_ = COMMUNICATION_FAILED; diff --git a/esphome/components/tuya/climate/__init__.py b/esphome/components/tuya/climate/__init__.py index 4dbdf07651..cefe5d5a4b 100644 --- a/esphome/components/tuya/climate/__init__.py +++ b/esphome/components/tuya/climate/__init__.py @@ -46,16 +46,15 @@ TuyaClimate = tuya_ns.class_("TuyaClimate", climate.Climate, cg.Component) def validate_temperature_multipliers(value): - if CONF_TEMPERATURE_MULTIPLIER in value: - if ( - CONF_CURRENT_TEMPERATURE_MULTIPLIER in value - or CONF_TARGET_TEMPERATURE_MULTIPLIER in value - ): - raise cv.Invalid( - f"Cannot have {CONF_TEMPERATURE_MULTIPLIER} at the same time as " - f"{CONF_CURRENT_TEMPERATURE_MULTIPLIER} and " - f"{CONF_TARGET_TEMPERATURE_MULTIPLIER}" - ) + if CONF_TEMPERATURE_MULTIPLIER in value and ( + CONF_CURRENT_TEMPERATURE_MULTIPLIER in value + or CONF_TARGET_TEMPERATURE_MULTIPLIER in value + ): + raise cv.Invalid( + f"Cannot have {CONF_TEMPERATURE_MULTIPLIER} at the same time as " + f"{CONF_CURRENT_TEMPERATURE_MULTIPLIER} and " + f"{CONF_TARGET_TEMPERATURE_MULTIPLIER}" + ) if ( CONF_CURRENT_TEMPERATURE_MULTIPLIER in value and CONF_TARGET_TEMPERATURE_MULTIPLIER not in value diff --git a/esphome/components/tuya/fan/__init__.py b/esphome/components/tuya/fan/__init__.py index c732bdaf31..de95888b6b 100644 --- a/esphome/components/tuya/fan/__init__.py +++ b/esphome/components/tuya/fan/__init__.py @@ -1,7 +1,7 @@ import esphome.codegen as cg from esphome.components import fan import esphome.config_validation as cv -from esphome.const import CONF_OUTPUT_ID, CONF_SPEED_COUNT, CONF_SWITCH_DATAPOINT +from esphome.const import CONF_ID, CONF_SPEED_COUNT, CONF_SWITCH_DATAPOINT from .. import CONF_TUYA_ID, Tuya, tuya_ns @@ -14,9 +14,9 @@ CONF_DIRECTION_DATAPOINT = "direction_datapoint" TuyaFan = tuya_ns.class_("TuyaFan", cg.Component, fan.Fan) CONFIG_SCHEMA = cv.All( - fan.FAN_SCHEMA.extend( + fan.fan_schema(TuyaFan) + .extend( { - cv.GenerateID(CONF_OUTPUT_ID): cv.declare_id(TuyaFan), cv.GenerateID(CONF_TUYA_ID): cv.use_id(Tuya), cv.Optional(CONF_OSCILLATION_DATAPOINT): cv.uint8_t, cv.Optional(CONF_SPEED_DATAPOINT): cv.uint8_t, @@ -24,7 +24,8 @@ CONFIG_SCHEMA = cv.All( cv.Optional(CONF_DIRECTION_DATAPOINT): cv.uint8_t, cv.Optional(CONF_SPEED_COUNT, default=3): cv.int_range(min=1, max=256), } - ).extend(cv.COMPONENT_SCHEMA), + ) + .extend(cv.COMPONENT_SCHEMA), cv.has_at_least_one_key(CONF_SPEED_DATAPOINT, CONF_SWITCH_DATAPOINT), ) @@ -32,7 +33,7 @@ CONFIG_SCHEMA = cv.All( async def to_code(config): parent = await cg.get_variable(config[CONF_TUYA_ID]) - var = cg.new_Pvariable(config[CONF_OUTPUT_ID], parent, config[CONF_SPEED_COUNT]) + var = cg.new_Pvariable(config[CONF_ID], parent, config[CONF_SPEED_COUNT]) await cg.register_component(var, config) await fan.register_fan(var, config) diff --git a/esphome/components/tuya/number/__init__.py b/esphome/components/tuya/number/__init__.py index bd57c8be14..7a61e69c5c 100644 --- a/esphome/components/tuya/number/__init__.py +++ b/esphome/components/tuya/number/__init__.py @@ -34,12 +34,14 @@ def validate_min_max(config): min_value = config[CONF_MIN_VALUE] if max_value <= min_value: raise cv.Invalid("max_value must be greater than min_value") - if hidden_config := config.get(CONF_DATAPOINT_HIDDEN): - if (initial_value := hidden_config.get(CONF_INITIAL_VALUE, None)) is not None: - if (initial_value > max_value) or (initial_value < min_value): - raise cv.Invalid( - f"{CONF_INITIAL_VALUE} must be a value between {CONF_MAX_VALUE} and {CONF_MIN_VALUE}" - ) + if ( + (hidden_config := config.get(CONF_DATAPOINT_HIDDEN)) + and (initial_value := hidden_config.get(CONF_INITIAL_VALUE, None)) is not None + and ((initial_value > max_value) or (initial_value < min_value)) + ): + raise cv.Invalid( + f"{CONF_INITIAL_VALUE} must be a value between {CONF_MAX_VALUE} and {CONF_MIN_VALUE}" + ) return config diff --git a/esphome/components/tuya/number/tuya_number.cpp b/esphome/components/tuya/number/tuya_number.cpp index 68a7f8f2a7..44b22167de 100644 --- a/esphome/components/tuya/number/tuya_number.cpp +++ b/esphome/components/tuya/number/tuya_number.cpp @@ -8,7 +8,7 @@ static const char *const TAG = "tuya.number"; void TuyaNumber::setup() { if (this->restore_value_) { - this->pref_ = global_preferences->make_preference(this->get_object_id_hash()); + this->pref_ = global_preferences->make_preference(this->get_preference_hash()); } this->parent_->register_listener(this->number_id_, [this](const TuyaDatapoint &datapoint) { diff --git a/esphome/components/tuya/select/tuya_select.cpp b/esphome/components/tuya/select/tuya_select.cpp index 07b0ff2815..91ddbc77ec 100644 --- a/esphome/components/tuya/select/tuya_select.cpp +++ b/esphome/components/tuya/select/tuya_select.cpp @@ -50,7 +50,7 @@ void TuyaSelect::dump_config() { " Options are:", this->select_id_, this->is_int_ ? "int" : "enum"); auto options = this->traits.get_options(); - for (auto i = 0; i < this->mappings_.size(); i++) { + for (size_t i = 0; i < this->mappings_.size(); i++) { ESP_LOGCONFIG(TAG, " %i: %s", this->mappings_.at(i), options.at(i).c_str()); } } diff --git a/esphome/components/tuya/tuya.cpp b/esphome/components/tuya/tuya.cpp index 1443d10254..12b14be9ff 100644 --- a/esphome/components/tuya/tuya.cpp +++ b/esphome/components/tuya/tuya.cpp @@ -215,12 +215,37 @@ void Tuya::handle_command_(uint8_t command, uint8_t version, const uint8_t *buff this->send_empty_command_(TuyaCommandType::DATAPOINT_QUERY); } break; - case TuyaCommandType::WIFI_RESET: - ESP_LOGE(TAG, "WIFI_RESET is not handled"); - break; case TuyaCommandType::WIFI_SELECT: - ESP_LOGE(TAG, "WIFI_SELECT is not handled"); + case TuyaCommandType::WIFI_RESET: { + const bool is_select = (len >= 1); + // Send WIFI_SELECT ACK + TuyaCommand ack; + ack.cmd = is_select ? TuyaCommandType::WIFI_SELECT : TuyaCommandType::WIFI_RESET; + ack.payload.clear(); + this->send_command_(ack); + // Establish pairing mode for correct first WIFI_STATE byte, EZ (0x00) default + uint8_t first = 0x00; + const char *mode_str = "EZ"; + if (is_select && buffer[0] == 0x01) { + first = 0x01; + mode_str = "AP"; + } + // Send WIFI_STATE response, MCU exits pairing mode + TuyaCommand st; + st.cmd = TuyaCommandType::WIFI_STATE; + st.payload.resize(1); + st.payload[0] = first; + this->send_command_(st); + st.payload[0] = 0x02; + this->send_command_(st); + st.payload[0] = 0x03; + this->send_command_(st); + st.payload[0] = 0x04; + this->send_command_(st); + ESP_LOGI(TAG, "%s received (%s), replied with WIFI_STATE confirming connection established", + is_select ? "WIFI_SELECT" : "WIFI_RESET", mode_str); break; + } case TuyaCommandType::DATAPOINT_DELIVER: break; case TuyaCommandType::DATAPOINT_REPORT_ASYNC: diff --git a/esphome/components/tx20/tx20.cpp b/esphome/components/tx20/tx20.cpp index 42e3955fc2..fd7b5fb03f 100644 --- a/esphome/components/tx20/tx20.cpp +++ b/esphome/components/tx20/tx20.cpp @@ -15,7 +15,6 @@ static const char *const DIRECTIONS[] = {"N", "NNE", "NE", "ENE", "E", "ESE", "S "S", "SSW", "SW", "WSW", "W", "WNW", "NW", "NNW"}; void Tx20Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->pin_->setup(); this->store_.buffer = new uint16_t[MAX_BUFFER_SIZE]; diff --git a/esphome/components/uart/__init__.py b/esphome/components/uart/__init__.py index 7d4c6360fe..f8f927d469 100644 --- a/esphome/components/uart/__init__.py +++ b/esphome/components/uart/__init__.py @@ -1,3 +1,4 @@ +import math import re from esphome import automation, pins @@ -14,9 +15,9 @@ from esphome.const import ( CONF_DIRECTION, CONF_DUMMY_RECEIVER, CONF_DUMMY_RECEIVER_ID, + CONF_FLOW_CONTROL_PIN, CONF_ID, CONF_INVERT, - CONF_INVERTED, CONF_LAMBDA, CONF_NUMBER, CONF_PORT, @@ -39,9 +40,6 @@ uart_ns = cg.esphome_ns.namespace("uart") UARTComponent = uart_ns.class_("UARTComponent") IDFUARTComponent = uart_ns.class_("IDFUARTComponent", UARTComponent, cg.Component) -ESP32ArduinoUARTComponent = uart_ns.class_( - "ESP32ArduinoUARTComponent", UARTComponent, cg.Component -) ESP8266UartComponent = uart_ns.class_( "ESP8266UartComponent", UARTComponent, cg.Component ) @@ -53,7 +51,6 @@ HostUartComponent = uart_ns.class_("HostUartComponent", UARTComponent, cg.Compon NATIVE_UART_CLASSES = ( str(IDFUARTComponent), - str(ESP32ArduinoUARTComponent), str(ESP8266UartComponent), str(RP2040UartComponent), str(LibreTinyUARTComponent), @@ -119,20 +116,6 @@ def validate_rx_pin(value): return value -def validate_invert_esp32(config): - if ( - CORE.is_esp32 - and CORE.using_arduino - and CONF_TX_PIN in config - and CONF_RX_PIN in config - and config[CONF_TX_PIN][CONF_INVERTED] != config[CONF_RX_PIN][CONF_INVERTED] - ): - raise cv.Invalid( - "Different invert values for TX and RX pin are not supported for ESP32 when using Arduino." - ) - return config - - def validate_host_config(config): if CORE.is_host: if CONF_TX_PIN in config or CONF_RX_PIN in config: @@ -151,10 +134,7 @@ def _uart_declare_type(value): if CORE.is_esp8266: return cv.declare_id(ESP8266UartComponent)(value) if CORE.is_esp32: - if CORE.using_arduino: - return cv.declare_id(ESP32ArduinoUARTComponent)(value) - if CORE.using_esp_idf: - return cv.declare_id(IDFUARTComponent)(value) + return cv.declare_id(IDFUARTComponent)(value) if CORE.is_rp2040: return cv.declare_id(RP2040UartComponent)(value) if CORE.is_libretiny: @@ -174,6 +154,8 @@ UART_PARITY_OPTIONS = { CONF_STOP_BITS = "stop_bits" CONF_DATA_BITS = "data_bits" CONF_PARITY = "parity" +CONF_RX_FULL_THRESHOLD = "rx_full_threshold" +CONF_RX_TIMEOUT = "rx_timeout" UARTDirection = uart_ns.enum("UARTDirection") UART_DIRECTIONS = { @@ -241,8 +223,17 @@ CONFIG_SCHEMA = cv.All( cv.Required(CONF_BAUD_RATE): cv.int_range(min=1), cv.Optional(CONF_TX_PIN): pins.internal_gpio_output_pin_schema, cv.Optional(CONF_RX_PIN): validate_rx_pin, + cv.Optional(CONF_FLOW_CONTROL_PIN): cv.All( + cv.only_on_esp32, pins.internal_gpio_output_pin_schema + ), cv.Optional(CONF_PORT): cv.All(validate_port, cv.only_on(PLATFORM_HOST)), cv.Optional(CONF_RX_BUFFER_SIZE, default=256): cv.validate_bytes, + cv.Optional(CONF_RX_FULL_THRESHOLD): cv.All( + cv.only_on_esp32, cv.validate_bytes, cv.int_range(min=1, max=120) + ), + cv.SplitDefault(CONF_RX_TIMEOUT, esp32=2): cv.All( + cv.only_on_esp32, cv.validate_bytes, cv.int_range(min=0, max=92) + ), cv.Optional(CONF_STOP_BITS, default=1): cv.one_of(1, 2, int=True), cv.Optional(CONF_DATA_BITS, default=8): cv.int_range(min=5, max=8), cv.Optional(CONF_PARITY, default="NONE"): cv.enum( @@ -255,7 +246,6 @@ CONFIG_SCHEMA = cv.All( } ).extend(cv.COMPONENT_SCHEMA), cv.has_at_least_one_key(CONF_TX_PIN, CONF_RX_PIN, CONF_PORT), - validate_invert_esp32, validate_host_config, ) @@ -298,9 +288,27 @@ async def to_code(config): if CONF_RX_PIN in config: rx_pin = await cg.gpio_pin_expression(config[CONF_RX_PIN]) cg.add(var.set_rx_pin(rx_pin)) + if CONF_FLOW_CONTROL_PIN in config: + flow_control_pin = await cg.gpio_pin_expression(config[CONF_FLOW_CONTROL_PIN]) + cg.add(var.set_flow_control_pin(flow_control_pin)) if CONF_PORT in config: cg.add(var.set_name(config[CONF_PORT])) cg.add(var.set_rx_buffer_size(config[CONF_RX_BUFFER_SIZE])) + if CORE.is_esp32: + if CONF_RX_FULL_THRESHOLD not in config: + # Calculate rx_full_threshold to be 10ms + bytelength = config[CONF_DATA_BITS] + config[CONF_STOP_BITS] + 1 + if config[CONF_PARITY] != "NONE": + bytelength += 1 + config[CONF_RX_FULL_THRESHOLD] = max( + 1, + min( + 120, + math.floor((config[CONF_BAUD_RATE] / (bytelength * 1000 / 10)) - 1), + ), + ) + cg.add(var.set_rx_full_threshold(config[CONF_RX_FULL_THRESHOLD])) + cg.add(var.set_rx_timeout(config[CONF_RX_TIMEOUT])) cg.add(var.set_stop_bits(config[CONF_STOP_BITS])) cg.add(var.set_data_bits(config[CONF_DATA_BITS])) cg.add(var.set_parity(config[CONF_PARITY])) @@ -339,7 +347,7 @@ def final_validate_device_schema( def validate_pin(opt, device): def validator(value): - if opt in device: + if opt in device and not CORE.testing_mode: raise cv.Invalid( f"The uart {opt} is used both by {name} and {device[opt]}, " f"but can only be used by one. Please create a new uart bus for {name}." @@ -444,8 +452,10 @@ async def uart_write_to_code(config, action_id, template_arg, args): FILTER_SOURCE_FILES = filter_source_files_from_platform( { - "uart_component_esp32_arduino.cpp": {PlatformFramework.ESP32_ARDUINO}, - "uart_component_esp_idf.cpp": {PlatformFramework.ESP32_IDF}, + "uart_component_esp_idf.cpp": { + PlatformFramework.ESP32_IDF, + PlatformFramework.ESP32_ARDUINO, + }, "uart_component_esp8266.cpp": {PlatformFramework.ESP8266_ARDUINO}, "uart_component_host.cpp": {PlatformFramework.HOST_NATIVE}, "uart_component_rp2040.cpp": {PlatformFramework.RP2040_ARDUINO}, diff --git a/esphome/components/uart/uart.h b/esphome/components/uart/uart.h index dc6962fbae..e2912db122 100644 --- a/esphome/components/uart/uart.h +++ b/esphome/components/uart/uart.h @@ -18,6 +18,12 @@ class UARTDevice { void write_byte(uint8_t data) { this->parent_->write_byte(data); } + void set_rx_full_threshold(size_t rx_full_threshold) { this->parent_->set_rx_full_threshold(rx_full_threshold); } + void set_rx_full_threshold_ms(size_t time) { this->parent_->set_rx_full_threshold_ms(time); } + size_t get_rx_full_threshold() { return this->parent_->get_rx_full_threshold(); } + void set_rx_timeout(size_t rx_timeout) { this->parent_->set_rx_timeout(rx_timeout); } + size_t get_rx_timeout() { return this->parent_->get_rx_timeout(); } + void write_array(const uint8_t *data, size_t len) { this->parent_->write_array(data, len); } void write_array(const std::vector &data) { this->parent_->write_array(data); } template void write_array(const std::array &data) { diff --git a/esphome/components/uart/uart_component.cpp b/esphome/components/uart/uart_component.cpp index 09b8c975ab..8f670275d4 100644 --- a/esphome/components/uart/uart_component.cpp +++ b/esphome/components/uart/uart_component.cpp @@ -20,5 +20,13 @@ bool UARTComponent::check_read_timeout_(size_t len) { return true; } +void UARTComponent::set_rx_full_threshold_ms(uint8_t time) { + uint8_t bytelength = this->data_bits_ + this->stop_bits_ + 1; + if (this->parity_ != UARTParityOptions::UART_CONFIG_PARITY_NONE) + bytelength += 1; + int32_t val = clamp((this->baud_rate_ / (bytelength * 1000 / time)) - 1, 1, 120); + this->set_rx_full_threshold(val); +} + } // namespace uart } // namespace esphome diff --git a/esphome/components/uart/uart_component.h b/esphome/components/uart/uart_component.h index a57910c1a1..452688b3e9 100644 --- a/esphome/components/uart/uart_component.h +++ b/esphome/components/uart/uart_component.h @@ -6,6 +6,7 @@ #include "esphome/core/component.h" #include "esphome/core/hal.h" #include "esphome/core/log.h" +#include "esphome/core/helpers.h" #ifdef USE_UART_DEBUGGER #include "esphome/core/automation.h" #endif @@ -82,6 +83,10 @@ class UARTComponent { // @param rx_pin Pointer to the internal GPIO pin used for reception. void set_rx_pin(InternalGPIOPin *rx_pin) { this->rx_pin_ = rx_pin; } + // Sets the flow control pin for the UART bus. + // @param flow_control_pin Pointer to the internal GPIO pin used for flow control. + void set_flow_control_pin(InternalGPIOPin *flow_control_pin) { this->flow_control_pin_ = flow_control_pin; } + // Sets the size of the RX buffer. // @param rx_buffer_size Size of the RX buffer in bytes. void set_rx_buffer_size(size_t rx_buffer_size) { this->rx_buffer_size_ = rx_buffer_size; } @@ -90,6 +95,26 @@ class UARTComponent { // @return Size of the RX buffer in bytes. size_t get_rx_buffer_size() { return this->rx_buffer_size_; } + // Sets the RX FIFO full interrupt threshold. + // @param rx_full_threshold RX full interrupt threshold in bytes. + virtual void set_rx_full_threshold(size_t rx_full_threshold) {} + + // Sets the RX FIFO full interrupt threshold. + // @param time RX full interrupt threshold in ms. + void set_rx_full_threshold_ms(uint8_t time); + + // Gets the RX FIFO full interrupt threshold. + // @return RX full interrupt threshold in bytes. + size_t get_rx_full_threshold() { return this->rx_full_threshold_; } + + // Sets the RX timeout interrupt threshold. + // @param rx_timeout RX timeout interrupt threshold (unit: time of sending one byte). + virtual void set_rx_timeout(size_t rx_timeout) {} + + // Gets the RX timeout interrupt threshold. + // @return RX timeout interrupt threshold (unit: time of sending one byte). + size_t get_rx_timeout() { return this->rx_timeout_; } + // Sets the number of stop bits used in UART communication. // @param stop_bits Number of stop bits. void set_stop_bits(uint8_t stop_bits) { this->stop_bits_ = stop_bits; } @@ -161,7 +186,10 @@ class UARTComponent { InternalGPIOPin *tx_pin_; InternalGPIOPin *rx_pin_; + InternalGPIOPin *flow_control_pin_; size_t rx_buffer_size_; + size_t rx_full_threshold_{1}; + size_t rx_timeout_{0}; uint32_t baud_rate_; uint8_t stop_bits_; uint8_t data_bits_; diff --git a/esphome/components/uart/uart_component_esp32_arduino.cpp b/esphome/components/uart/uart_component_esp32_arduino.cpp deleted file mode 100644 index 7441d8c1b3..0000000000 --- a/esphome/components/uart/uart_component_esp32_arduino.cpp +++ /dev/null @@ -1,215 +0,0 @@ -#ifdef USE_ESP32_FRAMEWORK_ARDUINO -#include "uart_component_esp32_arduino.h" -#include "esphome/core/application.h" -#include "esphome/core/defines.h" -#include "esphome/core/helpers.h" -#include "esphome/core/log.h" - -#ifdef USE_LOGGER -#include "esphome/components/logger/logger.h" -#endif - -namespace esphome { -namespace uart { -static const char *const TAG = "uart.arduino_esp32"; - -static const uint32_t UART_PARITY_EVEN = 0 << 0; -static const uint32_t UART_PARITY_ODD = 1 << 0; -static const uint32_t UART_PARITY_ENABLE = 1 << 1; -static const uint32_t UART_NB_BIT_5 = 0 << 2; -static const uint32_t UART_NB_BIT_6 = 1 << 2; -static const uint32_t UART_NB_BIT_7 = 2 << 2; -static const uint32_t UART_NB_BIT_8 = 3 << 2; -static const uint32_t UART_NB_STOP_BIT_1 = 1 << 4; -static const uint32_t UART_NB_STOP_BIT_2 = 3 << 4; -static const uint32_t UART_TICK_APB_CLOCK = 1 << 27; - -uint32_t ESP32ArduinoUARTComponent::get_config() { - uint32_t config = 0; - - /* - * All bits numbers below come from - * framework-arduinoespressif32/cores/esp32/esp32-hal-uart.h - * And more specifically conf0 union in uart_dev_t. - * - * Below is bit used from conf0 union. - * : - * parity:0 0:even 1:odd - * parity_en:1 Set this bit to enable uart parity check. - * bit_num:2-4 0:5bits 1:6bits 2:7bits 3:8bits - * stop_bit_num:4-6 stop bit. 1:1bit 2:1.5bits 3:2bits - * tick_ref_always_on:27 select the clock.1：apb clock：ref_tick - */ - - if (this->parity_ == UART_CONFIG_PARITY_EVEN) { - config |= UART_PARITY_EVEN | UART_PARITY_ENABLE; - } else if (this->parity_ == UART_CONFIG_PARITY_ODD) { - config |= UART_PARITY_ODD | UART_PARITY_ENABLE; - } - - switch (this->data_bits_) { - case 5: - config |= UART_NB_BIT_5; - break; - case 6: - config |= UART_NB_BIT_6; - break; - case 7: - config |= UART_NB_BIT_7; - break; - case 8: - config |= UART_NB_BIT_8; - break; - } - - if (this->stop_bits_ == 1) { - config |= UART_NB_STOP_BIT_1; - } else { - config |= UART_NB_STOP_BIT_2; - } - - config |= UART_TICK_APB_CLOCK; - - return config; -} - -void ESP32ArduinoUARTComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - // Use Arduino HardwareSerial UARTs if all used pins match the ones - // preconfigured by the platform. For example if RX disabled but TX pin - // is 1 we still want to use Serial. - bool is_default_tx, is_default_rx; -#ifdef CONFIG_IDF_TARGET_ESP32C3 - is_default_tx = tx_pin_ == nullptr || tx_pin_->get_pin() == 21; - is_default_rx = rx_pin_ == nullptr || rx_pin_->get_pin() == 20; -#else - is_default_tx = tx_pin_ == nullptr || tx_pin_->get_pin() == 1; - is_default_rx = rx_pin_ == nullptr || rx_pin_->get_pin() == 3; -#endif - static uint8_t next_uart_num = 0; - if (is_default_tx && is_default_rx && next_uart_num == 0) { -#if ARDUINO_USB_CDC_ON_BOOT - this->hw_serial_ = &Serial0; -#else - this->hw_serial_ = &Serial; -#endif - next_uart_num++; - } else { -#ifdef USE_LOGGER - bool logger_uses_hardware_uart = true; - -#ifdef USE_LOGGER_USB_CDC - if (logger::global_logger->get_uart() == logger::UART_SELECTION_USB_CDC) { - // this is not a hardware UART, ignore it - logger_uses_hardware_uart = false; - } -#endif // USE_LOGGER_USB_CDC - -#ifdef USE_LOGGER_USB_SERIAL_JTAG - if (logger::global_logger->get_uart() == logger::UART_SELECTION_USB_SERIAL_JTAG) { - // this is not a hardware UART, ignore it - logger_uses_hardware_uart = false; - } -#endif // USE_LOGGER_USB_SERIAL_JTAG - - if (logger_uses_hardware_uart && logger::global_logger->get_baud_rate() > 0 && - logger::global_logger->get_uart() == next_uart_num) { - next_uart_num++; - } -#endif // USE_LOGGER - - if (next_uart_num >= SOC_UART_NUM) { - ESP_LOGW(TAG, "Maximum number of UART components created already."); - this->mark_failed(); - return; - } - - this->number_ = next_uart_num; - this->hw_serial_ = new HardwareSerial(next_uart_num++); // NOLINT(cppcoreguidelines-owning-memory) - } - - this->load_settings(false); -} - -void ESP32ArduinoUARTComponent::load_settings(bool dump_config) { - int8_t tx = this->tx_pin_ != nullptr ? this->tx_pin_->get_pin() : -1; - int8_t rx = this->rx_pin_ != nullptr ? this->rx_pin_->get_pin() : -1; - bool invert = false; - if (tx_pin_ != nullptr && tx_pin_->is_inverted()) - invert = true; - if (rx_pin_ != nullptr && rx_pin_->is_inverted()) - invert = true; - this->hw_serial_->setRxBufferSize(this->rx_buffer_size_); - this->hw_serial_->begin(this->baud_rate_, get_config(), rx, tx, invert); - if (dump_config) { - ESP_LOGCONFIG(TAG, "UART %u was reloaded.", this->number_); - this->dump_config(); - } -} - -void ESP32ArduinoUARTComponent::dump_config() { - ESP_LOGCONFIG(TAG, "UART Bus %d:", this->number_); - LOG_PIN(" TX Pin: ", tx_pin_); - LOG_PIN(" RX Pin: ", rx_pin_); - if (this->rx_pin_ != nullptr) { - ESP_LOGCONFIG(TAG, " RX Buffer Size: %u", this->rx_buffer_size_); - } - ESP_LOGCONFIG(TAG, - " Baud Rate: %u baud\n" - " Data Bits: %u\n" - " Parity: %s\n" - " Stop bits: %u", - this->baud_rate_, this->data_bits_, LOG_STR_ARG(parity_to_str(this->parity_)), this->stop_bits_); - this->check_logger_conflict(); -} - -void ESP32ArduinoUARTComponent::write_array(const uint8_t *data, size_t len) { - this->hw_serial_->write(data, len); -#ifdef USE_UART_DEBUGGER - for (size_t i = 0; i < len; i++) { - this->debug_callback_.call(UART_DIRECTION_TX, data[i]); - } -#endif -} - -bool ESP32ArduinoUARTComponent::peek_byte(uint8_t *data) { - if (!this->check_read_timeout_()) - return false; - *data = this->hw_serial_->peek(); - return true; -} - -bool ESP32ArduinoUARTComponent::read_array(uint8_t *data, size_t len) { - if (!this->check_read_timeout_(len)) - return false; - this->hw_serial_->readBytes(data, len); -#ifdef USE_UART_DEBUGGER - for (size_t i = 0; i < len; i++) { - this->debug_callback_.call(UART_DIRECTION_RX, data[i]); - } -#endif - return true; -} - -int ESP32ArduinoUARTComponent::available() { return this->hw_serial_->available(); } -void ESP32ArduinoUARTComponent::flush() { - ESP_LOGVV(TAG, " Flushing"); - this->hw_serial_->flush(); -} - -void ESP32ArduinoUARTComponent::check_logger_conflict() { -#ifdef USE_LOGGER - if (this->hw_serial_ == nullptr || logger::global_logger->get_baud_rate() == 0) { - return; - } - - if (this->hw_serial_ == logger::global_logger->get_hw_serial()) { - ESP_LOGW(TAG, " You're using the same serial port for logging and the UART component. Please " - "disable logging over the serial port by setting logger->baud_rate to 0."); - } -#endif -} - -} // namespace uart -} // namespace esphome -#endif // USE_ESP32_FRAMEWORK_ARDUINO diff --git a/esphome/components/uart/uart_component_esp32_arduino.h b/esphome/components/uart/uart_component_esp32_arduino.h deleted file mode 100644 index de17d9718b..0000000000 --- a/esphome/components/uart/uart_component_esp32_arduino.h +++ /dev/null @@ -1,60 +0,0 @@ -#pragma once - -#ifdef USE_ESP32_FRAMEWORK_ARDUINO - -#include -#include -#include -#include "esphome/core/component.h" -#include "esphome/core/hal.h" -#include "esphome/core/log.h" -#include "uart_component.h" - -namespace esphome { -namespace uart { - -class ESP32ArduinoUARTComponent : public UARTComponent, public Component { - public: - void setup() override; - void dump_config() override; - float get_setup_priority() const override { return setup_priority::BUS; } - - void write_array(const uint8_t *data, size_t len) override; - - bool peek_byte(uint8_t *data) override; - bool read_array(uint8_t *data, size_t len) override; - - int available() override; - void flush() override; - - uint32_t get_config(); - - HardwareSerial *get_hw_serial() { return this->hw_serial_; } - uint8_t get_hw_serial_number() { return this->number_; } - - /** - * Load the UART with the current settings. - * @param dump_config (Optional, default `true`): True for displaying new settings or - * false to change it quitely - * - * Example: - * ```cpp - * id(uart1).load_settings(); - * ``` - * - * This will load the current UART interface with the latest settings (baud_rate, parity, etc). - */ - void load_settings(bool dump_config) override; - void load_settings() override { this->load_settings(true); } - - protected: - void check_logger_conflict() override; - - HardwareSerial *hw_serial_{nullptr}; - uint8_t number_{0}; -}; - -} // namespace uart -} // namespace esphome - -#endif // USE_ESP32_FRAMEWORK_ARDUINO diff --git a/esphome/components/uart/uart_component_esp8266.cpp b/esphome/components/uart/uart_component_esp8266.cpp index 7f4cc7b37c..b2bf2bacf1 100644 --- a/esphome/components/uart/uart_component_esp8266.cpp +++ b/esphome/components/uart/uart_component_esp8266.cpp @@ -56,7 +56,6 @@ uint32_t ESP8266UartComponent::get_config() { } void ESP8266UartComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); // Use Arduino HardwareSerial UARTs if all used pins match the ones // preconfigured by the platform. For example if RX disabled but TX pin // is 1 we still want to use Serial. diff --git a/esphome/components/uart/uart_component_esp_idf.cpp b/esphome/components/uart/uart_component_esp_idf.cpp index 63b2579c3f..7530856b1e 100644 --- a/esphome/components/uart/uart_component_esp_idf.cpp +++ b/esphome/components/uart/uart_component_esp_idf.cpp @@ -1,4 +1,4 @@ -#ifdef USE_ESP_IDF +#ifdef USE_ESP32 #include "uart_component_esp_idf.h" #include @@ -86,12 +86,16 @@ void IDFUARTComponent::setup() { return; } this->uart_num_ = static_cast(next_uart_num++); - ESP_LOGCONFIG(TAG, "Running setup for UART %u", this->uart_num_); - this->lock_ = xSemaphoreCreateMutex(); xSemaphoreTake(this->lock_, portMAX_DELAY); + this->load_settings(false); + + xSemaphoreGive(this->lock_); +} + +void IDFUARTComponent::load_settings(bool dump_config) { uart_config_t uart_config = this->get_config_(); esp_err_t err = uart_param_config(this->uart_num_, &uart_config); if (err != ESP_OK) { @@ -102,6 +106,7 @@ void IDFUARTComponent::setup() { int8_t tx = this->tx_pin_ != nullptr ? this->tx_pin_->get_pin() : -1; int8_t rx = this->rx_pin_ != nullptr ? this->rx_pin_->get_pin() : -1; + int8_t flow_control = this->flow_control_pin_ != nullptr ? this->flow_control_pin_->get_pin() : -1; uint32_t invert = 0; if (this->tx_pin_ != nullptr && this->tx_pin_->is_inverted()) @@ -116,13 +121,21 @@ void IDFUARTComponent::setup() { return; } - err = uart_set_pin(this->uart_num_, tx, rx, UART_PIN_NO_CHANGE, UART_PIN_NO_CHANGE); + err = uart_set_pin(this->uart_num_, tx, rx, flow_control, UART_PIN_NO_CHANGE); if (err != ESP_OK) { ESP_LOGW(TAG, "uart_set_pin failed: %s", esp_err_to_name(err)); this->mark_failed(); return; } + if (uart_is_driver_installed(this->uart_num_)) { + uart_driver_delete(this->uart_num_); + if (err != ESP_OK) { + ESP_LOGW(TAG, "uart_driver_delete failed: %s", esp_err_to_name(err)); + this->mark_failed(); + return; + } + } err = uart_driver_install(this->uart_num_, /* UART RX ring buffer size. */ this->rx_buffer_size_, /* UART TX ring buffer size. If set to zero, driver will not use TX buffer, TX function will block task until all data have been sent out.*/ @@ -135,17 +148,29 @@ void IDFUARTComponent::setup() { return; } - xSemaphoreGive(this->lock_); -} - -void IDFUARTComponent::load_settings(bool dump_config) { - uart_config_t uart_config = this->get_config_(); - esp_err_t err = uart_param_config(this->uart_num_, &uart_config); + err = uart_set_rx_full_threshold(this->uart_num_, this->rx_full_threshold_); if (err != ESP_OK) { - ESP_LOGW(TAG, "uart_param_config failed: %s", esp_err_to_name(err)); + ESP_LOGW(TAG, "uart_set_rx_full_threshold failed: %s", esp_err_to_name(err)); this->mark_failed(); return; - } else if (dump_config) { + } + + err = uart_set_rx_timeout(this->uart_num_, this->rx_timeout_); + if (err != ESP_OK) { + ESP_LOGW(TAG, "uart_set_rx_timeout failed: %s", esp_err_to_name(err)); + this->mark_failed(); + return; + } + + auto mode = this->flow_control_pin_ != nullptr ? UART_MODE_RS485_HALF_DUPLEX : UART_MODE_UART; + err = uart_set_mode(this->uart_num_, mode); + if (err != ESP_OK) { + ESP_LOGW(TAG, "uart_set_mode failed: %s", esp_err_to_name(err)); + this->mark_failed(); + return; + } + + if (dump_config) { ESP_LOGCONFIG(TAG, "UART %u was reloaded.", this->uart_num_); this->dump_config(); } @@ -155,8 +180,13 @@ void IDFUARTComponent::dump_config() { ESP_LOGCONFIG(TAG, "UART Bus %u:", this->uart_num_); LOG_PIN(" TX Pin: ", tx_pin_); LOG_PIN(" RX Pin: ", rx_pin_); + LOG_PIN(" Flow Control Pin: ", flow_control_pin_); if (this->rx_pin_ != nullptr) { - ESP_LOGCONFIG(TAG, " RX Buffer Size: %u", this->rx_buffer_size_); + ESP_LOGCONFIG(TAG, + " RX Buffer Size: %u\n" + " RX Full Threshold: %u\n" + " RX Timeout: %u", + this->rx_buffer_size_, this->rx_full_threshold_, this->rx_timeout_); } ESP_LOGCONFIG(TAG, " Baud Rate: %" PRIu32 " baud\n" @@ -167,6 +197,28 @@ void IDFUARTComponent::dump_config() { this->check_logger_conflict(); } +void IDFUARTComponent::set_rx_full_threshold(size_t rx_full_threshold) { + if (this->is_ready()) { + esp_err_t err = uart_set_rx_full_threshold(this->uart_num_, rx_full_threshold); + if (err != ESP_OK) { + ESP_LOGW(TAG, "uart_set_rx_full_threshold failed: %s", esp_err_to_name(err)); + return; + } + } + this->rx_full_threshold_ = rx_full_threshold; +} + +void IDFUARTComponent::set_rx_timeout(size_t rx_timeout) { + if (this->is_ready()) { + esp_err_t err = uart_set_rx_timeout(this->uart_num_, rx_timeout); + if (err != ESP_OK) { + ESP_LOGW(TAG, "uart_set_rx_timeout failed: %s", esp_err_to_name(err)); + return; + } + } + this->rx_timeout_ = rx_timeout; +} + void IDFUARTComponent::write_array(const uint8_t *data, size_t len) { xSemaphoreTake(this->lock_, portMAX_DELAY); uart_write_bytes(this->uart_num_, data, len); diff --git a/esphome/components/uart/uart_component_esp_idf.h b/esphome/components/uart/uart_component_esp_idf.h index 215641ebe2..a2ba2aa968 100644 --- a/esphome/components/uart/uart_component_esp_idf.h +++ b/esphome/components/uart/uart_component_esp_idf.h @@ -1,6 +1,6 @@ #pragma once -#ifdef USE_ESP_IDF +#ifdef USE_ESP32 #include #include "esphome/core/component.h" @@ -15,6 +15,9 @@ class IDFUARTComponent : public UARTComponent, public Component { void dump_config() override; float get_setup_priority() const override { return setup_priority::BUS; } + void set_rx_full_threshold(size_t rx_full_threshold) override; + void set_rx_timeout(size_t rx_timeout) override; + void write_array(const uint8_t *data, size_t len) override; bool peek_byte(uint8_t *data) override; @@ -55,4 +58,4 @@ class IDFUARTComponent : public UARTComponent, public Component { } // namespace uart } // namespace esphome -#endif // USE_ESP_IDF +#endif // USE_ESP32 diff --git a/esphome/components/uart/uart_component_libretiny.cpp b/esphome/components/uart/uart_component_libretiny.cpp index ffdb329669..8a7a301cfe 100644 --- a/esphome/components/uart/uart_component_libretiny.cpp +++ b/esphome/components/uart/uart_component_libretiny.cpp @@ -46,8 +46,6 @@ uint16_t LibreTinyUARTComponent::get_config() { } void LibreTinyUARTComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - int8_t tx_pin = tx_pin_ == nullptr ? -1 : tx_pin_->get_pin(); int8_t rx_pin = rx_pin_ == nullptr ? -1 : rx_pin_->get_pin(); bool tx_inverted = tx_pin_ != nullptr && tx_pin_->is_inverted(); diff --git a/esphome/components/uart/uart_component_rp2040.cpp b/esphome/components/uart/uart_component_rp2040.cpp index f375d4a93f..ae3042fb77 100644 --- a/esphome/components/uart/uart_component_rp2040.cpp +++ b/esphome/components/uart/uart_component_rp2040.cpp @@ -52,8 +52,6 @@ uint16_t RP2040UartComponent::get_config() { } void RP2040UartComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - uint16_t config = get_config(); constexpr uint32_t valid_tx_uart_0 = __bitset({0, 12, 16, 28}); diff --git a/esphome/components/udp/__init__.py b/esphome/components/udp/__init__.py index ed405d7c22..6b1e4f8ed8 100644 --- a/esphome/components/udp/__init__.py +++ b/esphome/components/udp/__init__.py @@ -1,6 +1,7 @@ from esphome import automation from esphome.automation import Trigger import esphome.codegen as cg +from esphome.components.const import CONF_ON_RECEIVE from esphome.components.packet_transport import ( CONF_BINARY_SENSORS, CONF_ENCRYPTION, @@ -27,7 +28,6 @@ trigger_args = cg.std_vector.template(cg.uint8) CONF_ADDRESSES = "addresses" CONF_LISTEN_ADDRESS = "listen_address" CONF_UDP_ID = "udp_id" -CONF_ON_RECEIVE = "on_receive" CONF_LISTEN_PORT = "listen_port" CONF_BROADCAST_PORT = "broadcast_port" diff --git a/esphome/components/udp/udp_component.cpp b/esphome/components/udp/udp_component.cpp index 62a1189355..8a9ce612b4 100644 --- a/esphome/components/udp/udp_component.cpp +++ b/esphome/components/udp/udp_component.cpp @@ -28,12 +28,12 @@ void UDPComponent::setup() { int enable = 1; auto err = this->broadcast_socket_->setsockopt(SOL_SOCKET, SO_REUSEADDR, &enable, sizeof(int)); if (err != 0) { - this->status_set_warning("Socket unable to set reuseaddr"); + this->status_set_warning(LOG_STR("Socket unable to set reuseaddr")); // we can still continue } err = this->broadcast_socket_->setsockopt(SOL_SOCKET, SO_BROADCAST, &enable, sizeof(int)); if (err != 0) { - this->status_set_warning("Socket unable to set broadcast"); + this->status_set_warning(LOG_STR("Socket unable to set broadcast")); } } // create listening socket if we either want to subscribe to providers, or need to listen @@ -55,7 +55,7 @@ void UDPComponent::setup() { int enable = 1; err = this->listen_socket_->setsockopt(SOL_SOCKET, SO_REUSEADDR, &enable, sizeof(enable)); if (err != 0) { - this->status_set_warning("Socket unable to set reuseaddr"); + this->status_set_warning(LOG_STR("Socket unable to set reuseaddr")); // we can still continue } struct sockaddr_in server {}; diff --git a/esphome/components/ufire_ec/sensor.py b/esphome/components/ufire_ec/sensor.py index 944fdfdee9..9edf0f89ff 100644 --- a/esphome/components/ufire_ec/sensor.py +++ b/esphome/components/ufire_ec/sensor.py @@ -122,5 +122,4 @@ UFIRE_EC_RESET_SCHEMA = cv.Schema( ) async def ufire_ec_reset_to_code(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + return cg.new_Pvariable(action_id, template_arg, paren) diff --git a/esphome/components/ufire_ec/ufire_ec.cpp b/esphome/components/ufire_ec/ufire_ec.cpp index 813e667a00..0a57ecc67b 100644 --- a/esphome/components/ufire_ec/ufire_ec.cpp +++ b/esphome/components/ufire_ec/ufire_ec.cpp @@ -7,8 +7,6 @@ namespace ufire_ec { static const char *const TAG = "ufire_ec"; void UFireECComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - uint8_t version; if (!this->read_byte(REGISTER_VERSION, &version) && version != 0xFF) { this->mark_failed(); @@ -106,10 +104,10 @@ void UFireECComponent::write_data_(uint8_t reg, float data) { void UFireECComponent::dump_config() { ESP_LOGCONFIG(TAG, "uFire-EC"); LOG_I2C_DEVICE(this) - LOG_UPDATE_INTERVAL(this) - LOG_SENSOR(" ", "EC Sensor", this->ec_sensor_) - LOG_SENSOR(" ", "Temperature Sensor", this->temperature_sensor_) - LOG_SENSOR(" ", "Temperature Sensor external", this->temperature_sensor_external_) + LOG_UPDATE_INTERVAL(this); + LOG_SENSOR(" ", "EC Sensor", this->ec_sensor_); + LOG_SENSOR(" ", "Temperature Sensor", this->temperature_sensor_); + LOG_SENSOR(" ", "Temperature Sensor external", this->temperature_sensor_external_); ESP_LOGCONFIG(TAG, " Temperature Compensation: %f\n" " Temperature Coefficient: %f", diff --git a/esphome/components/ufire_ise/sensor.py b/esphome/components/ufire_ise/sensor.py index e57a1155a4..8009cdaa6a 100644 --- a/esphome/components/ufire_ise/sensor.py +++ b/esphome/components/ufire_ise/sensor.py @@ -123,5 +123,4 @@ UFIRE_ISE_RESET_SCHEMA = cv.Schema({cv.GenerateID(): cv.use_id(UFireISEComponent ) async def ufire_ise_reset_to_code(config, action_id, template_arg, args): paren = await cg.get_variable(config[CONF_ID]) - var = cg.new_Pvariable(action_id, template_arg, paren) - return var + return cg.new_Pvariable(action_id, template_arg, paren) diff --git a/esphome/components/ufire_ise/ufire_ise.cpp b/esphome/components/ufire_ise/ufire_ise.cpp index 5d0cb6ec2f..486a506391 100644 --- a/esphome/components/ufire_ise/ufire_ise.cpp +++ b/esphome/components/ufire_ise/ufire_ise.cpp @@ -9,8 +9,6 @@ namespace ufire_ise { static const char *const TAG = "ufire_ise"; void UFireISEComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - uint8_t version; if (!this->read_byte(REGISTER_VERSION, &version) && version != 0xFF) { this->mark_failed(); @@ -143,10 +141,10 @@ void UFireISEComponent::write_data_(uint8_t reg, float data) { void UFireISEComponent::dump_config() { ESP_LOGCONFIG(TAG, "uFire-ISE"); LOG_I2C_DEVICE(this) - LOG_UPDATE_INTERVAL(this) - LOG_SENSOR(" ", "PH Sensor", this->ph_sensor_) - LOG_SENSOR(" ", "Temperature Sensor", this->temperature_sensor_) - LOG_SENSOR(" ", "Temperature Sensor external", this->temperature_sensor_external_) + LOG_UPDATE_INTERVAL(this); + LOG_SENSOR(" ", "PH Sensor", this->ph_sensor_); + LOG_SENSOR(" ", "Temperature Sensor", this->temperature_sensor_); + LOG_SENSOR(" ", "Temperature Sensor external", this->temperature_sensor_external_); } } // namespace ufire_ise diff --git a/esphome/components/ultrasonic/ultrasonic_sensor.cpp b/esphome/components/ultrasonic/ultrasonic_sensor.cpp index b737dfa4cd..e864ea6419 100644 --- a/esphome/components/ultrasonic/ultrasonic_sensor.cpp +++ b/esphome/components/ultrasonic/ultrasonic_sensor.cpp @@ -8,7 +8,6 @@ namespace ultrasonic { static const char *const TAG = "ultrasonic.sensor"; void UltrasonicSensorComponent::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->trigger_pin_->setup(); this->trigger_pin_->digital_write(false); this->echo_pin_->setup(); diff --git a/esphome/components/update/__init__.py b/esphome/components/update/__init__.py index 758267f412..35fc4eaf1d 100644 --- a/esphome/components/update/__init__.py +++ b/esphome/components/update/__init__.py @@ -14,7 +14,7 @@ from esphome.const import ( DEVICE_CLASS_FIRMWARE, ENTITY_CATEGORY_CONFIG, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass @@ -124,9 +124,8 @@ async def new_update(config): return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_UPDATE") cg.add_global(update_ns.using) diff --git a/esphome/components/uponor_smatrix/climate/uponor_smatrix_climate.cpp b/esphome/components/uponor_smatrix/climate/uponor_smatrix_climate.cpp index d7e672d8cf..19a9112c73 100644 --- a/esphome/components/uponor_smatrix/climate/uponor_smatrix_climate.cpp +++ b/esphome/components/uponor_smatrix/climate/uponor_smatrix_climate.cpp @@ -58,7 +58,7 @@ void UponorSmatrixClimate::control(const climate::ClimateCall &call) { } void UponorSmatrixClimate::on_device_data(const UponorSmatrixData *data, size_t data_len) { - for (int i = 0; i < data_len; i++) { + for (size_t i = 0; i < data_len; i++) { switch (data[i].id) { case UPONOR_ID_TARGET_TEMP_MIN: this->min_temperature_ = raw_to_celsius(data[i].value); diff --git a/esphome/components/uponor_smatrix/sensor/uponor_smatrix_sensor.cpp b/esphome/components/uponor_smatrix/sensor/uponor_smatrix_sensor.cpp index 452660dc14..a1d0db214f 100644 --- a/esphome/components/uponor_smatrix/sensor/uponor_smatrix_sensor.cpp +++ b/esphome/components/uponor_smatrix/sensor/uponor_smatrix_sensor.cpp @@ -18,7 +18,7 @@ void UponorSmatrixSensor::dump_config() { } void UponorSmatrixSensor::on_device_data(const UponorSmatrixData *data, size_t data_len) { - for (int i = 0; i < data_len; i++) { + for (size_t i = 0; i < data_len; i++) { switch (data[i].id) { case UPONOR_ID_ROOM_TEMP: if (this->temperature_sensor_ != nullptr) diff --git a/esphome/components/uponor_smatrix/uponor_smatrix.cpp b/esphome/components/uponor_smatrix/uponor_smatrix.cpp index a0017518bf..867305059f 100644 --- a/esphome/components/uponor_smatrix/uponor_smatrix.cpp +++ b/esphome/components/uponor_smatrix/uponor_smatrix.cpp @@ -122,7 +122,7 @@ bool UponorSmatrixComponent::parse_byte_(uint8_t byte) { // Decode packet payload data for easy access UponorSmatrixData data[data_len]; - for (int i = 0; i < data_len; i++) { + for (size_t i = 0; i < data_len; i++) { data[i].id = packet[(i * 3) + 4]; data[i].value = encode_uint16(packet[(i * 3) + 5], packet[(i * 3) + 6]); } @@ -135,7 +135,7 @@ bool UponorSmatrixComponent::parse_byte_(uint8_t byte) { // thermostat sending both room temperature and time information. bool found_temperature = false; bool found_time = false; - for (int i = 0; i < data_len; i++) { + for (size_t i = 0; i < data_len; i++) { if (data[i].id == UPONOR_ID_ROOM_TEMP) found_temperature = true; if (data[i].id == UPONOR_ID_DATETIME1) @@ -181,7 +181,7 @@ bool UponorSmatrixComponent::send(uint16_t device_address, const UponorSmatrixDa packet.push_back(device_address >> 8); packet.push_back(device_address >> 0); - for (int i = 0; i < data_len; i++) { + for (size_t i = 0; i < data_len; i++) { packet.push_back(data[i].id); packet.push_back(data[i].value >> 8); packet.push_back(data[i].value >> 0); diff --git a/esphome/components/uptime/sensor/uptime_seconds_sensor.cpp b/esphome/components/uptime/sensor/uptime_seconds_sensor.cpp index fa6b9d621d..54260d7e80 100644 --- a/esphome/components/uptime/sensor/uptime_seconds_sensor.cpp +++ b/esphome/components/uptime/sensor/uptime_seconds_sensor.cpp @@ -27,7 +27,6 @@ void UptimeSecondsSensor::update() { const float seconds = float(seconds_int) + (this->uptime_ % 1000ULL) / 1000.0f; this->publish_state(seconds); } -std::string UptimeSecondsSensor::unique_id() { return get_mac_address() + "-uptime"; } float UptimeSecondsSensor::get_setup_priority() const { return setup_priority::HARDWARE; } void UptimeSecondsSensor::dump_config() { LOG_SENSOR("", "Uptime Sensor", this); diff --git a/esphome/components/uptime/sensor/uptime_seconds_sensor.h b/esphome/components/uptime/sensor/uptime_seconds_sensor.h index 41b3647822..210195052f 100644 --- a/esphome/components/uptime/sensor/uptime_seconds_sensor.h +++ b/esphome/components/uptime/sensor/uptime_seconds_sensor.h @@ -13,8 +13,6 @@ class UptimeSecondsSensor : public sensor::Sensor, public PollingComponent { float get_setup_priority() const override; - std::string unique_id() override; - protected: uint64_t uptime_{0}; }; diff --git a/esphome/components/usb_host/__init__.py b/esphome/components/usb_host/__init__.py index 0fe3310127..d452e0e9fa 100644 --- a/esphome/components/usb_host/__init__.py +++ b/esphome/components/usb_host/__init__.py @@ -1,5 +1,6 @@ import esphome.codegen as cg from esphome.components.esp32 import ( + VARIANT_ESP32P4, VARIANT_ESP32S2, VARIANT_ESP32S3, add_idf_sdkconfig_option, @@ -8,6 +9,7 @@ from esphome.components.esp32 import ( import esphome.config_validation as cv from esphome.const import CONF_DEVICES, CONF_ID from esphome.cpp_types import Component +from esphome.types import ConfigType AUTO_LOAD = ["bytebuffer"] CODEOWNERS = ["@clydebarrow"] @@ -19,6 +21,7 @@ USBClient = usb_host_ns.class_("USBClient", Component) CONF_VID = "vid" CONF_PID = "pid" CONF_ENABLE_HUBS = "enable_hubs" +CONF_MAX_TRANSFER_REQUESTS = "max_transfer_requests" def usb_device_schema(cls=USBClient, vid: int = None, pid: [int] = None) -> cv.Schema: @@ -43,11 +46,14 @@ CONFIG_SCHEMA = cv.All( { cv.GenerateID(): cv.declare_id(USBHost), cv.Optional(CONF_ENABLE_HUBS, default=False): cv.boolean, + cv.Optional(CONF_MAX_TRANSFER_REQUESTS, default=16): cv.int_range( + min=1, max=32 + ), cv.Optional(CONF_DEVICES): cv.ensure_list(usb_device_schema()), } ), cv.only_with_esp_idf, - only_on_variant(supported=[VARIANT_ESP32S2, VARIANT_ESP32S3]), + only_on_variant(supported=[VARIANT_ESP32S2, VARIANT_ESP32S3, VARIANT_ESP32P4]), ) @@ -57,10 +63,14 @@ async def register_usb_client(config): return var -async def to_code(config): +async def to_code(config: ConfigType) -> None: add_idf_sdkconfig_option("CONFIG_USB_HOST_CONTROL_TRANSFER_MAX_SIZE", 1024) if config.get(CONF_ENABLE_HUBS): add_idf_sdkconfig_option("CONFIG_USB_HOST_HUBS_SUPPORTED", True) + + max_requests = config[CONF_MAX_TRANSFER_REQUESTS] + cg.add_define("USB_HOST_MAX_REQUESTS", max_requests) + var = cg.new_Pvariable(config[CONF_ID]) await cg.register_component(var, config) for device in config.get(CONF_DEVICES) or (): diff --git a/esphome/components/usb_host/usb_host.h b/esphome/components/usb_host/usb_host.h index c5466eb1f0..43b24a54a5 100644 --- a/esphome/components/usb_host/usb_host.h +++ b/esphome/components/usb_host/usb_host.h @@ -1,18 +1,48 @@ #pragma once // Should not be needed, but it's required to pass CI clang-tidy checks -#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) +#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) || defined(USE_ESP32_VARIANT_ESP32P4) +#include "esphome/core/defines.h" #include "esphome/core/component.h" #include #include "usb/usb_host.h" - -#include +#include +#include +#include "esphome/core/lock_free_queue.h" +#include "esphome/core/event_pool.h" +#include namespace esphome { namespace usb_host { +// THREADING MODEL: +// This component uses a dedicated USB task for event processing to prevent data loss. +// - USB Task (high priority): Handles USB events, executes transfer callbacks, releases transfer slots +// - Main Loop Task: Initiates transfers, processes device connect/disconnect events +// +// Thread-safe communication: +// - Lock-free queues for USB task -> main loop events (SPSC pattern) +// - Lock-free TransferRequest pool using atomic bitmask (MCMP pattern - multi-consumer, multi-producer) +// +// TransferRequest pool access pattern: +// - get_trq_() [allocate]: Called from BOTH USB task and main loop threads +// * USB task: via USB UART input callbacks that restart transfers immediately +// * Main loop: for output transfers and flow-controlled input restarts +// - release_trq() [deallocate]: Called from BOTH USB task and main loop threads +// * USB task: immediately after transfer callback completes (critical for preventing slot exhaustion) +// * Main loop: when transfer submission fails +// +// The multi-threaded allocation/deallocation is intentional for performance: +// - USB task can immediately restart input transfers and release slots without context switching +// - Main loop controls backpressure by deciding when to restart after consuming data +// The atomic bitmask ensures thread-safe allocation/deallocation without mutex blocking. + static const char *const TAG = "usb_host"; +// Forward declarations +struct TransferRequest; +class USBClient; + // constants for setup packet type static const uint8_t USB_RECIP_DEVICE = 0; static const uint8_t USB_RECIP_INTERFACE = 1; @@ -25,7 +55,20 @@ static const uint8_t USB_DIR_IN = 1 << 7; static const uint8_t USB_DIR_OUT = 0; static const size_t SETUP_PACKET_SIZE = 8; -static const size_t MAX_REQUESTS = 16; // maximum number of outstanding requests possible. +static const size_t MAX_REQUESTS = USB_HOST_MAX_REQUESTS; // maximum number of outstanding requests possible. +static_assert(MAX_REQUESTS >= 1 && MAX_REQUESTS <= 32, "MAX_REQUESTS must be between 1 and 32"); + +// Select appropriate bitmask type for tracking allocation of TransferRequest slots. +// The bitmask must have at least as many bits as MAX_REQUESTS, so: +// - Use uint16_t for up to 16 requests (MAX_REQUESTS <= 16) +// - Use uint32_t for 17-32 requests (MAX_REQUESTS > 16) +// This is tied to the static_assert above, which enforces MAX_REQUESTS is between 1 and 32. +// If MAX_REQUESTS is increased above 32, this logic and the static_assert must be updated. +using trq_bitmask_t = std::conditional<(MAX_REQUESTS <= 16), uint16_t, uint32_t>::type; + +static constexpr size_t USB_EVENT_QUEUE_SIZE = 32; // Size of event queue between USB task and main loop +static constexpr size_t USB_TASK_STACK_SIZE = 4096; // Stack size for USB task (same as ESP-IDF USB examples) +static constexpr UBaseType_t USB_TASK_PRIORITY = 5; // Higher priority than main loop (tskIDLE_PRIORITY + 5) // used to report a transfer status struct TransferStatus { @@ -49,6 +92,26 @@ struct TransferRequest { USBClient *client; }; +enum EventType : uint8_t { + EVENT_DEVICE_NEW, + EVENT_DEVICE_GONE, +}; + +struct UsbEvent { + EventType type; + union { + struct { + uint8_t address; + } device_new; + struct { + usb_device_handle_t handle; + } device_gone; + } data; + + // Required for EventPool - no cleanup needed for POD types + void release() {} +}; + // callback function type. enum ClientState { @@ -63,13 +126,7 @@ class USBClient : public Component { friend class USBHost; public: - USBClient(uint16_t vid, uint16_t pid) : vid_(vid), pid_(pid) { init_pool(); } - - void init_pool() { - this->trq_pool_.clear(); - for (size_t i = 0; i != MAX_REQUESTS; i++) - this->trq_pool_.push_back(&this->requests_[i]); - } + USBClient(uint16_t vid, uint16_t pid) : vid_(vid), pid_(pid), trq_in_use_(0) {} void setup() override; void loop() override; // setup must happen after the host bus has been setup @@ -84,12 +141,26 @@ class USBClient : public Component { bool control_transfer(uint8_t type, uint8_t request, uint16_t value, uint16_t index, const transfer_cb_t &callback, const std::vector &data = {}); + // Lock-free event queue and pool for USB task to main loop communication + // Must be public for access from static callbacks + LockFreeQueue event_queue; + EventPool event_pool; + protected: bool register_(); - TransferRequest *get_trq_(); + TransferRequest *get_trq_(); // Lock-free allocation using atomic bitmask (multi-consumer safe) virtual void disconnect(); virtual void on_connected() {} - virtual void on_disconnected() { this->init_pool(); } + virtual void on_disconnected() { + // Reset all requests to available (all bits to 0) + this->trq_in_use_.store(0); + } + + // USB task management + static void usb_task_fn(void *arg); + void usb_task_loop(); + + TaskHandle_t usb_task_handle_{nullptr}; usb_host_client_handle_t handle_{}; usb_device_handle_t device_handle_{}; @@ -97,7 +168,11 @@ class USBClient : public Component { int state_{USB_CLIENT_INIT}; uint16_t vid_{}; uint16_t pid_{}; - std::list trq_pool_{}; + // Lock-free pool management using atomic bitmask (no dynamic allocation) + // Bit i = 1: requests_[i] is in use, Bit i = 0: requests_[i] is available + // Supports multiple concurrent consumers and producers (both threads can allocate/deallocate) + // Bitmask type automatically selected: uint16_t for <= 16 slots, uint32_t for 17-32 slots + std::atomic trq_in_use_; TransferRequest requests_[MAX_REQUESTS]{}; }; class USBHost : public Component { diff --git a/esphome/components/usb_host/usb_host_client.cpp b/esphome/components/usb_host/usb_host_client.cpp index edf6c94b07..2139ed869a 100644 --- a/esphome/components/usb_host/usb_host_client.cpp +++ b/esphome/components/usb_host/usb_host_client.cpp @@ -1,5 +1,5 @@ // Should not be needed, but it's required to pass CI clang-tidy checks -#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) +#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) || defined(USE_ESP32_VARIANT_ESP32P4) #include "usb_host.h" #include "esphome/core/log.h" #include "esphome/core/hal.h" @@ -7,6 +7,7 @@ #include #include +#include namespace esphome { namespace usb_host { @@ -139,24 +140,40 @@ static std::string get_descriptor_string(const usb_str_desc_t *desc) { return {buffer}; } +// CALLBACK CONTEXT: USB task (called from usb_host_client_handle_events in USB task) static void client_event_cb(const usb_host_client_event_msg_t *event_msg, void *ptr) { auto *client = static_cast(ptr); + + // Allocate event from pool + UsbEvent *event = client->event_pool.allocate(); + if (event == nullptr) { + // No events available - increment counter for periodic logging + client->event_queue.increment_dropped_count(); + return; + } + + // Queue events to be processed in main loop switch (event_msg->event) { case USB_HOST_CLIENT_EVENT_NEW_DEV: { - auto addr = event_msg->new_dev.address; ESP_LOGD(TAG, "New device %d", event_msg->new_dev.address); - client->on_opened(addr); + event->type = EVENT_DEVICE_NEW; + event->data.device_new.address = event_msg->new_dev.address; break; } case USB_HOST_CLIENT_EVENT_DEV_GONE: { - client->on_removed(event_msg->dev_gone.dev_hdl); - ESP_LOGD(TAG, "Device gone %d", event_msg->new_dev.address); + ESP_LOGD(TAG, "Device gone"); + event->type = EVENT_DEVICE_GONE; + event->data.device_gone.handle = event_msg->dev_gone.dev_hdl; break; } default: ESP_LOGD(TAG, "Unknown event %d", event_msg->event); - break; + client->event_pool.release(event); + return; } + + // Push to lock-free queue (always succeeds since pool size == queue size) + client->event_queue.push(event); } void USBClient::setup() { usb_host_client_config_t config{.is_synchronous = false, @@ -169,14 +186,59 @@ void USBClient::setup() { this->mark_failed(); return; } - for (auto *trq : this->trq_pool_) { - usb_host_transfer_alloc(64, 0, &trq->transfer); - trq->client = this; + // Pre-allocate USB transfer buffers for all slots at startup + // This avoids any dynamic allocation during runtime + for (size_t i = 0; i < MAX_REQUESTS; i++) { + usb_host_transfer_alloc(64, 0, &this->requests_[i].transfer); + this->requests_[i].client = this; // Set once, never changes + } + + // Create and start USB task + xTaskCreate(usb_task_fn, "usb_task", + USB_TASK_STACK_SIZE, // Stack size + this, // Task parameter + USB_TASK_PRIORITY, // Priority (higher than main loop) + &this->usb_task_handle_); + + if (this->usb_task_handle_ == nullptr) { + ESP_LOGE(TAG, "Failed to create USB task"); + this->mark_failed(); + } +} + +void USBClient::usb_task_fn(void *arg) { + auto *client = static_cast(arg); + client->usb_task_loop(); +} + +void USBClient::usb_task_loop() { + while (true) { + usb_host_client_handle_events(this->handle_, portMAX_DELAY); } - ESP_LOGCONFIG(TAG, "client setup complete"); } void USBClient::loop() { + // Process any events from the USB task + UsbEvent *event; + while ((event = this->event_queue.pop()) != nullptr) { + switch (event->type) { + case EVENT_DEVICE_NEW: + this->on_opened(event->data.device_new.address); + break; + case EVENT_DEVICE_GONE: + this->on_removed(event->data.device_gone.handle); + break; + } + // Return event to pool for reuse + this->event_pool.release(event); + } + + // Log dropped events periodically + uint16_t dropped = this->event_queue.get_and_reset_dropped_count(); + if (dropped > 0) { + ESP_LOGW(TAG, "Dropped %u USB events due to queue overflow", dropped); + } + switch (this->state_) { case USB_CLIENT_OPEN: { int err; @@ -229,7 +291,6 @@ void USBClient::loop() { } default: - usb_host_client_handle_events(this->handle_, 0); break; } } @@ -246,6 +307,7 @@ void USBClient::on_removed(usb_device_handle_t handle) { } } +// CALLBACK CONTEXT: USB task (called from usb_host_client_handle_events in USB task) static void control_callback(const usb_transfer_t *xfer) { auto *trq = static_cast(xfer->context); trq->status.error_code = xfer->status; @@ -253,22 +315,55 @@ static void control_callback(const usb_transfer_t *xfer) { trq->status.endpoint = xfer->bEndpointAddress; trq->status.data = xfer->data_buffer; trq->status.data_len = xfer->actual_num_bytes; - if (trq->callback != nullptr) + + // Execute callback in USB task context + if (trq->callback != nullptr) { trq->callback(trq->status); + } + + // Release transfer slot immediately in USB task + // The release_trq() uses thread-safe atomic operations trq->client->release_trq(trq); } +// THREAD CONTEXT: Called from both USB task and main loop threads (multi-consumer) +// - USB task: USB UART input callbacks restart transfers for immediate data reception +// - Main loop: Output transfers and flow-controlled input restarts after consuming data +// +// THREAD SAFETY: Lock-free using atomic compare-and-swap on bitmask +// This multi-threaded access is intentional for performance - USB task can +// immediately restart transfers without waiting for main loop scheduling. TransferRequest *USBClient::get_trq_() { - if (this->trq_pool_.empty()) { - ESP_LOGE(TAG, "Too many requests queued"); - return nullptr; + trq_bitmask_t mask = this->trq_in_use_.load(std::memory_order_relaxed); + + // Find first available slot (bit = 0) and try to claim it atomically + // We use a while loop to allow retrying the same slot after CAS failure + size_t i = 0; + while (i != MAX_REQUESTS) { + if (mask & (static_cast(1) << i)) { + // Slot is in use, move to next slot + i++; + continue; + } + + // Slot i appears available, try to claim it atomically + trq_bitmask_t desired = mask | (static_cast(1) << i); // Set bit i to mark as in-use + + if (this->trq_in_use_.compare_exchange_weak(mask, desired, std::memory_order_acquire, std::memory_order_relaxed)) { + // Successfully claimed slot i - prepare the TransferRequest + auto *trq = &this->requests_[i]; + trq->transfer->context = trq; + trq->transfer->device_handle = this->device_handle_; + return trq; + } + // CAS failed - another thread modified the bitmask + // mask was already updated by compare_exchange_weak with the current value + // No need to reload - the CAS already did that for us + i = 0; } - auto *trq = this->trq_pool_.front(); - this->trq_pool_.pop_front(); - trq->client = this; - trq->transfer->context = trq; - trq->transfer->device_handle = this->device_handle_; - return trq; + + ESP_LOGE(TAG, "All %zu transfer slots in use", MAX_REQUESTS); + return nullptr; } void USBClient::disconnect() { this->on_disconnected(); @@ -281,6 +376,8 @@ void USBClient::disconnect() { this->device_addr_ = -1; } +// THREAD CONTEXT: Called from main loop thread only +// - Used for device configuration and control operations bool USBClient::control_transfer(uint8_t type, uint8_t request, uint16_t value, uint16_t index, const transfer_cb_t &callback, const std::vector &data) { auto *trq = this->get_trq_(); @@ -316,6 +413,7 @@ bool USBClient::control_transfer(uint8_t type, uint8_t request, uint16_t value, return true; } +// CALLBACK CONTEXT: USB task (called from usb_host_client_handle_events in USB task) static void transfer_callback(usb_transfer_t *xfer) { auto *trq = static_cast(xfer->context); trq->status.error_code = xfer->status; @@ -323,12 +421,24 @@ static void transfer_callback(usb_transfer_t *xfer) { trq->status.endpoint = xfer->bEndpointAddress; trq->status.data = xfer->data_buffer; trq->status.data_len = xfer->actual_num_bytes; - if (trq->callback != nullptr) + + // Always execute callback in USB task context + // Callbacks should be fast and non-blocking (e.g., copy data to queue) + if (trq->callback != nullptr) { trq->callback(trq->status); + } + + // Release transfer slot AFTER callback completes to prevent slot exhaustion + // This is critical for high-throughput transfers (e.g., USB UART at 115200 baud) + // The callback has finished accessing xfer->data_buffer, so it's safe to release + // The release_trq() uses thread-safe atomic operations trq->client->release_trq(trq); } /** * Performs a transfer input operation. + * THREAD CONTEXT: Called from both USB task and main loop threads! + * - USB task: USB UART input callbacks call start_input() which calls this + * - Main loop: Initial setup and other components * * @param ep_address The endpoint address. * @param callback The callback function to be called when the transfer is complete. @@ -355,6 +465,9 @@ void USBClient::transfer_in(uint8_t ep_address, const transfer_cb_t &callback, u /** * Performs an output transfer operation. + * THREAD CONTEXT: Called from main loop thread only + * - USB UART output uses defer() to ensure main loop context + * - Modbus and other components call from loop() * * @param ep_address The endpoint address. * @param callback The callback function to be called when the transfer is complete. @@ -387,7 +500,28 @@ void USBClient::dump_config() { " Product id %04X", this->vid_, this->pid_); } -void USBClient::release_trq(TransferRequest *trq) { this->trq_pool_.push_back(trq); } +// THREAD CONTEXT: Called from both USB task and main loop threads +// - USB task: Immediately after transfer callback completes +// - Main loop: When transfer submission fails +// +// THREAD SAFETY: Lock-free using atomic AND to clear bit +// Thread-safe atomic operation allows multi-threaded deallocation +void USBClient::release_trq(TransferRequest *trq) { + if (trq == nullptr) + return; + + // Calculate index from pointer arithmetic + size_t index = trq - this->requests_; + if (index >= MAX_REQUESTS) { + ESP_LOGE(TAG, "Invalid TransferRequest pointer"); + return; + } + + // Atomically clear bit i to mark slot as available + // fetch_and with inverted bitmask clears the bit atomically + trq_bitmask_t bit = static_cast(1) << index; + this->trq_in_use_.fetch_and(static_cast(~bit), std::memory_order_release); +} } // namespace usb_host } // namespace esphome diff --git a/esphome/components/usb_host/usb_host_component.cpp b/esphome/components/usb_host/usb_host_component.cpp index 63a2ab77cc..fb19239c73 100644 --- a/esphome/components/usb_host/usb_host_component.cpp +++ b/esphome/components/usb_host/usb_host_component.cpp @@ -1,5 +1,5 @@ // Should not be needed, but it's required to pass CI clang-tidy checks -#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) +#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) || defined(USE_ESP32_VARIANT_ESP32P4) #include "usb_host.h" #include #include "esphome/core/log.h" @@ -8,7 +8,6 @@ namespace esphome { namespace usb_host { void USBHost::setup() { - ESP_LOGCONFIG(TAG, "Setup starts"); usb_host_config_t config{}; if (usb_host_install(&config) != ESP_OK) { diff --git a/esphome/components/usb_uart/__init__.py b/esphome/components/usb_uart/__init__.py index 6999b1b955..a852e1f78b 100644 --- a/esphome/components/usb_uart/__init__.py +++ b/esphome/components/usb_uart/__init__.py @@ -24,7 +24,6 @@ usb_uart_ns = cg.esphome_ns.namespace("usb_uart") USBUartComponent = usb_uart_ns.class_("USBUartComponent", Component) USBUartChannel = usb_uart_ns.class_("USBUartChannel", UARTComponent) - UARTParityOptions = usb_uart_ns.enum("UARTParityOptions") UART_PARITY_OPTIONS = { "NONE": UARTParityOptions.UART_CONFIG_PARITY_NONE, diff --git a/esphome/components/usb_uart/ch34x.cpp b/esphome/components/usb_uart/ch34x.cpp index 74e7933824..889366b579 100644 --- a/esphome/components/usb_uart/ch34x.cpp +++ b/esphome/components/usb_uart/ch34x.cpp @@ -1,4 +1,4 @@ -#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) +#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) || defined(USE_ESP32_VARIANT_ESP32P4) #include "usb_uart.h" #include "usb/usb_host.h" #include "esphome/core/log.h" @@ -16,12 +16,12 @@ using namespace bytebuffer; void USBUartTypeCH34X::enable_channels() { // enable the channels for (auto channel : this->channels_) { - if (!channel->initialised_) + if (!channel->initialised_.load()) continue; usb_host::transfer_cb_t callback = [=](const usb_host::TransferStatus &status) { if (!status.success) { ESP_LOGE(TAG, "Control transfer failed, status=%s", esp_err_to_name(status.error_code)); - channel->initialised_ = false; + channel->initialised_.store(false); } }; @@ -48,7 +48,7 @@ void USBUartTypeCH34X::enable_channels() { auto factor = static_cast(clk / baud_rate); if (factor == 0 || factor == 0xFF) { ESP_LOGE(TAG, "Invalid baud rate %" PRIu32, baud_rate); - channel->initialised_ = false; + channel->initialised_.store(false); continue; } if ((clk / factor - baud_rate) > (baud_rate - clk / (factor + 1))) @@ -72,6 +72,7 @@ void USBUartTypeCH34X::enable_channels() { if (channel->index_ >= 2) cmd += 0xE; this->control_transfer(USB_VENDOR_DEV | usb_host::USB_DIR_OUT, cmd, value, (factor << 8) | divisor, callback); + this->control_transfer(USB_VENDOR_DEV | usb_host::USB_DIR_OUT, cmd + 3, 0x80, 0, callback); } USBUartTypeCdcAcm::enable_channels(); } diff --git a/esphome/components/usb_uart/cp210x.cpp b/esphome/components/usb_uart/cp210x.cpp index f7d60c307a..5fec0bed02 100644 --- a/esphome/components/usb_uart/cp210x.cpp +++ b/esphome/components/usb_uart/cp210x.cpp @@ -1,4 +1,4 @@ -#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) +#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) || defined(USE_ESP32_VARIANT_ESP32P4) #include "usb_uart.h" #include "usb/usb_host.h" #include "esphome/core/log.h" @@ -100,12 +100,12 @@ std::vector USBUartTypeCP210X::parse_descriptors(usb_device_handle_t dev void USBUartTypeCP210X::enable_channels() { // enable the channels for (auto channel : this->channels_) { - if (!channel->initialised_) + if (!channel->initialised_.load()) continue; usb_host::transfer_cb_t callback = [=](const usb_host::TransferStatus &status) { if (!status.success) { ESP_LOGE(TAG, "Control transfer failed, status=%s", esp_err_to_name(status.error_code)); - channel->initialised_ = false; + channel->initialised_.store(false); } }; this->control_transfer(USB_VENDOR_IFC | usb_host::USB_DIR_OUT, IFC_ENABLE, 1, channel->index_, callback); diff --git a/esphome/components/usb_uart/usb_uart.cpp b/esphome/components/usb_uart/usb_uart.cpp index 934306f480..29003e071e 100644 --- a/esphome/components/usb_uart/usb_uart.cpp +++ b/esphome/components/usb_uart/usb_uart.cpp @@ -1,5 +1,5 @@ // Should not be needed, but it's required to pass CI clang-tidy checks -#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) +#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) || defined(USE_ESP32_VARIANT_ESP32P4) #include "usb_uart.h" #include "esphome/core/log.h" #include "esphome/components/uart/uart_debugger.h" @@ -130,7 +130,7 @@ size_t RingBuffer::pop(uint8_t *data, size_t len) { return len; } void USBUartChannel::write_array(const uint8_t *data, size_t len) { - if (!this->initialised_) { + if (!this->initialised_.load()) { ESP_LOGV(TAG, "Channel not initialised - write ignored"); return; } @@ -152,7 +152,7 @@ bool USBUartChannel::peek_byte(uint8_t *data) { return true; } bool USBUartChannel::read_array(uint8_t *data, size_t len) { - if (!this->initialised_) { + if (!this->initialised_.load()) { ESP_LOGV(TAG, "Channel not initialised - read ignored"); return false; } @@ -170,7 +170,34 @@ bool USBUartChannel::read_array(uint8_t *data, size_t len) { return status; } void USBUartComponent::setup() { USBClient::setup(); } -void USBUartComponent::loop() { USBClient::loop(); } +void USBUartComponent::loop() { + USBClient::loop(); + + // Process USB data from the lock-free queue + UsbDataChunk *chunk; + while ((chunk = this->usb_data_queue_.pop()) != nullptr) { + auto *channel = chunk->channel; + +#ifdef USE_UART_DEBUGGER + if (channel->debug_) { + uart::UARTDebug::log_hex(uart::UART_DIRECTION_RX, std::vector(chunk->data, chunk->data + chunk->length), + ','); // NOLINT() + } +#endif + + // Push data to ring buffer (now safe in main loop) + channel->input_buffer_.push(chunk->data, chunk->length); + + // Return chunk to pool for reuse + this->chunk_pool_.release(chunk); + } + + // Log dropped USB data periodically + uint16_t dropped = this->usb_data_queue_.get_and_reset_dropped_count(); + if (dropped > 0) { + ESP_LOGW(TAG, "Dropped %u USB data chunks due to buffer overflow", dropped); + } +} void USBUartComponent::dump_config() { USBClient::dump_config(); for (auto &channel : this->channels_) { @@ -187,49 +214,77 @@ void USBUartComponent::dump_config() { } } void USBUartComponent::start_input(USBUartChannel *channel) { - if (!channel->initialised_ || channel->input_started_ || - channel->input_buffer_.get_free_space() < channel->cdc_dev_.in_ep->wMaxPacketSize) + if (!channel->initialised_.load() || channel->input_started_.load()) return; + // THREAD CONTEXT: Called from both USB task and main loop threads + // - USB task: Immediate restart after successful transfer for continuous data flow + // - Main loop: Controlled restart after consuming data (backpressure mechanism) + // + // This dual-thread access is intentional for performance: + // - USB task restarts avoid context switch delays for high-speed data + // - Main loop restarts provide flow control when buffers are full + // + // The underlying transfer_in() uses lock-free atomic allocation from the + // TransferRequest pool, making this multi-threaded access safe const auto *ep = channel->cdc_dev_.in_ep; + // CALLBACK CONTEXT: This lambda is executed in USB task via transfer_callback auto callback = [this, channel](const usb_host::TransferStatus &status) { ESP_LOGV(TAG, "Transfer result: length: %u; status %X", status.data_len, status.error_code); if (!status.success) { ESP_LOGE(TAG, "Control transfer failed, status=%s", esp_err_to_name(status.error_code)); + // On failure, don't restart - let next read_array() trigger it + channel->input_started_.store(false); return; } -#ifdef USE_UART_DEBUGGER - if (channel->debug_) { - uart::UARTDebug::log_hex(uart::UART_DIRECTION_RX, - std::vector(status.data, status.data + status.data_len), ','); // NOLINT() - } -#endif - channel->input_started_ = false; - if (!channel->dummy_receiver_) { - for (size_t i = 0; i != status.data_len; i++) { - channel->input_buffer_.push(status.data[i]); + + if (!channel->dummy_receiver_ && status.data_len > 0) { + // Allocate a chunk from the pool + UsbDataChunk *chunk = this->chunk_pool_.allocate(); + if (chunk == nullptr) { + // No chunks available - queue is full or we're out of memory + this->usb_data_queue_.increment_dropped_count(); + // Mark input as not started so we can retry + channel->input_started_.store(false); + return; } + + // Copy data to chunk (this is fast, happens in USB task) + memcpy(chunk->data, status.data, status.data_len); + chunk->length = status.data_len; + chunk->channel = channel; + + // Push to lock-free queue for main loop processing + // Push always succeeds because pool size == queue size + this->usb_data_queue_.push(chunk); } - if (channel->input_buffer_.get_free_space() >= channel->cdc_dev_.in_ep->wMaxPacketSize) { - this->defer([this, channel] { this->start_input(channel); }); - } + + // On success, restart input immediately from USB task for performance + // The lock-free queue will handle backpressure + channel->input_started_.store(false); + this->start_input(channel); }; - channel->input_started_ = true; + channel->input_started_.store(true); this->transfer_in(ep->bEndpointAddress, callback, ep->wMaxPacketSize); } void USBUartComponent::start_output(USBUartChannel *channel) { - if (channel->output_started_) + // IMPORTANT: This function must only be called from the main loop! + // The output_buffer_ is not thread-safe and can only be accessed from main loop. + // USB callbacks use defer() to ensure this function runs in the correct context. + if (channel->output_started_.load()) return; if (channel->output_buffer_.is_empty()) { return; } const auto *ep = channel->cdc_dev_.out_ep; + // CALLBACK CONTEXT: This lambda is executed in USB task via transfer_callback auto callback = [this, channel](const usb_host::TransferStatus &status) { ESP_LOGV(TAG, "Output Transfer result: length: %u; status %X", status.data_len, status.error_code); - channel->output_started_ = false; + channel->output_started_.store(false); + // Defer restart to main loop (defer is thread-safe) this->defer([this, channel] { this->start_output(channel); }); }; - channel->output_started_ = true; + channel->output_started_.store(true); uint8_t data[ep->wMaxPacketSize]; auto len = channel->output_buffer_.pop(data, ep->wMaxPacketSize); this->transfer_out(ep->bEndpointAddress, callback, data, len); @@ -249,7 +304,8 @@ static void fix_mps(const usb_ep_desc_t *ep) { if (ep != nullptr) { auto *ep_mutable = const_cast(ep); if (ep->wMaxPacketSize > 64) { - ESP_LOGW(TAG, "Corrected MPS of EP %u from %u to 64", ep->bEndpointAddress, ep->wMaxPacketSize); + ESP_LOGW(TAG, "Corrected MPS of EP 0x%02X from %u to 64", static_cast(ep->bEndpointAddress & 0xFF), + ep->wMaxPacketSize); ep_mutable->wMaxPacketSize = 64; } } @@ -272,7 +328,7 @@ void USBUartTypeCdcAcm::on_connected() { channel->cdc_dev_ = cdc_devs[i++]; fix_mps(channel->cdc_dev_.in_ep); fix_mps(channel->cdc_dev_.out_ep); - channel->initialised_ = true; + channel->initialised_.store(true); auto err = usb_host_interface_claim(this->handle_, this->device_handle_, channel->cdc_dev_.bulk_interface_number, 0); if (err != ESP_OK) { @@ -301,9 +357,9 @@ void USBUartTypeCdcAcm::on_disconnected() { usb_host_endpoint_flush(this->device_handle_, channel->cdc_dev_.notify_ep->bEndpointAddress); } usb_host_interface_release(this->handle_, this->device_handle_, channel->cdc_dev_.bulk_interface_number); - channel->initialised_ = false; - channel->input_started_ = false; - channel->output_started_ = false; + channel->initialised_.store(false); + channel->input_started_.store(false); + channel->output_started_.store(false); channel->input_buffer_.clear(); channel->output_buffer_.clear(); } @@ -312,10 +368,10 @@ void USBUartTypeCdcAcm::on_disconnected() { void USBUartTypeCdcAcm::enable_channels() { for (auto *channel : this->channels_) { - if (!channel->initialised_) + if (!channel->initialised_.load()) continue; - channel->input_started_ = false; - channel->output_started_ = false; + channel->input_started_.store(false); + channel->output_started_.store(false); this->start_input(channel); } } diff --git a/esphome/components/usb_uart/usb_uart.h b/esphome/components/usb_uart/usb_uart.h index a103c51add..a5e7905ac5 100644 --- a/esphome/components/usb_uart/usb_uart.h +++ b/esphome/components/usb_uart/usb_uart.h @@ -1,15 +1,19 @@ #pragma once -#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) +#if defined(USE_ESP32_VARIANT_ESP32S2) || defined(USE_ESP32_VARIANT_ESP32S3) || defined(USE_ESP32_VARIANT_ESP32P4) #include "esphome/core/component.h" #include "esphome/core/helpers.h" #include "esphome/components/uart/uart_component.h" #include "esphome/components/usb_host/usb_host.h" +#include "esphome/core/lock_free_queue.h" +#include "esphome/core/event_pool.h" +#include namespace esphome { namespace usb_uart { class USBUartTypeCdcAcm; class USBUartComponent; +class USBUartChannel; static const char *const TAG = "usb_uart"; @@ -68,6 +72,17 @@ class RingBuffer { uint8_t *buffer_; }; +// Structure for queuing received USB data chunks +struct UsbDataChunk { + static constexpr size_t MAX_CHUNK_SIZE = 64; // USB packet size + uint8_t data[MAX_CHUNK_SIZE]; + uint8_t length; // Max 64 bytes, so uint8_t is sufficient + USBUartChannel *channel; + + // Required for EventPool - no cleanup needed for POD types + void release() {} +}; + class USBUartChannel : public uart::UARTComponent, public Parented { friend class USBUartComponent; friend class USBUartTypeCdcAcm; @@ -90,16 +105,20 @@ class USBUartChannel : public uart::UARTComponent, public Parenteddummy_receiver_ = dummy_receiver; } protected: - const uint8_t index_; + // Larger structures first for better alignment RingBuffer input_buffer_; RingBuffer output_buffer_; - UARTParityOptions parity_{UART_CONFIG_PARITY_NONE}; - bool input_started_{true}; - bool output_started_{true}; CdcEps cdc_dev_{}; + // Enum (likely 4 bytes) + UARTParityOptions parity_{UART_CONFIG_PARITY_NONE}; + // Group atomics together (each 1 byte) + std::atomic input_started_{true}; + std::atomic output_started_{true}; + std::atomic initialised_{false}; + // Group regular bytes together to minimize padding + const uint8_t index_; bool debug_{}; bool dummy_receiver_{}; - bool initialised_{}; }; class USBUartComponent : public usb_host::USBClient { @@ -115,6 +134,11 @@ class USBUartComponent : public usb_host::USBClient { void start_input(USBUartChannel *channel); void start_output(USBUartChannel *channel); + // Lock-free data transfer from USB task to main loop + static constexpr int USB_DATA_QUEUE_SIZE = 32; + LockFreeQueue usb_data_queue_; + EventPool chunk_pool_; + protected: std::vector channels_{}; }; diff --git a/esphome/components/valve/__init__.py b/esphome/components/valve/__init__.py index cb27546120..6f31fc3a20 100644 --- a/esphome/components/valve/__init__.py +++ b/esphome/components/valve/__init__.py @@ -21,7 +21,7 @@ from esphome.const import ( DEVICE_CLASS_GAS, DEVICE_CLASS_WATER, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority from esphome.core.entity_helpers import entity_duplicate_validator, setup_entity from esphome.cpp_generator import MockObjClass @@ -202,9 +202,9 @@ async def valve_stop_to_code(config, action_id, template_arg, args): @automation.register_action("valve.toggle", ToggleAction, VALVE_ACTION_SCHEMA) -def valve_toggle_to_code(config, action_id, template_arg, args): - paren = yield cg.get_variable(config[CONF_ID]) - yield cg.new_Pvariable(action_id, template_arg, paren) +async def valve_toggle_to_code(config, action_id, template_arg, args): + paren = await cg.get_variable(config[CONF_ID]) + return cg.new_Pvariable(action_id, template_arg, paren) VALVE_CONTROL_ACTION_SCHEMA = cv.Schema( @@ -233,7 +233,6 @@ async def valve_control_to_code(config, action_id, template_arg, args): return var -@coroutine_with_priority(100.0) +@coroutine_with_priority(CoroPriority.CORE) async def to_code(config): - cg.add_define("USE_VALVE") cg.add_global(valve_ns.using) diff --git a/esphome/components/valve/valve.cpp b/esphome/components/valve/valve.cpp index d1ec17945a..b041fe8449 100644 --- a/esphome/components/valve/valve.cpp +++ b/esphome/components/valve/valve.cpp @@ -1,5 +1,6 @@ #include "valve.h" #include "esphome/core/log.h" +#include namespace esphome { namespace valve { @@ -155,7 +156,7 @@ void Valve::publish_state(bool save) { } } optional Valve::restore_state_() { - this->rtc_ = global_preferences->make_preference(this->get_object_id_hash()); + this->rtc_ = global_preferences->make_preference(this->get_preference_hash()); ValveRestoreState recovered{}; if (!this->rtc_.load(&recovered)) return {}; diff --git a/esphome/components/valve/valve.h b/esphome/components/valve/valve.h index 0e14a8d8f0..ab7ff5abe1 100644 --- a/esphome/components/valve/valve.h +++ b/esphome/components/valve/valve.h @@ -19,8 +19,8 @@ const extern float VALVE_CLOSED; if (traits_.get_is_assumed_state()) { \ ESP_LOGCONFIG(TAG, "%s Assumed State: YES", prefix); \ } \ - if (!(obj)->get_device_class().empty()) { \ - ESP_LOGCONFIG(TAG, "%s Device Class: '%s'", prefix, (obj)->get_device_class().c_str()); \ + if (!(obj)->get_device_class_ref().empty()) { \ + ESP_LOGCONFIG(TAG, "%s Device Class: '%s'", prefix, (obj)->get_device_class_ref().c_str()); \ } \ } diff --git a/esphome/components/veml3235/veml3235.cpp b/esphome/components/veml3235/veml3235.cpp index d5489216b6..1e02e3e802 100644 --- a/esphome/components/veml3235/veml3235.cpp +++ b/esphome/components/veml3235/veml3235.cpp @@ -9,22 +9,17 @@ static const char *const TAG = "veml3235.sensor"; void VEML3235Sensor::setup() { uint8_t device_id[] = {0, 0}; - - ESP_LOGCONFIG(TAG, "Running setup for '%s'", this->name_.c_str()); - if (!this->refresh_config_reg()) { ESP_LOGE(TAG, "Unable to write configuration"); this->mark_failed(); return; } - if ((this->write(&ID_REG, 1, false) != i2c::ERROR_OK) || !this->read_bytes_raw(device_id, 2)) { + if ((this->read_register(ID_REG, device_id, sizeof device_id) != i2c::ERROR_OK)) { ESP_LOGE(TAG, "Unable to read ID"); this->mark_failed(); - return; } else if (device_id[0] != DEVICE_ID) { ESP_LOGE(TAG, "Incorrect device ID - expected 0x%.2x, read 0x%.2x", DEVICE_ID, device_id[0]); this->mark_failed(); - return; } } @@ -52,7 +47,7 @@ float VEML3235Sensor::read_lx_() { } uint8_t als_regs[] = {0, 0}; - if ((this->write(&ALS_REG, 1, false) != i2c::ERROR_OK) || !this->read_bytes_raw(als_regs, 2)) { + if ((this->read_register(ALS_REG, als_regs, sizeof als_regs) != i2c::ERROR_OK)) { this->status_set_warning(); return NAN; } diff --git a/esphome/components/veml7700/veml7700.cpp b/esphome/components/veml7700/veml7700.cpp index 9d87a639a6..eb286ba21b 100644 --- a/esphome/components/veml7700/veml7700.cpp +++ b/esphome/components/veml7700/veml7700.cpp @@ -1,6 +1,7 @@ #include "veml7700.h" #include "esphome/core/application.h" #include "esphome/core/log.h" +#include namespace esphome { namespace veml7700 { @@ -12,30 +13,30 @@ static float reduce_to_zero(float a, float b) { return (a > b) ? (a - b) : 0; } template T get_next(const T (&array)[size], const T val) { size_t i = 0; - size_t idx = -1; - while (idx == -1 && i < size) { + size_t idx = std::numeric_limits::max(); + while (idx == std::numeric_limits::max() && i < size) { if (array[i] == val) { idx = i; break; } i++; } - if (idx == -1 || i + 1 >= size) + if (idx == std::numeric_limits::max() || i + 1 >= size) return val; return array[i + 1]; } template T get_prev(const T (&array)[size], const T val) { size_t i = size - 1; - size_t idx = -1; - while (idx == -1 && i > 0) { + size_t idx = std::numeric_limits::max(); + while (idx == std::numeric_limits::max() && i > 0) { if (array[i] == val) { idx = i; break; } i--; } - if (idx == -1 || i == 0) + if (idx == std::numeric_limits::max() || i == 0) return val; return array[i - 1]; } @@ -78,8 +79,6 @@ static const char *get_gain_str(Gain gain) { } void VEML7700Component::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); - auto err = this->configure_(); if (err != i2c::ERROR_OK) { ESP_LOGW(TAG, "Sensor configuration failed"); @@ -281,20 +280,18 @@ ErrorCode VEML7700Component::reconfigure_time_and_gain_(IntegrationTime time, Ga } ErrorCode VEML7700Component::read_sensor_output_(Readings &data) { - auto als_err = - this->read_register((uint8_t) CommandRegisters::ALS, (uint8_t *) &data.als_counts, VEML_REG_SIZE, false); + auto als_err = this->read_register((uint8_t) CommandRegisters::ALS, (uint8_t *) &data.als_counts, VEML_REG_SIZE); if (als_err != i2c::ERROR_OK) { ESP_LOGW(TAG, "Error reading ALS register, err = %d", als_err); } auto white_err = - this->read_register((uint8_t) CommandRegisters::WHITE, (uint8_t *) &data.white_counts, VEML_REG_SIZE, false); + this->read_register((uint8_t) CommandRegisters::WHITE, (uint8_t *) &data.white_counts, VEML_REG_SIZE); if (white_err != i2c::ERROR_OK) { ESP_LOGW(TAG, "Error reading WHITE register, err = %d", white_err); } ConfigurationRegister conf{0}; - auto err = - this->read_register((uint8_t) CommandRegisters::ALS_CONF_0, (uint8_t *) conf.raw_bytes, VEML_REG_SIZE, false); + auto err = this->read_register((uint8_t) CommandRegisters::ALS_CONF_0, (uint8_t *) conf.raw_bytes, VEML_REG_SIZE); if (err != i2c::ERROR_OK) { ESP_LOGW(TAG, "Error reading ALS_CONF_0 register, err = %d", white_err); } diff --git a/esphome/components/veml7700/veml7700.h b/esphome/components/veml7700/veml7700.h index b0d1451cf0..4b5edf733d 100644 --- a/esphome/components/veml7700/veml7700.h +++ b/esphome/components/veml7700/veml7700.h @@ -3,7 +3,6 @@ #include "esphome/components/i2c/i2c.h" #include "esphome/components/sensor/sensor.h" #include "esphome/core/component.h" -#include "esphome/core/optional.h" namespace esphome { namespace veml7700 { diff --git a/esphome/components/version/version_text_sensor.cpp b/esphome/components/version/version_text_sensor.cpp index 5b2437ab62..65dbfd27cf 100644 --- a/esphome/components/version/version_text_sensor.cpp +++ b/esphome/components/version/version_text_sensor.cpp @@ -2,6 +2,7 @@ #include "esphome/core/log.h" #include "esphome/core/application.h" #include "esphome/core/version.h" +#include "esphome/core/helpers.h" namespace esphome { namespace version { @@ -12,12 +13,11 @@ void VersionTextSensor::setup() { if (this->hide_timestamp_) { this->publish_state(ESPHOME_VERSION); } else { - this->publish_state(ESPHOME_VERSION " " + App.get_compilation_time()); + this->publish_state(str_sprintf(ESPHOME_VERSION " %s", App.get_compilation_time().c_str())); } } float VersionTextSensor::get_setup_priority() const { return setup_priority::DATA; } void VersionTextSensor::set_hide_timestamp(bool hide_timestamp) { this->hide_timestamp_ = hide_timestamp; } -std::string VersionTextSensor::unique_id() { return get_mac_address() + "-version"; } void VersionTextSensor::dump_config() { LOG_TEXT_SENSOR("", "Version Text Sensor", this); } } // namespace version diff --git a/esphome/components/version/version_text_sensor.h b/esphome/components/version/version_text_sensor.h index 9355e78442..6813da7830 100644 --- a/esphome/components/version/version_text_sensor.h +++ b/esphome/components/version/version_text_sensor.h @@ -12,7 +12,6 @@ class VersionTextSensor : public text_sensor::TextSensor, public Component { void setup() override; void dump_config() override; float get_setup_priority() const override; - std::string unique_id() override; protected: bool hide_timestamp_{false}; diff --git a/esphome/components/vl53l0x/vl53l0x_sensor.cpp b/esphome/components/vl53l0x/vl53l0x_sensor.cpp index d0b7116eb8..d2548a5bbd 100644 --- a/esphome/components/vl53l0x/vl53l0x_sensor.cpp +++ b/esphome/components/vl53l0x/vl53l0x_sensor.cpp @@ -32,8 +32,6 @@ void VL53L0XSensor::dump_config() { } void VL53L0XSensor::setup() { - ESP_LOGD(TAG, "'%s' - setup BEGIN", this->name_.c_str()); - if (!esphome::vl53l0x::VL53L0XSensor::enable_pin_setup_complete) { for (auto &vl53_sensor : vl53_sensors) { if (vl53_sensor->enable_pin_ != nullptr) { @@ -258,8 +256,6 @@ void VL53L0XSensor::setup() { // I2C_SXXXX__DEVICE_ADDRESS = 0x0001 for VL53L1X reg(0x8A) = final_address & 0x7F; this->set_i2c_address(final_address); - - ESP_LOGD(TAG, "'%s' - setup END", this->name_.c_str()); } void VL53L0XSensor::update() { diff --git a/esphome/components/voice_assistant/voice_assistant.cpp b/esphome/components/voice_assistant/voice_assistant.cpp index 9cf7d10936..7ece73994f 100644 --- a/esphome/components/voice_assistant/voice_assistant.cpp +++ b/esphome/components/voice_assistant/voice_assistant.cpp @@ -35,6 +35,27 @@ void VoiceAssistant::setup() { temp_ring_buffer->write((void *) data.data(), data.size()); } }); + +#ifdef USE_MEDIA_PLAYER + if (this->media_player_ != nullptr) { + this->media_player_->add_on_state_callback([this]() { + switch (this->media_player_->state) { + case media_player::MediaPlayerState::MEDIA_PLAYER_STATE_ANNOUNCING: + if (this->media_player_response_state_ == MediaPlayerResponseState::URL_SENT) { + // State changed to announcing after receiving the url + this->media_player_response_state_ = MediaPlayerResponseState::PLAYING; + } + break; + default: + if (this->media_player_response_state_ == MediaPlayerResponseState::PLAYING) { + // No longer announcing the TTS response + this->media_player_response_state_ = MediaPlayerResponseState::FINISHED; + } + break; + } + }); + } +#endif } float VoiceAssistant::get_setup_priority() const { return setup_priority::AFTER_CONNECTION; } @@ -217,13 +238,20 @@ void VoiceAssistant::loop() { api::VoiceAssistantRequest msg; msg.start = true; - msg.conversation_id = this->conversation_id_; + msg.set_conversation_id(StringRef(this->conversation_id_)); msg.flags = flags; msg.audio_settings = audio_settings; - msg.wake_word_phrase = this->wake_word_; - this->wake_word_ = ""; + msg.set_wake_word_phrase(StringRef(this->wake_word_)); - if (this->api_client_ == nullptr || !this->api_client_->send_message(msg)) { + // Reset media player state tracking +#ifdef USE_MEDIA_PLAYER + if (this->media_player_ != nullptr) { + this->media_player_response_state_ = MediaPlayerResponseState::IDLE; + } +#endif + + if (this->api_client_ == nullptr || + !this->api_client_->send_message(msg, api::VoiceAssistantRequest::MESSAGE_TYPE)) { ESP_LOGW(TAG, "Could not request start"); this->error_trigger_->trigger("not-connected", "Could not request start"); this->continuous_ = false; @@ -244,8 +272,8 @@ void VoiceAssistant::loop() { size_t read_bytes = this->ring_buffer_->read((void *) this->send_buffer_, SEND_BUFFER_SIZE, 0); if (this->audio_mode_ == AUDIO_MODE_API) { api::VoiceAssistantAudio msg; - msg.data.assign((char *) this->send_buffer_, read_bytes); - this->api_client_->send_message(msg); + msg.set_data(this->send_buffer_, read_bytes); + this->api_client_->send_message(msg, api::VoiceAssistantAudio::MESSAGE_TYPE); } else { if (!this->udp_socket_running_) { if (!this->start_udp_socket_()) { @@ -314,24 +342,17 @@ void VoiceAssistant::loop() { #endif #ifdef USE_MEDIA_PLAYER if (this->media_player_ != nullptr) { - playing = (this->media_player_->state == media_player::MediaPlayerState::MEDIA_PLAYER_STATE_ANNOUNCING); + playing = (this->media_player_response_state_ == MediaPlayerResponseState::PLAYING); - if (playing && this->media_player_wait_for_announcement_start_) { - // Announcement has started playing, wait for it to finish - this->media_player_wait_for_announcement_start_ = false; - this->media_player_wait_for_announcement_end_ = true; - } - - if (!playing && this->media_player_wait_for_announcement_end_) { - // Announcement has finished playing - this->media_player_wait_for_announcement_end_ = false; + if (this->media_player_response_state_ == MediaPlayerResponseState::FINISHED) { + this->media_player_response_state_ = MediaPlayerResponseState::IDLE; this->cancel_timeout("playing"); ESP_LOGD(TAG, "Announcement finished playing"); this->set_state_(State::RESPONSE_FINISHED, State::RESPONSE_FINISHED); api::VoiceAssistantAnnounceFinished msg; msg.success = true; - this->api_client_->send_message(msg); + this->api_client_->send_message(msg, api::VoiceAssistantAnnounceFinished::MESSAGE_TYPE); break; } } @@ -408,8 +429,9 @@ void VoiceAssistant::client_subscription(api::APIConnection *client, bool subscr if (this->api_client_ != nullptr) { ESP_LOGE(TAG, "Multiple API Clients attempting to connect to Voice Assistant"); - ESP_LOGE(TAG, "Current client: %s", this->api_client_->get_client_combined_info().c_str()); - ESP_LOGE(TAG, "New client: %s", client->get_client_combined_info().c_str()); + ESP_LOGE(TAG, "Current client: %s (%s)", this->api_client_->get_name().c_str(), + this->api_client_->get_peername().c_str()); + ESP_LOGE(TAG, "New client: %s (%s)", client->get_name().c_str(), client->get_peername().c_str()); return; } @@ -555,7 +577,7 @@ void VoiceAssistant::request_stop() { break; case State::AWAITING_RESPONSE: this->signal_stop_(); - // Fallthrough intended to stop a streaming TTS announcement that has potentially started + break; case State::STREAMING_RESPONSE: #ifdef USE_MEDIA_PLAYER // Stop any ongoing media player announcement @@ -565,6 +587,10 @@ void VoiceAssistant::request_stop() { .set_announcement(true) .perform(); } + if (this->started_streaming_tts_) { + // Haven't reached the TTS_END stage, so send the stop signal to HA. + this->signal_stop_(); + } #endif break; case State::RESPONSE_FINISHED: @@ -580,7 +606,7 @@ void VoiceAssistant::signal_stop_() { ESP_LOGD(TAG, "Signaling stop"); api::VoiceAssistantRequest msg; msg.start = false; - this->api_client_->send_message(msg); + this->api_client_->send_message(msg, api::VoiceAssistantRequest::MESSAGE_TYPE); } void VoiceAssistant::start_playback_timeout_() { @@ -590,7 +616,7 @@ void VoiceAssistant::start_playback_timeout_() { api::VoiceAssistantAnnounceFinished msg; msg.success = true; - this->api_client_->send_message(msg); + this->api_client_->send_message(msg, api::VoiceAssistantAnnounceFinished::MESSAGE_TYPE); }); } @@ -648,13 +674,16 @@ void VoiceAssistant::on_event(const api::VoiceAssistantEventResponse &msg) { if (this->media_player_ != nullptr) { for (const auto &arg : msg.data) { if ((arg.name == "tts_start_streaming") && (arg.value == "1") && !this->tts_response_url_.empty()) { + this->media_player_response_state_ = MediaPlayerResponseState::URL_SENT; + this->media_player_->make_call().set_media_url(this->tts_response_url_).set_announcement(true).perform(); - this->media_player_wait_for_announcement_start_ = true; - this->media_player_wait_for_announcement_end_ = false; this->started_streaming_tts_ = true; + this->start_playback_timeout_(); + tts_url_for_trigger = this->tts_response_url_; this->tts_response_url_.clear(); // Reset streaming URL + this->set_state_(State::STREAMING_RESPONSE, State::STREAMING_RESPONSE); } } } @@ -713,18 +742,22 @@ void VoiceAssistant::on_event(const api::VoiceAssistantEventResponse &msg) { this->defer([this, url]() { #ifdef USE_MEDIA_PLAYER if ((this->media_player_ != nullptr) && (!this->started_streaming_tts_)) { + this->media_player_response_state_ = MediaPlayerResponseState::URL_SENT; + this->media_player_->make_call().set_media_url(url).set_announcement(true).perform(); - this->media_player_wait_for_announcement_start_ = true; - this->media_player_wait_for_announcement_end_ = false; - // Start the playback timeout, as the media player state isn't immediately updated this->start_playback_timeout_(); } + this->started_streaming_tts_ = false; // Helps indicate reaching the TTS_END stage #endif this->tts_end_trigger_->trigger(url); }); State new_state = this->local_output_ ? State::STREAMING_RESPONSE : State::IDLE; - this->set_state_(new_state, new_state); + if (new_state != this->state_) { + // Don't needlessly change the state. The intent progress stage may have already changed the state to streaming + // response. + this->set_state_(new_state, new_state); + } break; } case api::enums::VOICE_ASSISTANT_RUN_END: { @@ -875,6 +908,9 @@ void VoiceAssistant::on_announce(const api::VoiceAssistantAnnounceRequest &msg) #ifdef USE_MEDIA_PLAYER if (this->media_player_ != nullptr) { this->tts_start_trigger_->trigger(msg.text); + + this->media_player_response_state_ = MediaPlayerResponseState::URL_SENT; + if (!msg.preannounce_media_id.empty()) { this->media_player_->make_call().set_media_url(msg.preannounce_media_id).set_announcement(true).perform(); } @@ -886,9 +922,6 @@ void VoiceAssistant::on_announce(const api::VoiceAssistantAnnounceRequest &msg) .perform(); this->continue_conversation_ = msg.start_conversation; - this->media_player_wait_for_announcement_start_ = true; - this->media_player_wait_for_announcement_end_ = false; - // Start the playback timeout, as the media player state isn't immediately updated this->start_playback_timeout_(); if (this->continuous_) { diff --git a/esphome/components/voice_assistant/voice_assistant.h b/esphome/components/voice_assistant/voice_assistant.h index 2424ea6052..95f77dbf09 100644 --- a/esphome/components/voice_assistant/voice_assistant.h +++ b/esphome/components/voice_assistant/voice_assistant.h @@ -90,6 +90,15 @@ struct Configuration { uint32_t max_active_wake_words; }; +#ifdef USE_MEDIA_PLAYER +enum class MediaPlayerResponseState { + IDLE, + URL_SENT, + PLAYING, + FINISHED, +}; +#endif + class VoiceAssistant : public Component { public: VoiceAssistant(); @@ -272,8 +281,8 @@ class VoiceAssistant : public Component { media_player::MediaPlayer *media_player_{nullptr}; std::string tts_response_url_{""}; bool started_streaming_tts_{false}; - bool media_player_wait_for_announcement_start_{false}; - bool media_player_wait_for_announcement_end_{false}; + + MediaPlayerResponseState media_player_response_state_{MediaPlayerResponseState::IDLE}; #endif bool local_output_{false}; diff --git a/esphome/components/wake_on_lan/wake_on_lan.cpp b/esphome/components/wake_on_lan/wake_on_lan.cpp index bed098755a..adf5a080e5 100644 --- a/esphome/components/wake_on_lan/wake_on_lan.cpp +++ b/esphome/components/wake_on_lan/wake_on_lan.cpp @@ -74,12 +74,12 @@ void WakeOnLanButton::setup() { int enable = 1; auto err = this->broadcast_socket_->setsockopt(SOL_SOCKET, SO_REUSEADDR, &enable, sizeof(int)); if (err != 0) { - this->status_set_warning("Socket unable to set reuseaddr"); + this->status_set_warning(LOG_STR("Socket unable to set reuseaddr")); // we can still continue } err = this->broadcast_socket_->setsockopt(SOL_SOCKET, SO_BROADCAST, &enable, sizeof(int)); if (err != 0) { - this->status_set_warning("Socket unable to set broadcast"); + this->status_set_warning(LOG_STR("Socket unable to set broadcast")); } #endif } diff --git a/esphome/components/waveshare_epaper/waveshare_213v3.cpp b/esphome/components/waveshare_epaper/waveshare_213v3.cpp index 316cd80ccd..068cb91d31 100644 --- a/esphome/components/waveshare_epaper/waveshare_213v3.cpp +++ b/esphome/components/waveshare_epaper/waveshare_213v3.cpp @@ -181,7 +181,7 @@ void WaveshareEPaper2P13InV3::dump_config() { LOG_PIN(" Reset Pin: ", this->reset_pin_) LOG_PIN(" DC Pin: ", this->dc_pin_) LOG_PIN(" Busy Pin: ", this->busy_pin_) - LOG_UPDATE_INTERVAL(this) + LOG_UPDATE_INTERVAL(this); } void WaveshareEPaper2P13InV3::set_full_update_every(uint32_t full_update_every) { diff --git a/esphome/components/waveshare_epaper/waveshare_epaper.cpp b/esphome/components/waveshare_epaper/waveshare_epaper.cpp index 75c6b84b79..3510d157d6 100644 --- a/esphome/components/waveshare_epaper/waveshare_epaper.cpp +++ b/esphome/components/waveshare_epaper/waveshare_epaper.cpp @@ -2274,11 +2274,11 @@ void GDEW0154M09::clear_() { uint32_t pixsize = this->get_buffer_length_(); for (uint8_t j = 0; j < 2; j++) { this->command(CMD_DTM1_DATA_START_TRANS); - for (int count = 0; count < pixsize; count++) { + for (uint32_t count = 0; count < pixsize; count++) { this->data(0x00); } this->command(CMD_DTM2_DATA_START_TRANS2); - for (int count = 0; count < pixsize; count++) { + for (uint32_t count = 0; count < pixsize; count++) { this->data(0xff); } this->command(CMD_DISPLAY_REFRESH); @@ -2291,11 +2291,11 @@ void HOT GDEW0154M09::display() { this->init_internal_(); // "Mode 0 display" for now this->command(CMD_DTM1_DATA_START_TRANS); - for (int i = 0; i < this->get_buffer_length_(); i++) { + for (uint32_t i = 0; i < this->get_buffer_length_(); i++) { this->data(0xff); } this->command(CMD_DTM2_DATA_START_TRANS2); // write 'new' data to SRAM - for (int i = 0; i < this->get_buffer_length_(); i++) { + for (uint32_t i = 0; i < this->get_buffer_length_(); i++) { this->data(this->buffer_[i]); } this->command(CMD_DISPLAY_REFRESH); diff --git a/esphome/components/web_server/__init__.py b/esphome/components/web_server/__init__.py index 6890f60014..288d928e80 100644 --- a/esphome/components/web_server/__init__.py +++ b/esphome/components/web_server/__init__.py @@ -31,7 +31,7 @@ from esphome.const import ( PLATFORM_LN882X, PLATFORM_RTL87XX, ) -from esphome.core import CORE, coroutine_with_priority +from esphome.core import CORE, CoroPriority, coroutine_with_priority import esphome.final_validate as fv from esphome.types import ConfigType @@ -52,9 +52,9 @@ def default_url(config: ConfigType) -> ConfigType: config = config.copy() if config[CONF_VERSION] == 1: if CONF_CSS_URL not in config: - config[CONF_CSS_URL] = "https://esphome.io/_static/webserver-v1.min.css" + config[CONF_CSS_URL] = "https://oi.esphome.io/v1/webserver-v1.min.css" if CONF_JS_URL not in config: - config[CONF_JS_URL] = "https://esphome.io/_static/webserver-v1.min.js" + config[CONF_JS_URL] = "https://oi.esphome.io/v1/webserver-v1.min.js" if config[CONF_VERSION] == 2: if CONF_CSS_URL not in config: config[CONF_CSS_URL] = "" @@ -74,13 +74,14 @@ def validate_local(config: ConfigType) -> ConfigType: return config -def validate_ota_removed(config: ConfigType) -> ConfigType: - # Only raise error if OTA is explicitly enabled (True) - # If it's False or not specified, we can safely ignore it - if config.get(CONF_OTA): +def validate_ota(config: ConfigType) -> ConfigType: + # The OTA option only accepts False to explicitly disable OTA for web_server + # IMPORTANT: Setting ota: false ONLY affects the web_server component + # The captive_portal component will still be able to perform OTA updates + if CONF_OTA in config and config[CONF_OTA] is not False: raise cv.Invalid( - f"The '{CONF_OTA}' option has been removed from 'web_server'. " - f"Please use the new OTA platform structure instead:\n\n" + f"The '{CONF_OTA}' option in 'web_server' only accepts 'false' to disable OTA. " + f"To enable OTA, please use the new OTA platform structure instead:\n\n" f"ota:\n" f" - platform: web_server\n\n" f"See https://esphome.io/components/ota for more information." @@ -185,7 +186,7 @@ CONFIG_SCHEMA = cv.All( web_server_base.WebServerBase ), cv.Optional(CONF_INCLUDE_INTERNAL, default=False): cv.boolean, - cv.Optional(CONF_OTA, default=False): cv.boolean, + cv.Optional(CONF_OTA): cv.boolean, cv.Optional(CONF_LOG, default=True): cv.boolean, cv.Optional(CONF_LOCAL): cv.boolean, cv.Optional(CONF_SORTING_GROUPS): cv.ensure_list(sorting_group), @@ -203,7 +204,7 @@ CONFIG_SCHEMA = cv.All( default_url, validate_local, validate_sorting_groups, - validate_ota_removed, + validate_ota, ) @@ -268,7 +269,7 @@ def add_resource_as_progmem( cg.add_global(cg.RawExpression(size_t)) -@coroutine_with_priority(40.0) +@coroutine_with_priority(CoroPriority.WEB) async def to_code(config): paren = await cg.get_variable(config[CONF_WEB_SERVER_BASE_ID]) @@ -288,11 +289,16 @@ async def to_code(config): cg.add(var.set_css_url(config[CONF_CSS_URL])) cg.add(var.set_js_url(config[CONF_JS_URL])) # OTA is now handled by the web_server OTA platform - # The CONF_OTA option is kept only for backwards compatibility validation + # The CONF_OTA option is kept to allow explicitly disabling OTA for web_server + # IMPORTANT: This ONLY affects the web_server component, NOT captive_portal + # Captive portal will still be able to perform OTA updates even when this is set + if config.get(CONF_OTA) is False: + cg.add_define("USE_WEBSERVER_OTA_DISABLED") cg.add(var.set_expose_log(config[CONF_LOG])) if config[CONF_ENABLE_PRIVATE_NETWORK_ACCESS]: cg.add_define("USE_WEBSERVER_PRIVATE_NETWORK_ACCESS") if CONF_AUTH in config: + cg.add_define("USE_WEBSERVER_AUTH") cg.add(paren.set_auth_username(config[CONF_AUTH][CONF_USERNAME])) cg.add(paren.set_auth_password(config[CONF_AUTH][CONF_PASSWORD])) if CONF_CSS_INCLUDE in config: @@ -312,3 +318,15 @@ async def to_code(config): if (sorting_group_config := config.get(CONF_SORTING_GROUPS)) is not None: cg.add_define("USE_WEBSERVER_SORTING") add_sorting_groups(var, sorting_group_config) + + +def FILTER_SOURCE_FILES() -> list[str]: + """Filter out web_server_v1.cpp when version is not 1.""" + files_to_filter: list[str] = [] + + # web_server_v1.cpp is only needed when version is 1 + config = CORE.config.get("web_server", {}) + if config.get(CONF_VERSION, 2) != 1: + files_to_filter.append("web_server_v1.cpp") + + return files_to_filter diff --git a/esphome/components/web_server/list_entities.cpp b/esphome/components/web_server/list_entities.cpp index fb02821760..6b27545549 100644 --- a/esphome/components/web_server/list_entities.cpp +++ b/esphome/components/web_server/list_entities.cpp @@ -9,83 +9,64 @@ namespace esphome { namespace web_server { -#ifdef USE_ARDUINO +#ifdef USE_ESP32 +ListEntitiesIterator::ListEntitiesIterator(const WebServer *ws, AsyncEventSource *es) : web_server_(ws), events_(es) {} +#elif USE_ARDUINO ListEntitiesIterator::ListEntitiesIterator(const WebServer *ws, DeferredUpdateEventSource *es) : web_server_(ws), events_(es) {} #endif -#ifdef USE_ESP_IDF -ListEntitiesIterator::ListEntitiesIterator(const WebServer *ws, AsyncEventSource *es) : web_server_(ws), events_(es) {} -#endif ListEntitiesIterator::~ListEntitiesIterator() {} #ifdef USE_BINARY_SENSOR bool ListEntitiesIterator::on_binary_sensor(binary_sensor::BinarySensor *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::binary_sensor_all_json_generator); return true; } #endif #ifdef USE_COVER bool ListEntitiesIterator::on_cover(cover::Cover *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::cover_all_json_generator); return true; } #endif #ifdef USE_FAN bool ListEntitiesIterator::on_fan(fan::Fan *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::fan_all_json_generator); return true; } #endif #ifdef USE_LIGHT bool ListEntitiesIterator::on_light(light::LightState *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::light_all_json_generator); return true; } #endif #ifdef USE_SENSOR bool ListEntitiesIterator::on_sensor(sensor::Sensor *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::sensor_all_json_generator); return true; } #endif #ifdef USE_SWITCH bool ListEntitiesIterator::on_switch(switch_::Switch *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::switch_all_json_generator); return true; } #endif #ifdef USE_BUTTON bool ListEntitiesIterator::on_button(button::Button *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::button_all_json_generator); return true; } #endif #ifdef USE_TEXT_SENSOR bool ListEntitiesIterator::on_text_sensor(text_sensor::TextSensor *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::text_sensor_all_json_generator); return true; } #endif #ifdef USE_LOCK bool ListEntitiesIterator::on_lock(lock::Lock *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::lock_all_json_generator); return true; } @@ -93,8 +74,6 @@ bool ListEntitiesIterator::on_lock(lock::Lock *obj) { #ifdef USE_VALVE bool ListEntitiesIterator::on_valve(valve::Valve *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::valve_all_json_generator); return true; } @@ -102,8 +81,6 @@ bool ListEntitiesIterator::on_valve(valve::Valve *obj) { #ifdef USE_CLIMATE bool ListEntitiesIterator::on_climate(climate::Climate *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::climate_all_json_generator); return true; } @@ -111,8 +88,6 @@ bool ListEntitiesIterator::on_climate(climate::Climate *obj) { #ifdef USE_NUMBER bool ListEntitiesIterator::on_number(number::Number *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::number_all_json_generator); return true; } @@ -120,8 +95,6 @@ bool ListEntitiesIterator::on_number(number::Number *obj) { #ifdef USE_DATETIME_DATE bool ListEntitiesIterator::on_date(datetime::DateEntity *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::date_all_json_generator); return true; } @@ -129,8 +102,6 @@ bool ListEntitiesIterator::on_date(datetime::DateEntity *obj) { #ifdef USE_DATETIME_TIME bool ListEntitiesIterator::on_time(datetime::TimeEntity *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::time_all_json_generator); return true; } @@ -138,8 +109,6 @@ bool ListEntitiesIterator::on_time(datetime::TimeEntity *obj) { #ifdef USE_DATETIME_DATETIME bool ListEntitiesIterator::on_datetime(datetime::DateTimeEntity *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::datetime_all_json_generator); return true; } @@ -147,8 +116,6 @@ bool ListEntitiesIterator::on_datetime(datetime::DateTimeEntity *obj) { #ifdef USE_TEXT bool ListEntitiesIterator::on_text(text::Text *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::text_all_json_generator); return true; } @@ -156,8 +123,6 @@ bool ListEntitiesIterator::on_text(text::Text *obj) { #ifdef USE_SELECT bool ListEntitiesIterator::on_select(select::Select *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::select_all_json_generator); return true; } @@ -165,8 +130,6 @@ bool ListEntitiesIterator::on_select(select::Select *obj) { #ifdef USE_ALARM_CONTROL_PANEL bool ListEntitiesIterator::on_alarm_control_panel(alarm_control_panel::AlarmControlPanel *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::alarm_control_panel_all_json_generator); return true; } @@ -174,8 +137,6 @@ bool ListEntitiesIterator::on_alarm_control_panel(alarm_control_panel::AlarmCont #ifdef USE_EVENT bool ListEntitiesIterator::on_event(event::Event *obj) { - if (this->events_->count() == 0) - return true; // Null event type, since we are just iterating over entities this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::event_all_json_generator); return true; @@ -184,8 +145,6 @@ bool ListEntitiesIterator::on_event(event::Event *obj) { #ifdef USE_UPDATE bool ListEntitiesIterator::on_update(update::UpdateEntity *obj) { - if (this->events_->count() == 0) - return true; this->events_->deferrable_send_state(obj, "state_detail_all", WebServer::update_all_json_generator); return true; } diff --git a/esphome/components/web_server/list_entities.h b/esphome/components/web_server/list_entities.h index ba81c70c86..43e1cc2544 100644 --- a/esphome/components/web_server/list_entities.h +++ b/esphome/components/web_server/list_entities.h @@ -5,25 +5,24 @@ #include "esphome/core/component.h" #include "esphome/core/component_iterator.h" namespace esphome { -#ifdef USE_ESP_IDF +#ifdef USE_ESP32 namespace web_server_idf { class AsyncEventSource; } #endif namespace web_server { -#ifdef USE_ARDUINO +#if !defined(USE_ESP32) && defined(USE_ARDUINO) class DeferredUpdateEventSource; #endif class WebServer; class ListEntitiesIterator : public ComponentIterator { public: -#ifdef USE_ARDUINO - ListEntitiesIterator(const WebServer *ws, DeferredUpdateEventSource *es); -#endif -#ifdef USE_ESP_IDF +#ifdef USE_ESP32 ListEntitiesIterator(const WebServer *ws, esphome::web_server_idf::AsyncEventSource *es); +#elif defined(USE_ARDUINO) + ListEntitiesIterator(const WebServer *ws, DeferredUpdateEventSource *es); #endif virtual ~ListEntitiesIterator(); #ifdef USE_BINARY_SENSOR @@ -90,11 +89,10 @@ class ListEntitiesIterator : public ComponentIterator { protected: const WebServer *web_server_; -#ifdef USE_ARDUINO - DeferredUpdateEventSource *events_; -#endif -#ifdef USE_ESP_IDF +#ifdef USE_ESP32 esphome::web_server_idf::AsyncEventSource *events_; +#elif USE_ARDUINO + DeferredUpdateEventSource *events_; #endif }; diff --git a/esphome/components/web_server/ota/__init__.py b/esphome/components/web_server/ota/__init__.py index 3af14fd453..4a98db8877 100644 --- a/esphome/components/web_server/ota/__init__.py +++ b/esphome/components/web_server/ota/__init__.py @@ -4,6 +4,7 @@ from esphome.components.ota import BASE_OTA_SCHEMA, OTAComponent, ota_to_code import esphome.config_validation as cv from esphome.const import CONF_ID from esphome.core import CORE, coroutine_with_priority +from esphome.coroutine import CoroPriority CODEOWNERS = ["@esphome/core"] DEPENDENCIES = ["network", "web_server_base"] @@ -22,11 +23,11 @@ CONFIG_SCHEMA = ( ) -@coroutine_with_priority(52.0) +@coroutine_with_priority(CoroPriority.WEB_SERVER_OTA) async def to_code(config): var = cg.new_Pvariable(config[CONF_ID]) await ota_to_code(var, config) await cg.register_component(var, config) cg.add_define("USE_WEBSERVER_OTA") - if CORE.using_esp_idf: + if CORE.is_esp32: add_idf_component(name="zorxx/multipart-parser", ref="1.0.1") diff --git a/esphome/components/web_server/ota/ota_web_server.cpp b/esphome/components/web_server/ota/ota_web_server.cpp index 4f8f6fda17..7929f3647f 100644 --- a/esphome/components/web_server/ota/ota_web_server.cpp +++ b/esphome/components/web_server/ota/ota_web_server.cpp @@ -5,6 +5,10 @@ #include "esphome/core/application.h" #include "esphome/core/log.h" +#ifdef USE_CAPTIVE_PORTAL +#include "esphome/components/captive_portal/captive_portal.h" +#endif + #ifdef USE_ARDUINO #ifdef USE_ESP8266 #include @@ -13,6 +17,12 @@ #endif #endif // USE_ARDUINO +#if USE_ESP32 +using PlatformString = std::string; +#elif USE_ARDUINO +using PlatformString = String; +#endif + namespace esphome { namespace web_server { @@ -22,10 +32,25 @@ class OTARequestHandler : public AsyncWebHandler { public: OTARequestHandler(WebServerOTAComponent *parent) : parent_(parent) {} void handleRequest(AsyncWebServerRequest *request) override; - void handleUpload(AsyncWebServerRequest *request, const String &filename, size_t index, uint8_t *data, size_t len, - bool final) override; + void handleUpload(AsyncWebServerRequest *request, const PlatformString &filename, size_t index, uint8_t *data, + size_t len, bool final) override; bool canHandle(AsyncWebServerRequest *request) const override { - return request->url() == "/update" && request->method() == HTTP_POST; + // Check if this is an OTA update request + bool is_ota_request = request->url() == "/update" && request->method() == HTTP_POST; + +#if defined(USE_WEBSERVER_OTA_DISABLED) && defined(USE_CAPTIVE_PORTAL) + // IMPORTANT: USE_WEBSERVER_OTA_DISABLED only disables OTA for the web_server component + // Captive portal can still perform OTA updates - check if request is from active captive portal + // Note: global_captive_portal is the standard way components communicate in ESPHome + return is_ota_request && captive_portal::global_captive_portal != nullptr && + captive_portal::global_captive_portal->is_active(); +#elif defined(USE_WEBSERVER_OTA_DISABLED) + // OTA disabled for web_server and no captive portal compiled in + return false; +#else + // OTA enabled for web_server + return is_ota_request; +#endif } // NOLINTNEXTLINE(readability-identifier-naming) @@ -57,7 +82,7 @@ void OTARequestHandler::report_ota_progress_(AsyncWebServerRequest *request) { percentage = (this->ota_read_length_ * 100.0f) / request->contentLength(); ESP_LOGD(TAG, "OTA in progress: %0.1f%%", percentage); } else { - ESP_LOGD(TAG, "OTA in progress: %u bytes read", this->ota_read_length_); + ESP_LOGD(TAG, "OTA in progress: %" PRIu32 " bytes read", this->ota_read_length_); } #ifdef USE_OTA_STATE_CALLBACK // Report progress - use call_deferred since we're in web server task @@ -81,7 +106,7 @@ void OTARequestHandler::ota_init_(const char *filename) { this->ota_success_ = false; } -void OTARequestHandler::handleUpload(AsyncWebServerRequest *request, const String &filename, size_t index, +void OTARequestHandler::handleUpload(AsyncWebServerRequest *request, const PlatformString &filename, size_t index, uint8_t *data, size_t len, bool final) { ota::OTAResponseTypes error_code = ota::OTA_RESPONSE_OK; @@ -152,7 +177,7 @@ void OTARequestHandler::handleUpload(AsyncWebServerRequest *request, const Strin // Finalize if (final) { - ESP_LOGD(TAG, "OTA final chunk: index=%u, len=%u, total_read=%u, contentLength=%u", index, len, + ESP_LOGD(TAG, "OTA final chunk: index=%zu, len=%zu, total_read=%" PRIu32 ", contentLength=%zu", index, len, this->ota_read_length_, request->contentLength()); // For Arduino framework, the Update library tracks expected size from firmware header @@ -179,9 +204,20 @@ void OTARequestHandler::handleUpload(AsyncWebServerRequest *request, const Strin void OTARequestHandler::handleRequest(AsyncWebServerRequest *request) { AsyncWebServerResponse *response; // Use the ota_success_ flag to determine the actual result +#ifdef USE_ESP8266 + static const char UPDATE_SUCCESS[] PROGMEM = "Update Successful!"; + static const char UPDATE_FAILED[] PROGMEM = "Update Failed!"; + static const char TEXT_PLAIN[] PROGMEM = "text/plain"; + static const char CONNECTION_STR[] PROGMEM = "Connection"; + static const char CLOSE_STR[] PROGMEM = "close"; + const char *msg = this->ota_success_ ? UPDATE_SUCCESS : UPDATE_FAILED; + response = request->beginResponse_P(200, TEXT_PLAIN, msg); + response->addHeader(CONNECTION_STR, CLOSE_STR); +#else const char *msg = this->ota_success_ ? "Update Successful!" : "Update Failed!"; response = request->beginResponse(200, "text/plain", msg); response->addHeader("Connection", "close"); +#endif request->send(response); } diff --git a/esphome/components/web_server/server_index_v2.h b/esphome/components/web_server/server_index_v2.h index ec093d3186..e675d81552 100644 --- a/esphome/components/web_server/server_index_v2.h +++ b/esphome/components/web_server/server_index_v2.h @@ -494,155 +494,155 @@ const uint8_t INDEX_GZ[] PROGMEM = { 0x1c, 0x40, 0xc8, 0x12, 0x7c, 0xa6, 0xc1, 0x29, 0x21, 0xa4, 0xd5, 0x9f, 0x05, 0x5f, 0xe2, 0x9b, 0x98, 0xa6, 0xc1, 0xbc, 0xe8, 0x96, 0x04, 0xa0, 0x22, 0xa6, 0x6f, 0x45, 0x79, 0x6f, 0x9c, 0xa4, 0x8a, 0xea, 0xb5, 0x82, 0xb3, 0x59, 0x52, 0xcf, 0x96, 0x58, 0x9a, 0xe5, 0x93, 0x19, 0x25, 0xfc, 0xa6, 0x79, 0xeb, 0xf6, 0x36, 0xc7, 0xd7, 0x60, 0x76, - 0x65, 0x7c, 0x4d, 0x02, 0x5b, 0x3e, 0xbd, 0x0f, 0xc7, 0xe5, 0xef, 0x57, 0x34, 0xcf, 0xc3, 0xb1, 0xae, 0xb9, 0x3d, - 0x9e, 0x26, 0x41, 0xb4, 0x63, 0x69, 0x06, 0x08, 0x88, 0x89, 0x01, 0x46, 0xc0, 0xa7, 0xa1, 0x43, 0x64, 0x30, 0xf5, - 0x7a, 0x74, 0x4d, 0x0e, 0x5f, 0x2f, 0x12, 0xe1, 0xb8, 0x2a, 0x38, 0x99, 0x66, 0x54, 0x96, 0x2a, 0x34, 0x16, 0x27, - 0xfb, 0x50, 0xa0, 0x5e, 0x6f, 0x89, 0xa2, 0x19, 0x07, 0xca, 0xf6, 0x58, 0x9a, 0x63, 0xa2, 0x68, 0x76, 0xa2, 0x52, - 0x99, 0xa5, 0xb4, 0x1e, 0xbb, 0xf9, 0xbc, 0x3d, 0x84, 0x3f, 0x3a, 0x32, 0xf4, 0xf9, 0x68, 0x34, 0xba, 0x37, 0xaa, - 0xf6, 0x79, 0x34, 0xa2, 0x1d, 0x7a, 0xd4, 0x85, 0x24, 0x96, 0xa6, 0x8e, 0xc5, 0xb4, 0x0b, 0x89, 0xbb, 0xc5, 0xc3, - 0x2a, 0x43, 0xd8, 0x46, 0xc4, 0x8b, 0x87, 0x47, 0xd8, 0x8a, 0x69, 0x46, 0x17, 0x93, 0x30, 0x1b, 0xb3, 0x34, 0x68, - 0x15, 0xfe, 0x5c, 0x87, 0xa4, 0x3e, 0x3f, 0x3e, 0x3e, 0x2e, 0xfc, 0xc8, 0x3c, 0xb5, 0xa2, 0xa8, 0xf0, 0x87, 0x8b, - 0x72, 0x1a, 0xad, 0xd6, 0x68, 0x54, 0xf8, 0xcc, 0x14, 0x1c, 0x74, 0x86, 0xd1, 0x41, 0xa7, 0xf0, 0x6f, 0xac, 0x1a, - 0x85, 0x4f, 0xf5, 0x53, 0x46, 0xa3, 0x5a, 0x26, 0xcc, 0xe3, 0x56, 0xab, 0xf0, 0x15, 0xa1, 0x2d, 0xc0, 0x2c, 0x55, - 0x3f, 0x83, 0x70, 0x26, 0x38, 0x30, 0xf7, 0x6e, 0x22, 0xbc, 0xc1, 0xa5, 0xbe, 0x65, 0x44, 0x7d, 0x93, 0xa3, 0x40, - 0x17, 0xf8, 0x67, 0x3b, 0x78, 0x04, 0xc4, 0x2c, 0x83, 0x46, 0x89, 0x89, 0x2d, 0xd5, 0x5e, 0x03, 0x65, 0xc9, 0xd7, - 0x3f, 0x93, 0xa4, 0x8a, 0x29, 0x01, 0x27, 0x83, 0x9a, 0xea, 0x32, 0x3c, 0x4a, 0xb7, 0xc8, 0x0f, 0xf6, 0x69, 0xf9, - 0x71, 0xf7, 0x10, 0xf1, 0xc1, 0xfe, 0x70, 0xf1, 0x41, 0xa9, 0x25, 0x3e, 0x14, 0xf3, 0xb8, 0x13, 0xc4, 0x1d, 0xc6, - 0x74, 0xf8, 0xf1, 0x9a, 0xdf, 0x36, 0x61, 0x4b, 0x64, 0xae, 0x14, 0x2c, 0xbb, 0xbf, 0x35, 0x6b, 0xc6, 0x74, 0x66, - 0x7d, 0xd1, 0x43, 0xaa, 0x0f, 0x6f, 0x52, 0xe2, 0xbe, 0x31, 0xb6, 0xad, 0x2a, 0x19, 0x8d, 0x88, 0xfb, 0x66, 0x34, - 0x72, 0xcd, 0x59, 0xc9, 0x50, 0x50, 0x59, 0xeb, 0x75, 0xad, 0x44, 0xd6, 0xfa, 0xf2, 0x4b, 0xbb, 0xcc, 0x2e, 0xd0, - 0xa1, 0x27, 0x3b, 0xcc, 0xa4, 0xdf, 0x44, 0x2c, 0x87, 0xad, 0x06, 0x1f, 0x1a, 0xa9, 0xdf, 0xd5, 0x98, 0xd6, 0xae, - 0xd5, 0x2e, 0x01, 0xde, 0x70, 0x17, 0xf8, 0xea, 0x45, 0x01, 0x63, 0x6a, 0xf2, 0x16, 0x9f, 0xde, 0x7d, 0x15, 0x79, - 0x77, 0x02, 0x15, 0x2c, 0x7f, 0x93, 0xae, 0x1c, 0x02, 0x52, 0x30, 0x12, 0x62, 0x4f, 0xab, 0x10, 0x7c, 0x3c, 0x4e, - 0xe0, 0x5b, 0x2f, 0x8b, 0xda, 0xfd, 0xb1, 0xaa, 0x79, 0xbf, 0x36, 0xdf, 0xc0, 0x6e, 0xa8, 0x6f, 0x5b, 0x95, 0x9f, - 0x9e, 0x52, 0xc9, 0xe3, 0x73, 0xfd, 0x0d, 0x22, 0x69, 0x16, 0x2f, 0x34, 0x93, 0x5f, 0xa8, 0x94, 0x63, 0x01, 0xe9, - 0x36, 0xaa, 0xe3, 0xa8, 0x28, 0xf4, 0x61, 0x8d, 0x88, 0xe5, 0x53, 0xb8, 0xd7, 0x54, 0xb5, 0xa4, 0x9f, 0x62, 0xe1, - 0xf9, 0x8d, 0x15, 0xdf, 0xa9, 0x2d, 0x57, 0x61, 0x02, 0x3c, 0xca, 0x61, 0x7e, 0x27, 0x0a, 0x57, 0xfb, 0xdd, 0x0d, - 0x12, 0x5d, 0x47, 0xe1, 0x53, 0x45, 0x9e, 0xac, 0x19, 0x82, 0xf3, 0xbb, 0x5c, 0x10, 0xf3, 0xca, 0x14, 0x14, 0x76, - 0xfc, 0x52, 0xbe, 0x51, 0xd8, 0x92, 0xd1, 0x92, 0x7c, 0x1a, 0xa6, 0x8a, 0x8d, 0x12, 0x57, 0xf1, 0x83, 0xdd, 0x45, - 0xb5, 0xf2, 0x85, 0x6b, 0xc0, 0x56, 0xc4, 0xdb, 0x3b, 0xd9, 0x87, 0x06, 0x3d, 0xa7, 0x06, 0x7a, 0xba, 0x16, 0x64, - 0xf9, 0x44, 0xba, 0xc3, 0x95, 0x9f, 0xdf, 0x60, 0x3f, 0xbf, 0x71, 0xfe, 0xbc, 0x68, 0xde, 0xd0, 0xeb, 0x8f, 0x4c, - 0x34, 0x45, 0x38, 0x6d, 0x82, 0xe1, 0x23, 0x9d, 0xa3, 0x9a, 0x3d, 0xcb, 0x2c, 0x3f, 0x75, 0xd5, 0x41, 0x77, 0x96, - 0x43, 0x56, 0x84, 0x54, 0xdf, 0x83, 0x94, 0xa7, 0xb4, 0x5b, 0xcf, 0xe6, 0xb4, 0x83, 0xec, 0x06, 0x5b, 0x17, 0x0b, - 0x0e, 0x59, 0x14, 0xe2, 0x2e, 0x68, 0x69, 0xb6, 0xde, 0x32, 0x11, 0xf4, 0xd6, 0xc6, 0xfa, 0x81, 0x46, 0x6e, 0x43, - 0x4a, 0xaf, 0x6c, 0x3d, 0x93, 0x60, 0x5b, 0x26, 0xc0, 0xa7, 0x72, 0x1b, 0xc1, 0xa5, 0x6a, 0xfe, 0x5a, 0x49, 0xa1, - 0xab, 0xc5, 0x32, 0xb7, 0xf1, 0x21, 0x90, 0x05, 0xe1, 0x48, 0xd0, 0x0c, 0x3f, 0xa4, 0xe6, 0xb5, 0x3c, 0x86, 0xb4, - 0x00, 0x31, 0x13, 0xb4, 0x8f, 0xa7, 0xb7, 0x0f, 0xef, 0xfe, 0xfe, 0xe9, 0x17, 0x1a, 0x47, 0xe6, 0x5a, 0x1e, 0xd7, - 0xed, 0xc2, 0x46, 0x48, 0xc2, 0xbb, 0x80, 0xa5, 0x52, 0xe6, 0x5d, 0x83, 0x5f, 0xb4, 0x3b, 0xe5, 0x3a, 0x49, 0x37, - 0xa3, 0x89, 0xfc, 0x0a, 0x9f, 0x5e, 0x8a, 0x83, 0x47, 0xd3, 0x5b, 0xb3, 0x1a, 0xed, 0x95, 0xe4, 0xdb, 0x3f, 0x34, - 0xc7, 0x76, 0x7b, 0x52, 0x6f, 0x3d, 0x4f, 0xf4, 0x68, 0x7a, 0xdb, 0x55, 0x82, 0xb6, 0x99, 0x29, 0xa8, 0x5a, 0xd3, - 0x5b, 0x3b, 0xcb, 0xb8, 0xea, 0xc8, 0xf1, 0x0f, 0x72, 0x87, 0x86, 0x39, 0xed, 0xc2, 0xbd, 0xe3, 0x6c, 0x18, 0x26, - 0x5a, 0x98, 0x4f, 0x58, 0x14, 0x25, 0xb4, 0x6b, 0xe4, 0xb5, 0xd3, 0x7e, 0x04, 0x49, 0xba, 0xf6, 0x92, 0xd5, 0x57, - 0xc5, 0x42, 0x5e, 0x89, 0xa7, 0xf0, 0x3a, 0xe7, 0x09, 0x7c, 0xf4, 0x63, 0x23, 0x3a, 0x75, 0xf6, 0x6a, 0xab, 0x42, - 0x9e, 0xfc, 0x5d, 0x9f, 0xcb, 0x51, 0xeb, 0x4f, 0x5d, 0xb9, 0xe0, 0xad, 0xae, 0xe0, 0xd3, 0xa0, 0x79, 0x50, 0x9f, - 0x08, 0xbc, 0x2a, 0xa7, 0x80, 0x37, 0x4c, 0x0b, 0x83, 0xb4, 0x52, 0x7c, 0xda, 0xf1, 0xdb, 0xba, 0x4c, 0x76, 0x00, - 0x79, 0x61, 0x65, 0x51, 0x51, 0x9f, 0xcc, 0xbf, 0xcd, 0x6e, 0x79, 0xb2, 0x79, 0xb7, 0x3c, 0x31, 0xbb, 0xe5, 0x7e, - 0x8a, 0xfd, 0x7c, 0xd4, 0x86, 0x3f, 0xdd, 0x6a, 0x42, 0x41, 0xcb, 0x39, 0x98, 0xde, 0x3a, 0xa0, 0xa7, 0x35, 0x3b, - 0xd3, 0x5b, 0x95, 0x63, 0x0d, 0xb1, 0x9b, 0x16, 0x64, 0x1d, 0xe3, 0x96, 0x03, 0x85, 0xf0, 0xb7, 0x55, 0x7b, 0xd5, - 0x3e, 0x84, 0x77, 0xd0, 0xea, 0x68, 0xfd, 0x5d, 0xe7, 0xfe, 0x4d, 0x1b, 0xa4, 0x5c, 0x78, 0x81, 0xe1, 0xc6, 0xc8, - 0x17, 0xe1, 0xf5, 0x35, 0x8d, 0x82, 0x11, 0x1f, 0xce, 0xf2, 0x7f, 0xd2, 0xf0, 0x6b, 0x24, 0xde, 0xbb, 0xa5, 0x57, - 0xfa, 0x31, 0x4d, 0x55, 0xc6, 0xb7, 0xe9, 0x61, 0x51, 0xae, 0x53, 0x90, 0x0f, 0xc3, 0x84, 0x7a, 0x1d, 0xff, 0x70, - 0xc3, 0x26, 0xf8, 0x77, 0x59, 0x9b, 0x8d, 0x93, 0xf9, 0xbd, 0xc8, 0xb8, 0x17, 0x09, 0xbf, 0x0a, 0x07, 0xf6, 0x1a, - 0xb6, 0x8e, 0x37, 0x83, 0x3b, 0x30, 0x23, 0x5d, 0x18, 0xa1, 0xa0, 0xe5, 0x4e, 0x44, 0x47, 0xe1, 0x2c, 0x11, 0xf7, - 0xf7, 0xba, 0x8d, 0x32, 0xd6, 0x7a, 0xbd, 0x87, 0xa1, 0x57, 0x75, 0x1f, 0xc8, 0xa5, 0x3f, 0x7f, 0x72, 0x08, 0x7f, - 0x54, 0xfe, 0xd7, 0x5d, 0xa5, 0xab, 0x2b, 0xbb, 0x17, 0x74, 0xf5, 0xdd, 0x9a, 0x32, 0xae, 0x44, 0xb8, 0xd4, 0xc7, - 0x1f, 0x5a, 0x1b, 0xb4, 0xca, 0x07, 0x55, 0xd7, 0x5a, 0xd6, 0xaf, 0xaa, 0xfd, 0xeb, 0x3a, 0x7f, 0x60, 0xdd, 0xa1, - 0xd2, 0x5c, 0xeb, 0x75, 0xf5, 0x67, 0x08, 0xd7, 0x2a, 0x1b, 0x8c, 0xcb, 0xfa, 0xbb, 0xe4, 0xae, 0x34, 0x51, 0x54, - 0x34, 0x16, 0xac, 0x94, 0x5d, 0x65, 0xa5, 0xe4, 0x94, 0x5c, 0x9d, 0xf4, 0x6f, 0x27, 0x89, 0x33, 0x57, 0xc7, 0x25, - 0x89, 0xdb, 0xf6, 0x5b, 0xae, 0x23, 0xf3, 0x00, 0xe0, 0xd6, 0x76, 0x57, 0x7e, 0xde, 0xd6, 0xed, 0x83, 0xa6, 0x35, - 0x1f, 0x4b, 0xcd, 0xee, 0x65, 0x78, 0x47, 0xb3, 0xcb, 0x8e, 0xeb, 0x80, 0x9f, 0xa6, 0xa9, 0x52, 0x26, 0x64, 0x99, - 0xd3, 0x71, 0x9d, 0xdb, 0x49, 0x92, 0xe6, 0xc4, 0x8d, 0x85, 0x98, 0x06, 0xea, 0xfb, 0xb7, 0x37, 0x07, 0x3e, 0xcf, - 0xc6, 0xfb, 0x9d, 0x56, 0xab, 0x05, 0x17, 0xc0, 0xba, 0xce, 0x9c, 0xd1, 0x9b, 0xa7, 0xfc, 0x96, 0xb8, 0x2d, 0xa7, - 0xe5, 0xb4, 0x3b, 0xc7, 0x4e, 0xbb, 0x73, 0xe8, 0x3f, 0x3a, 0x76, 0x7b, 0x9f, 0x39, 0xce, 0x49, 0x44, 0x47, 0x39, - 0xfc, 0x70, 0x9c, 0x13, 0xa9, 0x78, 0xa9, 0xdf, 0x8e, 0xe3, 0x0f, 0x93, 0xbc, 0xd9, 0x76, 0x16, 0xfa, 0xd1, 0x71, - 0xe0, 0x50, 0x69, 0xe0, 0x7c, 0x3e, 0xea, 0x8c, 0x0e, 0x47, 0x4f, 0xba, 0xba, 0xb8, 0xf8, 0xac, 0x56, 0x1d, 0xab, - 0xff, 0x3b, 0x56, 0xb3, 0x5c, 0x64, 0xfc, 0x23, 0xd5, 0x39, 0x89, 0x0e, 0x88, 0x9e, 0x8d, 0x4d, 0x3b, 0xeb, 0x23, - 0xb5, 0x8f, 0xaf, 0x87, 0xa3, 0x4e, 0x55, 0x5d, 0xc2, 0xb8, 0x5f, 0x02, 0x79, 0xb2, 0x6f, 0x40, 0x3f, 0xb1, 0xd1, - 0xd4, 0x6e, 0x6e, 0x42, 0x54, 0xdb, 0xd5, 0x73, 0x1c, 0x9b, 0xf9, 0x9d, 0xc0, 0x19, 0x06, 0xa3, 0xab, 0x4a, 0x08, - 0x5c, 0x27, 0x22, 0xee, 0xab, 0x76, 0xe7, 0x18, 0xb7, 0xdb, 0x8f, 0xfc, 0x47, 0xc7, 0xc3, 0x16, 0x3e, 0xf4, 0x0f, - 0x9b, 0x07, 0xfe, 0x23, 0x7c, 0xdc, 0x3c, 0xc6, 0xc7, 0x2f, 0x8e, 0x87, 0xcd, 0x43, 0xff, 0x10, 0xb7, 0x9a, 0xc7, - 0x50, 0xd8, 0x3c, 0x6e, 0x1e, 0xcf, 0x9b, 0x87, 0xc7, 0xc3, 0x96, 0x2c, 0xed, 0xf8, 0x47, 0x47, 0xcd, 0x76, 0xcb, - 0x3f, 0x3a, 0xc2, 0x47, 0xfe, 0xa3, 0x47, 0xcd, 0xf6, 0x81, 0xff, 0xe8, 0xd1, 0xcb, 0xa3, 0x63, 0xff, 0x00, 0xde, - 0x1d, 0x1c, 0x0c, 0x0f, 0xfc, 0x76, 0xbb, 0x09, 0xff, 0xe0, 0x63, 0xbf, 0xa3, 0x7e, 0xb4, 0xdb, 0xfe, 0x41, 0x1b, - 0xb7, 0x92, 0xa3, 0x8e, 0xff, 0xe8, 0x09, 0x96, 0xff, 0xca, 0x6a, 0x58, 0xfe, 0x03, 0xdd, 0xe0, 0x27, 0x7e, 0xe7, - 0x91, 0xfa, 0x25, 0x3b, 0x9c, 0x1f, 0x1e, 0xff, 0xe0, 0xee, 0x6f, 0x9d, 0x43, 0x5b, 0xcd, 0xe1, 0xf8, 0xc8, 0x3f, - 0x38, 0xc0, 0x87, 0x6d, 0xff, 0xf8, 0x20, 0x6e, 0x1e, 0x76, 0xfc, 0x47, 0x8f, 0x87, 0xcd, 0xb6, 0xff, 0xf8, 0x31, - 0x6e, 0x35, 0x0f, 0xfc, 0x0e, 0x6e, 0xfb, 0x87, 0x07, 0xf2, 0xc7, 0x81, 0xdf, 0x99, 0x3f, 0x7e, 0xe2, 0x3f, 0x3a, - 0x8a, 0x1f, 0xf9, 0x87, 0xdf, 0x1e, 0x1e, 0xfb, 0x9d, 0x83, 0xf8, 0xe0, 0x91, 0xdf, 0x79, 0x3c, 0x7f, 0xe4, 0x1f, - 0xc6, 0xcd, 0xce, 0xa3, 0x7b, 0x5b, 0xb6, 0x3b, 0x3e, 0xe0, 0x48, 0xbe, 0x86, 0x17, 0x58, 0xbf, 0x80, 0xbf, 0xb1, - 0x6c, 0xfb, 0xef, 0xd8, 0x4d, 0xbe, 0xde, 0xf4, 0x89, 0x7f, 0xfc, 0x78, 0xa8, 0xaa, 0x43, 0x41, 0xd3, 0xd4, 0x80, - 0x26, 0xf3, 0xa6, 0x1a, 0x56, 0x76, 0xd7, 0x34, 0x1d, 0x99, 0xbf, 0x7a, 0xb0, 0x79, 0x13, 0x06, 0x56, 0xe3, 0xfe, - 0x87, 0xf6, 0x53, 0x2e, 0xf9, 0xc9, 0xfe, 0x58, 0x91, 0xfe, 0xb8, 0xf7, 0x99, 0xba, 0xdd, 0xf9, 0xb3, 0x2b, 0x9c, - 0x6e, 0x73, 0x7c, 0x64, 0x9f, 0x76, 0x7c, 0x70, 0xfa, 0x10, 0xcf, 0x47, 0xf6, 0x87, 0x7b, 0x3e, 0x52, 0xba, 0xe2, - 0x38, 0xbf, 0x16, 0x6b, 0x0e, 0x8e, 0x55, 0xab, 0xf8, 0xa9, 0xf0, 0x06, 0x39, 0x7c, 0x47, 0xac, 0xe8, 0x5e, 0x0b, - 0xc2, 0xa9, 0xed, 0x07, 0xe2, 0xc0, 0x62, 0xaf, 0x85, 0xe2, 0xb1, 0xc9, 0x36, 0x84, 0x84, 0x9f, 0x46, 0xc8, 0xb7, - 0x0f, 0xc1, 0x47, 0xf8, 0x87, 0xe3, 0x23, 0xb1, 0xf1, 0x51, 0xf3, 0xe5, 0x4b, 0x4f, 0x83, 0xf4, 0x14, 0x9c, 0xcb, - 0x67, 0x0f, 0x0e, 0x51, 0x35, 0xdc, 0x7d, 0x0a, 0x45, 0xb9, 0xab, 0x22, 0x5f, 0xef, 0x7e, 0x4d, 0xd8, 0x41, 0x9d, - 0x98, 0x24, 0xae, 0x76, 0xcb, 0x4c, 0xa5, 0xd4, 0xd1, 0x0f, 0xa5, 0x50, 0xea, 0xf8, 0x2d, 0xbf, 0x55, 0xba, 0x74, - 0xe0, 0x94, 0x2c, 0x59, 0x70, 0x11, 0xc2, 0x17, 0x6b, 0x13, 0x3e, 0x96, 0xdf, 0xb6, 0x85, 0xaf, 0x09, 0x40, 0xd2, - 0xcf, 0x50, 0x7d, 0xc8, 0x21, 0x70, 0x5d, 0x7d, 0xb7, 0x06, 0x9c, 0xc2, 0xfc, 0x06, 0x4e, 0xaa, 0x9a, 0xa8, 0xc4, - 0x04, 0xbc, 0x1d, 0xaf, 0x68, 0xc4, 0x42, 0xcf, 0xf5, 0xa6, 0x19, 0x1d, 0xd1, 0x2c, 0x6f, 0xd6, 0x8e, 0x6f, 0xca, - 0x93, 0x9b, 0xc8, 0x35, 0x9f, 0x46, 0xcd, 0xe0, 0x76, 0x6c, 0x32, 0xd0, 0xfe, 0x46, 0x57, 0x1b, 0x60, 0x6e, 0x81, - 0x4d, 0x49, 0x06, 0xb2, 0xb6, 0x52, 0xda, 0x5c, 0xa5, 0xb5, 0xb5, 0xfd, 0xce, 0x11, 0x72, 0x64, 0x31, 0xdc, 0x3b, - 0xfc, 0xbd, 0xd7, 0x3c, 0x68, 0xfd, 0x09, 0x59, 0xcd, 0xca, 0x8e, 0x2e, 0xb4, 0xbb, 0x2d, 0xad, 0xbe, 0x29, 0x5d, - 0x3f, 0x5b, 0xeb, 0x2a, 0x8a, 0xf8, 0x5c, 0xcd, 0xdd, 0x45, 0xdd, 0x54, 0x47, 0xb8, 0xd5, 0x0d, 0x11, 0x23, 0x36, - 0xf6, 0xec, 0x2f, 0x06, 0xab, 0x7b, 0x8d, 0xe5, 0x87, 0xc6, 0x51, 0x51, 0x55, 0x49, 0xd1, 0x42, 0xc6, 0x5b, 0x58, - 0xea, 0xa4, 0xcb, 0xa5, 0x97, 0x82, 0x8b, 0x9c, 0x58, 0x38, 0x85, 0x67, 0x54, 0x43, 0x72, 0x8a, 0x4b, 0x80, 0x24, - 0x82, 0x49, 0xaa, 0xfe, 0xaf, 0x8a, 0xcd, 0x0f, 0xed, 0xf8, 0xf2, 0x93, 0x30, 0x1d, 0x03, 0x15, 0x86, 0xe9, 0x78, - 0xcd, 0xad, 0xa6, 0x42, 0x46, 0x2b, 0xa5, 0x55, 0x57, 0x95, 0xfb, 0x2c, 0x7f, 0x7a, 0xf7, 0x5e, 0x5f, 0x80, 0xe6, - 0x82, 0x77, 0x5a, 0x46, 0x38, 0xaa, 0xcb, 0x9a, 0x1b, 0xe4, 0x8b, 0x93, 0x09, 0x15, 0xa1, 0xca, 0xd7, 0x04, 0x7d, - 0x02, 0x4e, 0xcd, 0x3a, 0xda, 0x1a, 0x25, 0xae, 0x94, 0xee, 0x24, 0xa2, 0x73, 0x36, 0xd4, 0xa2, 0x1e, 0x3b, 0xfa, - 0xe6, 0x80, 0xa6, 0x5c, 0x1a, 0xd2, 0xc6, 0xca, 0x1f, 0x33, 0x0c, 0x65, 0x46, 0x3e, 0x49, 0xb9, 0xdb, 0xfb, 0xa2, - 0xfc, 0xfa, 0xe9, 0xb6, 0x45, 0x48, 0x58, 0xfa, 0x71, 0x90, 0xd1, 0xe4, 0x9f, 0xc8, 0x17, 0x6c, 0xc8, 0xd3, 0x2f, - 0x2e, 0xe0, 0xab, 0xf4, 0x7e, 0x9c, 0xd1, 0x11, 0xf9, 0x02, 0x64, 0x7c, 0x20, 0xad, 0x0f, 0x60, 0x84, 0x8d, 0xdb, - 0x49, 0x82, 0xa5, 0xc6, 0xf4, 0x00, 0x85, 0x48, 0x81, 0xeb, 0x76, 0x8e, 0x5c, 0x47, 0xd9, 0xc4, 0xf2, 0x77, 0x4f, - 0x89, 0x53, 0xa9, 0x04, 0x38, 0xed, 0x8e, 0x7f, 0x14, 0x77, 0xfc, 0x27, 0xf3, 0xc7, 0xfe, 0x71, 0xdc, 0x7e, 0x3c, - 0x6f, 0xc2, 0xff, 0x1d, 0xff, 0x49, 0xd2, 0xec, 0xf8, 0x4f, 0xe0, 0xef, 0xb7, 0x87, 0xfe, 0x51, 0xdc, 0x6c, 0xfb, - 0xc7, 0xf3, 0x03, 0xff, 0xe0, 0x65, 0xbb, 0xe3, 0x1f, 0x38, 0x6d, 0x47, 0xb5, 0x03, 0x76, 0xad, 0xb8, 0xf3, 0x17, - 0x2b, 0x1b, 0x62, 0x43, 0x38, 0x4e, 0xe5, 0x9c, 0xba, 0xd8, 0x2b, 0xbf, 0xb1, 0xa8, 0xf7, 0xa7, 0x76, 0xd6, 0x3d, - 0x0b, 0x33, 0xf8, 0xd0, 0x4d, 0x7d, 0xef, 0xd6, 0xde, 0xe1, 0x1a, 0xbf, 0xd8, 0x30, 0x04, 0xec, 0x70, 0x17, 0xdb, - 0x47, 0xef, 0xe1, 0xdc, 0xba, 0xbc, 0x17, 0xdc, 0x5c, 0x8f, 0xb8, 0x9d, 0xb4, 0x55, 0x45, 0x73, 0x05, 0xa3, 0x64, - 0x16, 0x4c, 0x7e, 0x81, 0x41, 0x0e, 0xf2, 0x55, 0x54, 0xac, 0x8e, 0x0f, 0xa9, 0xaf, 0x19, 0xb7, 0x6e, 0x1f, 0xa0, - 0xd5, 0x81, 0x8d, 0x88, 0xc1, 0x7d, 0x11, 0x45, 0x61, 0x40, 0xaf, 0xb9, 0x69, 0x2b, 0x2c, 0x49, 0x7e, 0x41, 0xf3, - 0xbe, 0x0b, 0x45, 0x6e, 0xe0, 0x4a, 0x17, 0x9f, 0x5b, 0x7e, 0xec, 0xa7, 0x24, 0xec, 0xaa, 0x00, 0xcb, 0x43, 0x57, - 0xb0, 0x6b, 0x01, 0x3f, 0x2e, 0xda, 0xdb, 0xdb, 0xba, 0x5f, 0xa4, 0x02, 0x09, 0x73, 0xad, 0xbe, 0x11, 0x62, 0xb3, - 0x22, 0xd7, 0x46, 0x74, 0xd9, 0xaf, 0x44, 0x21, 0xd2, 0x78, 0xba, 0xa6, 0xa1, 0xf0, 0xc3, 0x54, 0x25, 0xd1, 0x58, - 0x0c, 0x0b, 0xb7, 0xe9, 0x01, 0x2a, 0xb8, 0x08, 0xad, 0xef, 0x00, 0xeb, 0x7d, 0xce, 0x45, 0x68, 0xce, 0xd2, 0x5a, - 0xd7, 0x06, 0x81, 0xa3, 0x37, 0xee, 0xf4, 0xde, 0xbc, 0x3f, 0x75, 0xd4, 0xf6, 0x3c, 0xd9, 0x8f, 0x3b, 0xbd, 0x13, - 0xe9, 0x33, 0x51, 0x27, 0xf1, 0x88, 0x3a, 0x89, 0xe7, 0xe8, 0x53, 0x99, 0x10, 0x49, 0x2b, 0xf6, 0xd5, 0xb4, 0xa5, - 0xcd, 0xa0, 0xbc, 0xbd, 0x93, 0x59, 0x22, 0x18, 0xdc, 0x71, 0xbd, 0x2f, 0x8f, 0xe1, 0xc1, 0x82, 0x95, 0x79, 0xd8, - 0x5a, 0x3b, 0xbc, 0x16, 0xa9, 0xf1, 0x0d, 0x8f, 0x58, 0x42, 0x4d, 0xe6, 0xb5, 0xee, 0xaa, 0x3c, 0x29, 0xb0, 0x5e, - 0x3b, 0x9f, 0x5d, 0x4f, 0x98, 0x70, 0xcd, 0x79, 0x86, 0x0f, 0xba, 0xc1, 0x89, 0x1c, 0xaa, 0x77, 0x55, 0x68, 0xe7, - 0xb5, 0xf9, 0x9a, 0x4f, 0x7d, 0x49, 0xf5, 0xec, 0xb5, 0x84, 0x80, 0x13, 0x72, 0xf1, 0x41, 0xaf, 0x74, 0x17, 0xdb, - 0xef, 0x8a, 0x93, 0xfd, 0xf8, 0xa0, 0x77, 0x15, 0x4c, 0x75, 0x7f, 0x2f, 0xf9, 0x78, 0x73, 0x5f, 0x09, 0x1f, 0xf7, - 0xe5, 0x51, 0x10, 0x75, 0x48, 0xd9, 0x28, 0xbf, 0x3c, 0x71, 0x7b, 0x27, 0x5a, 0x19, 0x70, 0x64, 0x60, 0xdd, 0x3d, - 0x6a, 0x99, 0xd3, 0x25, 0x09, 0x1f, 0xc3, 0x86, 0x54, 0x4d, 0xac, 0x41, 0x6a, 0x1e, 0xf7, 0xb8, 0xdd, 0x3b, 0x09, - 0x1d, 0xc9, 0x5b, 0x24, 0xf3, 0xc8, 0x83, 0x7d, 0x68, 0x1c, 0xf3, 0x09, 0xf5, 0x19, 0xdf, 0xbf, 0xa1, 0xd7, 0xcd, - 0x70, 0xca, 0x2a, 0xf7, 0x36, 0x28, 0x1d, 0xe5, 0x90, 0xdc, 0x78, 0xc4, 0xf5, 0xd9, 0xab, 0x4e, 0xe5, 0x6e, 0x3b, - 0x04, 0x9b, 0xc7, 0xb8, 0xe6, 0xa4, 0x4f, 0xce, 0x02, 0x8b, 0xf7, 0x4e, 0xf6, 0xc3, 0x15, 0x8c, 0x48, 0x7e, 0x5f, - 0x68, 0x47, 0x3b, 0x18, 0x36, 0x40, 0x6f, 0xae, 0xa3, 0xc4, 0x81, 0x71, 0xc8, 0x6b, 0x41, 0x5d, 0xb8, 0xbd, 0x7f, - 0xfd, 0x1f, 0xff, 0x4b, 0xfb, 0xd8, 0x4f, 0xf6, 0xe3, 0xb6, 0xe9, 0x6b, 0x65, 0x55, 0x8a, 0x13, 0x38, 0xee, 0x59, - 0x05, 0x85, 0xe9, 0x6d, 0x73, 0x9c, 0xb1, 0xa8, 0x19, 0x87, 0xc9, 0xc8, 0xed, 0x6d, 0xc7, 0xa6, 0x7d, 0x6c, 0x4b, - 0x43, 0x5d, 0x2f, 0x02, 0x7a, 0xfd, 0x4d, 0x07, 0x8f, 0xcc, 0xf9, 0x15, 0xb9, 0xb5, 0xed, 0x63, 0x48, 0xd5, 0xee, - 0xab, 0x1d, 0x45, 0x4a, 0xf5, 0x27, 0xc2, 0x34, 0x07, 0x4c, 0x6b, 0x27, 0x90, 0x0a, 0xd7, 0x29, 0x83, 0x5a, 0xff, - 0xf7, 0x7f, 0xfe, 0x97, 0xff, 0x66, 0x1e, 0x21, 0x56, 0xf5, 0xaf, 0xff, 0xfd, 0x3f, 0xff, 0x9f, 0xff, 0xfd, 0x5f, - 0xe1, 0xd4, 0x8a, 0x8e, 0x67, 0x49, 0xa6, 0xe2, 0x54, 0xc1, 0x2c, 0xc5, 0x5d, 0x1c, 0x48, 0xec, 0x9c, 0xb0, 0x5c, - 0xb0, 0x61, 0xfd, 0x4c, 0xd2, 0xb9, 0x1c, 0x50, 0xee, 0x4c, 0x0d, 0x9d, 0xdc, 0xe1, 0x45, 0x45, 0x50, 0x35, 0x94, - 0x4b, 0xc2, 0x2d, 0x4e, 0xf6, 0x01, 0xdf, 0x0f, 0x3b, 0xc6, 0xe9, 0x97, 0xcb, 0xb1, 0x30, 0x64, 0x02, 0x25, 0x45, - 0x55, 0xee, 0x40, 0x6c, 0x65, 0x01, 0x8f, 0x41, 0xc7, 0x2a, 0x96, 0xab, 0x57, 0x6b, 0xd3, 0xfd, 0x69, 0x96, 0x0b, - 0x36, 0x02, 0x94, 0x2b, 0x3f, 0xb1, 0x0c, 0x63, 0x37, 0x41, 0x57, 0x4c, 0xee, 0x0a, 0xd9, 0x8b, 0x22, 0xd0, 0xc3, - 0xe3, 0x3f, 0x15, 0x7f, 0x99, 0x80, 0x46, 0xe6, 0x78, 0x93, 0xf0, 0x56, 0x9b, 0xe7, 0x8f, 0x5a, 0xad, 0xe9, 0x2d, - 0x5a, 0x54, 0x23, 0xe0, 0x6d, 0x83, 0x49, 0x3a, 0xb6, 0x3b, 0x94, 0xf1, 0xef, 0xd2, 0x8d, 0xdd, 0x72, 0xc0, 0x17, - 0xee, 0xb4, 0x8a, 0xe2, 0xcf, 0x0b, 0xe9, 0x49, 0x65, 0xbf, 0x40, 0x9c, 0x5a, 0x3b, 0x9d, 0xaf, 0xb9, 0x3d, 0xb9, - 0x85, 0xd5, 0xaa, 0xa3, 0x5a, 0xc5, 0xed, 0xf5, 0xd3, 0x89, 0x76, 0x9c, 0xdd, 0x8e, 0x90, 0x1f, 0x42, 0xcc, 0x3b, - 0x6e, 0xe3, 0xb8, 0xb3, 0x28, 0xbb, 0x17, 0x82, 0x4f, 0xec, 0xc0, 0x3a, 0x0d, 0xe9, 0x90, 0x8e, 0x8c, 0xb3, 0x5e, - 0xbf, 0x57, 0x41, 0xf3, 0x22, 0x3e, 0xd8, 0x30, 0x96, 0x06, 0x49, 0x06, 0xd4, 0x9d, 0x56, 0xf1, 0x39, 0xec, 0xc0, - 0xc5, 0x28, 0xe1, 0xa1, 0x08, 0x24, 0xc1, 0x76, 0xed, 0xf0, 0x7c, 0x08, 0x3c, 0x89, 0x2f, 0x2c, 0x78, 0xba, 0xaa, - 0x2a, 0xb8, 0xcd, 0xeb, 0x67, 0x48, 0x0b, 0x5f, 0x36, 0xb7, 0xbb, 0x52, 0x5e, 0xb7, 0x6f, 0x75, 0xd4, 0xfb, 0x5d, - 0xcd, 0x5d, 0xa5, 0x05, 0x52, 0x07, 0x6d, 0x7e, 0xaf, 0xe4, 0xba, 0x7a, 0xfb, 0xb5, 0xf0, 0x5c, 0x09, 0xa6, 0xbb, - 0x5a, 0x4b, 0x16, 0x42, 0xad, 0x77, 0xe4, 0xdb, 0xd2, 0x64, 0x0a, 0xa7, 0x53, 0x59, 0x11, 0x75, 0x4f, 0xf6, 0x95, - 0xa6, 0x0b, 0xdc, 0x43, 0xa6, 0x74, 0xa8, 0x0c, 0x0a, 0x5d, 0x49, 0x6f, 0x05, 0xf5, 0x4b, 0xe7, 0x56, 0xc0, 0xa7, - 0xe3, 0x7a, 0xff, 0x0f, 0x82, 0x7a, 0x0b, 0xa7, 0xcf, 0x89, 0x00, 0x00}; + 0x65, 0x7c, 0xed, 0x25, 0x00, 0x5b, 0x3e, 0xbd, 0x0f, 0xc7, 0xe5, 0xef, 0x57, 0x34, 0xcf, 0xc3, 0xb1, 0xae, 0xb9, + 0x3d, 0x9e, 0x26, 0x41, 0xb4, 0x63, 0x69, 0x06, 0x08, 0x88, 0x89, 0x01, 0x46, 0xc0, 0xa7, 0xa1, 0x43, 0x64, 0x30, + 0xf5, 0x7a, 0x74, 0x4d, 0xe2, 0xaa, 0x5e, 0x24, 0xc2, 0x71, 0x55, 0x70, 0x32, 0xcd, 0xa8, 0x2c, 0x55, 0x68, 0x2c, + 0x4e, 0xf6, 0xa1, 0x40, 0xbd, 0xde, 0x12, 0x45, 0x33, 0x0e, 0x94, 0xed, 0xb1, 0x34, 0xc7, 0x44, 0xd1, 0xec, 0x44, + 0xa5, 0x32, 0x4b, 0x69, 0x3d, 0x76, 0xf3, 0x79, 0x7b, 0x08, 0x7f, 0x74, 0x64, 0xe8, 0xf3, 0xd1, 0x68, 0x74, 0x6f, + 0x54, 0xed, 0xf3, 0x68, 0x44, 0x3b, 0xf4, 0xa8, 0x0b, 0x49, 0x2c, 0x4d, 0x1d, 0x8b, 0x69, 0x17, 0x12, 0x77, 0x8b, + 0x87, 0x55, 0x86, 0xb0, 0x8d, 0x88, 0x17, 0x0f, 0x8f, 0xb0, 0x15, 0xd3, 0x8c, 0x2e, 0x26, 0x61, 0x36, 0x66, 0x69, + 0xd0, 0x2a, 0xfc, 0xb9, 0x0e, 0x49, 0x7d, 0x7e, 0x7c, 0x7c, 0x5c, 0xf8, 0x91, 0x79, 0x6a, 0x45, 0x51, 0xe1, 0x0f, + 0x17, 0xe5, 0x34, 0x5a, 0xad, 0xd1, 0xa8, 0xf0, 0x99, 0x29, 0x38, 0xe8, 0x0c, 0xa3, 0x83, 0x4e, 0xe1, 0xdf, 0x58, + 0x35, 0x0a, 0x9f, 0xea, 0xa7, 0x8c, 0x46, 0xb5, 0x4c, 0x98, 0xc7, 0xad, 0x56, 0xe1, 0x2b, 0x42, 0x5b, 0x80, 0x59, + 0xaa, 0x7e, 0x06, 0xe1, 0x4c, 0x70, 0x60, 0xee, 0xdd, 0x44, 0x78, 0x83, 0x4b, 0x7d, 0xcb, 0x88, 0xfa, 0x26, 0x47, + 0x81, 0x2e, 0xf0, 0xcf, 0x76, 0xf0, 0x08, 0x88, 0x59, 0x06, 0x8d, 0x12, 0x13, 0x5b, 0xaa, 0xbd, 0x06, 0xca, 0x92, + 0xaf, 0x7f, 0x26, 0x49, 0x15, 0x53, 0x02, 0x4e, 0x06, 0x35, 0xd5, 0x65, 0x78, 0x94, 0x6e, 0x91, 0x1f, 0xec, 0xd3, + 0xf2, 0xe3, 0xee, 0x21, 0xe2, 0x83, 0xfd, 0xe1, 0xe2, 0x83, 0x52, 0x4b, 0x7c, 0x28, 0xe6, 0x71, 0x27, 0x88, 0x3b, + 0x8c, 0xe9, 0xf0, 0xe3, 0x35, 0xbf, 0x6d, 0xc2, 0x96, 0xc8, 0x5c, 0x29, 0x58, 0x76, 0x7f, 0x6b, 0xd6, 0x8c, 0xe9, + 0xcc, 0xfa, 0xa2, 0x87, 0x54, 0x1f, 0xde, 0xa4, 0xc4, 0x7d, 0x63, 0x6c, 0x5b, 0x55, 0x32, 0x1a, 0x11, 0xf7, 0xcd, + 0x68, 0xe4, 0x9a, 0xb3, 0x92, 0xa1, 0xa0, 0xb2, 0xd6, 0xeb, 0x5a, 0x89, 0xac, 0xf5, 0xe5, 0x97, 0x76, 0x99, 0x5d, + 0xa0, 0x43, 0x4f, 0x76, 0x98, 0x49, 0xbf, 0x89, 0x58, 0x0e, 0x5b, 0x0d, 0x3e, 0x34, 0x52, 0xbf, 0xab, 0x31, 0xad, + 0x5d, 0xab, 0x5d, 0x02, 0xbc, 0xe1, 0x2e, 0xf0, 0xd5, 0x8b, 0x02, 0xc6, 0xd4, 0xe4, 0x2d, 0x3e, 0xbd, 0xfb, 0x2a, + 0xf2, 0xee, 0x04, 0x2a, 0x58, 0xfe, 0x26, 0x5d, 0x39, 0x04, 0xa4, 0x60, 0x24, 0xc4, 0x9e, 0x56, 0x21, 0xf8, 0x78, + 0x9c, 0xc0, 0xb7, 0x5e, 0x16, 0xb5, 0xfb, 0x63, 0x55, 0xf3, 0x7e, 0x6d, 0xbe, 0x81, 0xdd, 0x50, 0xdf, 0xb6, 0x2a, + 0x3f, 0x3d, 0xa5, 0x92, 0xc7, 0xe7, 0xfa, 0x1b, 0x44, 0xd2, 0x2c, 0x5e, 0x68, 0x26, 0xbf, 0x50, 0x29, 0xc7, 0x02, + 0xd2, 0x6d, 0x54, 0xc7, 0x51, 0x51, 0xe8, 0xc3, 0x1a, 0x11, 0xcb, 0xa7, 0x70, 0xaf, 0xa9, 0x6a, 0x49, 0x3f, 0xc5, + 0xc2, 0xf3, 0x1b, 0x2b, 0xbe, 0x53, 0x5b, 0xae, 0xc2, 0x04, 0x78, 0x94, 0xc3, 0xfc, 0x4e, 0x14, 0xae, 0xf6, 0xbb, + 0x1b, 0x24, 0xba, 0x8e, 0xc2, 0xa7, 0x8a, 0x3c, 0x59, 0x33, 0x04, 0xe7, 0x77, 0xb9, 0x20, 0xe6, 0x95, 0x29, 0x28, + 0xec, 0xf8, 0xa5, 0x7c, 0xa3, 0xb0, 0x25, 0xa3, 0x25, 0xf9, 0x34, 0x4c, 0x15, 0x1b, 0x25, 0xae, 0xe2, 0x07, 0xbb, + 0x8b, 0x6a, 0xe5, 0x0b, 0xd7, 0x80, 0xad, 0x88, 0xb7, 0x77, 0xb2, 0x0f, 0x0d, 0x7a, 0x4e, 0x0d, 0xf4, 0x74, 0x2d, + 0xc8, 0xf2, 0x89, 0x74, 0x87, 0x2b, 0x3f, 0xbf, 0xc1, 0x7e, 0x7e, 0xe3, 0xfc, 0x79, 0xd1, 0xbc, 0xa1, 0xd7, 0x1f, + 0x99, 0x68, 0x8a, 0x70, 0xda, 0x04, 0xc3, 0x47, 0x3a, 0x47, 0x35, 0x7b, 0x96, 0x59, 0x7e, 0xea, 0xaa, 0x83, 0xee, + 0x2c, 0x87, 0xac, 0x08, 0xa9, 0xbe, 0x07, 0x29, 0x4f, 0x69, 0xb7, 0x9e, 0xcd, 0x69, 0x07, 0xd9, 0x0d, 0xb6, 0x2e, + 0x16, 0x1c, 0xb2, 0x28, 0xc4, 0x5d, 0xd0, 0xd2, 0x6c, 0xbd, 0x65, 0x22, 0xe8, 0xad, 0x8d, 0xf5, 0x03, 0x8d, 0xdc, + 0x86, 0x94, 0x5e, 0xd9, 0x7a, 0x26, 0xc1, 0xb6, 0x4c, 0x80, 0x4f, 0xe5, 0x36, 0x82, 0x4b, 0xd5, 0xfc, 0xb5, 0x92, + 0x42, 0x57, 0x8b, 0x65, 0x6e, 0xe3, 0x43, 0x20, 0x0b, 0xc2, 0x91, 0xa0, 0x19, 0x7e, 0x48, 0xcd, 0x6b, 0x79, 0x0c, + 0x69, 0x01, 0x62, 0x26, 0x68, 0x1f, 0x4f, 0x6f, 0x1f, 0xde, 0xfd, 0xfd, 0xd3, 0x2f, 0x34, 0x8e, 0xcc, 0xb5, 0x3c, + 0xae, 0xdb, 0x85, 0x8d, 0x90, 0x84, 0x77, 0x01, 0x4b, 0xa5, 0xcc, 0xbb, 0x06, 0xbf, 0x68, 0x77, 0xca, 0x75, 0x92, + 0x6e, 0x46, 0x13, 0xf9, 0x15, 0x3e, 0xbd, 0x14, 0x07, 0x8f, 0xa6, 0xb7, 0x66, 0x35, 0xda, 0x2b, 0xc9, 0xb7, 0x7f, + 0x68, 0x8e, 0xed, 0xf6, 0xa4, 0xde, 0x7a, 0x9e, 0xe8, 0xd1, 0xf4, 0xb6, 0xab, 0x04, 0x6d, 0x33, 0x53, 0x50, 0xb5, + 0xa6, 0xb7, 0x76, 0x96, 0x71, 0xd5, 0x91, 0xe3, 0x1f, 0xe4, 0x0e, 0x0d, 0x73, 0xda, 0x85, 0x7b, 0xc7, 0xd9, 0x30, + 0x4c, 0xb4, 0x30, 0x9f, 0xb0, 0x28, 0x4a, 0x68, 0xd7, 0xc8, 0x6b, 0xa7, 0xfd, 0x08, 0x92, 0x74, 0xed, 0x25, 0xab, + 0xaf, 0x8a, 0x85, 0xbc, 0x12, 0x4f, 0xe1, 0x75, 0xce, 0x13, 0xf8, 0xe8, 0xc7, 0x46, 0x74, 0xea, 0xec, 0xd5, 0x56, + 0x85, 0x3c, 0xf9, 0xbb, 0x3e, 0x97, 0xa3, 0xd6, 0x9f, 0xba, 0x72, 0xc1, 0x5b, 0x5d, 0xc1, 0xa7, 0x41, 0xf3, 0xa0, + 0x3e, 0x11, 0x78, 0x55, 0x4e, 0x01, 0x6f, 0x98, 0x16, 0x06, 0x69, 0xa5, 0xf8, 0xb4, 0xe3, 0xb7, 0x75, 0x99, 0xec, + 0x00, 0xf2, 0xc2, 0xca, 0xa2, 0xa2, 0x3e, 0x99, 0x7f, 0x9b, 0xdd, 0xf2, 0x64, 0xf3, 0x6e, 0x79, 0x62, 0x76, 0xcb, + 0xfd, 0x14, 0xfb, 0xf9, 0xa8, 0x0d, 0x7f, 0xba, 0xd5, 0x84, 0x82, 0x96, 0x73, 0x30, 0xbd, 0x75, 0x40, 0x4f, 0x6b, + 0x76, 0xa6, 0xb7, 0x2a, 0xc7, 0x1a, 0x62, 0x37, 0x2d, 0xc8, 0x3a, 0xc6, 0x2d, 0x07, 0x0a, 0xe1, 0x6f, 0xab, 0xf6, + 0xaa, 0x7d, 0x08, 0xef, 0xa0, 0xd5, 0xd1, 0xfa, 0xbb, 0xce, 0xfd, 0x9b, 0x36, 0x48, 0xb9, 0xf0, 0x02, 0xc3, 0x8d, + 0x91, 0x2f, 0xc2, 0xeb, 0x6b, 0x1a, 0x05, 0x23, 0x3e, 0x9c, 0xe5, 0xff, 0xa4, 0xe1, 0xd7, 0x48, 0xbc, 0x77, 0x4b, + 0xaf, 0xf4, 0x63, 0x9a, 0xaa, 0x8c, 0x6f, 0xd3, 0xc3, 0xa2, 0x5c, 0xa7, 0x20, 0x1f, 0x86, 0x09, 0xf5, 0x3a, 0xfe, + 0xe1, 0x86, 0x4d, 0xf0, 0xef, 0xb2, 0x36, 0x1b, 0x27, 0xf3, 0x7b, 0x91, 0x71, 0x2f, 0x12, 0x7e, 0x15, 0x0e, 0xec, + 0x35, 0x6c, 0x1d, 0x6f, 0x06, 0x77, 0x60, 0x46, 0xba, 0x30, 0x42, 0x41, 0xcb, 0x9d, 0x88, 0x8e, 0xc2, 0x59, 0x22, + 0xee, 0xef, 0x75, 0x1b, 0x65, 0xac, 0xf5, 0x7a, 0x0f, 0x43, 0xaf, 0xea, 0x3e, 0x90, 0x4b, 0x7f, 0xfe, 0xe4, 0x10, + 0xfe, 0xa8, 0xfc, 0xaf, 0xbb, 0x4a, 0x57, 0x57, 0x76, 0x2f, 0xe8, 0xea, 0xbb, 0x35, 0x65, 0x5c, 0x89, 0x70, 0xa9, + 0x8f, 0x3f, 0xb4, 0x36, 0x68, 0x95, 0x0f, 0xaa, 0xae, 0xb5, 0xac, 0x5f, 0x55, 0xfb, 0xd7, 0x75, 0xfe, 0xc0, 0xba, + 0x43, 0xa5, 0xb9, 0xd6, 0xeb, 0xea, 0xcf, 0x10, 0xae, 0x55, 0x36, 0x18, 0x97, 0xf5, 0x77, 0xc9, 0x5d, 0x69, 0xa2, + 0xa8, 0x68, 0x2c, 0x58, 0x29, 0xbb, 0xca, 0x4a, 0xc9, 0x29, 0xb9, 0x3a, 0xe9, 0xdf, 0x4e, 0x12, 0x67, 0xae, 0x8e, + 0x4b, 0x12, 0xb7, 0xed, 0xb7, 0x5c, 0x47, 0xe6, 0x01, 0xc0, 0xad, 0xed, 0xae, 0xfc, 0xbc, 0xad, 0xdb, 0x07, 0x4d, + 0x6b, 0x3e, 0x96, 0x9a, 0xdd, 0xcb, 0xf0, 0x8e, 0x66, 0x97, 0x1d, 0xd7, 0x01, 0x3f, 0x4d, 0x53, 0xa5, 0x4c, 0xc8, + 0x32, 0xa7, 0xe3, 0x3a, 0xb7, 0x93, 0x24, 0xcd, 0x89, 0x1b, 0x0b, 0x31, 0x0d, 0xd4, 0xf7, 0x6f, 0x6f, 0x0e, 0x7c, + 0x9e, 0x8d, 0xf7, 0x3b, 0xad, 0x56, 0x0b, 0x2e, 0x80, 0x75, 0x9d, 0x39, 0xa3, 0x37, 0x4f, 0xf9, 0x2d, 0x71, 0x5b, + 0x4e, 0xcb, 0x69, 0x77, 0x8e, 0x9d, 0x76, 0xe7, 0xd0, 0x7f, 0x74, 0xec, 0xf6, 0x3e, 0x73, 0x9c, 0x93, 0x88, 0x8e, + 0x72, 0xf8, 0xe1, 0x38, 0x27, 0x52, 0xf1, 0x52, 0xbf, 0x1d, 0xc7, 0x1f, 0x26, 0x79, 0xb3, 0xed, 0x2c, 0xf4, 0xa3, + 0xe3, 0xc0, 0xa1, 0xd2, 0xc0, 0xf9, 0x7c, 0xd4, 0x19, 0x1d, 0x8e, 0x9e, 0x74, 0x75, 0x71, 0xf1, 0x59, 0xad, 0x3a, + 0x56, 0xff, 0x77, 0xac, 0x66, 0xb9, 0xc8, 0xf8, 0x47, 0xaa, 0x73, 0x12, 0x1d, 0x10, 0x3d, 0x1b, 0x9b, 0x76, 0xd6, + 0x47, 0x6a, 0x1f, 0x5f, 0x0f, 0x47, 0x9d, 0xaa, 0xba, 0x84, 0x71, 0xbf, 0x04, 0xf2, 0x64, 0xdf, 0x80, 0x7e, 0x62, + 0xa3, 0xa9, 0xdd, 0xdc, 0x84, 0xa8, 0xb6, 0xab, 0xe7, 0x38, 0x36, 0xf3, 0x3b, 0x81, 0x33, 0x0c, 0x46, 0x57, 0x95, + 0x10, 0xb8, 0x4e, 0x44, 0xdc, 0x57, 0xed, 0xce, 0x31, 0x6e, 0xb7, 0x1f, 0xf9, 0x8f, 0x8e, 0x87, 0x2d, 0x7c, 0xe8, + 0x1f, 0x36, 0x0f, 0xfc, 0x47, 0xf8, 0xb8, 0x79, 0x8c, 0x8f, 0x5f, 0x1c, 0x0f, 0x9b, 0x87, 0xfe, 0x21, 0x6e, 0x35, + 0x8f, 0xa1, 0xb0, 0x79, 0xdc, 0x3c, 0x9e, 0x37, 0x0f, 0x8f, 0x87, 0x2d, 0x59, 0xda, 0xf1, 0x8f, 0x8e, 0x9a, 0xed, + 0x96, 0x7f, 0x74, 0x84, 0x8f, 0xfc, 0x47, 0x8f, 0x9a, 0xed, 0x03, 0xff, 0xd1, 0xa3, 0x97, 0x47, 0xc7, 0xfe, 0x01, + 0xbc, 0x3b, 0x38, 0x18, 0x1e, 0xf8, 0xed, 0x76, 0x13, 0xfe, 0xc1, 0xc7, 0x7e, 0x47, 0xfd, 0x68, 0xb7, 0xfd, 0x83, + 0x36, 0x6e, 0x25, 0x47, 0x1d, 0xff, 0xd1, 0x13, 0x2c, 0xff, 0x95, 0xd5, 0xb0, 0xfc, 0x07, 0xba, 0xc1, 0x4f, 0xfc, + 0xce, 0x23, 0xf5, 0x4b, 0x76, 0x38, 0x3f, 0x3c, 0xfe, 0xc1, 0xdd, 0xdf, 0x3a, 0x87, 0xb6, 0x9a, 0xc3, 0xf1, 0x91, + 0x7f, 0x70, 0x80, 0x0f, 0xdb, 0xfe, 0xf1, 0x41, 0xdc, 0x3c, 0xec, 0xf8, 0x8f, 0x1e, 0x0f, 0x9b, 0x6d, 0xff, 0xf1, + 0x63, 0xdc, 0x6a, 0x1e, 0xf8, 0x1d, 0xdc, 0xf6, 0x0f, 0x0f, 0xe4, 0x8f, 0x03, 0xbf, 0x33, 0x7f, 0xfc, 0xc4, 0x7f, + 0x74, 0x14, 0x3f, 0xf2, 0x0f, 0xbf, 0x3d, 0x3c, 0xf6, 0x3b, 0x07, 0xf1, 0xc1, 0x23, 0xbf, 0xf3, 0x78, 0xfe, 0xc8, + 0x3f, 0x8c, 0x9b, 0x9d, 0x47, 0xf7, 0xb6, 0x6c, 0x77, 0x7c, 0xc0, 0x91, 0x7c, 0x0d, 0x2f, 0xb0, 0x7e, 0x01, 0x7f, + 0x63, 0xd9, 0xf6, 0xdf, 0xb1, 0x9b, 0x7c, 0xbd, 0xe9, 0x13, 0xff, 0xf8, 0xf1, 0x50, 0x55, 0x87, 0x82, 0xa6, 0xa9, + 0x01, 0x4d, 0xe6, 0x4d, 0x35, 0xac, 0xec, 0xae, 0x69, 0x3a, 0x32, 0x7f, 0xf5, 0x60, 0xf3, 0x26, 0x0c, 0xac, 0xc6, + 0xfd, 0x0f, 0xed, 0xa7, 0x5c, 0xf2, 0x93, 0xfd, 0xb1, 0x22, 0xfd, 0x71, 0xef, 0x33, 0x75, 0xbb, 0xf3, 0x67, 0x57, + 0x38, 0xdd, 0xe6, 0xf8, 0xc8, 0x3e, 0xed, 0xf8, 0xe0, 0xf4, 0x21, 0x9e, 0x8f, 0xec, 0x0f, 0xf7, 0x7c, 0xa4, 0x74, + 0xc5, 0x71, 0x7e, 0x2d, 0xd6, 0x1c, 0x1c, 0xab, 0x56, 0xf1, 0x53, 0xe1, 0x0d, 0x72, 0xf8, 0x8e, 0x58, 0xd1, 0xbd, + 0x16, 0x84, 0x53, 0xdb, 0x0f, 0xc4, 0x81, 0xc5, 0x5e, 0x0b, 0xc5, 0x63, 0x93, 0x6d, 0x08, 0x09, 0x3f, 0x8d, 0x90, + 0x6f, 0x1f, 0x82, 0x8f, 0xf0, 0x0f, 0xc7, 0x47, 0x62, 0xe3, 0xa3, 0xe6, 0xcb, 0x97, 0x9e, 0x06, 0xe9, 0x29, 0x38, + 0x97, 0xcf, 0x1e, 0x1c, 0xa2, 0x6a, 0xb8, 0xfb, 0x14, 0x8a, 0x72, 0x57, 0x45, 0xbe, 0xde, 0xfd, 0x9a, 0xb0, 0x83, + 0x3a, 0x31, 0x49, 0x5c, 0xed, 0x96, 0x99, 0x4a, 0xa9, 0xa3, 0x1f, 0x4a, 0xa1, 0xd4, 0xf1, 0x5b, 0x7e, 0xab, 0x74, + 0xe9, 0xc0, 0x29, 0x59, 0xb2, 0xe0, 0x22, 0x84, 0x2f, 0xd6, 0x26, 0x7c, 0x2c, 0xbf, 0x6d, 0x0b, 0x5f, 0x13, 0x80, + 0xa4, 0x9f, 0xa1, 0xfa, 0x90, 0x43, 0xe0, 0xba, 0xfa, 0x6e, 0x0d, 0x38, 0x85, 0xf9, 0x0d, 0x9c, 0x54, 0x35, 0x51, + 0x89, 0x09, 0x78, 0x3b, 0x5e, 0xd1, 0x88, 0x85, 0x9e, 0xeb, 0x4d, 0x33, 0x3a, 0xa2, 0x59, 0xde, 0xac, 0x1d, 0xdf, + 0x94, 0x27, 0x37, 0x91, 0x6b, 0x3e, 0x8d, 0x9a, 0xc1, 0xed, 0xd8, 0x64, 0xa0, 0xfd, 0x8d, 0xae, 0x36, 0xc0, 0xdc, + 0x02, 0x9b, 0x92, 0x0c, 0x64, 0x6d, 0xa5, 0xb4, 0xb9, 0x4a, 0x6b, 0x6b, 0xfb, 0x9d, 0x23, 0xe4, 0xc8, 0x62, 0xb8, + 0x77, 0xf8, 0x7b, 0xaf, 0x79, 0xd0, 0xfa, 0x13, 0xb2, 0x9a, 0x95, 0x1d, 0x5d, 0x68, 0x77, 0x5b, 0x5a, 0x7d, 0x53, + 0xba, 0x7e, 0xb6, 0xd6, 0x55, 0x14, 0xf1, 0xb9, 0x9a, 0xbb, 0x8b, 0xba, 0xa9, 0x8e, 0x70, 0xab, 0x1b, 0x22, 0x46, + 0x6c, 0xec, 0xd9, 0x5f, 0x0c, 0x56, 0xf7, 0x1a, 0xcb, 0x0f, 0x8d, 0xa3, 0xa2, 0xaa, 0x92, 0xa2, 0x85, 0x8c, 0xb7, + 0xb0, 0xd4, 0x49, 0x97, 0x4b, 0x2f, 0x05, 0x17, 0x39, 0xb1, 0x70, 0x0a, 0xcf, 0xa8, 0x86, 0xe4, 0x14, 0x97, 0x00, + 0x49, 0x04, 0x93, 0x54, 0xfd, 0x5f, 0x15, 0x9b, 0x1f, 0xda, 0xf1, 0xe5, 0x27, 0x61, 0x3a, 0x06, 0x2a, 0x0c, 0xd3, + 0xf1, 0x9a, 0x5b, 0x4d, 0x85, 0x8c, 0x56, 0x4a, 0xab, 0xae, 0x2a, 0xf7, 0x59, 0xfe, 0xf4, 0xee, 0xbd, 0xbe, 0x00, + 0xcd, 0x05, 0xef, 0xb4, 0x8c, 0x70, 0x54, 0x97, 0x35, 0x37, 0xc8, 0x17, 0x27, 0x13, 0x2a, 0x42, 0x95, 0xaf, 0x09, + 0xfa, 0x04, 0x9c, 0x9a, 0x75, 0xb4, 0x35, 0x4a, 0x5c, 0x29, 0xdd, 0x49, 0x44, 0xe7, 0x6c, 0xa8, 0x45, 0x3d, 0x76, + 0xf4, 0xcd, 0x01, 0x4d, 0xb9, 0x34, 0xa4, 0x8d, 0x95, 0x3f, 0x66, 0x18, 0xca, 0x8c, 0x7c, 0x92, 0x72, 0xb7, 0xf7, + 0x45, 0xf9, 0xf5, 0xd3, 0x6d, 0x8b, 0x90, 0xb0, 0xf4, 0xe3, 0x20, 0xa3, 0xc9, 0x3f, 0x91, 0x2f, 0xd8, 0x90, 0xa7, + 0x5f, 0x5c, 0xc0, 0x57, 0xe9, 0xfd, 0x38, 0xa3, 0x23, 0xf2, 0x05, 0xc8, 0xf8, 0x40, 0x5a, 0x1f, 0xc0, 0x08, 0x1b, + 0xb7, 0x93, 0x04, 0x4b, 0x8d, 0xe9, 0x01, 0x0a, 0x91, 0x02, 0xd7, 0xed, 0x1c, 0xb9, 0x8e, 0xb2, 0x89, 0xe5, 0xef, + 0x9e, 0x12, 0xa7, 0x52, 0x09, 0x70, 0xda, 0x1d, 0xff, 0x28, 0xee, 0xf8, 0x4f, 0xe6, 0x8f, 0xfd, 0xe3, 0xb8, 0xfd, + 0x78, 0xde, 0x84, 0xff, 0x3b, 0xfe, 0x93, 0xa4, 0xd9, 0xf1, 0x9f, 0xc0, 0xdf, 0x6f, 0x0f, 0xfd, 0xa3, 0xb8, 0xd9, + 0xf6, 0x8f, 0xe7, 0x07, 0xfe, 0xc1, 0xcb, 0x76, 0xc7, 0x3f, 0x70, 0xda, 0x8e, 0x6a, 0x07, 0xec, 0x5a, 0x71, 0xe7, + 0x2f, 0x56, 0x36, 0xc4, 0x86, 0x70, 0x9c, 0xca, 0x39, 0x75, 0xb1, 0x57, 0x7e, 0x63, 0x51, 0xef, 0x4f, 0xed, 0xac, + 0x7b, 0x16, 0x66, 0xf0, 0xa1, 0x9b, 0xfa, 0xde, 0xad, 0xbd, 0xc3, 0x35, 0x7e, 0xb1, 0x61, 0x08, 0xd8, 0xe1, 0x2e, + 0xb6, 0x8f, 0xde, 0xc3, 0xb9, 0x75, 0x79, 0x2f, 0xb8, 0xb9, 0x1e, 0x71, 0x3b, 0x69, 0xab, 0x8a, 0xe6, 0x0a, 0x46, + 0xc9, 0x2c, 0x98, 0xfc, 0x02, 0x83, 0x1c, 0xe4, 0xab, 0xa8, 0x58, 0x1d, 0x1f, 0x52, 0x5f, 0x33, 0x6e, 0xdd, 0x3e, + 0x40, 0xab, 0x03, 0x1b, 0x11, 0x83, 0xfb, 0x22, 0x8a, 0xc2, 0x80, 0x5e, 0x73, 0xd3, 0x56, 0x58, 0x92, 0xfc, 0x82, + 0xe6, 0x7d, 0x17, 0x8a, 0xdc, 0xc0, 0x95, 0x2e, 0x3e, 0xb7, 0xfc, 0xd8, 0x4f, 0x49, 0xd8, 0x55, 0x01, 0x96, 0x87, + 0xae, 0x60, 0xd7, 0x02, 0x7e, 0x5c, 0xb4, 0xb7, 0xb7, 0x75, 0xbf, 0x48, 0x05, 0x12, 0xe6, 0x5a, 0x7d, 0x23, 0xc4, + 0x66, 0x45, 0xae, 0x8d, 0xe8, 0xb2, 0x5f, 0x89, 0x42, 0xa4, 0xf1, 0x74, 0x4d, 0x43, 0xe1, 0x87, 0xa9, 0x4a, 0xa2, + 0xb1, 0x18, 0x16, 0x6e, 0xd3, 0x03, 0x54, 0x70, 0x11, 0x5a, 0xdf, 0x01, 0xd6, 0xfb, 0x9c, 0x8b, 0xd0, 0x9c, 0xa5, + 0xb5, 0xae, 0x0d, 0x02, 0x47, 0x6f, 0xdc, 0xe9, 0xbd, 0x79, 0x7f, 0xea, 0xa8, 0xed, 0x79, 0xb2, 0x1f, 0x77, 0x7a, + 0x27, 0xd2, 0x67, 0xa2, 0x4e, 0xe2, 0x11, 0x75, 0x12, 0xcf, 0xd1, 0xa7, 0x32, 0x21, 0x92, 0x56, 0xec, 0xab, 0x69, + 0x4b, 0x9b, 0x41, 0x79, 0x7b, 0x27, 0xb3, 0x44, 0x30, 0xb8, 0xe3, 0x7a, 0x5f, 0x1e, 0xc3, 0x83, 0x05, 0x2b, 0xf3, + 0xb0, 0xb5, 0x76, 0x78, 0x2d, 0x52, 0xe3, 0x1b, 0x1e, 0xb1, 0x84, 0x9a, 0xcc, 0x6b, 0xdd, 0x55, 0x79, 0x52, 0x60, + 0xbd, 0x76, 0x3e, 0xbb, 0x9e, 0x30, 0xe1, 0x9a, 0xf3, 0x0c, 0x1f, 0x74, 0x83, 0x13, 0x39, 0x54, 0xef, 0xaa, 0xd0, + 0xce, 0x6b, 0xf3, 0x35, 0x9f, 0xfa, 0x92, 0xea, 0xd9, 0x6b, 0x09, 0x01, 0x27, 0xe4, 0xe2, 0x83, 0x5e, 0xe9, 0x2e, + 0xb6, 0xdf, 0x15, 0x27, 0xfb, 0xf1, 0x41, 0xef, 0x2a, 0x98, 0xea, 0xfe, 0x5e, 0xf2, 0xf1, 0xe6, 0xbe, 0x12, 0x3e, + 0xee, 0xcb, 0xa3, 0x20, 0xea, 0x90, 0xb2, 0x51, 0x7e, 0x79, 0xe2, 0xf6, 0x4e, 0xb4, 0x32, 0xe0, 0xc8, 0xc0, 0xba, + 0x7b, 0xd4, 0x32, 0xa7, 0x4b, 0x12, 0x3e, 0x86, 0x0d, 0xa9, 0x9a, 0x58, 0x83, 0xd4, 0x3c, 0xee, 0x71, 0xbb, 0x77, + 0x12, 0x3a, 0x92, 0xb7, 0x48, 0xe6, 0x91, 0x07, 0xfb, 0xd0, 0x38, 0xe6, 0x13, 0xea, 0x33, 0xbe, 0x7f, 0x43, 0xaf, + 0x9b, 0xe1, 0x94, 0x55, 0xee, 0x6d, 0x50, 0x3a, 0xca, 0x21, 0xb9, 0xf1, 0x88, 0xeb, 0xb3, 0x57, 0x9d, 0xca, 0xdd, + 0x76, 0x08, 0x36, 0x8f, 0x71, 0xcd, 0x49, 0x9f, 0x9c, 0x05, 0x16, 0xef, 0x9d, 0xec, 0x87, 0x2b, 0x18, 0x91, 0xfc, + 0xbe, 0xd0, 0x8e, 0x76, 0x30, 0x6c, 0x80, 0xde, 0x5c, 0x47, 0x89, 0x03, 0xe3, 0x90, 0xd7, 0x82, 0xba, 0x70, 0x7b, + 0xff, 0xfa, 0x3f, 0xfe, 0x97, 0xf6, 0xb1, 0x9f, 0xec, 0xc7, 0x6d, 0xd3, 0xd7, 0xca, 0xaa, 0x14, 0x27, 0x70, 0xdc, + 0xb3, 0x0a, 0x0a, 0xd3, 0xdb, 0xe6, 0x38, 0x63, 0x51, 0x33, 0x0e, 0x93, 0x91, 0xdb, 0xdb, 0x8e, 0x4d, 0xfb, 0xd8, + 0x96, 0x86, 0xba, 0x5e, 0x04, 0xf4, 0xfa, 0x9b, 0x0e, 0x1e, 0x99, 0xf3, 0x2b, 0x72, 0x6b, 0xdb, 0xc7, 0x90, 0xaa, + 0xdd, 0x57, 0x3b, 0x8a, 0x94, 0xea, 0x4f, 0x84, 0x69, 0x0e, 0x98, 0xd6, 0x4e, 0x20, 0x15, 0xae, 0x53, 0x06, 0xb5, + 0xfe, 0xef, 0xff, 0xfc, 0x2f, 0xff, 0xcd, 0x3c, 0x42, 0xac, 0xea, 0x5f, 0xff, 0xfb, 0x7f, 0xfe, 0x3f, 0xff, 0xfb, + 0xbf, 0xc2, 0xa9, 0x15, 0x1d, 0xcf, 0x92, 0x4c, 0xc5, 0xa9, 0x82, 0x59, 0x8a, 0xbb, 0x38, 0x90, 0xd8, 0x39, 0x61, + 0xb9, 0x60, 0xc3, 0xfa, 0x99, 0xa4, 0x73, 0x39, 0xa0, 0xdc, 0x99, 0x1a, 0x3a, 0xb9, 0xc3, 0x8b, 0x8a, 0xa0, 0x6a, + 0x28, 0x97, 0x84, 0x5b, 0x9c, 0xec, 0x03, 0xbe, 0x1f, 0x76, 0x8c, 0xd3, 0x2f, 0x97, 0x63, 0x61, 0xc8, 0x04, 0x4a, + 0x8a, 0xaa, 0xdc, 0x81, 0xd8, 0xca, 0x02, 0x1e, 0x83, 0x8e, 0x55, 0x2c, 0x57, 0xaf, 0xd6, 0xa6, 0xfb, 0xd3, 0x2c, + 0x17, 0x6c, 0x04, 0x28, 0x57, 0x7e, 0x62, 0x19, 0xc6, 0x6e, 0x82, 0xae, 0x98, 0xdc, 0x15, 0xb2, 0x17, 0x45, 0xa0, + 0x87, 0xc7, 0x7f, 0x2a, 0xfe, 0x32, 0x01, 0x8d, 0xcc, 0xf1, 0x26, 0xe1, 0xad, 0x36, 0xcf, 0x1f, 0xb5, 0x5a, 0xd3, + 0x5b, 0xb4, 0xa8, 0x46, 0xc0, 0xdb, 0x06, 0x93, 0x74, 0x6c, 0x77, 0x28, 0xe3, 0xdf, 0xa5, 0x1b, 0xbb, 0xe5, 0x80, + 0x2f, 0xdc, 0x69, 0x15, 0xc5, 0x9f, 0x17, 0xd2, 0x93, 0xca, 0x7e, 0x81, 0x38, 0xb5, 0x76, 0x3a, 0x5f, 0x73, 0x7b, + 0x72, 0x0b, 0xab, 0x55, 0x47, 0xb5, 0x8a, 0xdb, 0xeb, 0xa7, 0x13, 0xed, 0x38, 0xbb, 0x1d, 0x21, 0x3f, 0x84, 0x98, + 0x77, 0xdc, 0xc6, 0x71, 0x67, 0x51, 0x76, 0x2f, 0x04, 0x9f, 0xd8, 0x81, 0x75, 0x1a, 0xd2, 0x21, 0x1d, 0x19, 0x67, + 0xbd, 0x7e, 0xaf, 0x82, 0xe6, 0x45, 0x7c, 0xb0, 0x61, 0x2c, 0x0d, 0x92, 0x0c, 0xa8, 0x3b, 0xad, 0xe2, 0x73, 0xd8, + 0x81, 0x8b, 0x51, 0xc2, 0x43, 0x11, 0x48, 0x82, 0xed, 0xda, 0xe1, 0xf9, 0x10, 0x78, 0x12, 0x5f, 0x58, 0xf0, 0x74, + 0x55, 0x55, 0x70, 0x9b, 0xd7, 0xcf, 0x90, 0x16, 0xbe, 0x6c, 0x6e, 0x77, 0xa5, 0xbc, 0x6e, 0xdf, 0xea, 0xa8, 0xf7, + 0xbb, 0x9a, 0xbb, 0x4a, 0x0b, 0xa4, 0x0e, 0xda, 0xfc, 0x5e, 0xc9, 0x75, 0xf5, 0xf6, 0x6b, 0xe1, 0xb9, 0x12, 0x4c, + 0x77, 0xb5, 0x96, 0x2c, 0x84, 0x5a, 0xef, 0xc8, 0xb7, 0xa5, 0xc9, 0x14, 0x4e, 0xa7, 0xb2, 0x22, 0xea, 0x9e, 0xec, + 0x2b, 0x4d, 0x17, 0xb8, 0x87, 0x4c, 0xe9, 0x50, 0x19, 0x14, 0xba, 0x92, 0xde, 0x0a, 0xea, 0x97, 0xce, 0xad, 0x80, + 0x4f, 0xc7, 0xf5, 0xfe, 0x1f, 0xe7, 0xe0, 0x1c, 0x12, 0xcf, 0x89, 0x00, 0x00}; } // namespace web_server } // namespace esphome diff --git a/esphome/components/web_server/web_server.cpp b/esphome/components/web_server/web_server.cpp index 9ec667dbc5..1d08ef5a35 100644 --- a/esphome/components/web_server/web_server.cpp +++ b/esphome/components/web_server/web_server.cpp @@ -8,7 +8,7 @@ #include "esphome/core/log.h" #include "esphome/core/util.h" -#ifdef USE_ARDUINO +#if !defined(USE_ESP32) && defined(USE_ARDUINO) #include "StreamString.h" #endif @@ -103,19 +103,19 @@ static UrlMatch match_url(const char *url_ptr, size_t url_len, bool only_domain) return match; } -#ifdef USE_ARDUINO +#if !defined(USE_ESP32) && defined(USE_ARDUINO) // helper for allowing only unique entries in the queue void DeferredUpdateEventSource::deq_push_back_with_dedup_(void *source, message_generator_t *message_generator) { DeferredEvent item(source, message_generator); - auto iter = std::find_if(this->deferred_queue_.begin(), this->deferred_queue_.end(), - [&item](const DeferredEvent &test) -> bool { return test == item; }); - - if (iter != this->deferred_queue_.end()) { - (*iter) = item; - } else { - this->deferred_queue_.push_back(item); + // Use range-based for loop instead of std::find_if to reduce template instantiation overhead and binary size + for (auto &event : this->deferred_queue_) { + if (event == item) { + event = item; + return; + } } + this->deferred_queue_.push_back(item); } void DeferredUpdateEventSource::process_deferred_queue_() { @@ -127,6 +127,10 @@ void DeferredUpdateEventSource::process_deferred_queue_() { deferred_queue_.erase(deferred_queue_.begin()); this->consecutive_send_failures_ = 0; // Reset failure count on successful send } else { + // NOTE: Similar logic exists in web_server_idf/web_server_idf.cpp in AsyncEventSourceResponse::process_buffer_() + // The implementations differ due to platform-specific APIs (DISCARDED vs HTTPD_SOCK_ERR_TIMEOUT, close() vs + // fd_.store(0)), but the failure counting and timeout logic should be kept in sync. If you change this logic, + // also update the ESP-IDF implementation. this->consecutive_send_failures_++; if (this->consecutive_send_failures_ >= MAX_CONSECUTIVE_SEND_FAILURES) { // Too many failures, connection is likely dead @@ -148,6 +152,10 @@ void DeferredUpdateEventSource::loop() { void DeferredUpdateEventSource::deferrable_send_state(void *source, const char *event_type, message_generator_t *message_generator) { + // Skip if no connected clients to avoid unnecessary deferred queue processing + if (this->count() == 0) + return; + // allow all json "details_all" to go through before publishing bare state events, this avoids unnamed entries showing // up in the web GUI and reduces event load during initial connect if (!entities_iterator_.completed() && 0 != strcmp(event_type, "state_detail_all")) @@ -193,6 +201,9 @@ void DeferredUpdateEventSourceList::loop() { void DeferredUpdateEventSourceList::deferrable_send_state(void *source, const char *event_type, message_generator_t *message_generator) { + // Skip if no event sources (no connected clients) to avoid unnecessary iteration + if (this->empty()) + return; for (DeferredUpdateEventSource *dues : *this) { dues->deferrable_send_state(source, event_type, message_generator); } @@ -228,10 +239,11 @@ void DeferredUpdateEventSourceList::on_client_connect_(WebServer *ws, DeferredUp #ifdef USE_WEBSERVER_SORTING for (auto &group : ws->sorting_groups_) { - message = json::build_json([group](JsonObject root) { - root["name"] = group.second.name; - root["sorting_weight"] = group.second.weight; - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + root["name"] = group.second.name; + root["sorting_weight"] = group.second.weight; + message = builder.serialize(); // up to 31 groups should be able to be queued initially without defer source->try_send_nodefer(message.c_str(), "sorting_group"); @@ -265,21 +277,23 @@ void WebServer::set_js_include(const char *js_include) { this->js_include_ = js_ #endif std::string WebServer::get_config_json() { - return json::build_json([this](JsonObject root) { - root["title"] = App.get_friendly_name().empty() ? App.get_name() : App.get_friendly_name(); - root["comment"] = App.get_comment(); -#ifdef USE_WEBSERVER_OTA - root["ota"] = true; // web_server OTA platform is configured + json::JsonBuilder builder; + JsonObject root = builder.root(); + + root["title"] = App.get_friendly_name().empty() ? App.get_name() : App.get_friendly_name(); + root["comment"] = App.get_comment(); +#if defined(USE_WEBSERVER_OTA_DISABLED) || !defined(USE_WEBSERVER_OTA) + root["ota"] = false; // Note: USE_WEBSERVER_OTA_DISABLED only affects web_server, not captive_portal #else - root["ota"] = false; + root["ota"] = true; #endif - root["log"] = this->expose_log_; - root["lang"] = "en"; - }); + root["log"] = this->expose_log_; + root["lang"] = "en"; + + return builder.serialize(); } void WebServer::setup() { - ESP_LOGCONFIG(TAG, "Running setup"); this->setup_controller(this->include_internal_); this->base_->init(); @@ -294,7 +308,7 @@ void WebServer::setup() { } #endif -#ifdef USE_ESP_IDF +#ifdef USE_ESP32 this->base_->add_handler(&this->events_); #endif this->base_->add_handler(this); @@ -377,23 +391,37 @@ void WebServer::handle_js_request(AsyncWebServerRequest *request) { } #endif -#define set_json_id(root, obj, sensor, start_config) \ - (root)["id"] = sensor; \ - if (((start_config) == DETAIL_ALL)) { \ - (root)["name"] = (obj)->get_name(); \ - (root)["icon"] = (obj)->get_icon(); \ - (root)["entity_category"] = (obj)->get_entity_category(); \ - if ((obj)->is_disabled_by_default()) \ - (root)["is_disabled_by_default"] = (obj)->is_disabled_by_default(); \ +// Helper functions to reduce code size by avoiding macro expansion +static void set_json_id(JsonObject &root, EntityBase *obj, const char *prefix, JsonDetail start_config) { + char id_buf[160]; // object_id can be up to 128 chars + prefix + dash + null + const auto &object_id = obj->get_object_id(); + snprintf(id_buf, sizeof(id_buf), "%s-%s", prefix, object_id.c_str()); + root["id"] = id_buf; + if (start_config == DETAIL_ALL) { + root["name"] = obj->get_name(); + root["icon"] = obj->get_icon_ref(); + root["entity_category"] = obj->get_entity_category(); + bool is_disabled = obj->is_disabled_by_default(); + if (is_disabled) + root["is_disabled_by_default"] = is_disabled; } +} -#define set_json_value(root, obj, sensor, value, start_config) \ - set_json_id((root), (obj), sensor, start_config); \ - (root)["value"] = value; +// Keep as separate function even though only used once: reduces code size by ~48 bytes +// by allowing compiler to share code between template instantiations (bool, float, etc.) +template +static void set_json_value(JsonObject &root, EntityBase *obj, const char *prefix, const T &value, + JsonDetail start_config) { + set_json_id(root, obj, prefix, start_config); + root["value"] = value; +} -#define set_json_icon_state_value(root, obj, sensor, state, value, start_config) \ - set_json_value(root, obj, sensor, value, start_config); \ - (root)["state"] = state; +template +static void set_json_icon_state_value(JsonObject &root, EntityBase *obj, const char *prefix, const std::string &state, + const T &value, JsonDetail start_config) { + set_json_value(root, obj, prefix, value, start_config); + root["state"] = state; +} // Helper to get request detail parameter static JsonDetail get_request_detail(AsyncWebServerRequest *request) { @@ -403,8 +431,6 @@ static JsonDetail get_request_detail(AsyncWebServerRequest *request) { #ifdef USE_SENSOR void WebServer::on_sensor_update(sensor::Sensor *obj, float state) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", sensor_state_json_generator); } void WebServer::handle_sensor_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -427,29 +453,26 @@ std::string WebServer::sensor_all_json_generator(WebServer *web_server, void *so return web_server->sensor_json((sensor::Sensor *) (source), ((sensor::Sensor *) (source))->state, DETAIL_ALL); } std::string WebServer::sensor_json(sensor::Sensor *obj, float value, JsonDetail start_config) { - return json::build_json([this, obj, value, start_config](JsonObject root) { - std::string state; - if (std::isnan(value)) { - state = "NA"; - } else { - state = value_accuracy_to_string(value, obj->get_accuracy_decimals()); - if (!obj->get_unit_of_measurement().empty()) - state += " " + obj->get_unit_of_measurement(); - } - set_json_icon_state_value(root, obj, "sensor-" + obj->get_object_id(), state, value, start_config); - if (start_config == DETAIL_ALL) { - this->add_sorting_info_(root, obj); - if (!obj->get_unit_of_measurement().empty()) - root["uom"] = obj->get_unit_of_measurement(); - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + const auto uom_ref = obj->get_unit_of_measurement_ref(); + + std::string state = + std::isnan(value) ? "NA" : value_accuracy_with_uom_to_string(value, obj->get_accuracy_decimals(), uom_ref); + set_json_icon_state_value(root, obj, "sensor", state, value, start_config); + if (start_config == DETAIL_ALL) { + this->add_sorting_info_(root, obj); + if (!uom_ref.empty()) + root["uom"] = uom_ref; + } + + return builder.serialize(); } #endif #ifdef USE_TEXT_SENSOR void WebServer::on_text_sensor_update(text_sensor::TextSensor *obj, const std::string &state) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", text_sensor_state_json_generator); } void WebServer::handle_text_sensor_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -475,19 +498,20 @@ std::string WebServer::text_sensor_all_json_generator(WebServer *web_server, voi } std::string WebServer::text_sensor_json(text_sensor::TextSensor *obj, const std::string &value, JsonDetail start_config) { - return json::build_json([this, obj, value, start_config](JsonObject root) { - set_json_icon_state_value(root, obj, "text_sensor-" + obj->get_object_id(), value, value, start_config); - if (start_config == DETAIL_ALL) { - this->add_sorting_info_(root, obj); - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + set_json_icon_state_value(root, obj, "text_sensor", value, value, start_config); + if (start_config == DETAIL_ALL) { + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif #ifdef USE_SWITCH void WebServer::on_switch_update(switch_::Switch *obj, bool state) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", switch_state_json_generator); } void WebServer::handle_switch_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -499,14 +523,37 @@ void WebServer::handle_switch_request(AsyncWebServerRequest *request, const UrlM auto detail = get_request_detail(request); std::string data = this->switch_json(obj, obj->state, detail); request->send(200, "application/json", data.c_str()); - } else if (match.method_equals("toggle")) { - this->defer([obj]() { obj->toggle(); }); - request->send(200); + return; + } + + // Handle action methods with single defer and response + enum SwitchAction { NONE, TOGGLE, TURN_ON, TURN_OFF }; + SwitchAction action = NONE; + + if (match.method_equals("toggle")) { + action = TOGGLE; } else if (match.method_equals("turn_on")) { - this->defer([obj]() { obj->turn_on(); }); - request->send(200); + action = TURN_ON; } else if (match.method_equals("turn_off")) { - this->defer([obj]() { obj->turn_off(); }); + action = TURN_OFF; + } + + if (action != NONE) { + this->defer([obj, action]() { + switch (action) { + case TOGGLE: + obj->toggle(); + break; + case TURN_ON: + obj->turn_on(); + break; + case TURN_OFF: + obj->turn_off(); + break; + default: + break; + } + }); request->send(200); } else { request->send(404); @@ -522,13 +569,16 @@ std::string WebServer::switch_all_json_generator(WebServer *web_server, void *so return web_server->switch_json((switch_::Switch *) (source), ((switch_::Switch *) (source))->state, DETAIL_ALL); } std::string WebServer::switch_json(switch_::Switch *obj, bool value, JsonDetail start_config) { - return json::build_json([this, obj, value, start_config](JsonObject root) { - set_json_icon_state_value(root, obj, "switch-" + obj->get_object_id(), value ? "ON" : "OFF", value, start_config); - if (start_config == DETAIL_ALL) { - root["assumed_state"] = obj->assumed_state(); - this->add_sorting_info_(root, obj); - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + set_json_icon_state_value(root, obj, "switch", value ? "ON" : "OFF", value, start_config); + if (start_config == DETAIL_ALL) { + root["assumed_state"] = obj->assumed_state(); + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif @@ -559,19 +609,20 @@ std::string WebServer::button_all_json_generator(WebServer *web_server, void *so return web_server->button_json((button::Button *) (source), DETAIL_ALL); } std::string WebServer::button_json(button::Button *obj, JsonDetail start_config) { - return json::build_json([this, obj, start_config](JsonObject root) { - set_json_id(root, obj, "button-" + obj->get_object_id(), start_config); - if (start_config == DETAIL_ALL) { - this->add_sorting_info_(root, obj); - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + set_json_id(root, obj, "button", start_config); + if (start_config == DETAIL_ALL) { + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif #ifdef USE_BINARY_SENSOR void WebServer::on_binary_sensor_update(binary_sensor::BinarySensor *obj) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", binary_sensor_state_json_generator); } void WebServer::handle_binary_sensor_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -596,20 +647,20 @@ std::string WebServer::binary_sensor_all_json_generator(WebServer *web_server, v ((binary_sensor::BinarySensor *) (source))->state, DETAIL_ALL); } std::string WebServer::binary_sensor_json(binary_sensor::BinarySensor *obj, bool value, JsonDetail start_config) { - return json::build_json([this, obj, value, start_config](JsonObject root) { - set_json_icon_state_value(root, obj, "binary_sensor-" + obj->get_object_id(), value ? "ON" : "OFF", value, - start_config); - if (start_config == DETAIL_ALL) { - this->add_sorting_info_(root, obj); - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + set_json_icon_state_value(root, obj, "binary_sensor", value ? "ON" : "OFF", value, start_config); + if (start_config == DETAIL_ALL) { + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif #ifdef USE_FAN void WebServer::on_fan_update(fan::Fan *obj) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", fan_state_json_generator); } void WebServer::handle_fan_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -627,15 +678,8 @@ void WebServer::handle_fan_request(AsyncWebServerRequest *request, const UrlMatc } else if (match.method_equals("turn_on") || match.method_equals("turn_off")) { auto call = match.method_equals("turn_on") ? obj->turn_on() : obj->turn_off(); - if (request->hasParam("speed_level")) { - auto speed_level = request->getParam("speed_level")->value(); - auto val = parse_number(speed_level.c_str()); - if (!val.has_value()) { - ESP_LOGW(TAG, "Can't convert '%s' to number!", speed_level.c_str()); - return; - } - call.set_speed(*val); - } + parse_int_param_(request, "speed_level", call, &decltype(call)::set_speed); + if (request->hasParam("oscillation")) { auto speed = request->getParam("oscillation")->value(); auto val = parse_on_off(speed.c_str()); @@ -670,27 +714,27 @@ std::string WebServer::fan_all_json_generator(WebServer *web_server, void *sourc return web_server->fan_json((fan::Fan *) (source), DETAIL_ALL); } std::string WebServer::fan_json(fan::Fan *obj, JsonDetail start_config) { - return json::build_json([this, obj, start_config](JsonObject root) { - set_json_icon_state_value(root, obj, "fan-" + obj->get_object_id(), obj->state ? "ON" : "OFF", obj->state, - start_config); - const auto traits = obj->get_traits(); - if (traits.supports_speed()) { - root["speed_level"] = obj->speed; - root["speed_count"] = traits.supported_speed_count(); - } - if (obj->get_traits().supports_oscillation()) - root["oscillation"] = obj->oscillating; - if (start_config == DETAIL_ALL) { - this->add_sorting_info_(root, obj); - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + set_json_icon_state_value(root, obj, "fan", obj->state ? "ON" : "OFF", obj->state, start_config); + const auto traits = obj->get_traits(); + if (traits.supports_speed()) { + root["speed_level"] = obj->speed; + root["speed_count"] = traits.supported_speed_count(); + } + if (obj->get_traits().supports_oscillation()) + root["oscillation"] = obj->oscillating; + if (start_config == DETAIL_ALL) { + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif #ifdef USE_LIGHT void WebServer::on_light_update(light::LightState *obj) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", light_state_json_generator); } void WebServer::handle_light_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -707,69 +751,26 @@ void WebServer::handle_light_request(AsyncWebServerRequest *request, const UrlMa request->send(200); } else if (match.method_equals("turn_on")) { auto call = obj->turn_on(); - if (request->hasParam("brightness")) { - auto brightness = parse_number(request->getParam("brightness")->value().c_str()); - if (brightness.has_value()) { - call.set_brightness(*brightness / 255.0f); - } - } - if (request->hasParam("r")) { - auto r = parse_number(request->getParam("r")->value().c_str()); - if (r.has_value()) { - call.set_red(*r / 255.0f); - } - } - if (request->hasParam("g")) { - auto g = parse_number(request->getParam("g")->value().c_str()); - if (g.has_value()) { - call.set_green(*g / 255.0f); - } - } - if (request->hasParam("b")) { - auto b = parse_number(request->getParam("b")->value().c_str()); - if (b.has_value()) { - call.set_blue(*b / 255.0f); - } - } - if (request->hasParam("white_value")) { - auto white_value = parse_number(request->getParam("white_value")->value().c_str()); - if (white_value.has_value()) { - call.set_white(*white_value / 255.0f); - } - } - if (request->hasParam("color_temp")) { - auto color_temp = parse_number(request->getParam("color_temp")->value().c_str()); - if (color_temp.has_value()) { - call.set_color_temperature(*color_temp); - } - } - if (request->hasParam("flash")) { - auto flash = parse_number(request->getParam("flash")->value().c_str()); - if (flash.has_value()) { - call.set_flash_length(*flash * 1000); - } - } - if (request->hasParam("transition")) { - auto transition = parse_number(request->getParam("transition")->value().c_str()); - if (transition.has_value()) { - call.set_transition_length(*transition * 1000); - } - } - if (request->hasParam("effect")) { - const char *effect = request->getParam("effect")->value().c_str(); - call.set_effect(effect); - } + + // Parse color parameters + parse_light_param_(request, "brightness", call, &decltype(call)::set_brightness, 255.0f); + parse_light_param_(request, "r", call, &decltype(call)::set_red, 255.0f); + parse_light_param_(request, "g", call, &decltype(call)::set_green, 255.0f); + parse_light_param_(request, "b", call, &decltype(call)::set_blue, 255.0f); + parse_light_param_(request, "white_value", call, &decltype(call)::set_white, 255.0f); + parse_light_param_(request, "color_temp", call, &decltype(call)::set_color_temperature); + + // Parse timing parameters + parse_light_param_uint_(request, "flash", call, &decltype(call)::set_flash_length, 1000); + parse_light_param_uint_(request, "transition", call, &decltype(call)::set_transition_length, 1000); + + parse_string_param_(request, "effect", call, &decltype(call)::set_effect); this->defer([call]() mutable { call.perform(); }); request->send(200); } else if (match.method_equals("turn_off")) { auto call = obj->turn_off(); - if (request->hasParam("transition")) { - auto transition = parse_number(request->getParam("transition")->value().c_str()); - if (transition.has_value()) { - call.set_transition_length(*transition * 1000); - } - } + parse_light_param_uint_(request, "transition", call, &decltype(call)::set_transition_length, 1000); this->defer([call]() mutable { call.perform(); }); request->send(200); } else { @@ -786,27 +787,27 @@ std::string WebServer::light_all_json_generator(WebServer *web_server, void *sou return web_server->light_json((light::LightState *) (source), DETAIL_ALL); } std::string WebServer::light_json(light::LightState *obj, JsonDetail start_config) { - return json::build_json([this, obj, start_config](JsonObject root) { - set_json_id(root, obj, "light-" + obj->get_object_id(), start_config); - root["state"] = obj->remote_values.is_on() ? "ON" : "OFF"; + json::JsonBuilder builder; + JsonObject root = builder.root(); - light::LightJSONSchema::dump_json(*obj, root); - if (start_config == DETAIL_ALL) { - JsonArray opt = root["effects"].to(); - opt.add("None"); - for (auto const &option : obj->get_effects()) { - opt.add(option->get_name()); - } - this->add_sorting_info_(root, obj); + set_json_value(root, obj, "light", obj->remote_values.is_on() ? "ON" : "OFF", start_config); + + light::LightJSONSchema::dump_json(*obj, root); + if (start_config == DETAIL_ALL) { + JsonArray opt = root["effects"].to(); + opt.add("None"); + for (auto const &option : obj->get_effects()) { + opt.add(option->get_name()); } - }); + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif #ifdef USE_COVER void WebServer::on_cover_update(cover::Cover *obj) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", cover_state_json_generator); } void WebServer::handle_cover_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -822,15 +823,28 @@ void WebServer::handle_cover_request(AsyncWebServerRequest *request, const UrlMa } auto call = obj->make_call(); - if (match.method_equals("open")) { - call.set_command_open(); - } else if (match.method_equals("close")) { - call.set_command_close(); - } else if (match.method_equals("stop")) { - call.set_command_stop(); - } else if (match.method_equals("toggle")) { - call.set_command_toggle(); - } else if (!match.method_equals("set")) { + + // Lookup table for cover methods + static const struct { + const char *name; + cover::CoverCall &(cover::CoverCall::*action)(); + } METHODS[] = { + {"open", &cover::CoverCall::set_command_open}, + {"close", &cover::CoverCall::set_command_close}, + {"stop", &cover::CoverCall::set_command_stop}, + {"toggle", &cover::CoverCall::set_command_toggle}, + }; + + bool found = false; + for (const auto &method : METHODS) { + if (match.method_equals(method.name)) { + (call.*method.action)(); + found = true; + break; + } + } + + if (!found && !match.method_equals("set")) { request->send(404); return; } @@ -842,18 +856,8 @@ void WebServer::handle_cover_request(AsyncWebServerRequest *request, const UrlMa return; } - if (request->hasParam("position")) { - auto position = parse_number(request->getParam("position")->value().c_str()); - if (position.has_value()) { - call.set_position(*position); - } - } - if (request->hasParam("tilt")) { - auto tilt = parse_number(request->getParam("tilt")->value().c_str()); - if (tilt.has_value()) { - call.set_tilt(*tilt); - } - } + parse_float_param_(request, "position", call, &decltype(call)::set_position); + parse_float_param_(request, "tilt", call, &decltype(call)::set_tilt); this->defer([call]() mutable { call.perform(); }); request->send(200); @@ -865,29 +869,30 @@ std::string WebServer::cover_state_json_generator(WebServer *web_server, void *s return web_server->cover_json((cover::Cover *) (source), DETAIL_STATE); } std::string WebServer::cover_all_json_generator(WebServer *web_server, void *source) { - return web_server->cover_json((cover::Cover *) (source), DETAIL_STATE); + return web_server->cover_json((cover::Cover *) (source), DETAIL_ALL); } std::string WebServer::cover_json(cover::Cover *obj, JsonDetail start_config) { - return json::build_json([this, obj, start_config](JsonObject root) { - set_json_icon_state_value(root, obj, "cover-" + obj->get_object_id(), obj->is_fully_closed() ? "CLOSED" : "OPEN", - obj->position, start_config); - root["current_operation"] = cover::cover_operation_to_str(obj->current_operation); + json::JsonBuilder builder; + JsonObject root = builder.root(); - if (obj->get_traits().get_supports_position()) - root["position"] = obj->position; - if (obj->get_traits().get_supports_tilt()) - root["tilt"] = obj->tilt; - if (start_config == DETAIL_ALL) { - this->add_sorting_info_(root, obj); - } - }); + set_json_icon_state_value(root, obj, "cover", obj->is_fully_closed() ? "CLOSED" : "OPEN", obj->position, + start_config); + root["current_operation"] = cover::cover_operation_to_str(obj->current_operation); + + if (obj->get_traits().get_supports_position()) + root["position"] = obj->position; + if (obj->get_traits().get_supports_tilt()) + root["tilt"] = obj->tilt; + if (start_config == DETAIL_ALL) { + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif #ifdef USE_NUMBER void WebServer::on_number_update(number::Number *obj, float state) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", number_state_json_generator); } void WebServer::handle_number_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -907,11 +912,7 @@ void WebServer::handle_number_request(AsyncWebServerRequest *request, const UrlM } auto call = obj->make_call(); - if (request->hasParam("value")) { - auto value = parse_number(request->getParam("value")->value().c_str()); - if (value.has_value()) - call.set_value(*value); - } + parse_float_param_(request, "value", call, &decltype(call)::set_value); this->defer([call]() mutable { call.perform(); }); request->send(200); @@ -927,38 +928,36 @@ std::string WebServer::number_all_json_generator(WebServer *web_server, void *so return web_server->number_json((number::Number *) (source), ((number::Number *) (source))->state, DETAIL_ALL); } std::string WebServer::number_json(number::Number *obj, float value, JsonDetail start_config) { - return json::build_json([this, obj, value, start_config](JsonObject root) { - set_json_id(root, obj, "number-" + obj->get_object_id(), start_config); - if (start_config == DETAIL_ALL) { - root["min_value"] = - value_accuracy_to_string(obj->traits.get_min_value(), step_to_accuracy_decimals(obj->traits.get_step())); - root["max_value"] = - value_accuracy_to_string(obj->traits.get_max_value(), step_to_accuracy_decimals(obj->traits.get_step())); - root["step"] = - value_accuracy_to_string(obj->traits.get_step(), step_to_accuracy_decimals(obj->traits.get_step())); - root["mode"] = (int) obj->traits.get_mode(); - if (!obj->traits.get_unit_of_measurement().empty()) - root["uom"] = obj->traits.get_unit_of_measurement(); - this->add_sorting_info_(root, obj); - } - if (std::isnan(value)) { - root["value"] = "\"NaN\""; - root["state"] = "NA"; - } else { - root["value"] = value_accuracy_to_string(value, step_to_accuracy_decimals(obj->traits.get_step())); - std::string state = value_accuracy_to_string(value, step_to_accuracy_decimals(obj->traits.get_step())); - if (!obj->traits.get_unit_of_measurement().empty()) - state += " " + obj->traits.get_unit_of_measurement(); - root["state"] = state; - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + const auto uom_ref = obj->traits.get_unit_of_measurement_ref(); + + std::string val_str = std::isnan(value) + ? "\"NaN\"" + : value_accuracy_to_string(value, step_to_accuracy_decimals(obj->traits.get_step())); + std::string state_str = std::isnan(value) ? "NA" + : value_accuracy_with_uom_to_string( + value, step_to_accuracy_decimals(obj->traits.get_step()), uom_ref); + set_json_icon_state_value(root, obj, "number", state_str, val_str, start_config); + if (start_config == DETAIL_ALL) { + root["min_value"] = + value_accuracy_to_string(obj->traits.get_min_value(), step_to_accuracy_decimals(obj->traits.get_step())); + root["max_value"] = + value_accuracy_to_string(obj->traits.get_max_value(), step_to_accuracy_decimals(obj->traits.get_step())); + root["step"] = value_accuracy_to_string(obj->traits.get_step(), step_to_accuracy_decimals(obj->traits.get_step())); + root["mode"] = (int) obj->traits.get_mode(); + if (!uom_ref.empty()) + root["uom"] = uom_ref; + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif #ifdef USE_DATETIME_DATE void WebServer::on_date_update(datetime::DateEntity *obj) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", date_state_json_generator); } void WebServer::handle_date_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -983,10 +982,7 @@ void WebServer::handle_date_request(AsyncWebServerRequest *request, const UrlMat return; } - if (request->hasParam("value")) { - std::string value = request->getParam("value")->value().c_str(); // NOLINT - call.set_date(value); - } + parse_string_param_(request, "value", call, &decltype(call)::set_date); this->defer([call]() mutable { call.perform(); }); request->send(200); @@ -1002,22 +998,21 @@ std::string WebServer::date_all_json_generator(WebServer *web_server, void *sour return web_server->date_json((datetime::DateEntity *) (source), DETAIL_ALL); } std::string WebServer::date_json(datetime::DateEntity *obj, JsonDetail start_config) { - return json::build_json([this, obj, start_config](JsonObject root) { - set_json_id(root, obj, "date-" + obj->get_object_id(), start_config); - std::string value = str_sprintf("%d-%02d-%02d", obj->year, obj->month, obj->day); - root["value"] = value; - root["state"] = value; - if (start_config == DETAIL_ALL) { - this->add_sorting_info_(root, obj); - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + std::string value = str_sprintf("%d-%02d-%02d", obj->year, obj->month, obj->day); + set_json_icon_state_value(root, obj, "date", value, value, start_config); + if (start_config == DETAIL_ALL) { + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif // USE_DATETIME_DATE #ifdef USE_DATETIME_TIME void WebServer::on_time_update(datetime::TimeEntity *obj) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", time_state_json_generator); } void WebServer::handle_time_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -1042,10 +1037,7 @@ void WebServer::handle_time_request(AsyncWebServerRequest *request, const UrlMat return; } - if (request->hasParam("value")) { - std::string value = request->getParam("value")->value().c_str(); // NOLINT - call.set_time(value); - } + parse_string_param_(request, "value", call, &decltype(call)::set_time); this->defer([call]() mutable { call.perform(); }); request->send(200); @@ -1060,22 +1052,21 @@ std::string WebServer::time_all_json_generator(WebServer *web_server, void *sour return web_server->time_json((datetime::TimeEntity *) (source), DETAIL_ALL); } std::string WebServer::time_json(datetime::TimeEntity *obj, JsonDetail start_config) { - return json::build_json([this, obj, start_config](JsonObject root) { - set_json_id(root, obj, "time-" + obj->get_object_id(), start_config); - std::string value = str_sprintf("%02d:%02d:%02d", obj->hour, obj->minute, obj->second); - root["value"] = value; - root["state"] = value; - if (start_config == DETAIL_ALL) { - this->add_sorting_info_(root, obj); - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + std::string value = str_sprintf("%02d:%02d:%02d", obj->hour, obj->minute, obj->second); + set_json_icon_state_value(root, obj, "time", value, value, start_config); + if (start_config == DETAIL_ALL) { + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif // USE_DATETIME_TIME #ifdef USE_DATETIME_DATETIME void WebServer::on_datetime_update(datetime::DateTimeEntity *obj) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", datetime_state_json_generator); } void WebServer::handle_datetime_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -1100,10 +1091,7 @@ void WebServer::handle_datetime_request(AsyncWebServerRequest *request, const Ur return; } - if (request->hasParam("value")) { - std::string value = request->getParam("value")->value().c_str(); // NOLINT - call.set_datetime(value); - } + parse_string_param_(request, "value", call, &decltype(call)::set_datetime); this->defer([call]() mutable { call.perform(); }); request->send(200); @@ -1118,23 +1106,22 @@ std::string WebServer::datetime_all_json_generator(WebServer *web_server, void * return web_server->datetime_json((datetime::DateTimeEntity *) (source), DETAIL_ALL); } std::string WebServer::datetime_json(datetime::DateTimeEntity *obj, JsonDetail start_config) { - return json::build_json([this, obj, start_config](JsonObject root) { - set_json_id(root, obj, "datetime-" + obj->get_object_id(), start_config); - std::string value = str_sprintf("%d-%02d-%02d %02d:%02d:%02d", obj->year, obj->month, obj->day, obj->hour, - obj->minute, obj->second); - root["value"] = value; - root["state"] = value; - if (start_config == DETAIL_ALL) { - this->add_sorting_info_(root, obj); - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + std::string value = + str_sprintf("%d-%02d-%02d %02d:%02d:%02d", obj->year, obj->month, obj->day, obj->hour, obj->minute, obj->second); + set_json_icon_state_value(root, obj, "datetime", value, value, start_config); + if (start_config == DETAIL_ALL) { + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif // USE_DATETIME_DATETIME #ifdef USE_TEXT void WebServer::on_text_update(text::Text *obj, const std::string &state) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", text_state_json_generator); } void WebServer::handle_text_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -1154,10 +1141,7 @@ void WebServer::handle_text_request(AsyncWebServerRequest *request, const UrlMat } auto call = obj->make_call(); - if (request->hasParam("value")) { - String value = request->getParam("value")->value(); - call.set_value(value.c_str()); // NOLINT - } + parse_string_param_(request, "value", call, &decltype(call)::set_value); this->defer([call]() mutable { call.perform(); }); request->send(200); @@ -1173,29 +1157,25 @@ std::string WebServer::text_all_json_generator(WebServer *web_server, void *sour return web_server->text_json((text::Text *) (source), ((text::Text *) (source))->state, DETAIL_ALL); } std::string WebServer::text_json(text::Text *obj, const std::string &value, JsonDetail start_config) { - return json::build_json([this, obj, value, start_config](JsonObject root) { - set_json_id(root, obj, "text-" + obj->get_object_id(), start_config); - root["min_length"] = obj->traits.get_min_length(); - root["max_length"] = obj->traits.get_max_length(); - root["pattern"] = obj->traits.get_pattern(); - if (obj->traits.get_mode() == text::TextMode::TEXT_MODE_PASSWORD) { - root["state"] = "********"; - } else { - root["state"] = value; - } - root["value"] = value; - if (start_config == DETAIL_ALL) { - root["mode"] = (int) obj->traits.get_mode(); - this->add_sorting_info_(root, obj); - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + std::string state = obj->traits.get_mode() == text::TextMode::TEXT_MODE_PASSWORD ? "********" : value; + set_json_icon_state_value(root, obj, "text", state, value, start_config); + root["min_length"] = obj->traits.get_min_length(); + root["max_length"] = obj->traits.get_max_length(); + root["pattern"] = obj->traits.get_pattern(); + if (start_config == DETAIL_ALL) { + root["mode"] = (int) obj->traits.get_mode(); + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif #ifdef USE_SELECT void WebServer::on_select_update(select::Select *obj, const std::string &state, size_t index) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", select_state_json_generator); } void WebServer::handle_select_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -1216,11 +1196,7 @@ void WebServer::handle_select_request(AsyncWebServerRequest *request, const UrlM } auto call = obj->make_call(); - - if (request->hasParam("option")) { - auto option = request->getParam("option")->value(); - call.set_option(option.c_str()); // NOLINT - } + parse_string_param_(request, "option", call, &decltype(call)::set_option); this->defer([call]() mutable { call.perform(); }); request->send(200); @@ -1235,26 +1211,27 @@ std::string WebServer::select_all_json_generator(WebServer *web_server, void *so return web_server->select_json((select::Select *) (source), ((select::Select *) (source))->state, DETAIL_ALL); } std::string WebServer::select_json(select::Select *obj, const std::string &value, JsonDetail start_config) { - return json::build_json([this, obj, value, start_config](JsonObject root) { - set_json_icon_state_value(root, obj, "select-" + obj->get_object_id(), value, value, start_config); - if (start_config == DETAIL_ALL) { - JsonArray opt = root["option"].to(); - for (auto &option : obj->traits.get_options()) { - opt.add(option); - } - this->add_sorting_info_(root, obj); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + set_json_icon_state_value(root, obj, "select", value, value, start_config); + if (start_config == DETAIL_ALL) { + JsonArray opt = root["option"].to(); + for (auto &option : obj->traits.get_options()) { + opt.add(option); } - }); + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif // Longest: HORIZONTAL -#define PSTR_LOCAL(mode_s) strncpy_P(buf, (PGM_P) ((mode_s)), 15) +#define PSTR_LOCAL(mode_s) ESPHOME_strncpy_P(buf, (ESPHOME_PGM_P) ((mode_s)), 15) #ifdef USE_CLIMATE void WebServer::on_climate_update(climate::Climate *obj) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", climate_state_json_generator); } void WebServer::handle_climate_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -1276,38 +1253,15 @@ void WebServer::handle_climate_request(AsyncWebServerRequest *request, const Url auto call = obj->make_call(); - if (request->hasParam("mode")) { - auto mode = request->getParam("mode")->value(); - call.set_mode(mode.c_str()); // NOLINT - } + // Parse string mode parameters + parse_string_param_(request, "mode", call, &decltype(call)::set_mode); + parse_string_param_(request, "fan_mode", call, &decltype(call)::set_fan_mode); + parse_string_param_(request, "swing_mode", call, &decltype(call)::set_swing_mode); - if (request->hasParam("fan_mode")) { - auto mode = request->getParam("fan_mode")->value(); - call.set_fan_mode(mode.c_str()); // NOLINT - } - - if (request->hasParam("swing_mode")) { - auto mode = request->getParam("swing_mode")->value(); - call.set_swing_mode(mode.c_str()); // NOLINT - } - - if (request->hasParam("target_temperature_high")) { - auto target_temperature_high = parse_number(request->getParam("target_temperature_high")->value().c_str()); - if (target_temperature_high.has_value()) - call.set_target_temperature_high(*target_temperature_high); - } - - if (request->hasParam("target_temperature_low")) { - auto target_temperature_low = parse_number(request->getParam("target_temperature_low")->value().c_str()); - if (target_temperature_low.has_value()) - call.set_target_temperature_low(*target_temperature_low); - } - - if (request->hasParam("target_temperature")) { - auto target_temperature = parse_number(request->getParam("target_temperature")->value().c_str()); - if (target_temperature.has_value()) - call.set_target_temperature(*target_temperature); - } + // Parse temperature parameters + parse_float_param_(request, "target_temperature_high", call, &decltype(call)::set_target_temperature_high); + parse_float_param_(request, "target_temperature_low", call, &decltype(call)::set_target_temperature_low); + parse_float_param_(request, "target_temperature", call, &decltype(call)::set_target_temperature); this->defer([call]() mutable { call.perform(); }); request->send(200); @@ -1316,106 +1270,109 @@ void WebServer::handle_climate_request(AsyncWebServerRequest *request, const Url request->send(404); } std::string WebServer::climate_state_json_generator(WebServer *web_server, void *source) { + // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson return web_server->climate_json((climate::Climate *) (source), DETAIL_STATE); } std::string WebServer::climate_all_json_generator(WebServer *web_server, void *source) { + // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson return web_server->climate_json((climate::Climate *) (source), DETAIL_ALL); } std::string WebServer::climate_json(climate::Climate *obj, JsonDetail start_config) { // NOLINTBEGIN(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson - return json::build_json([this, obj, start_config](JsonObject root) { - set_json_id(root, obj, "climate-" + obj->get_object_id(), start_config); - const auto traits = obj->get_traits(); - int8_t target_accuracy = traits.get_target_temperature_accuracy_decimals(); - int8_t current_accuracy = traits.get_current_temperature_accuracy_decimals(); - char buf[16]; + json::JsonBuilder builder; + JsonObject root = builder.root(); + set_json_id(root, obj, "climate", start_config); + const auto traits = obj->get_traits(); + int8_t target_accuracy = traits.get_target_temperature_accuracy_decimals(); + int8_t current_accuracy = traits.get_current_temperature_accuracy_decimals(); + char buf[16]; - if (start_config == DETAIL_ALL) { - JsonArray opt = root["modes"].to(); - for (climate::ClimateMode m : traits.get_supported_modes()) - opt.add(PSTR_LOCAL(climate::climate_mode_to_string(m))); - if (!traits.get_supported_custom_fan_modes().empty()) { - JsonArray opt = root["fan_modes"].to(); - for (climate::ClimateFanMode m : traits.get_supported_fan_modes()) - opt.add(PSTR_LOCAL(climate::climate_fan_mode_to_string(m))); - } - - if (!traits.get_supported_custom_fan_modes().empty()) { - JsonArray opt = root["custom_fan_modes"].to(); - for (auto const &custom_fan_mode : traits.get_supported_custom_fan_modes()) - opt.add(custom_fan_mode); - } - if (traits.get_supports_swing_modes()) { - JsonArray opt = root["swing_modes"].to(); - for (auto swing_mode : traits.get_supported_swing_modes()) - opt.add(PSTR_LOCAL(climate::climate_swing_mode_to_string(swing_mode))); - } - if (traits.get_supports_presets() && obj->preset.has_value()) { - JsonArray opt = root["presets"].to(); - for (climate::ClimatePreset m : traits.get_supported_presets()) - opt.add(PSTR_LOCAL(climate::climate_preset_to_string(m))); - } - if (!traits.get_supported_custom_presets().empty() && obj->custom_preset.has_value()) { - JsonArray opt = root["custom_presets"].to(); - for (auto const &custom_preset : traits.get_supported_custom_presets()) - opt.add(custom_preset); - } - this->add_sorting_info_(root, obj); + if (start_config == DETAIL_ALL) { + JsonArray opt = root["modes"].to(); + for (climate::ClimateMode m : traits.get_supported_modes()) + opt.add(PSTR_LOCAL(climate::climate_mode_to_string(m))); + if (!traits.get_supported_custom_fan_modes().empty()) { + JsonArray opt = root["fan_modes"].to(); + for (climate::ClimateFanMode m : traits.get_supported_fan_modes()) + opt.add(PSTR_LOCAL(climate::climate_fan_mode_to_string(m))); } - bool has_state = false; - root["mode"] = PSTR_LOCAL(climate_mode_to_string(obj->mode)); - root["max_temp"] = value_accuracy_to_string(traits.get_visual_max_temperature(), target_accuracy); - root["min_temp"] = value_accuracy_to_string(traits.get_visual_min_temperature(), target_accuracy); - root["step"] = traits.get_visual_target_temperature_step(); - if (traits.get_supports_action()) { - root["action"] = PSTR_LOCAL(climate_action_to_string(obj->action)); - root["state"] = root["action"]; - has_state = true; - } - if (traits.get_supports_fan_modes() && obj->fan_mode.has_value()) { - root["fan_mode"] = PSTR_LOCAL(climate_fan_mode_to_string(obj->fan_mode.value())); - } - if (!traits.get_supported_custom_fan_modes().empty() && obj->custom_fan_mode.has_value()) { - root["custom_fan_mode"] = obj->custom_fan_mode.value().c_str(); - } - if (traits.get_supports_presets() && obj->preset.has_value()) { - root["preset"] = PSTR_LOCAL(climate_preset_to_string(obj->preset.value())); - } - if (!traits.get_supported_custom_presets().empty() && obj->custom_preset.has_value()) { - root["custom_preset"] = obj->custom_preset.value().c_str(); + if (!traits.get_supported_custom_fan_modes().empty()) { + JsonArray opt = root["custom_fan_modes"].to(); + for (auto const &custom_fan_mode : traits.get_supported_custom_fan_modes()) + opt.add(custom_fan_mode); } if (traits.get_supports_swing_modes()) { - root["swing_mode"] = PSTR_LOCAL(climate_swing_mode_to_string(obj->swing_mode)); + JsonArray opt = root["swing_modes"].to(); + for (auto swing_mode : traits.get_supported_swing_modes()) + opt.add(PSTR_LOCAL(climate::climate_swing_mode_to_string(swing_mode))); } - if (traits.get_supports_current_temperature()) { - if (!std::isnan(obj->current_temperature)) { - root["current_temperature"] = value_accuracy_to_string(obj->current_temperature, current_accuracy); - } else { - root["current_temperature"] = "NA"; - } + if (traits.get_supports_presets() && obj->preset.has_value()) { + JsonArray opt = root["presets"].to(); + for (climate::ClimatePreset m : traits.get_supported_presets()) + opt.add(PSTR_LOCAL(climate::climate_preset_to_string(m))); } - if (traits.get_supports_two_point_target_temperature()) { - root["target_temperature_low"] = value_accuracy_to_string(obj->target_temperature_low, target_accuracy); - root["target_temperature_high"] = value_accuracy_to_string(obj->target_temperature_high, target_accuracy); - if (!has_state) { - root["state"] = value_accuracy_to_string((obj->target_temperature_high + obj->target_temperature_low) / 2.0f, - target_accuracy); - } + if (!traits.get_supported_custom_presets().empty() && obj->custom_preset.has_value()) { + JsonArray opt = root["custom_presets"].to(); + for (auto const &custom_preset : traits.get_supported_custom_presets()) + opt.add(custom_preset); + } + this->add_sorting_info_(root, obj); + } + + bool has_state = false; + root["mode"] = PSTR_LOCAL(climate_mode_to_string(obj->mode)); + root["max_temp"] = value_accuracy_to_string(traits.get_visual_max_temperature(), target_accuracy); + root["min_temp"] = value_accuracy_to_string(traits.get_visual_min_temperature(), target_accuracy); + root["step"] = traits.get_visual_target_temperature_step(); + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_ACTION)) { + root["action"] = PSTR_LOCAL(climate_action_to_string(obj->action)); + root["state"] = root["action"]; + has_state = true; + } + if (traits.get_supports_fan_modes() && obj->fan_mode.has_value()) { + root["fan_mode"] = PSTR_LOCAL(climate_fan_mode_to_string(obj->fan_mode.value())); + } + if (!traits.get_supported_custom_fan_modes().empty() && obj->custom_fan_mode.has_value()) { + root["custom_fan_mode"] = obj->custom_fan_mode.value().c_str(); + } + if (traits.get_supports_presets() && obj->preset.has_value()) { + root["preset"] = PSTR_LOCAL(climate_preset_to_string(obj->preset.value())); + } + if (!traits.get_supported_custom_presets().empty() && obj->custom_preset.has_value()) { + root["custom_preset"] = obj->custom_preset.value().c_str(); + } + if (traits.get_supports_swing_modes()) { + root["swing_mode"] = PSTR_LOCAL(climate_swing_mode_to_string(obj->swing_mode)); + } + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_CURRENT_TEMPERATURE)) { + if (!std::isnan(obj->current_temperature)) { + root["current_temperature"] = value_accuracy_to_string(obj->current_temperature, current_accuracy); } else { - root["target_temperature"] = value_accuracy_to_string(obj->target_temperature, target_accuracy); - if (!has_state) - root["state"] = root["target_temperature"]; + root["current_temperature"] = "NA"; } - }); + } + if (traits.has_feature_flags(climate::CLIMATE_SUPPORTS_TWO_POINT_TARGET_TEMPERATURE | + climate::CLIMATE_REQUIRES_TWO_POINT_TARGET_TEMPERATURE)) { + root["target_temperature_low"] = value_accuracy_to_string(obj->target_temperature_low, target_accuracy); + root["target_temperature_high"] = value_accuracy_to_string(obj->target_temperature_high, target_accuracy); + if (!has_state) { + root["state"] = value_accuracy_to_string((obj->target_temperature_high + obj->target_temperature_low) / 2.0f, + target_accuracy); + } + } else { + root["target_temperature"] = value_accuracy_to_string(obj->target_temperature, target_accuracy); + if (!has_state) + root["state"] = root["target_temperature"]; + } + + return builder.serialize(); // NOLINTEND(clang-analyzer-cplusplus.NewDeleteLeaks) } #endif #ifdef USE_LOCK void WebServer::on_lock_update(lock::Lock *obj) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", lock_state_json_generator); } void WebServer::handle_lock_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -1427,14 +1384,37 @@ void WebServer::handle_lock_request(AsyncWebServerRequest *request, const UrlMat auto detail = get_request_detail(request); std::string data = this->lock_json(obj, obj->state, detail); request->send(200, "application/json", data.c_str()); - } else if (match.method_equals("lock")) { - this->defer([obj]() { obj->lock(); }); - request->send(200); + return; + } + + // Handle action methods with single defer and response + enum LockAction { NONE, LOCK, UNLOCK, OPEN }; + LockAction action = NONE; + + if (match.method_equals("lock")) { + action = LOCK; } else if (match.method_equals("unlock")) { - this->defer([obj]() { obj->unlock(); }); - request->send(200); + action = UNLOCK; } else if (match.method_equals("open")) { - this->defer([obj]() { obj->open(); }); + action = OPEN; + } + + if (action != NONE) { + this->defer([obj, action]() { + switch (action) { + case LOCK: + obj->lock(); + break; + case UNLOCK: + obj->unlock(); + break; + case OPEN: + obj->open(); + break; + default: + break; + } + }); request->send(200); } else { request->send(404); @@ -1450,20 +1430,20 @@ std::string WebServer::lock_all_json_generator(WebServer *web_server, void *sour return web_server->lock_json((lock::Lock *) (source), ((lock::Lock *) (source))->state, DETAIL_ALL); } std::string WebServer::lock_json(lock::Lock *obj, lock::LockState value, JsonDetail start_config) { - return json::build_json([this, obj, value, start_config](JsonObject root) { - set_json_icon_state_value(root, obj, "lock-" + obj->get_object_id(), lock::lock_state_to_string(value), value, - start_config); - if (start_config == DETAIL_ALL) { - this->add_sorting_info_(root, obj); - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + set_json_icon_state_value(root, obj, "lock", lock::lock_state_to_string(value), value, start_config); + if (start_config == DETAIL_ALL) { + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif #ifdef USE_VALVE void WebServer::on_valve_update(valve::Valve *obj) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", valve_state_json_generator); } void WebServer::handle_valve_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -1479,15 +1459,28 @@ void WebServer::handle_valve_request(AsyncWebServerRequest *request, const UrlMa } auto call = obj->make_call(); - if (match.method_equals("open")) { - call.set_command_open(); - } else if (match.method_equals("close")) { - call.set_command_close(); - } else if (match.method_equals("stop")) { - call.set_command_stop(); - } else if (match.method_equals("toggle")) { - call.set_command_toggle(); - } else if (!match.method_equals("set")) { + + // Lookup table for valve methods + static const struct { + const char *name; + valve::ValveCall &(valve::ValveCall::*action)(); + } METHODS[] = { + {"open", &valve::ValveCall::set_command_open}, + {"close", &valve::ValveCall::set_command_close}, + {"stop", &valve::ValveCall::set_command_stop}, + {"toggle", &valve::ValveCall::set_command_toggle}, + }; + + bool found = false; + for (const auto &method : METHODS) { + if (match.method_equals(method.name)) { + (call.*method.action)(); + found = true; + break; + } + } + + if (!found && !match.method_equals("set")) { request->send(404); return; } @@ -1498,12 +1491,7 @@ void WebServer::handle_valve_request(AsyncWebServerRequest *request, const UrlMa return; } - if (request->hasParam("position")) { - auto position = parse_number(request->getParam("position")->value().c_str()); - if (position.has_value()) { - call.set_position(*position); - } - } + parse_float_param_(request, "position", call, &decltype(call)::set_position); this->defer([call]() mutable { call.perform(); }); request->send(200); @@ -1518,24 +1506,25 @@ std::string WebServer::valve_all_json_generator(WebServer *web_server, void *sou return web_server->valve_json((valve::Valve *) (source), DETAIL_ALL); } std::string WebServer::valve_json(valve::Valve *obj, JsonDetail start_config) { - return json::build_json([this, obj, start_config](JsonObject root) { - set_json_icon_state_value(root, obj, "valve-" + obj->get_object_id(), obj->is_fully_closed() ? "CLOSED" : "OPEN", - obj->position, start_config); - root["current_operation"] = valve::valve_operation_to_str(obj->current_operation); + json::JsonBuilder builder; + JsonObject root = builder.root(); - if (obj->get_traits().get_supports_position()) - root["position"] = obj->position; - if (start_config == DETAIL_ALL) { - this->add_sorting_info_(root, obj); - } - }); + set_json_icon_state_value(root, obj, "valve", obj->is_fully_closed() ? "CLOSED" : "OPEN", obj->position, + start_config); + root["current_operation"] = valve::valve_operation_to_str(obj->current_operation); + + if (obj->get_traits().get_supports_position()) + root["position"] = obj->position; + if (start_config == DETAIL_ALL) { + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif #ifdef USE_ALARM_CONTROL_PANEL void WebServer::on_alarm_control_panel_update(alarm_control_panel::AlarmControlPanel *obj) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", alarm_control_panel_state_json_generator); } void WebServer::handle_alarm_control_panel_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -1551,21 +1540,30 @@ void WebServer::handle_alarm_control_panel_request(AsyncWebServerRequest *reques } auto call = obj->make_call(); - if (request->hasParam("code")) { - call.set_code(request->getParam("code")->value().c_str()); // NOLINT + parse_string_param_(request, "code", call, &decltype(call)::set_code); + + // Lookup table for alarm control panel methods + static const struct { + const char *name; + alarm_control_panel::AlarmControlPanelCall &(alarm_control_panel::AlarmControlPanelCall::*action)(); + } METHODS[] = { + {"disarm", &alarm_control_panel::AlarmControlPanelCall::disarm}, + {"arm_away", &alarm_control_panel::AlarmControlPanelCall::arm_away}, + {"arm_home", &alarm_control_panel::AlarmControlPanelCall::arm_home}, + {"arm_night", &alarm_control_panel::AlarmControlPanelCall::arm_night}, + {"arm_vacation", &alarm_control_panel::AlarmControlPanelCall::arm_vacation}, + }; + + bool found = false; + for (const auto &method : METHODS) { + if (match.method_equals(method.name)) { + (call.*method.action)(); + found = true; + break; + } } - if (match.method_equals("disarm")) { - call.disarm(); - } else if (match.method_equals("arm_away")) { - call.arm_away(); - } else if (match.method_equals("arm_home")) { - call.arm_home(); - } else if (match.method_equals("arm_night")) { - call.arm_night(); - } else if (match.method_equals("arm_vacation")) { - call.arm_vacation(); - } else { + if (!found) { request->send(404); return; } @@ -1589,14 +1587,17 @@ std::string WebServer::alarm_control_panel_all_json_generator(WebServer *web_ser std::string WebServer::alarm_control_panel_json(alarm_control_panel::AlarmControlPanel *obj, alarm_control_panel::AlarmControlPanelState value, JsonDetail start_config) { - return json::build_json([this, obj, value, start_config](JsonObject root) { - char buf[16]; - set_json_icon_state_value(root, obj, "alarm-control-panel-" + obj->get_object_id(), - PSTR_LOCAL(alarm_control_panel_state_to_string(value)), value, start_config); - if (start_config == DETAIL_ALL) { - this->add_sorting_info_(root, obj); - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + char buf[16]; + set_json_icon_state_value(root, obj, "alarm-control-panel", PSTR_LOCAL(alarm_control_panel_state_to_string(value)), + value, start_config); + if (start_config == DETAIL_ALL) { + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif @@ -1620,7 +1621,9 @@ void WebServer::handle_event_request(AsyncWebServerRequest *request, const UrlMa request->send(404); } -static std::string get_event_type(event::Event *event) { return event->last_event_type ? *event->last_event_type : ""; } +static std::string get_event_type(event::Event *event) { + return (event && event->last_event_type) ? *event->last_event_type : ""; +} std::string WebServer::event_state_json_generator(WebServer *web_server, void *source) { auto *event = static_cast(source); @@ -1631,27 +1634,41 @@ std::string WebServer::event_all_json_generator(WebServer *web_server, void *sou return web_server->event_json(event, get_event_type(event), DETAIL_ALL); } std::string WebServer::event_json(event::Event *obj, const std::string &event_type, JsonDetail start_config) { - return json::build_json([this, obj, event_type, start_config](JsonObject root) { - set_json_id(root, obj, "event-" + obj->get_object_id(), start_config); - if (!event_type.empty()) { - root["event_type"] = event_type; + json::JsonBuilder builder; + JsonObject root = builder.root(); + + set_json_id(root, obj, "event", start_config); + if (!event_type.empty()) { + root["event_type"] = event_type; + } + if (start_config == DETAIL_ALL) { + JsonArray event_types = root["event_types"].to(); + for (auto const &event_type : obj->get_event_types()) { + event_types.add(event_type); } - if (start_config == DETAIL_ALL) { - JsonArray event_types = root["event_types"].to(); - for (auto const &event_type : obj->get_event_types()) { - event_types.add(event_type); - } - root["device_class"] = obj->get_device_class(); - this->add_sorting_info_(root, obj); - } - }); + root["device_class"] = obj->get_device_class_ref(); + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); } #endif #ifdef USE_UPDATE +static const char *update_state_to_string(update::UpdateState state) { + switch (state) { + case update::UPDATE_STATE_NO_UPDATE: + return "NO UPDATE"; + case update::UPDATE_STATE_AVAILABLE: + return "UPDATE AVAILABLE"; + case update::UPDATE_STATE_INSTALLING: + return "INSTALLING"; + default: + return "UNKNOWN"; + } +} + void WebServer::on_update(update::UpdateEntity *obj) { - if (this->events_.empty()) - return; this->events_.deferrable_send_state(obj, "state", update_state_json_generator); } void WebServer::handle_update_request(AsyncWebServerRequest *request, const UrlMatch &match) { @@ -1678,38 +1695,29 @@ void WebServer::handle_update_request(AsyncWebServerRequest *request, const UrlM request->send(404); } std::string WebServer::update_state_json_generator(WebServer *web_server, void *source) { + // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson return web_server->update_json((update::UpdateEntity *) (source), DETAIL_STATE); } std::string WebServer::update_all_json_generator(WebServer *web_server, void *source) { + // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson return web_server->update_json((update::UpdateEntity *) (source), DETAIL_STATE); } std::string WebServer::update_json(update::UpdateEntity *obj, JsonDetail start_config) { // NOLINTBEGIN(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson - return json::build_json([this, obj, start_config](JsonObject root) { - set_json_id(root, obj, "update-" + obj->get_object_id(), start_config); - root["value"] = obj->update_info.latest_version; - switch (obj->state) { - case update::UPDATE_STATE_NO_UPDATE: - root["state"] = "NO UPDATE"; - break; - case update::UPDATE_STATE_AVAILABLE: - root["state"] = "UPDATE AVAILABLE"; - break; - case update::UPDATE_STATE_INSTALLING: - root["state"] = "INSTALLING"; - break; - default: - root["state"] = "UNKNOWN"; - break; - } - if (start_config == DETAIL_ALL) { - root["current_version"] = obj->update_info.current_version; - root["title"] = obj->update_info.title; - root["summary"] = obj->update_info.summary; - root["release_url"] = obj->update_info.release_url; - this->add_sorting_info_(root, obj); - } - }); + json::JsonBuilder builder; + JsonObject root = builder.root(); + + set_json_icon_state_value(root, obj, "update", update_state_to_string(obj->state), obj->update_info.latest_version, + start_config); + if (start_config == DETAIL_ALL) { + root["current_version"] = obj->update_info.current_version; + root["title"] = obj->update_info.title; + root["summary"] = obj->update_info.summary; + root["release_url"] = obj->update_info.release_url; + this->add_sorting_info_(root, obj); + } + + return builder.serialize(); // NOLINTEND(clang-analyzer-cplusplus.NewDeleteLeaks) } #endif @@ -1718,24 +1726,24 @@ bool WebServer::canHandle(AsyncWebServerRequest *request) const { const auto &url = request->url(); const auto method = request->method(); - // Simple URL checks - if (url == "/") - return true; - -#ifdef USE_ARDUINO - if (url == "/events") - return true; + // Static URL checks + static const char *const STATIC_URLS[] = { + "/", +#if !defined(USE_ESP32) && defined(USE_ARDUINO) + "/events", #endif - #ifdef USE_WEBSERVER_CSS_INCLUDE - if (url == "/0.css") - return true; + "/0.css", #endif - #ifdef USE_WEBSERVER_JS_INCLUDE - if (url == "/0.js") - return true; + "/0.js", #endif + }; + + for (const auto &static_url : STATIC_URLS) { + if (url == static_url) + return true; + } #ifdef USE_WEBSERVER_PRIVATE_NETWORK_ACCESS if (method == HTTP_OPTIONS && request->hasHeader(HEADER_CORS_REQ_PNA)) @@ -1755,92 +1763,87 @@ bool WebServer::canHandle(AsyncWebServerRequest *request) const { if (!is_get_or_post) return false; - // GET-only components - if (is_get) { + // Use lookup tables for domain checks + static const char *const GET_ONLY_DOMAINS[] = { #ifdef USE_SENSOR - if (match.domain_equals("sensor")) - return true; + "sensor", #endif #ifdef USE_BINARY_SENSOR - if (match.domain_equals("binary_sensor")) - return true; + "binary_sensor", #endif #ifdef USE_TEXT_SENSOR - if (match.domain_equals("text_sensor")) - return true; + "text_sensor", #endif #ifdef USE_EVENT - if (match.domain_equals("event")) - return true; + "event", #endif - } + }; - // GET+POST components - if (is_get_or_post) { + static const char *const GET_POST_DOMAINS[] = { #ifdef USE_SWITCH - if (match.domain_equals("switch")) - return true; + "switch", #endif #ifdef USE_BUTTON - if (match.domain_equals("button")) - return true; + "button", #endif #ifdef USE_FAN - if (match.domain_equals("fan")) - return true; + "fan", #endif #ifdef USE_LIGHT - if (match.domain_equals("light")) - return true; + "light", #endif #ifdef USE_COVER - if (match.domain_equals("cover")) - return true; + "cover", #endif #ifdef USE_NUMBER - if (match.domain_equals("number")) - return true; + "number", #endif #ifdef USE_DATETIME_DATE - if (match.domain_equals("date")) - return true; + "date", #endif #ifdef USE_DATETIME_TIME - if (match.domain_equals("time")) - return true; + "time", #endif #ifdef USE_DATETIME_DATETIME - if (match.domain_equals("datetime")) - return true; + "datetime", #endif #ifdef USE_TEXT - if (match.domain_equals("text")) - return true; + "text", #endif #ifdef USE_SELECT - if (match.domain_equals("select")) - return true; + "select", #endif #ifdef USE_CLIMATE - if (match.domain_equals("climate")) - return true; + "climate", #endif #ifdef USE_LOCK - if (match.domain_equals("lock")) - return true; + "lock", #endif #ifdef USE_VALVE - if (match.domain_equals("valve")) - return true; + "valve", #endif #ifdef USE_ALARM_CONTROL_PANEL - if (match.domain_equals("alarm_control_panel")) - return true; + "alarm_control_panel", #endif #ifdef USE_UPDATE - if (match.domain_equals("update")) - return true; + "update", #endif + }; + + // Check GET-only domains + if (is_get) { + for (const auto &domain : GET_ONLY_DOMAINS) { + if (match.domain_equals(domain)) + return true; + } + } + + // Check GET+POST domains + if (is_get_or_post) { + for (const auto &domain : GET_POST_DOMAINS) { + if (match.domain_equals(domain)) + return true; + } } return false; @@ -1854,7 +1857,7 @@ void WebServer::handleRequest(AsyncWebServerRequest *request) { return; } -#ifdef USE_ARDUINO +#if !defined(USE_ESP32) && defined(USE_ARDUINO) if (url == "/events") { this->events_.add_new_client(this, request); return; diff --git a/esphome/components/web_server/web_server.h b/esphome/components/web_server/web_server.h index ef1b03a73b..2e5d58d375 100644 --- a/esphome/components/web_server/web_server.h +++ b/esphome/components/web_server/web_server.h @@ -81,7 +81,7 @@ enum JsonDetail { DETAIL_ALL, DETAIL_STATE }; implemented in a more straightforward way for ESP-IDF. Arduino platform will eventually go away and this workaround can be forgotten. */ -#ifdef USE_ARDUINO +#if !defined(USE_ESP32) && defined(USE_ARDUINO) using message_generator_t = std::string(WebServer *, void *); class DeferredUpdateEventSourceList; @@ -164,7 +164,7 @@ class DeferredUpdateEventSourceList : public std::list that's sent to each client. Defaults to - * https://esphome.io/_static/webserver-v1.min.css + * https://oi.esphome.io/v1/webserver-v1.min.css * * @param css_url The url to the web server stylesheet. */ void set_css_url(const char *css_url); /** Set the URL to the script that's embedded in the index page. Defaults to - * https://esphome.io/_static/webserver-v1.min.js + * https://oi.esphome.io/v1/webserver-v1.min.js * * @param js_url The url to the web server script. */ @@ -498,12 +498,71 @@ class WebServer : public Controller, public Component, public AsyncWebHandler { protected: void add_sorting_info_(JsonObject &root, EntityBase *entity); - web_server_base::WebServerBase *base_; -#ifdef USE_ARDUINO - DeferredUpdateEventSourceList events_; + +#ifdef USE_LIGHT + // Helper to parse and apply a float parameter with optional scaling + template + void parse_light_param_(AsyncWebServerRequest *request, const char *param_name, T &call, Ret (T::*setter)(float), + float scale = 1.0f) { + if (request->hasParam(param_name)) { + auto value = parse_number(request->getParam(param_name)->value().c_str()); + if (value.has_value()) { + (call.*setter)(*value / scale); + } + } + } + + // Helper to parse and apply a uint32_t parameter with optional scaling + template + void parse_light_param_uint_(AsyncWebServerRequest *request, const char *param_name, T &call, + Ret (T::*setter)(uint32_t), uint32_t scale = 1) { + if (request->hasParam(param_name)) { + auto value = parse_number(request->getParam(param_name)->value().c_str()); + if (value.has_value()) { + (call.*setter)(*value * scale); + } + } + } #endif -#ifdef USE_ESP_IDF + + // Generic helper to parse and apply a float parameter + template + void parse_float_param_(AsyncWebServerRequest *request, const char *param_name, T &call, Ret (T::*setter)(float)) { + if (request->hasParam(param_name)) { + auto value = parse_number(request->getParam(param_name)->value().c_str()); + if (value.has_value()) { + (call.*setter)(*value); + } + } + } + + // Generic helper to parse and apply an int parameter + template + void parse_int_param_(AsyncWebServerRequest *request, const char *param_name, T &call, Ret (T::*setter)(int)) { + if (request->hasParam(param_name)) { + auto value = parse_number(request->getParam(param_name)->value().c_str()); + if (value.has_value()) { + (call.*setter)(*value); + } + } + } + + // Generic helper to parse and apply a string parameter + template + void parse_string_param_(AsyncWebServerRequest *request, const char *param_name, T &call, + Ret (T::*setter)(const std::string &)) { + if (request->hasParam(param_name)) { + // .c_str() is required for Arduino framework where value() returns Arduino String instead of std::string + std::string value = request->getParam(param_name)->value().c_str(); // NOLINT(readability-redundant-string-cstr) + (call.*setter)(value); + } + } + + web_server_base::WebServerBase *base_; +#ifdef USE_ESP32 AsyncEventSource events_{"/events", this}; +#elif USE_ARDUINO + DeferredUpdateEventSourceList events_; #endif #if USE_WEBSERVER_VERSION == 1 diff --git a/esphome/components/web_server/web_server_v1.cpp b/esphome/components/web_server/web_server_v1.cpp index 5db0f1cae9..870a338620 100644 --- a/esphome/components/web_server/web_server_v1.cpp +++ b/esphome/components/web_server/web_server_v1.cpp @@ -34,23 +34,23 @@ void WebServer::set_js_url(const char *js_url) { this->js_url_ = js_url; } void WebServer::handle_index_request(AsyncWebServerRequest *request) { AsyncResponseStream *stream = request->beginResponseStream("text/html"); const std::string &title = App.get_name(); - stream->print(F("")); + stream->print(ESPHOME_F("<!DOCTYPE html><html lang=\"en\"><head><meta charset=UTF-8><meta " + "name=viewport content=\"width=device-width, initial-scale=1,user-scalable=no\"><title>")); stream->print(title.c_str()); - stream->print(F("")); + stream->print(ESPHOME_F("")); #ifdef USE_WEBSERVER_CSS_INCLUDE - stream->print(F("")); + stream->print(ESPHOME_F("")); #endif if (strlen(this->css_url_) > 0) { - stream->print(F(R"(print(ESPHOME_F(R"(print(this->css_url_); - stream->print(F("\">")); + stream->print(ESPHOME_F("\">")); } - stream->print(F("")); - stream->print(F("

")); + stream->print(ESPHOME_F("")); + stream->print(ESPHOME_F("

")); stream->print(title.c_str()); - stream->print(F("

")); - stream->print(F("

States

")); + stream->print(ESPHOME_F("")); + stream->print(ESPHOME_F("

States

NameStateActions
")); #ifdef USE_SENSOR for (auto *obj : App.get_sensors()) { @@ -190,24 +190,28 @@ void WebServer::handle_index_request(AsyncWebServerRequest *request) { } #endif - stream->print(F("
NameStateActions

See ESPHome Web API for " - "REST API documentation.

")); -#ifdef USE_WEBSERVER_OTA - stream->print(F("

OTA Update

")); + stream->print( + ESPHOME_F("

See ESPHome Web API for " + "REST API documentation.

")); +#if defined(USE_WEBSERVER_OTA) && !defined(USE_WEBSERVER_OTA_DISABLED) + // Show OTA form only if web_server OTA is not explicitly disabled + // Note: USE_WEBSERVER_OTA_DISABLED only affects web_server, not captive_portal + stream->print( + ESPHOME_F("

OTA Update

")); #endif - stream->print(F("

Debug Log

"));
+  stream->print(ESPHOME_F("
Debug Log
"));
 #ifdef USE_WEBSERVER_JS_INCLUDE
   if (this->js_include_ != nullptr) {
-    stream->print(F(""));
+    stream->print(ESPHOME_F(""));
   }
 #endif
   if (strlen(this->js_url_) > 0) {
-    stream->print(F(""));
+    stream->print(ESPHOME_F("\">"));
   }
-  stream->print(F("
"));
+  stream->print(ESPHOME_F("
"));
   request->send(stream);
 }
 
diff --git a/esphome/components/web_server_base/__init__.py b/esphome/components/web_server_base/__init__.py
index 9f3371c233..4cf76eba0e 100644
--- a/esphome/components/web_server_base/__init__.py
+++ b/esphome/components/web_server_base/__init__.py
@@ -2,16 +2,17 @@ import esphome.codegen as cg
 import esphome.config_validation as cv
 from esphome.const import CONF_ID
 from esphome.core import CORE, coroutine_with_priority
+from esphome.coroutine import CoroPriority
 
-CODEOWNERS = ["@OttoWinter"]
+CODEOWNERS = ["@esphome/core"]
 DEPENDENCIES = ["network"]
 
 
 def AUTO_LOAD():
+    if CORE.is_esp32:
+        return ["web_server_idf"]
     if CORE.using_arduino:
         return ["async_tcp"]
-    if CORE.using_esp_idf:
-        return ["web_server_idf"]
     return []
 
 
@@ -26,12 +27,15 @@ CONFIG_SCHEMA = cv.Schema(
 )
 
 
-@coroutine_with_priority(65.0)
+@coroutine_with_priority(CoroPriority.WEB_SERVER_BASE)
 async def to_code(config):
     var = cg.new_Pvariable(config[CONF_ID])
     await cg.register_component(var, config)
     cg.add(cg.RawExpression(f"{web_server_base_ns}::global_web_server_base = {var}"))
 
+    if CORE.is_esp32:
+        return
+
     if CORE.using_arduino:
         if CORE.is_esp32:
             cg.add_library("WiFi", None)
@@ -39,5 +43,7 @@ async def to_code(config):
             cg.add_library("Update", None)
         if CORE.is_esp8266:
             cg.add_library("ESP8266WiFi", None)
+        if CORE.is_libretiny:
+            CORE.add_platformio_option("lib_ignore", ["ESPAsyncTCP", "RPAsyncTCP"])
         # https://github.com/ESP32Async/ESPAsyncWebServer/blob/main/library.json
         cg.add_library("ESP32Async/ESPAsyncWebServer", "3.7.10")
diff --git a/esphome/components/web_server_base/web_server_base.cpp b/esphome/components/web_server_base/web_server_base.cpp
index e1c2bc0b25..6e7097338c 100644
--- a/esphome/components/web_server_base/web_server_base.cpp
+++ b/esphome/components/web_server_base/web_server_base.cpp
@@ -14,9 +14,11 @@ WebServerBase *global_web_server_base = nullptr;  // NOLINT(cppcoreguidelines-av
 void WebServerBase::add_handler(AsyncWebHandler *handler) {
   // remove all handlers
 
+#ifdef USE_WEBSERVER_AUTH
   if (!credentials_.username.empty()) {
     handler = new internal::AuthMiddlewareHandler(handler, &credentials_);
   }
+#endif
   this->handlers_.push_back(handler);
   if (this->server_ != nullptr) {
     this->server_->addHandler(handler);
diff --git a/esphome/components/web_server_base/web_server_base.h b/esphome/components/web_server_base/web_server_base.h
index a475238a37..039a452d64 100644
--- a/esphome/components/web_server_base/web_server_base.h
+++ b/esphome/components/web_server_base/web_server_base.h
@@ -7,11 +7,31 @@
 
 #include "esphome/core/component.h"
 
-#ifdef USE_ARDUINO
-#include 
-#elif USE_ESP_IDF
+// Platform-agnostic macros for web server components
+// On ESP32 (both Arduino and IDF): Use plain strings (no PROGMEM)
+// On ESP8266: Use Arduino's F() macro for PROGMEM strings
+#ifdef USE_ESP32
+#define ESPHOME_F(string_literal) (string_literal)
+#define ESPHOME_PGM_P const char *
+#define ESPHOME_strncpy_P strncpy
+#else
+// ESP8266 uses Arduino macros
+#define ESPHOME_F(string_literal) F(string_literal)
+#define ESPHOME_PGM_P PGM_P
+#define ESPHOME_strncpy_P strncpy_P
+#endif
+
+#if USE_ESP32
 #include "esphome/core/hal.h"
 #include "esphome/components/web_server_idf/web_server_idf.h"
+#else
+#include 
+#endif
+
+#if USE_ESP32
+using PlatformString = std::string;
+#elif USE_ARDUINO
+using PlatformString = String;
 #endif
 
 namespace esphome {
@@ -28,8 +48,8 @@ class MiddlewareHandler : public AsyncWebHandler {
 
   bool canHandle(AsyncWebServerRequest *request) const override { return next_->canHandle(request); }
   void handleRequest(AsyncWebServerRequest *request) override { next_->handleRequest(request); }
-  void handleUpload(AsyncWebServerRequest *request, const String &filename, size_t index, uint8_t *data, size_t len,
-                    bool final) override {
+  void handleUpload(AsyncWebServerRequest *request, const PlatformString &filename, size_t index, uint8_t *data,
+                    size_t len, bool final) override {
     next_->handleUpload(request, filename, index, data, len, final);
   }
   void handleBody(AsyncWebServerRequest *request, uint8_t *data, size_t len, size_t index, size_t total) override {
@@ -41,6 +61,7 @@ class MiddlewareHandler : public AsyncWebHandler {
   AsyncWebHandler *next_;
 };
 
+#ifdef USE_WEBSERVER_AUTH
 struct Credentials {
   std::string username;
   std::string password;
@@ -64,8 +85,8 @@ class AuthMiddlewareHandler : public MiddlewareHandler {
       return;
     MiddlewareHandler::handleRequest(request);
   }
-  void handleUpload(AsyncWebServerRequest *request, const String &filename, size_t index, uint8_t *data, size_t len,
-                    bool final) override {
+  void handleUpload(AsyncWebServerRequest *request, const PlatformString &filename, size_t index, uint8_t *data,
+                    size_t len, bool final) override {
     if (!check_auth(request))
       return;
     MiddlewareHandler::handleUpload(request, filename, index, data, len, final);
@@ -79,6 +100,7 @@ class AuthMiddlewareHandler : public MiddlewareHandler {
  protected:
   Credentials *credentials_;
 };
+#endif
 
 }  // namespace internal
 
@@ -108,8 +130,10 @@ class WebServerBase : public Component {
   std::shared_ptr get_server() const { return server_; }
   float get_setup_priority() const override;
 
+#ifdef USE_WEBSERVER_AUTH
   void set_auth_username(std::string auth_username) { credentials_.username = std::move(auth_username); }
   void set_auth_password(std::string auth_password) { credentials_.password = std::move(auth_password); }
+#endif
 
   void add_handler(AsyncWebHandler *handler);
 
@@ -121,7 +145,9 @@ class WebServerBase : public Component {
   uint16_t port_{80};
   std::shared_ptr server_{nullptr};
   std::vector handlers_;
+#ifdef USE_WEBSERVER_AUTH
   internal::Credentials credentials_;
+#endif
 };
 
 }  // namespace web_server_base
diff --git a/esphome/components/web_server_idf/__init__.py b/esphome/components/web_server_idf/__init__.py
index 506e1c5c13..74a9d657a6 100644
--- a/esphome/components/web_server_idf/__init__.py
+++ b/esphome/components/web_server_idf/__init__.py
@@ -5,7 +5,7 @@ CODEOWNERS = ["@dentra"]
 
 CONFIG_SCHEMA = cv.All(
     cv.Schema({}),
-    cv.only_with_esp_idf,
+    cv.only_on_esp32,
 )
 
 
diff --git a/esphome/components/web_server_idf/multipart.cpp b/esphome/components/web_server_idf/multipart.cpp
index 8655226ab9..2092a41a8e 100644
--- a/esphome/components/web_server_idf/multipart.cpp
+++ b/esphome/components/web_server_idf/multipart.cpp
@@ -1,5 +1,5 @@
 #include "esphome/core/defines.h"
-#if defined(USE_ESP_IDF) && defined(USE_WEBSERVER_OTA)
+#if defined(USE_ESP32) && defined(USE_WEBSERVER_OTA)
 #include "multipart.h"
 #include "utils.h"
 #include "esphome/core/log.h"
@@ -251,4 +251,4 @@ std::string str_trim(const std::string &str) {
 
 }  // namespace web_server_idf
 }  // namespace esphome
-#endif  // defined(USE_ESP_IDF) && defined(USE_WEBSERVER_OTA)
+#endif  // defined(USE_ESP32) && defined(USE_WEBSERVER_OTA)
diff --git a/esphome/components/web_server_idf/multipart.h b/esphome/components/web_server_idf/multipart.h
index 967c72ffa5..8fbe90c4a0 100644
--- a/esphome/components/web_server_idf/multipart.h
+++ b/esphome/components/web_server_idf/multipart.h
@@ -1,6 +1,6 @@
 #pragma once
 #include "esphome/core/defines.h"
-#if defined(USE_ESP_IDF) && defined(USE_WEBSERVER_OTA)
+#if defined(USE_ESP32) && defined(USE_WEBSERVER_OTA)
 
 #include 
 #include 
@@ -83,4 +83,4 @@ std::string str_trim(const std::string &str);
 
 }  // namespace web_server_idf
 }  // namespace esphome
-#endif  // defined(USE_ESP_IDF) && defined(USE_WEBSERVER_OTA)
+#endif  // defined(USE_ESP32) && defined(USE_WEBSERVER_OTA)
diff --git a/esphome/components/web_server_idf/utils.cpp b/esphome/components/web_server_idf/utils.cpp
index ac5df90bb8..d5d34b520b 100644
--- a/esphome/components/web_server_idf/utils.cpp
+++ b/esphome/components/web_server_idf/utils.cpp
@@ -1,4 +1,4 @@
-#ifdef USE_ESP_IDF
+#ifdef USE_ESP32
 #include 
 #include 
 #include 
@@ -122,4 +122,4 @@ const char *stristr(const char *haystack, const char *needle) {
 
 }  // namespace web_server_idf
 }  // namespace esphome
-#endif  // USE_ESP_IDF
+#endif  // USE_ESP32
diff --git a/esphome/components/web_server_idf/utils.h b/esphome/components/web_server_idf/utils.h
index 988b962d72..f70a5f0760 100644
--- a/esphome/components/web_server_idf/utils.h
+++ b/esphome/components/web_server_idf/utils.h
@@ -1,5 +1,5 @@
 #pragma once
-#ifdef USE_ESP_IDF
+#ifdef USE_ESP32
 
 #include 
 #include 
@@ -24,4 +24,4 @@ const char *stristr(const char *haystack, const char *needle);
 
 }  // namespace web_server_idf
 }  // namespace esphome
-#endif  // USE_ESP_IDF
+#endif  // USE_ESP32
diff --git a/esphome/components/web_server_idf/web_server_idf.cpp b/esphome/components/web_server_idf/web_server_idf.cpp
index 734259093e..c3ba7ddc2b 100644
--- a/esphome/components/web_server_idf/web_server_idf.cpp
+++ b/esphome/components/web_server_idf/web_server_idf.cpp
@@ -1,4 +1,4 @@
-#ifdef USE_ESP_IDF
+#ifdef USE_ESP32
 
 #include 
 #include 
@@ -25,6 +25,10 @@
 #include "esphome/components/web_server/list_entities.h"
 #endif  // USE_WEBSERVER
 
+// Include socket headers after Arduino headers to avoid IPADDR_NONE/INADDR_NONE macro conflicts
+#include 
+#include 
+
 namespace esphome {
 namespace web_server_idf {
 
@@ -46,6 +50,42 @@ DefaultHeaders default_headers_instance;
 
 DefaultHeaders &DefaultHeaders::Instance() { return default_headers_instance; }
 
+namespace {
+// Non-blocking send function to prevent watchdog timeouts when TCP buffers are full
+/**
+ * Sends data on a socket in non-blocking mode.
+ *
+ * @param hd      HTTP server handle (unused).
+ * @param sockfd  Socket file descriptor.
+ * @param buf     Buffer to send.
+ * @param buf_len Length of buffer.
+ * @param flags   Flags for send().
+ * @return
+ *   - Number of bytes sent on success.
+ *   - HTTPD_SOCK_ERR_INVALID if buf is nullptr.
+ *   - HTTPD_SOCK_ERR_TIMEOUT if the send buffer is full (EAGAIN/EWOULDBLOCK).
+ *   - HTTPD_SOCK_ERR_FAIL for other errors.
+ */
+int nonblocking_send(httpd_handle_t hd, int sockfd, const char *buf, size_t buf_len, int flags) {
+  if (buf == nullptr) {
+    return HTTPD_SOCK_ERR_INVALID;
+  }
+
+  // Use MSG_DONTWAIT to prevent blocking when TCP send buffer is full
+  int ret = send(sockfd, buf, buf_len, flags | MSG_DONTWAIT);
+  if (ret < 0) {
+    if (errno == EAGAIN || errno == EWOULDBLOCK) {
+      // Buffer full - retry later
+      return HTTPD_SOCK_ERR_TIMEOUT;
+    }
+    // Real error
+    ESP_LOGD(TAG, "send error: errno %d", errno);
+    return HTTPD_SOCK_ERR_FAIL;
+  }
+  return ret;
+}
+}  // namespace
+
 void AsyncWebServer::end() {
   if (this->server_) {
     httpd_stop(this->server_);
@@ -116,7 +156,7 @@ esp_err_t AsyncWebServer::request_post_handler(httpd_req_t *r) {
   }
 
   // Handle regular form data
-  if (r->content_len > HTTPD_MAX_REQ_HDR_LEN) {
+  if (r->content_len > CONFIG_HTTPD_MAX_REQ_HDR_LEN) {
     ESP_LOGW(TAG, "Request size is to big: %zu", r->content_len);
     httpd_resp_send_err(r, HTTPD_400_BAD_REQUEST, nullptr);
     return ESP_FAIL;
@@ -164,8 +204,8 @@ esp_err_t AsyncWebServer::request_handler_(AsyncWebServerRequest *request) const
 
 AsyncWebServerRequest::~AsyncWebServerRequest() {
   delete this->rsp_;
-  for (const auto &pair : this->params_) {
-    delete pair.second;  // NOLINT(cppcoreguidelines-owning-memory)
+  for (auto *param : this->params_) {
+    delete param;  // NOLINT(cppcoreguidelines-owning-memory)
   }
 }
 
@@ -205,10 +245,22 @@ void AsyncWebServerRequest::redirect(const std::string &url) {
 }
 
 void AsyncWebServerRequest::init_response_(AsyncWebServerResponse *rsp, int code, const char *content_type) {
-  httpd_resp_set_status(*this, code == 200   ? HTTPD_200
-                               : code == 404 ? HTTPD_404
-                               : code == 409 ? HTTPD_409
-                                             : to_string(code).c_str());
+  // Set status code - use constants for common codes to avoid string allocation
+  const char *status = nullptr;
+  switch (code) {
+    case 200:
+      status = HTTPD_200;
+      break;
+    case 404:
+      status = HTTPD_404;
+      break;
+    case 409:
+      status = HTTPD_409;
+      break;
+    default:
+      break;
+  }
+  httpd_resp_set_status(*this, status == nullptr ? to_string(code).c_str() : status);
 
   if (content_type && *content_type) {
     httpd_resp_set_type(*this, content_type);
@@ -223,6 +275,7 @@ void AsyncWebServerRequest::init_response_(AsyncWebServerResponse *rsp, int code
   this->rsp_ = rsp;
 }
 
+#ifdef USE_WEBSERVER_AUTH
 bool AsyncWebServerRequest::authenticate(const char *username, const char *password) const {
   if (username == nullptr || password == nullptr || *username == 0) {
     return true;
@@ -252,7 +305,7 @@ bool AsyncWebServerRequest::authenticate(const char *username, const char *passw
   esp_crypto_base64_encode(reinterpret_cast(digest.get()), n, &out,
                            reinterpret_cast(user_info.c_str()), user_info.size());
 
-  return strncmp(digest.get(), auth_str + auth_prefix_len, auth.value().size() - auth_prefix_len) == 0;
+  return strcmp(digest.get(), auth_str + auth_prefix_len) == 0;
 }
 
 void AsyncWebServerRequest::requestAuthentication(const char *realm) const {
@@ -261,13 +314,17 @@ void AsyncWebServerRequest::requestAuthentication(const char *realm) const {
   httpd_resp_set_hdr(*this, "WWW-Authenticate", auth_val.c_str());
   httpd_resp_send_err(*this, HTTPD_401_UNAUTHORIZED, nullptr);
 }
+#endif
 
 AsyncWebParameter *AsyncWebServerRequest::getParam(const std::string &name) {
-  auto find = this->params_.find(name);
-  if (find != this->params_.end()) {
-    return find->second;
+  // Check cache first - only successful lookups are cached
+  for (auto *param : this->params_) {
+    if (param->name() == name) {
+      return param;
+    }
   }
 
+  // Look up value from query strings
   optional val = query_key_value(this->post_query_, name);
   if (!val.has_value()) {
     auto url_query = request_get_url_query(*this);
@@ -276,11 +333,14 @@ AsyncWebParameter *AsyncWebServerRequest::getParam(const std::string &name) {
     }
   }
 
-  AsyncWebParameter *param = nullptr;
-  if (val.has_value()) {
-    param = new AsyncWebParameter(val.value());  // NOLINT(cppcoreguidelines-owning-memory)
+  // Don't cache misses to avoid wasting memory when handlers check for
+  // optional parameters that don't exist in the request
+  if (!val.has_value()) {
+    return nullptr;
   }
-  this->params_.insert({name, param});
+
+  auto *param = new AsyncWebParameter(name, val.value());  // NOLINT(cppcoreguidelines-owning-memory)
+  this->params_.push_back(param);
   return param;
 }
 
@@ -315,29 +375,30 @@ AsyncEventSource::~AsyncEventSource() {
 }
 
 void AsyncEventSource::handleRequest(AsyncWebServerRequest *request) {
-  auto *rsp =  // NOLINT(cppcoreguidelines-owning-memory)
-      new AsyncEventSourceResponse(request, this, this->web_server_);
+  // NOLINTNEXTLINE(cppcoreguidelines-owning-memory,clang-analyzer-cplusplus.NewDeleteLeaks)
+  auto *rsp = new AsyncEventSourceResponse(request, this, this->web_server_);
   if (this->on_connect_) {
     this->on_connect_(rsp);
   }
-  this->sessions_.insert(rsp);
+  this->sessions_.push_back(rsp);
 }
 
 void AsyncEventSource::loop() {
   // Clean up dead sessions safely
   // This follows the ESP-IDF pattern where free_ctx marks resources as dead
   // and the main loop handles the actual cleanup to avoid race conditions
-  auto it = this->sessions_.begin();
-  while (it != this->sessions_.end()) {
-    auto *ses = *it;
+  for (size_t i = 0; i < this->sessions_.size();) {
+    auto *ses = this->sessions_[i];
     // If the session has a dead socket (marked by destroy callback)
     if (ses->fd_.load() == 0) {
       ESP_LOGD(TAG, "Removing dead event source session");
-      it = this->sessions_.erase(it);
       delete ses;  // NOLINT(cppcoreguidelines-owning-memory)
+      // Remove by swapping with last element (O(1) removal, order doesn't matter for sessions)
+      this->sessions_[i] = this->sessions_.back();
+      this->sessions_.pop_back();
     } else {
       ses->loop();
-      ++it;
+      ++i;
     }
   }
 }
@@ -352,6 +413,9 @@ void AsyncEventSource::try_send_nodefer(const char *message, const char *event,
 
 void AsyncEventSource::deferrable_send_state(void *source, const char *event_type,
                                              message_generator_t *message_generator) {
+  // Skip if no connected clients to avoid unnecessary processing
+  if (this->empty())
+    return;
   for (auto *ses : this->sessions_) {
     if (ses->fd_.load() != 0) {  // Skip dead sessions
       ses->deferrable_send_state(source, event_type, message_generator);
@@ -382,6 +446,9 @@ AsyncEventSourceResponse::AsyncEventSourceResponse(const AsyncWebServerRequest *
   this->hd_ = req->handle;
   this->fd_.store(httpd_req_to_sockfd(req));
 
+  // Use non-blocking send to prevent watchdog timeouts when TCP buffers are full
+  httpd_sess_set_send_override(this->hd_, this->fd_.load(), nonblocking_send);
+
   // Configure reconnect timeout and send config
   // this should always go through since the tcp send buffer is empty on connect
   std::string message = ws->get_config_json();
@@ -390,10 +457,11 @@ AsyncEventSourceResponse::AsyncEventSourceResponse(const AsyncWebServerRequest *
 #ifdef USE_WEBSERVER_SORTING
   for (auto &group : ws->sorting_groups_) {
     // NOLINTBEGIN(clang-analyzer-cplusplus.NewDeleteLeaks) false positive with ArduinoJson
-    message = json::build_json([group](JsonObject root) {
-      root["name"] = group.second.name;
-      root["sorting_weight"] = group.second.weight;
-    });
+    json::JsonBuilder builder;
+    JsonObject root = builder.root();
+    root["name"] = group.second.name;
+    root["sorting_weight"] = group.second.weight;
+    message = builder.serialize();
     // NOLINTEND(clang-analyzer-cplusplus.NewDeleteLeaks)
 
     // a (very) large number of these should be able to be queued initially without defer
@@ -423,14 +491,14 @@ void AsyncEventSourceResponse::destroy(void *ptr) {
 void AsyncEventSourceResponse::deq_push_back_with_dedup_(void *source, message_generator_t *message_generator) {
   DeferredEvent item(source, message_generator);
 
-  auto iter = std::find_if(this->deferred_queue_.begin(), this->deferred_queue_.end(),
-                           [&item](const DeferredEvent &test) -> bool { return test == item; });
-
-  if (iter != this->deferred_queue_.end()) {
-    (*iter) = item;
-  } else {
-    this->deferred_queue_.push_back(item);
+  // Use range-based for loop instead of std::find_if to reduce template instantiation overhead and binary size
+  for (auto &event : this->deferred_queue_) {
+    if (event == item) {
+      event = item;
+      return;
+    }
   }
+  this->deferred_queue_.push_back(item);
 }
 
 void AsyncEventSourceResponse::process_deferred_queue_() {
@@ -456,15 +524,45 @@ void AsyncEventSourceResponse::process_buffer_() {
     return;
   }
 
-  int bytes_sent = httpd_socket_send(this->hd_, this->fd_.load(), event_buffer_.c_str() + event_bytes_sent_,
-                                     event_buffer_.size() - event_bytes_sent_, 0);
-  if (bytes_sent == HTTPD_SOCK_ERR_TIMEOUT || bytes_sent == HTTPD_SOCK_ERR_FAIL) {
-    // Socket error - just return, the connection will be closed by httpd
-    // and our destroy callback will be called
+  size_t remaining = event_buffer_.size() - event_bytes_sent_;
+  int bytes_sent =
+      httpd_socket_send(this->hd_, this->fd_.load(), event_buffer_.c_str() + event_bytes_sent_, remaining, 0);
+  if (bytes_sent == HTTPD_SOCK_ERR_TIMEOUT) {
+    // EAGAIN/EWOULDBLOCK - socket buffer full, try again later
+    // NOTE: Similar logic exists in web_server/web_server.cpp in DeferredUpdateEventSource::process_deferred_queue_()
+    // The implementations differ due to platform-specific APIs (HTTPD_SOCK_ERR_TIMEOUT vs DISCARDED, fd_.store(0) vs
+    // close()), but the failure counting and timeout logic should be kept in sync. If you change this logic, also
+    // update the Arduino implementation.
+    this->consecutive_send_failures_++;
+    if (this->consecutive_send_failures_ >= MAX_CONSECUTIVE_SEND_FAILURES) {
+      // Too many failures, connection is likely dead
+      ESP_LOGW(TAG, "Closing stuck EventSource connection after %" PRIu16 " failed sends",
+               this->consecutive_send_failures_);
+      this->fd_.store(0);  // Mark for cleanup
+      this->deferred_queue_.clear();
+    }
     return;
   }
+  if (bytes_sent == HTTPD_SOCK_ERR_FAIL) {
+    // Real socket error - connection will be closed by httpd and destroy callback will be called
+    return;
+  }
+  if (bytes_sent <= 0) {
+    // Unexpected error or zero bytes sent
+    ESP_LOGW(TAG, "Unexpected send result: %d", bytes_sent);
+    return;
+  }
+
+  // Successful send - reset failure counter
+  this->consecutive_send_failures_ = 0;
   event_bytes_sent_ += bytes_sent;
 
+  // Log partial sends for debugging
+  if (event_bytes_sent_ < event_buffer_.size()) {
+    ESP_LOGV(TAG, "Partial send: %d/%zu bytes (total: %zu/%zu)", bytes_sent, remaining, event_bytes_sent_,
+             event_buffer_.size());
+  }
+
   if (event_bytes_sent_ == event_buffer_.size()) {
     event_buffer_.resize(0);
     event_bytes_sent_ = 0;
@@ -667,4 +765,4 @@ esp_err_t AsyncWebServer::handle_multipart_upload_(httpd_req_t *r, const char *c
 }  // namespace web_server_idf
 }  // namespace esphome
 
-#endif  // !defined(USE_ESP_IDF)
+#endif  // !defined(USE_ESP32)
diff --git a/esphome/components/web_server_idf/web_server_idf.h b/esphome/components/web_server_idf/web_server_idf.h
index e8e40ef9b0..5ec6fec009 100644
--- a/esphome/components/web_server_idf/web_server_idf.h
+++ b/esphome/components/web_server_idf/web_server_idf.h
@@ -1,5 +1,5 @@
 #pragma once
-#ifdef USE_ESP_IDF
+#ifdef USE_ESP32
 
 #include "esphome/core/defines.h"
 #include 
@@ -8,7 +8,6 @@
 #include 
 #include 
 #include 
-#include 
 #include 
 #include 
 #include 
@@ -22,18 +21,14 @@ class ListEntitiesIterator;
 #endif
 namespace web_server_idf {
 
-#define F(string_literal) (string_literal)
-#define PGM_P const char *
-#define strncpy_P strncpy
-
-using String = std::string;
-
 class AsyncWebParameter {
  public:
-  AsyncWebParameter(std::string value) : value_(std::move(value)) {}
+  AsyncWebParameter(std::string name, std::string value) : name_(std::move(name)), value_(std::move(value)) {}
+  const std::string &name() const { return this->name_; }
   const std::string &value() const { return this->value_; }
 
  protected:
+  std::string name_;
   std::string value_;
 };
 
@@ -115,9 +110,11 @@ class AsyncWebServerRequest {
   // NOLINTNEXTLINE(readability-identifier-naming)
   size_t contentLength() const { return this->req_->content_len; }
 
+#ifdef USE_WEBSERVER_AUTH
   bool authenticate(const char *username, const char *password) const;
   // NOLINTNEXTLINE(readability-identifier-naming)
   void requestAuthentication(const char *realm = nullptr) const;
+#endif
 
   void redirect(const std::string &url);
 
@@ -172,7 +169,11 @@ class AsyncWebServerRequest {
  protected:
   httpd_req_t *req_;
   AsyncWebServerResponse *rsp_{};
-  std::map params_;
+  // Use vector instead of map/unordered_map: most requests have 0-3 params, so linear search
+  // is faster than tree/hash overhead. AsyncWebParameter stores both name and value to avoid
+  // duplicate storage. Only successful lookups are cached to prevent cache pollution when
+  // handlers check for optional parameters that don't exist.
+  std::vector params_;
   std::string post_query_;
   AsyncWebServerRequest(httpd_req_t *req) : req_(req) {}
   AsyncWebServerRequest(httpd_req_t *req, std::string post_query) : req_(req), post_query_(std::move(post_query)) {}
@@ -281,6 +282,8 @@ class AsyncEventSourceResponse {
   std::unique_ptr entities_iterator_;
   std::string event_buffer_{""};
   size_t event_bytes_sent_;
+  uint16_t consecutive_send_failures_{0};
+  static constexpr uint16_t MAX_CONSECUTIVE_SEND_FAILURES = 2500;  // ~20 seconds at 125Hz loop rate
 };
 
 using AsyncEventSourceClient = AsyncEventSourceResponse;
@@ -311,7 +314,10 @@ class AsyncEventSource : public AsyncWebHandler {
 
  protected:
   std::string url_;
-  std::set sessions_;
+  // Use vector instead of set: SSE sessions are typically 1-5 connections (browsers, dashboards).
+  // Linear search is faster than red-black tree overhead for this small dataset.
+  // Only operations needed: add session, remove session, iterate sessions - no need for sorted order.
+  std::vector sessions_;
   connect_handler_t on_connect_{};
   esphome::web_server::WebServer *web_server_;
 };
@@ -339,4 +345,4 @@ class DefaultHeaders {
 
 using namespace esphome::web_server_idf;  // NOLINT(google-global-names-in-headers)
 
-#endif  // !defined(USE_ESP_IDF)
+#endif  // !defined(USE_ESP32)
diff --git a/esphome/components/weikai_i2c/weikai_i2c.cpp b/esphome/components/weikai_i2c/weikai_i2c.cpp
index 32e7ec4f23..03ac74e070 100644
--- a/esphome/components/weikai_i2c/weikai_i2c.cpp
+++ b/esphome/components/weikai_i2c/weikai_i2c.cpp
@@ -142,8 +142,7 @@ void WeikaiRegisterI2C::write_fifo(uint8_t *data, size_t length) {
 void WeikaiComponentI2C::setup() {
   // before any manipulation we store the address to base_address_ for future use
   this->base_address_ = this->address_;
-  ESP_LOGCONFIG(TAG, "Running setup for '%s' with %d UARTs at @%02X", this->get_name(), this->children_.size(),
-                this->base_address_);
+  ESP_LOGCONFIG(TAG, "Setup %s (%d UARTs) @ 0x%02X", this->get_name(), this->children_.size(), this->base_address_);
 
   // enable all channels
   this->reg(WKREG_GENA, 0) = GENA_C1EN | GENA_C2EN | GENA_C3EN | GENA_C4EN;
diff --git a/esphome/components/weikai_spi/weikai_spi.cpp b/esphome/components/weikai_spi/weikai_spi.cpp
index a43e0e6599..7bcb817f09 100644
--- a/esphome/components/weikai_spi/weikai_spi.cpp
+++ b/esphome/components/weikai_spi/weikai_spi.cpp
@@ -156,7 +156,7 @@ void WeikaiRegisterSPI::write_fifo(uint8_t *data, size_t length) {
 ///////////////////////////////////////////////////////////////////////////////
 void WeikaiComponentSPI::setup() {
   using namespace weikai;
-  ESP_LOGCONFIG(TAG, "Running setup for '%s' with %d UARTs", this->get_name(), this->children_.size());
+  ESP_LOGCONFIG(TAG, "Setup %s (%d UARTs)", this->get_name(), this->children_.size());
   this->spi_setup();
   // enable all channels
   this->reg(WKREG_GENA, 0) = GENA_C1EN | GENA_C2EN | GENA_C3EN | GENA_C4EN;
diff --git a/esphome/components/wifi/__init__.py b/esphome/components/wifi/__init__.py
index 61f37556ba..1f742dc1a8 100644
--- a/esphome/components/wifi/__init__.py
+++ b/esphome/components/wifi/__init__.py
@@ -1,10 +1,12 @@
 from esphome import automation
 from esphome.automation import Condition
 import esphome.codegen as cg
+from esphome.components.const import CONF_USE_PSRAM
 from esphome.components.esp32 import add_idf_sdkconfig_option, const, get_esp32_variant
 from esphome.components.network import IPAddress
 from esphome.config_helpers import filter_source_files_from_platform
 import esphome.config_validation as cv
+from esphome.config_validation import only_with_esp_idf
 from esphome.const import (
     CONF_AP,
     CONF_BSSID,
@@ -42,14 +44,14 @@ from esphome.const import (
     CONF_USERNAME,
     PlatformFramework,
 )
-from esphome.core import CORE, HexInt, coroutine_with_priority
+from esphome.core import CORE, CoroPriority, HexInt, coroutine_with_priority
 import esphome.final_validate as fv
 
 from . import wpa2_eap
 
 AUTO_LOAD = ["network"]
 
-NO_WIFI_VARIANTS = [const.VARIANT_ESP32H2]
+NO_WIFI_VARIANTS = [const.VARIANT_ESP32H2, const.VARIANT_ESP32P4]
 CONF_SAVE = "save"
 
 wifi_ns = cg.esphome_ns.namespace("wifi")
@@ -123,8 +125,8 @@ EAP_AUTH_SCHEMA = cv.All(
             cv.Optional(CONF_USERNAME): cv.string_strict,
             cv.Optional(CONF_PASSWORD): cv.string_strict,
             cv.Optional(CONF_CERTIFICATE_AUTHORITY): wpa2_eap.validate_certificate,
-            cv.SplitDefault(CONF_TTLS_PHASE_2, esp32_idf="mschapv2"): cv.All(
-                cv.enum(TTLS_PHASE_2), cv.only_with_esp_idf
+            cv.SplitDefault(CONF_TTLS_PHASE_2, esp32="mschapv2"): cv.All(
+                cv.enum(TTLS_PHASE_2), cv.only_on_esp32
             ),
             cv.Inclusive(
                 CONF_CERTIFICATE, "certificate_and_key"
@@ -177,8 +179,8 @@ WIFI_NETWORK_STA = WIFI_NETWORK_BASE.extend(
 def validate_variant(_):
     if CORE.is_esp32:
         variant = get_esp32_variant()
-        if variant in NO_WIFI_VARIANTS:
-            raise cv.Invalid(f"{variant} does not support WiFi")
+        if variant in NO_WIFI_VARIANTS and "esp32_hosted" not in fv.full_config.get():
+            raise cv.Invalid(f"WiFi requires component esp32_hosted on {variant}")
 
 
 def final_validate(config):
@@ -192,45 +194,7 @@ def final_validate(config):
         )
 
 
-def final_validate_power_esp32_ble(value):
-    if not CORE.is_esp32:
-        return
-    if value != "NONE":
-        # WiFi should be in modem sleep (!=NONE) with BLE coexistence
-        # https://docs.espressif.com/projects/esp-idf/en/v3.3.5/api-guides/wifi.html#station-sleep
-        return
-    for conflicting in [
-        "esp32_ble",
-        "esp32_ble_beacon",
-        "esp32_ble_server",
-        "esp32_ble_tracker",
-    ]:
-        if conflicting not in fv.full_config.get():
-            continue
-
-        try:
-            # Only arduino 1.0.5+ and esp-idf impacted
-            cv.require_framework_version(
-                esp32_arduino=cv.Version(1, 0, 5),
-                esp_idf=cv.Version(4, 0, 0),
-            )(None)
-        except cv.Invalid:
-            pass
-        else:
-            raise cv.Invalid(
-                f"power_save_mode NONE is incompatible with {conflicting}. "
-                f"Please remove the power save mode. See also "
-                f"https://github.com/esphome/issues/issues/2141#issuecomment-865688582"
-            )
-
-
 FINAL_VALIDATE_SCHEMA = cv.All(
-    cv.Schema(
-        {
-            cv.Optional(CONF_POWER_SAVE_MODE): final_validate_power_esp32_ble,
-        },
-        extra=cv.ALLOW_EXTRA,
-    ),
     final_validate,
     validate_variant,
 )
@@ -263,8 +227,6 @@ def _validate(config):
         networks = config.get(CONF_NETWORKS, [])
         if not networks:
             raise cv.Invalid("At least one network required for fast_connect!")
-        if len(networks) != 1:
-            raise cv.Invalid("Fast connect can only be used with one network!")
 
     if CONF_USE_ADDRESS not in config:
         use_address = CORE.name + config[CONF_DOMAIN]
@@ -318,11 +280,11 @@ CONFIG_SCHEMA = cv.All(
             cv.SplitDefault(CONF_OUTPUT_POWER, esp8266=20.0): cv.All(
                 cv.decibel, cv.float_range(min=8.5, max=20.5)
             ),
-            cv.SplitDefault(CONF_ENABLE_BTM, esp32_idf=False): cv.All(
-                cv.boolean, cv.only_with_esp_idf
+            cv.SplitDefault(CONF_ENABLE_BTM, esp32=False): cv.All(
+                cv.boolean, cv.only_on_esp32
             ),
-            cv.SplitDefault(CONF_ENABLE_RRM, esp32_idf=False): cv.All(
-                cv.boolean, cv.only_with_esp_idf
+            cv.SplitDefault(CONF_ENABLE_RRM, esp32=False): cv.All(
+                cv.boolean, cv.only_on_esp32
             ),
             cv.Optional(CONF_PASSIVE_SCAN, default=False): cv.boolean,
             cv.Optional("enable_mdns"): cv.invalid(
@@ -334,6 +296,9 @@ CONFIG_SCHEMA = cv.All(
             cv.Optional(CONF_ON_DISCONNECT): automation.validate_automation(
                 single=True
             ),
+            cv.Optional(CONF_USE_PSRAM): cv.All(
+                only_with_esp_idf, cv.requires_component("psram"), cv.boolean
+            ),
         }
     ),
     _validate,
@@ -405,16 +370,21 @@ def wifi_network(config, ap, static_ip):
     return ap
 
 
-@coroutine_with_priority(60.0)
+@coroutine_with_priority(CoroPriority.COMMUNICATION)
 async def to_code(config):
     var = cg.new_Pvariable(config[CONF_ID])
     cg.add(var.set_use_address(config[CONF_USE_ADDRESS]))
 
+    # Track if any network uses Enterprise authentication
+    has_eap = False
+
     def add_sta(ap, network):
         ip_config = network.get(CONF_MANUAL_IP, config.get(CONF_MANUAL_IP))
         cg.add(var.add_sta(wifi_network(network, ap, ip_config)))
 
     for network in config.get(CONF_NETWORKS, []):
+        if CONF_EAP in network:
+            has_eap = True
         cg.with_local_variable(network[CONF_ID], WiFiAP(), add_sta, network)
 
     if CONF_AP in config:
@@ -431,6 +401,10 @@ async def to_code(config):
         add_idf_sdkconfig_option("CONFIG_ESP_WIFI_SOFTAP_SUPPORT", False)
         add_idf_sdkconfig_option("CONFIG_LWIP_DHCPS", False)
 
+    # Disable Enterprise WiFi support if no EAP is configured
+    if CORE.is_esp32:
+        add_idf_sdkconfig_option("CONFIG_ESP_WIFI_ENTERPRISE_SUPPORT", has_eap)
+
     cg.add(var.set_reboot_timeout(config[CONF_REBOOT_TIMEOUT]))
     cg.add(var.set_power_save_mode(config[CONF_POWER_SAVE_MODE]))
     cg.add(var.set_fast_connect(config[CONF_FAST_CONNECT]))
@@ -442,12 +416,10 @@ async def to_code(config):
 
     if CORE.is_esp8266:
         cg.add_library("ESP8266WiFi", None)
-    elif CORE.is_esp32 and CORE.using_arduino:
-        cg.add_library("WiFi", None)
     elif CORE.is_rp2040:
         cg.add_library("WiFi", None)
 
-    if CORE.is_esp32 and CORE.using_esp_idf:
+    if CORE.is_esp32:
         if config[CONF_ENABLE_BTM] or config[CONF_ENABLE_RRM]:
             add_idf_sdkconfig_option("CONFIG_WPA_11KV_SUPPORT", True)
             cg.add_define("USE_WIFI_11KV_SUPPORT")
@@ -456,6 +428,8 @@ async def to_code(config):
         if config[CONF_ENABLE_RRM]:
             cg.add(var.set_rrm(config[CONF_ENABLE_RRM]))
 
+    if config.get(CONF_USE_PSRAM):
+        add_idf_sdkconfig_option("CONFIG_SPIRAM_TRY_ALLOCATE_WIFI_LWIP", True)
     cg.add_define("USE_WIFI")
 
     # must register before OTA safe mode check
@@ -473,6 +447,8 @@ async def to_code(config):
             var.get_disconnect_trigger(), [], on_disconnect_config
         )
 
+    CORE.add_job(final_step)
+
 
 @automation.register_condition("wifi.connected", WiFiConnectedCondition, cv.Schema({}))
 async def wifi_connected_to_code(config, condition_id, template_arg, args):
@@ -494,6 +470,28 @@ async def wifi_disable_to_code(config, action_id, template_arg, args):
     return cg.new_Pvariable(action_id, template_arg)
 
 
+KEEP_SCAN_RESULTS_KEY = "wifi_keep_scan_results"
+
+
+def request_wifi_scan_results():
+    """Request that WiFi scan results be kept in memory after connection.
+
+    Components that need access to scan results after WiFi is connected should
+    call this function during their code generation. This prevents the WiFi component from
+    freeing scan result memory after successful connection.
+    """
+    CORE.data[KEEP_SCAN_RESULTS_KEY] = True
+
+
+@coroutine_with_priority(CoroPriority.FINAL)
+async def final_step():
+    """Final code generation step to configure scan result retention."""
+    if CORE.data.get(KEEP_SCAN_RESULTS_KEY, False):
+        cg.add(
+            cg.RawExpression("wifi::global_wifi_component->set_keep_scan_results(true)")
+        )
+
+
 @automation.register_action(
     "wifi.configure",
     WiFiConfigureAction,
@@ -532,8 +530,10 @@ async def wifi_set_sta_to_code(config, action_id, template_arg, args):
 
 FILTER_SOURCE_FILES = filter_source_files_from_platform(
     {
-        "wifi_component_esp32_arduino.cpp": {PlatformFramework.ESP32_ARDUINO},
-        "wifi_component_esp_idf.cpp": {PlatformFramework.ESP32_IDF},
+        "wifi_component_esp_idf.cpp": {
+            PlatformFramework.ESP32_IDF,
+            PlatformFramework.ESP32_ARDUINO,
+        },
         "wifi_component_esp8266.cpp": {PlatformFramework.ESP8266_ARDUINO},
         "wifi_component_libretiny.cpp": {
             PlatformFramework.BK72XX_ARDUINO,
diff --git a/esphome/components/wifi/wifi_component.cpp b/esphome/components/wifi/wifi_component.cpp
index d717b68340..5aa2a03a14 100644
--- a/esphome/components/wifi/wifi_component.cpp
+++ b/esphome/components/wifi/wifi_component.cpp
@@ -1,9 +1,8 @@
 #include "wifi_component.h"
 #ifdef USE_WIFI
 #include 
-#include 
 
-#ifdef USE_ESP_IDF
+#ifdef USE_ESP32
 #if (ESP_IDF_VERSION_MAJOR >= 5 && ESP_IDF_VERSION_MINOR >= 1)
 #include 
 #else
@@ -11,7 +10,7 @@
 #endif
 #endif
 
-#if defined(USE_ESP32) || defined(USE_ESP_IDF)
+#if defined(USE_ESP32)
 #include 
 #endif
 #ifdef USE_ESP8266
@@ -42,10 +41,28 @@ namespace wifi {
 
 static const char *const TAG = "wifi";
 
+#if defined(USE_ESP32) && defined(USE_WIFI_WPA2_EAP) && ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE
+static const char *eap_phase2_to_str(esp_eap_ttls_phase2_types type) {
+  switch (type) {
+    case ESP_EAP_TTLS_PHASE2_PAP:
+      return "pap";
+    case ESP_EAP_TTLS_PHASE2_CHAP:
+      return "chap";
+    case ESP_EAP_TTLS_PHASE2_MSCHAP:
+      return "mschap";
+    case ESP_EAP_TTLS_PHASE2_MSCHAPV2:
+      return "mschapv2";
+    case ESP_EAP_TTLS_PHASE2_EAP:
+      return "eap";
+    default:
+      return "unknown";
+  }
+}
+#endif
+
 float WiFiComponent::get_setup_priority() const { return setup_priority::WIFI; }
 
 void WiFiComponent::setup() {
-  ESP_LOGCONFIG(TAG, "Running setup");
   this->wifi_pre_setup_();
   if (this->enable_on_boot_) {
     this->start();
@@ -92,8 +109,11 @@ void WiFiComponent::start() {
     }
 
     if (this->fast_connect_) {
-      this->selected_ap_ = this->sta_[0];
-      this->load_fast_connect_settings_();
+      this->trying_loaded_ap_ = this->load_fast_connect_settings_();
+      if (!this->trying_loaded_ap_) {
+        this->ap_index_ = 0;
+        this->selected_ap_ = this->sta_[this->ap_index_];
+      }
       this->start_connecting(this->selected_ap_, false);
     } else {
       this->start_scanning();
@@ -122,6 +142,14 @@ void WiFiComponent::start() {
   this->wifi_apply_hostname_();
 }
 
+void WiFiComponent::restart_adapter() {
+  ESP_LOGW(TAG, "Restarting adapter");
+  this->wifi_mode_(false, {});
+  delay(100);  // NOLINT
+  this->num_retried_ = 0;
+  this->retry_hidden_ = false;
+}
+
 void WiFiComponent::loop() {
   this->wifi_loop_();
   const uint32_t now = App.get_loop_component_start_time();
@@ -138,10 +166,12 @@ void WiFiComponent::loop() {
 
     switch (this->state_) {
       case WIFI_COMPONENT_STATE_COOLDOWN: {
-        this->status_set_warning("waiting to reconnect");
+        this->status_set_warning(LOG_STR("waiting to reconnect"));
         if (millis() - this->action_started_ > 5000) {
           if (this->fast_connect_ || this->retry_hidden_) {
-            this->start_connecting(this->sta_[0], false);
+            if (!this->selected_ap_.get_bssid().has_value())
+              this->selected_ap_ = this->sta_[0];
+            this->start_connecting(this->selected_ap_, false);
           } else {
             this->start_scanning();
           }
@@ -149,13 +179,13 @@ void WiFiComponent::loop() {
         break;
       }
       case WIFI_COMPONENT_STATE_STA_SCANNING: {
-        this->status_set_warning("scanning for networks");
+        this->status_set_warning(LOG_STR("scanning for networks"));
         this->check_scanning_finished();
         break;
       }
       case WIFI_COMPONENT_STATE_STA_CONNECTING:
       case WIFI_COMPONENT_STATE_STA_CONNECTING_2: {
-        this->status_set_warning("associating to network");
+        this->status_set_warning(LOG_STR("associating to network"));
         this->check_connecting_finished();
         break;
       }
@@ -235,12 +265,9 @@ network::IPAddress WiFiComponent::get_dns_address(int num) {
     return this->wifi_dns_ip_(num);
   return {};
 }
-std::string WiFiComponent::get_use_address() const {
-  if (this->use_address_.empty()) {
-    return App.get_name() + ".local";
-  }
-  return this->use_address_;
-}
+// set_use_address() is guaranteed to be called during component setup by Python code generation,
+// so use_address_ will always be valid when get_use_address() is called - no fallback needed.
+const std::string &WiFiComponent::get_use_address() const { return this->use_address_; }
 void WiFiComponent::set_use_address(const std::string &use_address) { this->use_address_ = use_address; }
 
 #ifdef USE_WIFI_AP
@@ -254,32 +281,34 @@ void WiFiComponent::setup_ap_config_() {
     std::string name = App.get_name();
     if (name.length() > 32) {
       if (App.is_name_add_mac_suffix_enabled()) {
-        name.erase(name.begin() + 25, name.end() - 7);  // Remove characters between 25 and the mac address
+        // Keep first 25 chars and last 7 chars (MAC suffix), remove middle
+        name.erase(25, name.length() - 32);
       } else {
-        name = name.substr(0, 32);
+        name.resize(32);
       }
     }
     this->ap_.set_ssid(name);
   }
+  this->ap_setup_ = this->wifi_start_ap_(this->ap_);
 
-  ESP_LOGCONFIG(TAG, "Setting up AP");
-
+  auto ip_address = this->wifi_soft_ap_ip().str();
   ESP_LOGCONFIG(TAG,
+                "Setting up AP:\n"
                 "  AP SSID: '%s'\n"
-                "  AP Password: '%s'",
-                this->ap_.get_ssid().c_str(), this->ap_.get_password().c_str());
-  if (this->ap_.get_manual_ip().has_value()) {
-    auto manual = *this->ap_.get_manual_ip();
+                "  AP Password: '%s'\n"
+                "  IP Address: %s",
+                this->ap_.get_ssid().c_str(), this->ap_.get_password().c_str(), ip_address.c_str());
+
+  auto manual_ip = this->ap_.get_manual_ip();
+  if (manual_ip.has_value()) {
     ESP_LOGCONFIG(TAG,
                   "  AP Static IP: '%s'\n"
                   "  AP Gateway: '%s'\n"
                   "  AP Subnet: '%s'",
-                  manual.static_ip.str().c_str(), manual.gateway.str().c_str(), manual.subnet.str().c_str());
+                  manual_ip->static_ip.str().c_str(), manual_ip->gateway.str().c_str(),
+                  manual_ip->subnet.str().c_str());
   }
 
-  this->ap_setup_ = this->wifi_start_ap_(this->ap_);
-  ESP_LOGCONFIG(TAG, "  IP Address: %s", this->wifi_soft_ap_ip().str().c_str());
-
   if (!this->has_sta()) {
     this->state_ = WIFI_COMPONENT_STATE_AP;
   }
@@ -302,9 +331,9 @@ void WiFiComponent::set_sta(const WiFiAP &ap) {
 }
 void WiFiComponent::clear_sta() { this->sta_.clear(); }
 void WiFiComponent::save_wifi_sta(const std::string &ssid, const std::string &password) {
-  SavedWifiSettings save{};
-  snprintf(save.ssid, sizeof(save.ssid), "%s", ssid.c_str());
-  snprintf(save.password, sizeof(save.password), "%s", password.c_str());
+  SavedWifiSettings save{};  // zero-initialized - all bytes set to \0, guaranteeing null termination
+  strncpy(save.ssid, ssid.c_str(), sizeof(save.ssid) - 1);              // max 32 chars, byte 32 remains \0
+  strncpy(save.password, password.c_str(), sizeof(save.password) - 1);  // max 64 chars, byte 64 remains \0
   this->pref_.save(&save);
   // ensure it's written immediately
   global_preferences->sync();
@@ -321,8 +350,7 @@ void WiFiComponent::start_connecting(const WiFiAP &ap, bool two) {
   ESP_LOGV(TAG, "Connection Params:");
   ESP_LOGV(TAG, "  SSID: '%s'", ap.get_ssid().c_str());
   if (ap.get_bssid().has_value()) {
-    bssid_t b = *ap.get_bssid();
-    ESP_LOGV(TAG, "  BSSID: %02X:%02X:%02X:%02X:%02X:%02X", b[0], b[1], b[2], b[3], b[4], b[5]);
+    ESP_LOGV(TAG, "  BSSID: %s", format_mac_address_pretty(ap.get_bssid()->data()).c_str());
   } else {
     ESP_LOGV(TAG, "  BSSID: Not Set");
   }
@@ -334,15 +362,8 @@ void WiFiComponent::start_connecting(const WiFiAP &ap, bool two) {
     ESP_LOGV(TAG, "    Identity: " LOG_SECRET("'%s'"), eap_config.identity.c_str());
     ESP_LOGV(TAG, "    Username: " LOG_SECRET("'%s'"), eap_config.username.c_str());
     ESP_LOGV(TAG, "    Password: " LOG_SECRET("'%s'"), eap_config.password.c_str());
-#ifdef USE_ESP_IDF
-#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE
-    std::map phase2types = {{ESP_EAP_TTLS_PHASE2_PAP, "pap"},
-                                                                    {ESP_EAP_TTLS_PHASE2_CHAP, "chap"},
-                                                                    {ESP_EAP_TTLS_PHASE2_MSCHAP, "mschap"},
-                                                                    {ESP_EAP_TTLS_PHASE2_MSCHAPV2, "mschapv2"},
-                                                                    {ESP_EAP_TTLS_PHASE2_EAP, "eap"}};
-    ESP_LOGV(TAG, "    TTLS Phase 2: " LOG_SECRET("'%s'"), phase2types[eap_config.ttls_phase_2].c_str());
-#endif
+#if defined(USE_ESP32) && defined(USE_WIFI_WPA2_EAP) && ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_VERBOSE
+    ESP_LOGV(TAG, "    TTLS Phase 2: " LOG_SECRET("'%s'"), eap_phase2_to_str(eap_config.ttls_phase_2));
 #endif
     bool ca_cert_present = eap_config.ca_cert != nullptr && strlen(eap_config.ca_cert);
     bool client_cert_present = eap_config.client_cert != nullptr && strlen(eap_config.client_cert);
@@ -436,7 +457,6 @@ void WiFiComponent::print_connect_params_() {
     ESP_LOGCONFIG(TAG, "  Disabled");
     return;
   }
-  ESP_LOGCONFIG(TAG, "  SSID: " LOG_SECRET("'%s'"), wifi_ssid().c_str());
   for (auto &ip : wifi_sta_ip_addresses()) {
     if (ip.is_set()) {
       ESP_LOGCONFIG(TAG, "  IP Address: %s", ip.str().c_str());
@@ -444,22 +464,23 @@ void WiFiComponent::print_connect_params_() {
   }
   int8_t rssi = wifi_rssi();
   ESP_LOGCONFIG(TAG,
-                "  BSSID: " LOG_SECRET("%02X:%02X:%02X:%02X:%02X:%02X") "\n"
-                                                                        "  Hostname: '%s'\n"
-                                                                        "  Signal strength: %d dB %s",
-                bssid[0], bssid[1], bssid[2], bssid[3], bssid[4], bssid[5], App.get_name().c_str(), rssi,
-                LOG_STR_ARG(get_signal_bars(rssi)));
+                "  SSID: " LOG_SECRET("'%s'") "\n"
+                                              "  BSSID: " LOG_SECRET("%s") "\n"
+                                                                           "  Hostname: '%s'\n"
+                                                                           "  Signal strength: %d dB %s\n"
+                                                                           "  Channel: %" PRId32 "\n"
+                                                                           "  Subnet: %s\n"
+                                                                           "  Gateway: %s\n"
+                                                                           "  DNS1: %s\n"
+                                                                           "  DNS2: %s",
+                wifi_ssid().c_str(), format_mac_address_pretty(bssid.data()).c_str(), App.get_name().c_str(), rssi,
+                LOG_STR_ARG(get_signal_bars(rssi)), get_wifi_channel(), wifi_subnet_mask_().str().c_str(),
+                wifi_gateway_ip_().str().c_str(), wifi_dns_ip_(0).str().c_str(), wifi_dns_ip_(1).str().c_str());
+#ifdef ESPHOME_LOG_HAS_VERBOSE
   if (this->selected_ap_.get_bssid().has_value()) {
     ESP_LOGV(TAG, "  Priority: %.1f", this->get_sta_priority(*this->selected_ap_.get_bssid()));
   }
-  ESP_LOGCONFIG(TAG,
-                "  Channel: %" PRId32 "\n"
-                "  Subnet: %s\n"
-                "  Gateway: %s\n"
-                "  DNS1: %s\n"
-                "  DNS2: %s",
-                get_wifi_channel(), wifi_subnet_mask_().str().c_str(), wifi_gateway_ip_().str().c_str(),
-                wifi_dns_ip_(0).str().c_str(), wifi_dns_ip_(1).str().c_str());
+#endif
 #ifdef USE_WIFI_11KV_SUPPORT
   ESP_LOGCONFIG(TAG,
                 "  BTM: %s\n"
@@ -497,6 +518,74 @@ void WiFiComponent::start_scanning() {
   this->state_ = WIFI_COMPONENT_STATE_STA_SCANNING;
 }
 
+// Helper function for WiFi scan result comparison
+// Returns true if 'a' should be placed before 'b' in the sorted order
+[[nodiscard]] inline static bool wifi_scan_result_is_better(const WiFiScanResult &a, const WiFiScanResult &b) {
+  // Matching networks always come before non-matching
+  if (a.get_matches() && !b.get_matches())
+    return true;
+  if (!a.get_matches() && b.get_matches())
+    return false;
+
+  if (a.get_matches() && b.get_matches()) {
+    // For APs with the same SSID, always prefer stronger signal
+    // This helps with mesh networks and multiple APs
+    if (a.get_ssid() == b.get_ssid()) {
+      return a.get_rssi() > b.get_rssi();
+    }
+
+    // For different SSIDs, check priority first
+    if (a.get_priority() != b.get_priority())
+      return a.get_priority() > b.get_priority();
+    // If priorities are equal, prefer stronger signal
+    return a.get_rssi() > b.get_rssi();
+  }
+
+  // Both don't match - sort by signal strength
+  return a.get_rssi() > b.get_rssi();
+}
+
+// Helper function for insertion sort of WiFi scan results
+// Using insertion sort instead of std::stable_sort saves flash memory
+// by avoiding template instantiations (std::rotate, std::stable_sort, lambdas)
+// IMPORTANT: This sort is stable (preserves relative order of equal elements)
+template static void insertion_sort_scan_results(VectorType &results) {
+  const size_t size = results.size();
+  for (size_t i = 1; i < size; i++) {
+    // Make a copy to avoid issues with move semantics during comparison
+    WiFiScanResult key = results[i];
+    int32_t j = i - 1;
+
+    // Move elements that are worse than key to the right
+    // For stability, we only move if key is strictly better than results[j]
+    while (j >= 0 && wifi_scan_result_is_better(key, results[j])) {
+      results[j + 1] = results[j];
+      j--;
+    }
+    results[j + 1] = key;
+  }
+}
+
+// Helper function to log scan results - marked noinline to prevent re-inlining into loop
+__attribute__((noinline)) static void log_scan_result(const WiFiScanResult &res) {
+  char bssid_s[18];
+  auto bssid = res.get_bssid();
+  format_mac_addr_upper(bssid.data(), bssid_s);
+
+  if (res.get_matches()) {
+    ESP_LOGI(TAG, "- '%s' %s" LOG_SECRET("(%s) ") "%s", res.get_ssid().c_str(),
+             res.get_is_hidden() ? LOG_STR_LITERAL("(HIDDEN) ") : LOG_STR_LITERAL(""), bssid_s,
+             LOG_STR_ARG(get_signal_bars(res.get_rssi())));
+    ESP_LOGD(TAG,
+             "    Channel: %u\n"
+             "    RSSI: %d dB",
+             res.get_channel(), res.get_rssi());
+  } else {
+    ESP_LOGD(TAG, "- " LOG_SECRET("'%s'") " " LOG_SECRET("(%s) ") "%s", res.get_ssid().c_str(), bssid_s,
+             LOG_STR_ARG(get_signal_bars(res.get_rssi())));
+  }
+}
+
 void WiFiComponent::check_scanning_finished() {
   if (!this->scan_done_) {
     if (millis() - this->action_started_ > 30000) {
@@ -527,37 +616,11 @@ void WiFiComponent::check_scanning_finished() {
     }
   }
 
-  std::stable_sort(this->scan_result_.begin(), this->scan_result_.end(),
-                   [](const WiFiScanResult &a, const WiFiScanResult &b) {
-                     // return true if a is better than b
-                     if (a.get_matches() && !b.get_matches())
-                       return true;
-                     if (!a.get_matches() && b.get_matches())
-                       return false;
-
-                     if (a.get_matches() && b.get_matches()) {
-                       // if both match, check priority
-                       if (a.get_priority() != b.get_priority())
-                         return a.get_priority() > b.get_priority();
-                     }
-
-                     return a.get_rssi() > b.get_rssi();
-                   });
+  // Sort scan results using insertion sort for better memory efficiency
+  insertion_sort_scan_results(this->scan_result_);
 
   for (auto &res : this->scan_result_) {
-    char bssid_s[18];
-    auto bssid = res.get_bssid();
-    sprintf(bssid_s, "%02X:%02X:%02X:%02X:%02X:%02X", bssid[0], bssid[1], bssid[2], bssid[3], bssid[4], bssid[5]);
-
-    if (res.get_matches()) {
-      ESP_LOGI(TAG, "- '%s' %s" LOG_SECRET("(%s) ") "%s", res.get_ssid().c_str(),
-               res.get_is_hidden() ? "(HIDDEN) " : "", bssid_s, LOG_STR_ARG(get_signal_bars(res.get_rssi())));
-      ESP_LOGD(TAG, "    Channel: %u", res.get_channel());
-      ESP_LOGD(TAG, "    RSSI: %d dB", res.get_rssi());
-    } else {
-      ESP_LOGD(TAG, "- " LOG_SECRET("'%s'") " " LOG_SECRET("(%s) ") "%s", res.get_ssid().c_str(), bssid_s,
-               LOG_STR_ARG(get_signal_bars(res.get_rssi())));
-    }
+    log_scan_result(res);
   }
 
   if (!this->scan_result_[0].get_matches()) {
@@ -624,10 +687,12 @@ void WiFiComponent::check_connecting_finished() {
       return;
     }
 
+    ESP_LOGI(TAG, "Connected");
     // We won't retry hidden networks unless a reconnect fails more than three times again
+    if (this->retry_hidden_ && !this->selected_ap_.get_hidden())
+      ESP_LOGW(TAG, "Network '%s' should be marked as hidden", this->selected_ap_.get_ssid().c_str());
     this->retry_hidden_ = false;
 
-    ESP_LOGI(TAG, "Connected");
     this->print_connect_params_();
 
     if (this->has_ap()) {
@@ -648,6 +713,12 @@ void WiFiComponent::check_connecting_finished() {
     this->state_ = WIFI_COMPONENT_STATE_STA_CONNECTED;
     this->num_retried_ = 0;
 
+    // Free scan results memory unless a component needs them
+    if (!this->keep_scan_results_) {
+      this->scan_result_.clear();
+      this->scan_result_.shrink_to_fit();
+    }
+
     if (this->fast_connect_) {
       this->save_fast_connect_settings_();
     }
@@ -698,18 +769,30 @@ void WiFiComponent::retry_connect() {
   delay(10);
   if (!this->is_captive_portal_active_() && !this->is_esp32_improv_active_() &&
       (this->num_retried_ > 3 || this->error_from_callback_)) {
-    if (this->num_retried_ > 5) {
-      // If retry failed for more than 5 times, let's restart STA
-      ESP_LOGW(TAG, "Restarting adapter");
-      this->wifi_mode_(false, {});
-      delay(100);  // NOLINT
+    if (this->fast_connect_) {
+      if (this->trying_loaded_ap_) {
+        this->trying_loaded_ap_ = false;
+        this->ap_index_ = 0;  // Retry from the first configured AP
+      } else if (this->ap_index_ >= this->sta_.size() - 1) {
+        ESP_LOGW(TAG, "No more APs to try");
+        this->ap_index_ = 0;
+        this->restart_adapter();
+      } else {
+        // Try next AP
+        this->ap_index_++;
+      }
       this->num_retried_ = 0;
-      this->retry_hidden_ = false;
+      this->selected_ap_ = this->sta_[this->ap_index_];
     } else {
-      // Try hidden networks after 3 failed retries
-      ESP_LOGD(TAG, "Retrying with hidden networks");
-      this->retry_hidden_ = true;
-      this->num_retried_++;
+      if (this->num_retried_ > 5) {
+        // If retry failed for more than 5 times, let's restart STA
+        this->restart_adapter();
+      } else {
+        // Try hidden networks after 3 failed retries
+        ESP_LOGD(TAG, "Retrying with hidden networks");
+        this->retry_hidden_ = true;
+        this->num_retried_++;
+      }
     }
   } else {
     this->num_retried_++;
@@ -756,17 +839,22 @@ bool WiFiComponent::is_esp32_improv_active_() {
 #endif
 }
 
-void WiFiComponent::load_fast_connect_settings_() {
+bool WiFiComponent::load_fast_connect_settings_() {
   SavedWifiFastConnectSettings fast_connect_save{};
 
   if (this->fast_connect_pref_.load(&fast_connect_save)) {
     bssid_t bssid{};
     std::copy(fast_connect_save.bssid, fast_connect_save.bssid + 6, bssid.begin());
+    this->ap_index_ = fast_connect_save.ap_index;
+    this->selected_ap_ = this->sta_[this->ap_index_];
     this->selected_ap_.set_bssid(bssid);
     this->selected_ap_.set_channel(fast_connect_save.channel);
 
     ESP_LOGD(TAG, "Loaded fast_connect settings");
+    return true;
   }
+
+  return false;
 }
 
 void WiFiComponent::save_fast_connect_settings_() {
@@ -778,6 +866,7 @@ void WiFiComponent::save_fast_connect_settings_() {
 
     memcpy(fast_connect_save.bssid, bssid.data(), 6);
     fast_connect_save.channel = channel;
+    fast_connect_save.ap_index = this->ap_index_;
 
     this->fast_connect_pref_.save(&fast_connect_save);
 
diff --git a/esphome/components/wifi/wifi_component.h b/esphome/components/wifi/wifi_component.h
index 64797a5801..9d32071b2b 100644
--- a/esphome/components/wifi/wifi_component.h
+++ b/esphome/components/wifi/wifi_component.h
@@ -20,7 +20,7 @@
 #include 
 #endif
 
-#if defined(USE_ESP_IDF) && defined(USE_WIFI_WPA2_EAP)
+#if defined(USE_ESP32) && defined(USE_WIFI_WPA2_EAP)
 #if (ESP_IDF_VERSION_MAJOR >= 5) && (ESP_IDF_VERSION_MINOR >= 1)
 #include 
 #else
@@ -60,6 +60,7 @@ struct SavedWifiSettings {
 struct SavedWifiFastConnectSettings {
   uint8_t bssid[6];
   uint8_t channel;
+  int8_t ap_index;
 } PACKED;  // NOLINT
 
 enum WiFiComponentState : uint8_t {
@@ -112,7 +113,7 @@ struct EAPAuth {
   const char *client_cert;
   const char *client_key;
 // used for EAP-TTLS
-#ifdef USE_ESP_IDF
+#ifdef USE_ESP32
   esp_eap_ttls_phase2_types ttls_phase_2;
 #endif
 };
@@ -120,6 +121,14 @@ struct EAPAuth {
 
 using bssid_t = std::array;
 
+// Use std::vector for RP2040 since scan count is unknown (callback-based)
+// Use FixedVector for other platforms where count is queried first
+#ifdef USE_RP2040
+template using wifi_scan_vector_t = std::vector;
+#else
+template using wifi_scan_vector_t = FixedVector;
+#endif
+
 class WiFiAP {
  public:
   void set_ssid(const std::string &ssid);
@@ -198,7 +207,7 @@ enum WiFiPowerSaveMode : uint8_t {
   WIFI_POWER_SAVE_HIGH,
 };
 
-#ifdef USE_ESP_IDF
+#ifdef USE_ESP32
 struct IDFWiFiEvent;
 #endif
 
@@ -256,6 +265,7 @@ class WiFiComponent : public Component {
   void setup() override;
   void start();
   void dump_config() override;
+  void restart_adapter();
   /// WIFI setup_priority.
   float get_setup_priority() const override;
   float get_loop_priority() const override;
@@ -273,10 +283,10 @@ class WiFiComponent : public Component {
 
   network::IPAddress get_dns_address(int num);
   network::IPAddresses get_ip_addresses();
-  std::string get_use_address() const;
+  const std::string &get_use_address() const;
   void set_use_address(const std::string &use_address);
 
-  const std::vector &get_scan_result() const { return scan_result_; }
+  const wifi_scan_vector_t &get_scan_result() const { return scan_result_; }
 
   network::IPAddress wifi_soft_ap_ip();
 
@@ -314,6 +324,7 @@ class WiFiComponent : public Component {
   int8_t wifi_rssi();
 
   void set_enable_on_boot(bool enable_on_boot) { this->enable_on_boot_ = enable_on_boot; }
+  void set_keep_scan_results(bool keep_scan_results) { this->keep_scan_results_ = keep_scan_results; }
 
   Trigger<> *get_connect_trigger() const { return this->connect_trigger_; };
   Trigger<> *get_disconnect_trigger() const { return this->disconnect_trigger_; };
@@ -353,7 +364,7 @@ class WiFiComponent : public Component {
   bool is_captive_portal_active_();
   bool is_esp32_improv_active_();
 
-  void load_fast_connect_settings_();
+  bool load_fast_connect_settings_();
   void save_fast_connect_settings_();
 
 #ifdef USE_ESP8266
@@ -366,7 +377,7 @@ class WiFiComponent : public Component {
   void wifi_event_callback_(arduino_event_id_t event, arduino_event_info_t info);
   void wifi_scan_done_callback_();
 #endif
-#ifdef USE_ESP_IDF
+#ifdef USE_ESP32
   void wifi_process_event_(IDFWiFiEvent *data);
 #endif
 
@@ -383,7 +394,7 @@ class WiFiComponent : public Component {
   std::string use_address_;
   std::vector sta_;
   std::vector sta_priorities_;
-  std::vector scan_result_;
+  wifi_scan_vector_t scan_result_;
   WiFiAP selected_ap_;
   WiFiAP ap_;
   optional output_power_;
@@ -400,12 +411,14 @@ class WiFiComponent : public Component {
   WiFiComponentState state_{WIFI_COMPONENT_STATE_OFF};
   WiFiPowerSaveMode power_save_{WIFI_POWER_SAVE_NONE};
   uint8_t num_retried_{0};
+  uint8_t ap_index_{0};
 #if USE_NETWORK_IPV6
   uint8_t num_ipv6_addresses_{0};
 #endif /* USE_NETWORK_IPV6 */
 
   // Group all boolean values together
   bool fast_connect_{false};
+  bool trying_loaded_ap_{false};
   bool retry_hidden_{false};
   bool has_ap_{false};
   bool handled_connected_state_{false};
@@ -420,6 +433,7 @@ class WiFiComponent : public Component {
 #endif
   bool enable_on_boot_;
   bool got_ipv4_address_{false};
+  bool keep_scan_results_{false};
 
   // Pointers at the end (naturally aligned)
   Trigger<> *connect_trigger_{new Trigger<>()};
diff --git a/esphome/components/wifi/wifi_component_esp32_arduino.cpp b/esphome/components/wifi/wifi_component_esp32_arduino.cpp
deleted file mode 100644
index a7877eb90b..0000000000
--- a/esphome/components/wifi/wifi_component_esp32_arduino.cpp
+++ /dev/null
@@ -1,849 +0,0 @@
-#include "wifi_component.h"
-
-#ifdef USE_WIFI
-#ifdef USE_ESP32_FRAMEWORK_ARDUINO
-
-#include 
-#include 
-
-#include 
-#include 
-#ifdef USE_WIFI_WPA2_EAP
-#include 
-#endif
-
-#ifdef USE_WIFI_AP
-#include "dhcpserver/dhcpserver.h"
-#endif  // USE_WIFI_AP
-
-#include "lwip/apps/sntp.h"
-#include "lwip/dns.h"
-#include "lwip/err.h"
-
-#ifdef CONFIG_LWIP_TCPIP_CORE_LOCKING
-#include "lwip/priv/tcpip_priv.h"
-#endif
-
-#include "esphome/core/application.h"
-#include "esphome/core/hal.h"
-#include "esphome/core/helpers.h"
-#include "esphome/core/log.h"
-#include "esphome/core/util.h"
-
-namespace esphome {
-namespace wifi {
-
-static const char *const TAG = "wifi_esp32";
-
-static esp_netif_t *s_sta_netif = nullptr;  // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
-#ifdef USE_WIFI_AP
-static esp_netif_t *s_ap_netif = nullptr;  // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
-#endif                                     // USE_WIFI_AP
-
-static bool s_sta_connecting = false;  // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
-
-void WiFiComponent::wifi_pre_setup_() {
-  uint8_t mac[6];
-  if (has_custom_mac_address()) {
-    get_mac_address_raw(mac);
-    set_mac_address(mac);
-  }
-  auto f = std::bind(&WiFiComponent::wifi_event_callback_, this, std::placeholders::_1, std::placeholders::_2);
-  WiFi.onEvent(f);
-  WiFi.persistent(false);
-  // Make sure WiFi is in clean state before anything starts
-  this->wifi_mode_(false, false);
-}
-
-bool WiFiComponent::wifi_mode_(optional sta, optional ap) {
-  wifi_mode_t current_mode = WiFiClass::getMode();
-  bool current_sta = current_mode == WIFI_MODE_STA || current_mode == WIFI_MODE_APSTA;
-  bool current_ap = current_mode == WIFI_MODE_AP || current_mode == WIFI_MODE_APSTA;
-
-  bool set_sta = sta.value_or(current_sta);
-  bool set_ap = ap.value_or(current_ap);
-
-  wifi_mode_t set_mode;
-  if (set_sta && set_ap) {
-    set_mode = WIFI_MODE_APSTA;
-  } else if (set_sta && !set_ap) {
-    set_mode = WIFI_MODE_STA;
-  } else if (!set_sta && set_ap) {
-    set_mode = WIFI_MODE_AP;
-  } else {
-    set_mode = WIFI_MODE_NULL;
-  }
-
-  if (current_mode == set_mode)
-    return true;
-
-  if (set_sta && !current_sta) {
-    ESP_LOGV(TAG, "Enabling STA");
-  } else if (!set_sta && current_sta) {
-    ESP_LOGV(TAG, "Disabling STA");
-  }
-  if (set_ap && !current_ap) {
-    ESP_LOGV(TAG, "Enabling AP");
-  } else if (!set_ap && current_ap) {
-    ESP_LOGV(TAG, "Disabling AP");
-  }
-
-  bool ret = WiFiClass::mode(set_mode);
-
-  if (!ret) {
-    ESP_LOGW(TAG, "Setting mode failed");
-    return false;
-  }
-
-  // WiFiClass::mode above calls esp_netif_create_default_wifi_sta() and
-  // esp_netif_create_default_wifi_ap(), which creates the interfaces.
-  // s_sta_netif handle is set during ESPHOME_EVENT_ID_WIFI_STA_START event
-
-#ifdef USE_WIFI_AP
-  if (set_ap)
-    s_ap_netif = esp_netif_get_handle_from_ifkey("WIFI_AP_DEF");
-#endif
-
-  return ret;
-}
-
-bool WiFiComponent::wifi_sta_pre_setup_() {
-  if (!this->wifi_mode_(true, {}))
-    return false;
-
-  WiFi.setAutoReconnect(false);
-  delay(10);
-  return true;
-}
-
-bool WiFiComponent::wifi_apply_output_power_(float output_power) {
-  int8_t val = static_cast(output_power * 4);
-  return esp_wifi_set_max_tx_power(val) == ESP_OK;
-}
-
-bool WiFiComponent::wifi_apply_power_save_() {
-  wifi_ps_type_t power_save;
-  switch (this->power_save_) {
-    case WIFI_POWER_SAVE_LIGHT:
-      power_save = WIFI_PS_MIN_MODEM;
-      break;
-    case WIFI_POWER_SAVE_HIGH:
-      power_save = WIFI_PS_MAX_MODEM;
-      break;
-    case WIFI_POWER_SAVE_NONE:
-    default:
-      power_save = WIFI_PS_NONE;
-      break;
-  }
-  return esp_wifi_set_ps(power_save) == ESP_OK;
-}
-
-bool WiFiComponent::wifi_sta_connect_(const WiFiAP &ap) {
-  // enable STA
-  if (!this->wifi_mode_(true, {}))
-    return false;
-
-  // https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/network/esp_wifi.html#_CPPv417wifi_sta_config_t
-  wifi_config_t conf;
-  memset(&conf, 0, sizeof(conf));
-  if (ap.get_ssid().size() > sizeof(conf.sta.ssid)) {
-    ESP_LOGE(TAG, "SSID too long");
-    return false;
-  }
-  if (ap.get_password().size() > sizeof(conf.sta.password)) {
-    ESP_LOGE(TAG, "Password too long");
-    return false;
-  }
-  memcpy(reinterpret_cast(conf.sta.ssid), ap.get_ssid().c_str(), ap.get_ssid().size());
-  memcpy(reinterpret_cast(conf.sta.password), ap.get_password().c_str(), ap.get_password().size());
-
-  // The weakest authmode to accept in the fast scan mode
-  if (ap.get_password().empty()) {
-    conf.sta.threshold.authmode = WIFI_AUTH_OPEN;
-  } else {
-    conf.sta.threshold.authmode = WIFI_AUTH_WPA_WPA2_PSK;
-  }
-
-#ifdef USE_WIFI_WPA2_EAP
-  if (ap.get_eap().has_value()) {
-    conf.sta.threshold.authmode = WIFI_AUTH_WPA2_ENTERPRISE;
-  }
-#endif
-
-  if (ap.get_bssid().has_value()) {
-    conf.sta.bssid_set = true;
-    memcpy(conf.sta.bssid, ap.get_bssid()->data(), 6);
-  } else {
-    conf.sta.bssid_set = false;
-  }
-  if (ap.get_channel().has_value()) {
-    conf.sta.channel = *ap.get_channel();
-    conf.sta.scan_method = WIFI_FAST_SCAN;
-  } else {
-    conf.sta.scan_method = WIFI_ALL_CHANNEL_SCAN;
-  }
-  // Listen interval for ESP32 station to receive beacon when WIFI_PS_MAX_MODEM is set.
-  // Units: AP beacon intervals. Defaults to 3 if set to 0.
-  conf.sta.listen_interval = 0;
-
-  // Protected Management Frame
-  // Device will prefer to connect in PMF mode if other device also advertises PMF capability.
-  conf.sta.pmf_cfg.capable = true;
-  conf.sta.pmf_cfg.required = false;
-
-  // note, we do our own filtering
-  // The minimum rssi to accept in the fast scan mode
-  conf.sta.threshold.rssi = -127;
-
-  conf.sta.threshold.authmode = WIFI_AUTH_OPEN;
-
-  wifi_config_t current_conf;
-  esp_err_t err;
-  err = esp_wifi_get_config(WIFI_IF_STA, ¤t_conf);
-  if (err != ERR_OK) {
-    ESP_LOGW(TAG, "esp_wifi_get_config failed: %s", esp_err_to_name(err));
-    // can continue
-  }
-
-  if (memcmp(¤t_conf, &conf, sizeof(wifi_config_t)) != 0) {  // NOLINT
-    err = esp_wifi_disconnect();
-    if (err != ESP_OK) {
-      ESP_LOGV(TAG, "esp_wifi_disconnect failed: %s", esp_err_to_name(err));
-      return false;
-    }
-  }
-
-  err = esp_wifi_set_config(WIFI_IF_STA, &conf);
-  if (err != ESP_OK) {
-    ESP_LOGV(TAG, "esp_wifi_set_config failed: %s", esp_err_to_name(err));
-    return false;
-  }
-
-  if (!this->wifi_sta_ip_config_(ap.get_manual_ip())) {
-    return false;
-  }
-
-  // setup enterprise authentication if required
-#ifdef USE_WIFI_WPA2_EAP
-  if (ap.get_eap().has_value()) {
-    // note: all certificates and keys have to be null terminated. Lengths are appended by +1 to include \0.
-    EAPAuth eap = ap.get_eap().value();
-    err = esp_eap_client_set_identity((uint8_t *) eap.identity.c_str(), eap.identity.length());
-    if (err != ESP_OK) {
-      ESP_LOGV(TAG, "esp_eap_client_set_identity failed: %d", err);
-    }
-    int ca_cert_len = strlen(eap.ca_cert);
-    int client_cert_len = strlen(eap.client_cert);
-    int client_key_len = strlen(eap.client_key);
-    if (ca_cert_len) {
-      err = esp_eap_client_set_ca_cert((uint8_t *) eap.ca_cert, ca_cert_len + 1);
-      if (err != ESP_OK) {
-        ESP_LOGV(TAG, "esp_eap_client_set_ca_cert failed: %d", err);
-      }
-    }
-    // workout what type of EAP this is
-    // validation is not required as the config tool has already validated it
-    if (client_cert_len && client_key_len) {
-      // if we have certs, this must be EAP-TLS
-      err = esp_eap_client_set_certificate_and_key((uint8_t *) eap.client_cert, client_cert_len + 1,
-                                                   (uint8_t *) eap.client_key, client_key_len + 1,
-                                                   (uint8_t *) eap.password.c_str(), strlen(eap.password.c_str()));
-      if (err != ESP_OK) {
-        ESP_LOGV(TAG, "esp_eap_client_set_certificate_and_key failed: %d", err);
-      }
-    } else {
-      // in the absence of certs, assume this is username/password based
-      err = esp_eap_client_set_username((uint8_t *) eap.username.c_str(), eap.username.length());
-      if (err != ESP_OK) {
-        ESP_LOGV(TAG, "esp_eap_client_set_username failed: %d", err);
-      }
-      err = esp_eap_client_set_password((uint8_t *) eap.password.c_str(), eap.password.length());
-      if (err != ESP_OK) {
-        ESP_LOGV(TAG, "esp_eap_client_set_password failed: %d", err);
-      }
-    }
-    err = esp_wifi_sta_enterprise_enable();
-    if (err != ESP_OK) {
-      ESP_LOGV(TAG, "esp_wifi_sta_enterprise_enable failed: %d", err);
-    }
-  }
-#endif  // USE_WIFI_WPA2_EAP
-
-  this->wifi_apply_hostname_();
-
-  s_sta_connecting = true;
-
-  err = esp_wifi_connect();
-  if (err != ESP_OK) {
-    ESP_LOGW(TAG, "esp_wifi_connect failed: %s", esp_err_to_name(err));
-    return false;
-  }
-
-  return true;
-}
-
-bool WiFiComponent::wifi_sta_ip_config_(optional manual_ip) {
-  // enable STA
-  if (!this->wifi_mode_(true, {}))
-    return false;
-
-  esp_netif_dhcp_status_t dhcp_status;
-  esp_err_t err = esp_netif_dhcpc_get_status(s_sta_netif, &dhcp_status);
-  if (err != ESP_OK) {
-    ESP_LOGV(TAG, "esp_netif_dhcpc_get_status failed: %s", esp_err_to_name(err));
-    return false;
-  }
-
-  if (!manual_ip.has_value()) {
-// sntp_servermode_dhcp lwip/sntp.c (Required to lock TCPIP core functionality!)
-// https://github.com/esphome/issues/issues/6591
-// https://github.com/espressif/arduino-esp32/issues/10526
-#ifdef CONFIG_LWIP_TCPIP_CORE_LOCKING
-    if (!sys_thread_tcpip(LWIP_CORE_LOCK_QUERY_HOLDER)) {
-      LOCK_TCPIP_CORE();
-    }
-#endif
-
-    // lwIP starts the SNTP client if it gets an SNTP server from DHCP. We don't need the time, and more importantly,
-    // the built-in SNTP client has a memory leak in certain situations. Disable this feature.
-    // https://github.com/esphome/issues/issues/2299
-    sntp_servermode_dhcp(false);
-
-#ifdef CONFIG_LWIP_TCPIP_CORE_LOCKING
-    if (sys_thread_tcpip(LWIP_CORE_LOCK_QUERY_HOLDER)) {
-      UNLOCK_TCPIP_CORE();
-    }
-#endif
-
-    // No manual IP is set; use DHCP client
-    if (dhcp_status != ESP_NETIF_DHCP_STARTED) {
-      err = esp_netif_dhcpc_start(s_sta_netif);
-      if (err != ESP_OK) {
-        ESP_LOGV(TAG, "Starting DHCP client failed: %d", err);
-      }
-      return err == ESP_OK;
-    }
-    return true;
-  }
-
-  esp_netif_ip_info_t info;  // struct of ip4_addr_t with ip, netmask, gw
-  info.ip = manual_ip->static_ip;
-  info.gw = manual_ip->gateway;
-  info.netmask = manual_ip->subnet;
-  err = esp_netif_dhcpc_stop(s_sta_netif);
-  if (err != ESP_OK && err != ESP_ERR_ESP_NETIF_DHCP_ALREADY_STOPPED) {
-    ESP_LOGV(TAG, "Stopping DHCP client failed: %s", esp_err_to_name(err));
-  }
-
-  err = esp_netif_set_ip_info(s_sta_netif, &info);
-  if (err != ESP_OK) {
-    ESP_LOGV(TAG, "Setting manual IP info failed: %s", esp_err_to_name(err));
-  }
-
-  esp_netif_dns_info_t dns;
-  if (manual_ip->dns1.is_set()) {
-    dns.ip = manual_ip->dns1;
-    esp_netif_set_dns_info(s_sta_netif, ESP_NETIF_DNS_MAIN, &dns);
-  }
-  if (manual_ip->dns2.is_set()) {
-    dns.ip = manual_ip->dns2;
-    esp_netif_set_dns_info(s_sta_netif, ESP_NETIF_DNS_BACKUP, &dns);
-  }
-
-  return true;
-}
-
-network::IPAddresses WiFiComponent::wifi_sta_ip_addresses() {
-  if (!this->has_sta())
-    return {};
-  network::IPAddresses addresses;
-  esp_netif_ip_info_t ip;
-  esp_err_t err = esp_netif_get_ip_info(s_sta_netif, &ip);
-  if (err != ESP_OK) {
-    ESP_LOGV(TAG, "esp_netif_get_ip_info failed: %s", esp_err_to_name(err));
-    // TODO: do something smarter
-    // return false;
-  } else {
-    addresses[0] = network::IPAddress(&ip.ip);
-  }
-#if USE_NETWORK_IPV6
-  struct esp_ip6_addr if_ip6s[CONFIG_LWIP_IPV6_NUM_ADDRESSES];
-  uint8_t count = 0;
-  count = esp_netif_get_all_ip6(s_sta_netif, if_ip6s);
-  assert(count <= CONFIG_LWIP_IPV6_NUM_ADDRESSES);
-  for (int i = 0; i < count; i++) {
-    addresses[i + 1] = network::IPAddress(&if_ip6s[i]);
-  }
-#endif /* USE_NETWORK_IPV6 */
-  return addresses;
-}
-
-bool WiFiComponent::wifi_apply_hostname_() {
-  // setting is done in SYSTEM_EVENT_STA_START callback
-  return true;
-}
-const char *get_auth_mode_str(uint8_t mode) {
-  switch (mode) {
-    case WIFI_AUTH_OPEN:
-      return "OPEN";
-    case WIFI_AUTH_WEP:
-      return "WEP";
-    case WIFI_AUTH_WPA_PSK:
-      return "WPA PSK";
-    case WIFI_AUTH_WPA2_PSK:
-      return "WPA2 PSK";
-    case WIFI_AUTH_WPA_WPA2_PSK:
-      return "WPA/WPA2 PSK";
-    case WIFI_AUTH_WPA2_ENTERPRISE:
-      return "WPA2 Enterprise";
-    case WIFI_AUTH_WPA3_PSK:
-      return "WPA3 PSK";
-    case WIFI_AUTH_WPA2_WPA3_PSK:
-      return "WPA2/WPA3 PSK";
-    case WIFI_AUTH_WAPI_PSK:
-      return "WAPI PSK";
-    default:
-      return "UNKNOWN";
-  }
-}
-
-using esphome_ip4_addr_t = esp_ip4_addr_t;
-
-std::string format_ip4_addr(const esphome_ip4_addr_t &ip) {
-  char buf[20];
-  sprintf(buf, "%u.%u.%u.%u", uint8_t(ip.addr >> 0), uint8_t(ip.addr >> 8), uint8_t(ip.addr >> 16),
-          uint8_t(ip.addr >> 24));
-  return buf;
-}
-const char *get_op_mode_str(uint8_t mode) {
-  switch (mode) {
-    case WIFI_OFF:
-      return "OFF";
-    case WIFI_STA:
-      return "STA";
-    case WIFI_AP:
-      return "AP";
-    case WIFI_AP_STA:
-      return "AP+STA";
-    default:
-      return "UNKNOWN";
-  }
-}
-const char *get_disconnect_reason_str(uint8_t reason) {
-  switch (reason) {
-    case WIFI_REASON_AUTH_EXPIRE:
-      return "Auth Expired";
-    case WIFI_REASON_AUTH_LEAVE:
-      return "Auth Leave";
-    case WIFI_REASON_ASSOC_EXPIRE:
-      return "Association Expired";
-    case WIFI_REASON_ASSOC_TOOMANY:
-      return "Too Many Associations";
-    case WIFI_REASON_NOT_AUTHED:
-      return "Not Authenticated";
-    case WIFI_REASON_NOT_ASSOCED:
-      return "Not Associated";
-    case WIFI_REASON_ASSOC_LEAVE:
-      return "Association Leave";
-    case WIFI_REASON_ASSOC_NOT_AUTHED:
-      return "Association not Authenticated";
-    case WIFI_REASON_DISASSOC_PWRCAP_BAD:
-      return "Disassociate Power Cap Bad";
-    case WIFI_REASON_DISASSOC_SUPCHAN_BAD:
-      return "Disassociate Supported Channel Bad";
-    case WIFI_REASON_IE_INVALID:
-      return "IE Invalid";
-    case WIFI_REASON_MIC_FAILURE:
-      return "Mic Failure";
-    case WIFI_REASON_4WAY_HANDSHAKE_TIMEOUT:
-      return "4-Way Handshake Timeout";
-    case WIFI_REASON_GROUP_KEY_UPDATE_TIMEOUT:
-      return "Group Key Update Timeout";
-    case WIFI_REASON_IE_IN_4WAY_DIFFERS:
-      return "IE In 4-Way Handshake Differs";
-    case WIFI_REASON_GROUP_CIPHER_INVALID:
-      return "Group Cipher Invalid";
-    case WIFI_REASON_PAIRWISE_CIPHER_INVALID:
-      return "Pairwise Cipher Invalid";
-    case WIFI_REASON_AKMP_INVALID:
-      return "AKMP Invalid";
-    case WIFI_REASON_UNSUPP_RSN_IE_VERSION:
-      return "Unsupported RSN IE version";
-    case WIFI_REASON_INVALID_RSN_IE_CAP:
-      return "Invalid RSN IE Cap";
-    case WIFI_REASON_802_1X_AUTH_FAILED:
-      return "802.1x Authentication Failed";
-    case WIFI_REASON_CIPHER_SUITE_REJECTED:
-      return "Cipher Suite Rejected";
-    case WIFI_REASON_BEACON_TIMEOUT:
-      return "Beacon Timeout";
-    case WIFI_REASON_NO_AP_FOUND:
-      return "AP Not Found";
-    case WIFI_REASON_AUTH_FAIL:
-      return "Authentication Failed";
-    case WIFI_REASON_ASSOC_FAIL:
-      return "Association Failed";
-    case WIFI_REASON_HANDSHAKE_TIMEOUT:
-      return "Handshake Failed";
-    case WIFI_REASON_CONNECTION_FAIL:
-      return "Connection Failed";
-    case WIFI_REASON_ROAMING:
-      return "Station Roaming";
-    case WIFI_REASON_UNSPECIFIED:
-    default:
-      return "Unspecified";
-  }
-}
-
-void WiFiComponent::wifi_loop_() {}
-
-#define ESPHOME_EVENT_ID_WIFI_READY ARDUINO_EVENT_WIFI_READY
-#define ESPHOME_EVENT_ID_WIFI_SCAN_DONE ARDUINO_EVENT_WIFI_SCAN_DONE
-#define ESPHOME_EVENT_ID_WIFI_STA_START ARDUINO_EVENT_WIFI_STA_START
-#define ESPHOME_EVENT_ID_WIFI_STA_STOP ARDUINO_EVENT_WIFI_STA_STOP
-#define ESPHOME_EVENT_ID_WIFI_STA_CONNECTED ARDUINO_EVENT_WIFI_STA_CONNECTED
-#define ESPHOME_EVENT_ID_WIFI_STA_DISCONNECTED ARDUINO_EVENT_WIFI_STA_DISCONNECTED
-#define ESPHOME_EVENT_ID_WIFI_STA_AUTHMODE_CHANGE ARDUINO_EVENT_WIFI_STA_AUTHMODE_CHANGE
-#define ESPHOME_EVENT_ID_WIFI_STA_GOT_IP ARDUINO_EVENT_WIFI_STA_GOT_IP
-#define ESPHOME_EVENT_ID_WIFI_STA_GOT_IP6 ARDUINO_EVENT_WIFI_STA_GOT_IP6
-#define ESPHOME_EVENT_ID_WIFI_STA_LOST_IP ARDUINO_EVENT_WIFI_STA_LOST_IP
-#define ESPHOME_EVENT_ID_WIFI_AP_START ARDUINO_EVENT_WIFI_AP_START
-#define ESPHOME_EVENT_ID_WIFI_AP_STOP ARDUINO_EVENT_WIFI_AP_STOP
-#define ESPHOME_EVENT_ID_WIFI_AP_STACONNECTED ARDUINO_EVENT_WIFI_AP_STACONNECTED
-#define ESPHOME_EVENT_ID_WIFI_AP_STADISCONNECTED ARDUINO_EVENT_WIFI_AP_STADISCONNECTED
-#define ESPHOME_EVENT_ID_WIFI_AP_STAIPASSIGNED ARDUINO_EVENT_WIFI_AP_STAIPASSIGNED
-#define ESPHOME_EVENT_ID_WIFI_AP_PROBEREQRECVED ARDUINO_EVENT_WIFI_AP_PROBEREQRECVED
-#define ESPHOME_EVENT_ID_WIFI_AP_GOT_IP6 ARDUINO_EVENT_WIFI_AP_GOT_IP6
-using esphome_wifi_event_id_t = arduino_event_id_t;
-using esphome_wifi_event_info_t = arduino_event_info_t;
-
-void WiFiComponent::wifi_event_callback_(esphome_wifi_event_id_t event, esphome_wifi_event_info_t info) {
-  switch (event) {
-    case ESPHOME_EVENT_ID_WIFI_READY: {
-      ESP_LOGV(TAG, "Ready");
-      break;
-    }
-    case ESPHOME_EVENT_ID_WIFI_SCAN_DONE: {
-      auto it = info.wifi_scan_done;
-      ESP_LOGV(TAG, "Scan done: status=%u number=%u scan_id=%u", it.status, it.number, it.scan_id);
-
-      this->wifi_scan_done_callback_();
-      break;
-    }
-    case ESPHOME_EVENT_ID_WIFI_STA_START: {
-      ESP_LOGV(TAG, "STA start");
-      // apply hostname
-      s_sta_netif = esp_netif_get_handle_from_ifkey("WIFI_STA_DEF");
-      esp_err_t err = esp_netif_set_hostname(s_sta_netif, App.get_name().c_str());
-      if (err != ERR_OK) {
-        ESP_LOGW(TAG, "esp_netif_set_hostname failed: %s", esp_err_to_name(err));
-      }
-      break;
-    }
-    case ESPHOME_EVENT_ID_WIFI_STA_STOP: {
-      ESP_LOGV(TAG, "STA stop");
-      break;
-    }
-    case ESPHOME_EVENT_ID_WIFI_STA_CONNECTED: {
-      auto it = info.wifi_sta_connected;
-      char buf[33];
-      memcpy(buf, it.ssid, it.ssid_len);
-      buf[it.ssid_len] = '\0';
-      ESP_LOGV(TAG, "Connected ssid='%s' bssid=" LOG_SECRET("%s") " channel=%u, authmode=%s", buf,
-               format_mac_address_pretty(it.bssid).c_str(), it.channel, get_auth_mode_str(it.authmode));
-#if USE_NETWORK_IPV6
-      this->set_timeout(100, [] { WiFi.enableIPv6(); });
-#endif /* USE_NETWORK_IPV6 */
-
-      break;
-    }
-    case ESPHOME_EVENT_ID_WIFI_STA_DISCONNECTED: {
-      auto it = info.wifi_sta_disconnected;
-      char buf[33];
-      memcpy(buf, it.ssid, it.ssid_len);
-      buf[it.ssid_len] = '\0';
-      if (it.reason == WIFI_REASON_NO_AP_FOUND) {
-        ESP_LOGW(TAG, "Disconnected ssid='%s' reason='Probe Request Unsuccessful'", buf);
-      } else {
-        ESP_LOGW(TAG, "Disconnected ssid='%s' bssid=" LOG_SECRET("%s") " reason='%s'", buf,
-                 format_mac_address_pretty(it.bssid).c_str(), get_disconnect_reason_str(it.reason));
-      }
-
-      uint8_t reason = it.reason;
-      if (reason == WIFI_REASON_AUTH_EXPIRE || reason == WIFI_REASON_BEACON_TIMEOUT ||
-          reason == WIFI_REASON_NO_AP_FOUND || reason == WIFI_REASON_ASSOC_FAIL ||
-          reason == WIFI_REASON_HANDSHAKE_TIMEOUT) {
-        err_t err = esp_wifi_disconnect();
-        if (err != ESP_OK) {
-          ESP_LOGV(TAG, "Disconnect failed: %s", esp_err_to_name(err));
-        }
-        this->error_from_callback_ = true;
-      }
-
-      s_sta_connecting = false;
-      break;
-    }
-    case ESPHOME_EVENT_ID_WIFI_STA_AUTHMODE_CHANGE: {
-      auto it = info.wifi_sta_authmode_change;
-      ESP_LOGV(TAG, "Authmode Change old=%s new=%s", get_auth_mode_str(it.old_mode), get_auth_mode_str(it.new_mode));
-      // Mitigate CVE-2020-12638
-      // https://lbsfilm.at/blog/wpa2-authenticationmode-downgrade-in-espressif-microprocessors
-      if (it.old_mode != WIFI_AUTH_OPEN && it.new_mode == WIFI_AUTH_OPEN) {
-        ESP_LOGW(TAG, "Potential Authmode downgrade detected, disconnecting");
-        // we can't call retry_connect() from this context, so disconnect immediately
-        // and notify main thread with error_from_callback_
-        err_t err = esp_wifi_disconnect();
-        if (err != ESP_OK) {
-          ESP_LOGW(TAG, "Disconnect failed: %s", esp_err_to_name(err));
-        }
-        this->error_from_callback_ = true;
-      }
-      break;
-    }
-    case ESPHOME_EVENT_ID_WIFI_STA_GOT_IP: {
-      auto it = info.got_ip.ip_info;
-      ESP_LOGV(TAG, "static_ip=%s gateway=%s", format_ip4_addr(it.ip).c_str(), format_ip4_addr(it.gw).c_str());
-      this->got_ipv4_address_ = true;
-#if USE_NETWORK_IPV6
-      s_sta_connecting = this->num_ipv6_addresses_ < USE_NETWORK_MIN_IPV6_ADDR_COUNT;
-#else
-      s_sta_connecting = false;
-#endif /* USE_NETWORK_IPV6 */
-      break;
-    }
-#if USE_NETWORK_IPV6
-    case ESPHOME_EVENT_ID_WIFI_STA_GOT_IP6: {
-      auto it = info.got_ip6.ip6_info;
-      ESP_LOGV(TAG, "IPv6 address=" IPV6STR, IPV62STR(it.ip));
-      this->num_ipv6_addresses_++;
-      s_sta_connecting = !(this->got_ipv4_address_ & (this->num_ipv6_addresses_ >= USE_NETWORK_MIN_IPV6_ADDR_COUNT));
-      break;
-    }
-#endif /* USE_NETWORK_IPV6 */
-    case ESPHOME_EVENT_ID_WIFI_STA_LOST_IP: {
-      ESP_LOGV(TAG, "Lost IP");
-      this->got_ipv4_address_ = false;
-      break;
-    }
-    case ESPHOME_EVENT_ID_WIFI_AP_START: {
-      ESP_LOGV(TAG, "AP start");
-      break;
-    }
-    case ESPHOME_EVENT_ID_WIFI_AP_STOP: {
-      ESP_LOGV(TAG, "AP stop");
-      break;
-    }
-    case ESPHOME_EVENT_ID_WIFI_AP_STACONNECTED: {
-      auto it = info.wifi_sta_connected;
-      auto &mac = it.bssid;
-      ESP_LOGV(TAG, "AP client connected MAC=%s", format_mac_address_pretty(mac).c_str());
-      break;
-    }
-    case ESPHOME_EVENT_ID_WIFI_AP_STADISCONNECTED: {
-      auto it = info.wifi_sta_disconnected;
-      auto &mac = it.bssid;
-      ESP_LOGV(TAG, "AP client disconnected MAC=%s", format_mac_address_pretty(mac).c_str());
-      break;
-    }
-    case ESPHOME_EVENT_ID_WIFI_AP_STAIPASSIGNED: {
-      ESP_LOGV(TAG, "AP client assigned IP");
-      break;
-    }
-    case ESPHOME_EVENT_ID_WIFI_AP_PROBEREQRECVED: {
-      auto it = info.wifi_ap_probereqrecved;
-      ESP_LOGVV(TAG, "AP receive Probe Request MAC=%s RSSI=%d", format_mac_address_pretty(it.mac).c_str(), it.rssi);
-      break;
-    }
-    default:
-      break;
-  }
-}
-
-WiFiSTAConnectStatus WiFiComponent::wifi_sta_connect_status_() {
-  const auto status = WiFi.status();
-  if (status == WL_CONNECT_FAILED || status == WL_CONNECTION_LOST) {
-    return WiFiSTAConnectStatus::ERROR_CONNECT_FAILED;
-  }
-  if (status == WL_NO_SSID_AVAIL) {
-    return WiFiSTAConnectStatus::ERROR_NETWORK_NOT_FOUND;
-  }
-  if (s_sta_connecting) {
-    return WiFiSTAConnectStatus::CONNECTING;
-  }
-  if (status == WL_CONNECTED) {
-    return WiFiSTAConnectStatus::CONNECTED;
-  }
-  return WiFiSTAConnectStatus::IDLE;
-}
-bool WiFiComponent::wifi_scan_start_(bool passive) {
-  // enable STA
-  if (!this->wifi_mode_(true, {}))
-    return false;
-
-  // need to use WiFi because of WiFiScanClass allocations :(
-  int16_t err = WiFi.scanNetworks(true, true, passive, 200);
-  if (err != WIFI_SCAN_RUNNING) {
-    ESP_LOGV(TAG, "WiFi.scanNetworks failed: %d", err);
-    return false;
-  }
-
-  return true;
-}
-void WiFiComponent::wifi_scan_done_callback_() {
-  this->scan_result_.clear();
-
-  int16_t num = WiFi.scanComplete();
-  if (num < 0)
-    return;
-
-  this->scan_result_.reserve(static_cast(num));
-  for (int i = 0; i < num; i++) {
-    String ssid = WiFi.SSID(i);
-    wifi_auth_mode_t authmode = WiFi.encryptionType(i);
-    int32_t rssi = WiFi.RSSI(i);
-    uint8_t *bssid = WiFi.BSSID(i);
-    int32_t channel = WiFi.channel(i);
-
-    WiFiScanResult scan({bssid[0], bssid[1], bssid[2], bssid[3], bssid[4], bssid[5]}, std::string(ssid.c_str()),
-                        channel, rssi, authmode != WIFI_AUTH_OPEN, ssid.length() == 0);
-    this->scan_result_.push_back(scan);
-  }
-  WiFi.scanDelete();
-  this->scan_done_ = true;
-}
-
-#ifdef USE_WIFI_AP
-bool WiFiComponent::wifi_ap_ip_config_(optional manual_ip) {
-  esp_err_t err;
-
-  // enable AP
-  if (!this->wifi_mode_({}, true))
-    return false;
-
-  esp_netif_ip_info_t info;
-  if (manual_ip.has_value()) {
-    info.ip = manual_ip->static_ip;
-    info.gw = manual_ip->gateway;
-    info.netmask = manual_ip->subnet;
-  } else {
-    info.ip = network::IPAddress(192, 168, 4, 1);
-    info.gw = network::IPAddress(192, 168, 4, 1);
-    info.netmask = network::IPAddress(255, 255, 255, 0);
-  }
-
-  err = esp_netif_dhcps_stop(s_ap_netif);
-  if (err != ESP_OK && err != ESP_ERR_ESP_NETIF_DHCP_ALREADY_STOPPED) {
-    ESP_LOGE(TAG, "esp_netif_dhcps_stop failed: %s", esp_err_to_name(err));
-    return false;
-  }
-
-  err = esp_netif_set_ip_info(s_ap_netif, &info);
-  if (err != ESP_OK) {
-    ESP_LOGE(TAG, "esp_netif_set_ip_info failed: %d", err);
-    return false;
-  }
-
-  dhcps_lease_t lease;
-  lease.enable = true;
-  network::IPAddress start_address = network::IPAddress(&info.ip);
-  start_address += 99;
-  lease.start_ip = start_address;
-  ESP_LOGV(TAG, "DHCP server IP lease start: %s", start_address.str().c_str());
-  start_address += 10;
-  lease.end_ip = start_address;
-  ESP_LOGV(TAG, "DHCP server IP lease end: %s", start_address.str().c_str());
-  err = esp_netif_dhcps_option(s_ap_netif, ESP_NETIF_OP_SET, ESP_NETIF_REQUESTED_IP_ADDRESS, &lease, sizeof(lease));
-
-  if (err != ESP_OK) {
-    ESP_LOGE(TAG, "esp_netif_dhcps_option failed: %d", err);
-    return false;
-  }
-
-  err = esp_netif_dhcps_start(s_ap_netif);
-
-  if (err != ESP_OK) {
-    ESP_LOGE(TAG, "esp_netif_dhcps_start failed: %d", err);
-    return false;
-  }
-
-  return true;
-}
-
-bool WiFiComponent::wifi_start_ap_(const WiFiAP &ap) {
-  // enable AP
-  if (!this->wifi_mode_({}, true))
-    return false;
-
-  wifi_config_t conf;
-  memset(&conf, 0, sizeof(conf));
-  if (ap.get_ssid().size() > sizeof(conf.ap.ssid)) {
-    ESP_LOGE(TAG, "AP SSID too long");
-    return false;
-  }
-  memcpy(reinterpret_cast(conf.ap.ssid), ap.get_ssid().c_str(), ap.get_ssid().size());
-  conf.ap.channel = ap.get_channel().value_or(1);
-  conf.ap.ssid_hidden = ap.get_ssid().size();
-  conf.ap.max_connection = 5;
-  conf.ap.beacon_interval = 100;
-
-  if (ap.get_password().empty()) {
-    conf.ap.authmode = WIFI_AUTH_OPEN;
-    *conf.ap.password = 0;
-  } else {
-    conf.ap.authmode = WIFI_AUTH_WPA2_PSK;
-    if (ap.get_password().size() > sizeof(conf.ap.password)) {
-      ESP_LOGE(TAG, "AP password too long");
-      return false;
-    }
-    memcpy(reinterpret_cast(conf.ap.password), ap.get_password().c_str(), ap.get_password().size());
-  }
-
-  // pairwise cipher of SoftAP, group cipher will be derived using this.
-  conf.ap.pairwise_cipher = WIFI_CIPHER_TYPE_CCMP;
-
-  esp_err_t err = esp_wifi_set_config(WIFI_IF_AP, &conf);
-  if (err != ESP_OK) {
-    ESP_LOGV(TAG, "esp_wifi_set_config failed: %d", err);
-    return false;
-  }
-
-  yield();
-
-  if (!this->wifi_ap_ip_config_(ap.get_manual_ip())) {
-    ESP_LOGV(TAG, "wifi_ap_ip_config_ failed");
-    return false;
-  }
-
-  return true;
-}
-
-network::IPAddress WiFiComponent::wifi_soft_ap_ip() {
-  esp_netif_ip_info_t ip;
-  esp_netif_get_ip_info(s_ap_netif, &ip);
-  return network::IPAddress(&ip.ip);
-}
-#endif  // USE_WIFI_AP
-
-bool WiFiComponent::wifi_disconnect_() { return esp_wifi_disconnect(); }
-
-bssid_t WiFiComponent::wifi_bssid() {
-  bssid_t bssid{};
-  uint8_t *raw_bssid = WiFi.BSSID();
-  if (raw_bssid != nullptr) {
-    for (size_t i = 0; i < bssid.size(); i++)
-      bssid[i] = raw_bssid[i];
-  }
-  return bssid;
-}
-std::string WiFiComponent::wifi_ssid() { return WiFi.SSID().c_str(); }
-int8_t WiFiComponent::wifi_rssi() { return WiFi.RSSI(); }
-int32_t WiFiComponent::get_wifi_channel() { return WiFi.channel(); }
-network::IPAddress WiFiComponent::wifi_subnet_mask_() { return network::IPAddress(WiFi.subnetMask()); }
-network::IPAddress WiFiComponent::wifi_gateway_ip_() { return network::IPAddress(WiFi.gatewayIP()); }
-network::IPAddress WiFiComponent::wifi_dns_ip_(int num) { return network::IPAddress(WiFi.dnsIP(num)); }
-
-}  // namespace wifi
-}  // namespace esphome
-
-#endif  // USE_ESP32_FRAMEWORK_ARDUINO
-#endif
diff --git a/esphome/components/wifi/wifi_component_esp8266.cpp b/esphome/components/wifi/wifi_component_esp8266.cpp
index ae1daed8b5..59909b2cb5 100644
--- a/esphome/components/wifi/wifi_component_esp8266.cpp
+++ b/esphome/components/wifi/wifi_component_esp8266.cpp
@@ -301,7 +301,7 @@ bool WiFiComponent::wifi_sta_connect_(const WiFiAP &ap) {
       // if we have certs, this must be EAP-TLS
       ret = wifi_station_set_enterprise_cert_key((uint8_t *) eap.client_cert, client_cert_len + 1,
                                                  (uint8_t *) eap.client_key, client_key_len + 1,
-                                                 (uint8_t *) eap.password.c_str(), strlen(eap.password.c_str()));
+                                                 (uint8_t *) eap.password.c_str(), eap.password.length());
       if (ret) {
         ESP_LOGV(TAG, "esp_wifi_sta_wpa2_ent_set_cert_key failed: %d", ret);
       }
@@ -696,7 +696,15 @@ void WiFiComponent::wifi_scan_done_callback_(void *arg, STATUS status) {
     this->retry_connect();
     return;
   }
+
+  // Count the number of results first
   auto *head = reinterpret_cast(arg);
+  size_t count = 0;
+  for (bss_info *it = head; it != nullptr; it = STAILQ_NEXT(it, next)) {
+    count++;
+  }
+
+  this->scan_result_.init(count);
   for (bss_info *it = head; it != nullptr; it = STAILQ_NEXT(it, next)) {
     WiFiScanResult res({it->bssid[0], it->bssid[1], it->bssid[2], it->bssid[3], it->bssid[4], it->bssid[5]},
                        std::string(reinterpret_cast(it->ssid), it->ssid_len), it->channel, it->rssi,
diff --git a/esphome/components/wifi/wifi_component_esp_idf.cpp b/esphome/components/wifi/wifi_component_esp_idf.cpp
index f0655a6d1d..951f5803a6 100644
--- a/esphome/components/wifi/wifi_component_esp_idf.cpp
+++ b/esphome/components/wifi/wifi_component_esp_idf.cpp
@@ -1,7 +1,7 @@
 #include "wifi_component.h"
 
 #ifdef USE_WIFI
-#ifdef USE_ESP_IDF
+#ifdef USE_ESP32
 
 #include 
 #include 
@@ -27,6 +27,10 @@
 #include "dhcpserver/dhcpserver.h"
 #endif  // USE_WIFI_AP
 
+#ifdef USE_CAPTIVE_PORTAL
+#include "esphome/components/captive_portal/captive_portal.h"
+#endif
+
 #include "lwip/apps/sntp.h"
 #include "lwip/dns.h"
 #include "lwip/err.h"
@@ -404,11 +408,11 @@ bool WiFiComponent::wifi_sta_connect_(const WiFiAP &ap) {
 #if (ESP_IDF_VERSION_MAJOR >= 5) && (ESP_IDF_VERSION_MINOR >= 1)
       err = esp_eap_client_set_certificate_and_key((uint8_t *) eap.client_cert, client_cert_len + 1,
                                                    (uint8_t *) eap.client_key, client_key_len + 1,
-                                                   (uint8_t *) eap.password.c_str(), strlen(eap.password.c_str()));
+                                                   (uint8_t *) eap.password.c_str(), eap.password.length());
 #else
       err = esp_wifi_sta_wpa2_ent_set_cert_key((uint8_t *) eap.client_cert, client_cert_len + 1,
                                                (uint8_t *) eap.client_key, client_key_len + 1,
-                                               (uint8_t *) eap.password.c_str(), strlen(eap.password.c_str()));
+                                               (uint8_t *) eap.password.c_str(), eap.password.length());
 #endif
       if (err != ESP_OK) {
         ESP_LOGV(TAG, "set_cert_key failed %d", err);
@@ -473,6 +477,12 @@ bool WiFiComponent::wifi_sta_ip_config_(optional manual_ip) {
   if (!this->wifi_mode_(true, {}))
     return false;
 
+  // Check if the STA interface is initialized before using it
+  if (s_sta_netif == nullptr) {
+    ESP_LOGW(TAG, "STA interface not initialized");
+    return false;
+  }
+
   esp_netif_dhcp_status_t dhcp_status;
   esp_err_t err = esp_netif_dhcpc_get_status(s_sta_netif, &dhcp_status);
   if (err != ESP_OK) {
@@ -640,8 +650,22 @@ const char *get_disconnect_reason_str(uint8_t reason) {
       return "Handshake Failed";
     case WIFI_REASON_CONNECTION_FAIL:
       return "Connection Failed";
+    case WIFI_REASON_AP_TSF_RESET:
+      return "AP TSF reset";
     case WIFI_REASON_ROAMING:
       return "Station Roaming";
+    case WIFI_REASON_ASSOC_COMEBACK_TIME_TOO_LONG:
+      return "Association comeback time too long";
+    case WIFI_REASON_SA_QUERY_TIMEOUT:
+      return "SA query timeout";
+#if (ESP_IDF_VERSION_MAJOR >= 5) && (ESP_IDF_VERSION_MINOR >= 2)
+    case WIFI_REASON_NO_AP_FOUND_W_COMPATIBLE_SECURITY:
+      return "No AP found with compatible security";
+    case WIFI_REASON_NO_AP_FOUND_IN_AUTHMODE_THRESHOLD:
+      return "No AP found in auth mode threshold";
+    case WIFI_REASON_NO_AP_FOUND_IN_RSSI_THRESHOLD:
+      return "No AP found in RSSI threshold";
+#endif
     case WIFI_REASON_UNSPECIFIED:
     default:
       return "Unspecified";
@@ -760,7 +784,7 @@ void WiFiComponent::wifi_process_event_(IDFWiFiEvent *data) {
     }
     records.resize(number);
 
-    scan_result_.reserve(number);
+    scan_result_.init(number);
     for (int i = 0; i < number; i++) {
       auto &record = records[i];
       bssid_t bssid;
@@ -853,6 +877,12 @@ bool WiFiComponent::wifi_ap_ip_config_(optional manual_ip) {
   if (!this->wifi_mode_({}, true))
     return false;
 
+  // Check if the AP interface is initialized before using it
+  if (s_ap_netif == nullptr) {
+    ESP_LOGW(TAG, "AP interface not initialized");
+    return false;
+  }
+
   esp_netif_ip_info_t info;
   if (manual_ip.has_value()) {
     info.ip = manual_ip->static_ip;
@@ -892,6 +922,22 @@ bool WiFiComponent::wifi_ap_ip_config_(optional manual_ip) {
     return false;
   }
 
+#if defined(USE_CAPTIVE_PORTAL) && ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(5, 4, 0)
+  // Configure DHCP Option 114 (Captive Portal URI) if captive portal is enabled
+  // This provides a standards-compliant way for clients to discover the captive portal
+  if (captive_portal::global_captive_portal != nullptr) {
+    static char captive_portal_uri[32];
+    snprintf(captive_portal_uri, sizeof(captive_portal_uri), "http://%s", network::IPAddress(&info.ip).str().c_str());
+    err = esp_netif_dhcps_option(s_ap_netif, ESP_NETIF_OP_SET, ESP_NETIF_CAPTIVEPORTAL_URI, captive_portal_uri,
+                                 strlen(captive_portal_uri));
+    if (err != ESP_OK) {
+      ESP_LOGV(TAG, "Failed to set DHCP captive portal URI: %s", esp_err_to_name(err));
+    } else {
+      ESP_LOGV(TAG, "DHCP Captive Portal URI set to: %s", captive_portal_uri);
+    }
+  }
+#endif
+
   err = esp_netif_dhcps_start(s_ap_netif);
 
   if (err != ESP_OK) {
@@ -1024,5 +1070,5 @@ network::IPAddress WiFiComponent::wifi_dns_ip_(int num) {
 }  // namespace wifi
 }  // namespace esphome
 
-#endif  // USE_ESP_IDF
+#endif  // USE_ESP32
 #endif
diff --git a/esphome/components/wifi/wifi_component_libretiny.cpp b/esphome/components/wifi/wifi_component_libretiny.cpp
index b15f710150..cb179d9022 100644
--- a/esphome/components/wifi/wifi_component_libretiny.cpp
+++ b/esphome/components/wifi/wifi_component_libretiny.cpp
@@ -411,7 +411,7 @@ void WiFiComponent::wifi_scan_done_callback_() {
   if (num < 0)
     return;
 
-  this->scan_result_.reserve(static_cast(num));
+  this->scan_result_.init(static_cast(num));
   for (int i = 0; i < num; i++) {
     String ssid = WiFi.SSID(i);
     wifi_auth_mode_t authmode = WiFi.encryptionType(i);
diff --git a/esphome/components/wifi_info/text_sensor.py b/esphome/components/wifi_info/text_sensor.py
index 4ceb73a695..a4da582c55 100644
--- a/esphome/components/wifi_info/text_sensor.py
+++ b/esphome/components/wifi_info/text_sensor.py
@@ -1,5 +1,5 @@
 import esphome.codegen as cg
-from esphome.components import text_sensor
+from esphome.components import text_sensor, wifi
 import esphome.config_validation as cv
 from esphome.const import (
     CONF_BSSID,
@@ -77,7 +77,9 @@ async def to_code(config):
     await setup_conf(config, CONF_SSID)
     await setup_conf(config, CONF_BSSID)
     await setup_conf(config, CONF_MAC_ADDRESS)
-    await setup_conf(config, CONF_SCAN_RESULTS)
+    if CONF_SCAN_RESULTS in config:
+        await setup_conf(config, CONF_SCAN_RESULTS)
+        wifi.request_wifi_scan_results()
     await setup_conf(config, CONF_DNS_ADDRESS)
     if conf := config.get(CONF_IP_ADDRESS):
         wifi_info = await text_sensor.new_text_sensor(config[CONF_IP_ADDRESS])
diff --git a/esphome/components/wifi_info/wifi_info_text_sensor.h b/esphome/components/wifi_info/wifi_info_text_sensor.h
index 0aa44a0894..2cb96123a0 100644
--- a/esphome/components/wifi_info/wifi_info_text_sensor.h
+++ b/esphome/components/wifi_info/wifi_info_text_sensor.h
@@ -1,6 +1,7 @@
 #pragma once
 
 #include "esphome/core/component.h"
+#include "esphome/core/helpers.h"
 #include "esphome/components/text_sensor/text_sensor.h"
 #include "esphome/components/wifi/wifi_component.h"
 #ifdef USE_WIFI
@@ -28,7 +29,6 @@ class IPAddressWiFiInfo : public PollingComponent, public text_sensor::TextSenso
     }
   }
   float get_setup_priority() const override { return setup_priority::AFTER_WIFI; }
-  std::string unique_id() override { return get_mac_address() + "-wifiinfo-ip"; }
   void dump_config() override;
   void add_ip_sensors(uint8_t index, text_sensor::TextSensor *s) { this->ip_sensors_[index] = s; }
 
@@ -51,7 +51,6 @@ class DNSAddressWifiInfo : public PollingComponent, public text_sensor::TextSens
     }
   }
   float get_setup_priority() const override { return setup_priority::AFTER_WIFI; }
-  std::string unique_id() override { return get_mac_address() + "-wifiinfo-dns"; }
   void dump_config() override;
 
  protected:
@@ -80,7 +79,6 @@ class ScanResultsWiFiInfo : public PollingComponent, public text_sensor::TextSen
     }
   }
   float get_setup_priority() const override { return setup_priority::AFTER_WIFI; }
-  std::string unique_id() override { return get_mac_address() + "-wifiinfo-scanresults"; }
   void dump_config() override;
 
  protected:
@@ -97,7 +95,6 @@ class SSIDWiFiInfo : public PollingComponent, public text_sensor::TextSensor {
     }
   }
   float get_setup_priority() const override { return setup_priority::AFTER_WIFI; }
-  std::string unique_id() override { return get_mac_address() + "-wifiinfo-ssid"; }
   void dump_config() override;
 
  protected:
@@ -110,13 +107,12 @@ class BSSIDWiFiInfo : public PollingComponent, public text_sensor::TextSensor {
     wifi::bssid_t bssid = wifi::global_wifi_component->wifi_bssid();
     if (memcmp(bssid.data(), last_bssid_.data(), 6) != 0) {
       std::copy(bssid.begin(), bssid.end(), last_bssid_.begin());
-      char buf[30];
-      sprintf(buf, "%02X:%02X:%02X:%02X:%02X:%02X", bssid[0], bssid[1], bssid[2], bssid[3], bssid[4], bssid[5]);
+      char buf[18];
+      format_mac_addr_upper(bssid.data(), buf);
       this->publish_state(buf);
     }
   }
   float get_setup_priority() const override { return setup_priority::AFTER_WIFI; }
-  std::string unique_id() override { return get_mac_address() + "-wifiinfo-bssid"; }
   void dump_config() override;
 
  protected:
@@ -126,7 +122,6 @@ class BSSIDWiFiInfo : public PollingComponent, public text_sensor::TextSensor {
 class MacAddressWifiInfo : public Component, public text_sensor::TextSensor {
  public:
   void setup() override { this->publish_state(get_mac_address_pretty()); }
-  std::string unique_id() override { return get_mac_address() + "-wifiinfo-macadr"; }
   void dump_config() override;
 };
 
diff --git a/esphome/components/wifi_signal/wifi_signal_sensor.h b/esphome/components/wifi_signal/wifi_signal_sensor.h
index fbe03a6404..5cfd19b523 100644
--- a/esphome/components/wifi_signal/wifi_signal_sensor.h
+++ b/esphome/components/wifi_signal/wifi_signal_sensor.h
@@ -13,7 +13,6 @@ class WiFiSignalSensor : public sensor::Sensor, public PollingComponent {
   void update() override { this->publish_state(wifi::global_wifi_component->wifi_rssi()); }
   void dump_config() override;
 
-  std::string unique_id() override { return get_mac_address() + "-wifisignal"; }
   float get_setup_priority() const override { return setup_priority::AFTER_WIFI; }
 };
 
diff --git a/esphome/components/wireguard/__init__.py b/esphome/components/wireguard/__init__.py
index 8eff8e7b2a..50c7980215 100644
--- a/esphome/components/wireguard/__init__.py
+++ b/esphome/components/wireguard/__init__.py
@@ -118,7 +118,7 @@ async def to_code(config):
 
     # Workaround for crash on IDF 5+
     # See https://github.com/trombik/esp_wireguard/issues/33#issuecomment-1568503651
-    if CORE.using_esp_idf:
+    if CORE.is_esp32:
         add_idf_sdkconfig_option("CONFIG_LWIP_PPP_SUPPORT", True)
 
     # This flag is added here because the esp_wireguard library statically
diff --git a/esphome/components/wireguard/wireguard.cpp b/esphome/components/wireguard/wireguard.cpp
index 1f61e2dda3..2de6f0d2e3 100644
--- a/esphome/components/wireguard/wireguard.cpp
+++ b/esphome/components/wireguard/wireguard.cpp
@@ -8,6 +8,7 @@
 #include "esphome/core/log.h"
 #include "esphome/core/time.h"
 #include "esphome/components/network/util.h"
+#include "esphome/core/helpers.h"
 
 #include 
 #include 
@@ -27,8 +28,6 @@ static const char *const LOGMSG_ONLINE = "online";
 static const char *const LOGMSG_OFFLINE = "offline";
 
 void Wireguard::setup() {
-  ESP_LOGCONFIG(TAG, "Running setup");
-
   this->wg_config_.address = this->address_.c_str();
   this->wg_config_.private_key = this->private_key_.c_str();
   this->wg_config_.endpoint = this->peer_endpoint_.c_str();
@@ -42,7 +41,10 @@ void Wireguard::setup() {
 
   this->publish_enabled_state();
 
-  this->wg_initialized_ = esp_wireguard_init(&(this->wg_config_), &(this->wg_ctx_));
+  {
+    LwIPLock lock;
+    this->wg_initialized_ = esp_wireguard_init(&(this->wg_config_), &(this->wg_ctx_));
+  }
 
   if (this->wg_initialized_ == ESP_OK) {
     ESP_LOGI(TAG, "Initialized");
@@ -249,7 +251,10 @@ void Wireguard::start_connection_() {
   }
 
   ESP_LOGD(TAG, "Starting connection");
-  this->wg_connected_ = esp_wireguard_connect(&(this->wg_ctx_));
+  {
+    LwIPLock lock;
+    this->wg_connected_ = esp_wireguard_connect(&(this->wg_ctx_));
+  }
 
   if (this->wg_connected_ == ESP_OK) {
     ESP_LOGI(TAG, "Connection started");
@@ -280,7 +285,10 @@ void Wireguard::start_connection_() {
 void Wireguard::stop_connection_() {
   if (this->wg_initialized_ == ESP_OK && this->wg_connected_ == ESP_OK) {
     ESP_LOGD(TAG, "Stopping connection");
-    esp_wireguard_disconnect(&(this->wg_ctx_));
+    {
+      LwIPLock lock;
+      esp_wireguard_disconnect(&(this->wg_ctx_));
+    }
     this->wg_connected_ = ESP_FAIL;
   }
 }
diff --git a/esphome/components/wts01/__init__.py b/esphome/components/wts01/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/esphome/components/wts01/sensor.py b/esphome/components/wts01/sensor.py
new file mode 100644
index 0000000000..bf4f0262ad
--- /dev/null
+++ b/esphome/components/wts01/sensor.py
@@ -0,0 +1,41 @@
+import esphome.codegen as cg
+from esphome.components import sensor, uart
+import esphome.config_validation as cv
+from esphome.const import (
+    DEVICE_CLASS_TEMPERATURE,
+    STATE_CLASS_MEASUREMENT,
+    UNIT_CELSIUS,
+)
+
+CONF_WTS01_ID = "wts01_id"
+CODEOWNERS = ["@alepee"]
+DEPENDENCIES = ["uart"]
+
+wts01_ns = cg.esphome_ns.namespace("wts01")
+WTS01Sensor = wts01_ns.class_(
+    "WTS01Sensor", cg.Component, uart.UARTDevice, sensor.Sensor
+)
+
+CONFIG_SCHEMA = (
+    sensor.sensor_schema(
+        WTS01Sensor,
+        unit_of_measurement=UNIT_CELSIUS,
+        accuracy_decimals=1,
+        device_class=DEVICE_CLASS_TEMPERATURE,
+        state_class=STATE_CLASS_MEASUREMENT,
+    )
+    .extend(cv.COMPONENT_SCHEMA)
+    .extend(uart.UART_DEVICE_SCHEMA)
+)
+
+FINAL_VALIDATE_SCHEMA = uart.final_validate_device_schema(
+    "wts01",
+    baud_rate=9600,
+    require_rx=True,
+)
+
+
+async def to_code(config):
+    var = await sensor.new_sensor(config)
+    await cg.register_component(var, config)
+    await uart.register_uart_device(var, config)
diff --git a/esphome/components/wts01/wts01.cpp b/esphome/components/wts01/wts01.cpp
new file mode 100644
index 0000000000..cb910d89cf
--- /dev/null
+++ b/esphome/components/wts01/wts01.cpp
@@ -0,0 +1,91 @@
+#include "wts01.h"
+#include "esphome/core/log.h"
+#include 
+
+namespace esphome {
+namespace wts01 {
+
+constexpr uint8_t HEADER_1 = 0x55;
+constexpr uint8_t HEADER_2 = 0x01;
+constexpr uint8_t HEADER_3 = 0x01;
+constexpr uint8_t HEADER_4 = 0x04;
+
+static const char *const TAG = "wts01";
+
+void WTS01Sensor::loop() {
+  // Process all available data at once
+  while (this->available()) {
+    uint8_t c;
+    if (this->read_byte(&c)) {
+      this->handle_char_(c);
+    }
+  }
+}
+
+void WTS01Sensor::dump_config() { LOG_SENSOR("", "WTS01 Sensor", this); }
+
+void WTS01Sensor::handle_char_(uint8_t c) {
+  // State machine for processing the header. Reset if something doesn't match.
+  if (this->buffer_pos_ == 0 && c != HEADER_1) {
+    return;
+  }
+
+  if (this->buffer_pos_ == 1 && c != HEADER_2) {
+    this->buffer_pos_ = 0;
+    return;
+  }
+
+  if (this->buffer_pos_ == 2 && c != HEADER_3) {
+    this->buffer_pos_ = 0;
+    return;
+  }
+
+  if (this->buffer_pos_ == 3 && c != HEADER_4) {
+    this->buffer_pos_ = 0;
+    return;
+  }
+
+  // Add byte to buffer
+  this->buffer_[this->buffer_pos_++] = c;
+
+  // Process complete packet
+  if (this->buffer_pos_ >= PACKET_SIZE) {
+    this->process_packet_();
+    this->buffer_pos_ = 0;
+  }
+}
+
+void WTS01Sensor::process_packet_() {
+  // Based on Tasmota implementation
+  // Format: 55 01 01 04 01 11 16 12 95
+  // header            T  Td Ck  - T = Temperature, Td = Temperature decimal, Ck = Checksum
+  uint8_t calculated_checksum = 0;
+  for (uint8_t i = 0; i < PACKET_SIZE - 1; i++) {
+    calculated_checksum += this->buffer_[i];
+  }
+
+  uint8_t received_checksum = this->buffer_[PACKET_SIZE - 1];
+  if (calculated_checksum != received_checksum) {
+    ESP_LOGW(TAG, "WTS01 Checksum doesn't match: 0x%02X != 0x%02X", received_checksum, calculated_checksum);
+    return;
+  }
+
+  // Extract temperature value
+  int8_t temp = this->buffer_[6];
+  int32_t sign = 1;
+
+  // Handle negative temperatures
+  if (temp < 0) {
+    sign = -1;
+  }
+
+  // Calculate temperature (temp + decimal/100)
+  float temperature = static_cast(temp) + (sign * static_cast(this->buffer_[7]) / 100.0f);
+
+  ESP_LOGV(TAG, "Received new temperature: %.2f°C", temperature);
+
+  this->publish_state(temperature);
+}
+
+}  // namespace wts01
+}  // namespace esphome
diff --git a/esphome/components/wts01/wts01.h b/esphome/components/wts01/wts01.h
new file mode 100644
index 0000000000..298595a5d6
--- /dev/null
+++ b/esphome/components/wts01/wts01.h
@@ -0,0 +1,27 @@
+#pragma once
+
+#include "esphome/core/component.h"
+#include "esphome/components/sensor/sensor.h"
+#include "esphome/components/uart/uart.h"
+
+namespace esphome {
+namespace wts01 {
+
+constexpr uint8_t PACKET_SIZE = 9;
+
+class WTS01Sensor : public sensor::Sensor, public uart::UARTDevice, public Component {
+ public:
+  void loop() override;
+  void dump_config() override;
+  float get_setup_priority() const override { return setup_priority::DATA; }
+
+ protected:
+  uint8_t buffer_[PACKET_SIZE];
+  uint8_t buffer_pos_{0};
+
+  void handle_char_(uint8_t c);
+  void process_packet_();
+};
+
+}  // namespace wts01
+}  // namespace esphome
diff --git a/esphome/components/x9c/x9c.cpp b/esphome/components/x9c/x9c.cpp
index ccd0c60b50..5cd4fba8c0 100644
--- a/esphome/components/x9c/x9c.cpp
+++ b/esphome/components/x9c/x9c.cpp
@@ -34,8 +34,6 @@ void X9cOutput::trim_value(int change_amount) {
 }
 
 void X9cOutput::setup() {
-  ESP_LOGCONFIG(TAG, "Running setup");
-
   this->inc_pin_->get_pin();
   this->inc_pin_->setup();
   this->inc_pin_->digital_write(false);
diff --git a/esphome/components/xgzp68xx/sensor.py b/esphome/components/xgzp68xx/sensor.py
index 74cef3bf7b..2b38392a02 100644
--- a/esphome/components/xgzp68xx/sensor.py
+++ b/esphome/components/xgzp68xx/sensor.py
@@ -3,6 +3,7 @@ from esphome.components import i2c, sensor
 import esphome.config_validation as cv
 from esphome.const import (
     CONF_ID,
+    CONF_OVERSAMPLING,
     CONF_PRESSURE,
     CONF_TEMPERATURE,
     DEVICE_CLASS_PRESSURE,
@@ -18,6 +19,17 @@ CODEOWNERS = ["@gcormier"]
 CONF_K_VALUE = "k_value"
 
 xgzp68xx_ns = cg.esphome_ns.namespace("xgzp68xx")
+XGZP68XXOversampling = xgzp68xx_ns.enum("XGZP68XXOversampling")
+OVERSAMPLING_OPTIONS = {
+    "256X": XGZP68XXOversampling.XGZP68XX_OVERSAMPLING_256X,
+    "512X": XGZP68XXOversampling.XGZP68XX_OVERSAMPLING_512X,
+    "1024X": XGZP68XXOversampling.XGZP68XX_OVERSAMPLING_1024X,
+    "2048X": XGZP68XXOversampling.XGZP68XX_OVERSAMPLING_2048X,
+    "4096X": XGZP68XXOversampling.XGZP68XX_OVERSAMPLING_4096X,
+    "8192X": XGZP68XXOversampling.XGZP68XX_OVERSAMPLING_8192X,
+    "16384X": XGZP68XXOversampling.XGZP68XX_OVERSAMPLING_16384X,
+    "32768X": XGZP68XXOversampling.XGZP68XX_OVERSAMPLING_32768X,
+}
 XGZP68XXComponent = xgzp68xx_ns.class_(
     "XGZP68XXComponent", cg.PollingComponent, i2c.I2CDevice
 )
@@ -31,6 +43,12 @@ CONFIG_SCHEMA = (
                 accuracy_decimals=1,
                 device_class=DEVICE_CLASS_PRESSURE,
                 state_class=STATE_CLASS_MEASUREMENT,
+            ).extend(
+                {
+                    cv.Optional(CONF_OVERSAMPLING, default="4096X"): cv.enum(
+                        OVERSAMPLING_OPTIONS, upper=True
+                    ),
+                }
             ),
             cv.Optional(CONF_TEMPERATURE): sensor.sensor_schema(
                 unit_of_measurement=UNIT_CELSIUS,
@@ -58,5 +76,6 @@ async def to_code(config):
     if pressure_config := config.get(CONF_PRESSURE):
         sens = await sensor.new_sensor(pressure_config)
         cg.add(var.set_pressure_sensor(sens))
+        cg.add(var.set_pressure_oversampling(pressure_config[CONF_OVERSAMPLING]))
 
     cg.add(var.set_k_value(config[CONF_K_VALUE]))
diff --git a/esphome/components/xgzp68xx/xgzp68xx.cpp b/esphome/components/xgzp68xx/xgzp68xx.cpp
index 52933ebdef..2b0824de0a 100644
--- a/esphome/components/xgzp68xx/xgzp68xx.cpp
+++ b/esphome/components/xgzp68xx/xgzp68xx.cpp
@@ -16,16 +16,49 @@ static const uint8_t SYSCONFIG_ADDRESS = 0xA5;
 static const uint8_t PCONFIG_ADDRESS = 0xA6;
 static const uint8_t READ_COMMAND = 0x0A;
 
+[[maybe_unused]] static const char *oversampling_to_str(XGZP68XXOversampling oversampling) {
+  switch (oversampling) {
+    case XGZP68XX_OVERSAMPLING_256X:
+      return "256x";
+    case XGZP68XX_OVERSAMPLING_512X:
+      return "512x";
+    case XGZP68XX_OVERSAMPLING_1024X:
+      return "1024x";
+    case XGZP68XX_OVERSAMPLING_2048X:
+      return "2048x";
+    case XGZP68XX_OVERSAMPLING_4096X:
+      return "4096x";
+    case XGZP68XX_OVERSAMPLING_8192X:
+      return "8192x";
+    case XGZP68XX_OVERSAMPLING_16384X:
+      return "16384x";
+    case XGZP68XX_OVERSAMPLING_32768X:
+      return "32768x";
+    default:
+      return "UNKNOWN";
+  }
+}
+
 void XGZP68XXComponent::update() {
+  // Do we need to change oversampling?
+  if (this->last_pressure_oversampling_ != this->pressure_oversampling_) {
+    uint8_t oldconfig = 0;
+    this->read_register(PCONFIG_ADDRESS, &oldconfig, 1);
+    uint8_t newconfig = (oldconfig & 0xf8) | (this->pressure_oversampling_ & 0x7);
+    this->write_register(PCONFIG_ADDRESS, &newconfig, 1);
+    ESP_LOGD(TAG, "oversampling to %s: oldconfig = 0x%x newconfig = 0x%x",
+             oversampling_to_str(this->pressure_oversampling_), oldconfig, newconfig);
+    this->last_pressure_oversampling_ = this->pressure_oversampling_;
+  }
+
   // Request temp + pressure acquisition
   this->write_register(0x30, &READ_COMMAND, 1);
 
   // Wait 20mS per datasheet
   this->set_timeout("measurement", 20, [this]() {
-    uint8_t data[5];
-    uint32_t pressure_raw;
-    uint16_t temperature_raw;
-    float pressure_in_pa, temperature;
+    uint8_t data[5] = {};
+    uint32_t pressure_raw = 0;
+    uint16_t temperature_raw = 0;
     int success;
 
     // Read the sensor data
@@ -42,23 +75,11 @@ void XGZP68XXComponent::update() {
     ESP_LOGV(TAG, "Got raw pressure=%" PRIu32 ", raw temperature=%u", pressure_raw, temperature_raw);
     ESP_LOGV(TAG, "K value is %u", this->k_value_);
 
-    // The most significant bit of both pressure and temperature will be 1 to indicate a negative value.
-    // This is directly from the datasheet, and the calculations below will handle this.
-    if (pressure_raw > pow(2, 23)) {
-      // Negative pressure
-      pressure_in_pa = (pressure_raw - pow(2, 24)) / (float) (this->k_value_);
-    } else {
-      // Positive pressure
-      pressure_in_pa = pressure_raw / (float) (this->k_value_);
-    }
+    // Sign extend the pressure
+    float pressure_in_pa = (float) (((int32_t) pressure_raw << 8) >> 8);
+    pressure_in_pa /= (float) (this->k_value_);
 
-    if (temperature_raw > pow(2, 15)) {
-      // Negative temperature
-      temperature = (float) (temperature_raw - pow(2, 16)) / 256.0f;
-    } else {
-      // Positive temperature
-      temperature = (float) temperature_raw / 256.0f;
-    }
+    float temperature = ((float) (int16_t) temperature_raw) / 256.0f;
 
     if (this->pressure_sensor_ != nullptr)
       this->pressure_sensor_->publish_state(pressure_in_pa);
@@ -69,21 +90,27 @@ void XGZP68XXComponent::update() {
 }
 
 void XGZP68XXComponent::setup() {
-  ESP_LOGCONFIG(TAG, "Running setup");
-  uint8_t config;
+  uint8_t config1 = 0, config2 = 0;
 
   // Display some sample bits to confirm we are talking to the sensor
-  this->read_register(SYSCONFIG_ADDRESS, &config, 1);
-  ESP_LOGCONFIG(TAG,
-                "Gain value is %d\n"
-                "XGZP68xx started!",
-                (config >> 3) & 0b111);
+  if (i2c::ErrorCode::ERROR_OK != this->read_register(SYSCONFIG_ADDRESS, &config1, 1)) {
+    this->mark_failed();
+    return;
+  }
+  if (i2c::ErrorCode::ERROR_OK != this->read_register(PCONFIG_ADDRESS, &config2, 1)) {
+    this->mark_failed();
+    return;
+  }
+  ESP_LOGD(TAG, "sys_config 0x%x, p_config 0x%x", config1, config2);
 }
 
 void XGZP68XXComponent::dump_config() {
   ESP_LOGCONFIG(TAG, "XGZP68xx:");
   LOG_SENSOR("  ", "Temperature: ", this->temperature_sensor_);
   LOG_SENSOR("  ", "Pressure: ", this->pressure_sensor_);
+  if (this->pressure_sensor_ != nullptr) {
+    ESP_LOGCONFIG(TAG, "    Oversampling: %s", oversampling_to_str(this->pressure_oversampling_));
+  }
   LOG_I2C_DEVICE(this);
   if (this->is_failed()) {
     ESP_LOGE(TAG, "  Connection failed");
diff --git a/esphome/components/xgzp68xx/xgzp68xx.h b/esphome/components/xgzp68xx/xgzp68xx.h
index 1bb7304b15..ce9cfd6b78 100644
--- a/esphome/components/xgzp68xx/xgzp68xx.h
+++ b/esphome/components/xgzp68xx/xgzp68xx.h
@@ -7,11 +7,29 @@
 namespace esphome {
 namespace xgzp68xx {
 
+/// Enum listing all oversampling options for the XGZP68XX.
+enum XGZP68XXOversampling : uint8_t {
+  XGZP68XX_OVERSAMPLING_256X = 0b100,
+  XGZP68XX_OVERSAMPLING_512X = 0b101,
+  XGZP68XX_OVERSAMPLING_1024X = 0b000,
+  XGZP68XX_OVERSAMPLING_2048X = 0b001,
+  XGZP68XX_OVERSAMPLING_4096X = 0b010,
+  XGZP68XX_OVERSAMPLING_8192X = 0b011,
+  XGZP68XX_OVERSAMPLING_16384X = 0b110,
+  XGZP68XX_OVERSAMPLING_32768X = 0b111,
+
+  XGZP68XX_OVERSAMPLING_UNKNOWN = (uint8_t) -1,
+};
+
 class XGZP68XXComponent : public PollingComponent, public sensor::Sensor, public i2c::I2CDevice {
  public:
   SUB_SENSOR(temperature)
   SUB_SENSOR(pressure)
   void set_k_value(uint16_t k_value) { this->k_value_ = k_value; }
+  /// Set the pressure oversampling value. Defaults to 4096X.
+  void set_pressure_oversampling(XGZP68XXOversampling pressure_oversampling) {
+    this->pressure_oversampling_ = pressure_oversampling;
+  }
 
   void update() override;
   void setup() override;
@@ -21,6 +39,8 @@ class XGZP68XXComponent : public PollingComponent, public sensor::Sensor, public
   /// Internal method to read the pressure from the component after it has been scheduled.
   void read_pressure_();
   uint16_t k_value_;
+  XGZP68XXOversampling pressure_oversampling_{XGZP68XX_OVERSAMPLING_4096X};
+  XGZP68XXOversampling last_pressure_oversampling_{XGZP68XX_OVERSAMPLING_UNKNOWN};
 };
 
 }  // namespace xgzp68xx
diff --git a/esphome/components/xl9535/xl9535.cpp b/esphome/components/xl9535/xl9535.cpp
index 7bcd98070f..958fc5eede 100644
--- a/esphome/components/xl9535/xl9535.cpp
+++ b/esphome/components/xl9535/xl9535.cpp
@@ -7,8 +7,6 @@ namespace xl9535 {
 static const char *const TAG = "xl9535";
 
 void XL9535Component::setup() {
-  ESP_LOGCONFIG(TAG, "Running setup");
-
   // Check to see if the device can read from the register
   uint8_t port = 0;
   if (this->read_register(XL9535_INPUT_PORT_0_REGISTER, &port, 1) != i2c::ERROR_OK) {
diff --git a/esphome/components/zephyr/__init__.py b/esphome/components/zephyr/__init__.py
new file mode 100644
index 0000000000..634c99876b
--- /dev/null
+++ b/esphome/components/zephyr/__init__.py
@@ -0,0 +1,259 @@
+from pathlib import Path
+from typing import TypedDict
+
+import esphome.codegen as cg
+from esphome.const import CONF_BOARD
+from esphome.core import CORE
+from esphome.helpers import copy_file_if_changed, write_file_if_changed
+
+from .const import (
+    BOOTLOADER_MCUBOOT,
+    KEY_BOARD,
+    KEY_BOOTLOADER,
+    KEY_EXTRA_BUILD_FILES,
+    KEY_OVERLAY,
+    KEY_PM_STATIC,
+    KEY_PRJ_CONF,
+    KEY_USER,
+    KEY_ZEPHYR,
+    zephyr_ns,
+)
+
+CODEOWNERS = ["@tomaszduda23"]
+AUTO_LOAD = ["preferences"]
+
+PrjConfValueType = bool | str | int
+
+
+class Section:
+    def __init__(self, name, address, size, region):
+        self.name = name
+        self.address = address
+        self.size = size
+        self.region = region
+        self.end_address = self.address + self.size
+
+    def __str__(self):
+        return (
+            f"{self.name}:\n"
+            f"  address: 0x{self.address:X}\n"
+            f"  end_address: 0x{self.end_address:X}\n"
+            f"  region: {self.region}\n"
+            f"  size: 0x{self.size:X}"
+        )
+
+
+class ZephyrData(TypedDict):
+    board: str
+    bootloader: str
+    prj_conf: dict[str, tuple[PrjConfValueType, bool]]
+    overlay: str
+    extra_build_files: dict[str, Path]
+    pm_static: list[Section]
+    user: dict[str, list[str]]
+
+
+def zephyr_set_core_data(config):
+    CORE.data[KEY_ZEPHYR] = ZephyrData(
+        board=config[CONF_BOARD],
+        bootloader=config[KEY_BOOTLOADER],
+        prj_conf={},
+        overlay="",
+        extra_build_files={},
+        pm_static=[],
+        user={},
+    )
+    return config
+
+
+def zephyr_data() -> ZephyrData:
+    return CORE.data[KEY_ZEPHYR]
+
+
+def zephyr_add_prj_conf(
+    name: str, value: PrjConfValueType, required: bool = True
+) -> None:
+    """Set an zephyr prj conf value."""
+    if not name.startswith("CONFIG_"):
+        name = "CONFIG_" + name
+    prj_conf = zephyr_data()[KEY_PRJ_CONF]
+    if name not in prj_conf:
+        prj_conf[name] = (value, required)
+        return
+    old_value, old_required = prj_conf[name]
+    if old_value != value and old_required:
+        raise ValueError(
+            f"{name} already set with value '{old_value}', cannot set again to '{value}'"
+        )
+    if required:
+        prj_conf[name] = (value, required)
+
+
+def zephyr_add_overlay(content):
+    zephyr_data()[KEY_OVERLAY] += content
+
+
+def add_extra_build_file(filename: str, path: Path) -> bool:
+    """Add an extra build file to the project."""
+    extra_build_files = zephyr_data()[KEY_EXTRA_BUILD_FILES]
+    if filename not in extra_build_files:
+        extra_build_files[filename] = path
+        return True
+    return False
+
+
+def add_extra_script(stage: str, filename: str, path: Path) -> None:
+    """Add an extra script to the project."""
+    key = f"{stage}:{filename}"
+    if add_extra_build_file(filename, path):
+        cg.add_platformio_option("extra_scripts", [key])
+
+
+def zephyr_to_code(config):
+    cg.add(zephyr_ns.setup_preferences())
+    cg.add_build_flag("-DUSE_ZEPHYR")
+    cg.set_cpp_standard("gnu++20")
+    # build is done by west so bypass board checking in platformio
+    cg.add_platformio_option("boards_dir", CORE.relative_build_path("boards"))
+
+    # c++ support
+    zephyr_add_prj_conf("NEWLIB_LIBC", True)
+    zephyr_add_prj_conf("CONFIG_FPU", True)
+    zephyr_add_prj_conf("NEWLIB_LIBC_FLOAT_PRINTF", True)
+    zephyr_add_prj_conf("CPLUSPLUS", True)
+    zephyr_add_prj_conf("CONFIG_STD_CPP20", True)
+    zephyr_add_prj_conf("LIB_CPLUSPLUS", True)
+    # preferences
+    zephyr_add_prj_conf("SETTINGS", True)
+    zephyr_add_prj_conf("NVS", True)
+    zephyr_add_prj_conf("FLASH_MAP", True)
+    zephyr_add_prj_conf("CONFIG_FLASH", True)
+    # watchdog
+    zephyr_add_prj_conf("WATCHDOG", True)
+    zephyr_add_prj_conf("WDT_DISABLE_AT_BOOT", False)
+    # disable console
+    zephyr_add_prj_conf("UART_CONSOLE", False)
+    zephyr_add_prj_conf("CONSOLE", False, False)
+    # use NFC pins as GPIO
+    zephyr_add_prj_conf("NFCT_PINS_AS_GPIOS", True)
+
+    #  os: ***** USAGE FAULT *****
+    #  os:   Illegal load of EXC_RETURN into PC
+    zephyr_add_prj_conf("MAIN_STACK_SIZE", 2048)
+
+    add_extra_script(
+        "pre",
+        "pre_build.py",
+        Path(__file__).parent / "pre_build.py.script",
+    )
+
+
+def _format_prj_conf_val(value: PrjConfValueType) -> str:
+    if isinstance(value, bool):
+        return "y" if value else "n"
+    if isinstance(value, int):
+        return str(value)
+    if isinstance(value, str):
+        return f'"{value}"'
+    raise ValueError
+
+
+def zephyr_add_cdc_acm(config, id):
+    zephyr_add_prj_conf("USB_DEVICE_STACK", True)
+    zephyr_add_prj_conf("USB_CDC_ACM", True)
+    # prevent device to go to susspend, without this communication stop working in python
+    # there should be a way to solve it
+    zephyr_add_prj_conf("USB_DEVICE_REMOTE_WAKEUP", False)
+    # prevent logging when buffer is full
+    zephyr_add_prj_conf("USB_CDC_ACM_LOG_LEVEL_WRN", True)
+    zephyr_add_overlay(
+        f"""
+&zephyr_udc0 {{
+    cdc_acm_uart{id}: cdc_acm_uart{id} {{
+        compatible = "zephyr,cdc-acm-uart";
+    }};
+}};
+"""
+    )
+
+
+def zephyr_add_pm_static(section: Section):
+    CORE.data[KEY_ZEPHYR][KEY_PM_STATIC].extend(section)
+
+
+def zephyr_add_user(key, value):
+    user = zephyr_data()[KEY_USER]
+    if key not in user:
+        user[key] = []
+    user[key] += [value]
+
+
+def copy_files():
+    user = zephyr_data()[KEY_USER]
+    if user:
+        zephyr_add_overlay(
+            f"""
+/ {{
+    zephyr,user {{
+        {[f"{key} = {', '.join(value)};" for key, value in user.items()][0]}
+}};
+}};"""
+        )
+
+    want_opts = zephyr_data()[KEY_PRJ_CONF]
+
+    prj_conf = (
+        "\n".join(
+            f"{name}={_format_prj_conf_val(value[0])}"
+            for name, value in sorted(want_opts.items())
+        )
+        + "\n"
+    )
+
+    write_file_if_changed(CORE.relative_build_path("zephyr/prj.conf"), prj_conf)
+
+    write_file_if_changed(
+        CORE.relative_build_path("zephyr/app.overlay"),
+        zephyr_data()[KEY_OVERLAY],
+    )
+
+    if zephyr_data()[KEY_BOOTLOADER] == BOOTLOADER_MCUBOOT or zephyr_data()[
+        KEY_BOARD
+    ] in ["xiao_ble"]:
+        fake_board_manifest = """
+{
+    "frameworks": [
+        "zephyr"
+    ],
+    "name": "esphome nrf52",
+    "upload": {
+        "maximum_ram_size": 248832,
+        "maximum_size": 815104,
+        "speed": 115200
+    },
+    "url": "https://esphome.io/",
+    "vendor": "esphome",
+    "build": {
+        "softdevice": {
+            "sd_fwid": "0x00B6"
+        }
+    }
+}
+"""
+
+        write_file_if_changed(
+            CORE.relative_build_path(f"boards/{zephyr_data()[KEY_BOARD]}.json"),
+            fake_board_manifest,
+        )
+
+    for filename, path in zephyr_data()[KEY_EXTRA_BUILD_FILES].items():
+        copy_file_if_changed(
+            path,
+            CORE.relative_build_path(filename),
+        )
+
+    pm_static = "\n".join(str(item) for item in zephyr_data()[KEY_PM_STATIC])
+    if pm_static:
+        write_file_if_changed(
+            CORE.relative_build_path("zephyr/pm_static.yml"), pm_static
+        )
diff --git a/esphome/components/zephyr/const.py b/esphome/components/zephyr/const.py
new file mode 100644
index 0000000000..06a4fc42bc
--- /dev/null
+++ b/esphome/components/zephyr/const.py
@@ -0,0 +1,16 @@
+from typing import Final
+
+import esphome.codegen as cg
+
+BOOTLOADER_MCUBOOT = "mcuboot"
+
+KEY_BOOTLOADER: Final = "bootloader"
+KEY_EXTRA_BUILD_FILES: Final = "extra_build_files"
+KEY_OVERLAY: Final = "overlay"
+KEY_PM_STATIC: Final = "pm_static"
+KEY_PRJ_CONF: Final = "prj_conf"
+KEY_ZEPHYR = "zephyr"
+KEY_BOARD: Final = "board"
+KEY_USER: Final = "user"
+
+zephyr_ns = cg.esphome_ns.namespace("zephyr")
diff --git a/esphome/components/zephyr/core.cpp b/esphome/components/zephyr/core.cpp
new file mode 100644
index 0000000000..ad7a148cdb
--- /dev/null
+++ b/esphome/components/zephyr/core.cpp
@@ -0,0 +1,90 @@
+#ifdef USE_ZEPHYR
+
+#include 
+#include 
+#include 
+#include 
+#include "esphome/core/hal.h"
+#include "esphome/core/helpers.h"
+
+namespace esphome {
+
+static int wdt_channel_id = -1;  // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
+static const device *const WDT = DEVICE_DT_GET(DT_ALIAS(watchdog0));
+
+void yield() { ::k_yield(); }
+uint32_t millis() { return k_ticks_to_ms_floor32(k_uptime_ticks()); }
+uint32_t micros() { return k_ticks_to_us_floor32(k_uptime_ticks()); }
+void delayMicroseconds(uint32_t us) { ::k_usleep(us); }
+void delay(uint32_t ms) { ::k_msleep(ms); }
+
+void arch_init() {
+  if (device_is_ready(WDT)) {
+    static wdt_timeout_cfg wdt_config{};
+    wdt_config.flags = WDT_FLAG_RESET_SOC;
+    wdt_config.window.max = 2000;
+    wdt_channel_id = wdt_install_timeout(WDT, &wdt_config);
+    if (wdt_channel_id >= 0) {
+      wdt_setup(WDT, WDT_OPT_PAUSE_HALTED_BY_DBG | WDT_OPT_PAUSE_IN_SLEEP);
+    }
+  }
+}
+
+void arch_feed_wdt() {
+  if (wdt_channel_id >= 0) {
+    wdt_feed(WDT, wdt_channel_id);
+  }
+}
+
+void arch_restart() { sys_reboot(SYS_REBOOT_COLD); }
+uint32_t arch_get_cpu_cycle_count() { return k_cycle_get_32(); }
+uint32_t arch_get_cpu_freq_hz() { return sys_clock_hw_cycles_per_sec(); }
+uint8_t progmem_read_byte(const uint8_t *addr) { return *addr; }
+
+Mutex::Mutex() {
+  auto *mutex = new k_mutex();
+  this->handle_ = mutex;
+  k_mutex_init(mutex);
+}
+Mutex::~Mutex() { delete static_cast(this->handle_); }
+void Mutex::lock() { k_mutex_lock(static_cast(this->handle_), K_FOREVER); }
+bool Mutex::try_lock() { return k_mutex_lock(static_cast(this->handle_), K_NO_WAIT) == 0; }
+void Mutex::unlock() { k_mutex_unlock(static_cast(this->handle_)); }
+
+IRAM_ATTR InterruptLock::InterruptLock() { state_ = irq_lock(); }
+IRAM_ATTR InterruptLock::~InterruptLock() { irq_unlock(state_); }
+
+// Zephyr doesn't support lwIP core locking, so this is a no-op
+LwIPLock::LwIPLock() {}
+LwIPLock::~LwIPLock() {}
+
+uint32_t random_uint32() { return rand(); }  // NOLINT(cert-msc30-c, cert-msc50-cpp)
+bool random_bytes(uint8_t *data, size_t len) {
+  sys_rand_get(data, len);
+  return true;
+}
+
+void get_mac_address_raw(uint8_t *mac) {  // NOLINT(readability-non-const-parameter)
+  mac[0] = ((NRF_FICR->DEVICEADDR[1] & 0xFFFF) >> 8) | 0xC0;
+  mac[1] = NRF_FICR->DEVICEADDR[1] & 0xFFFF;
+  mac[2] = NRF_FICR->DEVICEADDR[0] >> 24;
+  mac[3] = NRF_FICR->DEVICEADDR[0] >> 16;
+  mac[4] = NRF_FICR->DEVICEADDR[0] >> 8;
+  mac[5] = NRF_FICR->DEVICEADDR[0];
+}
+
+}  // namespace esphome
+
+void setup();
+void loop();
+
+int main() {
+  setup();
+  while (true) {
+    loop();
+    esphome::yield();
+  }
+  return 0;
+}
+
+#endif
diff --git a/esphome/components/zephyr/gpio.cpp b/esphome/components/zephyr/gpio.cpp
new file mode 100644
index 0000000000..4b84910368
--- /dev/null
+++ b/esphome/components/zephyr/gpio.cpp
@@ -0,0 +1,120 @@
+#ifdef USE_ZEPHYR
+#include "gpio.h"
+#include 
+#include "esphome/core/log.h"
+
+namespace esphome {
+namespace zephyr {
+
+static const char *const TAG = "zephyr";
+
+static int flags_to_mode(gpio::Flags flags, bool inverted, bool value) {
+  int ret = 0;
+  if (flags & gpio::FLAG_INPUT) {
+    ret |= GPIO_INPUT;
+  }
+  if (flags & gpio::FLAG_OUTPUT) {
+    ret |= GPIO_OUTPUT;
+    if (value != inverted) {
+      ret |= GPIO_OUTPUT_INIT_HIGH;
+    } else {
+      ret |= GPIO_OUTPUT_INIT_LOW;
+    }
+  }
+  if (flags & gpio::FLAG_PULLUP) {
+    ret |= GPIO_PULL_UP;
+  }
+  if (flags & gpio::FLAG_PULLDOWN) {
+    ret |= GPIO_PULL_DOWN;
+  }
+  if (flags & gpio::FLAG_OPEN_DRAIN) {
+    ret |= GPIO_OPEN_DRAIN;
+  }
+  return ret;
+}
+
+struct ISRPinArg {
+  uint8_t pin;
+  bool inverted;
+};
+
+ISRInternalGPIOPin ZephyrGPIOPin::to_isr() const {
+  auto *arg = new ISRPinArg{};  // NOLINT(cppcoreguidelines-owning-memory)
+  arg->pin = this->pin_;
+  arg->inverted = this->inverted_;
+  return ISRInternalGPIOPin((void *) arg);
+}
+
+void ZephyrGPIOPin::attach_interrupt(void (*func)(void *), void *arg, gpio::InterruptType type) const {
+  // TODO
+}
+
+void ZephyrGPIOPin::setup() {
+  const struct device *gpio = nullptr;
+  if (this->pin_ < 32) {
+#define GPIO0 DT_NODELABEL(gpio0)
+#if DT_NODE_HAS_STATUS(GPIO0, okay)
+    gpio = DEVICE_DT_GET(GPIO0);
+#else
+#error "gpio0 is disabled"
+#endif
+  } else {
+#define GPIO1 DT_NODELABEL(gpio1)
+#if DT_NODE_HAS_STATUS(GPIO1, okay)
+    gpio = DEVICE_DT_GET(GPIO1);
+#else
+#error "gpio1 is disabled"
+#endif
+  }
+  if (device_is_ready(gpio)) {
+    this->gpio_ = gpio;
+  } else {
+    ESP_LOGE(TAG, "gpio %u is not ready.", this->pin_);
+    return;
+  }
+  this->pin_mode(this->flags_);
+}
+
+void ZephyrGPIOPin::pin_mode(gpio::Flags flags) {
+  if (nullptr == this->gpio_) {
+    return;
+  }
+  gpio_pin_configure(this->gpio_, this->pin_ % 32, flags_to_mode(flags, this->inverted_, this->value_));
+}
+
+std::string ZephyrGPIOPin::dump_summary() const {
+  char buffer[32];
+  snprintf(buffer, sizeof(buffer), "GPIO%u, P%u.%u", this->pin_, this->pin_ / 32, this->pin_ % 32);
+  return buffer;
+}
+
+bool ZephyrGPIOPin::digital_read() {
+  if (nullptr == this->gpio_) {
+    return false;
+  }
+  return bool(gpio_pin_get(this->gpio_, this->pin_ % 32) != this->inverted_);
+}
+
+void ZephyrGPIOPin::digital_write(bool value) {
+  // make sure that value is not ignored since it can be inverted e.g. on switch side
+  // that way init state should be correct
+  this->value_ = value;
+  if (nullptr == this->gpio_) {
+    return;
+  }
+  gpio_pin_set(this->gpio_, this->pin_ % 32, value != this->inverted_ ? 1 : 0);
+}
+void ZephyrGPIOPin::detach_interrupt() const {
+  // TODO
+}
+
+}  // namespace zephyr
+
+bool IRAM_ATTR ISRInternalGPIOPin::digital_read() {
+  // TODO
+  return false;
+}
+
+}  // namespace esphome
+
+#endif
diff --git a/esphome/components/zephyr/gpio.h b/esphome/components/zephyr/gpio.h
new file mode 100644
index 0000000000..f512ae4648
--- /dev/null
+++ b/esphome/components/zephyr/gpio.h
@@ -0,0 +1,38 @@
+#pragma once
+
+#ifdef USE_ZEPHYR
+#include "esphome/core/hal.h"
+struct device;
+namespace esphome {
+namespace zephyr {
+
+class ZephyrGPIOPin : public InternalGPIOPin {
+ public:
+  void set_pin(uint8_t pin) { this->pin_ = pin; }
+  void set_inverted(bool inverted) { this->inverted_ = inverted; }
+  void set_flags(gpio::Flags flags) { this->flags_ = flags; }
+
+  void setup() override;
+  void pin_mode(gpio::Flags flags) override;
+  bool digital_read() override;
+  void digital_write(bool value) override;
+  std::string dump_summary() const override;
+  void detach_interrupt() const override;
+  ISRInternalGPIOPin to_isr() const override;
+  uint8_t get_pin() const override { return this->pin_; }
+  bool is_inverted() const override { return this->inverted_; }
+  gpio::Flags get_flags() const override { return flags_; }
+
+ protected:
+  void attach_interrupt(void (*func)(void *), void *arg, gpio::InterruptType type) const override;
+  uint8_t pin_;
+  bool inverted_;
+  gpio::Flags flags_;
+  const device *gpio_ = nullptr;
+  bool value_ = false;
+};
+
+}  // namespace zephyr
+}  // namespace esphome
+
+#endif  // USE_ZEPHYR
diff --git a/esphome/components/zephyr/pre_build.py.script b/esphome/components/zephyr/pre_build.py.script
new file mode 100644
index 0000000000..3731fccf53
--- /dev/null
+++ b/esphome/components/zephyr/pre_build.py.script
@@ -0,0 +1,4 @@
+Import("env")
+
+board_config = env.BoardConfig()
+board_config.update("frameworks", ["arduino", "zephyr"])
diff --git a/esphome/components/zephyr/preferences.cpp b/esphome/components/zephyr/preferences.cpp
new file mode 100644
index 0000000000..d702366044
--- /dev/null
+++ b/esphome/components/zephyr/preferences.cpp
@@ -0,0 +1,156 @@
+#ifdef USE_ZEPHYR
+
+#include 
+#include "esphome/core/preferences.h"
+#include "esphome/core/log.h"
+#include 
+
+namespace esphome {
+namespace zephyr {
+
+static const char *const TAG = "zephyr.preferences";
+
+#define ESPHOME_SETTINGS_KEY "esphome"
+
+class ZephyrPreferenceBackend : public ESPPreferenceBackend {
+ public:
+  ZephyrPreferenceBackend(uint32_t type) { this->type_ = type; }
+  ZephyrPreferenceBackend(uint32_t type, std::vector &&data) : data(std::move(data)) { this->type_ = type; }
+
+  bool save(const uint8_t *data, size_t len) override {
+    this->data.resize(len);
+    std::memcpy(this->data.data(), data, len);
+    ESP_LOGVV(TAG, "save key: %u, len: %d", this->type_, len);
+    return true;
+  }
+
+  bool load(uint8_t *data, size_t len) override {
+    if (len != this->data.size()) {
+      ESP_LOGE(TAG, "size of setting key %s changed, from: %u, to: %u", get_key().c_str(), this->data.size(), len);
+      return false;
+    }
+    std::memcpy(data, this->data.data(), len);
+    ESP_LOGVV(TAG, "load key: %u, len: %d", this->type_, len);
+    return true;
+  }
+
+  uint32_t get_type() const { return this->type_; }
+  std::string get_key() const { return str_sprintf(ESPHOME_SETTINGS_KEY "/%" PRIx32, this->type_); }
+
+  std::vector data;
+
+ protected:
+  uint32_t type_ = 0;
+};
+
+class ZephyrPreferences : public ESPPreferences {
+ public:
+  void open() {
+    int err = settings_subsys_init();
+    if (err) {
+      ESP_LOGE(TAG, "Failed to initialize settings subsystem, err: %d", err);
+      return;
+    }
+
+    static struct settings_handler settings_cb = {
+        .name = ESPHOME_SETTINGS_KEY,
+        .h_set = load_setting,
+        .h_export = export_settings,
+    };
+
+    err = settings_register(&settings_cb);
+    if (err) {
+      ESP_LOGE(TAG, "setting_register failed, err, %d", err);
+      return;
+    }
+
+    err = settings_load_subtree(ESPHOME_SETTINGS_KEY);
+    if (err) {
+      ESP_LOGE(TAG, "Cannot load settings, err: %d", err);
+      return;
+    }
+    ESP_LOGD(TAG, "Loaded %u settings.", this->backends_.size());
+  }
+
+  ESPPreferenceObject make_preference(size_t length, uint32_t type, bool in_flash) override {
+    return make_preference(length, type);
+  }
+
+  ESPPreferenceObject make_preference(size_t length, uint32_t type) override {
+    for (auto *backend : this->backends_) {
+      if (backend->get_type() == type) {
+        return ESPPreferenceObject(backend);
+      }
+    }
+    printf("type %u size %u\n", type, this->backends_.size());
+    auto *pref = new ZephyrPreferenceBackend(type);  // NOLINT(cppcoreguidelines-owning-memory)
+    ESP_LOGD(TAG, "Add new setting %s.", pref->get_key().c_str());
+    this->backends_.push_back(pref);
+    return ESPPreferenceObject(pref);
+  }
+
+  bool sync() override {
+    ESP_LOGD(TAG, "Save settings");
+    int err = settings_save();
+    if (err) {
+      ESP_LOGE(TAG, "Cannot save settings, err: %d", err);
+      return false;
+    }
+    return true;
+  }
+
+  bool reset() override {
+    ESP_LOGD(TAG, "Reset settings");
+    for (auto *backend : this->backends_) {
+      // save empty delete data
+      backend->data.clear();
+    }
+    sync();
+    return true;
+  }
+
+ protected:
+  std::vector backends_;
+
+  static int load_setting(const char *name, size_t len, settings_read_cb read_cb, void *cb_arg) {
+    auto type = parse_hex(name);
+    if (!type.has_value()) {
+      std::string full_name(ESPHOME_SETTINGS_KEY);
+      full_name += "/";
+      full_name += name;
+      // Delete unusable keys. Otherwise it will stay in flash forever.
+      settings_delete(full_name.c_str());
+      return 1;
+    }
+    std::vector data(len);
+    int err = read_cb(cb_arg, data.data(), len);
+
+    ESP_LOGD(TAG, "load setting, name: %s(%u), len %u, err %u", name, *type, len, err);
+    auto *pref = new ZephyrPreferenceBackend(*type, std::move(data));  // NOLINT(cppcoreguidelines-owning-memory)
+    static_cast(global_preferences)->backends_.push_back(pref);
+    return 0;
+  }
+
+  static int export_settings(int (*cb)(const char *name, const void *value, size_t val_len)) {
+    for (auto *backend : static_cast(global_preferences)->backends_) {
+      auto name = backend->get_key();
+      int err = cb(name.c_str(), backend->data.data(), backend->data.size());
+      ESP_LOGD(TAG, "save in flash, name %s, len %u, err %d", name.c_str(), backend->data.size(), err);
+    }
+    return 0;
+  }
+};
+
+void setup_preferences() {
+  auto *prefs = new ZephyrPreferences();  // NOLINT(cppcoreguidelines-owning-memory)
+  global_preferences = prefs;
+  prefs->open();
+}
+
+}  // namespace zephyr
+
+ESPPreferences *global_preferences;  // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
+
+}  // namespace esphome
+
+#endif
diff --git a/esphome/components/zephyr/preferences.h b/esphome/components/zephyr/preferences.h
new file mode 100644
index 0000000000..6a37e41b46
--- /dev/null
+++ b/esphome/components/zephyr/preferences.h
@@ -0,0 +1,13 @@
+#pragma once
+
+#ifdef USE_ZEPHYR
+
+namespace esphome {
+namespace zephyr {
+
+void setup_preferences();
+
+}  // namespace zephyr
+}  // namespace esphome
+
+#endif
diff --git a/esphome/components/zephyr_ble_server/__init__.py b/esphome/components/zephyr_ble_server/__init__.py
new file mode 100644
index 0000000000..211941e984
--- /dev/null
+++ b/esphome/components/zephyr_ble_server/__init__.py
@@ -0,0 +1,34 @@
+import esphome.codegen as cg
+from esphome.components.zephyr import zephyr_add_prj_conf
+import esphome.config_validation as cv
+from esphome.const import CONF_ESPHOME, CONF_ID, CONF_NAME, Framework
+import esphome.final_validate as fv
+
+zephyr_ble_server_ns = cg.esphome_ns.namespace("zephyr_ble_server")
+BLEServer = zephyr_ble_server_ns.class_("BLEServer", cg.Component)
+
+CONFIG_SCHEMA = cv.All(
+    cv.Schema(
+        {
+            cv.GenerateID(): cv.declare_id(BLEServer),
+        }
+    ).extend(cv.COMPONENT_SCHEMA),
+    cv.only_with_framework(Framework.ZEPHYR),
+)
+
+
+def _final_validate(_):
+    full_config = fv.full_config.get()
+    zephyr_add_prj_conf("BT_DEVICE_NAME", full_config[CONF_ESPHOME][CONF_NAME])
+
+
+FINAL_VALIDATE_SCHEMA = _final_validate
+
+
+async def to_code(config):
+    var = cg.new_Pvariable(config[CONF_ID])
+    zephyr_add_prj_conf("BT", True)
+    zephyr_add_prj_conf("BT_PERIPHERAL", True)
+    zephyr_add_prj_conf("BT_RX_STACK_SIZE", 1536)
+    # zephyr_add_prj_conf("BT_LL_SW_SPLIT", True)
+    await cg.register_component(var, config)
diff --git a/esphome/components/zephyr_ble_server/ble_server.cpp b/esphome/components/zephyr_ble_server/ble_server.cpp
new file mode 100644
index 0000000000..9f7e606a90
--- /dev/null
+++ b/esphome/components/zephyr_ble_server/ble_server.cpp
@@ -0,0 +1,100 @@
+#ifdef USE_ZEPHYR
+#include "ble_server.h"
+#include "esphome/core/defines.h"
+#include "esphome/core/log.h"
+#include 
+#include 
+
+namespace esphome::zephyr_ble_server {
+
+static const char *const TAG = "zephyr_ble_server";
+
+static struct k_work advertise_work;  // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
+
+#define DEVICE_NAME CONFIG_BT_DEVICE_NAME
+#define DEVICE_NAME_LEN (sizeof(DEVICE_NAME) - 1)
+
+static const struct bt_data AD[] = {
+    BT_DATA_BYTES(BT_DATA_FLAGS, (BT_LE_AD_GENERAL | BT_LE_AD_NO_BREDR)),
+    BT_DATA(BT_DATA_NAME_COMPLETE, DEVICE_NAME, DEVICE_NAME_LEN),
+};
+
+static const struct bt_data SD[] = {
+#ifdef USE_OTA
+    BT_DATA_BYTES(BT_DATA_UUID128_ALL, 0x84, 0xaa, 0x60, 0x74, 0x52, 0x8a, 0x8b, 0x86, 0xd3, 0x4c, 0xb7, 0x1d, 0x1d,
+                  0xdc, 0x53, 0x8d),
+#endif
+};
+
+const struct bt_le_adv_param *const ADV_PARAM = BT_LE_ADV_CONN;
+
+static void advertise(struct k_work *work) {
+  int rc = bt_le_adv_stop();
+  if (rc) {
+    ESP_LOGE(TAG, "Advertising failed to stop (rc %d)", rc);
+  }
+
+  rc = bt_le_adv_start(ADV_PARAM, AD, ARRAY_SIZE(AD), SD, ARRAY_SIZE(SD));
+  if (rc) {
+    ESP_LOGE(TAG, "Advertising failed to start (rc %d)", rc);
+    return;
+  }
+  ESP_LOGI(TAG, "Advertising successfully started");
+}
+
+static void connected(struct bt_conn *conn, uint8_t err) {
+  if (err) {
+    ESP_LOGE(TAG, "Connection failed (err 0x%02x)", err);
+  } else {
+    ESP_LOGI(TAG, "Connected");
+  }
+}
+
+static void disconnected(struct bt_conn *conn, uint8_t reason) {
+  ESP_LOGI(TAG, "Disconnected (reason 0x%02x)", reason);
+  k_work_submit(&advertise_work);
+}
+
+static void bt_ready(int err) {
+  if (err != 0) {
+    ESP_LOGE(TAG, "Bluetooth failed to initialise: %d", err);
+  } else {
+    k_work_submit(&advertise_work);
+  }
+}
+
+BT_CONN_CB_DEFINE(conn_callbacks) = {
+    .connected = connected,
+    .disconnected = disconnected,
+};
+
+void BLEServer::setup() {
+  k_work_init(&advertise_work, advertise);
+  resume_();
+}
+
+void BLEServer::loop() {
+  if (this->suspended_) {
+    resume_();
+    this->suspended_ = false;
+  }
+}
+
+void BLEServer::resume_() {
+  int rc = bt_enable(bt_ready);
+  if (rc != 0) {
+    ESP_LOGE(TAG, "Bluetooth enable failed: %d", rc);
+    return;
+  }
+}
+
+void BLEServer::on_shutdown() {
+  struct k_work_sync sync;
+  k_work_cancel_sync(&advertise_work, &sync);
+  bt_disable();
+  this->suspended_ = true;
+}
+
+}  // namespace esphome::zephyr_ble_server
+
+#endif
diff --git a/esphome/components/zephyr_ble_server/ble_server.h b/esphome/components/zephyr_ble_server/ble_server.h
new file mode 100644
index 0000000000..1b32e9b58c
--- /dev/null
+++ b/esphome/components/zephyr_ble_server/ble_server.h
@@ -0,0 +1,19 @@
+#pragma once
+#ifdef USE_ZEPHYR
+#include "esphome/core/component.h"
+
+namespace esphome::zephyr_ble_server {
+
+class BLEServer : public Component {
+ public:
+  void setup() override;
+  void loop() override;
+  void on_shutdown() override;
+
+ protected:
+  void resume_();
+  bool suspended_ = false;
+};
+
+}  // namespace esphome::zephyr_ble_server
+#endif
diff --git a/esphome/components/zwave_proxy/__init__.py b/esphome/components/zwave_proxy/__init__.py
new file mode 100644
index 0000000000..d88f9f7041
--- /dev/null
+++ b/esphome/components/zwave_proxy/__init__.py
@@ -0,0 +1,43 @@
+import esphome.codegen as cg
+from esphome.components import uart
+import esphome.config_validation as cv
+from esphome.const import CONF_ID, CONF_POWER_SAVE_MODE, CONF_WIFI
+import esphome.final_validate as fv
+
+CODEOWNERS = ["@kbx81"]
+DEPENDENCIES = ["api", "uart"]
+
+zwave_proxy_ns = cg.esphome_ns.namespace("zwave_proxy")
+ZWaveProxy = zwave_proxy_ns.class_("ZWaveProxy", cg.Component, uart.UARTDevice)
+
+
+def final_validate(config):
+    full_config = fv.full_config.get()
+    if (wifi_conf := full_config.get(CONF_WIFI)) and (
+        wifi_conf.get(CONF_POWER_SAVE_MODE).lower() != "none"
+    ):
+        raise cv.Invalid(
+            f"{CONF_WIFI} {CONF_POWER_SAVE_MODE} must be set to 'none' when using Z-Wave proxy"
+        )
+
+    return config
+
+
+CONFIG_SCHEMA = (
+    cv.Schema(
+        {
+            cv.GenerateID(): cv.declare_id(ZWaveProxy),
+        }
+    )
+    .extend(cv.COMPONENT_SCHEMA)
+    .extend(uart.UART_DEVICE_SCHEMA)
+)
+
+FINAL_VALIDATE_SCHEMA = final_validate
+
+
+async def to_code(config):
+    var = cg.new_Pvariable(config[CONF_ID])
+    await cg.register_component(var, config)
+    await uart.register_uart_device(var, config)
+    cg.add_define("USE_ZWAVE_PROXY")
diff --git a/esphome/components/zwave_proxy/zwave_proxy.cpp b/esphome/components/zwave_proxy/zwave_proxy.cpp
new file mode 100644
index 0000000000..a26a9b2335
--- /dev/null
+++ b/esphome/components/zwave_proxy/zwave_proxy.cpp
@@ -0,0 +1,346 @@
+#include "zwave_proxy.h"
+#include "esphome/components/api/api_server.h"
+#include "esphome/core/application.h"
+#include "esphome/core/helpers.h"
+#include "esphome/core/log.h"
+#include "esphome/core/util.h"
+
+namespace esphome {
+namespace zwave_proxy {
+
+static const char *const TAG = "zwave_proxy";
+
+static constexpr uint8_t ZWAVE_COMMAND_GET_NETWORK_IDS = 0x20;
+// GET_NETWORK_IDS response: [SOF][LENGTH][TYPE][CMD][HOME_ID(4)][NODE_ID][...]
+static constexpr uint8_t ZWAVE_COMMAND_TYPE_RESPONSE = 0x01;    // Response type field value
+static constexpr uint8_t ZWAVE_MIN_GET_NETWORK_IDS_LENGTH = 9;  // TYPE + CMD + HOME_ID(4) + NODE_ID + checksum
+static constexpr uint32_t HOME_ID_TIMEOUT_MS = 100;             // Timeout for waiting for home ID during setup
+
+static uint8_t calculate_frame_checksum(const uint8_t *data, uint8_t length) {
+  // Calculate Z-Wave frame checksum
+  // XOR all bytes between SOF and checksum position (exclusive)
+  // Initial value is 0xFF per Z-Wave protocol specification
+  uint8_t checksum = 0xFF;
+  for (uint8_t i = 1; i < length - 1; i++) {
+    checksum ^= data[i];
+  }
+  return checksum;
+}
+
+ZWaveProxy::ZWaveProxy() { global_zwave_proxy = this; }
+
+void ZWaveProxy::setup() {
+  this->setup_time_ = App.get_loop_component_start_time();
+  this->send_simple_command_(ZWAVE_COMMAND_GET_NETWORK_IDS);
+}
+
+float ZWaveProxy::get_setup_priority() const {
+  // Set up before API so home ID is ready when API starts
+  return setup_priority::BEFORE_CONNECTION;
+}
+
+bool ZWaveProxy::can_proceed() {
+  // If we already have the home ID, we can proceed
+  if (this->home_id_ready_) {
+    return true;
+  }
+
+  // Handle any pending responses
+  if (this->response_handler_()) {
+    ESP_LOGV(TAG, "Handled response during setup");
+  }
+
+  // Process UART data to check for home ID
+  this->process_uart_();
+
+  // Check if we got the home ID after processing
+  if (this->home_id_ready_) {
+    return true;
+  }
+
+  // Wait up to HOME_ID_TIMEOUT_MS for home ID response
+  const uint32_t now = App.get_loop_component_start_time();
+  if (now - this->setup_time_ > HOME_ID_TIMEOUT_MS) {
+    ESP_LOGW(TAG, "Timeout reading Home ID during setup");
+    return true;  // Proceed anyway after timeout
+  }
+
+  return false;  // Keep waiting
+}
+
+void ZWaveProxy::loop() {
+  if (this->response_handler_()) {
+    ESP_LOGV(TAG, "Handled late response");
+  }
+  if (this->api_connection_ != nullptr && (!this->api_connection_->is_connection_setup() || !api_is_connected())) {
+    ESP_LOGW(TAG, "Subscriber disconnected");
+    this->api_connection_ = nullptr;  // Unsubscribe if disconnected
+  }
+
+  this->process_uart_();
+  this->status_clear_warning();
+}
+
+void ZWaveProxy::process_uart_() {
+  while (this->available()) {
+    uint8_t byte;
+    if (!this->read_byte(&byte)) {
+      this->status_set_warning("UART read failed");
+      return;
+    }
+    if (this->parse_byte_(byte)) {
+      // Check if this is a GET_NETWORK_IDS response frame
+      // Frame format: [SOF][LENGTH][TYPE][CMD][HOME_ID(4)][NODE_ID][...]
+      // We verify:
+      // - buffer_[0]: Start of frame marker (0x01)
+      // - buffer_[1]: Length field must be >= 9 to contain all required data
+      // - buffer_[2]: Command type (0x01 for response)
+      // - buffer_[3]: Command ID (0x20 for GET_NETWORK_IDS)
+      if (this->buffer_[3] == ZWAVE_COMMAND_GET_NETWORK_IDS && this->buffer_[2] == ZWAVE_COMMAND_TYPE_RESPONSE &&
+          this->buffer_[1] >= ZWAVE_MIN_GET_NETWORK_IDS_LENGTH && this->buffer_[0] == ZWAVE_FRAME_TYPE_START) {
+        // Store the 4-byte Home ID, which starts at offset 4, and notify connected clients if it changed
+        // The frame parser has already validated the checksum and ensured all bytes are present
+        if (this->set_home_id(&this->buffer_[4])) {
+          this->send_homeid_changed_msg_();
+        }
+      }
+      ESP_LOGV(TAG, "Sending to client: %s", YESNO(this->api_connection_ != nullptr));
+      if (this->api_connection_ != nullptr) {
+        // Zero-copy: point directly to our buffer
+        this->outgoing_proto_msg_.data = this->buffer_.data();
+        if (this->in_bootloader_) {
+          this->outgoing_proto_msg_.data_len = this->buffer_index_;
+        } else {
+          // If this is a data frame, use frame length indicator + 2 (for SoF + checksum), else assume 1 for ACK/NAK/CAN
+          this->outgoing_proto_msg_.data_len = this->buffer_[0] == ZWAVE_FRAME_TYPE_START ? this->buffer_[1] + 2 : 1;
+        }
+        this->api_connection_->send_message(this->outgoing_proto_msg_, api::ZWaveProxyFrame::MESSAGE_TYPE);
+      }
+    }
+  }
+}
+
+void ZWaveProxy::dump_config() {
+  ESP_LOGCONFIG(TAG,
+                "Z-Wave Proxy:\n"
+                "  Home ID: %s",
+                format_hex_pretty(this->home_id_.data(), this->home_id_.size(), ':', false).c_str());
+}
+
+void ZWaveProxy::api_connection_authenticated(api::APIConnection *conn) {
+  if (this->home_id_ready_) {
+    // If a client just authenticated & HomeID is ready, send the current HomeID
+    this->send_homeid_changed_msg_(conn);
+  }
+}
+
+void ZWaveProxy::zwave_proxy_request(api::APIConnection *api_connection, api::enums::ZWaveProxyRequestType type) {
+  switch (type) {
+    case api::enums::ZWAVE_PROXY_REQUEST_TYPE_SUBSCRIBE:
+      if (this->api_connection_ != nullptr) {
+        ESP_LOGE(TAG, "Only one API subscription is allowed at a time");
+        return;
+      }
+      this->api_connection_ = api_connection;
+      ESP_LOGV(TAG, "API connection is now subscribed");
+      break;
+    case api::enums::ZWAVE_PROXY_REQUEST_TYPE_UNSUBSCRIBE:
+      if (this->api_connection_ != api_connection) {
+        ESP_LOGV(TAG, "API connection is not subscribed");
+        return;
+      }
+      this->api_connection_ = nullptr;
+      break;
+    default:
+      ESP_LOGW(TAG, "Unknown request type: %d", type);
+      break;
+  }
+}
+
+bool ZWaveProxy::set_home_id(const uint8_t *new_home_id) {
+  if (std::memcmp(this->home_id_.data(), new_home_id, this->home_id_.size()) == 0) {
+    ESP_LOGV(TAG, "Home ID unchanged");
+    return false;  // No change
+  }
+  std::memcpy(this->home_id_.data(), new_home_id, this->home_id_.size());
+  ESP_LOGI(TAG, "Home ID: %s", format_hex_pretty(this->home_id_.data(), this->home_id_.size(), ':', false).c_str());
+  this->home_id_ready_ = true;
+  return true;  // Home ID was changed
+}
+
+void ZWaveProxy::send_frame(const uint8_t *data, size_t length) {
+  if (length == 1 && data[0] == this->last_response_) {
+    ESP_LOGV(TAG, "Skipping sending duplicate response: 0x%02X", data[0]);
+    return;
+  }
+  ESP_LOGVV(TAG, "Sending: %s", format_hex_pretty(data, length).c_str());
+  this->write_array(data, length);
+}
+
+void ZWaveProxy::send_homeid_changed_msg_(api::APIConnection *conn) {
+  api::ZWaveProxyRequest msg;
+  msg.type = api::enums::ZWAVE_PROXY_REQUEST_TYPE_HOME_ID_CHANGE;
+  msg.data = this->home_id_.data();
+  msg.data_len = this->home_id_.size();
+  if (conn != nullptr) {
+    // Send to specific connection
+    conn->send_message(msg, api::ZWaveProxyRequest::MESSAGE_TYPE);
+  } else if (api::global_api_server != nullptr) {
+    // We could add code to manage a second subscription type, but, since this message is
+    //  very infrequent and small, we simply send it to all clients
+    api::global_api_server->on_zwave_proxy_request(msg);
+  }
+}
+
+void ZWaveProxy::send_simple_command_(const uint8_t command_id) {
+  // Send a simple Z-Wave command with no parameters
+  // Frame format: [SOF][LENGTH][TYPE][CMD][CHECKSUM]
+  // Where LENGTH=0x03 (3 bytes: TYPE + CMD + CHECKSUM)
+  uint8_t cmd[] = {0x01, 0x03, 0x00, command_id, 0x00};
+  cmd[4] = calculate_frame_checksum(cmd, sizeof(cmd));
+  this->send_frame(cmd, sizeof(cmd));
+}
+
+bool ZWaveProxy::parse_byte_(uint8_t byte) {
+  bool frame_completed = false;
+  // Basic parsing logic for received frames
+  switch (this->parsing_state_) {
+    case ZWAVE_PARSING_STATE_WAIT_START:
+      this->parse_start_(byte);
+      break;
+    case ZWAVE_PARSING_STATE_WAIT_LENGTH:
+      if (!byte) {
+        ESP_LOGW(TAG, "Invalid LENGTH: %u", byte);
+        this->parsing_state_ = ZWAVE_PARSING_STATE_SEND_NAK;
+        return false;
+      }
+      ESP_LOGVV(TAG, "Received LENGTH: %u", byte);
+      this->end_frame_after_ = this->buffer_index_ + byte;
+      ESP_LOGVV(TAG, "Calculated EOF: %u", this->end_frame_after_);
+      this->buffer_[this->buffer_index_++] = byte;
+      this->parsing_state_ = ZWAVE_PARSING_STATE_WAIT_TYPE;
+      break;
+    case ZWAVE_PARSING_STATE_WAIT_TYPE:
+      this->buffer_[this->buffer_index_++] = byte;
+      ESP_LOGVV(TAG, "Received TYPE: 0x%02X", byte);
+      this->parsing_state_ = ZWAVE_PARSING_STATE_WAIT_COMMAND_ID;
+      break;
+    case ZWAVE_PARSING_STATE_WAIT_COMMAND_ID:
+      this->buffer_[this->buffer_index_++] = byte;
+      ESP_LOGVV(TAG, "Received COMMAND ID: 0x%02X", byte);
+      this->parsing_state_ = ZWAVE_PARSING_STATE_WAIT_PAYLOAD;
+      break;
+    case ZWAVE_PARSING_STATE_WAIT_PAYLOAD:
+      this->buffer_[this->buffer_index_++] = byte;
+      ESP_LOGVV(TAG, "Received PAYLOAD: 0x%02X", byte);
+      if (this->buffer_index_ >= this->end_frame_after_) {
+        this->parsing_state_ = ZWAVE_PARSING_STATE_WAIT_CHECKSUM;
+      }
+      break;
+    case ZWAVE_PARSING_STATE_WAIT_CHECKSUM: {
+      this->buffer_[this->buffer_index_++] = byte;
+      auto checksum = calculate_frame_checksum(this->buffer_.data(), this->buffer_index_);
+      ESP_LOGVV(TAG, "CHECKSUM Received: 0x%02X - Calculated: 0x%02X", byte, checksum);
+      if (checksum != byte) {
+        ESP_LOGW(TAG, "Bad checksum: expected 0x%02X, got 0x%02X", checksum, byte);
+        this->parsing_state_ = ZWAVE_PARSING_STATE_SEND_NAK;
+      } else {
+        this->parsing_state_ = ZWAVE_PARSING_STATE_SEND_ACK;
+        ESP_LOGVV(TAG, "Received frame: %s", format_hex_pretty(this->buffer_.data(), this->buffer_index_).c_str());
+        frame_completed = true;
+      }
+      this->response_handler_();
+      break;
+    }
+    case ZWAVE_PARSING_STATE_READ_BL_MENU:
+      this->buffer_[this->buffer_index_++] = byte;
+      if (!byte) {
+        this->parsing_state_ = ZWAVE_PARSING_STATE_WAIT_START;
+        frame_completed = true;
+      }
+      break;
+    case ZWAVE_PARSING_STATE_SEND_ACK:
+    case ZWAVE_PARSING_STATE_SEND_NAK:
+      break;  // Should not happen, handled in loop()
+    default:
+      ESP_LOGW(TAG, "Bad parsing state; resetting");
+      this->parsing_state_ = ZWAVE_PARSING_STATE_WAIT_START;
+      break;
+  }
+  return frame_completed;
+}
+
+void ZWaveProxy::parse_start_(uint8_t byte) {
+  this->buffer_index_ = 0;
+  this->parsing_state_ = ZWAVE_PARSING_STATE_WAIT_START;
+  switch (byte) {
+    case ZWAVE_FRAME_TYPE_START:
+      ESP_LOGVV(TAG, "Received START");
+      if (this->in_bootloader_) {
+        ESP_LOGD(TAG, "Exited bootloader mode");
+        this->in_bootloader_ = false;
+      }
+      this->buffer_[this->buffer_index_++] = byte;
+      this->parsing_state_ = ZWAVE_PARSING_STATE_WAIT_LENGTH;
+      return;
+    case ZWAVE_FRAME_TYPE_BL_MENU:
+      ESP_LOGVV(TAG, "Received BL_MENU");
+      if (!this->in_bootloader_) {
+        ESP_LOGD(TAG, "Entered bootloader mode");
+        this->in_bootloader_ = true;
+      }
+      this->buffer_[this->buffer_index_++] = byte;
+      this->parsing_state_ = ZWAVE_PARSING_STATE_READ_BL_MENU;
+      return;
+    case ZWAVE_FRAME_TYPE_BL_BEGIN_UPLOAD:
+      ESP_LOGVV(TAG, "Received BL_BEGIN_UPLOAD");
+      break;
+    case ZWAVE_FRAME_TYPE_ACK:
+      ESP_LOGVV(TAG, "Received ACK");
+      break;
+    case ZWAVE_FRAME_TYPE_NAK:
+      ESP_LOGW(TAG, "Received NAK");
+      break;
+    case ZWAVE_FRAME_TYPE_CAN:
+      ESP_LOGW(TAG, "Received CAN");
+      break;
+    default:
+      ESP_LOGW(TAG, "Unrecognized START: 0x%02X", byte);
+      return;
+  }
+  // Forward response (ACK/NAK/CAN) back to client for processing
+  if (this->api_connection_ != nullptr) {
+    // Store single byte in buffer and point to it
+    this->buffer_[0] = byte;
+    this->outgoing_proto_msg_.data = this->buffer_.data();
+    this->outgoing_proto_msg_.data_len = 1;
+    this->api_connection_->send_message(this->outgoing_proto_msg_, api::ZWaveProxyFrame::MESSAGE_TYPE);
+  }
+}
+
+bool ZWaveProxy::response_handler_() {
+  switch (this->parsing_state_) {
+    case ZWAVE_PARSING_STATE_SEND_ACK:
+      this->last_response_ = ZWAVE_FRAME_TYPE_ACK;
+      break;
+    case ZWAVE_PARSING_STATE_SEND_CAN:
+      this->last_response_ = ZWAVE_FRAME_TYPE_CAN;
+      break;
+    case ZWAVE_PARSING_STATE_SEND_NAK:
+      this->last_response_ = ZWAVE_FRAME_TYPE_NAK;
+      break;
+    default:
+      return false;  // No response handled
+  }
+
+  ESP_LOGVV(TAG, "Sending %s (0x%02X)", this->last_response_ == ZWAVE_FRAME_TYPE_ACK ? "ACK" : "NAK/CAN",
+            this->last_response_);
+  this->write_byte(this->last_response_);
+  this->parsing_state_ = ZWAVE_PARSING_STATE_WAIT_START;
+  return true;
+}
+
+ZWaveProxy *global_zwave_proxy = nullptr;  // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
+
+}  // namespace zwave_proxy
+}  // namespace esphome
diff --git a/esphome/components/zwave_proxy/zwave_proxy.h b/esphome/components/zwave_proxy/zwave_proxy.h
new file mode 100644
index 0000000000..20d9090d98
--- /dev/null
+++ b/esphome/components/zwave_proxy/zwave_proxy.h
@@ -0,0 +1,93 @@
+#pragma once
+
+#include "esphome/components/api/api_connection.h"
+#include "esphome/components/api/api_pb2.h"
+#include "esphome/core/component.h"
+#include "esphome/core/helpers.h"
+#include "esphome/components/uart/uart.h"
+
+#include 
+
+namespace esphome {
+namespace zwave_proxy {
+
+static constexpr size_t MAX_ZWAVE_FRAME_SIZE = 257;  // Maximum Z-Wave frame size
+
+enum ZWaveResponseTypes : uint8_t {
+  ZWAVE_FRAME_TYPE_ACK = 0x06,
+  ZWAVE_FRAME_TYPE_CAN = 0x18,
+  ZWAVE_FRAME_TYPE_NAK = 0x15,
+  ZWAVE_FRAME_TYPE_START = 0x01,
+  ZWAVE_FRAME_TYPE_BL_MENU = 0x0D,
+  ZWAVE_FRAME_TYPE_BL_BEGIN_UPLOAD = 0x43,
+};
+
+enum ZWaveParsingState : uint8_t {
+  ZWAVE_PARSING_STATE_WAIT_START,
+  ZWAVE_PARSING_STATE_WAIT_LENGTH,
+  ZWAVE_PARSING_STATE_WAIT_TYPE,
+  ZWAVE_PARSING_STATE_WAIT_COMMAND_ID,
+  ZWAVE_PARSING_STATE_WAIT_PAYLOAD,
+  ZWAVE_PARSING_STATE_WAIT_CHECKSUM,
+  ZWAVE_PARSING_STATE_SEND_ACK,
+  ZWAVE_PARSING_STATE_SEND_CAN,
+  ZWAVE_PARSING_STATE_SEND_NAK,
+  ZWAVE_PARSING_STATE_READ_BL_MENU,
+};
+
+enum ZWaveProxyFeature : uint32_t {
+  FEATURE_ZWAVE_PROXY_ENABLED = 1 << 0,
+};
+
+class ZWaveProxy : public uart::UARTDevice, public Component {
+ public:
+  ZWaveProxy();
+
+  void setup() override;
+  void loop() override;
+  void dump_config() override;
+  float get_setup_priority() const override;
+  bool can_proceed() override;
+
+  void api_connection_authenticated(api::APIConnection *conn);
+  void zwave_proxy_request(api::APIConnection *api_connection, api::enums::ZWaveProxyRequestType type);
+  api::APIConnection *get_api_connection() { return this->api_connection_; }
+
+  uint32_t get_feature_flags() const { return ZWaveProxyFeature::FEATURE_ZWAVE_PROXY_ENABLED; }
+  uint32_t get_home_id() {
+    return encode_uint32(this->home_id_[0], this->home_id_[1], this->home_id_[2], this->home_id_[3]);
+  }
+  bool set_home_id(const uint8_t *new_home_id);  // Store a new home ID. Returns true if it changed.
+
+  void send_frame(const uint8_t *data, size_t length);
+
+ protected:
+  void send_homeid_changed_msg_(api::APIConnection *conn = nullptr);
+  void send_simple_command_(uint8_t command_id);
+  bool parse_byte_(uint8_t byte);  // Returns true if frame parsing was completed (a frame is ready in the buffer)
+  void parse_start_(uint8_t byte);
+  bool response_handler_();
+  void process_uart_();  // Process all available UART data
+
+  // Pre-allocated message - always ready to send
+  api::ZWaveProxyFrame outgoing_proto_msg_;
+  std::array buffer_;  // Fixed buffer for incoming data
+  std::array home_id_{0, 0, 0, 0};        // Fixed buffer for home ID
+
+  // Pointers and 32-bit values (aligned together)
+  api::APIConnection *api_connection_{nullptr};  // Current subscribed client
+  uint32_t setup_time_{0};                       // Time when setup() was called
+
+  // 8-bit values (grouped together to minimize padding)
+  uint8_t buffer_index_{0};     // Index for populating the data buffer
+  uint8_t end_frame_after_{0};  // Payload reception ends after this index
+  uint8_t last_response_{0};    // Last response type sent
+  ZWaveParsingState parsing_state_{ZWAVE_PARSING_STATE_WAIT_START};
+  bool in_bootloader_{false};  // True if the device is detected to be in bootloader mode
+  bool home_id_ready_{false};  // True when home ID has been received from Z-Wave module
+};
+
+extern ZWaveProxy *global_zwave_proxy;  // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
+
+}  // namespace zwave_proxy
+}  // namespace esphome
diff --git a/esphome/config.py b/esphome/config.py
index c4aa9aea24..6adecb5c65 100644
--- a/esphome/config.py
+++ b/esphome/config.py
@@ -12,7 +12,7 @@ from typing import Any
 import voluptuous as vol
 
 from esphome import core, loader, pins, yaml_util
-from esphome.config_helpers import Extend, Remove
+from esphome.config_helpers import Extend, Remove, merge_dicts_ordered
 import esphome.config_validation as cv
 from esphome.const import (
     CONF_ESPHOME,
@@ -32,7 +32,7 @@ from esphome.log import AnsiFore, color
 from esphome.types import ConfigFragmentType, ConfigType
 from esphome.util import OrderedDict, safe_print
 from esphome.voluptuous_schema import ExtraKeysInvalid
-from esphome.yaml_util import ESPForceValue, ESPHomeDataBase, is_secret
+from esphome.yaml_util import ESPHomeDataBase, ESPLiteralValue, is_secret
 
 _LOGGER = logging.getLogger(__name__)
 
@@ -67,6 +67,31 @@ ConfigPath = list[str | int]
 path_context = contextvars.ContextVar("Config path")
 
 
+def _add_auto_load_steps(result: Config, loads: list[str]) -> None:
+    """Add AutoLoadValidationStep for each component in loads that isn't already loaded."""
+    for load in loads:
+        if load not in result:
+            result.add_validation_step(AutoLoadValidationStep(load))
+
+
+def _process_auto_load(
+    result: Config, platform: ComponentManifest, path: ConfigPath
+) -> None:
+    # Process platform's AUTO_LOAD
+    auto_load = platform.auto_load
+    if isinstance(auto_load, list):
+        _add_auto_load_steps(result, auto_load)
+    elif callable(auto_load):
+        import inspect
+
+        if inspect.signature(auto_load).parameters:
+            result.add_validation_step(
+                AddDynamicAutoLoadsValidationStep(path, platform)
+            )
+        else:
+            _add_auto_load_steps(result, auto_load())
+
+
 def _process_platform_config(
     result: Config,
     component_name: str,
@@ -91,9 +116,7 @@ def _process_platform_config(
     CORE.loaded_platforms.add(f"{component_name}/{platform_name}")
 
     # Process platform's AUTO_LOAD
-    for load in platform.auto_load:
-        if load not in result:
-            result.add_validation_step(AutoLoadValidationStep(load))
+    _process_auto_load(result, platform, path)
 
     # Add validation steps for the platform
     p_domain = f"{component_name}.{platform_name}"
@@ -198,10 +221,7 @@ class Config(OrderedDict, fv.FinalValidateConfig):
         self.output_paths.remove((path, domain))
 
     def is_in_error_path(self, path: ConfigPath) -> bool:
-        for err in self.errors:
-            if _path_begins_with(err.path, path):
-                return True
-        return False
+        return any(_path_begins_with(err.path, path) for err in self.errors)
 
     def set_by_path(self, path, value):
         conf = self
@@ -224,7 +244,7 @@ class Config(OrderedDict, fv.FinalValidateConfig):
         for index, path_item in enumerate(path):
             try:
                 if path_item in data:
-                    key_data = [x for x in data.keys() if x == path_item][0]
+                    key_data = [x for x in data if x == path_item][0]
                     if isinstance(key_data, ESPHomeDataBase):
                         doc_range = key_data.esp_range
                         if get_key and index == len(path) - 1:
@@ -309,7 +329,7 @@ def recursive_check_replaceme(value):
         return cv.Schema([recursive_check_replaceme])(value)
     if isinstance(value, dict):
         return cv.Schema({cv.valid: recursive_check_replaceme})(value)
-    if isinstance(value, ESPForceValue):
+    if isinstance(value, ESPLiteralValue):
         pass
     if isinstance(value, str) and value == "REPLACEME":
         raise cv.Invalid(
@@ -317,7 +337,7 @@ def recursive_check_replaceme(value):
             "Please make sure you have replaced all fields from the sample "
             "configuration.\n"
             "If you want to use the literal REPLACEME string, "
-            'please use "!force REPLACEME"'
+            'please use "!literal REPLACEME"'
         )
     return value
 
@@ -332,6 +352,28 @@ class ConfigValidationStep(abc.ABC):
     def run(self, result: Config) -> None: ...  # noqa: E704
 
 
+class LoadTargetPlatformValidationStep(ConfigValidationStep):
+    """Load target platform step."""
+
+    def __init__(self, domain: str, conf: ConfigType):
+        self.domain = domain
+        self.conf = conf
+
+    def run(self, result: Config) -> None:
+        if self.conf is None:
+            result[self.domain] = self.conf = {}
+        result.add_output_path([self.domain], self.domain)
+        component = get_component(self.domain)
+
+        result[self.domain] = self.conf
+        path = [self.domain]
+        CORE.loaded_integrations.add(self.domain)
+
+        result.add_validation_step(
+            SchemaValidationStep(self.domain, path, self.conf, component)
+        )
+
+
 class LoadValidationStep(ConfigValidationStep):
     """Load step, this step is called once for each domain config fragment.
 
@@ -363,11 +405,15 @@ class LoadValidationStep(ConfigValidationStep):
             result.add_str_error(f"Component not found: {self.domain}", path)
             return
         CORE.loaded_integrations.add(self.domain)
+        # For platform components, normalize conf before creating MetadataValidationStep
+        if component.is_platform_component:
+            if not self.conf:
+                result[self.domain] = self.conf = []
+            elif not isinstance(self.conf, list):
+                result[self.domain] = self.conf = [self.conf]
 
         # Process AUTO_LOAD
-        for load in component.auto_load:
-            if load not in result:
-                result.add_validation_step(AutoLoadValidationStep(load))
+        _process_auto_load(result, component, path)
 
         result.add_validation_step(
             MetadataValidationStep([self.domain], self.domain, self.conf, component)
@@ -380,12 +426,6 @@ class LoadValidationStep(ConfigValidationStep):
         # Remove this is as an output path
         result.remove_output_path([self.domain], self.domain)
 
-        # Ensure conf is a list
-        if not self.conf:
-            result[self.domain] = self.conf = []
-        elif not isinstance(self.conf, list):
-            result[self.domain] = self.conf = [self.conf]
-
         for i, p_config in enumerate(self.conf):
             path = [self.domain, i]
             # Construct temporary unknown output path
@@ -585,16 +625,46 @@ class MetadataValidationStep(ConfigValidationStep):
                 )
                 return
             for i, part_conf in enumerate(self.conf):
+                path = self.path + [i]
                 result.add_validation_step(
-                    SchemaValidationStep(
-                        self.domain, self.path + [i], part_conf, self.comp
-                    )
+                    SchemaValidationStep(self.domain, path, part_conf, self.comp)
                 )
+                result.add_validation_step(FinalValidateValidationStep(path, self.comp))
+
             return
 
         result.add_validation_step(
             SchemaValidationStep(self.domain, self.path, self.conf, self.comp)
         )
+        result.add_validation_step(FinalValidateValidationStep(self.path, self.comp))
+
+
+class AddDynamicAutoLoadsValidationStep(ConfigValidationStep):
+    """Add dynamic auto loads step.
+
+    This step is used to auto-load components where one component can alter its
+    AUTO_LOAD based on its configuration.
+    """
+
+    # Has to happen after normal schema is validated and before final schema validation
+    priority = -5.0
+
+    def __init__(self, path: ConfigPath, comp: ComponentManifest) -> None:
+        self.path = path
+        self.comp = comp
+
+    def run(self, result: Config) -> None:
+        if result.errors:
+            # If result already has errors, skip this step
+            return
+
+        conf = result.get_nested_item(self.path)
+        with result.catch_error(self.path):
+            auto_load = self.comp.auto_load
+            if not callable(auto_load):
+                return
+            loads = auto_load(conf)
+            _add_auto_load_steps(result, loads)
 
 
 class SchemaValidationStep(ConfigValidationStep):
@@ -606,13 +676,15 @@ class SchemaValidationStep(ConfigValidationStep):
     def __init__(
         self, domain: str, path: ConfigPath, conf: ConfigType, comp: ComponentManifest
     ):
+        self.domain = domain
         self.path = path
         self.conf = conf
         self.comp = comp
 
     def run(self, result: Config) -> None:
         token = path_context.set(self.path)
-        with result.catch_error(self.path):
+        # The domain already contains the full component path (e.g., "sensor.template", "sensor.uptime")
+        with CORE.component_context(self.domain), result.catch_error(self.path):
             if self.comp.is_platform:
                 # Remove 'platform' key for validation
                 input_conf = OrderedDict(self.conf)
@@ -631,7 +703,6 @@ class SchemaValidationStep(ConfigValidationStep):
                 result.set_by_path(self.path, validated)
 
         path_context.reset(token)
-        result.add_validation_step(FinalValidateValidationStep(self.path, self.comp))
 
 
 class IDPassValidationStep(ConfigValidationStep):
@@ -824,7 +895,9 @@ class PinUseValidationCheck(ConfigValidationStep):
 
 
 def validate_config(
-    config: dict[str, Any], command_line_substitutions: dict[str, Any]
+    config: dict[str, Any],
+    command_line_substitutions: dict[str, Any],
+    skip_external_update: bool = False,
 ) -> Config:
     result = Config()
 
@@ -837,7 +910,7 @@ def validate_config(
 
         result.add_output_path([CONF_PACKAGES], CONF_PACKAGES)
         try:
-            config = do_packages_pass(config)
+            config = do_packages_pass(config, skip_update=skip_external_update)
         except vol.Invalid as err:
             result.update(config)
             result.add_error(err)
@@ -849,10 +922,9 @@ def validate_config(
     if CONF_SUBSTITUTIONS in config or command_line_substitutions:
         from esphome.components import substitutions
 
-        result[CONF_SUBSTITUTIONS] = {
-            **(config.get(CONF_SUBSTITUTIONS) or {}),
-            **command_line_substitutions,
-        }
+        result[CONF_SUBSTITUTIONS] = merge_dicts_ordered(
+            config.get(CONF_SUBSTITUTIONS) or {}, command_line_substitutions
+        )
         result.add_output_path([CONF_SUBSTITUTIONS], CONF_SUBSTITUTIONS)
         try:
             substitutions.do_substitution_pass(config, command_line_substitutions)
@@ -874,7 +946,7 @@ def validate_config(
 
         result.add_output_path([CONF_EXTERNAL_COMPONENTS], CONF_EXTERNAL_COMPONENTS)
         try:
-            do_external_components_pass(config)
+            do_external_components_pass(config, skip_update=skip_external_update)
         except vol.Invalid as err:
             result.update(config)
             result.add_error(err)
@@ -912,7 +984,7 @@ def validate_config(
 
     # First run platform validation steps
     result.add_validation_step(
-        LoadValidationStep(target_platform, config[target_platform])
+        LoadTargetPlatformValidationStep(target_platform, config[target_platform])
     )
     result.run_validation_steps()
 
@@ -920,6 +992,9 @@ def validate_config(
         # do not try to validate further as we don't know what the target is
         return result
 
+    # Reset the pin registry so that any target platforms with pin validations do not get the duplicate pin warning.
+    pins.PIN_SCHEMA_REGISTRY.reset()
+
     for domain, conf in config.items():
         result.add_validation_step(LoadValidationStep(domain, conf))
     result.add_validation_step(IDPassValidationStep())
@@ -995,7 +1070,9 @@ class InvalidYAMLError(EsphomeError):
         self.base_exc = base_exc
 
 
-def _load_config(command_line_substitutions: dict[str, Any]) -> Config:
+def _load_config(
+    command_line_substitutions: dict[str, Any], skip_external_update: bool = False
+) -> Config:
     """Load the configuration file."""
     try:
         config = yaml_util.load_yaml(CORE.config_path)
@@ -1003,7 +1080,7 @@ def _load_config(command_line_substitutions: dict[str, Any]) -> Config:
         raise InvalidYAMLError(e) from e
 
     try:
-        return validate_config(config, command_line_substitutions)
+        return validate_config(config, command_line_substitutions, skip_external_update)
     except EsphomeError:
         raise
     except Exception:
@@ -1011,9 +1088,11 @@ def _load_config(command_line_substitutions: dict[str, Any]) -> Config:
         raise
 
 
-def load_config(command_line_substitutions: dict[str, Any]) -> Config:
+def load_config(
+    command_line_substitutions: dict[str, Any], skip_external_update: bool = False
+) -> Config:
     try:
-        return _load_config(command_line_substitutions)
+        return _load_config(command_line_substitutions, skip_external_update)
     except vol.Invalid as err:
         raise EsphomeError(f"Error while parsing config: {err}") from err
 
@@ -1081,7 +1160,7 @@ def dump_dict(
             ret += "{}"
             multiline = False
 
-        for k in conf.keys():
+        for k in conf:
             path_ = path + [k]
             error = config.get_error_for_path(path_)
             if error is not None:
@@ -1097,10 +1176,7 @@ def dump_dict(
                 msg = f"\n{indent(msg)}"
 
             if inf is not None:
-                if m:
-                    msg = f" {inf}{msg}"
-                else:
-                    msg = f"{msg} {inf}"
+                msg = f" {inf}{msg}" if m else f"{msg} {inf}"
             ret += f"{st + msg}\n"
     elif isinstance(conf, str):
         if is_secret(conf):
@@ -1156,10 +1232,10 @@ def strip_default_ids(config):
     return config
 
 
-def read_config(command_line_substitutions):
+def read_config(command_line_substitutions, skip_external_update=False):
     _LOGGER.info("Reading configuration %s...", CORE.config_path)
     try:
-        res = load_config(command_line_substitutions)
+        res = load_config(command_line_substitutions, skip_external_update)
     except EsphomeError as err:
         _LOGGER.error("Error while reading config: %s", err)
         return None
diff --git a/esphome/config_helpers.py b/esphome/config_helpers.py
index 50ce4e8e34..88cfa49fdc 100644
--- a/esphome/config_helpers.py
+++ b/esphome/config_helpers.py
@@ -10,6 +10,7 @@ from esphome.const import (
     PlatformFramework,
 )
 from esphome.core import CORE
+from esphome.util import OrderedDict
 
 # Pre-build lookup map from (platform, framework) tuples to PlatformFramework enum
 _PLATFORM_FRAMEWORK_LOOKUP = {
@@ -17,6 +18,25 @@ _PLATFORM_FRAMEWORK_LOOKUP = {
 }
 
 
+def merge_dicts_ordered(*dicts: dict) -> OrderedDict:
+    """Merge multiple dicts into an OrderedDict, preserving key order.
+
+    This is a helper to ensure that dictionary merging preserves OrderedDict type,
+    which is important for operations like move_to_end().
+
+    Args:
+        *dicts: Variable number of dictionaries to merge (later dicts override earlier ones)
+
+    Returns:
+        OrderedDict with merged contents
+    """
+    result = OrderedDict()
+    for d in dicts:
+        if d:
+            result.update(d)
+    return result
+
+
 class Extend:
     def __init__(self, value):
         self.value = value
@@ -60,7 +80,11 @@ def merge_config(full_old, full_new):
         if isinstance(new, dict):
             if not isinstance(old, dict):
                 return new
-            res = old.copy()
+            # Preserve OrderedDict type by copying to OrderedDict if either input is OrderedDict
+            if isinstance(old, OrderedDict) or isinstance(new, OrderedDict):
+                res = OrderedDict(old)
+            else:
+                res = old.copy()
             for k, v in new.items():
                 if isinstance(v, Remove) and k in old:
                     del res[k]
@@ -111,8 +135,7 @@ def merge_config(full_old, full_new):
                     else:
                         ids[new_id] = len(res)
                 res.append(v)
-            res = [v for i, v in enumerate(res) if i not in ids_to_delete]
-            return res
+            return [v for i, v in enumerate(res) if i not in ids_to_delete]
         if new is None:
             return old
 
diff --git a/esphome/config_validation.py b/esphome/config_validation.py
index b1691fa43e..e2f0b835c9 100644
--- a/esphome/config_validation.py
+++ b/esphome/config_validation.py
@@ -2,7 +2,7 @@
 
 from __future__ import annotations
 
-from contextlib import contextmanager
+from contextlib import contextmanager, suppress
 from dataclasses import dataclass
 from datetime import datetime
 from ipaddress import (
@@ -15,7 +15,7 @@ from ipaddress import (
     ip_network,
 )
 import logging
-import os
+from pathlib import Path
 import re
 from string import ascii_letters, digits
 import uuid as uuid_
@@ -73,6 +73,7 @@ from esphome.const import (
     TYPE_GIT,
     TYPE_LOCAL,
     VALID_SUBSTITUTIONS_CHARACTERS,
+    Framework,
     __version__ as ESPHOME_VERSION,
 )
 from esphome.core import (
@@ -86,7 +87,7 @@ from esphome.core import (
     TimePeriodNanoseconds,
     TimePeriodSeconds,
 )
-from esphome.helpers import add_class_to_obj, list_starts_with
+from esphome.helpers import add_class_to_obj, docs_url, list_starts_with
 from esphome.schema_extractors import (
     SCHEMA_EXTRACT,
     schema_extractor,
@@ -243,6 +244,20 @@ RESERVED_IDS = [
     "uart0",
     "uart1",
     "uart2",
+    # ESP32 ROM functions
+    "crc16_be",
+    "crc16_le",
+    "crc32_be",
+    "crc32_le",
+    "crc8_be",
+    "crc8_le",
+    "dbg_state",
+    "debug_timer",
+    "one_bits",
+    "recv_packet",
+    "send_packet",
+    "check_pos",
+    "software_reset",
 ]
 
 
@@ -282,6 +297,40 @@ class FinalExternalInvalid(Invalid):
     """Represents an invalid value in the final validation phase where the path should not be prepended."""
 
 
+@dataclass(frozen=True, order=True)
+class Version:
+    major: int
+    minor: int
+    patch: int
+    extra: str = ""
+
+    def __str__(self):
+        if self.extra:
+            return f"{self.major}.{self.minor}.{self.patch}-{self.extra}"
+        return f"{self.major}.{self.minor}.{self.patch}"
+
+    @classmethod
+    def parse(cls, value: str) -> Version:
+        match = re.match(r"^(\d+).(\d+).(\d+)-?(\w*)$", value)
+        if match is None:
+            raise ValueError(f"Not a valid version number {value}")
+        major = int(match[1])
+        minor = int(match[2])
+        patch = int(match[3])
+        extra = match[4] or ""
+        return Version(major=major, minor=minor, patch=patch, extra=extra)
+
+    @property
+    def is_beta(self) -> bool:
+        """Check if this version is a beta version."""
+        return self.extra.startswith("b")
+
+    @property
+    def is_dev(self) -> bool:
+        """Check if this version is a development version."""
+        return self.extra.startswith("dev")
+
+
 def check_not_templatable(value):
     if isinstance(value, Lambda):
         raise Invalid("This option is not templatable!")
@@ -358,10 +407,13 @@ def icon(value):
     )
 
 
-def sub_device_id(value: str | None) -> core.ID:
+def sub_device_id(value: str | None) -> core.ID | None:
     # Lazy import to avoid circular imports
     from esphome.core.config import Device
 
+    if not value:
+        return None
+
     return use_id(Device)(value)
 
 
@@ -619,16 +671,27 @@ def only_on(platforms):
     return validator_
 
 
-def only_with_framework(frameworks):
+def only_with_framework(
+    frameworks: Framework | str | list[Framework | str], suggestions=None
+):
     """Validate that this option can only be specified on the given frameworks."""
     if not isinstance(frameworks, list):
         frameworks = [frameworks]
 
+    frameworks = [Framework(framework) for framework in frameworks]
+
+    if suggestions is None:
+        suggestions = {}
+
     def validator_(obj):
         if CORE.target_framework not in frameworks:
-            raise Invalid(
-                f"This feature is only available with frameworks {frameworks}"
-            )
+            err_str = f"This feature is only available with framework(s) {', '.join([framework.value for framework in frameworks])}"
+            if suggestion := suggestions.get(CORE.target_framework, None):
+                (component, docs_path) = suggestion
+                err_str += f"\nPlease use '{component}'"
+                if docs_path:
+                    err_str += f": {docs_url(path=f'components/{docs_path}')}"
+            raise Invalid(err_str)
         return obj
 
     return validator_
@@ -637,8 +700,8 @@ def only_with_framework(frameworks):
 only_on_esp32 = only_on(PLATFORM_ESP32)
 only_on_esp8266 = only_on(PLATFORM_ESP8266)
 only_on_rp2040 = only_on(PLATFORM_RP2040)
-only_with_arduino = only_with_framework("arduino")
-only_with_esp_idf = only_with_framework("esp-idf")
+only_with_arduino = only_with_framework(Framework.ARDUINO)
+only_with_esp_idf = only_with_framework(Framework.ESP_IDF)
 
 
 # Adapted from:
@@ -1063,8 +1126,8 @@ voltage = float_with_unit("voltage", "(v|V|volt|Volts)?")
 distance = float_with_unit("distance", "(m)")
 framerate = float_with_unit("framerate", "(FPS|fps|Fps|FpS|Hz)")
 angle = float_with_unit("angle", "(°|deg)", optional_unit=True)
-_temperature_c = float_with_unit("temperature", "(°C|° C|°|C)?")
-_temperature_k = float_with_unit("temperature", "(° K|° K|K)?")
+_temperature_c = float_with_unit("temperature", "(°C|° C|C|°)?")
+_temperature_k = float_with_unit("temperature", "(°K|° K|K)?")
 _temperature_f = float_with_unit("temperature", "(°F|° F|F)?")
 decibel = float_with_unit("decibel", "(dB|dBm|db|dbm)", optional_unit=True)
 pressure = float_with_unit("pressure", "(bar|Bar)", optional_unit=True)
@@ -1146,6 +1209,13 @@ def validate_bytes(value):
 
 
 def hostname(value):
+    """Validate that the value is a valid hostname.
+
+    Maximum length is 63 characters per RFC 1035.
+
+    Note: If this limit is changed, update MAX_NAME_WITH_SUFFIX_SIZE in
+    esphome/core/helpers.cpp to accommodate the new maximum length.
+    """
     value = string(value)
     if re.match(r"^[a-z0-9-]{1,63}$", value, re.IGNORECASE) is not None:
         return value
@@ -1560,34 +1630,32 @@ def dimensions(value):
     return dimensions([match.group(1), match.group(2)])
 
 
-def directory(value):
+def directory(value: object) -> Path:
     value = string(value)
     path = CORE.relative_config_path(value)
 
-    if not os.path.exists(path):
+    if not path.exists():
         raise Invalid(
-            f"Could not find directory '{path}'. Please make sure it exists (full path: {os.path.abspath(path)})."
+            f"Could not find directory '{path}'. Please make sure it exists (full path: {path.resolve()})."
         )
-    if not os.path.isdir(path):
+    if not path.is_dir():
         raise Invalid(
-            f"Path '{path}' is not a directory (full path: {os.path.abspath(path)})."
+            f"Path '{path}' is not a directory (full path: {path.resolve()})."
         )
-    return value
+    return path
 
 
-def file_(value):
+def file_(value: object) -> Path:
     value = string(value)
     path = CORE.relative_config_path(value)
 
-    if not os.path.exists(path):
+    if not path.exists():
         raise Invalid(
-            f"Could not find file '{path}'. Please make sure it exists (full path: {os.path.abspath(path)})."
+            f"Could not find file '{path}'. Please make sure it exists (full path: {path.resolve()})."
         )
-    if not os.path.isfile(path):
-        raise Invalid(
-            f"Path '{path}' is not a file (full path: {os.path.abspath(path)})."
-        )
-    return value
+    if not path.is_file():
+        raise Invalid(f"Path '{path}' is not a file (full path: {path.resolve()}).")
+    return path
 
 
 ENTITY_ID_CHARACTERS = "abcdefghijklmnopqrstuvwxyz0123456789_"
@@ -1814,7 +1882,7 @@ def validate_registry_entry(name, registry):
 
 def none(value):
     if value in ("none", "None"):
-        return None
+        return
     raise Invalid("Must be none")
 
 
@@ -1966,26 +2034,6 @@ def source_refresh(value: str):
     return positive_time_period_seconds(value)
 
 
-@dataclass(frozen=True, order=True)
-class Version:
-    major: int
-    minor: int
-    patch: int
-
-    def __str__(self):
-        return f"{self.major}.{self.minor}.{self.patch}"
-
-    @classmethod
-    def parse(cls, value: str) -> Version:
-        match = re.match(r"^(\d+).(\d+).(\d+)-?\w*$", value)
-        if match is None:
-            raise ValueError(f"Not a valid version number {value}")
-        major = int(match[1])
-        minor = int(match[2])
-        patch = int(match[3])
-        return Version(major=major, minor=minor, patch=patch)
-
-
 def version_number(value):
     value = string_strict(value)
     try:
@@ -2081,10 +2129,8 @@ def require_esphome_version(year, month, patch):
 
 @contextmanager
 def suppress_invalid():
-    try:
+    with suppress(vol.Invalid):
         yield
-    except vol.Invalid:
-        pass
 
 
 GIT_SCHEMA = Schema(
diff --git a/esphome/const.py b/esphome/const.py
index a30df6ef35..ce1c033e41 100644
--- a/esphome/const.py
+++ b/esphome/const.py
@@ -4,7 +4,7 @@ from enum import Enum
 
 from esphome.enum import StrEnum
 
-__version__ = "2025.8.0-dev"
+__version__ = "2025.11.0-dev"
 
 ALLOWED_NAME_CHARS = "abcdefghijklmnopqrstuvwxyz0123456789-_"
 VALID_SUBSTITUTIONS_CHARACTERS = (
@@ -21,6 +21,7 @@ class Platform(StrEnum):
     HOST = "host"
     LIBRETINY_OLDSTYLE = "libretiny"
     LN882X = "ln882x"
+    NRF52 = "nrf52"
     RP2040 = "rp2040"
     RTL87XX = "rtl87xx"
 
@@ -31,6 +32,15 @@ class Framework(StrEnum):
     ARDUINO = "arduino"
     ESP_IDF = "esp-idf"
     NATIVE = "host"
+    ZEPHYR = "zephyr"
+
+
+class ThreadModel(StrEnum):
+    """Threading model identifiers for ESPHome scheduler."""
+
+    SINGLE = "ESPHOME_THREAD_SINGLE"
+    MULTI_NO_ATOMICS = "ESPHOME_THREAD_MULTI_NO_ATOMICS"
+    MULTI_ATOMICS = "ESPHOME_THREAD_MULTI_ATOMICS"
 
 
 class PlatformFramework(Enum):
@@ -47,6 +57,9 @@ class PlatformFramework(Enum):
     RTL87XX_ARDUINO = (Platform.RTL87XX, Framework.ARDUINO)
     LN882X_ARDUINO = (Platform.LN882X, Framework.ARDUINO)
 
+    # Zephyr framework platforms
+    NRF52_ZEPHYR = (Platform.NRF52, Framework.ZEPHYR)
+
     # Host platform (native)
     HOST_NATIVE = (Platform.HOST, Framework.NATIVE)
 
@@ -58,6 +71,7 @@ PLATFORM_ESP8266 = Platform.ESP8266
 PLATFORM_HOST = Platform.HOST
 PLATFORM_LIBRETINY_OLDSTYLE = Platform.LIBRETINY_OLDSTYLE
 PLATFORM_LN882X = Platform.LN882X
+PLATFORM_NRF52 = Platform.NRF52
 PLATFORM_RP2040 = Platform.RP2040
 PLATFORM_RTL87XX = Platform.RTL87XX
 
@@ -90,6 +104,7 @@ CONF_ALL = "all"
 CONF_ALLOW_OTHER_USES = "allow_other_uses"
 CONF_ALPHA = "alpha"
 CONF_ALTITUDE = "altitude"
+CONF_ALTITUDE_COMPENSATION = "altitude_compensation"
 CONF_AMBIENT_LIGHT = "ambient_light"
 CONF_AMBIENT_PRESSURE_COMPENSATION = "ambient_pressure_compensation"
 CONF_AMBIENT_PRESSURE_COMPENSATION_SOURCE = "ambient_pressure_compensation_source"
@@ -99,6 +114,7 @@ CONF_AND = "and"
 CONF_ANGLE = "angle"
 CONF_ANY = "any"
 CONF_AP = "ap"
+CONF_API = "api"
 CONF_APPARENT_POWER = "apparent_power"
 CONF_ARDUINO_VERSION = "arduino_version"
 CONF_AREA = "area"
@@ -158,6 +174,7 @@ CONF_CALIBRATE_LINEAR = "calibrate_linear"
 CONF_CALIBRATION = "calibration"
 CONF_CAPACITANCE = "capacitance"
 CONF_CAPACITY = "capacity"
+CONF_CAPTURE_RESPONSE = "capture_response"
 CONF_CARBON_MONOXIDE = "carbon_monoxide"
 CONF_CARRIER_DUTY_PERCENT = "carrier_duty_percent"
 CONF_CARRIER_FREQUENCY = "carrier_frequency"
@@ -170,6 +187,7 @@ CONF_CHARACTERISTIC_UUID = "characteristic_uuid"
 CONF_CHECK = "check"
 CONF_CHIPSET = "chipset"
 CONF_CLEAN_SESSION = "clean_session"
+CONF_CLEAR = "clear"
 CONF_CLEAR_IMPEDANCE = "clear_impedance"
 CONF_CLIENT_CERTIFICATE = "client_certificate"
 CONF_CLIENT_CERTIFICATE_KEY = "client_certificate_key"
@@ -368,6 +386,8 @@ CONF_FINGER_ID = "finger_id"
 CONF_FINGERPRINT_COUNT = "fingerprint_count"
 CONF_FLASH_LENGTH = "flash_length"
 CONF_FLASH_TRANSITION_LENGTH = "flash_transition_length"
+CONF_FLIP_X = "flip_x"
+CONF_FLIP_Y = "flip_y"
 CONF_FLOW = "flow"
 CONF_FLOW_CONTROL_PIN = "flow_control_pin"
 CONF_FONT = "font"
@@ -407,6 +427,7 @@ CONF_HEAD = "head"
 CONF_HEADING = "heading"
 CONF_HEARTBEAT = "heartbeat"
 CONF_HEAT_ACTION = "heat_action"
+CONF_HEAT_COOL_MODE = "heat_cool_mode"
 CONF_HEAT_DEADBAND = "heat_deadband"
 CONF_HEAT_MODE = "heat_mode"
 CONF_HEAT_OVERRUN = "heat_overrun"
@@ -506,6 +527,7 @@ CONF_LOADED_INTEGRATIONS = "loaded_integrations"
 CONF_LOCAL = "local"
 CONF_LOCK_ACTION = "lock_action"
 CONF_LOG = "log"
+CONF_LOG_LEVEL = "log_level"
 CONF_LOG_TOPIC = "log_topic"
 CONF_LOGGER = "logger"
 CONF_LOGS = "logs"
@@ -521,6 +543,7 @@ CONF_MANUAL_IP = "manual_ip"
 CONF_MANUFACTURER_ID = "manufacturer_id"
 CONF_MASK_DISTURBER = "mask_disturber"
 CONF_MAX_BRIGHTNESS = "max_brightness"
+CONF_MAX_CONNECTIONS = "max_connections"
 CONF_MAX_COOLING_RUN_TIME = "max_cooling_run_time"
 CONF_MAX_CURRENT = "max_current"
 CONF_MAX_DURATION = "max_duration"
@@ -650,9 +673,11 @@ CONF_ON_PRESET_SET = "on_preset_set"
 CONF_ON_PRESS = "on_press"
 CONF_ON_RAW_VALUE = "on_raw_value"
 CONF_ON_RELEASE = "on_release"
+CONF_ON_RESPONSE = "on_response"
 CONF_ON_SHUTDOWN = "on_shutdown"
 CONF_ON_SPEED_SET = "on_speed_set"
 CONF_ON_STATE = "on_state"
+CONF_ON_SUCCESS = "on_success"
 CONF_ON_TAG = "on_tag"
 CONF_ON_TAG_REMOVED = "on_tag_removed"
 CONF_ON_TIME = "on_time"
@@ -671,6 +696,7 @@ CONF_OPEN_DRAIN = "open_drain"
 CONF_OPEN_DRAIN_INTERRUPT = "open_drain_interrupt"
 CONF_OPEN_DURATION = "open_duration"
 CONF_OPEN_ENDSTOP = "open_endstop"
+CONF_OPENTHREAD = "openthread"
 CONF_OPERATION = "operation"
 CONF_OPTIMISTIC = "optimistic"
 CONF_OPTION = "option"
@@ -743,6 +769,7 @@ CONF_POSITION_COMMAND_TOPIC = "position_command_topic"
 CONF_POSITION_STATE_TOPIC = "position_state_topic"
 CONF_POWER = "power"
 CONF_POWER_FACTOR = "power_factor"
+CONF_POWER_MODE = "power_mode"
 CONF_POWER_ON_VALUE = "power_on_value"
 CONF_POWER_SAVE_MODE = "power_save_mode"
 CONF_POWER_SUPPLY = "power_supply"
@@ -794,6 +821,7 @@ CONF_RESET_DURATION = "reset_duration"
 CONF_RESET_PIN = "reset_pin"
 CONF_RESIZE = "resize"
 CONF_RESOLUTION = "resolution"
+CONF_RESPONSE_TEMPLATE = "response_template"
 CONF_RESTART = "restart"
 CONF_RESTORE = "restore"
 CONF_RESTORE_MODE = "restore_mode"
@@ -809,6 +837,7 @@ CONF_RMT_CHANNEL = "rmt_channel"
 CONF_RMT_SYMBOLS = "rmt_symbols"
 CONF_ROTATION = "rotation"
 CONF_ROW = "row"
+CONF_ROWS = "rows"
 CONF_RS_PIN = "rs_pin"
 CONF_RTD_NOMINAL_RESISTANCE = "rtd_nominal_resistance"
 CONF_RTD_WIRES = "rtd_wires"
@@ -915,6 +944,7 @@ CONF_SWING_MODE_COMMAND_TOPIC = "swing_mode_command_topic"
 CONF_SWING_MODE_STATE_TOPIC = "swing_mode_state_topic"
 CONF_SWING_OFF_ACTION = "swing_off_action"
 CONF_SWING_VERTICAL_ACTION = "swing_vertical_action"
+CONF_SWITCH = "switch"
 CONF_SWITCH_DATAPOINT = "switch_datapoint"
 CONF_SWITCHES = "switches"
 CONF_SYNC = "sync"
@@ -1145,7 +1175,7 @@ UNIT_KILOMETER = "km"
 UNIT_KILOMETER_PER_HOUR = "km/h"
 UNIT_KILOVOLT_AMPS = "kVA"
 UNIT_KILOVOLT_AMPS_HOURS = "kVAh"
-UNIT_KILOVOLT_AMPS_REACTIVE = "kVAR"
+UNIT_KILOVOLT_AMPS_REACTIVE = "kvar"
 UNIT_KILOVOLT_AMPS_REACTIVE_HOURS = "kvarh"
 UNIT_KILOWATT = "kW"
 UNIT_KILOWATT_HOURS = "kWh"
@@ -1187,6 +1217,7 @@ UNIT_WATT = "W"
 UNIT_WATT_HOURS = "Wh"
 
 # device classes
+DEVICE_CLASS_ABSOLUTE_HUMIDITY = "absolute_humidity"
 DEVICE_CLASS_APPARENT_POWER = "apparent_power"
 DEVICE_CLASS_AQI = "aqi"
 DEVICE_CLASS_AREA = "area"
@@ -1245,6 +1276,7 @@ DEVICE_CLASS_PLUG = "plug"
 DEVICE_CLASS_PM1 = "pm1"
 DEVICE_CLASS_PM10 = "pm10"
 DEVICE_CLASS_PM25 = "pm25"
+DEVICE_CLASS_PM4 = "pm4"
 DEVICE_CLASS_POWER = "power"
 DEVICE_CLASS_POWER_FACTOR = "power_factor"
 DEVICE_CLASS_PRECIPITATION = "precipitation"
@@ -1268,6 +1300,7 @@ DEVICE_CLASS_SULPHUR_DIOXIDE = "sulphur_dioxide"
 DEVICE_CLASS_SWITCH = "switch"
 DEVICE_CLASS_TAMPER = "tamper"
 DEVICE_CLASS_TEMPERATURE = "temperature"
+DEVICE_CLASS_TEMPERATURE_DELTA = "temperature_delta"
 DEVICE_CLASS_TIMESTAMP = "timestamp"
 DEVICE_CLASS_UPDATE = "update"
 DEVICE_CLASS_VIBRATION = "vibration"
@@ -1311,3 +1344,7 @@ ENTITY_CATEGORY_CONFIG = "config"
 
 # The entity category for read only diagnostic values, for example RSSI, uptime or MAC Address
 ENTITY_CATEGORY_DIAGNOSTIC = "diagnostic"
+
+# The corresponding constant exists in c++
+# when update_interval is set to never, it becomes SCHEDULER_DONT_RUN milliseconds
+SCHEDULER_DONT_RUN = 4294967295
diff --git a/esphome/core/__init__.py b/esphome/core/__init__.py
index e33bbcf726..43febd28a2 100644
--- a/esphome/core/__init__.py
+++ b/esphome/core/__init__.py
@@ -1,7 +1,9 @@
 from collections import defaultdict
+from contextlib import contextmanager
 import logging
 import math
 import os
+from pathlib import Path
 import re
 from typing import TYPE_CHECKING
 
@@ -21,12 +23,14 @@ from esphome.const import (
     PLATFORM_ESP8266,
     PLATFORM_HOST,
     PLATFORM_LN882X,
+    PLATFORM_NRF52,
     PLATFORM_RP2040,
     PLATFORM_RTL87XX,
 )
 
 # pylint: disable=unused-import
 from esphome.coroutine import (  # noqa: F401
+    CoroPriority,
     FakeAwaitable as _FakeAwaitable,
     FakeEventLoop as _FakeEventLoop,
     coroutine,
@@ -36,8 +40,10 @@ from esphome.helpers import ensure_unique_string, get_str_env, is_ha_addon
 from esphome.util import OrderedDict
 
 if TYPE_CHECKING:
+    from esphome.address_cache import AddressCache
+
     from ..cpp_generator import MockObj, MockObjClass, Statement
-    from ..types import ConfigType
+    from ..types import ConfigType, EntityMetadata
 
 _LOGGER = logging.getLogger(__name__)
 
@@ -262,7 +268,7 @@ class TimePeriodMinutes(TimePeriod):
     pass
 
 
-LAMBDA_PROG = re.compile(r"id\(\s*([a-zA-Z_][a-zA-Z0-9_]*)\s*\)(\.?)")
+LAMBDA_PROG = re.compile(r"\bid\(\s*([a-zA-Z_][a-zA-Z0-9_]*)\s*\)(\.?)")
 
 
 class Lambda:
@@ -378,7 +384,7 @@ class DocumentLocation:
 
     @classmethod
     def from_mark(cls, mark):
-        return cls(mark.name, mark.line, mark.column)
+        return cls(str(mark.name), mark.line, mark.column)
 
     def __str__(self):
         return f"{self.document} {self.line}:{self.column}"
@@ -469,6 +475,52 @@ class Library:
             return self.as_tuple == other.as_tuple
         return NotImplemented
 
+    def reconcile_with(self, other):
+        """Merge two libraries, reconciling any conflicts."""
+
+        if self.name != other.name:
+            # Different libraries, no reconciliation possible
+            raise ValueError(
+                f"Cannot reconcile libraries with different names: {self.name} and {other.name}"
+            )
+
+        # repository specificity takes precedence over version specificity
+        if self.repository is None and other.repository is None:
+            pass  # No repositories, no conflict, continue on
+
+        elif self.repository is None:
+            # incoming library has a repository, use it
+            self.repository = other.repository
+            self.version = other.version
+            return self
+
+        elif other.repository is None:
+            return self  # use the repository/version already present
+
+        elif self.repository != other.repository:
+            raise ValueError(
+                f"Reconciliation failed! Libraries {self} and {other} requested with conflicting repositories!"
+            )
+
+        if self.version is None and other.version is None:
+            return self  # Arduino library reconciled against another Arduino library, current is acceptable
+
+        if self.version is None:
+            # incoming library has a version, use it
+            self.version = other.version
+            return self
+
+        if other.version is None:
+            return self  # incoming library has no version, current is acceptable
+
+            # Same versions, current library is acceptable
+        if self.version != other.version:
+            raise ValueError(
+                f"Version pinning failed! Libraries {other} and {self} "
+                "requested with conflicting versions!"
+            )
+        return self
+
 
 # pylint: disable=too-many-public-methods
 class EsphomeCore:
@@ -477,6 +529,8 @@ class EsphomeCore:
         self.dashboard = False
         # True if command is run from vscode api
         self.vscode = False
+        # True if running in testing mode (disables validation checks for grouped testing)
+        self.testing_mode = False
         # The name of the node
         self.name: str | None = None
         # The friendly name of the node
@@ -487,9 +541,9 @@ class EsphomeCore:
         # The first key to this dict should always be the integration name
         self.data = {}
         # The relative path to the configuration YAML
-        self.config_path: str | None = None
+        self.config_path: Path | None = None
         # The relative path to where all build files are stored
-        self.build_path: str | None = None
+        self.build_path: Path | None = None
         # The validated configuration, this is None until the config has been validated
         self.config: ConfigType | None = None
         # The pending tasks in the task queue (mostly for C++ generation)
@@ -504,8 +558,8 @@ class EsphomeCore:
         self.main_statements: list[Statement] = []
         # A list of statements to insert in the global block (includes and global variables)
         self.global_statements: list[Statement] = []
-        # A set of platformio libraries to add to the project
-        self.libraries: list[Library] = []
+        # A map of platformio libraries to add to the project (shortname: (name, version, repository))
+        self.platformio_libraries: dict[str, Library] = {}
         # A set of build flags to set in the platformio project
         self.build_flags: set[str] = set()
         # A set of build unflags to set in the platformio project
@@ -524,14 +578,18 @@ class EsphomeCore:
         # Key: platform name (e.g. "sensor", "binary_sensor"), Value: count
         self.platform_counts: defaultdict[str, int] = defaultdict(int)
         # Track entity unique IDs to handle duplicates
-        # Set of (device_id, platform, sanitized_name) tuples
-        self.unique_ids: set[tuple[str, str, str]] = set()
+        # Dict mapping (device_id, platform, sanitized_name) -> entity metadata
+        self.unique_ids: dict[tuple[str, str, str], EntityMetadata] = {}
         # Whether ESPHome was started in verbose mode
         self.verbose = False
         # Whether ESPHome was started in quiet mode
         self.quiet = False
         # A list of all known ID classes
         self.id_classes = {}
+        # The current component being processed during validation
+        self.current_component: str | None = None
+        # Address cache for DNS and mDNS lookups from command line arguments
+        self.address_cache: AddressCache | None = None
 
     def reset(self):
         from esphome.pins import PIN_SCHEMA_REGISTRY
@@ -549,7 +607,7 @@ class EsphomeCore:
         self.variables = {}
         self.main_statements = []
         self.global_statements = []
-        self.libraries = []
+        self.platformio_libraries = {}
         self.build_flags = set()
         self.build_unflags = set()
         self.defines = set()
@@ -557,9 +615,21 @@ class EsphomeCore:
         self.loaded_integrations = set()
         self.component_ids = set()
         self.platform_counts = defaultdict(int)
-        self.unique_ids = set()
+        self.unique_ids = {}
+        self.current_component = None
+        self.address_cache = None
         PIN_SCHEMA_REGISTRY.reset()
 
+    @contextmanager
+    def component_context(self, component: str):
+        """Context manager to set the current component being processed."""
+        old_component = self.current_component
+        self.current_component = component
+        try:
+            yield
+        finally:
+            self.current_component = old_component
+
     @property
     def address(self) -> str | None:
         if self.config is None:
@@ -597,43 +667,46 @@ class EsphomeCore:
         return None
 
     @property
-    def config_dir(self):
-        return os.path.abspath(os.path.dirname(self.config_path))
+    def config_dir(self) -> Path:
+        if self.config_path.is_dir():
+            return self.config_path.absolute()
+        return self.config_path.absolute().parent
 
     @property
-    def data_dir(self):
+    def data_dir(self) -> Path:
         if is_ha_addon():
-            return os.path.join("/data")
+            return Path("/data")
         if "ESPHOME_DATA_DIR" in os.environ:
-            return get_str_env("ESPHOME_DATA_DIR", None)
+            return Path(get_str_env("ESPHOME_DATA_DIR", None))
         return self.relative_config_path(".esphome")
 
     @property
-    def config_filename(self):
-        return os.path.basename(self.config_path)
+    def config_filename(self) -> str:
+        return self.config_path.name
 
-    def relative_config_path(self, *path):
-        path_ = os.path.expanduser(os.path.join(*path))
-        return os.path.join(self.config_dir, path_)
+    def relative_config_path(self, *path: str | Path) -> Path:
+        path_ = Path(*path).expanduser()
+        return self.config_dir / path_
 
-    def relative_internal_path(self, *path: str) -> str:
-        return os.path.join(self.data_dir, *path)
+    def relative_internal_path(self, *path: str | Path) -> Path:
+        path_ = Path(*path).expanduser()
+        return self.data_dir / path_
 
-    def relative_build_path(self, *path):
-        path_ = os.path.expanduser(os.path.join(*path))
-        return os.path.join(self.build_path, path_)
+    def relative_build_path(self, *path: str | Path) -> Path:
+        path_ = Path(*path).expanduser()
+        return self.build_path / path_
 
-    def relative_src_path(self, *path):
+    def relative_src_path(self, *path: str | Path) -> Path:
         return self.relative_build_path("src", *path)
 
-    def relative_pioenvs_path(self, *path):
+    def relative_pioenvs_path(self, *path: str | Path) -> Path:
         return self.relative_build_path(".pioenvs", *path)
 
-    def relative_piolibdeps_path(self, *path):
+    def relative_piolibdeps_path(self, *path: str | Path) -> Path:
         return self.relative_build_path(".piolibdeps", *path)
 
     @property
-    def firmware_bin(self):
+    def firmware_bin(self) -> Path:
         if self.is_libretiny:
             return self.relative_pioenvs_path(self.name, "firmware.uf2")
         return self.relative_pioenvs_path(self.name, "firmware.bin")
@@ -670,6 +743,10 @@ class EsphomeCore:
     def is_libretiny(self):
         return self.is_bk72xx or self.is_rtl87xx or self.is_ln882x
 
+    @property
+    def is_nrf52(self):
+        return self.target_platform == PLATFORM_NRF52
+
     @property
     def is_host(self):
         return self.target_platform == PLATFORM_HOST
@@ -686,6 +763,10 @@ class EsphomeCore:
     def using_esp_idf(self):
         return self.target_framework == "esp-idf"
 
+    @property
+    def using_zephyr(self):
+        return self.target_framework == "zephyr"
+
     def add_job(self, func, *args, **kwargs) -> None:
         self.event_loop.add_job(func, *args, **kwargs)
 
@@ -729,54 +810,26 @@ class EsphomeCore:
         _LOGGER.debug("Adding global: %s", expression)
         return expression
 
-    def add_library(self, library):
+    def add_library(self, library: Library):
         if not isinstance(library, Library):
-            raise ValueError(
+            raise TypeError(
                 f"Library {library} must be instance of Library, not {type(library)}"
             )
-        for other in self.libraries[:]:
-            if other.name is None or library.name is None:
-                continue
-            library_name = (
-                library.name if "/" not in library.name else library.name.split("/")[1]
-            )
-            other_name = (
-                other.name if "/" not in other.name else other.name.split("/")[1]
-            )
-            if other_name != library_name:
-                continue
-            if other.repository is not None:
-                if library.repository is None or other.repository == library.repository:
-                    # Other is using a/the same repository, takes precedence
-                    break
-                raise ValueError(
-                    f"Adding named Library with repository failed! Libraries {library} and {other} "
-                    "requested with conflicting repositories!"
-                )
 
-            if library.repository is not None:
-                # This is more specific since its using a repository
-                self.libraries.remove(other)
-                continue
+        if not library.name:
+            raise ValueError(f"The library for {library.repository} must have a name")
 
-            if library.version is None:
-                # Other requirement is more specific
-                break
-            if other.version is None:
-                # Found more specific version requirement
-                self.libraries.remove(other)
-                continue
-            if other.version == library.version:
-                break
+        short_name = (
+            library.name if "/" not in library.name else library.name.split("/")[-1]
+        )
 
-            raise ValueError(
-                f"Version pinning failed! Libraries {library} and {other} "
-                "requested with conflicting versions!"
-            )
-        else:
+        if short_name not in self.platformio_libraries:
             _LOGGER.debug("Adding library: %s", library)
-            self.libraries.append(library)
-        return library
+            self.platformio_libraries[short_name] = library
+            return library
+
+        self.platformio_libraries[short_name].reconcile_with(library)
+        return self.platformio_libraries[short_name]
 
     def add_build_flag(self, build_flag: str) -> str:
         self.build_flags.add(build_flag)
diff --git a/esphome/core/application.cpp b/esphome/core/application.cpp
index d6fab018cc..c745aa0ae5 100644
--- a/esphome/core/application.cpp
+++ b/esphome/core/application.cpp
@@ -4,6 +4,9 @@
 #include "esphome/core/hal.h"
 #include 
 #include 
+#ifdef USE_RUNTIME_STATS
+#include "esphome/components/runtime_stats/runtime_stats.h"
+#endif
 
 #ifdef USE_STATUS_LED
 #include "esphome/components/status_led/status_led.h"
@@ -31,6 +34,27 @@ namespace esphome {
 
 static const char *const TAG = "app";
 
+// Helper function for insertion sort of components by priority
+// Using insertion sort instead of std::stable_sort saves ~1.3KB of flash
+// by avoiding template instantiations (std::rotate, std::stable_sort, lambdas)
+// IMPORTANT: This sort is stable (preserves relative order of equal elements),
+// which is necessary to maintain user-defined component order for same priority
+template
+static void insertion_sort_by_priority(Iterator first, Iterator last) {
+  for (auto it = first + 1; it != last; ++it) {
+    auto key = *it;
+    float key_priority = (key->*GetPriority)();
+    auto j = it - 1;
+
+    // Using '<' (not '<=') ensures stability - equal priority components keep their order
+    while (j >= first && ((*j)->*GetPriority)() < key_priority) {
+      *(j + 1) = *j;
+      j--;
+    }
+    *(j + 1) = key;
+  }
+}
+
 void Application::register_component_(Component *comp) {
   if (comp == nullptr) {
     ESP_LOGW(TAG, "Tried to register null component!");
@@ -39,7 +63,7 @@ void Application::register_component_(Component *comp) {
 
   for (auto *c : this->components_) {
     if (comp == c) {
-      ESP_LOGW(TAG, "Component %s already registered! (%p)", c->get_component_source(), c);
+      ESP_LOGW(TAG, "Component %s already registered! (%p)", LOG_STR_ARG(c->get_component_log_str()), c);
       return;
     }
   }
@@ -48,9 +72,13 @@ void Application::register_component_(Component *comp) {
 void Application::setup() {
   ESP_LOGI(TAG, "Running through setup()");
   ESP_LOGV(TAG, "Sorting components by setup priority");
-  std::stable_sort(this->components_.begin(), this->components_.end(), [](const Component *a, const Component *b) {
-    return a->get_actual_setup_priority() > b->get_actual_setup_priority();
-  });
+
+  // Sort by setup priority using our helper function
+  insertion_sort_by_prioritycomponents_.begin()), &Component::get_actual_setup_priority>(
+      this->components_.begin(), this->components_.end());
+
+  // Initialize looping_components_ early so enable_pending_loops_() works during setup
+  this->calculate_looping_components_();
 
   for (uint32_t i = 0; i < this->components_.size(); i++) {
     Component *component = this->components_[i];
@@ -63,13 +91,17 @@ void Application::setup() {
     if (component->can_proceed())
       continue;
 
-    std::stable_sort(this->components_.begin(), this->components_.begin() + i + 1,
-                     [](Component *a, Component *b) { return a->get_loop_priority() > b->get_loop_priority(); });
+    // Sort components 0 through i by loop priority
+    insertion_sort_by_prioritycomponents_.begin()), &Component::get_loop_priority>(
+        this->components_.begin(), this->components_.begin() + i + 1);
 
     do {
       uint8_t new_app_state = STATUS_LED_WARNING;
-      this->scheduler.call();
-      this->feed_wdt();
+      uint32_t now = millis();
+
+      // Process pending loop enables to handle GPIO interrupts during setup
+      this->before_loop_tasks_(now);
+
       for (uint32_t j = 0; j <= i; j++) {
         // Update loop_component_start_time_ right before calling each component
         this->loop_component_start_time_ = millis();
@@ -78,6 +110,8 @@ void Application::setup() {
         this->app_state_ |= new_app_state;
         this->feed_wdt();
       }
+
+      this->after_loop_tasks_();
       this->app_state_ = new_app_state;
       yield();
     } while (!component->can_proceed());
@@ -89,35 +123,14 @@ void Application::setup() {
   clear_setup_priority_overrides();
 
   this->schedule_dump_config();
-  this->calculate_looping_components_();
 }
 void Application::loop() {
   uint8_t new_app_state = 0;
 
-  this->scheduler.call();
-
   // Get the initial loop time at the start
   uint32_t last_op_end_time = millis();
 
-  // Feed WDT with time
-  this->feed_wdt(last_op_end_time);
-
-  // Process any pending enable_loop requests from ISRs
-  // This must be done before marking in_loop_ = true to avoid race conditions
-  if (this->has_pending_enable_loop_requests_) {
-    // Clear flag BEFORE processing to avoid race condition
-    // If ISR sets it during processing, we'll catch it next loop iteration
-    // This is safe because:
-    // 1. Each component has its own pending_enable_loop_ flag that we check
-    // 2. If we can't process a component (wrong state), enable_pending_loops_()
-    //    will set this flag back to true
-    // 3. Any new ISR requests during processing will set the flag again
-    this->has_pending_enable_loop_requests_ = false;
-    this->enable_pending_loops_();
-  }
-
-  // Mark that we're in the loop for safe reentrant modifications
-  this->in_loop_ = true;
+  this->before_loop_tasks_(last_op_end_time);
 
   for (this->current_loop_index_ = 0; this->current_loop_index_ < this->looping_components_active_end_;
        this->current_loop_index_++) {
@@ -138,9 +151,17 @@ void Application::loop() {
     this->feed_wdt(last_op_end_time);
   }
 
-  this->in_loop_ = false;
+  this->after_loop_tasks_();
   this->app_state_ = new_app_state;
 
+#ifdef USE_RUNTIME_STATS
+  // Process any pending runtime stats printing after all components have run
+  // This ensures stats printing doesn't affect component timing measurements
+  if (global_runtime_stats != nullptr) {
+    global_runtime_stats->process_pending_stats(last_op_end_time);
+  }
+#endif
+
   // Use the last component's end time instead of calling millis() again
   auto elapsed = last_op_end_time - this->last_loop_;
   if (elapsed >= this->loop_interval_ || HighFrequencyLoopRequester::is_high_frequency()) {
@@ -149,7 +170,7 @@ void Application::loop() {
     this->yield_with_select_(0);
   } else {
     uint32_t delay_time = this->loop_interval_ - elapsed;
-    uint32_t next_schedule = this->scheduler.next_schedule_in().value_or(delay_time);
+    uint32_t next_schedule = this->scheduler.next_schedule_in(last_op_end_time).value_or(delay_time);
     // next_schedule is max 0.5*delay_time
     // otherwise interval=0 schedules result in constant looping with almost no sleep
     next_schedule = std::max(next_schedule, delay_time / 2);
@@ -220,30 +241,79 @@ void Application::run_powerdown_hooks() {
 void Application::teardown_components(uint32_t timeout_ms) {
   uint32_t start_time = millis();
 
-  // Copy all components in reverse order using reverse iterators
+  // Use a StaticVector instead of std::vector to avoid heap allocation
+  // since we know the actual size at compile time
+  StaticVector pending_components;
+
+  // Copy all components in reverse order
   // Reverse order matches the behavior of run_safe_shutdown_hooks() above and ensures
   // components are torn down in the opposite order of their setup_priority (which is
   // used to sort components during Application::setup())
-  std::vector pending_components(this->components_.rbegin(), this->components_.rend());
+  size_t num_components = this->components_.size();
+  for (size_t i = 0; i < num_components; ++i) {
+    pending_components[i] = this->components_[num_components - 1 - i];
+  }
 
   uint32_t now = start_time;
-  while (!pending_components.empty() && (now - start_time) < timeout_ms) {
+  size_t pending_count = num_components;
+
+  // Teardown Algorithm
+  // ==================
+  // We iterate through pending components, calling teardown() on each.
+  // Components that return false (need more time) are copied forward
+  // in the array. Components that return true (finished) are skipped.
+  //
+  // The compaction happens in-place during iteration:
+  //   - still_pending tracks the write position (where to put next pending component)
+  //   - i tracks the read position (which component we're testing)
+  //   - When teardown() returns false, we copy component[i] to component[still_pending]
+  //   - When teardown() returns true, we just skip it (don't increment still_pending)
+  //
+  // Example with 4 components where B can teardown immediately:
+  //
+  // Start:
+  //   pending_components: [A, B, C, D]
+  //   pending_count: 4    ^----------^
+  //
+  // Iteration 1:
+  //   i=0: A needs more time → keep at pos 0 (no copy needed)
+  //   i=1: B finished → skip
+  //   i=2: C needs more time → copy to pos 1
+  //   i=3: D needs more time → copy to pos 2
+  //
+  // After iteration 1:
+  //   pending_components: [A, C, D | D]
+  //   pending_count: 3    ^--------^
+  //
+  // Iteration 2:
+  //   i=0: A finished → skip
+  //   i=1: C needs more time → copy to pos 0
+  //   i=2: D finished → skip
+  //
+  // After iteration 2:
+  //   pending_components: [C | C, D, D]  (positions 1-3 have old values)
+  //   pending_count: 1    ^--^
+
+  while (pending_count > 0 && (now - start_time) < timeout_ms) {
     // Feed watchdog during teardown to prevent triggering
     this->feed_wdt(now);
 
-    // Use iterator to safely erase elements
-    for (auto it = pending_components.begin(); it != pending_components.end();) {
-      if ((*it)->teardown()) {
-        // Component finished teardown, erase it
-        it = pending_components.erase(it);
-      } else {
-        // Component still needs time
-        ++it;
+    // Process components and compact the array, keeping only those still pending
+    size_t still_pending = 0;
+    for (size_t i = 0; i < pending_count; ++i) {
+      if (!pending_components[i]->teardown()) {
+        // Component still needs time, copy it forward
+        if (still_pending != i) {
+          pending_components[still_pending] = pending_components[i];
+        }
+        ++still_pending;
       }
+      // Component finished teardown, skip it (don't increment still_pending)
     }
+    pending_count = still_pending;
 
     // Give some time for I/O operations if components are still pending
-    if (!pending_components.empty()) {
+    if (pending_count > 0) {
       this->yield_with_select_(1);
     }
 
@@ -251,12 +321,12 @@ void Application::teardown_components(uint32_t timeout_ms) {
     now = millis();
   }
 
-  if (!pending_components.empty()) {
+  if (pending_count > 0) {
     // Note: At this point, connections are either disconnected or in a bad state,
     // so this warning will only appear via serial rather than being transmitted to clients
-    for (auto *component : pending_components) {
-      ESP_LOGW(TAG, "%s did not complete teardown within %" PRIu32 " ms", component->get_component_source(),
-               timeout_ms);
+    for (size_t i = 0; i < pending_count; ++i) {
+      ESP_LOGW(TAG, "%s did not complete teardown within %" PRIu32 " ms",
+               LOG_STR_ARG(pending_components[i]->get_component_log_str()), timeout_ms);
     }
   }
 }
@@ -270,24 +340,25 @@ void Application::calculate_looping_components_() {
     }
   }
 
-  // Pre-reserve vector to avoid reallocations
-  this->looping_components_.reserve(total_looping);
+  // Initialize FixedVector with exact size - no reallocation possible
+  this->looping_components_.init(total_looping);
 
-  // First add all active components
-  for (auto *obj : this->components_) {
-    if (obj->has_overridden_loop() &&
-        (obj->get_component_state() & COMPONENT_STATE_MASK) != COMPONENT_STATE_LOOP_DONE) {
-      this->looping_components_.push_back(obj);
-    }
-  }
+  // Add all components with loop override that aren't already LOOP_DONE
+  // Some components (like logger) may call disable_loop() during initialization
+  // before setup runs, so we need to respect their LOOP_DONE state
+  this->add_looping_components_by_state_(false);
 
   this->looping_components_active_end_ = this->looping_components_.size();
 
-  // Then add all inactive (LOOP_DONE) components
-  // This handles components that called disable_loop() during setup, before this method runs
+  // Then add any components that are already LOOP_DONE to the inactive section
+  // This handles components that called disable_loop() during initialization
+  this->add_looping_components_by_state_(true);
+}
+
+void Application::add_looping_components_by_state_(bool match_loop_done) {
   for (auto *obj : this->components_) {
     if (obj->has_overridden_loop() &&
-        (obj->get_component_state() & COMPONENT_STATE_MASK) == COMPONENT_STATE_LOOP_DONE) {
+        ((obj->get_component_state() & COMPONENT_STATE_MASK) == COMPONENT_STATE_LOOP_DONE) == match_loop_done) {
       this->looping_components_.push_back(obj);
     }
   }
@@ -309,6 +380,12 @@ void Application::disable_component_loop_(Component *component) {
         if (this->in_loop_ && i == this->current_loop_index_) {
           // Decrement so we'll process the swapped component next
           this->current_loop_index_--;
+          // Update the loop start time to current time so the swapped component
+          // gets correct timing instead of inheriting stale timing.
+          // This prevents integer underflow in timing calculations by ensuring
+          // the swapped component starts with a fresh timing reference, avoiding
+          // errors caused by stale or wrapped timing values.
+          this->loop_component_start_time_ = millis();
         }
       }
       return;
@@ -380,7 +457,7 @@ void Application::enable_pending_loops_() {
 
     // Clear the pending flag and enable the loop
     component->pending_enable_loop_ = false;
-    ESP_LOGVV(TAG, "%s loop enabled from ISR", component->get_component_source());
+    ESP_LOGVV(TAG, "%s loop enabled from ISR", LOG_STR_ARG(component->get_component_log_str()));
     component->component_state_ &= ~COMPONENT_STATE_MASK;
     component->component_state_ |= COMPONENT_STATE_LOOP;
 
@@ -394,6 +471,36 @@ void Application::enable_pending_loops_() {
   }
 }
 
+void Application::before_loop_tasks_(uint32_t loop_start_time) {
+  // Process scheduled tasks
+  this->scheduler.call(loop_start_time);
+
+  // Feed the watchdog timer
+  this->feed_wdt(loop_start_time);
+
+  // Process any pending enable_loop requests from ISRs
+  // This must be done before marking in_loop_ = true to avoid race conditions
+  if (this->has_pending_enable_loop_requests_) {
+    // Clear flag BEFORE processing to avoid race condition
+    // If ISR sets it during processing, we'll catch it next loop iteration
+    // This is safe because:
+    // 1. Each component has its own pending_enable_loop_ flag that we check
+    // 2. If we can't process a component (wrong state), enable_pending_loops_()
+    //    will set this flag back to true
+    // 3. Any new ISR requests during processing will set the flag again
+    this->has_pending_enable_loop_requests_ = false;
+    this->enable_pending_loops_();
+  }
+
+  // Mark that we're in the loop for safe reentrant modifications
+  this->in_loop_ = true;
+}
+
+void Application::after_loop_tasks_() {
+  // Clear the in_loop_ flag to indicate we're done processing components
+  this->in_loop_ = false;
+}
+
 #ifdef USE_SOCKET_SELECT_SUPPORT
 bool Application::register_socket_fd(int fd) {
   // WARNING: This function is NOT thread-safe and must only be called from the main loop
@@ -401,11 +508,16 @@ bool Application::register_socket_fd(int fd) {
   if (fd < 0)
     return false;
 
+#ifndef USE_ESP32
+  // Only check on non-ESP32 platforms
+  // On ESP32 (both Arduino and ESP-IDF), CONFIG_LWIP_MAX_SOCKETS is always <= FD_SETSIZE by design
+  // (LWIP_SOCKET_OFFSET = FD_SETSIZE - CONFIG_LWIP_MAX_SOCKETS per lwipopts.h)
+  // Other platforms may not have this guarantee
   if (fd >= FD_SETSIZE) {
-    ESP_LOGE(TAG, "Cannot monitor socket fd %d: exceeds FD_SETSIZE (%d)", fd, FD_SETSIZE);
-    ESP_LOGE(TAG, "Socket will not be monitored for data - may cause performance issues!");
+    ESP_LOGE(TAG, "fd %d exceeds FD_SETSIZE %d", fd, FD_SETSIZE);
     return false;
   }
+#endif
 
   this->socket_fds_.push_back(fd);
   this->socket_fds_changed_ = true;
@@ -423,24 +535,25 @@ void Application::unregister_socket_fd(int fd) {
   if (fd < 0)
     return;
 
-  auto it = std::find(this->socket_fds_.begin(), this->socket_fds_.end(), fd);
-  if (it != this->socket_fds_.end()) {
+  for (size_t i = 0; i < this->socket_fds_.size(); i++) {
+    if (this->socket_fds_[i] != fd)
+      continue;
+
     // Swap with last element and pop - O(1) removal since order doesn't matter
-    if (it != this->socket_fds_.end() - 1) {
-      std::swap(*it, this->socket_fds_.back());
-    }
+    if (i < this->socket_fds_.size() - 1)
+      this->socket_fds_[i] = this->socket_fds_.back();
     this->socket_fds_.pop_back();
     this->socket_fds_changed_ = true;
 
     // Only recalculate max_fd if we removed the current max
     if (fd == this->max_fd_) {
-      if (this->socket_fds_.empty()) {
-        this->max_fd_ = -1;
-      } else {
-        // Find new max using std::max_element
-        this->max_fd_ = *std::max_element(this->socket_fds_.begin(), this->socket_fds_.end());
+      this->max_fd_ = -1;
+      for (int sock_fd : this->socket_fds_) {
+        if (sock_fd > this->max_fd_)
+          this->max_fd_ = sock_fd;
       }
     }
+    return;
   }
 }
 
diff --git a/esphome/core/application.h b/esphome/core/application.h
index f2b5cb5c89..6e7f1b49f2 100644
--- a/esphome/core/application.h
+++ b/esphome/core/application.h
@@ -10,6 +10,7 @@
 #include "esphome/core/helpers.h"
 #include "esphome/core/preferences.h"
 #include "esphome/core/scheduler.h"
+#include "esphome/core/string_ref.h"
 
 #ifdef USE_DEVICES
 #include "esphome/core/device.h"
@@ -101,11 +102,14 @@ class Application {
     arch_init();
     this->name_add_mac_suffix_ = name_add_mac_suffix;
     if (name_add_mac_suffix) {
-      this->name_ = name + "-" + get_mac_address().substr(6);
-      if (friendly_name.empty()) {
-        this->friendly_name_ = "";
-      } else {
-        this->friendly_name_ = friendly_name + " " + get_mac_address().substr(6);
+      // MAC address suffix length (last 6 characters of 12-char MAC address string)
+      constexpr size_t mac_address_suffix_len = 6;
+      const std::string mac_addr = get_mac_address();
+      // Use pointer + offset to avoid substr() allocation
+      const char *mac_suffix_ptr = mac_addr.c_str() + mac_address_suffix_len;
+      this->name_ = make_name_with_suffix(name, '-', mac_suffix_ptr, mac_address_suffix_len);
+      if (!friendly_name.empty()) {
+        this->friendly_name_ = make_name_with_suffix(friendly_name, ' ', mac_suffix_ptr, mac_address_suffix_len);
       }
     } else {
       this->name_ = name;
@@ -214,77 +218,6 @@ class Application {
 #endif
 
   /// Reserve space for components to avoid memory fragmentation
-  void reserve_components(size_t count) { this->components_.reserve(count); }
-
-#ifdef USE_BINARY_SENSOR
-  void reserve_binary_sensor(size_t count) { this->binary_sensors_.reserve(count); }
-#endif
-#ifdef USE_SWITCH
-  void reserve_switch(size_t count) { this->switches_.reserve(count); }
-#endif
-#ifdef USE_BUTTON
-  void reserve_button(size_t count) { this->buttons_.reserve(count); }
-#endif
-#ifdef USE_SENSOR
-  void reserve_sensor(size_t count) { this->sensors_.reserve(count); }
-#endif
-#ifdef USE_TEXT_SENSOR
-  void reserve_text_sensor(size_t count) { this->text_sensors_.reserve(count); }
-#endif
-#ifdef USE_FAN
-  void reserve_fan(size_t count) { this->fans_.reserve(count); }
-#endif
-#ifdef USE_COVER
-  void reserve_cover(size_t count) { this->covers_.reserve(count); }
-#endif
-#ifdef USE_CLIMATE
-  void reserve_climate(size_t count) { this->climates_.reserve(count); }
-#endif
-#ifdef USE_LIGHT
-  void reserve_light(size_t count) { this->lights_.reserve(count); }
-#endif
-#ifdef USE_NUMBER
-  void reserve_number(size_t count) { this->numbers_.reserve(count); }
-#endif
-#ifdef USE_DATETIME_DATE
-  void reserve_date(size_t count) { this->dates_.reserve(count); }
-#endif
-#ifdef USE_DATETIME_TIME
-  void reserve_time(size_t count) { this->times_.reserve(count); }
-#endif
-#ifdef USE_DATETIME_DATETIME
-  void reserve_datetime(size_t count) { this->datetimes_.reserve(count); }
-#endif
-#ifdef USE_SELECT
-  void reserve_select(size_t count) { this->selects_.reserve(count); }
-#endif
-#ifdef USE_TEXT
-  void reserve_text(size_t count) { this->texts_.reserve(count); }
-#endif
-#ifdef USE_LOCK
-  void reserve_lock(size_t count) { this->locks_.reserve(count); }
-#endif
-#ifdef USE_VALVE
-  void reserve_valve(size_t count) { this->valves_.reserve(count); }
-#endif
-#ifdef USE_MEDIA_PLAYER
-  void reserve_media_player(size_t count) { this->media_players_.reserve(count); }
-#endif
-#ifdef USE_ALARM_CONTROL_PANEL
-  void reserve_alarm_control_panel(size_t count) { this->alarm_control_panels_.reserve(count); }
-#endif
-#ifdef USE_EVENT
-  void reserve_event(size_t count) { this->events_.reserve(count); }
-#endif
-#ifdef USE_UPDATE
-  void reserve_update(size_t count) { this->updates_.reserve(count); }
-#endif
-#ifdef USE_AREAS
-  void reserve_area(size_t count) { this->areas_.reserve(count); }
-#endif
-#ifdef USE_DEVICES
-  void reserve_device(size_t count) { this->devices_.reserve(count); }
-#endif
 
   /// Register the component in this Application instance.
   template C *register_component(C *c) {
@@ -322,6 +255,8 @@ class Application {
   bool is_name_add_mac_suffix_enabled() const { return this->name_add_mac_suffix_; }
 
   std::string get_compilation_time() const { return this->compilation_time_; }
+  /// Get the compilation time as StringRef (for API usage)
+  StringRef get_compilation_time_ref() const { return StringRef(this->compilation_time_); }
 
   /// Get the cached time in milliseconds from when the current component started its loop execution
   inline uint32_t IRAM_ATTR HOT get_loop_component_start_time() const { return this->loop_component_start_time_; }
@@ -368,8 +303,19 @@ class Application {
 
   uint8_t get_app_state() const { return this->app_state_; }
 
-// Helper macro for entity getter method declarations - reduces code duplication
-// When USE_DEVICE_ID is enabled in the future, this can be conditionally compiled to add device_id parameter
+// Helper macro for entity getter method declarations
+#ifdef USE_DEVICES
+#define GET_ENTITY_METHOD(entity_type, entity_name, entities_member) \
+  entity_type *get_##entity_name##_by_key(uint32_t key, uint32_t device_id, bool include_internal = false) { \
+    for (auto *obj : this->entities_member##_) { \
+      if (obj->get_object_id_hash() == key && obj->get_device_id() == device_id && \
+          (include_internal || !obj->is_internal())) \
+        return obj; \
+    } \
+    return nullptr; \
+  }
+  const auto &get_devices() { return this->devices_; }
+#else
 #define GET_ENTITY_METHOD(entity_type, entity_name, entities_member) \
   entity_type *get_##entity_name##_by_key(uint32_t key, bool include_internal = false) { \
     for (auto *obj : this->entities_member##_) { \
@@ -378,100 +324,95 @@ class Application {
     } \
     return nullptr; \
   }
-
-#ifdef USE_DEVICES
-  const std::vector &get_devices() { return this->devices_; }
-#endif
+#endif  // USE_DEVICES
 #ifdef USE_AREAS
-  const std::vector &get_areas() { return this->areas_; }
+  const auto &get_areas() { return this->areas_; }
 #endif
 #ifdef USE_BINARY_SENSOR
-  const std::vector &get_binary_sensors() { return this->binary_sensors_; }
+  auto &get_binary_sensors() const { return this->binary_sensors_; }
   GET_ENTITY_METHOD(binary_sensor::BinarySensor, binary_sensor, binary_sensors)
 #endif
 #ifdef USE_SWITCH
-  const std::vector &get_switches() { return this->switches_; }
+  auto &get_switches() const { return this->switches_; }
   GET_ENTITY_METHOD(switch_::Switch, switch, switches)
 #endif
 #ifdef USE_BUTTON
-  const std::vector &get_buttons() { return this->buttons_; }
+  auto &get_buttons() const { return this->buttons_; }
   GET_ENTITY_METHOD(button::Button, button, buttons)
 #endif
 #ifdef USE_SENSOR
-  const std::vector &get_sensors() { return this->sensors_; }
+  auto &get_sensors() const { return this->sensors_; }
   GET_ENTITY_METHOD(sensor::Sensor, sensor, sensors)
 #endif
 #ifdef USE_TEXT_SENSOR
-  const std::vector &get_text_sensors() { return this->text_sensors_; }
+  auto &get_text_sensors() const { return this->text_sensors_; }
   GET_ENTITY_METHOD(text_sensor::TextSensor, text_sensor, text_sensors)
 #endif
 #ifdef USE_FAN
-  const std::vector &get_fans() { return this->fans_; }
+  auto &get_fans() const { return this->fans_; }
   GET_ENTITY_METHOD(fan::Fan, fan, fans)
 #endif
 #ifdef USE_COVER
-  const std::vector &get_covers() { return this->covers_; }
+  auto &get_covers() const { return this->covers_; }
   GET_ENTITY_METHOD(cover::Cover, cover, covers)
 #endif
 #ifdef USE_LIGHT
-  const std::vector &get_lights() { return this->lights_; }
+  auto &get_lights() const { return this->lights_; }
   GET_ENTITY_METHOD(light::LightState, light, lights)
 #endif
 #ifdef USE_CLIMATE
-  const std::vector &get_climates() { return this->climates_; }
+  auto &get_climates() const { return this->climates_; }
   GET_ENTITY_METHOD(climate::Climate, climate, climates)
 #endif
 #ifdef USE_NUMBER
-  const std::vector &get_numbers() { return this->numbers_; }
+  auto &get_numbers() const { return this->numbers_; }
   GET_ENTITY_METHOD(number::Number, number, numbers)
 #endif
 #ifdef USE_DATETIME_DATE
-  const std::vector &get_dates() { return this->dates_; }
+  auto &get_dates() const { return this->dates_; }
   GET_ENTITY_METHOD(datetime::DateEntity, date, dates)
 #endif
 #ifdef USE_DATETIME_TIME
-  const std::vector &get_times() { return this->times_; }
+  auto &get_times() const { return this->times_; }
   GET_ENTITY_METHOD(datetime::TimeEntity, time, times)
 #endif
 #ifdef USE_DATETIME_DATETIME
-  const std::vector &get_datetimes() { return this->datetimes_; }
+  auto &get_datetimes() const { return this->datetimes_; }
   GET_ENTITY_METHOD(datetime::DateTimeEntity, datetime, datetimes)
 #endif
 #ifdef USE_TEXT
-  const std::vector &get_texts() { return this->texts_; }
+  auto &get_texts() const { return this->texts_; }
   GET_ENTITY_METHOD(text::Text, text, texts)
 #endif
 #ifdef USE_SELECT
-  const std::vector &get_selects() { return this->selects_; }
+  auto &get_selects() const { return this->selects_; }
   GET_ENTITY_METHOD(select::Select, select, selects)
 #endif
 #ifdef USE_LOCK
-  const std::vector &get_locks() { return this->locks_; }
+  auto &get_locks() const { return this->locks_; }
   GET_ENTITY_METHOD(lock::Lock, lock, locks)
 #endif
 #ifdef USE_VALVE
-  const std::vector &get_valves() { return this->valves_; }
+  auto &get_valves() const { return this->valves_; }
   GET_ENTITY_METHOD(valve::Valve, valve, valves)
 #endif
 #ifdef USE_MEDIA_PLAYER
-  const std::vector &get_media_players() { return this->media_players_; }
+  auto &get_media_players() const { return this->media_players_; }
   GET_ENTITY_METHOD(media_player::MediaPlayer, media_player, media_players)
 #endif
 
 #ifdef USE_ALARM_CONTROL_PANEL
-  const std::vector &get_alarm_control_panels() {
-    return this->alarm_control_panels_;
-  }
+  auto &get_alarm_control_panels() const { return this->alarm_control_panels_; }
   GET_ENTITY_METHOD(alarm_control_panel::AlarmControlPanel, alarm_control_panel, alarm_control_panels)
 #endif
 
 #ifdef USE_EVENT
-  const std::vector &get_events() { return this->events_; }
+  auto &get_events() const { return this->events_; }
   GET_ENTITY_METHOD(event::Event, event, events)
 #endif
 
 #ifdef USE_UPDATE
-  const std::vector &get_updates() { return this->updates_; }
+  auto &get_updates() const { return this->updates_; }
   GET_ENTITY_METHOD(update::UpdateEntity, update, updates)
 #endif
 
@@ -496,6 +437,7 @@ class Application {
   void register_component_(Component *comp);
 
   void calculate_looping_components_();
+  void add_looping_components_by_state_(bool match_loop_done);
 
   // These methods are called by Component::disable_loop() and Component::enable_loop()
   // Components should not call these directly - use this->disable_loop() or this->enable_loop()
@@ -504,6 +446,8 @@ class Application {
   void enable_component_loop_(Component *component);
   void enable_pending_loops_();
   void activate_looping_component_(uint16_t index);
+  void before_loop_tasks_(uint32_t loop_start_time);
+  void after_loop_tasks_();
 
   void feed_wdt_arch_();
 
@@ -517,12 +461,7 @@ class Application {
   const char *comment_{nullptr};
   const char *compilation_time_{nullptr};
 
-  // size_t members
-  size_t dump_config_at_{SIZE_MAX};
-
-  // Vectors (largest members)
-  std::vector components_{};
-
+  // std::vector (3 pointers each: begin, end, capacity)
   // Partitioned vector design for looping components
   // =================================================
   // Components are partitioned into [active | inactive] sections:
@@ -539,86 +478,18 @@ class Application {
   // - When a component is enabled, it's swapped with the first inactive component
   //   and active_end_ is incremented
   // - This eliminates branch mispredictions from flag checking in the hot loop
-  std::vector looping_components_{};
-
-#ifdef USE_DEVICES
-  std::vector devices_{};
-#endif
-#ifdef USE_AREAS
-  std::vector areas_{};
-#endif
-#ifdef USE_BINARY_SENSOR
-  std::vector binary_sensors_{};
-#endif
-#ifdef USE_SWITCH
-  std::vector switches_{};
-#endif
-#ifdef USE_BUTTON
-  std::vector buttons_{};
-#endif
-#ifdef USE_EVENT
-  std::vector events_{};
-#endif
-#ifdef USE_SENSOR
-  std::vector sensors_{};
-#endif
-#ifdef USE_TEXT_SENSOR
-  std::vector text_sensors_{};
-#endif
-#ifdef USE_FAN
-  std::vector fans_{};
-#endif
-#ifdef USE_COVER
-  std::vector covers_{};
-#endif
-#ifdef USE_CLIMATE
-  std::vector climates_{};
-#endif
-#ifdef USE_LIGHT
-  std::vector lights_{};
-#endif
-#ifdef USE_NUMBER
-  std::vector numbers_{};
-#endif
-#ifdef USE_DATETIME_DATE
-  std::vector dates_{};
-#endif
-#ifdef USE_DATETIME_TIME
-  std::vector times_{};
-#endif
-#ifdef USE_DATETIME_DATETIME
-  std::vector datetimes_{};
-#endif
-#ifdef USE_SELECT
-  std::vector selects_{};
-#endif
-#ifdef USE_TEXT
-  std::vector texts_{};
-#endif
-#ifdef USE_LOCK
-  std::vector locks_{};
-#endif
-#ifdef USE_VALVE
-  std::vector valves_{};
-#endif
-#ifdef USE_MEDIA_PLAYER
-  std::vector media_players_{};
-#endif
-#ifdef USE_ALARM_CONTROL_PANEL
-  std::vector alarm_control_panels_{};
-#endif
-#ifdef USE_UPDATE
-  std::vector updates_{};
-#endif
-
+  FixedVector looping_components_{};
 #ifdef USE_SOCKET_SELECT_SUPPORT
   std::vector socket_fds_;  // Vector of all monitored socket file descriptors
 #endif
 
-  // String members
+  // std::string members (typically 24-32 bytes each)
   std::string name_;
   std::string friendly_name_;
 
+  // size_t members
+  size_t dump_config_at_{SIZE_MAX};
+
   // 4-byte members
   uint32_t last_loop_{0};
   uint32_t loop_component_start_time_{0};
@@ -628,9 +499,9 @@ class Application {
 #endif
 
   // 2-byte members (grouped together for alignment)
-  uint16_t loop_interval_{16};  // Loop interval in ms (max 65535ms = 65.5 seconds)
-  uint16_t looping_components_active_end_{0};
-  uint16_t current_loop_index_{0};  // For safe reentrant modifications during iteration
+  uint16_t loop_interval_{16};                 // Loop interval in ms (max 65535ms = 65.5 seconds)
+  uint16_t looping_components_active_end_{0};  // Index marking end of active components in looping_components_
+  uint16_t current_loop_index_{0};             // For safe reentrant modifications during iteration
 
   // 1-byte members (grouped together to minimize padding)
   uint8_t app_state_{0};
@@ -640,11 +511,87 @@ class Application {
 
 #ifdef USE_SOCKET_SELECT_SUPPORT
   bool socket_fds_changed_{false};  // Flag to rebuild base_read_fds_ when socket_fds_ changes
+#endif
 
-  // Variable-sized members at end
+#ifdef USE_SOCKET_SELECT_SUPPORT
+  // Variable-sized members
   fd_set base_read_fds_{};  // Cached fd_set rebuilt only when socket_fds_ changes
   fd_set read_fds_{};       // Working fd_set for select(), copied from base_read_fds_
 #endif
+
+  // StaticVectors (largest members - contain actual array data inline)
+  StaticVector components_{};
+
+#ifdef USE_DEVICES
+  StaticVector devices_{};
+#endif
+#ifdef USE_AREAS
+  StaticVector areas_{};
+#endif
+#ifdef USE_BINARY_SENSOR
+  StaticVector binary_sensors_{};
+#endif
+#ifdef USE_SWITCH
+  StaticVector switches_{};
+#endif
+#ifdef USE_BUTTON
+  StaticVector buttons_{};
+#endif
+#ifdef USE_EVENT
+  StaticVector events_{};
+#endif
+#ifdef USE_SENSOR
+  StaticVector sensors_{};
+#endif
+#ifdef USE_TEXT_SENSOR
+  StaticVector text_sensors_{};
+#endif
+#ifdef USE_FAN
+  StaticVector fans_{};
+#endif
+#ifdef USE_COVER
+  StaticVector covers_{};
+#endif
+#ifdef USE_CLIMATE
+  StaticVector climates_{};
+#endif
+#ifdef USE_LIGHT
+  StaticVector lights_{};
+#endif
+#ifdef USE_NUMBER
+  StaticVector numbers_{};
+#endif
+#ifdef USE_DATETIME_DATE
+  StaticVector dates_{};
+#endif
+#ifdef USE_DATETIME_TIME
+  StaticVector times_{};
+#endif
+#ifdef USE_DATETIME_DATETIME
+  StaticVector datetimes_{};
+#endif
+#ifdef USE_SELECT
+  StaticVector selects_{};
+#endif
+#ifdef USE_TEXT
+  StaticVector texts_{};
+#endif
+#ifdef USE_LOCK
+  StaticVector locks_{};
+#endif
+#ifdef USE_VALVE
+  StaticVector valves_{};
+#endif
+#ifdef USE_MEDIA_PLAYER
+  StaticVector media_players_{};
+#endif
+#ifdef USE_ALARM_CONTROL_PANEL
+  StaticVector
+      alarm_control_panels_{};
+#endif
+#ifdef USE_UPDATE
+  StaticVector updates_{};
+#endif
 };
 
 /// Global storage of Application pointer - only one Application can exist.
diff --git a/esphome/core/base_automation.h b/esphome/core/base_automation.h
index 13179b90bb..f1248e0035 100644
--- a/esphome/core/base_automation.h
+++ b/esphome/core/base_automation.h
@@ -5,6 +5,9 @@
 #include "esphome/core/hal.h"
 #include "esphome/core/defines.h"
 #include "esphome/core/preferences.h"
+#include "esphome/core/scheduler.h"
+#include "esphome/core/application.h"
+#include "esphome/core/helpers.h"
 
 #include 
 
@@ -12,7 +15,7 @@ namespace esphome {
 
 template class AndCondition : public Condition {
  public:
-  explicit AndCondition(const std::vector *> &conditions) : conditions_(conditions) {}
+  explicit AndCondition(std::initializer_list *> conditions) : conditions_(conditions) {}
   bool check(Ts... x) override {
     for (auto *condition : this->conditions_) {
       if (!condition->check(x...))
@@ -23,12 +26,12 @@ template class AndCondition : public Condition {
   }
 
  protected:
-  std::vector *> conditions_;
+  FixedVector *> conditions_;
 };
 
 template class OrCondition : public Condition {
  public:
-  explicit OrCondition(const std::vector *> &conditions) : conditions_(conditions) {}
+  explicit OrCondition(std::initializer_list *> conditions) : conditions_(conditions) {}
   bool check(Ts... x) override {
     for (auto *condition : this->conditions_) {
       if (condition->check(x...))
@@ -39,7 +42,7 @@ template class OrCondition : public Condition {
   }
 
  protected:
-  std::vector *> conditions_;
+  FixedVector *> conditions_;
 };
 
 template class NotCondition : public Condition {
@@ -53,7 +56,7 @@ template class NotCondition : public Condition {
 
 template class XorCondition : public Condition {
  public:
-  explicit XorCondition(const std::vector *> &conditions) : conditions_(conditions) {}
+  explicit XorCondition(std::initializer_list *> conditions) : conditions_(conditions) {}
   bool check(Ts... x) override {
     size_t result = 0;
     for (auto *condition : this->conditions_) {
@@ -64,7 +67,7 @@ template class XorCondition : public Condition {
   }
 
  protected:
-  std::vector *> conditions_;
+  FixedVector *> conditions_;
 };
 
 template class LambdaCondition : public Condition {
@@ -158,14 +161,23 @@ template class DelayAction : public Action, public Compon
   void play_complex(Ts... x) override {
     auto f = std::bind(&DelayAction::play_next_, this, x...);
     this->num_running_++;
-    this->set_timeout(this->delay_.value(x...), f);
+
+    // If num_running_ > 1, we have multiple instances running in parallel
+    // In single/restart/queued modes, only one instance runs at a time
+    // Parallel mode uses skip_cancel=true to allow multiple delays to coexist
+    // WARNING: This can accumulate delays if scripts are triggered faster than they complete!
+    // Users should set max_runs on parallel scripts to limit concurrent executions.
+    // Issue #10264: This is a workaround for parallel script delays interfering with each other.
+    App.scheduler.set_timer_common_(this, Scheduler::SchedulerItem::TIMEOUT,
+                                    /* is_static_string= */ true, "delay", this->delay_.value(x...), std::move(f),
+                                    /* is_retry= */ false, /* skip_cancel= */ this->num_running_ > 1);
   }
   float get_setup_priority() const override { return setup_priority::HARDWARE; }
 
   void play(Ts... x) override { /* ignore - see play_complex */
   }
 
-  void stop() override { this->cancel_timeout(""); }
+  void stop() override { this->cancel_timeout("delay"); }
 };
 
 template class LambdaAction : public Action {
diff --git a/esphome/core/color.h b/esphome/core/color.h
index 2b307bb438..5dce58a485 100644
--- a/esphome/core/color.h
+++ b/esphome/core/color.h
@@ -1,8 +1,13 @@
 #pragma once
 
+#include "defines.h"
 #include "component.h"
 #include "helpers.h"
 
+#ifdef USE_LVGL
+#include "esphome/components/lvgl/lvgl_proxy.h"
+#endif  // USE_LVGL
+
 namespace esphome {
 
 inline static constexpr uint8_t esp_scale8(uint8_t i, uint8_t scale) {
@@ -33,6 +38,11 @@ struct Color {
     uint32_t raw_32;
   };
 
+#ifdef USE_LVGL
+  // convenience function for Color to get a lv_color_t representation
+  operator lv_color_t() const { return lv_color_make(this->r, this->g, this->b); }
+#endif
+
   inline constexpr Color() ESPHOME_ALWAYS_INLINE : raw_32(0) {}  // NOLINT
   inline constexpr Color(uint8_t red, uint8_t green, uint8_t blue) ESPHOME_ALWAYS_INLINE : r(red),
                                                                                            g(green),
diff --git a/esphome/core/component.cpp b/esphome/core/component.cpp
index c47f16b5f7..11d9501bb8 100644
--- a/esphome/core/component.cpp
+++ b/esphome/core/component.cpp
@@ -9,6 +9,9 @@
 #include "esphome/core/hal.h"
 #include "esphome/core/helpers.h"
 #include "esphome/core/log.h"
+#ifdef USE_RUNTIME_STATS
+#include "esphome/components/runtime_stats/runtime_stats.h"
+#endif
 
 namespace esphome {
 
@@ -30,12 +33,22 @@ static const char *const TAG = "component";
 // Using namespace-scope static to avoid guard variables (saves 16 bytes total)
 // This is safe because ESPHome is single-threaded during initialization
 namespace {
+struct ComponentErrorMessage {
+  const Component *component;
+  const char *message;
+};
+
+struct ComponentPriorityOverride {
+  const Component *component;
+  float priority;
+};
+
 // Error messages for failed components
 // NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
-std::unique_ptr>> component_error_messages;
+std::unique_ptr> component_error_messages;
 // Setup priority overrides - freed after setup completes
 // NOLINTNEXTLINE(cppcoreguidelines-avoid-non-const-global-variables)
-std::unique_ptr>> setup_priority_overrides;
+std::unique_ptr> setup_priority_overrides;
 }  // namespace
 
 namespace setup_priority {
@@ -129,16 +142,17 @@ void Component::call_dump_config() {
   this->dump_config();
   if (this->is_failed()) {
     // Look up error message from global vector
-    const char *error_msg = "unspecified";
+    const char *error_msg = nullptr;
     if (component_error_messages) {
-      for (const auto &pair : *component_error_messages) {
-        if (pair.first == this) {
-          error_msg = pair.second;
+      for (const auto &entry : *component_error_messages) {
+        if (entry.component == this) {
+          error_msg = entry.message;
           break;
         }
       }
     }
-    ESP_LOGE(TAG, "  %s is marked FAILED: %s", this->get_component_source(), error_msg);
+    ESP_LOGE(TAG, "  %s is marked FAILED: %s", LOG_STR_ARG(this->get_component_log_str()),
+             error_msg ? error_msg : LOG_STR_LITERAL("unspecified"));
   }
 }
 
@@ -146,36 +160,39 @@ uint8_t Component::get_component_state() const { return this->component_state_;
 void Component::call() {
   uint8_t state = this->component_state_ & COMPONENT_STATE_MASK;
   switch (state) {
-    case COMPONENT_STATE_CONSTRUCTION:
+    case COMPONENT_STATE_CONSTRUCTION: {
       // State Construction: Call setup and set state to setup
-      this->component_state_ &= ~COMPONENT_STATE_MASK;
-      this->component_state_ |= COMPONENT_STATE_SETUP;
+      this->set_component_state_(COMPONENT_STATE_SETUP);
+      ESP_LOGV(TAG, "Setup %s", LOG_STR_ARG(this->get_component_log_str()));
+#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_DEBUG
+      uint32_t start_time = millis();
+#endif
       this->call_setup();
+#if ESPHOME_LOG_LEVEL >= ESPHOME_LOG_LEVEL_DEBUG
+      uint32_t setup_time = millis() - start_time;
+      ESP_LOGCONFIG(TAG, "Setup %s took %ums", LOG_STR_ARG(this->get_component_log_str()), (unsigned) setup_time);
+#endif
       break;
+    }
     case COMPONENT_STATE_SETUP:
       // State setup: Call first loop and set state to loop
-      this->component_state_ &= ~COMPONENT_STATE_MASK;
-      this->component_state_ |= COMPONENT_STATE_LOOP;
+      this->set_component_state_(COMPONENT_STATE_LOOP);
       this->call_loop();
       break;
     case COMPONENT_STATE_LOOP:
       // State loop: Call loop
       this->call_loop();
       break;
-    case COMPONENT_STATE_FAILED:  // NOLINT(bugprone-branch-clone)
+    case COMPONENT_STATE_FAILED:
       // State failed: Do nothing
-      break;
-    case COMPONENT_STATE_LOOP_DONE:  // NOLINT(bugprone-branch-clone)
+    case COMPONENT_STATE_LOOP_DONE:
       // State loop done: Do nothing, component has finished its work
-      break;
     default:
       break;
   }
 }
-const char *Component::get_component_source() const {
-  if (this->component_source_ == nullptr)
-    return "";
-  return this->component_source_;
+const LogString *Component::get_component_log_str() const {
+  return this->component_source_ == nullptr ? LOG_STR("") : this->component_source_;
 }
 bool Component::should_warn_of_blocking(uint32_t blocking_time) {
   if (blocking_time > this->warn_if_blocking_over_) {
@@ -191,26 +208,27 @@ bool Component::should_warn_of_blocking(uint32_t blocking_time) {
   return false;
 }
 void Component::mark_failed() {
-  ESP_LOGE(TAG, "%s was marked as failed", this->get_component_source());
-  this->component_state_ &= ~COMPONENT_STATE_MASK;
-  this->component_state_ |= COMPONENT_STATE_FAILED;
+  ESP_LOGE(TAG, "%s was marked as failed", LOG_STR_ARG(this->get_component_log_str()));
+  this->set_component_state_(COMPONENT_STATE_FAILED);
   this->status_set_error();
   // Also remove from loop since failed components shouldn't loop
   App.disable_component_loop_(this);
 }
+void Component::set_component_state_(uint8_t state) {
+  this->component_state_ &= ~COMPONENT_STATE_MASK;
+  this->component_state_ |= state;
+}
 void Component::disable_loop() {
   if ((this->component_state_ & COMPONENT_STATE_MASK) != COMPONENT_STATE_LOOP_DONE) {
-    ESP_LOGVV(TAG, "%s loop disabled", this->get_component_source());
-    this->component_state_ &= ~COMPONENT_STATE_MASK;
-    this->component_state_ |= COMPONENT_STATE_LOOP_DONE;
+    ESP_LOGVV(TAG, "%s loop disabled", LOG_STR_ARG(this->get_component_log_str()));
+    this->set_component_state_(COMPONENT_STATE_LOOP_DONE);
     App.disable_component_loop_(this);
   }
 }
 void Component::enable_loop() {
   if ((this->component_state_ & COMPONENT_STATE_MASK) == COMPONENT_STATE_LOOP_DONE) {
-    ESP_LOGVV(TAG, "%s loop enabled", this->get_component_source());
-    this->component_state_ &= ~COMPONENT_STATE_MASK;
-    this->component_state_ |= COMPONENT_STATE_LOOP;
+    ESP_LOGVV(TAG, "%s loop enabled", LOG_STR_ARG(this->get_component_log_str()));
+    this->set_component_state_(COMPONENT_STATE_LOOP);
     App.enable_component_loop_(this);
   }
 }
@@ -229,9 +247,8 @@ void IRAM_ATTR HOT Component::enable_loop_soon_any_context() {
 }
 void Component::reset_to_construction_state() {
   if ((this->component_state_ & COMPONENT_STATE_MASK) == COMPONENT_STATE_FAILED) {
-    ESP_LOGI(TAG, "%s is being reset to construction state", this->get_component_source());
-    this->component_state_ &= ~COMPONENT_STATE_MASK;
-    this->component_state_ |= COMPONENT_STATE_CONSTRUCTION;
+    ESP_LOGI(TAG, "%s is being reset to construction state", LOG_STR_ARG(this->get_component_log_str()));
+    this->set_component_state_(COMPONENT_STATE_CONSTRUCTION);
     // Clear error status when resetting
     this->status_clear_error();
   }
@@ -252,10 +269,10 @@ void Component::defer(const char *name, std::function &&f) {  // NOLINT
   App.scheduler.set_timeout(this, name, 0, std::move(f));
 }
 void Component::set_timeout(uint32_t timeout, std::function &&f) {  // NOLINT
-  App.scheduler.set_timeout(this, "", timeout, std::move(f));
+  App.scheduler.set_timeout(this, static_cast(nullptr), timeout, std::move(f));
 }
 void Component::set_interval(uint32_t interval, std::function &&f) {  // NOLINT
-  App.scheduler.set_interval(this, "", interval, std::move(f));
+  App.scheduler.set_interval(this, static_cast(nullptr), interval, std::move(f));
 }
 void Component::set_retry(uint32_t initial_wait_time, uint8_t max_attempts, std::function &&f,
                           float backoff_increase_factor) {  // NOLINT
@@ -270,47 +287,59 @@ bool Component::is_ready() const {
 bool Component::can_proceed() { return true; }
 bool Component::status_has_warning() const { return this->component_state_ & STATUS_LED_WARNING; }
 bool Component::status_has_error() const { return this->component_state_ & STATUS_LED_ERROR; }
+
 void Component::status_set_warning(const char *message) {
   // Don't spam the log. This risks missing different warning messages though.
   if ((this->component_state_ & STATUS_LED_WARNING) != 0)
     return;
   this->component_state_ |= STATUS_LED_WARNING;
   App.app_state_ |= STATUS_LED_WARNING;
-  ESP_LOGW(TAG, "%s set Warning flag: %s", this->get_component_source(), message);
+  ESP_LOGW(TAG, "%s set Warning flag: %s", LOG_STR_ARG(this->get_component_log_str()),
+           message ? message : LOG_STR_LITERAL("unspecified"));
+}
+void Component::status_set_warning(const LogString *message) {
+  // Don't spam the log. This risks missing different warning messages though.
+  if ((this->component_state_ & STATUS_LED_WARNING) != 0)
+    return;
+  this->component_state_ |= STATUS_LED_WARNING;
+  App.app_state_ |= STATUS_LED_WARNING;
+  ESP_LOGW(TAG, "%s set Warning flag: %s", LOG_STR_ARG(this->get_component_log_str()),
+           message ? LOG_STR_ARG(message) : LOG_STR_LITERAL("unspecified"));
 }
 void Component::status_set_error(const char *message) {
   if ((this->component_state_ & STATUS_LED_ERROR) != 0)
     return;
   this->component_state_ |= STATUS_LED_ERROR;
   App.app_state_ |= STATUS_LED_ERROR;
-  ESP_LOGE(TAG, "%s set Error flag: %s", this->get_component_source(), message);
-  if (strcmp(message, "unspecified") != 0) {
+  ESP_LOGE(TAG, "%s set Error flag: %s", LOG_STR_ARG(this->get_component_log_str()),
+           message ? message : LOG_STR_LITERAL("unspecified"));
+  if (message != nullptr) {
     // Lazy allocate the error messages vector if needed
     if (!component_error_messages) {
-      component_error_messages = std::make_unique>>();
+      component_error_messages = std::make_unique>();
     }
     // Check if this component already has an error message
-    for (auto &pair : *component_error_messages) {
-      if (pair.first == this) {
-        pair.second = message;
+    for (auto &entry : *component_error_messages) {
+      if (entry.component == this) {
+        entry.message = message;
         return;
       }
     }
     // Add new error message
-    component_error_messages->emplace_back(this, message);
+    component_error_messages->emplace_back(ComponentErrorMessage{this, message});
   }
 }
 void Component::status_clear_warning() {
   if ((this->component_state_ & STATUS_LED_WARNING) == 0)
     return;
   this->component_state_ &= ~STATUS_LED_WARNING;
-  ESP_LOGW(TAG, "%s cleared Warning flag", this->get_component_source());
+  ESP_LOGW(TAG, "%s cleared Warning flag", LOG_STR_ARG(this->get_component_log_str()));
 }
 void Component::status_clear_error() {
   if ((this->component_state_ & STATUS_LED_ERROR) == 0)
     return;
   this->component_state_ &= ~STATUS_LED_ERROR;
-  ESP_LOGE(TAG, "%s cleared Error flag", this->get_component_source());
+  ESP_LOGE(TAG, "%s cleared Error flag", LOG_STR_ARG(this->get_component_log_str()));
 }
 void Component::status_momentary_warning(const std::string &name, uint32_t length) {
   this->status_set_warning();
@@ -321,13 +350,25 @@ void Component::status_momentary_error(const std::string &name, uint32_t length)
   this->set_timeout(name, length, [this]() { this->status_clear_error(); });
 }
 void Component::dump_config() {}
+
+// Function implementation of LOG_UPDATE_INTERVAL macro to reduce code size
+void log_update_interval(const char *tag, PollingComponent *component) {
+  uint32_t update_interval = component->get_update_interval();
+  if (update_interval == SCHEDULER_DONT_RUN) {
+    ESP_LOGCONFIG(tag, "  Update Interval: never");
+  } else if (update_interval < 100) {
+    ESP_LOGCONFIG(tag, "  Update Interval: %.3fs", update_interval / 1000.0f);
+  } else {
+    ESP_LOGCONFIG(tag, "  Update Interval: %.1fs", update_interval / 1000.0f);
+  }
+}
 float Component::get_actual_setup_priority() const {
   // Check if there's an override in the global vector
   if (setup_priority_overrides) {
     // Linear search is fine for small n (typically < 5 overrides)
-    for (const auto &pair : *setup_priority_overrides) {
-      if (pair.first == this) {
-        return pair.second;
+    for (const auto &entry : *setup_priority_overrides) {
+      if (entry.component == this) {
+        return entry.priority;
       }
     }
   }
@@ -336,21 +377,21 @@ float Component::get_actual_setup_priority() const {
 void Component::set_setup_priority(float priority) {
   // Lazy allocate the vector if needed
   if (!setup_priority_overrides) {
-    setup_priority_overrides = std::make_unique>>();
+    setup_priority_overrides = std::make_unique>();
     // Reserve some space to avoid reallocations (most configs have < 10 overrides)
     setup_priority_overrides->reserve(10);
   }
 
   // Check if this component already has an override
-  for (auto &pair : *setup_priority_overrides) {
-    if (pair.first == this) {
-      pair.second = priority;
+  for (auto &entry : *setup_priority_overrides) {
+    if (entry.component == this) {
+      entry.priority = priority;
       return;
     }
   }
 
   // Add new override
-  setup_priority_overrides->emplace_back(this, priority);
+  setup_priority_overrides->emplace_back(ComponentPriorityOverride{this, priority});
 }
 
 bool Component::has_overridden_loop() const {
@@ -370,11 +411,10 @@ bool Component::has_overridden_loop() const {
 PollingComponent::PollingComponent(uint32_t update_interval) : update_interval_(update_interval) {}
 
 void PollingComponent::call_setup() {
+  // init the poller before calling setup, allowing setup to cancel it if desired
+  this->start_poller();
   // Let the polling component subclass setup their HW.
   this->setup();
-
-  // init the poller
-  this->start_poller();
 }
 
 void PollingComponent::start_poller() {
@@ -396,6 +436,13 @@ uint32_t WarnIfComponentBlockingGuard::finish() {
   uint32_t curr_time = millis();
 
   uint32_t blocking_time = curr_time - this->started_;
+
+#ifdef USE_RUNTIME_STATS
+  // Record component runtime stats
+  if (global_runtime_stats != nullptr) {
+    global_runtime_stats->record_component_time(this->component_, blocking_time, curr_time);
+  }
+#endif
   bool should_warn;
   if (this->component_ != nullptr) {
     should_warn = this->component_->should_warn_of_blocking(blocking_time);
@@ -403,8 +450,9 @@ uint32_t WarnIfComponentBlockingGuard::finish() {
     should_warn = blocking_time > WARN_IF_BLOCKING_OVER_MS;
   }
   if (should_warn) {
-    const char *src = component_ == nullptr ? "" : component_->get_component_source();
-    ESP_LOGW(TAG, "%s took a long time for an operation (%" PRIu32 " ms)", src, blocking_time);
+    ESP_LOGW(TAG, "%s took a long time for an operation (%" PRIu32 " ms)",
+             component_ == nullptr ? LOG_STR_LITERAL("") : LOG_STR_ARG(component_->get_component_log_str()),
+             blocking_time);
     ESP_LOGW(TAG, "Components should block for at most 30 ms");
   }
 
diff --git a/esphome/core/component.h b/esphome/core/component.h
index 3734473a02..e97941374d 100644
--- a/esphome/core/component.h
+++ b/esphome/core/component.h
@@ -5,10 +5,14 @@
 #include 
 #include 
 
+#include "esphome/core/log.h"
 #include "esphome/core/optional.h"
 
 namespace esphome {
 
+// Forward declaration for LogString
+struct LogString;
+
 /** Default setup priorities for components of different types.
  *
  * Components should return one of these setup priorities in get_setup_priority.
@@ -44,14 +48,13 @@ extern const float LATE;
 
 static const uint32_t SCHEDULER_DONT_RUN = 4294967295UL;
 
-#define LOG_UPDATE_INTERVAL(this) \
-  if (this->get_update_interval() == SCHEDULER_DONT_RUN) { \
-    ESP_LOGCONFIG(TAG, "  Update Interval: never"); \
-  } else if (this->get_update_interval() < 100) { \
-    ESP_LOGCONFIG(TAG, "  Update Interval: %.3fs", this->get_update_interval() / 1000.0f); \
-  } else { \
-    ESP_LOGCONFIG(TAG, "  Update Interval: %.1fs", this->get_update_interval() / 1000.0f); \
-  }
+// Forward declaration
+class PollingComponent;
+
+// Function declaration for LOG_UPDATE_INTERVAL
+void log_update_interval(const char *tag, PollingComponent *component);
+
+#define LOG_UPDATE_INTERVAL(this) log_update_interval(TAG, this)
 
 extern const uint8_t COMPONENT_STATE_MASK;
 extern const uint8_t COMPONENT_STATE_CONSTRUCTION;
@@ -202,9 +205,10 @@ class Component {
 
   bool status_has_error() const;
 
-  void status_set_warning(const char *message = "unspecified");
+  void status_set_warning(const char *message = nullptr);
+  void status_set_warning(const LogString *message);
 
-  void status_set_error(const char *message = "unspecified");
+  void status_set_error(const char *message = nullptr);
 
   void status_clear_warning();
 
@@ -220,12 +224,12 @@ class Component {
    *
    * This is set by the ESPHome core, and should not be called manually.
    */
-  void set_component_source(const char *source) { component_source_ = source; }
-  /** Get the integration where this component was declared as a string.
+  void set_component_source(const LogString *source) { component_source_ = source; }
+  /** Get the integration where this component was declared as a LogString for logging.
    *
-   * Returns "" if source not set
+   * Returns LOG_STR("") if source not set
    */
-  const char *get_component_source() const;
+  const LogString *get_component_log_str() const;
 
   bool should_warn_of_blocking(uint32_t blocking_time);
 
@@ -236,6 +240,9 @@ class Component {
   virtual void call_setup();
   virtual void call_dump_config();
 
+  /// Helper to set component state (clears state bits and sets new state)
+  void set_component_state_(uint8_t state);
+
   /** Set an interval function with a unique name. Empty name means no cancelling possible.
    *
    * This will call f every interval ms. Can be cancelled via CancelInterval().
@@ -402,13 +409,13 @@ class Component {
   bool cancel_defer(const std::string &name);  // NOLINT
 
   // Ordered for optimal packing on 32-bit systems
-  const char *component_source_{nullptr};
+  const LogString *component_source_{nullptr};
   uint16_t warn_if_blocking_over_{WARN_IF_BLOCKING_OVER_MS};  ///< Warn if blocked for this many ms (max 65.5s)
   /// State of this component - each bit has a purpose:
-  /// Bits 0-1: Component state (0x00=CONSTRUCTION, 0x01=SETUP, 0x02=LOOP, 0x03=FAILED)
-  /// Bit 2: STATUS_LED_WARNING
-  /// Bit 3: STATUS_LED_ERROR
-  /// Bits 4-7: Unused - reserved for future expansion (50% of the bits are free)
+  /// Bits 0-2: Component state (0x00=CONSTRUCTION, 0x01=SETUP, 0x02=LOOP, 0x03=FAILED, 0x04=LOOP_DONE)
+  /// Bit 3: STATUS_LED_WARNING
+  /// Bit 4: STATUS_LED_ERROR
+  /// Bits 5-7: Unused - reserved for future expansion
   uint8_t component_state_{0x00};
   volatile bool pending_enable_loop_{false};  ///< ISR-safe flag for enable_loop_soon_any_context
 };
diff --git a/esphome/core/component_iterator.cpp b/esphome/core/component_iterator.cpp
index d27c4e70ba..668c4a1fda 100644
--- a/esphome/core/component_iterator.cpp
+++ b/esphome/core/component_iterator.cpp
@@ -16,373 +16,173 @@ void ComponentIterator::begin(bool include_internal) {
   this->at_ = 0;
   this->include_internal_ = include_internal;
 }
+
+void ComponentIterator::advance_platform_() {
+  this->state_ = static_cast(static_cast(this->state_) + 1);
+  this->at_ = 0;
+}
+
 void ComponentIterator::advance() {
-  bool advance_platform = false;
-  bool success = true;
   switch (this->state_) {
     case IteratorState::NONE:
       // not started
       return;
     case IteratorState::BEGIN:
       if (this->on_begin()) {
-        advance_platform = true;
-      } else {
-        return;
+        advance_platform_();
       }
       break;
+
 #ifdef USE_BINARY_SENSOR
     case IteratorState::BINARY_SENSOR:
-      if (this->at_ >= App.get_binary_sensors().size()) {
-        advance_platform = true;
-      } else {
-        auto *binary_sensor = App.get_binary_sensors()[this->at_];
-        if (binary_sensor->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_binary_sensor(binary_sensor);
-        }
-      }
+      this->process_platform_item_(App.get_binary_sensors(), &ComponentIterator::on_binary_sensor);
       break;
 #endif
+
 #ifdef USE_COVER
     case IteratorState::COVER:
-      if (this->at_ >= App.get_covers().size()) {
-        advance_platform = true;
-      } else {
-        auto *cover = App.get_covers()[this->at_];
-        if (cover->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_cover(cover);
-        }
-      }
+      this->process_platform_item_(App.get_covers(), &ComponentIterator::on_cover);
       break;
 #endif
+
 #ifdef USE_FAN
     case IteratorState::FAN:
-      if (this->at_ >= App.get_fans().size()) {
-        advance_platform = true;
-      } else {
-        auto *fan = App.get_fans()[this->at_];
-        if (fan->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_fan(fan);
-        }
-      }
+      this->process_platform_item_(App.get_fans(), &ComponentIterator::on_fan);
       break;
 #endif
+
 #ifdef USE_LIGHT
     case IteratorState::LIGHT:
-      if (this->at_ >= App.get_lights().size()) {
-        advance_platform = true;
-      } else {
-        auto *light = App.get_lights()[this->at_];
-        if (light->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_light(light);
-        }
-      }
+      this->process_platform_item_(App.get_lights(), &ComponentIterator::on_light);
       break;
 #endif
+
 #ifdef USE_SENSOR
     case IteratorState::SENSOR:
-      if (this->at_ >= App.get_sensors().size()) {
-        advance_platform = true;
-      } else {
-        auto *sensor = App.get_sensors()[this->at_];
-        if (sensor->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_sensor(sensor);
-        }
-      }
+      this->process_platform_item_(App.get_sensors(), &ComponentIterator::on_sensor);
       break;
 #endif
+
 #ifdef USE_SWITCH
     case IteratorState::SWITCH:
-      if (this->at_ >= App.get_switches().size()) {
-        advance_platform = true;
-      } else {
-        auto *a_switch = App.get_switches()[this->at_];
-        if (a_switch->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_switch(a_switch);
-        }
-      }
+      this->process_platform_item_(App.get_switches(), &ComponentIterator::on_switch);
       break;
 #endif
+
 #ifdef USE_BUTTON
     case IteratorState::BUTTON:
-      if (this->at_ >= App.get_buttons().size()) {
-        advance_platform = true;
-      } else {
-        auto *button = App.get_buttons()[this->at_];
-        if (button->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_button(button);
-        }
-      }
+      this->process_platform_item_(App.get_buttons(), &ComponentIterator::on_button);
       break;
 #endif
+
 #ifdef USE_TEXT_SENSOR
     case IteratorState::TEXT_SENSOR:
-      if (this->at_ >= App.get_text_sensors().size()) {
-        advance_platform = true;
-      } else {
-        auto *text_sensor = App.get_text_sensors()[this->at_];
-        if (text_sensor->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_text_sensor(text_sensor);
-        }
-      }
+      this->process_platform_item_(App.get_text_sensors(), &ComponentIterator::on_text_sensor);
       break;
 #endif
+
 #ifdef USE_API_SERVICES
-    case IteratorState ::SERVICE:
-      if (this->at_ >= api::global_api_server->get_user_services().size()) {
-        advance_platform = true;
-      } else {
-        auto *service = api::global_api_server->get_user_services()[this->at_];
-        success = this->on_service(service);
-      }
+    case IteratorState::SERVICE:
+      this->process_platform_item_(api::global_api_server->get_user_services(), &ComponentIterator::on_service);
       break;
 #endif
+
 #ifdef USE_CAMERA
-    case IteratorState::CAMERA:
-      if (camera::Camera::instance() == nullptr) {
-        advance_platform = true;
-      } else {
-        if (camera::Camera::instance()->is_internal() && !this->include_internal_) {
-          advance_platform = success = true;
-          break;
-        } else {
-          advance_platform = success = this->on_camera(camera::Camera::instance());
-        }
+    case IteratorState::CAMERA: {
+      camera::Camera *camera_instance = camera::Camera::instance();
+      if (camera_instance != nullptr && (!camera_instance->is_internal() || this->include_internal_)) {
+        this->on_camera(camera_instance);
       }
-      break;
+      advance_platform_();
+    } break;
 #endif
+
 #ifdef USE_CLIMATE
     case IteratorState::CLIMATE:
-      if (this->at_ >= App.get_climates().size()) {
-        advance_platform = true;
-      } else {
-        auto *climate = App.get_climates()[this->at_];
-        if (climate->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_climate(climate);
-        }
-      }
+      this->process_platform_item_(App.get_climates(), &ComponentIterator::on_climate);
       break;
 #endif
+
 #ifdef USE_NUMBER
     case IteratorState::NUMBER:
-      if (this->at_ >= App.get_numbers().size()) {
-        advance_platform = true;
-      } else {
-        auto *number = App.get_numbers()[this->at_];
-        if (number->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_number(number);
-        }
-      }
+      this->process_platform_item_(App.get_numbers(), &ComponentIterator::on_number);
       break;
 #endif
+
 #ifdef USE_DATETIME_DATE
     case IteratorState::DATETIME_DATE:
-      if (this->at_ >= App.get_dates().size()) {
-        advance_platform = true;
-      } else {
-        auto *date = App.get_dates()[this->at_];
-        if (date->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_date(date);
-        }
-      }
+      this->process_platform_item_(App.get_dates(), &ComponentIterator::on_date);
       break;
 #endif
+
 #ifdef USE_DATETIME_TIME
     case IteratorState::DATETIME_TIME:
-      if (this->at_ >= App.get_times().size()) {
-        advance_platform = true;
-      } else {
-        auto *time = App.get_times()[this->at_];
-        if (time->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_time(time);
-        }
-      }
+      this->process_platform_item_(App.get_times(), &ComponentIterator::on_time);
       break;
 #endif
+
 #ifdef USE_DATETIME_DATETIME
     case IteratorState::DATETIME_DATETIME:
-      if (this->at_ >= App.get_datetimes().size()) {
-        advance_platform = true;
-      } else {
-        auto *datetime = App.get_datetimes()[this->at_];
-        if (datetime->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_datetime(datetime);
-        }
-      }
+      this->process_platform_item_(App.get_datetimes(), &ComponentIterator::on_datetime);
       break;
 #endif
+
 #ifdef USE_TEXT
     case IteratorState::TEXT:
-      if (this->at_ >= App.get_texts().size()) {
-        advance_platform = true;
-      } else {
-        auto *text = App.get_texts()[this->at_];
-        if (text->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_text(text);
-        }
-      }
+      this->process_platform_item_(App.get_texts(), &ComponentIterator::on_text);
       break;
 #endif
+
 #ifdef USE_SELECT
     case IteratorState::SELECT:
-      if (this->at_ >= App.get_selects().size()) {
-        advance_platform = true;
-      } else {
-        auto *select = App.get_selects()[this->at_];
-        if (select->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_select(select);
-        }
-      }
+      this->process_platform_item_(App.get_selects(), &ComponentIterator::on_select);
       break;
 #endif
+
 #ifdef USE_LOCK
     case IteratorState::LOCK:
-      if (this->at_ >= App.get_locks().size()) {
-        advance_platform = true;
-      } else {
-        auto *a_lock = App.get_locks()[this->at_];
-        if (a_lock->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_lock(a_lock);
-        }
-      }
+      this->process_platform_item_(App.get_locks(), &ComponentIterator::on_lock);
       break;
 #endif
+
 #ifdef USE_VALVE
     case IteratorState::VALVE:
-      if (this->at_ >= App.get_valves().size()) {
-        advance_platform = true;
-      } else {
-        auto *valve = App.get_valves()[this->at_];
-        if (valve->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_valve(valve);
-        }
-      }
+      this->process_platform_item_(App.get_valves(), &ComponentIterator::on_valve);
       break;
 #endif
+
 #ifdef USE_MEDIA_PLAYER
     case IteratorState::MEDIA_PLAYER:
-      if (this->at_ >= App.get_media_players().size()) {
-        advance_platform = true;
-      } else {
-        auto *media_player = App.get_media_players()[this->at_];
-        if (media_player->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_media_player(media_player);
-        }
-      }
+      this->process_platform_item_(App.get_media_players(), &ComponentIterator::on_media_player);
       break;
 #endif
+
 #ifdef USE_ALARM_CONTROL_PANEL
     case IteratorState::ALARM_CONTROL_PANEL:
-      if (this->at_ >= App.get_alarm_control_panels().size()) {
-        advance_platform = true;
-      } else {
-        auto *a_alarm_control_panel = App.get_alarm_control_panels()[this->at_];
-        if (a_alarm_control_panel->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_alarm_control_panel(a_alarm_control_panel);
-        }
-      }
+      this->process_platform_item_(App.get_alarm_control_panels(), &ComponentIterator::on_alarm_control_panel);
       break;
 #endif
+
 #ifdef USE_EVENT
     case IteratorState::EVENT:
-      if (this->at_ >= App.get_events().size()) {
-        advance_platform = true;
-      } else {
-        auto *event = App.get_events()[this->at_];
-        if (event->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_event(event);
-        }
-      }
+      this->process_platform_item_(App.get_events(), &ComponentIterator::on_event);
       break;
 #endif
+
 #ifdef USE_UPDATE
     case IteratorState::UPDATE:
-      if (this->at_ >= App.get_updates().size()) {
-        advance_platform = true;
-      } else {
-        auto *update = App.get_updates()[this->at_];
-        if (update->is_internal() && !this->include_internal_) {
-          success = true;
-          break;
-        } else {
-          success = this->on_update(update);
-        }
-      }
+      this->process_platform_item_(App.get_updates(), &ComponentIterator::on_update);
       break;
 #endif
+
     case IteratorState::MAX:
       if (this->on_end()) {
         this->state_ = IteratorState::NONE;
       }
       return;
   }
-
-  if (advance_platform) {
-    this->state_ = static_cast(static_cast(this->state_) + 1);
-    this->at_ = 0;
-  } else if (success) {
-    this->at_++;
-  }
 }
+
 bool ComponentIterator::on_end() { return true; }
 bool ComponentIterator::on_begin() { return true; }
 #ifdef USE_API_SERVICES
diff --git a/esphome/core/component_iterator.h b/esphome/core/component_iterator.h
index ea2c8004ac..641d42898a 100644
--- a/esphome/core/component_iterator.h
+++ b/esphome/core/component_iterator.h
@@ -168,9 +168,25 @@ class ComponentIterator {
     UPDATE,
 #endif
     MAX,
-  } state_{IteratorState::NONE};
+  };
   uint16_t at_{0};  // Supports up to 65,535 entities per type
+  IteratorState state_{IteratorState::NONE};
   bool include_internal_{false};
+
+  template
+  void process_platform_item_(const Container &items,
+                              bool (ComponentIterator::*on_item)(typename Container::value_type)) {
+    if (this->at_ >= items.size()) {
+      this->advance_platform_();
+    } else {
+      typename Container::value_type item = items[this->at_];
+      if ((item->is_internal() && !this->include_internal_) || (this->*on_item)(item)) {
+        this->at_++;
+      }
+    }
+  }
+
+  void advance_platform_();
 };
 
 }  // namespace esphome
diff --git a/esphome/core/config.py b/esphome/core/config.py
index f73369f28f..8a5876dbcf 100644
--- a/esphome/core/config.py
+++ b/esphome/core/config.py
@@ -39,7 +39,7 @@ from esphome.const import (
     PlatformFramework,
     __version__ as ESPHOME_VERSION,
 )
-from esphome.core import CORE, coroutine_with_priority
+from esphome.core import CORE, CoroPriority, coroutine_with_priority
 from esphome.helpers import (
     copy_file_if_changed,
     fnv1a_32bit_hash,
@@ -136,21 +136,21 @@ def validate_ids_and_references(config: ConfigType) -> ConfigType:
     return config
 
 
-def valid_include(value):
+def valid_include(value: str) -> str:
     # Look for "<...>" includes
     if value.startswith("<") and value.endswith(">"):
         return value
     try:
-        return cv.directory(value)
+        return str(cv.directory(value))
     except cv.Invalid:
         pass
-    value = cv.file_(value)
-    _, ext = os.path.splitext(value)
+    path = cv.file_(value)
+    ext = path.suffix
     if ext not in VALID_INCLUDE_EXTS:
         raise cv.Invalid(
             f"Include has invalid file extension {ext} - valid extensions are {', '.join(VALID_INCLUDE_EXTS)}"
         )
-    return value
+    return str(path)
 
 
 def valid_project_name(value: str):
@@ -200,7 +200,7 @@ CONFIG_SCHEMA = cv.All(
     cv.Schema(
         {
             cv.Required(CONF_NAME): cv.valid_name,
-            cv.Optional(CONF_FRIENDLY_NAME, ""): cv.string,
+            cv.Optional(CONF_FRIENDLY_NAME, ""): cv.All(cv.string, cv.Length(max=120)),
             cv.Optional(CONF_AREA): validate_area_config,
             cv.Optional(CONF_COMMENT): cv.string,
             cv.Required(CONF_BUILD_PATH): cv.string,
@@ -311,13 +311,13 @@ def preload_core_config(config, result) -> str:
     CORE.data[KEY_CORE] = {}
 
     if CONF_BUILD_PATH not in conf:
-        build_path = get_str_env("ESPHOME_BUILD_PATH", "build")
-        conf[CONF_BUILD_PATH] = os.path.join(build_path, CORE.name)
-    CORE.build_path = CORE.relative_internal_path(conf[CONF_BUILD_PATH])
+        build_path = Path(get_str_env("ESPHOME_BUILD_PATH", "build"))
+        conf[CONF_BUILD_PATH] = str(build_path / CORE.name)
+    CORE.build_path = CORE.data_dir / conf[CONF_BUILD_PATH]
 
     target_platforms = []
 
-    for domain, _ in config.items():
+    for domain in config:
         if domain.startswith("."):
             continue
         if _is_target_platform(domain):
@@ -339,12 +339,12 @@ def preload_core_config(config, result) -> str:
     return target_platforms[0]
 
 
-def include_file(path, basename):
-    parts = basename.split(os.path.sep)
+def include_file(path: Path, basename: Path):
+    parts = basename.parts
     dst = CORE.relative_src_path(*parts)
     copy_file_if_changed(path, dst)
 
-    _, ext = os.path.splitext(path)
+    ext = path.suffix
     if ext in [".h", ".hpp", ".tcc"]:
         # Header, add include statement
         cg.add_global(cg.RawStatement(f'#include "{basename}"'))
@@ -359,7 +359,7 @@ ARDUINO_GLUE_CODE = """\
 """
 
 
-@coroutine_with_priority(-999.0)
+@coroutine_with_priority(CoroPriority.WORKAROUNDS)
 async def add_arduino_global_workaround():
     # The Arduino framework defined these itself in the global
     # namespace. For the esphome codebase that is not a problem,
@@ -376,32 +376,32 @@ async def add_arduino_global_workaround():
         cg.add_global(cg.RawStatement(line))
 
 
-@coroutine_with_priority(-1000.0)
-async def add_includes(includes):
+@coroutine_with_priority(CoroPriority.FINAL)
+async def add_includes(includes: list[str]) -> None:
     # Add includes at the very end, so that the included files can access global variables
     for include in includes:
         path = CORE.relative_config_path(include)
-        if os.path.isdir(path):
+        if path.is_dir():
             # Directory, copy tree
             for p in walk_files(path):
-                basename = os.path.relpath(p, os.path.dirname(path))
+                basename = p.relative_to(path.parent)
                 include_file(p, basename)
         else:
             # Copy file
-            basename = os.path.basename(path)
+            basename = Path(path.name)
             include_file(path, basename)
 
 
-@coroutine_with_priority(-1000.0)
+@coroutine_with_priority(CoroPriority.FINAL)
 async def _add_platformio_options(pio_options):
     # Add includes at the very end, so that they override everything
     for key, val in pio_options.items():
-        if key == "build_flags" and not isinstance(val, list):
+        if key in ["build_flags", "lib_ignore"] and not isinstance(val, list):
             val = [val]
         cg.add_platformio_option(key, val)
 
 
-@coroutine_with_priority(30.0)
+@coroutine_with_priority(CoroPriority.AUTOMATION)
 async def _add_automations(config):
     for conf in config.get(CONF_ON_BOOT, []):
         trigger = cg.new_Pvariable(conf[CONF_TRIGGER_ID], conf.get(CONF_PRIORITY))
@@ -419,13 +419,30 @@ async def _add_automations(config):
         await automation.build_automation(trigger, [], conf)
 
 
-@coroutine_with_priority(-100.0)
-async def _add_platform_reserves() -> None:
+# Datetime component has special subtypes that need additional defines
+DATETIME_SUBTYPES = {"date", "time", "datetime"}
+
+
+@coroutine_with_priority(CoroPriority.FINAL)
+async def _add_platform_defines() -> None:
+    # Generate compile-time defines for platforms that have actual entities
+    # Only add USE_* and count defines when there are entities
     for platform_name, count in sorted(CORE.platform_counts.items()):
-        cg.add(cg.RawStatement(f"App.reserve_{platform_name}({count});"), prepend=True)
+        if count <= 0:
+            continue
+
+        define_name = f"ESPHOME_ENTITY_{platform_name.upper()}_COUNT"
+        cg.add_define(define_name, count)
+
+        # Datetime subtypes only use USE_DATETIME_* defines
+        if platform_name in DATETIME_SUBTYPES:
+            cg.add_define(f"USE_DATETIME_{platform_name.upper()}")
+        else:
+            # Regular platforms use USE_* defines
+            cg.add_define(f"USE_{platform_name.upper()}")
 
 
-@coroutine_with_priority(100.0)
+@coroutine_with_priority(CoroPriority.CORE)
 async def to_code(config: ConfigType) -> None:
     cg.add_global(cg.global_ns.namespace("esphome").using)
     # These can be used by user lambdas, put them to default scope
@@ -442,12 +459,10 @@ async def to_code(config: ConfigType) -> None:
             config[CONF_NAME_ADD_MAC_SUFFIX],
         )
     )
-    # Reserve space for components to avoid reallocation during registration
-    cg.add(
-        cg.RawStatement(f"App.reserve_components({len(CORE.component_ids)});"),
-    )
+    # Define component count for static allocation
+    cg.add_define("ESPHOME_COMPONENT_COUNT", len(CORE.component_ids))
 
-    CORE.add_job(_add_platform_reserves)
+    CORE.add_job(_add_platform_defines)
 
     CORE.add_job(_add_automations, config)
 
@@ -514,8 +529,8 @@ async def to_code(config: ConfigType) -> None:
     all_areas.extend(config[CONF_AREAS])
 
     if all_areas:
-        cg.add(cg.RawStatement(f"App.reserve_area({len(all_areas)});"))
         cg.add_define("USE_AREAS")
+        cg.add_define("ESPHOME_AREA_COUNT", len(all_areas))
 
         for area_conf in all_areas:
             area_id: core.ID = area_conf[CONF_ID]
@@ -532,9 +547,9 @@ async def to_code(config: ConfigType) -> None:
     if not devices:
         return
 
-    # Reserve space for devices
-    cg.add(cg.RawStatement(f"App.reserve_device({len(devices)});"))
+    # Define device count for static allocation
     cg.add_define("USE_DEVICES")
+    cg.add_define("ESPHOME_DEVICE_COUNT", len(devices))
 
     # Process each device
     for dev_conf in devices:
diff --git a/esphome/core/defines.h b/esphome/core/defines.h
index 8ed8f4b5aa..1afb296fc0 100644
--- a/esphome/core/defines.h
+++ b/esphome/core/defines.h
@@ -15,6 +15,9 @@
 #define ESPHOME_VARIANT "ESP32"
 #define ESPHOME_DEBUG_SCHEDULER
 
+// Default threading model for static analysis (ESP32 is multi-threaded with atomics)
+#define ESPHOME_THREAD_MULTI_ATOMICS
+
 // logger
 #define ESPHOME_LOG_LEVEL ESPHOME_LOG_LEVEL_VERY_VERBOSE
 
@@ -45,6 +48,7 @@
 #define USE_LIGHT
 #define USE_LOCK
 #define USE_LOGGER
+#define USE_LOGGER_RUNTIME_TAG_LEVELS
 #define USE_LVGL
 #define USE_LVGL_ANIMIMG
 #define USE_LVGL_ARC
@@ -79,6 +83,9 @@
 #define USE_LVGL_TILEVIEW
 #define USE_LVGL_TOUCHSCREEN
 #define USE_MDNS
+#define USE_MDNS_STORE_SERVICES
+#define MDNS_SERVICE_COUNT 3
+#define MDNS_DYNAMIC_TXT_COUNT 3
 #define USE_MEDIA_PLAYER
 #define USE_NEXTION_TFT_UPLOAD
 #define USE_NUMBER
@@ -97,6 +104,7 @@
 #define USE_UART_DEBUGGER
 #define USE_UPDATE
 #define USE_VALVE
+#define USE_ZWAVE_PROXY
 
 // Feature flags which do not work for zephyr
 #ifndef USE_ZEPHYR
@@ -106,17 +114,26 @@
 #define USE_API
 #define USE_API_CLIENT_CONNECTED_TRIGGER
 #define USE_API_CLIENT_DISCONNECTED_TRIGGER
+#define USE_API_HOMEASSISTANT_ACTION_RESPONSES
+#define USE_API_HOMEASSISTANT_ACTION_RESPONSES_JSON
+#define USE_API_HOMEASSISTANT_SERVICES
+#define USE_API_HOMEASSISTANT_STATES
 #define USE_API_NOISE
 #define USE_API_PLAINTEXT
 #define USE_API_SERVICES
+#define API_MAX_SEND_QUEUE 8
 #define USE_MD5
+#define USE_SHA256
 #define USE_MQTT
 #define USE_NETWORK
 #define USE_ONLINE_IMAGE_BMP_SUPPORT
 #define USE_ONLINE_IMAGE_PNG_SUPPORT
 #define USE_ONLINE_IMAGE_JPEG_SUPPORT
 #define USE_OTA
+#define USE_OTA_MD5
 #define USE_OTA_PASSWORD
+#define USE_OTA_SHA256
+#define ALLOW_OTA_DOWNGRADE_MD5
 #define USE_OTA_STATE_CALLBACK
 #define USE_OTA_VERSION 2
 #define USE_TIME_TIMEZONE
@@ -142,10 +159,31 @@
 #define USE_ESPHOME_TASK_LOG_BUFFER
 
 #define USE_BLUETOOTH_PROXY
+#define BLUETOOTH_PROXY_MAX_CONNECTIONS 3
+#define BLUETOOTH_PROXY_ADVERTISEMENT_BATCH_SIZE 16
 #define USE_CAPTIVE_PORTAL
 #define USE_ESP32_BLE
+#define USE_ESP32_BLE_MAX_CONNECTIONS 3
 #define USE_ESP32_BLE_CLIENT
+#define USE_ESP32_BLE_DEVICE
 #define USE_ESP32_BLE_SERVER
+#define USE_ESP32_BLE_UUID
+#define USE_ESP32_BLE_ADVERTISING
+#define USE_ESP32_BLE_SERVER_SET_VALUE_ACTION
+#define USE_ESP32_BLE_SERVER_DESCRIPTOR_SET_VALUE_ACTION
+#define USE_ESP32_BLE_SERVER_NOTIFY_ACTION
+#define USE_ESP32_BLE_SERVER_CHARACTERISTIC_ON_WRITE
+#define USE_ESP32_BLE_SERVER_DESCRIPTOR_ON_WRITE
+#define USE_ESP32_BLE_SERVER_ON_CONNECT
+#define USE_ESP32_BLE_SERVER_ON_DISCONNECT
+#define ESPHOME_ESP32_BLE_TRACKER_LISTENER_COUNT 1
+#define ESPHOME_ESP32_BLE_TRACKER_CLIENT_COUNT 1
+#define ESPHOME_ESP32_BLE_GAP_EVENT_HANDLER_COUNT 2
+#define ESPHOME_ESP32_BLE_GAP_SCAN_EVENT_HANDLER_COUNT 1
+#define ESPHOME_ESP32_BLE_GATTC_EVENT_HANDLER_COUNT 1
+#define ESPHOME_ESP32_BLE_GATTS_EVENT_HANDLER_COUNT 1
+#define ESPHOME_ESP32_BLE_BLE_STATUS_EVENT_HANDLER_COUNT 2
+#define USE_ESP32_CAMERA_JPEG_ENCODER
 #define USE_I2C
 #define USE_IMPROV
 #define USE_MICROPHONE
@@ -156,18 +194,20 @@
 #define USE_SPI
 #define USE_VOICE_ASSISTANT
 #define USE_WEBSERVER
+#define USE_WEBSERVER_AUTH
 #define USE_WEBSERVER_OTA
 #define USE_WEBSERVER_PORT 80  // NOLINT
 #define USE_WEBSERVER_SORTING
 #define USE_WIFI_11KV_SUPPORT
+#define USB_HOST_MAX_REQUESTS 16
 
 #ifdef USE_ARDUINO
-#define USE_ARDUINO_VERSION_CODE VERSION_CODE(3, 1, 3)
+#define USE_ARDUINO_VERSION_CODE VERSION_CODE(3, 3, 2)
 #define USE_ETHERNET
+#define USE_ETHERNET_KSZ8081
 #endif
 
 #ifdef USE_ESP_IDF
-#define USE_ESP_IDF_VERSION_CODE VERSION_CODE(5, 3, 2)
 #define USE_MICRO_WAKE_WORD
 #define USE_MICRO_WAKE_WORD_VAD
 #if defined(USE_ESP32_VARIANT_ESP32C6) || defined(USE_ESP32_VARIANT_ESP32H2)
@@ -204,6 +244,7 @@
   {}
 
 #define USE_WEBSERVER
+#define USE_WEBSERVER_AUTH
 #define USE_WEBSERVER_PORT 80  // NOLINT
 #endif
 
@@ -220,6 +261,7 @@
 #define USE_SOCKET_IMPL_LWIP_SOCKETS
 #define USE_SOCKET_SELECT_SUPPORT
 #define USE_WEBSERVER
+#define USE_WEBSERVER_AUTH
 #define USE_WEBSERVER_PORT 80  // NOLINT
 #endif
 
@@ -228,8 +270,38 @@
 #define USE_SOCKET_SELECT_SUPPORT
 #endif
 
+#ifdef USE_NRF52
+#define USE_NRF52_DFU
+#endif
+
 // Disabled feature flags
 // #define USE_BSEC   // Requires a library with proprietary license
 // #define USE_BSEC2  // Requires a library with proprietary license
 
 #define USE_DASHBOARD_IMPORT
+
+// Default counts for static analysis
+#define ESPHOME_COMPONENT_COUNT 50
+#define ESPHOME_DEVICE_COUNT 10
+#define ESPHOME_AREA_COUNT 10
+#define ESPHOME_ENTITY_ALARM_CONTROL_PANEL_COUNT 1
+#define ESPHOME_ENTITY_BINARY_SENSOR_COUNT 1
+#define ESPHOME_ENTITY_BUTTON_COUNT 1
+#define ESPHOME_ENTITY_CLIMATE_COUNT 1
+#define ESPHOME_ENTITY_COVER_COUNT 1
+#define ESPHOME_ENTITY_DATE_COUNT 1
+#define ESPHOME_ENTITY_DATETIME_COUNT 1
+#define ESPHOME_ENTITY_EVENT_COUNT 1
+#define ESPHOME_ENTITY_FAN_COUNT 1
+#define ESPHOME_ENTITY_LIGHT_COUNT 1
+#define ESPHOME_ENTITY_LOCK_COUNT 1
+#define ESPHOME_ENTITY_MEDIA_PLAYER_COUNT 1
+#define ESPHOME_ENTITY_NUMBER_COUNT 1
+#define ESPHOME_ENTITY_SELECT_COUNT 1
+#define ESPHOME_ENTITY_SENSOR_COUNT 1
+#define ESPHOME_ENTITY_SWITCH_COUNT 1
+#define ESPHOME_ENTITY_TEXT_COUNT 1
+#define ESPHOME_ENTITY_TEXT_SENSOR_COUNT 1
+#define ESPHOME_ENTITY_TIME_COUNT 1
+#define ESPHOME_ENTITY_UPDATE_COUNT 1
+#define ESPHOME_ENTITY_VALVE_COUNT 1
diff --git a/esphome/core/entity_base.cpp b/esphome/core/entity_base.cpp
index 2ea9c77a3e..4883c72cf1 100644
--- a/esphome/core/entity_base.cpp
+++ b/esphome/core/entity_base.cpp
@@ -1,6 +1,7 @@
 #include "esphome/core/entity_base.h"
 #include "esphome/core/application.h"
 #include "esphome/core/helpers.h"
+#include "esphome/core/string_ref.h"
 
 namespace esphome {
 
@@ -44,19 +45,29 @@ void EntityBase::set_icon(const char *icon) {
 #endif
 }
 
+// Check if the object_id is dynamic (changes with MAC suffix)
+bool EntityBase::is_object_id_dynamic_() const {
+  return !this->flags_.has_own_name && App.is_name_add_mac_suffix_enabled();
+}
+
 // Entity Object ID
 std::string EntityBase::get_object_id() const {
   // Check if `App.get_friendly_name()` is constant or dynamic.
-  if (!this->flags_.has_own_name && App.is_name_add_mac_suffix_enabled()) {
+  if (this->is_object_id_dynamic_()) {
     // `App.get_friendly_name()` is dynamic.
     return str_sanitize(str_snake_case(App.get_friendly_name()));
-  } else {
-    // `App.get_friendly_name()` is constant.
-    if (this->object_id_c_str_ == nullptr) {
-      return "";
-    }
-    return this->object_id_c_str_;
   }
+  // `App.get_friendly_name()` is constant.
+  return this->object_id_c_str_ == nullptr ? "" : this->object_id_c_str_;
+}
+StringRef EntityBase::get_object_id_ref_for_api_() const {
+  static constexpr auto EMPTY_STRING = StringRef::from_lit("");
+  // Return empty for dynamic case (MAC suffix)
+  if (this->is_object_id_dynamic_()) {
+    return EMPTY_STRING;
+  }
+  // For static case, return the string or empty if null
+  return this->object_id_c_str_ == nullptr ? EMPTY_STRING : StringRef(this->object_id_c_str_);
 }
 void EntityBase::set_object_id(const char *object_id) {
   this->object_id_c_str_ = object_id;
@@ -64,7 +75,10 @@ void EntityBase::set_object_id(const char *object_id) {
 }
 
 // Calculate Object ID Hash from Entity Name
-void EntityBase::calc_object_id_() { this->object_id_hash_ = fnv1_hash(this->get_object_id()); }
+void EntityBase::calc_object_id_() {
+  this->object_id_hash_ =
+      fnv1_hash(this->is_object_id_dynamic_() ? this->get_object_id().c_str() : this->object_id_c_str_);
+}
 
 uint32_t EntityBase::get_object_id_hash() { return this->object_id_hash_; }
 
diff --git a/esphome/core/entity_base.h b/esphome/core/entity_base.h
index 00b1264ed0..4a6460e708 100644
--- a/esphome/core/entity_base.h
+++ b/esphome/core/entity_base.h
@@ -12,6 +12,11 @@
 
 namespace esphome {
 
+// Forward declaration for friend access
+namespace api {
+class APIConnection;
+}  // namespace api
+
 enum EntityCategory : uint8_t {
   ENTITY_CATEGORY_NONE = 0,
   ENTITY_CATEGORY_CONFIG = 1,
@@ -54,6 +59,14 @@ class EntityBase {
   // Get/set this entity's icon
   std::string get_icon() const;
   void set_icon(const char *icon);
+  StringRef get_icon_ref() const {
+    static constexpr auto EMPTY_STRING = StringRef::from_lit("");
+#ifdef USE_ENTITY_ICON
+    return this->icon_c_str_ == nullptr ? EMPTY_STRING : StringRef(this->icon_c_str_);
+#else
+    return EMPTY_STRING;
+#endif
+  }
 
 #ifdef USE_DEVICES
   // Get/set this entity's device id
@@ -72,12 +85,50 @@ class EntityBase {
   // Set has_state - for components that need to manually set this
   void set_has_state(bool state) { this->flags_.has_state = state; }
 
+  /**
+   * @brief Get a unique hash for storing preferences/settings for this entity.
+   *
+   * This method returns a hash that uniquely identifies the entity for the purpose of
+   * storing preferences (such as calibration, state, etc.). Unlike get_object_id_hash(),
+   * this hash also incorporates the device_id (if devices are enabled), ensuring uniqueness
+   * across multiple devices that may have entities with the same object_id.
+   *
+   * Use this method when storing or retrieving preferences/settings that should be unique
+   * per device-entity pair. Use get_object_id_hash() when you need a hash that identifies
+   * the entity regardless of the device it belongs to.
+   *
+   * For backward compatibility, if device_id is 0 (the main device), the hash is unchanged
+   * from previous versions, so existing single-device configurations will continue to work.
+   *
+   * @return uint32_t The unique hash for preferences, including device_id if available.
+   */
+  uint32_t get_preference_hash() {
+#ifdef USE_DEVICES
+    // Combine object_id_hash with device_id to ensure uniqueness across devices
+    // Note: device_id is 0 for the main device, so XORing with 0 preserves the original hash
+    // This ensures backward compatibility for existing single-device configurations
+    return this->get_object_id_hash() ^ this->get_device_id();
+#else
+    // Without devices, just use object_id_hash as before
+    return this->get_object_id_hash();
+#endif
+  }
+
  protected:
+  friend class api::APIConnection;
+
+  // Get object_id as StringRef when it's static (for API usage)
+  // Returns empty StringRef if object_id is dynamic (needs allocation)
+  StringRef get_object_id_ref_for_api_() const;
+
   /// The hash_base() function has been deprecated. It is kept in this
   /// class for now, to prevent external components from not compiling.
   virtual uint32_t hash_base() { return 0L; }
   void calc_object_id_();
 
+  /// Check if the object_id is dynamic (changes with MAC suffix)
+  bool is_object_id_dynamic_() const;
+
   StringRef name_;
   const char *object_id_c_str_{nullptr};
 #ifdef USE_ENTITY_ICON
@@ -105,6 +156,11 @@ class EntityBase_DeviceClass {  // NOLINT(readability-identifier-naming)
   std::string get_device_class();
   /// Manually set the device class.
   void set_device_class(const char *device_class);
+  /// Get the device class as StringRef
+  StringRef get_device_class_ref() const {
+    static constexpr auto EMPTY_STRING = StringRef::from_lit("");
+    return this->device_class_ == nullptr ? EMPTY_STRING : StringRef(this->device_class_);
+  }
 
  protected:
   const char *device_class_{nullptr};  ///< Device class override
@@ -116,6 +172,11 @@ class EntityBase_UnitOfMeasurement {  // NOLINT(readability-identifier-naming)
   std::string get_unit_of_measurement();
   /// Manually set the unit of measurement.
   void set_unit_of_measurement(const char *unit_of_measurement);
+  /// Get the unit of measurement as StringRef
+  StringRef get_unit_of_measurement_ref() const {
+    static constexpr auto EMPTY_STRING = StringRef::from_lit("");
+    return this->unit_of_measurement_ == nullptr ? EMPTY_STRING : StringRef(this->unit_of_measurement_);
+  }
 
  protected:
   const char *unit_of_measurement_{nullptr};  ///< Unit of measurement override
diff --git a/esphome/core/entity_helpers.py b/esphome/core/entity_helpers.py
index 5ad16ac76c..f0a04b4860 100644
--- a/esphome/core/entity_helpers.py
+++ b/esphome/core/entity_helpers.py
@@ -16,7 +16,7 @@ from esphome.core import CORE, ID
 from esphome.cpp_generator import MockObj, add, get_variable
 import esphome.final_validate as fv
 from esphome.helpers import sanitize, snake_case
-from esphome.types import ConfigType
+from esphome.types import ConfigType, EntityMetadata
 
 _LOGGER = logging.getLogger(__name__)
 
@@ -77,8 +77,8 @@ async def setup_entity(var: MockObj, config: ConfigType, platform: str) -> None:
     """
     # Get device info
     device_name: str | None = None
-    if CONF_DEVICE_ID in config:
-        device_id_obj: ID = config[CONF_DEVICE_ID]
+    device_id_obj: ID | None
+    if device_id_obj := config.get(CONF_DEVICE_ID):
         device: MockObj = await get_variable(device_id_obj)
         add(var.set_device(device))
         # Get device name for object ID calculation
@@ -198,9 +198,12 @@ def entity_duplicate_validator(platform: str) -> Callable[[ConfigType], ConfigTy
 
         # Get device name if entity is on a sub-device
         device_name = None
-        if CONF_DEVICE_ID in config:
-            device_id_obj = config[CONF_DEVICE_ID]
+        device_id = ""  # Empty string for main device
+        device_id_obj: ID | None
+        if device_id_obj := config.get(CONF_DEVICE_ID):
             device_name = device_id_obj.id
+            # Use the device ID string directly for uniqueness
+            device_id = device_id_obj.id
 
         # Calculate what object_id will actually be used
         # This handles empty names correctly by using device/friendly names
@@ -209,15 +212,61 @@ def entity_duplicate_validator(platform: str) -> Callable[[ConfigType], ConfigTy
         )
 
         # Check for duplicates
-        unique_key = (platform, name_key)
+        unique_key = (device_id, platform, name_key)
         if unique_key in CORE.unique_ids:
-            raise cv.Invalid(
-                f"Duplicate {platform} entity with name '{entity_name}' found. "
-                f"Each entity must have a unique name within its platform across all devices."
-            )
+            # Get the existing entity metadata
+            existing = CORE.unique_ids[unique_key]
+            existing_name = existing.get("name", entity_name)
+            existing_device = existing.get("device_id", "")
+            existing_id = existing.get("entity_id", "unknown")
 
-        # Add to tracking set
-        CORE.unique_ids.add(unique_key)
+            # Build detailed error message
+            device_prefix = f" on device '{device_id}'" if device_id else ""
+            existing_device_prefix = (
+                f" on device '{existing_device}'" if existing_device else ""
+            )
+            existing_component = existing.get("component", "unknown")
+
+            # Provide more context about where the duplicate was found
+            conflict_msg = (
+                f"Conflicts with entity '{existing_name}'{existing_device_prefix}"
+            )
+            if existing_id != "unknown":
+                conflict_msg += f" (id: {existing_id})"
+            if existing_component != "unknown":
+                conflict_msg += f" from component '{existing_component}'"
+
+            # Show both original names and their ASCII-only versions if they differ
+            sanitized_msg = ""
+            if entity_name != existing_name:
+                sanitized_msg = (
+                    f"\n  Original names: '{entity_name}' and '{existing_name}'"
+                    f"\n  Both convert to ASCII ID: '{name_key}'"
+                    "\n  To fix: Add unique ASCII characters (e.g., '1', '2', or 'A', 'B')"
+                    "\n          to distinguish them"
+                )
+
+            # Skip duplicate entity name validation when testing_mode is enabled
+            # This flag is used for grouped component testing
+            if not CORE.testing_mode:
+                raise cv.Invalid(
+                    f"Duplicate {platform} entity with name '{entity_name}' found{device_prefix}. "
+                    f"{conflict_msg}. "
+                    "Each entity on a device must have a unique name within its platform."
+                    f"{sanitized_msg}"
+                )
+
+        # Store metadata about this entity
+        entity_metadata: EntityMetadata = {
+            "name": entity_name,
+            "device_id": device_id,
+            "platform": platform,
+            "entity_id": str(config.get(CONF_ID, "unknown")),
+            "component": CORE.current_component or "unknown",
+        }
+
+        # Add to tracking dict
+        CORE.unique_ids[unique_key] = entity_metadata
         return config
 
     return validator
diff --git a/esphome/core/event_pool.h b/esphome/core/event_pool.h
index 69e03bafac..928a4e7dee 100644
--- a/esphome/core/event_pool.h
+++ b/esphome/core/event_pool.h
@@ -1,6 +1,6 @@
 #pragma once
 
-#if defined(USE_ESP32) || defined(USE_LIBRETINY)
+#if defined(USE_ESP32)
 
 #include 
 #include 
@@ -78,4 +78,4 @@ template class EventPool {
 
 }  // namespace esphome
 
-#endif  // defined(USE_ESP32) || defined(USE_LIBRETINY)
+#endif  // defined(USE_ESP32)
diff --git a/esphome/core/hash_base.h b/esphome/core/hash_base.h
new file mode 100644
index 0000000000..c45c4df70b
--- /dev/null
+++ b/esphome/core/hash_base.h
@@ -0,0 +1,56 @@
+#pragma once
+
+#include 
+#include 
+#include 
+#include "esphome/core/helpers.h"
+
+namespace esphome {
+
+/// Base class for hash algorithms
+class HashBase {
+ public:
+  virtual ~HashBase() = default;
+
+  /// Initialize a new hash computation
+  virtual void init() = 0;
+
+  /// Add bytes of data for the hash
+  virtual void add(const uint8_t *data, size_t len) = 0;
+  void add(const char *data, size_t len) { this->add((const uint8_t *) data, len); }
+
+  /// Compute the hash based on provided data
+  virtual void calculate() = 0;
+
+  /// Retrieve the hash as bytes
+  void get_bytes(uint8_t *output) { memcpy(output, this->digest_, this->get_size()); }
+
+  /// Retrieve the hash as hex characters
+  void get_hex(char *output) {
+    for (size_t i = 0; i < this->get_size(); i++) {
+      uint8_t byte = this->digest_[i];
+      output[i * 2] = format_hex_char(byte >> 4);
+      output[i * 2 + 1] = format_hex_char(byte & 0x0F);
+    }
+  }
+
+  /// Compare the hash against a provided byte-encoded hash
+  bool equals_bytes(const uint8_t *expected) { return memcmp(this->digest_, expected, this->get_size()) == 0; }
+
+  /// Compare the hash against a provided hex-encoded hash
+  bool equals_hex(const char *expected) {
+    uint8_t parsed[32];  // Fixed size for max hash (SHA256 = 32 bytes)
+    if (!parse_hex(expected, parsed, this->get_size())) {
+      return false;
+    }
+    return this->equals_bytes(parsed);
+  }
+
+  /// Get the size of the hash in bytes (16 for MD5, 32 for SHA256)
+  virtual size_t get_size() const = 0;
+
+ protected:
+  uint8_t digest_[32];  // Storage sized for max(MD5=16, SHA256=32) bytes
+};
+
+}  // namespace esphome
diff --git a/esphome/core/helpers.cpp b/esphome/core/helpers.cpp
index e84f5a7317..fb8b220b2f 100644
--- a/esphome/core/helpers.cpp
+++ b/esphome/core/helpers.cpp
@@ -3,6 +3,7 @@
 #include "esphome/core/defines.h"
 #include "esphome/core/hal.h"
 #include "esphome/core/log.h"
+#include "esphome/core/string_ref.h"
 
 #include 
 #include 
@@ -41,17 +42,28 @@ static const uint16_t CRC16_1021_BE_LUT_H[] = {0x0000, 0x1231, 0x2462, 0x3653, 0
 
 // Mathematics
 
-uint8_t crc8(const uint8_t *data, uint8_t len) {
-  uint8_t crc = 0;
-
+uint8_t crc8(const uint8_t *data, uint8_t len, uint8_t crc, uint8_t poly, bool msb_first) {
   while ((len--) != 0u) {
     uint8_t inbyte = *data++;
-    for (uint8_t i = 8; i != 0u; i--) {
-      bool mix = (crc ^ inbyte) & 0x01;
-      crc >>= 1;
-      if (mix)
-        crc ^= 0x8C;
-      inbyte >>= 1;
+    if (msb_first) {
+      // MSB first processing (for polynomials like 0x31, 0x07)
+      crc ^= inbyte;
+      for (uint8_t i = 8; i != 0u; i--) {
+        if (crc & 0x80) {
+          crc = (crc << 1) ^ poly;
+        } else {
+          crc <<= 1;
+        }
+      }
+    } else {
+      // LSB first processing (default for Dallas/Maxim 0x8C)
+      for (uint8_t i = 8; i != 0u; i--) {
+        bool mix = (crc ^ inbyte) & 0x01;
+        crc >>= 1;
+        if (mix)
+          crc ^= poly;
+        inbyte >>= 1;
+      }
     }
   }
   return crc;
@@ -131,11 +143,13 @@ uint16_t crc16be(const uint8_t *data, uint16_t len, uint16_t crc, uint16_t poly,
   return refout ? (crc ^ 0xffff) : crc;
 }
 
-uint32_t fnv1_hash(const std::string &str) {
+uint32_t fnv1_hash(const char *str) {
   uint32_t hash = 2166136261UL;
-  for (char c : str) {
-    hash *= 16777619UL;
-    hash ^= c;
+  if (str) {
+    while (*str) {
+      hash *= 16777619UL;
+      hash ^= *str++;
+    }
   }
   return hash;
 }
@@ -221,6 +235,30 @@ std::string str_sprintf(const char *fmt, ...) {
   return str;
 }
 
+// Maximum size for name with suffix: 120 (max friendly name) + 1 (separator) + 6 (MAC suffix) + 1 (null term)
+static constexpr size_t MAX_NAME_WITH_SUFFIX_SIZE = 128;
+
+std::string make_name_with_suffix(const std::string &name, char sep, const char *suffix_ptr, size_t suffix_len) {
+  char buffer[MAX_NAME_WITH_SUFFIX_SIZE];
+  size_t name_len = name.size();
+  size_t total_len = name_len + 1 + suffix_len;
+
+  // Silently truncate if needed: prioritize keeping the full suffix
+  if (total_len >= MAX_NAME_WITH_SUFFIX_SIZE) {
+    // NOTE: This calculation could underflow if suffix_len >= MAX_NAME_WITH_SUFFIX_SIZE - 2,
+    // but this is safe because this helper is only called with small suffixes:
+    // MAC suffixes (6-12 bytes), ".local" (5 bytes), etc.
+    name_len = MAX_NAME_WITH_SUFFIX_SIZE - suffix_len - 2;  // -2 for separator and null terminator
+    total_len = name_len + 1 + suffix_len;
+  }
+
+  memcpy(buffer, name.c_str(), name_len);
+  buffer[name_len] = sep;
+  memcpy(buffer + name_len + 1, suffix_ptr, suffix_len);
+  buffer[total_len] = '\0';
+  return std::string(buffer, total_len);
+}
+
 // Parsing & formatting
 
 size_t parse_hex(const char *str, size_t length, uint8_t *data, size_t count) {
@@ -242,23 +280,22 @@ size_t parse_hex(const char *str, size_t length, uint8_t *data, size_t count) {
 }
 
 std::string format_mac_address_pretty(const uint8_t *mac) {
-  return str_snprintf("%02X:%02X:%02X:%02X:%02X:%02X", 17, mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
+  char buf[18];
+  format_mac_addr_upper(mac, buf);
+  return std::string(buf);
 }
 
-static char format_hex_char(uint8_t v) { return v >= 10 ? 'a' + (v - 10) : '0' + v; }
 std::string format_hex(const uint8_t *data, size_t length) {
   std::string ret;
   ret.resize(length * 2);
   for (size_t i = 0; i < length; i++) {
-    ret[2 * i] = format_hex_char((data[i] & 0xF0) >> 4);
+    ret[2 * i] = format_hex_char(data[i] >> 4);
     ret[2 * i + 1] = format_hex_char(data[i] & 0x0F);
   }
   return ret;
 }
 std::string format_hex(const std::vector &data) { return format_hex(data.data(), data.size()); }
 
-static char format_hex_pretty_char(uint8_t v) { return v >= 10 ? 'A' + (v - 10) : '0' + v; }
-
 // Shared implementation for uint8_t and string hex formatting
 static std::string format_hex_pretty_uint8(const uint8_t *data, size_t length, char separator, bool show_length) {
   if (data == nullptr || length == 0)
@@ -267,7 +304,7 @@ static std::string format_hex_pretty_uint8(const uint8_t *data, size_t length, c
   uint8_t multiple = separator ? 3 : 2;  // 3 if separator is not \0, 2 otherwise
   ret.resize(multiple * length - (separator ? 1 : 0));
   for (size_t i = 0; i < length; i++) {
-    ret[multiple * i] = format_hex_pretty_char((data[i] & 0xF0) >> 4);
+    ret[multiple * i] = format_hex_pretty_char(data[i] >> 4);
     ret[multiple * i + 1] = format_hex_pretty_char(data[i] & 0x0F);
     if (separator && i != length - 1)
       ret[multiple * i + 2] = separator;
@@ -336,17 +373,34 @@ ParseOnOffState parse_on_off(const char *str, const char *on, const char *off) {
   return PARSE_NONE;
 }
 
-std::string value_accuracy_to_string(float value, int8_t accuracy_decimals) {
+static inline void normalize_accuracy_decimals(float &value, int8_t &accuracy_decimals) {
   if (accuracy_decimals < 0) {
     auto multiplier = powf(10.0f, accuracy_decimals);
     value = roundf(value * multiplier) / multiplier;
     accuracy_decimals = 0;
   }
+}
+
+std::string value_accuracy_to_string(float value, int8_t accuracy_decimals) {
+  normalize_accuracy_decimals(value, accuracy_decimals);
   char tmp[32];  // should be enough, but we should maybe improve this at some point.
   snprintf(tmp, sizeof(tmp), "%.*f", accuracy_decimals, value);
   return std::string(tmp);
 }
 
+std::string value_accuracy_with_uom_to_string(float value, int8_t accuracy_decimals, StringRef unit_of_measurement) {
+  normalize_accuracy_decimals(value, accuracy_decimals);
+  // Buffer sized for float (up to ~15 chars) + space + typical UOM (usually <20 chars like "μS/cm")
+  // snprintf truncates safely if exceeded, though ESPHome UOMs are typically short
+  char tmp[64];
+  if (unit_of_measurement.empty()) {
+    snprintf(tmp, sizeof(tmp), "%.*f", accuracy_decimals, value);
+  } else {
+    snprintf(tmp, sizeof(tmp), "%.*f %s", accuracy_decimals, value, unit_of_measurement.c_str());
+  }
+  return std::string(tmp);
+}
+
 int8_t step_to_accuracy_decimals(float step) {
   // use printf %g to find number of digits based on temperature step
   char buf[32];
@@ -578,7 +632,9 @@ bool HighFrequencyLoopRequester::is_high_frequency() { return num_requests > 0;
 std::string get_mac_address() {
   uint8_t mac[6];
   get_mac_address_raw(mac);
-  return str_snprintf("%02x%02x%02x%02x%02x%02x", 12, mac[0], mac[1], mac[2], mac[3], mac[4], mac[5]);
+  char buf[13];
+  format_mac_addr_lower_no_sep(mac, buf);
+  return std::string(buf);
 }
 
 std::string get_mac_address_pretty() {
@@ -599,8 +655,6 @@ bool mac_address_is_valid(const uint8_t *mac) {
     if (mac[i] != 0) {
       is_all_zeros = false;
     }
-  }
-  for (uint8_t i = 0; i < 6; i++) {
     if (mac[i] != 0xFF) {
       is_all_ones = false;
     }
diff --git a/esphome/core/helpers.h b/esphome/core/helpers.h
index c3b404ae60..326718e974 100644
--- a/esphome/core/helpers.h
+++ b/esphome/core/helpers.h
@@ -1,9 +1,11 @@
 #pragma once
 
+#include 
 #include 
 #include 
 #include 
 #include 
+#include 
 #include 
 #include 
 #include 
@@ -43,6 +45,9 @@
 
 namespace esphome {
 
+// Forward declaration to avoid circular dependency with string_ref.h
+class StringRef;
+
 /// @name STL backports
 ///@{
 
@@ -67,7 +72,10 @@ To bit_cast(const From &src) {
   return dst;
 }
 #endif
-using std::lerp;
+
+// clang-format off
+inline float lerp(float completion, float start, float end) = delete;  // Please use std::lerp. Notice that it has different order on arguments!
+// clang-format on
 
 // std::byteswap from C++23
 template constexpr T byteswap(T n) {
@@ -77,6 +85,16 @@ template constexpr T byteswap(T n) {
   return m;
 }
 template<> constexpr uint8_t byteswap(uint8_t n) { return n; }
+#ifdef USE_LIBRETINY
+// LibreTiny's Beken framework redefines __builtin_bswap functions as non-constexpr
+template<> inline uint16_t byteswap(uint16_t n) { return __builtin_bswap16(n); }
+template<> inline uint32_t byteswap(uint32_t n) { return __builtin_bswap32(n); }
+template<> inline uint64_t byteswap(uint64_t n) { return __builtin_bswap64(n); }
+template<> inline int8_t byteswap(int8_t n) { return n; }
+template<> inline int16_t byteswap(int16_t n) { return __builtin_bswap16(n); }
+template<> inline int32_t byteswap(int32_t n) { return __builtin_bswap32(n); }
+template<> inline int64_t byteswap(int64_t n) { return __builtin_bswap64(n); }
+#else
 template<> constexpr uint16_t byteswap(uint16_t n) { return __builtin_bswap16(n); }
 template<> constexpr uint32_t byteswap(uint32_t n) { return __builtin_bswap32(n); }
 template<> constexpr uint64_t byteswap(uint64_t n) { return __builtin_bswap64(n); }
@@ -84,6 +102,215 @@ template<> constexpr int8_t byteswap(int8_t n) { return n; }
 template<> constexpr int16_t byteswap(int16_t n) { return __builtin_bswap16(n); }
 template<> constexpr int32_t byteswap(int32_t n) { return __builtin_bswap32(n); }
 template<> constexpr int64_t byteswap(int64_t n) { return __builtin_bswap64(n); }
+#endif
+
+///@}
+
+/// @name Container utilities
+///@{
+
+/// Minimal static vector - saves memory by avoiding std::vector overhead
+template class StaticVector {
+ public:
+  using value_type = T;
+  using iterator = typename std::array::iterator;
+  using const_iterator = typename std::array::const_iterator;
+  using reverse_iterator = std::reverse_iterator;
+  using const_reverse_iterator = std::reverse_iterator;
+
+ private:
+  std::array data_{};
+  size_t count_{0};
+
+ public:
+  // Minimal vector-compatible interface - only what we actually use
+  void push_back(const T &value) {
+    if (count_ < N) {
+      data_[count_++] = value;
+    }
+  }
+
+  // Return reference to next element and increment count (with bounds checking)
+  T &emplace_next() {
+    if (count_ >= N) {
+      // Should never happen with proper size calculation
+      // Return reference to last element to avoid crash
+      return data_[N - 1];
+    }
+    return data_[count_++];
+  }
+
+  size_t size() const { return count_; }
+  bool empty() const { return count_ == 0; }
+
+  T &operator[](size_t i) { return data_[i]; }
+  const T &operator[](size_t i) const { return data_[i]; }
+
+  // For range-based for loops
+  iterator begin() { return data_.begin(); }
+  iterator end() { return data_.begin() + count_; }
+  const_iterator begin() const { return data_.begin(); }
+  const_iterator end() const { return data_.begin() + count_; }
+
+  // Reverse iterators
+  reverse_iterator rbegin() { return reverse_iterator(end()); }
+  reverse_iterator rend() { return reverse_iterator(begin()); }
+  const_reverse_iterator rbegin() const { return const_reverse_iterator(end()); }
+  const_reverse_iterator rend() const { return const_reverse_iterator(begin()); }
+};
+
+/// Fixed-capacity vector - allocates once at runtime, never reallocates
+/// This avoids std::vector template overhead (_M_realloc_insert, _M_default_append)
+/// when size is known at initialization but not at compile time
+template class FixedVector {
+ private:
+  T *data_{nullptr};
+  size_t size_{0};
+  size_t capacity_{0};
+
+  // Helper to destroy all elements without freeing memory
+  void destroy_elements_() {
+    // Only call destructors for non-trivially destructible types
+    if constexpr (!std::is_trivially_destructible::value) {
+      for (size_t i = 0; i < size_; i++) {
+        data_[i].~T();
+      }
+    }
+  }
+
+  // Helper to destroy elements and free memory
+  void cleanup_() {
+    if (data_ != nullptr) {
+      destroy_elements_();
+      // Free raw memory
+      ::operator delete(data_);
+    }
+  }
+
+  // Helper to reset pointers after cleanup
+  void reset_() {
+    data_ = nullptr;
+    capacity_ = 0;
+    size_ = 0;
+  }
+
+ public:
+  FixedVector() = default;
+
+  /// Constructor from initializer list - allocates exact size needed
+  /// This enables brace initialization: FixedVector v = {1, 2, 3};
+  FixedVector(std::initializer_list init_list) {
+    init(init_list.size());
+    size_t idx = 0;
+    for (const auto &item : init_list) {
+      new (data_ + idx) T(item);
+      ++idx;
+    }
+    size_ = init_list.size();
+  }
+
+  ~FixedVector() { cleanup_(); }
+
+  // Disable copy operations (avoid accidental expensive copies)
+  FixedVector(const FixedVector &) = delete;
+  FixedVector &operator=(const FixedVector &) = delete;
+
+  // Enable move semantics (allows use in move-only containers like std::vector)
+  FixedVector(FixedVector &&other) noexcept : data_(other.data_), size_(other.size_), capacity_(other.capacity_) {
+    other.reset_();
+  }
+
+  FixedVector &operator=(FixedVector &&other) noexcept {
+    if (this != &other) {
+      // Delete our current data
+      cleanup_();
+      // Take ownership of other's data
+      data_ = other.data_;
+      size_ = other.size_;
+      capacity_ = other.capacity_;
+      // Leave other in valid empty state
+      other.reset_();
+    }
+    return *this;
+  }
+
+  // Allocate capacity - can be called multiple times to reinit
+  void init(size_t n) {
+    cleanup_();
+    reset_();
+    if (n > 0) {
+      // Allocate raw memory without calling constructors
+      // sizeof(T) is correct here for any type T (value types, pointers, etc.)
+      // NOLINTNEXTLINE(bugprone-sizeof-expression)
+      data_ = static_cast(::operator new(n * sizeof(T)));
+      capacity_ = n;
+    }
+  }
+
+  // Clear the vector (destroy all elements, reset size to 0, keep capacity)
+  void clear() {
+    destroy_elements_();
+    size_ = 0;
+  }
+
+  // Shrink capacity to fit current size (frees all memory)
+  void shrink_to_fit() {
+    cleanup_();
+    reset_();
+  }
+
+  /// Add element without bounds checking
+  /// Caller must ensure sufficient capacity was allocated via init()
+  /// Silently ignores pushes beyond capacity (no exception or assertion)
+  void push_back(const T &value) {
+    if (size_ < capacity_) {
+      // Use placement new to construct the object in pre-allocated memory
+      new (&data_[size_]) T(value);
+      size_++;
+    }
+  }
+
+  /// Add element by move without bounds checking
+  /// Caller must ensure sufficient capacity was allocated via init()
+  /// Silently ignores pushes beyond capacity (no exception or assertion)
+  void push_back(T &&value) {
+    if (size_ < capacity_) {
+      // Use placement new to move-construct the object in pre-allocated memory
+      new (&data_[size_]) T(std::move(value));
+      size_++;
+    }
+  }
+
+  /// Emplace element without bounds checking - constructs in-place
+  /// Caller must ensure sufficient capacity was allocated via init()
+  /// Returns reference to the newly constructed element
+  /// NOTE: Caller MUST ensure size_ < capacity_ before calling
+  T &emplace_back() {
+    // Use placement new to default-construct the object in pre-allocated memory
+    new (&data_[size_]) T();
+    size_++;
+    return data_[size_ - 1];
+  }
+
+  /// Access last element (no bounds checking - matches std::vector behavior)
+  /// Caller must ensure vector is not empty (size() > 0)
+  T &back() { return data_[size_ - 1]; }
+  const T &back() const { return data_[size_ - 1]; }
+
+  size_t size() const { return size_; }
+  bool empty() const { return size_ == 0; }
+
+  /// Access element without bounds checking (matches std::vector behavior)
+  /// Caller must ensure index is valid (i < size())
+  T &operator[](size_t i) { return data_[i]; }
+  const T &operator[](size_t i) const { return data_[i]; }
+
+  // Iterator support for range-based for loops
+  T *begin() { return data_; }
+  T *end() { return data_ + size_; }
+  const T *begin() const { return data_; }
+  const T *end() const { return data_ + size_; }
+};
 
 ///@}
 
@@ -95,8 +322,8 @@ template T remap(U value, U min, U max, T min_out, T max
   return (value - min) * (max_out - min_out) / (max - min) + min_out;
 }
 
-/// Calculate a CRC-8 checksum of \p data with size \p len using the CRC-8-Dallas/Maxim polynomial.
-uint8_t crc8(const uint8_t *data, uint8_t len);
+/// Calculate a CRC-8 checksum of \p data with size \p len.
+uint8_t crc8(const uint8_t *data, uint8_t len, uint8_t crc = 0x00, uint8_t poly = 0x8C, bool msb_first = false);
 
 /// Calculate a CRC-16 checksum of \p data with size \p len.
 uint16_t crc16(const uint8_t *data, uint16_t len, uint16_t crc = 0xffff, uint16_t reverse_poly = 0xa001,
@@ -105,7 +332,8 @@ uint16_t crc16be(const uint8_t *data, uint16_t len, uint16_t crc = 0, uint16_t p
                  bool refout = false);
 
 /// Calculate a FNV-1 hash of \p str.
-uint32_t fnv1_hash(const std::string &str);
+uint32_t fnv1_hash(const char *str);
+inline uint32_t fnv1_hash(const std::string &str) { return fnv1_hash(str.c_str()); }
 
 /// Return a random 32-bit unsigned integer.
 uint32_t random_uint32();
@@ -231,6 +459,16 @@ std::string __attribute__((format(printf, 1, 3))) str_snprintf(const char *fmt,
 /// sprintf-like function returning std::string.
 std::string __attribute__((format(printf, 1, 2))) str_sprintf(const char *fmt, ...);
 
+/// Concatenate a name with a separator and suffix using an efficient stack-based approach.
+/// This avoids multiple heap allocations during string construction.
+/// Maximum name length supported is 120 characters for friendly names.
+/// @param name The base name string
+/// @param sep The separator character (e.g., '-', ' ', or '.')
+/// @param suffix_ptr Pointer to the suffix characters
+/// @param suffix_len Length of the suffix
+/// @return The concatenated string: name + sep + suffix
+std::string make_name_with_suffix(const std::string &name, char sep, const char *suffix_ptr, size_t suffix_len);
+
 ///@}
 
 /// @name Parsing & formatting
@@ -329,6 +567,35 @@ template::value, int> = 0> optional<
   return parse_hex(str.c_str(), str.length());
 }
 
+/// Convert a nibble (0-15) to lowercase hex char
+inline char format_hex_char(uint8_t v) { return v >= 10 ? 'a' + (v - 10) : '0' + v; }
+
+/// Convert a nibble (0-15) to uppercase hex char (used for pretty printing)
+/// This always uses uppercase (A-F) for pretty/human-readable output
+inline char format_hex_pretty_char(uint8_t v) { return v >= 10 ? 'A' + (v - 10) : '0' + v; }
+
+/// Format MAC address as XX:XX:XX:XX:XX:XX (uppercase)
+inline void format_mac_addr_upper(const uint8_t *mac, char *output) {
+  for (size_t i = 0; i < 6; i++) {
+    uint8_t byte = mac[i];
+    output[i * 3] = format_hex_pretty_char(byte >> 4);
+    output[i * 3 + 1] = format_hex_pretty_char(byte & 0x0F);
+    if (i < 5)
+      output[i * 3 + 2] = ':';
+  }
+  output[17] = '\0';
+}
+
+/// Format MAC address as xxxxxxxxxxxxxx (lowercase, no separators)
+inline void format_mac_addr_lower_no_sep(const uint8_t *mac, char *output) {
+  for (size_t i = 0; i < 6; i++) {
+    uint8_t byte = mac[i];
+    output[i * 2] = format_hex_char(byte >> 4);
+    output[i * 2 + 1] = format_hex_char(byte & 0x0F);
+  }
+  output[12] = '\0';
+}
+
 /// Format the six-byte array \p mac into a MAC address.
 std::string format_mac_address_pretty(const uint8_t mac[6]);
 /// Format the byte array \p data of length \p len in lowercased hex.
@@ -509,6 +776,8 @@ ParseOnOffState parse_on_off(const char *str, const char *on = nullptr, const ch
 
 /// Create a string from a value and an accuracy in decimals.
 std::string value_accuracy_to_string(float value, int8_t accuracy_decimals);
+/// Create a string from a value, an accuracy in decimals, and a unit of measurement.
+std::string value_accuracy_with_uom_to_string(float value, int8_t accuracy_decimals, StringRef unit_of_measurement);
 
 /// Derive accuracy in decimals from an increment step.
 int8_t step_to_accuracy_decimals(float step);
@@ -577,21 +846,28 @@ template class CallbackManager {
 /// Helper class to deduplicate items in a series of values.
 template class Deduplicator {
  public:
-  /// Feeds the next item in the series to the deduplicator and returns whether this is a duplicate.
+  /// Feeds the next item in the series to the deduplicator and returns false if this is a duplicate.
   bool next(T value) {
-    if (this->has_value_) {
-      if (this->last_value_ == value)
-        return false;
+    if (this->has_value_ && !this->value_unknown_ && this->last_value_ == value) {
+      return false;
     }
     this->has_value_ = true;
+    this->value_unknown_ = false;
     this->last_value_ = value;
     return true;
   }
-  /// Returns whether this deduplicator has processed any items so far.
+  /// Returns true if the deduplicator's value was previously known.
+  bool next_unknown() {
+    bool ret = !this->value_unknown_;
+    this->value_unknown_ = true;
+    return ret;
+  }
+  /// Returns true if this deduplicator has processed any items.
   bool has_value() const { return this->has_value_; }
 
  protected:
   bool has_value_{false};
+  bool value_unknown_{false};
   T last_value_{};
 };
 
@@ -678,11 +954,28 @@ class InterruptLock {
   ~InterruptLock();
 
  protected:
-#if defined(USE_ESP8266) || defined(USE_RP2040)
+#if defined(USE_ESP8266) || defined(USE_RP2040) || defined(USE_ZEPHYR)
   uint32_t state_;
 #endif
 };
 
+/** Helper class to lock the lwIP TCPIP core when making lwIP API calls from non-TCPIP threads.
+ *
+ * This is needed on multi-threaded platforms (ESP32) when CONFIG_LWIP_TCPIP_CORE_LOCKING is enabled.
+ * It ensures thread-safe access to lwIP APIs.
+ *
+ * @note This follows the same pattern as InterruptLock - platform-specific implementations in helpers.cpp
+ */
+class LwIPLock {
+ public:
+  LwIPLock();
+  ~LwIPLock();
+
+  // Delete copy constructor and copy assignment operator to prevent accidental copying
+  LwIPLock(const LwIPLock &) = delete;
+  LwIPLock &operator=(const LwIPLock &) = delete;
+};
+
 /** Helper class to request `loop()` to be called as fast as possible.
  *
  * Usually the ESPHome main loop runs at 60 Hz, sleeping in between invocations of `loop()` if necessary. When a higher
diff --git a/esphome/core/lock_free_queue.h b/esphome/core/lock_free_queue.h
index f35cfa5af9..68e2825d09 100644
--- a/esphome/core/lock_free_queue.h
+++ b/esphome/core/lock_free_queue.h
@@ -1,17 +1,12 @@
 #pragma once
 
-#if defined(USE_ESP32) || defined(USE_LIBRETINY)
+#if defined(USE_ESP32)
 
 #include 
 #include 
 
-#if defined(USE_ESP32)
 #include 
 #include 
-#elif defined(USE_LIBRETINY)
-#include 
-#include 
-#endif
 
 /*
  * Lock-free queue for single-producer single-consumer scenarios.
@@ -34,7 +29,7 @@ namespace esphome {
 // Base lock-free queue without task notification
 template class LockFreeQueue {
  public:
-  LockFreeQueue() : head_(0), tail_(0), dropped_count_(0) {}
+  LockFreeQueue() : dropped_count_(0), head_(0), tail_(0) {}
 
   bool push(T *element) {
     bool was_empty;
@@ -148,4 +143,4 @@ template class NotifyingLockFreeQueue : public LockFreeQu
 
 }  // namespace esphome
 
-#endif  // defined(USE_ESP32) || defined(USE_LIBRETINY)
+#endif  // defined(USE_ESP32)
diff --git a/esphome/core/ring_buffer.cpp b/esphome/core/ring_buffer.cpp
index b77a02b2a7..6a2232599f 100644
--- a/esphome/core/ring_buffer.cpp
+++ b/esphome/core/ring_buffer.cpp
@@ -78,9 +78,13 @@ size_t RingBuffer::write(const void *data, size_t len) {
   return this->write_without_replacement(data, len, 0);
 }
 
-size_t RingBuffer::write_without_replacement(const void *data, size_t len, TickType_t ticks_to_wait) {
+size_t RingBuffer::write_without_replacement(const void *data, size_t len, TickType_t ticks_to_wait,
+                                             bool write_partial) {
   if (!xRingbufferSend(this->handle_, data, len, ticks_to_wait)) {
-    // Couldn't fit all the data, so only write what will fit
+    if (!write_partial) {
+      return 0;  // Not enough space available and not allowed to write partial data
+    }
+    // Couldn't fit all the data, write what will fit
     size_t free = std::min(this->free(), len);
     if (xRingbufferSend(this->handle_, data, free, 0)) {
       return free;
diff --git a/esphome/core/ring_buffer.h b/esphome/core/ring_buffer.h
index bad96d3181..98a273781f 100644
--- a/esphome/core/ring_buffer.h
+++ b/esphome/core/ring_buffer.h
@@ -50,7 +50,8 @@ class RingBuffer {
    * @param ticks_to_wait Maximum number of FreeRTOS ticks to wait (default: 0)
    * @return Number of bytes written
    */
-  size_t write_without_replacement(const void *data, size_t len, TickType_t ticks_to_wait = 0);
+  size_t write_without_replacement(const void *data, size_t len, TickType_t ticks_to_wait = 0,
+                                   bool write_partial = true);
 
   /**
    * @brief Returns the number of available bytes in the ring buffer.
diff --git a/esphome/core/scheduler.cpp b/esphome/core/scheduler.cpp
index c6893b128f..402084f306 100644
--- a/esphome/core/scheduler.cpp
+++ b/esphome/core/scheduler.cpp
@@ -8,12 +8,29 @@
 #include 
 #include 
 #include 
+#include 
 
 namespace esphome {
 
 static const char *const TAG = "scheduler";
 
-static const uint32_t MAX_LOGICALLY_DELETED_ITEMS = 10;
+// Memory pool configuration constants
+// Pool size of 5 matches typical usage patterns (2-4 active timers)
+// - Minimal memory overhead (~250 bytes on ESP32)
+// - Sufficient for most configs with a couple sensors/components
+// - Still prevents heap fragmentation and allocation stalls
+// - Complex setups with many timers will just allocate beyond the pool
+// See https://github.com/esphome/backlog/issues/52
+static constexpr size_t MAX_POOL_SIZE = 5;
+
+// Maximum number of logically deleted (cancelled) items before forcing cleanup.
+// Set to 5 to match the pool size - when we have as many cancelled items as our
+// pool can hold, it's time to clean up and recycle them.
+static constexpr uint32_t MAX_LOGICALLY_DELETED_ITEMS = 5;
+// Half the 32-bit range - used to detect rollovers vs normal time progression
+static constexpr uint32_t HALF_MAX_UINT32 = std::numeric_limits::max() / 2;
+// max delay to start an interval sequence
+static constexpr uint32_t MAX_INTERVAL_DELAY = 5000;
 
 // Uncomment to debug scheduler
 // #define ESPHOME_DEBUG_SCHEDULER
@@ -51,7 +68,7 @@ static void validate_static_string(const char *name) {
     ESP_LOGW(TAG, "WARNING: Scheduler name '%s' at %p might be on heap (static ref at %p)", name, name, static_str);
   }
 }
-#endif
+#endif /* ESPHOME_DEBUG_SCHEDULER */
 
 // A note on locking: the `lock_` lock protects the `items_` and `to_add_` containers. It must be taken when writing to
 // them (i.e. when adding/removing items, but not when changing items). As items are only deleted from the loop task,
@@ -60,48 +77,79 @@ static void validate_static_string(const char *name) {
 
 // Common implementation for both timeout and interval
 void HOT Scheduler::set_timer_common_(Component *component, SchedulerItem::Type type, bool is_static_string,
-                                      const void *name_ptr, uint32_t delay, std::function func) {
+                                      const void *name_ptr, uint32_t delay, std::function func, bool is_retry,
+                                      bool skip_cancel) {
   // Get the name as const char*
   const char *name_cstr = this->get_name_cstr_(is_static_string, name_ptr);
 
   if (delay == SCHEDULER_DONT_RUN) {
     // Still need to cancel existing timer if name is not empty
-    LockGuard guard{this->lock_};
-    this->cancel_item_locked_(component, name_cstr, type);
+    if (!skip_cancel) {
+      LockGuard guard{this->lock_};
+      this->cancel_item_locked_(component, name_cstr, type);
+    }
     return;
   }
 
+  // Get fresh timestamp BEFORE taking lock - millis_64_ may need to acquire lock itself
+  const uint64_t now = this->millis_64_(millis());
+
+  // Take lock early to protect scheduler_item_pool_ access
+  LockGuard guard{this->lock_};
+
   // Create and populate the scheduler item
-  auto item = make_unique();
+  std::unique_ptr item;
+  if (!this->scheduler_item_pool_.empty()) {
+    // Reuse from pool
+    item = std::move(this->scheduler_item_pool_.back());
+    this->scheduler_item_pool_.pop_back();
+#ifdef ESPHOME_DEBUG_SCHEDULER
+    ESP_LOGD(TAG, "Reused item from pool (pool size now: %zu)", this->scheduler_item_pool_.size());
+#endif
+  } else {
+    // Allocate new if pool is empty
+    item = make_unique();
+#ifdef ESPHOME_DEBUG_SCHEDULER
+    ESP_LOGD(TAG, "Allocated new item (pool empty)");
+#endif
+  }
   item->component = component;
   item->set_name(name_cstr, !is_static_string);
   item->type = type;
   item->callback = std::move(func);
+  // Initialize remove to false (though it should already be from constructor)
+#ifdef ESPHOME_THREAD_MULTI_ATOMICS
+  item->remove.store(false, std::memory_order_relaxed);
+#else
   item->remove = false;
+#endif
+  item->is_retry = is_retry;
 
-#if !defined(USE_ESP8266) && !defined(USE_RP2040)
+#ifndef ESPHOME_THREAD_SINGLE
   // Special handling for defer() (delay = 0, type = TIMEOUT)
-  // ESP8266 and RP2040 are excluded because they don't need thread-safe defer handling
+  // Single-core platforms don't need thread-safe defer handling
   if (delay == 0 && type == SchedulerItem::TIMEOUT) {
     // Put in defer queue for guaranteed FIFO execution
-    LockGuard guard{this->lock_};
-    this->cancel_item_locked_(component, name_cstr, type);
+    if (!skip_cancel) {
+      this->cancel_item_locked_(component, name_cstr, type);
+    }
     this->defer_queue_.push_back(std::move(item));
     return;
   }
-#endif
-
-  const auto now = this->millis_();
+#endif /* not ESPHOME_THREAD_SINGLE */
 
   // Type-specific setup
   if (type == SchedulerItem::INTERVAL) {
     item->interval = delay;
-    // Calculate random offset (0 to interval/2)
-    uint32_t offset = (delay != 0) ? (random_uint32() % delay) / 2 : 0;
-    item->next_execution_ = now + offset;
+    // first execution happens immediately after a random smallish offset
+    // Calculate random offset (0 to min(interval/2, 5s))
+    uint32_t offset = (uint32_t) (std::min(delay / 2, MAX_INTERVAL_DELAY) * random_float());
+    item->set_next_execution(now + offset);
+    ESP_LOGV(TAG, "Scheduler interval for %s is %" PRIu32 "ms, offset %" PRIu32 "ms", name_cstr ? name_cstr : "", delay,
+             offset);
   } else {
     item->interval = 0;
-    item->next_execution_ = now + delay;
+    item->set_next_execution(now + delay);
   }
 
 #ifdef ESPHOME_DEBUG_SCHEDULER
@@ -113,18 +161,31 @@ void HOT Scheduler::set_timer_common_(Component *component, SchedulerItem::Type
   // Debug logging
   const char *type_str = (type == SchedulerItem::TIMEOUT) ? "timeout" : "interval";
   if (type == SchedulerItem::TIMEOUT) {
-    ESP_LOGD(TAG, "set_%s(name='%s/%s', %s=%" PRIu32 ")", type_str, item->get_source(),
+    ESP_LOGD(TAG, "set_%s(name='%s/%s', %s=%" PRIu32 ")", type_str, LOG_STR_ARG(item->get_source()),
              name_cstr ? name_cstr : "(null)", type_str, delay);
   } else {
-    ESP_LOGD(TAG, "set_%s(name='%s/%s', %s=%" PRIu32 ", offset=%" PRIu32 ")", type_str, item->get_source(),
-             name_cstr ? name_cstr : "(null)", type_str, delay, static_cast(item->next_execution_ - now));
+    ESP_LOGD(TAG, "set_%s(name='%s/%s', %s=%" PRIu32 ", offset=%" PRIu32 ")", type_str, LOG_STR_ARG(item->get_source()),
+             name_cstr ? name_cstr : "(null)", type_str, delay,
+             static_cast(item->get_next_execution() - now));
   }
-#endif
+#endif /* ESPHOME_DEBUG_SCHEDULER */
 
-  LockGuard guard{this->lock_};
-  // If name is provided, do atomic cancel-and-add
+  // For retries, check if there's a cancelled timeout first
+  if (is_retry && name_cstr != nullptr && type == SchedulerItem::TIMEOUT &&
+      (has_cancelled_timeout_in_container_(this->items_, component, name_cstr, /* match_retry= */ true) ||
+       has_cancelled_timeout_in_container_(this->to_add_, component, name_cstr, /* match_retry= */ true))) {
+    // Skip scheduling - the retry was cancelled
+#ifdef ESPHOME_DEBUG_SCHEDULER
+    ESP_LOGD(TAG, "Skipping retry '%s' - found cancelled item", name_cstr);
+#endif
+    return;
+  }
+
+  // If name is provided, do atomic cancel-and-add (unless skip_cancel is true)
   // Cancel existing items
-  this->cancel_item_locked_(component, name_cstr, type);
+  if (!skip_cancel) {
+    this->cancel_item_locked_(component, name_cstr, type);
+  }
   // Add new item directly to to_add_
   // since we have the lock held
   this->to_add_.push_back(std::move(item));
@@ -170,32 +231,34 @@ struct RetryArgs {
   Scheduler *scheduler;
 };
 
-static void retry_handler(const std::shared_ptr &args) {
+void retry_handler(const std::shared_ptr &args) {
   RetryResult const retry_result = args->func(--args->retry_countdown);
   if (retry_result == RetryResult::DONE || args->retry_countdown <= 0)
     return;
   // second execution of `func` happens after `initial_wait_time`
-  args->scheduler->set_timeout(args->component, args->name, args->current_interval, [args]() { retry_handler(args); });
+  args->scheduler->set_timer_common_(
+      args->component, Scheduler::SchedulerItem::TIMEOUT, false, &args->name, args->current_interval,
+      [args]() { retry_handler(args); }, /* is_retry= */ true);
   // backoff_increase_factor applied to third & later executions
   args->current_interval *= args->backoff_increase_factor;
 }
 
-void HOT Scheduler::set_retry(Component *component, const std::string &name, uint32_t initial_wait_time,
-                              uint8_t max_attempts, std::function func,
-                              float backoff_increase_factor) {
-  if (!name.empty())
-    this->cancel_retry(component, name);
+void HOT Scheduler::set_retry_common_(Component *component, bool is_static_string, const void *name_ptr,
+                                      uint32_t initial_wait_time, uint8_t max_attempts,
+                                      std::function func, float backoff_increase_factor) {
+  const char *name_cstr = this->get_name_cstr_(is_static_string, name_ptr);
+
+  if (name_cstr != nullptr)
+    this->cancel_retry(component, name_cstr);
 
   if (initial_wait_time == SCHEDULER_DONT_RUN)
     return;
 
   ESP_LOGVV(TAG, "set_retry(name='%s', initial_wait_time=%" PRIu32 ", max_attempts=%u, backoff_factor=%0.1f)",
-            name.c_str(), initial_wait_time, max_attempts, backoff_increase_factor);
+            name_cstr ? name_cstr : "", initial_wait_time, max_attempts, backoff_increase_factor);
 
   if (backoff_increase_factor < 0.0001) {
-    ESP_LOGE(TAG,
-             "set_retry(name='%s'): backoff_factor cannot be close to zero nor negative (%0.1f). Using 1.0 instead",
-             name.c_str(), backoff_increase_factor);
+    ESP_LOGE(TAG, "backoff_factor %0.1f too small, using 1.0: %s", backoff_increase_factor, name_cstr ? name_cstr : "");
     backoff_increase_factor = 1;
   }
 
@@ -204,31 +267,57 @@ void HOT Scheduler::set_retry(Component *component, const std::string &name, uin
   args->retry_countdown = max_attempts;
   args->current_interval = initial_wait_time;
   args->component = component;
-  args->name = "retry$" + name;
+  args->name = name_cstr ? name_cstr : "";  // Convert to std::string for RetryArgs
   args->backoff_increase_factor = backoff_increase_factor;
   args->scheduler = this;
 
-  // First execution of `func` immediately
-  this->set_timeout(component, args->name, 0, [args]() { retry_handler(args); });
-}
-bool HOT Scheduler::cancel_retry(Component *component, const std::string &name) {
-  return this->cancel_timeout(component, "retry$" + name);
+  // First execution of `func` immediately - use set_timer_common_ with is_retry=true
+  this->set_timer_common_(
+      component, SchedulerItem::TIMEOUT, false, &args->name, 0, [args]() { retry_handler(args); },
+      /* is_retry= */ true);
 }
 
-optional HOT Scheduler::next_schedule_in() {
-  // IMPORTANT: This method should only be called from the main thread (loop task).
-  // It calls empty_() and accesses items_[0] without holding a lock, which is only
-  // safe when called from the main thread. Other threads must not call this method.
-  if (this->empty_())
-    return {};
-  auto &item = this->items_[0];
-  const auto now = this->millis_();
-  if (item->next_execution_ < now)
-    return 0;
-  return item->next_execution_ - now;
+void HOT Scheduler::set_retry(Component *component, const std::string &name, uint32_t initial_wait_time,
+                              uint8_t max_attempts, std::function func,
+                              float backoff_increase_factor) {
+  this->set_retry_common_(component, false, &name, initial_wait_time, max_attempts, std::move(func),
+                          backoff_increase_factor);
 }
-void HOT Scheduler::call() {
-#if !defined(USE_ESP8266) && !defined(USE_RP2040)
+
+void HOT Scheduler::set_retry(Component *component, const char *name, uint32_t initial_wait_time, uint8_t max_attempts,
+                              std::function func, float backoff_increase_factor) {
+  this->set_retry_common_(component, true, name, initial_wait_time, max_attempts, std::move(func),
+                          backoff_increase_factor);
+}
+bool HOT Scheduler::cancel_retry(Component *component, const std::string &name) {
+  return this->cancel_retry(component, name.c_str());
+}
+
+bool HOT Scheduler::cancel_retry(Component *component, const char *name) {
+  // Cancel timeouts that have is_retry flag set
+  LockGuard guard{this->lock_};
+  return this->cancel_item_locked_(component, name, SchedulerItem::TIMEOUT, /* match_retry= */ true);
+}
+
+optional HOT Scheduler::next_schedule_in(uint32_t now) {
+  // IMPORTANT: This method should only be called from the main thread (loop task).
+  // It performs cleanup and accesses items_[0] without holding a lock, which is only
+  // safe when called from the main thread. Other threads must not call this method.
+
+  // If no items, return empty optional
+  if (this->cleanup_() == 0)
+    return {};
+
+  auto &item = this->items_[0];
+  // Convert the fresh timestamp from caller (usually Application::loop()) to 64-bit
+  const auto now_64 = this->millis_64_(now);  // 'now' from parameter - fresh from caller
+  const uint64_t next_exec = item->get_next_execution();
+  if (next_exec < now_64)
+    return 0;
+  return next_exec - now_64;
+}
+void HOT Scheduler::call(uint32_t now) {
+#ifndef ESPHOME_THREAD_SINGLE
   // Process defer queue first to guarantee FIFO execution order for deferred items.
   // Previously, defer() used the heap which gave undefined order for equal timestamps,
   // causing race conditions on multi-core systems (ESP32, BK7200).
@@ -236,8 +325,7 @@ void HOT Scheduler::call() {
   // - Deferred items (delay=0) go directly to defer_queue_ in set_timer_common_
   // - Items execute in exact order they were deferred (FIFO guarantee)
   // - No deferred items exist in to_add_, so processing order doesn't affect correctness
-  // ESP8266 and RP2040 don't use this queue - they fall back to the heap-based approach
-  // (ESP8266: single-core, RP2040: empty mutex implementation).
+  // Single-core platforms don't use this queue and fall back to the heap-based approach.
   //
   // Note: Items cancelled via cancel_item_locked_() are marked with remove=true but still
   // processed here. They are removed from the queue normally via pop_front() but skipped
@@ -256,23 +344,38 @@ void HOT Scheduler::call() {
     // Execute callback without holding lock to prevent deadlocks
     // if the callback tries to call defer() again
     if (!this->should_skip_item_(item.get())) {
-      this->execute_item_(item.get());
+      now = this->execute_item_(item.get(), now);
     }
+    // Recycle the defer item after execution
+    this->recycle_item_(std::move(item));
   }
-#endif
+#endif /* not ESPHOME_THREAD_SINGLE */
 
-  const auto now = this->millis_();
+  // Convert the fresh timestamp from main loop to 64-bit for scheduler operations
+  const auto now_64 = this->millis_64_(now);  // 'now' from parameter - fresh from Application::loop()
   this->process_to_add();
 
+  // Track if any items were added to to_add_ during this call (intervals or from callbacks)
+  bool has_added_items = false;
+
 #ifdef ESPHOME_DEBUG_SCHEDULER
   static uint64_t last_print = 0;
 
-  if (now - last_print > 2000) {
-    last_print = now;
+  if (now_64 - last_print > 2000) {
+    last_print = now_64;
     std::vector> old_items;
-    ESP_LOGD(TAG, "Items: count=%zu, now=%" PRIu64 " (%u, %" PRIu32 ")", this->items_.size(), now, this->millis_major_,
-             this->last_millis_);
-    while (!this->empty_()) {
+#ifdef ESPHOME_THREAD_MULTI_ATOMICS
+    const auto last_dbg = this->last_millis_.load(std::memory_order_relaxed);
+    const auto major_dbg = this->millis_major_.load(std::memory_order_relaxed);
+    ESP_LOGD(TAG, "Items: count=%zu, pool=%zu, now=%" PRIu64 " (%" PRIu16 ", %" PRIu32 ")", this->items_.size(),
+             this->scheduler_item_pool_.size(), now_64, major_dbg, last_dbg);
+#else  /* not ESPHOME_THREAD_MULTI_ATOMICS */
+    ESP_LOGD(TAG, "Items: count=%zu, pool=%zu, now=%" PRIu64 " (%" PRIu16 ", %" PRIu32 ")", this->items_.size(),
+             this->scheduler_item_pool_.size(), now_64, this->millis_major_, this->last_millis_);
+#endif /* else ESPHOME_THREAD_MULTI_ATOMICS */
+    // Cleanup before debug output
+    this->cleanup_();
+    while (!this->items_.empty()) {
       std::unique_ptr item;
       {
         LockGuard guard{this->lock_};
@@ -281,9 +384,10 @@ void HOT Scheduler::call() {
       }
 
       const char *name = item->get_name();
-      ESP_LOGD(TAG, "  %s '%s/%s' interval=%" PRIu32 " next_execution in %" PRIu64 "ms at %" PRIu64,
-               item->get_type_str(), item->get_source(), name ? name : "(null)", item->interval,
-               item->next_execution_ - now, item->next_execution_);
+      bool is_cancelled = is_item_removed_(item.get());
+      ESP_LOGD(TAG, "  %s '%s/%s' interval=%" PRIu32 " next_execution in %" PRIu64 "ms at %" PRIu64 "%s",
+               item->get_type_str(), LOG_STR_ARG(item->get_source()), name ? name : "(null)", item->interval,
+               item->get_next_execution() - now_64, item->get_next_execution(), is_cancelled ? " [CANCELLED]" : "");
 
       old_items.push_back(std::move(item));
     }
@@ -296,10 +400,15 @@ void HOT Scheduler::call() {
       std::make_heap(this->items_.begin(), this->items_.end(), SchedulerItem::cmp);
     }
   }
-#endif  // ESPHOME_DEBUG_SCHEDULER
+#endif /* ESPHOME_DEBUG_SCHEDULER */
 
-  // If we have too many items to remove
-  if (this->to_remove_ > MAX_LOGICALLY_DELETED_ITEMS) {
+  // Cleanup removed items before processing
+  // First try to clean items from the top of the heap (fast path)
+  this->cleanup_();
+
+  // If we still have too many cancelled items, do a full cleanup
+  // This only happens if cancelled items are stuck in the middle/bottom of the heap
+  if (this->to_remove_ >= MAX_LOGICALLY_DELETED_ITEMS) {
     // We hold the lock for the entire cleanup operation because:
     // 1. We're rebuilding the entire items_ list, so we need exclusive access throughout
     // 2. Other threads must see either the old state or the new state, not intermediate states
@@ -309,10 +418,13 @@ void HOT Scheduler::call() {
 
     std::vector> valid_items;
 
-    // Move all non-removed items to valid_items
+    // Move all non-removed items to valid_items, recycle removed ones
     for (auto &item : this->items_) {
-      if (!item->remove) {
+      if (!is_item_removed_(item.get())) {
         valid_items.push_back(std::move(item));
+      } else {
+        // Recycle removed items
+        this->recycle_item_(std::move(item));
       }
     }
 
@@ -322,65 +434,92 @@ void HOT Scheduler::call() {
     std::make_heap(this->items_.begin(), this->items_.end(), SchedulerItem::cmp);
     this->to_remove_ = 0;
   }
-
-  while (!this->empty_()) {
-    // use scoping to indicate visibility of `item` variable
-    {
-      // Don't copy-by value yet
-      auto &item = this->items_[0];
-      if (item->next_execution_ > now) {
-        // Not reached timeout yet, done for this call
-        break;
-      }
-      // Don't run on failed components
-      if (item->component != nullptr && item->component->is_failed()) {
-        LockGuard guard{this->lock_};
-        this->pop_raw_();
-        continue;
-      }
-#ifdef ESPHOME_DEBUG_SCHEDULER
-      const char *item_name = item->get_name();
-      ESP_LOGV(TAG, "Running %s '%s/%s' with interval=%" PRIu32 " next_execution=%" PRIu64 " (now=%" PRIu64 ")",
-               item->get_type_str(), item->get_source(), item_name ? item_name : "(null)", item->interval,
-               item->next_execution_, now);
-#endif
-
-      // Warning: During callback(), a lot of stuff can happen, including:
-      //  - timeouts/intervals get added, potentially invalidating vector pointers
-      //  - timeouts/intervals get cancelled
-      this->execute_item_(item.get());
+  while (!this->items_.empty()) {
+    // Don't copy-by value yet
+    auto &item = this->items_[0];
+    if (item->get_next_execution() > now_64) {
+      // Not reached timeout yet, done for this call
+      break;
+    }
+    // Don't run on failed components
+    if (item->component != nullptr && item->component->is_failed()) {
+      LockGuard guard{this->lock_};
+      this->pop_raw_();
+      continue;
     }
 
+    // Check if item is marked for removal
+    // This handles two cases:
+    // 1. Item was marked for removal after cleanup_() but before we got here
+    // 2. Item is marked for removal but wasn't at the front of the heap during cleanup_()
+#ifdef ESPHOME_THREAD_MULTI_NO_ATOMICS
+    // Multi-threaded platforms without atomics: must take lock to safely read remove flag
     {
       LockGuard guard{this->lock_};
-
-      // new scope, item from before might have been moved in the vector
-      auto item = std::move(this->items_[0]);
-      // Only pop after function call, this ensures we were reachable
-      // during the function call and know if we were cancelled.
-      this->pop_raw_();
-
-      if (item->remove) {
-        // We were removed/cancelled in the function call, stop
+      if (is_item_removed_(item.get())) {
+        this->pop_raw_();
         this->to_remove_--;
         continue;
       }
-
-      if (item->type == SchedulerItem::INTERVAL) {
-        item->next_execution_ = now + item->interval;
-        // Add new item directly to to_add_
-        // since we have the lock held
-        this->to_add_.push_back(std::move(item));
-      }
     }
+#else
+    // Single-threaded or multi-threaded with atomics: can check without lock
+    if (is_item_removed_(item.get())) {
+      LockGuard guard{this->lock_};
+      this->pop_raw_();
+      this->to_remove_--;
+      continue;
+    }
+#endif
+
+#ifdef ESPHOME_DEBUG_SCHEDULER
+    const char *item_name = item->get_name();
+    ESP_LOGV(TAG, "Running %s '%s/%s' with interval=%" PRIu32 " next_execution=%" PRIu64 " (now=%" PRIu64 ")",
+             item->get_type_str(), LOG_STR_ARG(item->get_source()), item_name ? item_name : "(null)", item->interval,
+             item->get_next_execution(), now_64);
+#endif /* ESPHOME_DEBUG_SCHEDULER */
+
+    // Warning: During callback(), a lot of stuff can happen, including:
+    //  - timeouts/intervals get added, potentially invalidating vector pointers
+    //  - timeouts/intervals get cancelled
+    now = this->execute_item_(item.get(), now);
+
+    LockGuard guard{this->lock_};
+
+    auto executed_item = std::move(this->items_[0]);
+    // Only pop after function call, this ensures we were reachable
+    // during the function call and know if we were cancelled.
+    this->pop_raw_();
+
+    if (executed_item->remove) {
+      // We were removed/cancelled in the function call, stop
+      this->to_remove_--;
+      continue;
+    }
+
+    if (executed_item->type == SchedulerItem::INTERVAL) {
+      executed_item->set_next_execution(now_64 + executed_item->interval);
+      // Add new item directly to to_add_
+      // since we have the lock held
+      this->to_add_.push_back(std::move(executed_item));
+    } else {
+      // Timeout completed - recycle it
+      this->recycle_item_(std::move(executed_item));
+    }
+
+    has_added_items |= !this->to_add_.empty();
   }
 
-  this->process_to_add();
+  if (has_added_items) {
+    this->process_to_add();
+  }
 }
 void HOT Scheduler::process_to_add() {
   LockGuard guard{this->lock_};
   for (auto &it : this->to_add_) {
-    if (it->remove) {
+    if (is_item_removed_(it.get())) {
+      // Recycle cancelled items
+      this->recycle_item_(std::move(it));
       continue;
     }
 
@@ -389,8 +528,8 @@ void HOT Scheduler::process_to_add() {
   }
   this->to_add_.clear();
 }
-void HOT Scheduler::cleanup_() {
-  // Fast path: if nothing to remove, just return
+size_t HOT Scheduler::cleanup_() {
+  // Fast path: if nothing to remove, just return the current size
   // Reading to_remove_ without lock is safe because:
   // 1. We only call this from the main thread during call()
   // 2. If it's 0, there's definitely nothing to cleanup
@@ -398,7 +537,7 @@ void HOT Scheduler::cleanup_() {
   // 4. Not all platforms support atomics, so we accept this race in favor of performance
   // 5. The worst case is a one-loop-iteration delay in cleanup, which is harmless
   if (this->to_remove_ == 0)
-    return;
+    return this->items_.size();
 
   // We must hold the lock for the entire cleanup operation because:
   // 1. We're modifying items_ (via pop_raw_) which requires exclusive access
@@ -412,24 +551,27 @@ void HOT Scheduler::cleanup_() {
   while (!this->items_.empty()) {
     auto &item = this->items_[0];
     if (!item->remove)
-      return;
+      break;
     this->to_remove_--;
     this->pop_raw_();
   }
+  return this->items_.size();
 }
 void HOT Scheduler::pop_raw_() {
   std::pop_heap(this->items_.begin(), this->items_.end(), SchedulerItem::cmp);
+
+  // Instead of destroying, recycle the item
+  this->recycle_item_(std::move(this->items_.back()));
+
   this->items_.pop_back();
 }
 
 // Helper to execute a scheduler item
-void HOT Scheduler::execute_item_(SchedulerItem *item) {
+uint32_t HOT Scheduler::execute_item_(SchedulerItem *item, uint32_t now) {
   App.set_current_component(item->component);
-
-  uint32_t now_ms = millis();
-  WarnIfComponentBlockingGuard guard{item->component, now_ms};
+  WarnIfComponentBlockingGuard guard{item->component, now};
   item->callback();
-  guard.finish();
+  return guard.finish();
 }
 
 // Common implementation for cancel operations
@@ -444,66 +586,230 @@ bool HOT Scheduler::cancel_item_(Component *component, bool is_static_string, co
 }
 
 // Helper to cancel items by name - must be called with lock held
-bool HOT Scheduler::cancel_item_locked_(Component *component, const char *name_cstr, SchedulerItem::Type type) {
+bool HOT Scheduler::cancel_item_locked_(Component *component, const char *name_cstr, SchedulerItem::Type type,
+                                        bool match_retry) {
   // Early return if name is invalid - no items to cancel
-  if (name_cstr == nullptr || name_cstr[0] == '\0') {
+  if (name_cstr == nullptr) {
     return false;
   }
 
   size_t total_cancelled = 0;
 
   // Check all containers for matching items
-#if !defined(USE_ESP8266) && !defined(USE_RP2040)
-  // Only check defer queue for timeouts (intervals never go there)
+#ifndef ESPHOME_THREAD_SINGLE
+  // Mark items in defer queue as cancelled (they'll be skipped when processed)
   if (type == SchedulerItem::TIMEOUT) {
-    for (auto &item : this->defer_queue_) {
-      if (this->matches_item_(item, component, name_cstr, type)) {
-        item->remove = true;
-        total_cancelled++;
-      }
-    }
+    total_cancelled += this->mark_matching_items_removed_(this->defer_queue_, component, name_cstr, type, match_retry);
   }
-#endif
+#endif /* not ESPHOME_THREAD_SINGLE */
 
   // Cancel items in the main heap
-  for (auto &item : this->items_) {
-    if (this->matches_item_(item, component, name_cstr, type)) {
-      item->remove = true;
+  // Special case: if the last item in the heap matches, we can remove it immediately
+  // (removing the last element doesn't break heap structure)
+  if (!this->items_.empty()) {
+    auto &last_item = this->items_.back();
+    if (this->matches_item_(last_item, component, name_cstr, type, match_retry)) {
+      this->recycle_item_(std::move(this->items_.back()));
+      this->items_.pop_back();
       total_cancelled++;
-      this->to_remove_++;  // Track removals for heap items
     }
+    // For other items in heap, we can only mark for removal (can't remove from middle of heap)
+    size_t heap_cancelled = this->mark_matching_items_removed_(this->items_, component, name_cstr, type, match_retry);
+    total_cancelled += heap_cancelled;
+    this->to_remove_ += heap_cancelled;  // Track removals for heap items
   }
 
   // Cancel items in to_add_
-  for (auto &item : this->to_add_) {
-    if (this->matches_item_(item, component, name_cstr, type)) {
-      item->remove = true;
-      total_cancelled++;
-      // Don't track removals for to_add_ items
-    }
-  }
+  total_cancelled += this->mark_matching_items_removed_(this->to_add_, component, name_cstr, type, match_retry);
 
   return total_cancelled > 0;
 }
 
-uint64_t Scheduler::millis_() {
-  // Get the current 32-bit millis value
-  const uint32_t now = millis();
-  // Check for rollover by comparing with last value
-  if (now < this->last_millis_) {
-    // Detected rollover (happens every ~49.7 days)
+uint64_t Scheduler::millis_64_(uint32_t now) {
+  // THREAD SAFETY NOTE:
+  // This function has three implementations, based on the precompiler flags
+  // - ESPHOME_THREAD_SINGLE - Runs on single-threaded platforms (ESP8266, RP2040, etc.)
+  // - ESPHOME_THREAD_MULTI_NO_ATOMICS - Runs on multi-threaded platforms without atomics (LibreTiny)
+  // - ESPHOME_THREAD_MULTI_ATOMICS - Runs on multi-threaded platforms with atomics (ESP32, HOST, etc.)
+  //
+  // Make sure all changes are synchronized if you edit this function.
+  //
+  // IMPORTANT: Always pass fresh millis() values to this function. The implementation
+  // handles out-of-order timestamps between threads, but minimizing time differences
+  // helps maintain accuracy.
+  //
+
+#ifdef ESPHOME_THREAD_SINGLE
+  // This is the single core implementation.
+  //
+  // Single-core platforms have no concurrency, so this is a simple implementation
+  // that just tracks 32-bit rollover (every 49.7 days) without any locking or atomics.
+
+  uint16_t major = this->millis_major_;
+  uint32_t last = this->last_millis_;
+
+  // Check for rollover
+  if (now < last && (last - now) > HALF_MAX_UINT32) {
     this->millis_major_++;
-    ESP_LOGD(TAG, "Incrementing scheduler major at %" PRIu64 "ms",
-             now + (static_cast(this->millis_major_) << 32));
+    major++;
+#ifdef ESPHOME_DEBUG_SCHEDULER
+    ESP_LOGD(TAG, "Detected true 32-bit rollover at %" PRIu32 "ms (was %" PRIu32 ")", now, last);
+#endif /* ESPHOME_DEBUG_SCHEDULER */
   }
-  this->last_millis_ = now;
+
+  // Only update if time moved forward
+  if (now > last) {
+    this->last_millis_ = now;
+  }
+
   // Combine major (high 32 bits) and now (low 32 bits) into 64-bit time
-  return now + (static_cast(this->millis_major_) << 32);
+  return now + (static_cast(major) << 32);
+
+#elif defined(ESPHOME_THREAD_MULTI_NO_ATOMICS)
+  // This is the multi core no atomics implementation.
+  //
+  // Without atomics, this implementation uses locks more aggressively:
+  // 1. Always locks when near the rollover boundary (within 10 seconds)
+  // 2. Always locks when detecting a large backwards jump
+  // 3. Updates without lock in normal forward progression (accepting minor races)
+  // This is less efficient but necessary without atomic operations.
+  uint16_t major = this->millis_major_;
+  uint32_t last = this->last_millis_;
+
+  // Define a safe window around the rollover point (10 seconds)
+  // This covers any reasonable scheduler delays or thread preemption
+  static const uint32_t ROLLOVER_WINDOW = 10000;  // 10 seconds in milliseconds
+
+  // Check if we're near the rollover boundary (close to std::numeric_limits::max() or just past 0)
+  bool near_rollover = (last > (std::numeric_limits::max() - ROLLOVER_WINDOW)) || (now < ROLLOVER_WINDOW);
+
+  if (near_rollover || (now < last && (last - now) > HALF_MAX_UINT32)) {
+    // Near rollover or detected a rollover - need lock for safety
+    LockGuard guard{this->lock_};
+    // Re-read with lock held
+    last = this->last_millis_;
+
+    if (now < last && (last - now) > HALF_MAX_UINT32) {
+      // True rollover detected (happens every ~49.7 days)
+      this->millis_major_++;
+      major++;
+#ifdef ESPHOME_DEBUG_SCHEDULER
+      ESP_LOGD(TAG, "Detected true 32-bit rollover at %" PRIu32 "ms (was %" PRIu32 ")", now, last);
+#endif /* ESPHOME_DEBUG_SCHEDULER */
+    }
+    // Update last_millis_ while holding lock
+    this->last_millis_ = now;
+  } else if (now > last) {
+    // Normal case: Not near rollover and time moved forward
+    // Update without lock. While this may cause minor races (microseconds of
+    // backwards time movement), they're acceptable because:
+    // 1. The scheduler operates at millisecond resolution, not microsecond
+    // 2. We've already prevented the critical rollover race condition
+    // 3. Any backwards movement is orders of magnitude smaller than scheduler delays
+    this->last_millis_ = now;
+  }
+  // If now <= last and we're not near rollover, don't update
+  // This minimizes backwards time movement
+
+  // Combine major (high 32 bits) and now (low 32 bits) into 64-bit time
+  return now + (static_cast(major) << 32);
+
+#elif defined(ESPHOME_THREAD_MULTI_ATOMICS)
+  // This is the multi core with atomics implementation.
+  //
+  // Uses atomic operations with acquire/release semantics to ensure coherent
+  // reads of millis_major_ and last_millis_ across cores. Features:
+  // 1. Epoch-coherency retry loop to handle concurrent updates
+  // 2. Lock only taken for actual rollover detection and update
+  // 3. Lock-free CAS updates for normal forward time progression
+  // 4. Memory ordering ensures cores see consistent time values
+
+  for (;;) {
+    uint16_t major = this->millis_major_.load(std::memory_order_acquire);
+
+    /*
+     * Acquire so that if we later decide **not** to take the lock we still
+     * observe a `millis_major_` value coherent with the loaded `last_millis_`.
+     * The acquire load ensures any later read of `millis_major_` sees its
+     * corresponding increment.
+     */
+    uint32_t last = this->last_millis_.load(std::memory_order_acquire);
+
+    // If we might be near a rollover (large backwards jump), take the lock for the entire operation
+    // This ensures rollover detection and last_millis_ update are atomic together
+    if (now < last && (last - now) > HALF_MAX_UINT32) {
+      // Potential rollover - need lock for atomic rollover detection + update
+      LockGuard guard{this->lock_};
+      // Re-read with lock held; mutex already provides ordering
+      last = this->last_millis_.load(std::memory_order_relaxed);
+
+      if (now < last && (last - now) > HALF_MAX_UINT32) {
+        // True rollover detected (happens every ~49.7 days)
+        this->millis_major_.fetch_add(1, std::memory_order_relaxed);
+        major++;
+#ifdef ESPHOME_DEBUG_SCHEDULER
+        ESP_LOGD(TAG, "Detected true 32-bit rollover at %" PRIu32 "ms (was %" PRIu32 ")", now, last);
+#endif /* ESPHOME_DEBUG_SCHEDULER */
+      }
+      /*
+       * Update last_millis_ while holding the lock to prevent races
+       * Publish the new low-word *after* bumping `millis_major_` (done above)
+       * so readers never see a mismatched pair.
+       */
+      this->last_millis_.store(now, std::memory_order_release);
+    } else {
+      // Normal case: Try lock-free update, but only allow forward movement within same epoch
+      // This prevents accidentally moving backwards across a rollover boundary
+      while (now > last && (now - last) < HALF_MAX_UINT32) {
+        if (this->last_millis_.compare_exchange_weak(last, now,
+                                                     std::memory_order_release,     // success
+                                                     std::memory_order_relaxed)) {  // failure
+          break;
+        }
+        // CAS failure means no data was published; relaxed is fine
+        // last is automatically updated by compare_exchange_weak if it fails
+      }
+    }
+    uint16_t major_end = this->millis_major_.load(std::memory_order_relaxed);
+    if (major_end == major)
+      return now + (static_cast(major) << 32);
+  }
+  // Unreachable - the loop always returns when major_end == major
+  __builtin_unreachable();
+
+#else
+#error \
+    "No platform threading model defined. One of ESPHOME_THREAD_SINGLE, ESPHOME_THREAD_MULTI_NO_ATOMICS, or ESPHOME_THREAD_MULTI_ATOMICS must be defined."
+#endif
 }
 
 bool HOT Scheduler::SchedulerItem::cmp(const std::unique_ptr &a,
                                        const std::unique_ptr &b) {
-  return a->next_execution_ > b->next_execution_;
+  // High bits are almost always equal (change only on 32-bit rollover ~49 days)
+  // Optimize for common case: check low bits first when high bits are equal
+  return (a->next_execution_high_ == b->next_execution_high_) ? (a->next_execution_low_ > b->next_execution_low_)
+                                                              : (a->next_execution_high_ > b->next_execution_high_);
+}
+
+void Scheduler::recycle_item_(std::unique_ptr item) {
+  if (!item)
+    return;
+
+  if (this->scheduler_item_pool_.size() < MAX_POOL_SIZE) {
+    // Clear callback to release captured resources
+    item->callback = nullptr;
+    // Clear dynamic name if any
+    item->clear_dynamic_name();
+    this->scheduler_item_pool_.push_back(std::move(item));
+#ifdef ESPHOME_DEBUG_SCHEDULER
+    ESP_LOGD(TAG, "Recycled item to pool (pool size now: %zu)", this->scheduler_item_pool_.size());
+#endif
+  } else {
+#ifdef ESPHOME_DEBUG_SCHEDULER
+    ESP_LOGD(TAG, "Pool full (size: %zu), deleting item", this->scheduler_item_pool_.size());
+#endif
+  }
+  // else: unique_ptr will delete the item when it goes out of scope
 }
 
 }  // namespace esphome
diff --git a/esphome/core/scheduler.h b/esphome/core/scheduler.h
index 39cee5a876..2237915e07 100644
--- a/esphome/core/scheduler.h
+++ b/esphome/core/scheduler.h
@@ -1,9 +1,13 @@
 #pragma once
 
+#include "esphome/core/defines.h"
 #include 
 #include 
 #include 
 #include 
+#ifdef ESPHOME_THREAD_MULTI_ATOMICS
+#include 
+#endif
 
 #include "esphome/core/component.h"
 #include "esphome/core/helpers.h"
@@ -11,8 +15,20 @@
 namespace esphome {
 
 class Component;
+struct RetryArgs;
+
+// Forward declaration of retry_handler - needs to be non-static for friend declaration
+void retry_handler(const std::shared_ptr &args);
 
 class Scheduler {
+  // Allow retry_handler to access protected members for internal retry mechanism
+  friend void ::esphome::retry_handler(const std::shared_ptr &args);
+  // Allow DelayAction to call set_timer_common_ with skip_cancel=true for parallel script delays.
+  // This is needed to fix issue #10264 where parallel scripts with delays interfere with each other.
+  // We use friend instead of a public API because skip_cancel is dangerous - it can cause delays
+  // to accumulate and overload the scheduler if misused.
+  template friend class DelayAction;
+
  public:
   // Public API - accepts std::string for backward compatibility
   void set_timeout(Component *component, const std::string &name, uint32_t timeout, std::function func);
@@ -50,11 +66,21 @@ class Scheduler {
   bool cancel_interval(Component *component, const char *name);
   void set_retry(Component *component, const std::string &name, uint32_t initial_wait_time, uint8_t max_attempts,
                  std::function func, float backoff_increase_factor = 1.0f);
+  void set_retry(Component *component, const char *name, uint32_t initial_wait_time, uint8_t max_attempts,
+                 std::function func, float backoff_increase_factor = 1.0f);
   bool cancel_retry(Component *component, const std::string &name);
+  bool cancel_retry(Component *component, const char *name);
 
-  optional next_schedule_in();
+  // Calculate when the next scheduled item should run
+  // @param now Fresh timestamp from millis() - must not be stale/cached
+  // Returns the time in milliseconds until the next scheduled item, or nullopt if no items
+  // This method performs cleanup of removed items before checking the schedule
+  // IMPORTANT: This method should only be called from the main thread (loop task).
+  optional next_schedule_in(uint32_t now);
 
-  void call();
+  // Execute all scheduled items that are ready
+  // @param now Fresh timestamp from millis() - must not be stale/cached
+  void call(uint32_t now);
 
   void process_to_add();
 
@@ -62,38 +88,65 @@ class Scheduler {
   struct SchedulerItem {
     // Ordered by size to minimize padding
     Component *component;
-    uint32_t interval;
-    // 64-bit time to handle millis() rollover. The scheduler combines the 32-bit millis()
-    // with a 16-bit rollover counter to create a 64-bit time that won't roll over for
-    // billions of years. This ensures correct scheduling even when devices run for months.
-    uint64_t next_execution_;
-
     // Optimized name storage using tagged union
     union {
       const char *static_name;  // For string literals (no allocation)
       char *dynamic_name;       // For allocated strings
     } name_;
-
+    uint32_t interval;
+    // Split time to handle millis() rollover. The scheduler combines the 32-bit millis()
+    // with a 16-bit rollover counter to create a 48-bit time space (using 32+16 bits).
+    // This is intentionally limited to 48 bits, not stored as a full 64-bit value.
+    // With 49.7 days per 32-bit rollover, the 16-bit counter supports
+    // 49.7 days × 65536 = ~8900 years. This ensures correct scheduling
+    // even when devices run for months. Split into two fields for better memory
+    // alignment on 32-bit systems.
+    uint32_t next_execution_low_;  // Lower 32 bits of execution time (millis value)
     std::function callback;
+    uint16_t next_execution_high_;  // Upper 16 bits (millis_major counter)
 
-    // Bit-packed fields to minimize padding
+#ifdef ESPHOME_THREAD_MULTI_ATOMICS
+    // Multi-threaded with atomics: use atomic for lock-free access
+    // Place atomic separately since it can't be packed with bit fields
+    std::atomic remove{false};
+
+    // Bit-packed fields (3 bits used, 5 bits padding in 1 byte)
+    enum Type : uint8_t { TIMEOUT, INTERVAL } type : 1;
+    bool name_is_dynamic : 1;  // True if name was dynamically allocated (needs delete[])
+    bool is_retry : 1;         // True if this is a retry timeout
+                               // 5 bits padding
+#else
+    // Single-threaded or multi-threaded without atomics: can pack all fields together
+    // Bit-packed fields (4 bits used, 4 bits padding in 1 byte)
     enum Type : uint8_t { TIMEOUT, INTERVAL } type : 1;
     bool remove : 1;
     bool name_is_dynamic : 1;  // True if name was dynamically allocated (needs delete[])
-    // 5 bits padding
+    bool is_retry : 1;         // True if this is a retry timeout
+                               // 4 bits padding
+#endif
 
     // Constructor
     SchedulerItem()
-        : component(nullptr), interval(0), next_execution_(0), type(TIMEOUT), remove(false), name_is_dynamic(false) {
+        : component(nullptr),
+          interval(0),
+          next_execution_low_(0),
+          next_execution_high_(0),
+#ifdef ESPHOME_THREAD_MULTI_ATOMICS
+          // remove is initialized in the member declaration as std::atomic{false}
+          type(TIMEOUT),
+          name_is_dynamic(false),
+          is_retry(false) {
+#else
+          type(TIMEOUT),
+          remove(false),
+          name_is_dynamic(false),
+          is_retry(false) {
+#endif
       name_.static_name = nullptr;
     }
 
     // Destructor to clean up dynamic names
-    ~SchedulerItem() {
-      if (name_is_dynamic) {
-        delete[] name_.dynamic_name;
-      }
-    }
+    ~SchedulerItem() { clear_dynamic_name(); }
 
     // Delete copy operations to prevent accidental copies
     SchedulerItem(const SchedulerItem &) = delete;
@@ -106,44 +159,72 @@ class Scheduler {
     // Helper to get the name regardless of storage type
     const char *get_name() const { return name_is_dynamic ? name_.dynamic_name : name_.static_name; }
 
+    // Helper to clear dynamic name if allocated
+    void clear_dynamic_name() {
+      if (name_is_dynamic && name_.dynamic_name) {
+        delete[] name_.dynamic_name;
+        name_.dynamic_name = nullptr;
+        name_is_dynamic = false;
+      }
+    }
+
     // Helper to set name with proper ownership
     void set_name(const char *name, bool make_copy = false) {
       // Clean up old dynamic name if any
-      if (name_is_dynamic && name_.dynamic_name) {
-        delete[] name_.dynamic_name;
-        name_is_dynamic = false;
-      }
+      clear_dynamic_name();
 
-      if (!name || !name[0]) {
+      if (!name) {
+        // nullptr case - no name provided
         name_.static_name = nullptr;
       } else if (make_copy) {
-        // Make a copy for dynamic strings
+        // Make a copy for dynamic strings (including empty strings)
         size_t len = strlen(name);
         name_.dynamic_name = new char[len + 1];
         memcpy(name_.dynamic_name, name, len + 1);
         name_is_dynamic = true;
       } else {
-        // Use static string directly
+        // Use static string directly (including empty strings)
         name_.static_name = name;
       }
     }
 
     static bool cmp(const std::unique_ptr &a, const std::unique_ptr &b);
-    const char *get_type_str() const { return (type == TIMEOUT) ? "timeout" : "interval"; }
-    const char *get_source() const { return component ? component->get_component_source() : "unknown"; }
+
+    // Note: We use 48 bits total (32 + 16), stored in a 64-bit value for API compatibility.
+    // The upper 16 bits of the 64-bit value are always zero, which is fine since
+    // millis_major_ is also 16 bits and they must match.
+    constexpr uint64_t get_next_execution() const {
+      return (static_cast(next_execution_high_) << 32) | next_execution_low_;
+    }
+
+    constexpr void set_next_execution(uint64_t value) {
+      next_execution_low_ = static_cast(value);
+      // Cast to uint16_t intentionally truncates to lower 16 bits of the upper 32 bits.
+      // This is correct because millis_major_ that creates these values is also 16 bits.
+      next_execution_high_ = static_cast(value >> 32);
+    }
+    constexpr const char *get_type_str() const { return (type == TIMEOUT) ? "timeout" : "interval"; }
+    const LogString *get_source() const { return component ? component->get_component_log_str() : LOG_STR("unknown"); }
   };
 
   // Common implementation for both timeout and interval
   void set_timer_common_(Component *component, SchedulerItem::Type type, bool is_static_string, const void *name_ptr,
-                         uint32_t delay, std::function func);
+                         uint32_t delay, std::function func, bool is_retry = false, bool skip_cancel = false);
 
-  uint64_t millis_();
-  void cleanup_();
+  // Common implementation for retry
+  void set_retry_common_(Component *component, bool is_static_string, const void *name_ptr, uint32_t initial_wait_time,
+                         uint8_t max_attempts, std::function func, float backoff_increase_factor);
+
+  uint64_t millis_64_(uint32_t now);
+  // Cleanup logically deleted items from the scheduler
+  // Returns the number of items remaining after cleanup
+  // IMPORTANT: This method should only be called from the main thread (loop task).
+  size_t cleanup_();
   void pop_raw_();
 
  private:
   // Helper to cancel items by name - must be called with lock held
-  bool cancel_item_locked_(Component *component, const char *name, SchedulerItem::Type type);
+  bool cancel_item_locked_(Component *component, const char *name, SchedulerItem::Type type, bool match_retry = false);
 
   // Helper to extract name as const char* from either static string or std::string
   inline const char *get_name_cstr_(bool is_static_string, const void *name_ptr) {
@@ -153,59 +234,139 @@ class Scheduler {
   // Common implementation for cancel operations
   bool cancel_item_(Component *component, bool is_static_string, const void *name_ptr, SchedulerItem::Type type);
 
+  // Helper to check if two scheduler item names match
+  inline bool HOT names_match_(const char *name1, const char *name2) const {
+    // Check pointer equality first (common for static strings), then string contents
+    // The core ESPHome codebase uses static strings (const char*) for component names,
+    // making pointer comparison effective. The std::string overloads exist only for
+    // compatibility with external components but are rarely used in practice.
+    return (name1 != nullptr && name2 != nullptr) && ((name1 == name2) || (strcmp(name1, name2) == 0));
+  }
+
   // Helper function to check if item matches criteria for cancellation
   inline bool HOT matches_item_(const std::unique_ptr &item, Component *component, const char *name_cstr,
-                                SchedulerItem::Type type) {
-    if (item->component != component || item->type != type || item->remove) {
+                                SchedulerItem::Type type, bool match_retry, bool skip_removed = true) const {
+    if (item->component != component || item->type != type || (skip_removed && item->remove) ||
+        (match_retry && !item->is_retry)) {
       return false;
     }
-    const char *item_name = item->get_name();
-    if (item_name == nullptr) {
-      return false;
-    }
-    // Fast path: if pointers are equal
-    // This is effective because the core ESPHome codebase uses static strings (const char*)
-    // for component names. The std::string overloads exist only for compatibility with
-    // external components, but are rarely used in practice.
-    if (item_name == name_cstr) {
-      return true;
-    }
-    // Slow path: compare string contents
-    return strcmp(name_cstr, item_name) == 0;
+    return this->names_match_(item->get_name(), name_cstr);
   }
 
   // Helper to execute a scheduler item
-  void execute_item_(SchedulerItem *item);
+  uint32_t execute_item_(SchedulerItem *item, uint32_t now);
 
   // Helper to check if item should be skipped
-  bool should_skip_item_(const SchedulerItem *item) const {
-    return item->remove || (item->component != nullptr && item->component->is_failed());
+  bool should_skip_item_(SchedulerItem *item) const {
+    return is_item_removed_(item) || (item->component != nullptr && item->component->is_failed());
   }
 
-  // Check if the scheduler has no items.
-  // IMPORTANT: This method should only be called from the main thread (loop task).
-  // It performs cleanup of removed items and checks if the queue is empty.
-  // The items_.empty() check at the end is done without a lock for performance,
-  // which is safe because this is only called from the main thread while other
-  // threads only add items (never remove them).
-  bool empty_() {
-    this->cleanup_();
-    return this->items_.empty();
+  // Helper to recycle a SchedulerItem
+  void recycle_item_(std::unique_ptr item);
+
+  // Helper to check if item is marked for removal (platform-specific)
+  // Returns true if item should be skipped, handles platform-specific synchronization
+  // For ESPHOME_THREAD_MULTI_NO_ATOMICS platforms, the caller must hold the scheduler lock before calling this
+  // function.
+  bool is_item_removed_(SchedulerItem *item) const {
+#ifdef ESPHOME_THREAD_MULTI_ATOMICS
+    // Multi-threaded with atomics: use atomic load for lock-free access
+    return item->remove.load(std::memory_order_acquire);
+#else
+    // Single-threaded (ESPHOME_THREAD_SINGLE) or
+    // multi-threaded without atomics (ESPHOME_THREAD_MULTI_NO_ATOMICS): direct read
+    // For ESPHOME_THREAD_MULTI_NO_ATOMICS, caller MUST hold lock!
+    return item->remove;
+#endif
+  }
+
+  // Helper to mark matching items in a container as removed
+  // Returns the number of items marked for removal
+  // For ESPHOME_THREAD_MULTI_NO_ATOMICS platforms, the caller must hold the scheduler lock before calling this
+  // function.
+  template
+  size_t mark_matching_items_removed_(Container &container, Component *component, const char *name_cstr,
+                                      SchedulerItem::Type type, bool match_retry) {
+    size_t count = 0;
+    for (auto &item : container) {
+      if (this->matches_item_(item, component, name_cstr, type, match_retry)) {
+        // Mark item for removal (platform-specific)
+#ifdef ESPHOME_THREAD_MULTI_ATOMICS
+        // Multi-threaded with atomics: use atomic store
+        item->remove.store(true, std::memory_order_release);
+#else
+        // Single-threaded (ESPHOME_THREAD_SINGLE) or
+        // multi-threaded without atomics (ESPHOME_THREAD_MULTI_NO_ATOMICS): direct write
+        // For ESPHOME_THREAD_MULTI_NO_ATOMICS, caller MUST hold lock!
+        item->remove = true;
+#endif
+        count++;
+      }
+    }
+    return count;
+  }
+
+  // Template helper to check if any item in a container matches our criteria
+  template
+  bool has_cancelled_timeout_in_container_(const Container &container, Component *component, const char *name_cstr,
+                                           bool match_retry) const {
+    for (const auto &item : container) {
+      if (is_item_removed_(item.get()) &&
+          this->matches_item_(item, component, name_cstr, SchedulerItem::TIMEOUT, match_retry,
+                              /* skip_removed= */ false)) {
+        return true;
+      }
+    }
+    return false;
   }
 
   Mutex lock_;
   std::vector> items_;
   std::vector> to_add_;
-#if !defined(USE_ESP8266) && !defined(USE_RP2040)
-  // ESP8266 and RP2040 don't need the defer queue because:
-  // ESP8266: Single-core with no preemptive multitasking
-  // RP2040: Currently has empty mutex implementation in ESPHome
-  // Both platforms save 40 bytes of RAM by excluding this
+#ifndef ESPHOME_THREAD_SINGLE
+  // Single-core platforms don't need the defer queue and save 40 bytes of RAM
   std::deque> defer_queue_;  // FIFO queue for defer() calls
-#endif
-  uint32_t last_millis_{0};
-  uint16_t millis_major_{0};
+#endif                                                      /* ESPHOME_THREAD_SINGLE */
   uint32_t to_remove_{0};
+
+  // Memory pool for recycling SchedulerItem objects to reduce heap churn.
+  // Design decisions:
+  // - std::vector is used instead of a fixed array because many systems only need 1-2 scheduler items
+  // - The vector grows dynamically up to MAX_POOL_SIZE (5) only when needed, saving memory on simple setups
+  // - Pool size of 5 matches typical usage (2-4 timers) while keeping memory overhead low (~250 bytes on ESP32)
+  // - The pool significantly reduces heap fragmentation which is critical because heap allocation/deallocation
+  //   can stall the entire system, causing timing issues and dropped events for any components that need
+  //   to synchronize between tasks (see https://github.com/esphome/backlog/issues/52)
+  std::vector> scheduler_item_pool_;
+
+#ifdef ESPHOME_THREAD_MULTI_ATOMICS
+  /*
+   * Multi-threaded platforms with atomic support: last_millis_ needs atomic for lock-free updates
+   *
+   * MEMORY-ORDERING NOTE
+   * --------------------
+   * `last_millis_` and `millis_major_` form a single 64-bit timestamp split in half.
+   * Writers publish `last_millis_` with memory_order_release and readers use
+   * memory_order_acquire. This ensures that once a reader sees the new low word,
+   * it also observes the corresponding increment of `millis_major_`.
+   */
+  std::atomic last_millis_{0};
+#else  /* not ESPHOME_THREAD_MULTI_ATOMICS */
+  // Platforms without atomic support or single-threaded platforms
+  uint32_t last_millis_{0};
+#endif /* else ESPHOME_THREAD_MULTI_ATOMICS */
+
+  /*
+   * Upper 16 bits of the 64-bit millis counter. Incremented only while holding
+   * `lock_`; read concurrently. Atomic (relaxed) avoids a formal data race.
+   * Ordering relative to `last_millis_` is provided by its release store and the
+   * corresponding acquire loads.
+   */
+#ifdef ESPHOME_THREAD_MULTI_ATOMICS
+  std::atomic millis_major_{0};
+#else  /* not ESPHOME_THREAD_MULTI_ATOMICS */
+  uint16_t millis_major_{0};
+#endif /* else ESPHOME_THREAD_MULTI_ATOMICS */
 };
 
 }  // namespace esphome
diff --git a/esphome/core/string_ref.cpp b/esphome/core/string_ref.cpp
deleted file mode 100644
index ce1e33cbb7..0000000000
--- a/esphome/core/string_ref.cpp
+++ /dev/null
@@ -1,12 +0,0 @@
-#include "string_ref.h"
-
-namespace esphome {
-
-#ifdef USE_JSON
-
-// NOLINTNEXTLINE(readability-identifier-naming)
-void convertToJson(const StringRef &src, JsonVariant dst) { dst.set(src.c_str()); }
-
-#endif  // USE_JSON
-
-}  // namespace esphome
diff --git a/esphome/core/string_ref.h b/esphome/core/string_ref.h
index c4320107e3..efaa17181d 100644
--- a/esphome/core/string_ref.h
+++ b/esphome/core/string_ref.h
@@ -130,7 +130,7 @@ inline std::string operator+(const StringRef &lhs, const char *rhs) {
 
 #ifdef USE_JSON
 // NOLINTNEXTLINE(readability-identifier-naming)
-void convertToJson(const StringRef &src, JsonVariant dst);
+inline void convertToJson(const StringRef &src, JsonVariant dst) { dst.set(src.c_str()); }
 #endif  // USE_JSON
 
 }  // namespace esphome
diff --git a/esphome/core/time.cpp b/esphome/core/time.cpp
index f9652b5329..1285ec6448 100644
--- a/esphome/core/time.cpp
+++ b/esphome/core/time.cpp
@@ -77,7 +77,7 @@ bool ESPTime::strptime(const std::string &time_to_parse, ESPTime &esp_time) {
              &hour,                                                                                      // NOLINT
              &minute,                                                                                    // NOLINT
              &second, &num) == 6 &&                                                                      // NOLINT
-      num == time_to_parse.size()) {
+      num == static_cast(time_to_parse.size())) {
     esp_time.year = year;
     esp_time.month = month;
     esp_time.day_of_month = day;
@@ -87,7 +87,7 @@ bool ESPTime::strptime(const std::string &time_to_parse, ESPTime &esp_time) {
   } else if (sscanf(time_to_parse.c_str(), "%04hu-%02hhu-%02hhu %02hhu:%02hhu %n", &year, &month, &day,  // NOLINT
                     &hour,                                                                               // NOLINT
                     &minute, &num) == 5 &&                                                               // NOLINT
-             num == time_to_parse.size()) {
+             num == static_cast(time_to_parse.size())) {
     esp_time.year = year;
     esp_time.month = month;
     esp_time.day_of_month = day;
@@ -95,17 +95,17 @@ bool ESPTime::strptime(const std::string &time_to_parse, ESPTime &esp_time) {
     esp_time.minute = minute;
     esp_time.second = 0;
   } else if (sscanf(time_to_parse.c_str(), "%02hhu:%02hhu:%02hhu %n", &hour, &minute, &second, &num) == 3 &&  // NOLINT
-             num == time_to_parse.size()) {
+             num == static_cast(time_to_parse.size())) {
     esp_time.hour = hour;
     esp_time.minute = minute;
     esp_time.second = second;
   } else if (sscanf(time_to_parse.c_str(), "%02hhu:%02hhu %n", &hour, &minute, &num) == 2 &&  // NOLINT
-             num == time_to_parse.size()) {
+             num == static_cast(time_to_parse.size())) {
     esp_time.hour = hour;
     esp_time.minute = minute;
     esp_time.second = 0;
   } else if (sscanf(time_to_parse.c_str(), "%04hu-%02hhu-%02hhu %n", &year, &month, &day, &num) == 3 &&  // NOLINT
-             num == time_to_parse.size()) {
+             num == static_cast(time_to_parse.size())) {
     esp_time.year = year;
     esp_time.month = month;
     esp_time.day_of_month = day;
@@ -203,27 +203,13 @@ void ESPTime::recalc_timestamp_local() {
 }
 
 int32_t ESPTime::timezone_offset() {
-  int32_t offset = 0;
   time_t now = ::time(nullptr);
-  auto local = ESPTime::from_epoch_local(now);
-  auto utc = ESPTime::from_epoch_utc(now);
-  bool negative = utc.hour > local.hour && local.day_of_year <= utc.day_of_year;
-
-  if (utc.minute > local.minute) {
-    local.minute += 60;
-    local.hour -= 1;
-  }
-  offset += (local.minute - utc.minute) * 60;
-
-  if (negative) {
-    offset -= (utc.hour - local.hour) * 3600;
-  } else {
-    if (utc.hour > local.hour) {
-      local.hour += 24;
-    }
-    offset += (local.hour - utc.hour) * 3600;
-  }
-  return offset;
+  struct tm local_tm = *::localtime(&now);
+  local_tm.tm_isdst = 0;  // Cause mktime to ignore daylight saving time because we want to include it in the offset.
+  time_t local_time = mktime(&local_tm);
+  struct tm utc_tm = *::gmtime(&now);
+  time_t utc_time = mktime(&utc_tm);
+  return static_cast(local_time - utc_time);
 }
 
 bool ESPTime::operator<(const ESPTime &other) const { return this->timestamp < other.timestamp; }
diff --git a/esphome/coroutine.py b/esphome/coroutine.py
index 8d952246f3..0331c602c5 100644
--- a/esphome/coroutine.py
+++ b/esphome/coroutine.py
@@ -42,7 +42,10 @@ Here everything is combined in `yield` expressions. You await other coroutines u
 the last `yield` expression defines what is returned.
 """
 
+from __future__ import annotations
+
 from collections.abc import Awaitable, Callable, Generator, Iterator
+import enum
 import functools
 import heapq
 import inspect
@@ -53,6 +56,98 @@ from typing import Any
 _LOGGER = logging.getLogger(__name__)
 
 
+class CoroPriority(enum.IntEnum):
+    """Execution priority stages for ESPHome code generation.
+
+    Higher values run first. These stages ensure proper dependency
+    resolution during code generation.
+    """
+
+    # Platform initialization - must run first
+    # Examples: esp32, esp8266, rp2040
+    PLATFORM = 1000
+
+    # Network infrastructure setup
+    # Examples: network (201)
+    NETWORK = 201
+
+    # Network transport layer
+    # Examples: async_tcp (200)
+    NETWORK_TRANSPORT = 200
+
+    # Core system components
+    # Examples: esphome core, most entity base components (cover, update, datetime,
+    # valve, alarm_control_panel, lock, event, binary_sensor, button, climate, fan,
+    # light, media_player, number, select, sensor, switch, text_sensor, text),
+    # microphone, speaker, audio_dac, touchscreen, stepper
+    CORE = 100
+
+    # Diagnostic and debugging systems
+    # Examples: logger (90)
+    DIAGNOSTICS = 90
+
+    # Status and monitoring systems
+    # Examples: status_led (80)
+    STATUS = 80
+
+    # Web server infrastructure
+    # Examples: web_server_base (65)
+    WEB_SERVER_BASE = 65
+
+    # Network portal services
+    # Examples: captive_portal (64)
+    CAPTIVE_PORTAL = 64
+
+    # Communication protocols and services
+    # Examples: wifi (60), ethernet (60)
+    COMMUNICATION = 60
+
+    # Network discovery and management services
+    # Examples: mdns (55)
+    NETWORK_SERVICES = 55
+
+    # OTA update services
+    # Examples: ota_updates (54)
+    OTA_UPDATES = 54
+
+    # Web-based OTA services
+    # Examples: web_server_ota (52)
+    WEB_SERVER_OTA = 52
+
+    # Application-level services
+    # Examples: safe_mode (50)
+    APPLICATION = 50
+
+    # Web and UI services
+    # Examples: web_server (40)
+    WEB = 40
+
+    # Automations and user logic
+    # Examples: esphome core automations (30)
+    AUTOMATION = 30
+
+    # Bus and peripheral setup
+    # Examples: i2c (1)
+    BUS = 1
+
+    # Standard component priority (default)
+    # Components without explicit priority run at 0
+    COMPONENT = 0
+
+    # Components that need others to be registered first
+    # Examples: globals (-100)
+    LATE = -100
+
+    # Platform-specific workarounds and fixes
+    # Examples: add_arduino_global_workaround (-999), esp8266 pin states (-999)
+    WORKAROUNDS = -999
+
+    # Final setup that requires all components to be registered
+    # Examples: add_includes, _add_platformio_options, _add_platform_defines (all -1000),
+    # esp32_ble_tracker feature defines (-1000)
+    FINAL = -1000
+
+
 def coroutine(func: Callable[..., Any]) -> Callable[..., Awaitable[Any]]:
     """Decorator to apply to methods to convert them to ESPHome coroutines."""
     if getattr(func, "_esphome_coroutine", False):
@@ -95,15 +190,16 @@ def coroutine(func: Callable[..., Any]) -> Callable[..., Awaitable[Any]]:
     return coro
 
 
-def coroutine_with_priority(priority: float):
+def coroutine_with_priority(priority: float | CoroPriority):
     """Decorator to apply to functions to convert them to ESPHome coroutines.
 
     :param priority: priority with which to schedule the coroutine, higher priorities run first.
+                     Can be a float or a CoroPriority enum value.
     """
 
     def decorator(func):
         coro = coroutine(func)
-        coro.priority = priority
+        coro.priority = float(priority)
         return coro
 
     return decorator
@@ -173,7 +269,7 @@ class _Task:
         self.iterator = iterator
         self.original_function = original_function
 
-    def with_priority(self, priority: float) -> "_Task":
+    def with_priority(self, priority: float) -> _Task:
         return _Task(priority, self.id_number, self.iterator, self.original_function)
 
     @property
diff --git a/esphome/cpp_generator.py b/esphome/cpp_generator.py
index 060dd36f8f..b2022c7ae6 100644
--- a/esphome/cpp_generator.py
+++ b/esphome/cpp_generator.py
@@ -1,5 +1,5 @@
 import abc
-from collections.abc import Callable, Sequence
+from collections.abc import Callable
 import inspect
 import math
 import re
@@ -13,7 +13,6 @@ from esphome.core import (
     HexInt,
     Lambda,
     Library,
-    TimePeriod,
     TimePeriodMicroseconds,
     TimePeriodMilliseconds,
     TimePeriodMinutes,
@@ -21,35 +20,11 @@ from esphome.core import (
     TimePeriodSeconds,
 )
 from esphome.helpers import cpp_string_escape, indent_all_but_first_and_last
+from esphome.types import Expression, SafeExpType, TemplateArgsType
 from esphome.util import OrderedDict
 from esphome.yaml_util import ESPHomeDataBase
 
 
-class Expression(abc.ABC):
-    __slots__ = ()
-
-    @abc.abstractmethod
-    def __str__(self):
-        """
-        Convert expression into C++ code
-        """
-
-
-SafeExpType = (
-    Expression
-    | bool
-    | str
-    | str
-    | int
-    | float
-    | TimePeriod
-    | type[bool]
-    | type[int]
-    | type[float]
-    | Sequence[Any]
-)
-
-
 class RawExpression(Expression):
     __slots__ = ("text",)
 
@@ -253,6 +228,19 @@ class StringLiteral(Literal):
         return cpp_string_escape(self.string)
 
 
+class LogStringLiteral(Literal):
+    """A string literal that uses LOG_STR() macro for flash storage on ESP8266."""
+
+    __slots__ = ("string",)
+
+    def __init__(self, string: str) -> None:
+        super().__init__()
+        self.string = string
+
+    def __str__(self) -> str:
+        return f"LOG_STR({cpp_string_escape(self.string)})"
+
+
 class IntLiteral(Literal):
     __slots__ = ("i",)
 
@@ -562,7 +550,7 @@ def Pvariable(id_: ID, rhs: SafeExpType, type_: "MockObj" = None) -> "MockObj":
     return obj
 
 
-def new_Pvariable(id_: ID, *args: SafeExpType) -> Pvariable:
+def new_Pvariable(id_: ID, *args: SafeExpType) -> "MockObj":
     """Declare a new pointer variable in the code generation by calling it's constructor
     with the given arguments.
 
@@ -668,7 +656,7 @@ async def get_variable_with_full_id(id_: ID) -> tuple[ID, "MockObj"]:
 
 async def process_lambda(
     value: Lambda,
-    parameters: list[tuple[SafeExpType, str]],
+    parameters: TemplateArgsType,
     capture: str = "=",
     return_type: SafeExpType = None,
 ) -> LambdaExpression | None:
@@ -771,8 +759,7 @@ class MockObj(Expression):
         if attr.startswith("P") and self.op not in ["::", ""]:
             attr = attr[1:]
             next_op = "->"
-        if attr.startswith("_"):
-            attr = attr[1:]
+        attr = attr.removeprefix("_")
         return MockObj(f"{self.base}{self.op}{attr}", next_op)
 
     def __call__(self, *args: SafeExpType) -> "MockObj":
@@ -1037,10 +1024,7 @@ class MockObjClass(MockObj):
     def inherits_from(self, other: "MockObjClass") -> bool:
         if str(self) == str(other):
             return True
-        for parent in self._parents:
-            if str(parent) == str(other):
-                return True
-        return False
+        return any(str(parent) == str(other) for parent in self._parents)
 
     def template(self, *args: SafeExpType) -> "MockObjClass":
         if len(args) != 1 or not isinstance(args[0], TemplateArguments):
diff --git a/esphome/cpp_helpers.py b/esphome/cpp_helpers.py
index 3f64be6154..2698b9b3d5 100644
--- a/esphome/cpp_helpers.py
+++ b/esphome/cpp_helpers.py
@@ -9,7 +9,7 @@ from esphome.const import (
 )
 from esphome.core import CORE, ID, coroutine
 from esphome.coroutine import FakeAwaitable
-from esphome.cpp_generator import add, get_variable
+from esphome.cpp_generator import LogStringLiteral, add, get_variable
 from esphome.cpp_types import App
 from esphome.types import ConfigFragmentType, ConfigType
 from esphome.util import Registry, RegistryEntry
@@ -76,7 +76,7 @@ async def register_component(var, config):
             "Error while finding name of component, please report this", exc_info=e
         )
     if name is not None:
-        add(var.set_component_source(name))
+        add(var.set_component_source(LogStringLiteral(name)))
 
     add(App.register_component(var))
     return var
@@ -115,7 +115,7 @@ async def build_registry_list(registry, config):
 
 async def past_safe_mode():
     if CONF_SAFE_MODE not in CORE.config:
-        return
+        return None
 
     def _safe_mode_generator():
         while True:
diff --git a/esphome/dashboard/const.py b/esphome/dashboard/const.py
index db66cb5ead..ada5575d0e 100644
--- a/esphome/dashboard/const.py
+++ b/esphome/dashboard/const.py
@@ -1,9 +1,26 @@
 from __future__ import annotations
 
-EVENT_ENTRY_ADDED = "entry_added"
-EVENT_ENTRY_REMOVED = "entry_removed"
-EVENT_ENTRY_UPDATED = "entry_updated"
-EVENT_ENTRY_STATE_CHANGED = "entry_state_changed"
+from esphome.enum import StrEnum
+
+
+class DashboardEvent(StrEnum):
+    """Dashboard WebSocket event types."""
+
+    # Server -> Client events (backend sends to frontend)
+    ENTRY_ADDED = "entry_added"
+    ENTRY_REMOVED = "entry_removed"
+    ENTRY_UPDATED = "entry_updated"
+    ENTRY_STATE_CHANGED = "entry_state_changed"
+    IMPORTABLE_DEVICE_ADDED = "importable_device_added"
+    IMPORTABLE_DEVICE_REMOVED = "importable_device_removed"
+    INITIAL_STATE = "initial_state"  # Sent on WebSocket connection
+    PONG = "pong"  # Response to client ping
+
+    # Client -> Server events (frontend sends to backend)
+    PING = "ping"  # WebSocket keepalive from client
+    REFRESH = "refresh"  # Force backend to poll for changes
+
+
 MAX_EXECUTOR_WORKERS = 48
 
 
diff --git a/esphome/dashboard/core.py b/esphome/dashboard/core.py
index 410ef0c29d..b9ec56cd00 100644
--- a/esphome/dashboard/core.py
+++ b/esphome/dashboard/core.py
@@ -7,13 +7,13 @@ from dataclasses import dataclass
 from functools import partial
 import json
 import logging
-from pathlib import Path
 import threading
 from typing import Any
 
 from esphome.storage_json import ignored_devices_storage_path
 
 from ..zeroconf import DiscoveredImport
+from .const import DashboardEvent
 from .dns import DNSCache
 from .entries import DashboardEntries
 from .settings import DashboardSettings
@@ -31,7 +31,7 @@ MDNS_BOOTSTRAP_TIME = 7.5
 class Event:
     """Dashboard Event."""
 
-    event_type: str
+    event_type: DashboardEvent
     data: dict[str, Any]
 
 
@@ -40,22 +40,24 @@ class EventBus:
 
     def __init__(self) -> None:
         """Initialize the Dashboard event bus."""
-        self._listeners: dict[str, set[Callable[[Event], None]]] = {}
+        self._listeners: dict[DashboardEvent, set[Callable[[Event], None]]] = {}
 
     def async_add_listener(
-        self, event_type: str, listener: Callable[[Event], None]
+        self, event_type: DashboardEvent, listener: Callable[[Event], None]
     ) -> Callable[[], None]:
         """Add a listener to the event bus."""
         self._listeners.setdefault(event_type, set()).add(listener)
         return partial(self._async_remove_listener, event_type, listener)
 
     def _async_remove_listener(
-        self, event_type: str, listener: Callable[[Event], None]
+        self, event_type: DashboardEvent, listener: Callable[[Event], None]
     ) -> None:
         """Remove a listener from the event bus."""
         self._listeners[event_type].discard(listener)
 
-    def async_fire(self, event_type: str, event_data: dict[str, Any]) -> None:
+    def async_fire(
+        self, event_type: DashboardEvent, event_data: dict[str, Any]
+    ) -> None:
         """Fire an event."""
         event = Event(event_type, event_data)
 
@@ -108,7 +110,7 @@ class ESPHomeDashboard:
         await self.loop.run_in_executor(None, self.load_ignored_devices)
 
     def load_ignored_devices(self) -> None:
-        storage_path = Path(ignored_devices_storage_path())
+        storage_path = ignored_devices_storage_path()
         try:
             with storage_path.open("r", encoding="utf-8") as f_handle:
                 data = json.load(f_handle)
@@ -117,7 +119,7 @@ class ESPHomeDashboard:
             pass
 
     def save_ignored_devices(self) -> None:
-        storage_path = Path(ignored_devices_storage_path())
+        storage_path = ignored_devices_storage_path()
         with storage_path.open("w", encoding="utf-8") as f_handle:
             json.dump(
                 {"ignored_devices": sorted(self.ignored_devices)}, indent=2, fp=f_handle
diff --git a/esphome/dashboard/dashboard.py b/esphome/dashboard/dashboard.py
index 9de2d39ce2..81c10763e7 100644
--- a/esphome/dashboard/dashboard.py
+++ b/esphome/dashboard/dashboard.py
@@ -3,6 +3,7 @@ from __future__ import annotations
 import asyncio
 from asyncio import events
 from concurrent.futures import ThreadPoolExecutor
+import contextlib
 import logging
 import os
 import socket
@@ -125,10 +126,8 @@ def start_dashboard(args) -> None:
 
     asyncio.set_event_loop_policy(DashboardEventLoopPolicy(settings.verbose))
 
-    try:
+    with contextlib.suppress(KeyboardInterrupt):
         asyncio.run(async_start(args))
-    except KeyboardInterrupt:
-        pass
 
 
 async def async_start(args) -> None:
diff --git a/esphome/dashboard/dns.py b/esphome/dashboard/dns.py
index ea85d338bf..58867f7bc1 100644
--- a/esphome/dashboard/dns.py
+++ b/esphome/dashboard/dns.py
@@ -3,15 +3,9 @@ from __future__ import annotations
 import asyncio
 from contextlib import suppress
 from ipaddress import ip_address
-import sys
 
 from icmplib import NameLookupError, async_resolve
 
-if sys.version_info >= (3, 11):
-    from asyncio import timeout as async_timeout
-else:
-    from async_timeout import timeout as async_timeout
-
 RESOLVE_TIMEOUT = 3.0
 
 
@@ -20,9 +14,9 @@ async def _async_resolve_wrapper(hostname: str) -> list[str] | Exception:
     with suppress(ValueError):
         return [str(ip_address(hostname))]
     try:
-        async with async_timeout(RESOLVE_TIMEOUT):
+        async with asyncio.timeout(RESOLVE_TIMEOUT):
             return await async_resolve(hostname)
-    except (asyncio.TimeoutError, NameLookupError, UnicodeError) as ex:
+    except (TimeoutError, NameLookupError, UnicodeError) as ex:
         return ex
 
 
@@ -34,6 +28,21 @@ class DNSCache:
         self._cache: dict[str, tuple[float, list[str] | Exception]] = {}
         self._ttl = ttl
 
+    def get_cached_addresses(
+        self, hostname: str, now_monotonic: float
+    ) -> list[str] | None:
+        """Get cached addresses without triggering resolution.
+
+        Returns None if not in cache, list of addresses if found.
+        """
+        # Normalize hostname for consistent lookups
+        normalized = hostname.rstrip(".").lower()
+        if expire_time_addresses := self._cache.get(normalized):
+            expire_time, addresses = expire_time_addresses
+            if expire_time > now_monotonic and not isinstance(addresses, Exception):
+                return addresses
+        return None
+
     async def async_resolve(
         self, hostname: str, now_monotonic: float
     ) -> list[str] | Exception:
diff --git a/esphome/dashboard/entries.py b/esphome/dashboard/entries.py
index b138cfd272..95b8a7b2ae 100644
--- a/esphome/dashboard/entries.py
+++ b/esphome/dashboard/entries.py
@@ -5,20 +5,14 @@ from collections import defaultdict
 from dataclasses import dataclass
 from functools import lru_cache
 import logging
-import os
+from pathlib import Path
 from typing import TYPE_CHECKING, Any
 
 from esphome import const, util
 from esphome.enum import StrEnum
 from esphome.storage_json import StorageJSON, ext_storage_path
 
-from .const import (
-    DASHBOARD_COMMAND,
-    EVENT_ENTRY_ADDED,
-    EVENT_ENTRY_REMOVED,
-    EVENT_ENTRY_STATE_CHANGED,
-    EVENT_ENTRY_UPDATED,
-)
+from .const import DASHBOARD_COMMAND, DashboardEvent
 from .util.subprocess import async_run_system_command
 
 if TYPE_CHECKING:
@@ -102,12 +96,12 @@ class DashboardEntries:
         #   "path/to/file.yaml": DashboardEntry,
         #   ...
         # }
-        self._entries: dict[str, DashboardEntry] = {}
+        self._entries: dict[Path, DashboardEntry] = {}
         self._loaded_entries = False
         self._update_lock = asyncio.Lock()
         self._name_to_entry: dict[str, set[DashboardEntry]] = defaultdict(set)
 
-    def get(self, path: str) -> DashboardEntry | None:
+    def get(self, path: Path) -> DashboardEntry | None:
         """Get an entry by path."""
         return self._entries.get(path)
 
@@ -192,7 +186,7 @@ class DashboardEntries:
             return
         entry.state = state
         self._dashboard.bus.async_fire(
-            EVENT_ENTRY_STATE_CHANGED, {"entry": entry, "state": state}
+            DashboardEvent.ENTRY_STATE_CHANGED, {"entry": entry, "state": state}
         )
 
     async def async_request_update_entries(self) -> None:
@@ -260,22 +254,22 @@ class DashboardEntries:
         for entry in added:
             entries[entry.path] = entry
             name_to_entry[entry.name].add(entry)
-            bus.async_fire(EVENT_ENTRY_ADDED, {"entry": entry})
+            bus.async_fire(DashboardEvent.ENTRY_ADDED, {"entry": entry})
 
         for entry in removed:
             del entries[entry.path]
             name_to_entry[entry.name].discard(entry)
-            bus.async_fire(EVENT_ENTRY_REMOVED, {"entry": entry})
+            bus.async_fire(DashboardEvent.ENTRY_REMOVED, {"entry": entry})
 
         for entry in updated:
             if (original_name := original_names[entry]) != (current_name := entry.name):
                 name_to_entry[original_name].discard(entry)
                 name_to_entry[current_name].add(entry)
-            bus.async_fire(EVENT_ENTRY_UPDATED, {"entry": entry})
+            bus.async_fire(DashboardEvent.ENTRY_UPDATED, {"entry": entry})
 
-    def _get_path_to_cache_key(self) -> dict[str, DashboardCacheKeyType]:
+    def _get_path_to_cache_key(self) -> dict[Path, DashboardCacheKeyType]:
         """Return a dict of path to cache key."""
-        path_to_cache_key: dict[str, DashboardCacheKeyType] = {}
+        path_to_cache_key: dict[Path, DashboardCacheKeyType] = {}
         #
         # The cache key is (inode, device, mtime, size)
         # which allows us to avoid locking since it ensures
@@ -287,12 +281,12 @@ class DashboardEntries:
         for file in util.list_yaml_files([self._config_dir]):
             try:
                 # Prefer the json storage path if it exists
-                stat = os.stat(ext_storage_path(os.path.basename(file)))
+                stat = ext_storage_path(file.name).stat()
             except OSError:
                 try:
                     # Fallback to the yaml file if the storage
                     # file does not exist or could not be generated
-                    stat = os.stat(file)
+                    stat = file.stat()
                 except OSError:
                     # File was deleted, ignore
                     continue
@@ -329,10 +323,10 @@ class DashboardEntry:
         "_to_dict",
     )
 
-    def __init__(self, path: str, cache_key: DashboardCacheKeyType) -> None:
+    def __init__(self, path: Path, cache_key: DashboardCacheKeyType) -> None:
         """Initialize the DashboardEntry."""
         self.path = path
-        self.filename: str = os.path.basename(path)
+        self.filename: str = path.name
         self._storage_path = ext_storage_path(self.filename)
         self.cache_key = cache_key
         self.storage: StorageJSON | None = None
@@ -365,7 +359,7 @@ class DashboardEntry:
                 "loaded_integrations": sorted(self.loaded_integrations),
                 "deployed_version": self.update_old,
                 "current_version": self.update_new,
-                "path": self.path,
+                "path": str(self.path),
                 "comment": self.comment,
                 "address": self.address,
                 "web_port": self.web_port,
diff --git a/esphome/dashboard/models.py b/esphome/dashboard/models.py
new file mode 100644
index 0000000000..47ddddd5ce
--- /dev/null
+++ b/esphome/dashboard/models.py
@@ -0,0 +1,76 @@
+"""Data models and builders for the dashboard."""
+
+from __future__ import annotations
+
+from typing import TYPE_CHECKING, TypedDict
+
+if TYPE_CHECKING:
+    from esphome.zeroconf import DiscoveredImport
+
+    from .core import ESPHomeDashboard
+    from .entries import DashboardEntry
+
+
+class ImportableDeviceDict(TypedDict):
+    """Dictionary representation of an importable device."""
+
+    name: str
+    friendly_name: str | None
+    package_import_url: str
+    project_name: str
+    project_version: str
+    network: str
+    ignored: bool
+
+
+class ConfiguredDeviceDict(TypedDict, total=False):
+    """Dictionary representation of a configured device."""
+
+    name: str
+    friendly_name: str | None
+    configuration: str
+    loaded_integrations: list[str] | None
+    deployed_version: str | None
+    current_version: str | None
+    path: str
+    comment: str | None
+    address: str | None
+    web_port: int | None
+    target_platform: str | None
+
+
+class DeviceListResponse(TypedDict):
+    """Response for device list API."""
+
+    configured: list[ConfiguredDeviceDict]
+    importable: list[ImportableDeviceDict]
+
+
+def build_importable_device_dict(
+    dashboard: ESPHomeDashboard, discovered: DiscoveredImport
+) -> ImportableDeviceDict:
+    """Build the importable device dictionary."""
+    return ImportableDeviceDict(
+        name=discovered.device_name,
+        friendly_name=discovered.friendly_name,
+        package_import_url=discovered.package_import_url,
+        project_name=discovered.project_name,
+        project_version=discovered.project_version,
+        network=discovered.network,
+        ignored=discovered.device_name in dashboard.ignored_devices,
+    )
+
+
+def build_device_list_response(
+    dashboard: ESPHomeDashboard, entries: list[DashboardEntry]
+) -> DeviceListResponse:
+    """Build the device list response data."""
+    configured = {entry.name for entry in entries}
+    return DeviceListResponse(
+        configured=[entry.to_dict() for entry in entries],
+        importable=[
+            build_importable_device_dict(dashboard, res)
+            for res in dashboard.import_result.values()
+            if res.device_name not in configured
+        ],
+    )
diff --git a/esphome/dashboard/settings.py b/esphome/dashboard/settings.py
index fa39b55016..6035b4a1d6 100644
--- a/esphome/dashboard/settings.py
+++ b/esphome/dashboard/settings.py
@@ -10,6 +10,10 @@ from esphome.helpers import get_bool_env
 
 from .util.password import password_hash
 
+# Sentinel file name used for CORE.config_path when dashboard initializes.
+# This ensures .parent returns the config directory instead of root.
+_DASHBOARD_SENTINEL_FILE = "___DASHBOARD_SENTINEL___.yaml"
+
 
 class DashboardSettings:
     """Settings for the dashboard."""
@@ -27,7 +31,7 @@ class DashboardSettings:
 
     def __init__(self) -> None:
         """Initialize the dashboard settings."""
-        self.config_dir: str = ""
+        self.config_dir: Path = None
         self.password_hash: str = ""
         self.username: str = ""
         self.using_password: bool = False
@@ -45,10 +49,15 @@ class DashboardSettings:
             self.using_password = bool(password)
         if self.using_password:
             self.password_hash = password_hash(password)
-        self.config_dir = args.configuration
-        self.absolute_config_dir = Path(self.config_dir).resolve()
+        self.config_dir = Path(args.configuration)
+        self.absolute_config_dir = self.config_dir.resolve()
         self.verbose = args.verbose
-        CORE.config_path = os.path.join(self.config_dir, ".")
+        # Set to a sentinel file so .parent gives us the config directory.
+        # Previously this was `os.path.join(self.config_dir, ".")` which worked because
+        # os.path.dirname("/config/.") returns "/config", but Path("/config/.").parent
+        # normalizes to Path("/config") first, then .parent returns Path("/"), breaking
+        # secret resolution. Using a sentinel file ensures .parent gives the correct directory.
+        CORE.config_path = self.config_dir / _DASHBOARD_SENTINEL_FILE
 
     @property
     def relative_url(self) -> str:
@@ -81,9 +90,9 @@ class DashboardSettings:
         # Compare password in constant running time (to prevent timing attacks)
         return hmac.compare_digest(self.password_hash, password_hash(password))
 
-    def rel_path(self, *args: Any) -> str:
+    def rel_path(self, *args: Any) -> Path:
         """Return a path relative to the ESPHome config folder."""
-        joined_path = os.path.join(self.config_dir, *args)
+        joined_path = self.config_dir / Path(*args)
         # Raises ValueError if not relative to ESPHome config folder
-        Path(joined_path).resolve().relative_to(self.absolute_config_dir)
+        joined_path.resolve().relative_to(self.absolute_config_dir)
         return joined_path
diff --git a/esphome/dashboard/status/mdns.py b/esphome/dashboard/status/mdns.py
index f9ac7b4289..881340ab24 100644
--- a/esphome/dashboard/status/mdns.py
+++ b/esphome/dashboard/status/mdns.py
@@ -4,16 +4,21 @@ import asyncio
 import logging
 import typing
 
+from zeroconf import AddressResolver, IPVersion
+
+from esphome.address_cache import normalize_hostname
 from esphome.zeroconf import (
     ESPHOME_SERVICE_TYPE,
     AsyncEsphomeZeroconf,
     DashboardBrowser,
     DashboardImportDiscovery,
     DashboardStatus,
+    DiscoveredImport,
 )
 
-from ..const import SENTINEL
+from ..const import SENTINEL, DashboardEvent
 from ..entries import DashboardEntry, EntryStateSource, bool_to_entry_state
+from ..models import build_importable_device_dict
 
 if typing.TYPE_CHECKING:
     from ..core import ESPHomeDashboard
@@ -50,6 +55,44 @@ class MDNSStatus:
             return await aiozc.async_resolve_host(host_name)
         return None
 
+    def get_cached_addresses(self, host_name: str) -> list[str] | None:
+        """Get cached addresses for a host without triggering resolution.
+
+        Returns None if not in cache or no zeroconf available.
+        """
+        if not self.aiozc:
+            _LOGGER.debug("No zeroconf instance available for %s", host_name)
+            return None
+
+        # Normalize hostname and get the base name
+        normalized = normalize_hostname(host_name)
+        base_name = normalized.partition(".")[0]
+
+        # Try to load from zeroconf cache without triggering resolution
+        resolver_name = f"{base_name}.local."
+        info = AddressResolver(resolver_name)
+        # Let zeroconf use its own current time for cache checking
+        if info.load_from_cache(self.aiozc.zeroconf):
+            addresses = info.parsed_scoped_addresses(IPVersion.All)
+            _LOGGER.debug("Found %s in zeroconf cache: %s", resolver_name, addresses)
+            return addresses
+        _LOGGER.debug("Not found in zeroconf cache: %s", resolver_name)
+        return None
+
+    def _on_import_update(self, name: str, discovered: DiscoveredImport | None) -> None:
+        """Handle importable device updates."""
+        if discovered is None:
+            # Device removed
+            self.dashboard.bus.async_fire(
+                DashboardEvent.IMPORTABLE_DEVICE_REMOVED, {"name": name}
+            )
+        else:
+            # Device added
+            self.dashboard.bus.async_fire(
+                DashboardEvent.IMPORTABLE_DEVICE_ADDED,
+                {"device": build_importable_device_dict(self.dashboard, discovered)},
+            )
+
     async def async_refresh_hosts(self) -> None:
         """Refresh the hosts to track."""
         dashboard = self.dashboard
@@ -106,7 +149,8 @@ class MDNSStatus:
                         self._async_set_state(entry, result)
 
         stat = DashboardStatus(on_update)
-        imports = DashboardImportDiscovery()
+
+        imports = DashboardImportDiscovery(self._on_import_update)
         dashboard.import_result = imports.import_state
 
         browser = DashboardBrowser(
diff --git a/esphome/dashboard/util/file.py b/esphome/dashboard/util/file.py
deleted file mode 100644
index bb263f9ad7..0000000000
--- a/esphome/dashboard/util/file.py
+++ /dev/null
@@ -1,63 +0,0 @@
-import logging
-import os
-from pathlib import Path
-import tempfile
-
-_LOGGER = logging.getLogger(__name__)
-
-
-def write_utf8_file(
-    filename: Path,
-    utf8_str: str,
-    private: bool = False,
-) -> None:
-    """Write a file and rename it into place.
-
-    Writes all or nothing.
-    """
-    write_file(filename, utf8_str.encode("utf-8"), private)
-
-
-# from https://github.com/home-assistant/core/blob/dev/homeassistant/util/file.py
-def write_file(
-    filename: Path,
-    utf8_data: bytes,
-    private: bool = False,
-) -> None:
-    """Write a file and rename it into place.
-
-    Writes all or nothing.
-    """
-
-    tmp_filename = ""
-    missing_fchmod = False
-    try:
-        # Modern versions of Python tempfile create this file with mode 0o600
-        with tempfile.NamedTemporaryFile(
-            mode="wb", dir=os.path.dirname(filename), delete=False
-        ) as fdesc:
-            fdesc.write(utf8_data)
-            tmp_filename = fdesc.name
-            if not private:
-                try:
-                    os.fchmod(fdesc.fileno(), 0o644)
-                except AttributeError:
-                    # os.fchmod is not available on Windows
-                    missing_fchmod = True
-
-        os.replace(tmp_filename, filename)
-        if missing_fchmod:
-            os.chmod(filename, 0o644)
-    finally:
-        if os.path.exists(tmp_filename):
-            try:
-                os.remove(tmp_filename)
-            except OSError as err:
-                # If we are cleaning up then something else went wrong, so
-                # we should suppress likely follow-on errors in the cleanup
-                _LOGGER.error(
-                    "File replacement cleanup failed for %s while saving %s: %s",
-                    tmp_filename,
-                    filename,
-                    err,
-                )
diff --git a/esphome/dashboard/web_server.py b/esphome/dashboard/web_server.py
index 480285b6c1..804a2b99af 100644
--- a/esphome/dashboard/web_server.py
+++ b/esphome/dashboard/web_server.py
@@ -2,9 +2,12 @@ from __future__ import annotations
 
 import asyncio
 import base64
+import binascii
 from collections.abc import Callable, Iterable
+import contextlib
 import datetime
 import functools
+from functools import partial
 import gzip
 import hashlib
 import importlib
@@ -48,10 +51,11 @@ from esphome.storage_json import (
 from esphome.util import get_serial_ports, shlex_quote
 from esphome.yaml_util import FastestAvailableSafeLoader
 
-from .const import DASHBOARD_COMMAND
-from .core import DASHBOARD
-from .entries import UNKNOWN_STATE, entry_state_to_bool
-from .util.file import write_file
+from ..helpers import write_file
+from .const import DASHBOARD_COMMAND, DashboardEvent
+from .core import DASHBOARD, ESPHomeDashboard, Event
+from .entries import UNKNOWN_STATE, DashboardEntry, entry_state_to_bool
+from .models import build_device_list_response
 from .util.subprocess import async_run_system_command
 from .util.text import friendly_name_slugify
 
@@ -144,7 +148,7 @@ def websocket_class(cls):
     if not hasattr(cls, "_message_handlers"):
         cls._message_handlers = {}
 
-    for _, method in cls.__dict__.items():
+    for method in cls.__dict__.values():
         if hasattr(method, "_message_handler"):
             cls._message_handlers[method._message_handler] = method
 
@@ -229,6 +233,7 @@ class EsphomeCommandWebSocket(tornado.websocket.WebSocketHandler):
                 stdin=subprocess.PIPE,
                 stdout=subprocess.PIPE,
                 stderr=subprocess.STDOUT,
+                close_fds=False,
             )
             stdout_thread = threading.Thread(target=self._stdout_thread)
             stdout_thread.daemon = True
@@ -281,11 +286,23 @@ class EsphomeCommandWebSocket(tornado.websocket.WebSocketHandler):
     def _stdout_thread(self) -> None:
         if not self._use_popen:
             return
+        line = b""
+        cr = False
         while True:
-            data = self._proc.stdout.readline()
+            data = self._proc.stdout.read(1)
             if data:
-                data = data.replace(b"\r", b"")
-                self._queue.put_nowait(data)
+                if data == b"\r":
+                    cr = True
+                elif data == b"\n":
+                    self._queue.put_nowait(line + b"\n")
+                    line = b""
+                    cr = False
+                elif cr:
+                    self._queue.put_nowait(line + b"\r")
+                    line = data
+                    cr = False
+                else:
+                    line += data
             if self._proc.poll() is not None:
                 break
         self._proc.wait(1.0)
@@ -312,6 +329,73 @@ class EsphomeCommandWebSocket(tornado.websocket.WebSocketHandler):
         raise NotImplementedError
 
 
+def build_cache_arguments(
+    entry: DashboardEntry | None,
+    dashboard: ESPHomeDashboard,
+    now: float,
+) -> list[str]:
+    """Build cache arguments for passing to CLI.
+
+    Args:
+        entry: Dashboard entry for the configuration
+        dashboard: Dashboard instance with cache access
+        now: Current monotonic time for DNS cache expiry checks
+
+    Returns:
+        List of cache arguments to pass to CLI
+    """
+    cache_args: list[str] = []
+
+    if not entry:
+        return cache_args
+
+    _LOGGER.debug(
+        "Building cache for entry (address=%s, name=%s)",
+        entry.address,
+        entry.name,
+    )
+
+    def add_cache_entry(hostname: str, addresses: list[str], cache_type: str) -> None:
+        """Add a cache entry to the command arguments."""
+        if not addresses:
+            return
+        normalized = hostname.rstrip(".").lower()
+        cache_args.extend(
+            [
+                f"--{cache_type}-address-cache",
+                f"{normalized}={','.join(sort_ip_addresses(addresses))}",
+            ]
+        )
+
+    # Check entry.address for cached addresses
+    if use_address := entry.address:
+        if use_address.endswith(".local"):
+            # mDNS cache for .local addresses
+            if (mdns := dashboard.mdns_status) and (
+                cached := mdns.get_cached_addresses(use_address)
+            ):
+                _LOGGER.debug("mDNS cache hit for %s: %s", use_address, cached)
+                add_cache_entry(use_address, cached, "mdns")
+        # DNS cache for non-.local addresses
+        elif cached := dashboard.dns_cache.get_cached_addresses(use_address, now):
+            _LOGGER.debug("DNS cache hit for %s: %s", use_address, cached)
+            add_cache_entry(use_address, cached, "dns")
+
+    # Check entry.name if we haven't already cached via address
+    # For mDNS devices, entry.name typically doesn't have .local suffix
+    if entry.name and not use_address:
+        mdns_name = (
+            f"{entry.name}.local" if not entry.name.endswith(".local") else entry.name
+        )
+        if (mdns := dashboard.mdns_status) and (
+            cached := mdns.get_cached_addresses(mdns_name)
+        ):
+            _LOGGER.debug("mDNS cache hit for %s: %s", mdns_name, cached)
+            add_cache_entry(mdns_name, cached, "mdns")
+
+    return cache_args
+
+
 class EsphomePortCommandWebSocket(EsphomeCommandWebSocket):
     """Base class for commands that require a port."""
 
@@ -324,38 +408,22 @@ class EsphomePortCommandWebSocket(EsphomeCommandWebSocket):
         configuration = json_message["configuration"]
         config_file = settings.rel_path(configuration)
         port = json_message["port"]
+
+        # Build cache arguments to pass to CLI
+        cache_args: list[str] = []
+
         if (
             port == "OTA"  # pylint: disable=too-many-boolean-expressions
             and (entry := entries.get(config_file))
             and entry.loaded_integrations
             and "api" in entry.loaded_integrations
         ):
-            if (mdns := dashboard.mdns_status) and (
-                address_list := await mdns.async_resolve_host(entry.name)
-            ):
-                # Use the IP address if available but only
-                # if the API is loaded and the device is online
-                # since MQTT logging will not work otherwise
-                port = sort_ip_addresses(address_list)[0]
-            elif (
-                entry.address
-                and (
-                    address_list := await dashboard.dns_cache.async_resolve(
-                        entry.address, time.monotonic()
-                    )
-                )
-                and not isinstance(address_list, Exception)
-            ):
-                # If mdns is not available, try to use the DNS cache
-                port = sort_ip_addresses(address_list)[0]
+            cache_args = build_cache_arguments(entry, dashboard, time.monotonic())
 
-        return [
-            *DASHBOARD_COMMAND,
-            *args,
-            config_file,
-            "--device",
-            port,
-        ]
+        # Cache arguments must come before the subcommand
+        cmd = [*DASHBOARD_COMMAND, *cache_args, *args, config_file, "--device", port]
+        _LOGGER.debug("Built command: %s", cmd)
+        return cmd
 
 
 class EsphomeLogsHandler(EsphomePortCommandWebSocket):
@@ -426,6 +494,14 @@ class EsphomeCleanMqttHandler(EsphomeCommandWebSocket):
         return [*DASHBOARD_COMMAND, "clean-mqtt", config_file]
 
 
+class EsphomeCleanAllHandler(EsphomeCommandWebSocket):
+    async def build_command(self, json_message: dict[str, Any]) -> list[str]:
+        clean_build_dir = json_message.get("clean_build_dir", True)
+        if clean_build_dir:
+            return [*DASHBOARD_COMMAND, "clean-all", settings.config_dir]
+        return [*DASHBOARD_COMMAND, "clean-all"]
+
+
 class EsphomeCleanHandler(EsphomeCommandWebSocket):
     async def build_command(self, json_message: dict[str, Any]) -> list[str]:
         config_file = settings.rel_path(json_message["configuration"])
@@ -447,6 +523,243 @@ class EsphomeUpdateAllHandler(EsphomeCommandWebSocket):
         return [*DASHBOARD_COMMAND, "update-all", settings.config_dir]
 
 
+# Dashboard polling constants
+DASHBOARD_POLL_INTERVAL = 2  # seconds
+DASHBOARD_ENTRIES_UPDATE_INTERVAL = 10  # seconds
+DASHBOARD_ENTRIES_UPDATE_ITERATIONS = (
+    DASHBOARD_ENTRIES_UPDATE_INTERVAL // DASHBOARD_POLL_INTERVAL
+)
+
+
+class DashboardSubscriber:
+    """Manages dashboard event polling task lifecycle based on active subscribers."""
+
+    def __init__(self) -> None:
+        """Initialize the dashboard subscriber."""
+        self._subscribers: set[DashboardEventsWebSocket] = set()
+        self._event_loop_task: asyncio.Task | None = None
+        self._refresh_event: asyncio.Event = asyncio.Event()
+
+    def subscribe(self, subscriber: DashboardEventsWebSocket) -> Callable[[], None]:
+        """Subscribe to dashboard updates and start event loop if needed."""
+        self._subscribers.add(subscriber)
+        if not self._event_loop_task or self._event_loop_task.done():
+            self._event_loop_task = asyncio.create_task(self._event_loop())
+            _LOGGER.info("Started dashboard event loop")
+        return partial(self._unsubscribe, subscriber)
+
+    def _unsubscribe(self, subscriber: DashboardEventsWebSocket) -> None:
+        """Unsubscribe from dashboard updates and stop event loop if no subscribers."""
+        self._subscribers.discard(subscriber)
+        if (
+            not self._subscribers
+            and self._event_loop_task
+            and not self._event_loop_task.done()
+        ):
+            self._event_loop_task.cancel()
+            self._event_loop_task = None
+            _LOGGER.info("Stopped dashboard event loop - no subscribers")
+
+    def request_refresh(self) -> None:
+        """Signal the polling loop to refresh immediately."""
+        self._refresh_event.set()
+
+    async def _event_loop(self) -> None:
+        """Run the event polling loop while there are subscribers."""
+        dashboard = DASHBOARD
+        entries_update_counter = 0
+
+        while self._subscribers:
+            # Signal that we need ping updates (non-blocking)
+            dashboard.ping_request.set()
+            if settings.status_use_mqtt:
+                dashboard.mqtt_ping_request.set()
+
+            # Check if it's time to update entries or if refresh was requested
+            entries_update_counter += 1
+            if (
+                entries_update_counter >= DASHBOARD_ENTRIES_UPDATE_ITERATIONS
+                or self._refresh_event.is_set()
+            ):
+                entries_update_counter = 0
+                await dashboard.entries.async_request_update_entries()
+                # Clear the refresh event if it was set
+                self._refresh_event.clear()
+
+            # Wait for either timeout or refresh event
+            try:
+                async with asyncio.timeout(DASHBOARD_POLL_INTERVAL):
+                    await self._refresh_event.wait()
+                    # If we get here, refresh was requested - continue loop immediately
+            except TimeoutError:
+                # Normal timeout - continue with regular polling
+                pass
+
+
+# Global dashboard subscriber instance
+DASHBOARD_SUBSCRIBER = DashboardSubscriber()
+
+
+@websocket_class
+class DashboardEventsWebSocket(tornado.websocket.WebSocketHandler):
+    """WebSocket handler for real-time dashboard events."""
+
+    _event_listeners: list[Callable[[], None]] | None = None
+    _dashboard_unsubscribe: Callable[[], None] | None = None
+
+    async def get(self, *args: str, **kwargs: str) -> None:
+        """Handle WebSocket upgrade request."""
+        if not is_authenticated(self):
+            self.set_status(401)
+            self.finish("Unauthorized")
+            return
+        await super().get(*args, **kwargs)
+
+    async def open(self, *args: str, **kwargs: str) -> None:  # pylint: disable=invalid-overridden-method
+        """Handle new WebSocket connection."""
+        # Ensure messages are sent immediately to avoid
+        # a 200-500ms delay when nodelay is not set.
+        self.set_nodelay(True)
+
+        # Update entries first
+        await DASHBOARD.entries.async_request_update_entries()
+        # Send initial state
+        self._send_initial_state()
+        # Subscribe to events
+        self._subscribe_to_events()
+        # Subscribe to dashboard updates
+        self._dashboard_unsubscribe = DASHBOARD_SUBSCRIBER.subscribe(self)
+        _LOGGER.debug("Dashboard status WebSocket opened")
+
+    def _send_initial_state(self) -> None:
+        """Send initial device list and ping status."""
+        entries = DASHBOARD.entries.async_all()
+
+        # Send initial state
+        self._safe_send_message(
+            {
+                "event": DashboardEvent.INITIAL_STATE,
+                "data": {
+                    "devices": build_device_list_response(DASHBOARD, entries),
+                    "ping": {
+                        entry.filename: entry_state_to_bool(entry.state)
+                        for entry in entries
+                    },
+                },
+            }
+        )
+
+    def _subscribe_to_events(self) -> None:
+        """Subscribe to dashboard events."""
+        async_add_listener = DASHBOARD.bus.async_add_listener
+        # Subscribe to all events
+        self._event_listeners = [
+            async_add_listener(
+                DashboardEvent.ENTRY_STATE_CHANGED, self._on_entry_state_changed
+            ),
+            async_add_listener(
+                DashboardEvent.ENTRY_ADDED,
+                self._make_entry_handler(DashboardEvent.ENTRY_ADDED),
+            ),
+            async_add_listener(
+                DashboardEvent.ENTRY_REMOVED,
+                self._make_entry_handler(DashboardEvent.ENTRY_REMOVED),
+            ),
+            async_add_listener(
+                DashboardEvent.ENTRY_UPDATED,
+                self._make_entry_handler(DashboardEvent.ENTRY_UPDATED),
+            ),
+            async_add_listener(
+                DashboardEvent.IMPORTABLE_DEVICE_ADDED, self._on_importable_added
+            ),
+            async_add_listener(
+                DashboardEvent.IMPORTABLE_DEVICE_REMOVED,
+                self._on_importable_removed,
+            ),
+        ]
+
+    def _on_entry_state_changed(self, event: Event) -> None:
+        """Handle entry state change event."""
+        entry = event.data["entry"]
+        state = event.data["state"]
+        self._safe_send_message(
+            {
+                "event": DashboardEvent.ENTRY_STATE_CHANGED,
+                "data": {
+                    "filename": entry.filename,
+                    "name": entry.name,
+                    "state": entry_state_to_bool(state),
+                },
+            }
+        )
+
+    def _make_entry_handler(
+        self, event_type: DashboardEvent
+    ) -> Callable[[Event], None]:
+        """Create an entry event handler."""
+
+        def handler(event: Event) -> None:
+            self._safe_send_message(
+                {"event": event_type, "data": {"device": event.data["entry"].to_dict()}}
+            )
+
+        return handler
+
+    def _on_importable_added(self, event: Event) -> None:
+        """Handle importable device added event."""
+        # Don't send if device is already configured
+        device_name = event.data.get("device", {}).get("name")
+        if device_name and DASHBOARD.entries.get_by_name(device_name):
+            return
+        self._safe_send_message(
+            {"event": DashboardEvent.IMPORTABLE_DEVICE_ADDED, "data": event.data}
+        )
+
+    def _on_importable_removed(self, event: Event) -> None:
+        """Handle importable device removed event."""
+        self._safe_send_message(
+            {"event": DashboardEvent.IMPORTABLE_DEVICE_REMOVED, "data": event.data}
+        )
+
+    def _safe_send_message(self, message: dict[str, Any]) -> None:
+        """Send a message to the WebSocket client, ignoring closed errors."""
+        with contextlib.suppress(tornado.websocket.WebSocketClosedError):
+            self.write_message(json.dumps(message))
+
+    def on_message(self, message: str) -> None:
+        """Handle incoming WebSocket messages."""
+        _LOGGER.debug("WebSocket received message: %s", message)
+        try:
+            data = json.loads(message)
+        except json.JSONDecodeError as err:
+            _LOGGER.debug("Failed to parse WebSocket message: %s", err)
+            return
+
+        event = data.get("event")
+        _LOGGER.debug("WebSocket message event: %s", event)
+        if event == DashboardEvent.PING:
+            # Send pong response for client ping
+            _LOGGER.debug("Received client ping, sending pong")
+            self._safe_send_message({"event": DashboardEvent.PONG})
+        elif event == DashboardEvent.REFRESH:
+            # Signal the polling loop to refresh immediately
+            _LOGGER.debug("Received refresh request, signaling polling loop")
+            DASHBOARD_SUBSCRIBER.request_refresh()
+
+    def on_close(self) -> None:
+        """Handle WebSocket close."""
+        # Unsubscribe from dashboard updates
+        if self._dashboard_unsubscribe:
+            self._dashboard_unsubscribe()
+            self._dashboard_unsubscribe = None
+
+        # Unsubscribe from events
+        for remove_listener in self._event_listeners or []:
+            remove_listener()
+
+        _LOGGER.debug("Dashboard status WebSocket closed")
+
+
 class SerialPortRequestHandler(BaseHandler):
     @authenticated
     async def get(self) -> None:
@@ -475,7 +788,17 @@ class WizardRequestHandler(BaseHandler):
         kwargs = {
             k: v
             for k, v in json.loads(self.request.body.decode()).items()
-            if k in ("name", "platform", "board", "ssid", "psk", "password")
+            if k
+            in (
+                "type",
+                "name",
+                "platform",
+                "board",
+                "ssid",
+                "psk",
+                "password",
+                "file_content",
+            )
         }
         if not kwargs["name"]:
             self.set_status(422)
@@ -483,19 +806,65 @@ class WizardRequestHandler(BaseHandler):
             self.write(json.dumps({"error": "Name is required"}))
             return
 
+        if "type" not in kwargs:
+            # Default to basic wizard type for backwards compatibility
+            kwargs["type"] = "basic"
+
         kwargs["friendly_name"] = kwargs["name"]
         kwargs["name"] = friendly_name_slugify(kwargs["friendly_name"])
-
-        kwargs["ota_password"] = secrets.token_hex(16)
-        noise_psk = secrets.token_bytes(32)
-        kwargs["api_encryption_key"] = base64.b64encode(noise_psk).decode()
+        if kwargs["type"] == "basic":
+            kwargs["ota_password"] = secrets.token_hex(16)
+            noise_psk = secrets.token_bytes(32)
+            kwargs["api_encryption_key"] = base64.b64encode(noise_psk).decode()
+        elif kwargs["type"] == "upload":
+            try:
+                kwargs["file_text"] = base64.b64decode(kwargs["file_content"]).decode(
+                    "utf-8"
+                )
+            except (binascii.Error, UnicodeDecodeError):
+                self.set_status(422)
+                self.set_header("content-type", "application/json")
+                self.write(
+                    json.dumps({"error": "The uploaded file is not correctly encoded."})
+                )
+                return
+        elif kwargs["type"] != "empty":
+            self.set_status(422)
+            self.set_header("content-type", "application/json")
+            self.write(
+                json.dumps(
+                    {"error": f"Invalid wizard type specified: {kwargs['type']}"}
+                )
+            )
+            return
         filename = f"{kwargs['name']}.yaml"
         destination = settings.rel_path(filename)
-        wizard.wizard_write(path=destination, **kwargs)
-        self.set_status(200)
-        self.set_header("content-type", "application/json")
-        self.write(json.dumps({"configuration": filename}))
-        self.finish()
+
+        # Check if destination file already exists
+        if destination.exists():
+            self.set_status(409)  # Conflict status code
+            self.set_header("content-type", "application/json")
+            self.write(
+                json.dumps({"error": f"Configuration file '{filename}' already exists"})
+            )
+            self.finish()
+            return
+
+        success = wizard.wizard_write(path=destination, **kwargs)
+        if success:
+            self.set_status(200)
+            self.set_header("content-type", "application/json")
+            self.write(json.dumps({"configuration": filename}))
+            self.finish()
+        else:
+            self.set_status(500)
+            self.set_header("content-type", "application/json")
+            self.write(
+                json.dumps(
+                    {"error": "Failed to write configuration, see logs for details"}
+                )
+            )
+            self.finish()
 
 
 class ImportRequestHandler(BaseHandler):
@@ -689,10 +1058,10 @@ class DownloadBinaryRequestHandler(BaseHandler):
             "download",
             f"{storage_json.name}-{file_name}",
         )
-        path = os.path.dirname(storage_json.firmware_bin_path)
-        path = os.path.join(path, file_name)
 
-        if not Path(path).is_file():
+        path = storage_json.firmware_bin_path.parent.joinpath(file_name)
+
+        if not path.is_file():
             args = ["esphome", "idedata", settings.rel_path(configuration)]
             rc, stdout, _ = await async_run_system_command(args)
 
@@ -746,28 +1115,7 @@ class ListDevicesHandler(BaseHandler):
         await dashboard.entries.async_request_update_entries()
         entries = dashboard.entries.async_all()
         self.set_header("content-type", "application/json")
-        configured = {entry.name for entry in entries}
-
-        self.write(
-            json.dumps(
-                {
-                    "configured": [entry.to_dict() for entry in entries],
-                    "importable": [
-                        {
-                            "name": res.device_name,
-                            "friendly_name": res.friendly_name,
-                            "package_import_url": res.package_import_url,
-                            "project_name": res.project_name,
-                            "project_version": res.project_version,
-                            "network": res.network,
-                            "ignored": res.device_name in dashboard.ignored_devices,
-                        }
-                        for res in dashboard.import_result.values()
-                        if res.device_name not in configured
-                    ],
-                }
-            )
-        )
+        self.write(json.dumps(build_device_list_response(dashboard, entries)))
 
 
 class MainRequestHandler(BaseHandler):
@@ -907,7 +1255,7 @@ class EditRequestHandler(BaseHandler):
             return
 
         filename = settings.rel_path(configuration)
-        if Path(filename).resolve().parent != settings.absolute_config_dir:
+        if filename.resolve().parent != settings.absolute_config_dir:
             self.send_error(404)
             return
 
@@ -930,10 +1278,6 @@ class EditRequestHandler(BaseHandler):
             self.set_status(404)
             return None
 
-    def _write_file(self, filename: str, content: bytes) -> None:
-        """Write a file with the given content."""
-        write_file(filename, content)
-
     @authenticated
     @bind_config
     async def post(self, configuration: str | None = None) -> None:
@@ -943,12 +1287,12 @@ class EditRequestHandler(BaseHandler):
             return
 
         filename = settings.rel_path(configuration)
-        if Path(filename).resolve().parent != settings.absolute_config_dir:
+        if filename.resolve().parent != settings.absolute_config_dir:
             self.send_error(404)
             return
 
         loop = asyncio.get_running_loop()
-        await loop.run_in_executor(None, self._write_file, filename, self.request.body)
+        await loop.run_in_executor(None, write_file, filename, self.request.body)
         # Ensure the StorageJSON is updated as well
         DASHBOARD.entries.async_schedule_storage_json_update(filename)
         self.set_status(200)
@@ -963,15 +1307,12 @@ class ArchiveRequestHandler(BaseHandler):
 
         archive_path = archive_storage_path()
         mkdir_p(archive_path)
-        shutil.move(config_file, os.path.join(archive_path, configuration))
+        shutil.move(config_file, archive_path / configuration)
 
         storage_json = StorageJSON.load(storage_path)
-        if storage_json is not None:
+        if storage_json is not None and storage_json.build_path:
             # Delete build folder (if exists)
-            name = storage_json.name
-            build_folder = os.path.join(settings.config_dir, name)
-            if build_folder is not None:
-                shutil.rmtree(build_folder, os.path.join(archive_path, name))
+            shutil.rmtree(storage_json.build_path, ignore_errors=True)
 
 
 class UnArchiveRequestHandler(BaseHandler):
@@ -980,7 +1321,7 @@ class UnArchiveRequestHandler(BaseHandler):
     def post(self, configuration: str | None = None) -> None:
         config_file = settings.rel_path(configuration)
         archive_path = archive_storage_path()
-        shutil.move(os.path.join(archive_path, configuration), config_file)
+        shutil.move(archive_path / configuration, config_file)
 
 
 class LoginHandler(BaseHandler):
@@ -1067,7 +1408,7 @@ class SecretKeysRequestHandler(BaseHandler):
 
         for secret_filename in const.SECRETS_FILES:
             relative_filename = settings.rel_path(secret_filename)
-            if os.path.isfile(relative_filename):
+            if relative_filename.is_file():
                 filename = relative_filename
                 break
 
@@ -1100,16 +1441,17 @@ class JsonConfigRequestHandler(BaseHandler):
     @bind_config
     async def get(self, configuration: str | None = None) -> None:
         filename = settings.rel_path(configuration)
-        if not os.path.isfile(filename):
+        if not filename.is_file():
             self.send_error(404)
             return
 
-        args = ["esphome", "config", filename, "--show-secrets"]
+        args = ["esphome", "config", str(filename), "--show-secrets"]
 
-        rc, stdout, _ = await async_run_system_command(args)
+        rc, stdout, stderr = await async_run_system_command(args)
 
         if rc != 0:
-            self.send_error(422)
+            self.set_status(422)
+            self.write(stderr)
             return
 
         data = yaml.load(stdout, Loader=SafeLoaderIgnoreUnknown)
@@ -1118,7 +1460,7 @@ class JsonConfigRequestHandler(BaseHandler):
         self.finish()
 
 
-def get_base_frontend_path() -> str:
+def get_base_frontend_path() -> Path:
     if ENV_DEV not in os.environ:
         import esphome_dashboard
 
@@ -1129,11 +1471,12 @@ def get_base_frontend_path() -> str:
         static_path += "/"
 
     # This path can be relative, so resolve against the root or else templates don't work
-    return os.path.abspath(os.path.join(os.getcwd(), static_path, "esphome_dashboard"))
+    path = Path(os.getcwd()) / static_path / "esphome_dashboard"
+    return path.resolve()
 
 
-def get_static_path(*args: Iterable[str]) -> str:
-    return os.path.join(get_base_frontend_path(), "static", *args)
+def get_static_path(*args: Iterable[str]) -> Path:
+    return get_base_frontend_path() / "static" / Path(*args)
 
 
 @functools.cache
@@ -1150,8 +1493,7 @@ def get_static_file_url(name: str) -> str:
         return base.replace("index.js", esphome_dashboard.entrypoint())
 
     path = get_static_path(name)
-    with open(path, "rb") as f_handle:
-        hash_ = hashlib.md5(f_handle.read()).hexdigest()[:8]
+    hash_ = hashlib.md5(path.read_bytes()).hexdigest()[:8]
     return f"{base}?hash={hash_}"
 
 
@@ -1211,6 +1553,7 @@ def make_app(debug=get_bool_env(ENV_DEV)) -> tornado.web.Application:
             (f"{rel}compile", EsphomeCompileHandler),
             (f"{rel}validate", EsphomeValidateHandler),
             (f"{rel}clean-mqtt", EsphomeCleanMqttHandler),
+            (f"{rel}clean-all", EsphomeCleanAllHandler),
             (f"{rel}clean", EsphomeCleanHandler),
             (f"{rel}vscode", EsphomeVscodeHandler),
             (f"{rel}ace", EsphomeAceEditorHandler),
@@ -1228,6 +1571,7 @@ def make_app(debug=get_bool_env(ENV_DEV)) -> tornado.web.Application:
             (f"{rel}wizard", WizardRequestHandler),
             (f"{rel}static/(.*)", StaticFileHandler, {"path": get_static_path()}),
             (f"{rel}devices", ListDevicesHandler),
+            (f"{rel}events", DashboardEventsWebSocket),
             (f"{rel}import", ImportRequestHandler),
             (f"{rel}secret_keys", SecretKeysRequestHandler),
             (f"{rel}json-config", JsonConfigRequestHandler),
@@ -1251,7 +1595,7 @@ def start_web_server(
     """Start the web server listener."""
 
     trash_path = trash_storage_path()
-    if os.path.exists(trash_path):
+    if trash_path.is_dir() and trash_path.exists():
         _LOGGER.info("Renaming 'trash' folder to 'archive'")
         archive_path = archive_storage_path()
         shutil.move(trash_path, archive_path)
diff --git a/esphome/espota2.py b/esphome/espota2.py
index 4f2a08fb4a..2b1b9a8328 100644
--- a/esphome/espota2.py
+++ b/esphome/espota2.py
@@ -1,19 +1,23 @@
 from __future__ import annotations
 
+from collections.abc import Callable
 import gzip
 import hashlib
 import io
 import logging
+from pathlib import Path
 import random
 import socket
 import sys
 import time
+from typing import Any
 
 from esphome.core import EsphomeError
 from esphome.helpers import resolve_ip_address
 
 RESPONSE_OK = 0x00
 RESPONSE_REQUEST_AUTH = 0x01
+RESPONSE_REQUEST_SHA256_AUTH = 0x02
 
 RESPONSE_HEADER_OK = 0x40
 RESPONSE_AUTH_OK = 0x41
@@ -44,6 +48,7 @@ OTA_VERSION_2_0 = 2
 MAGIC_BYTES = [0x6C, 0x26, 0xF7, 0x5C, 0x45]
 
 FEATURE_SUPPORTS_COMPRESSION = 0x01
+FEATURE_SUPPORTS_SHA256_AUTH = 0x02
 
 
 UPLOAD_BLOCK_SIZE = 8192
@@ -51,6 +56,12 @@ UPLOAD_BUFFER_SIZE = UPLOAD_BLOCK_SIZE * 8
 
 _LOGGER = logging.getLogger(__name__)
 
+# Authentication method lookup table: response -> (hash_func, nonce_size, name)
+_AUTH_METHODS: dict[int, tuple[Callable[..., Any], int, str]] = {
+    RESPONSE_REQUEST_SHA256_AUTH: (hashlib.sha256, 64, "SHA256"),
+    RESPONSE_REQUEST_AUTH: (hashlib.md5, 32, "MD5"),
+}
+
 
 class ProgressBar:
     def __init__(self):
@@ -80,21 +91,43 @@ class OTAError(EsphomeError):
     pass
 
 
-def recv_decode(sock, amount, decode=True):
+def recv_decode(
+    sock: socket.socket, amount: int, decode: bool = True
+) -> bytes | list[int]:
+    """Receive data from socket and optionally decode to list of integers.
+
+    :param sock: Socket to receive data from.
+    :param amount: Number of bytes to receive.
+    :param decode: If True, convert bytes to list of integers, otherwise return raw bytes.
+    :return: List of integers if decode=True, otherwise raw bytes.
+    """
     data = sock.recv(amount)
     if not decode:
         return data
     return list(data)
 
 
-def receive_exactly(sock, amount, msg, expect, decode=True):
-    if decode:
-        data = []
-    else:
-        data = b""
+def receive_exactly(
+    sock: socket.socket,
+    amount: int,
+    msg: str,
+    expect: int | list[int] | None,
+    decode: bool = True,
+) -> list[int] | bytes:
+    """Receive exactly the specified amount of data from socket with error checking.
+
+    :param sock: Socket to receive data from.
+    :param amount: Exact number of bytes to receive.
+    :param msg: Description of what is being received for error messages.
+    :param expect: Expected response code(s) for validation, None to skip validation.
+    :param decode: If True, return list of integers, otherwise return raw bytes.
+    :return: List of integers if decode=True, otherwise raw bytes.
+    :raises OTAError: If receiving fails or response doesn't match expected.
+    """
+    data: list[int] | bytes = [] if decode else b""
 
     try:
-        data += recv_decode(sock, 1, decode=decode)
+        data += recv_decode(sock, 1, decode=decode)  # type: ignore[operator]
     except OSError as err:
         raise OTAError(f"Error receiving acknowledge {msg}: {err}") from err
 
@@ -106,13 +139,19 @@ def receive_exactly(sock, amount, msg, expect, decode=True):
 
     while len(data) < amount:
         try:
-            data += recv_decode(sock, amount - len(data), decode=decode)
+            data += recv_decode(sock, amount - len(data), decode=decode)  # type: ignore[operator]
         except OSError as err:
             raise OTAError(f"Error receiving {msg}: {err}") from err
     return data
 
 
-def check_error(data, expect):
+def check_error(data: list[int] | bytes, expect: int | list[int] | None) -> None:
+    """Check response data for error codes and validate against expected response.
+
+    :param data: Response data from device (first byte is the response code).
+    :param expect: Expected response code(s), None to skip validation.
+    :raises OTAError: If an error code is detected or response doesn't match expected.
+    """
     if not expect:
         return
     dat = data[0]
@@ -127,7 +166,7 @@ def check_error(data, expect):
         raise OTAError("Error: Authentication invalid. Is the password correct?")
     if dat == RESPONSE_ERROR_WRITING_FLASH:
         raise OTAError(
-            "Error: Wring OTA data to flash memory failed. See USB logs for more "
+            "Error: Writing OTA data to flash memory failed. See USB logs for more "
             "information."
         )
     if dat == RESPONSE_ERROR_UPDATE_END:
@@ -179,7 +218,16 @@ def check_error(data, expect):
         raise OTAError(f"Unexpected response from ESP: 0x{data[0]:02X}")
 
 
-def send_check(sock, data, msg):
+def send_check(
+    sock: socket.socket, data: list[int] | tuple[int, ...] | int | str | bytes, msg: str
+) -> None:
+    """Send data to socket with error handling.
+
+    :param sock: Socket to send data to.
+    :param data: Data to send (can be list/tuple of ints, single int, string, or bytes).
+    :param msg: Description of what is being sent for error messages.
+    :raises OTAError: If sending fails.
+    """
     try:
         if isinstance(data, (list, tuple)):
             data = bytes(data)
@@ -194,7 +242,7 @@ def send_check(sock, data, msg):
 
 
 def perform_ota(
-    sock: socket.socket, password: str, file_handle: io.IOBase, filename: str
+    sock: socket.socket, password: str | None, file_handle: io.IOBase, filename: Path
 ) -> None:
     file_contents = file_handle.read()
     file_size = len(file_contents)
@@ -212,10 +260,14 @@ def perform_ota(
             f"Device uses unsupported OTA version {version}, this ESPHome supports {supported_versions}"
         )
 
-    # Features
-    send_check(sock, FEATURE_SUPPORTS_COMPRESSION, "features")
+    # Features - send both compression and SHA256 auth support
+    features_to_send = FEATURE_SUPPORTS_COMPRESSION | FEATURE_SUPPORTS_SHA256_AUTH
+    send_check(sock, features_to_send, "features")
     features = receive_exactly(
-        sock, 1, "features", [RESPONSE_HEADER_OK, RESPONSE_SUPPORTS_COMPRESSION]
+        sock,
+        1,
+        "features",
+        None,  # Accept any response
     )[0]
 
     if features == RESPONSE_SUPPORTS_COMPRESSION:
@@ -224,31 +276,52 @@ def perform_ota(
     else:
         upload_contents = file_contents
 
-    (auth,) = receive_exactly(
-        sock, 1, "auth", [RESPONSE_REQUEST_AUTH, RESPONSE_AUTH_OK]
-    )
-    if auth == RESPONSE_REQUEST_AUTH:
-        if not password:
+    def perform_auth(
+        sock: socket.socket,
+        password: str | None,
+        hash_func: Callable[..., Any],
+        nonce_size: int,
+        hash_name: str,
+    ) -> None:
+        """Perform challenge-response authentication using specified hash algorithm."""
+        if password is None:
             raise OTAError("ESP requests password, but no password given!")
-        nonce = receive_exactly(
-            sock, 32, "authentication nonce", [], decode=False
-        ).decode()
-        _LOGGER.debug("Auth: Nonce is %s", nonce)
-        cnonce = hashlib.md5(str(random.random()).encode()).hexdigest()
-        _LOGGER.debug("Auth: CNonce is %s", cnonce)
+
+        nonce_bytes = receive_exactly(
+            sock, nonce_size, f"{hash_name} authentication nonce", [], decode=False
+        )
+        assert isinstance(nonce_bytes, bytes)
+        nonce = nonce_bytes.decode()
+        _LOGGER.debug("Auth: %s Nonce is %s", hash_name, nonce)
+
+        # Generate cnonce
+        cnonce = hash_func(str(random.random()).encode()).hexdigest()
+        _LOGGER.debug("Auth: %s CNonce is %s", hash_name, cnonce)
 
         send_check(sock, cnonce, "auth cnonce")
 
-        result_md5 = hashlib.md5()
-        result_md5.update(password.encode("utf-8"))
-        result_md5.update(nonce.encode())
-        result_md5.update(cnonce.encode())
-        result = result_md5.hexdigest()
-        _LOGGER.debug("Auth: Result is %s", result)
+        # Calculate challenge response
+        hasher = hash_func()
+        hasher.update(password.encode("utf-8"))
+        hasher.update(nonce.encode())
+        hasher.update(cnonce.encode())
+        result = hasher.hexdigest()
+        _LOGGER.debug("Auth: %s Result is %s", hash_name, result)
 
         send_check(sock, result, "auth result")
         receive_exactly(sock, 1, "auth result", RESPONSE_AUTH_OK)
 
+    (auth,) = receive_exactly(
+        sock,
+        1,
+        "auth",
+        [RESPONSE_REQUEST_AUTH, RESPONSE_REQUEST_SHA256_AUTH, RESPONSE_AUTH_OK],
+    )
+
+    if auth != RESPONSE_AUTH_OK:
+        hash_func, nonce_size, hash_name = _AUTH_METHODS[auth]
+        perform_auth(sock, password, hash_func, nonce_size, hash_name)
+
     # Set higher timeout during upload
     sock.settimeout(30.0)
 
@@ -311,9 +384,17 @@ def perform_ota(
     time.sleep(1)
 
 
-def run_ota_impl_(remote_host, remote_port, password, filename):
+def run_ota_impl_(
+    remote_host: str | list[str], remote_port: int, password: str | None, filename: Path
+) -> tuple[int, str | None]:
+    from esphome.core import CORE
+
+    # Handle both single host and list of hosts
     try:
-        res = resolve_ip_address(remote_host, remote_port)
+        # Resolve all hosts at once for parallel DNS resolution
+        res = resolve_ip_address(
+            remote_host, remote_port, address_cache=CORE.address_cache
+        )
     except EsphomeError as err:
         _LOGGER.error(
             "Error resolving IP address of %s. Is it connected to WiFi?",
@@ -329,7 +410,7 @@ def run_ota_impl_(remote_host, remote_port, password, filename):
         af, socktype, _, _, sa = r
         _LOGGER.info("Connecting to %s port %s...", sa[0], sa[1])
         sock = socket.socket(af, socktype)
-        sock.settimeout(10.0)
+        sock.settimeout(20.0)
         try:
             sock.connect(sa)
         except OSError as err:
@@ -343,19 +424,22 @@ def run_ota_impl_(remote_host, remote_port, password, filename):
                 perform_ota(sock, password, file_handle, filename)
             except OTAError as err:
                 _LOGGER.error(str(err))
-                return 1
+                return 1, None
             finally:
                 sock.close()
 
-        return 0
+        # Successfully uploaded to sa[0]
+        return 0, sa[0]
 
     _LOGGER.error("Connection failed.")
-    return 1
+    return 1, None
 
 
-def run_ota(remote_host, remote_port, password, filename):
+def run_ota(
+    remote_host: str | list[str], remote_port: int, password: str | None, filename: Path
+) -> tuple[int, str | None]:
     try:
         return run_ota_impl_(remote_host, remote_port, password, filename)
     except OTAError as err:
         _LOGGER.error(err)
-        return 1
+        return 1, None
diff --git a/esphome/external_files.py b/esphome/external_files.py
index 057ff52f3f..80b54ebb2f 100644
--- a/esphome/external_files.py
+++ b/esphome/external_files.py
@@ -2,7 +2,6 @@ from __future__ import annotations
 
 from datetime import datetime
 import logging
-import os
 from pathlib import Path
 
 import requests
@@ -23,11 +22,11 @@ CONTENT_DISPOSITION = "content-disposition"
 TEMP_DIR = "temp"
 
 
-def has_remote_file_changed(url, local_file_path):
-    if os.path.exists(local_file_path):
+def has_remote_file_changed(url: str, local_file_path: Path) -> bool:
+    if local_file_path.exists():
         _LOGGER.debug("has_remote_file_changed: File exists at %s", local_file_path)
         try:
-            local_modification_time = os.path.getmtime(local_file_path)
+            local_modification_time = local_file_path.stat().st_mtime
             local_modification_time_str = datetime.utcfromtimestamp(
                 local_modification_time
             ).strftime("%a, %d %b %Y %H:%M:%S GMT")
@@ -65,9 +64,9 @@ def has_remote_file_changed(url, local_file_path):
     return True
 
 
-def is_file_recent(file_path: str, refresh: TimePeriodSeconds) -> bool:
-    if os.path.exists(file_path):
-        creation_time = os.path.getctime(file_path)
+def is_file_recent(file_path: Path, refresh: TimePeriodSeconds) -> bool:
+    if file_path.exists():
+        creation_time = file_path.stat().st_ctime
         current_time = datetime.now().timestamp()
         return current_time - creation_time <= refresh.total_seconds
     return False
diff --git a/esphome/git.py b/esphome/git.py
index 005bcae702..4ff07ffe75 100644
--- a/esphome/git.py
+++ b/esphome/git.py
@@ -5,6 +5,7 @@ import hashlib
 import logging
 from pathlib import Path
 import re
+import shutil
 import subprocess
 import urllib.parse
 
@@ -13,13 +14,64 @@ from esphome.core import CORE, TimePeriodSeconds
 
 _LOGGER = logging.getLogger(__name__)
 
+# Special value to indicate never refresh
+NEVER_REFRESH = TimePeriodSeconds(seconds=-1)
+
+
+class GitException(cv.Invalid):
+    """Base exception for git-related errors."""
+
+
+class GitNotInstalledError(GitException):
+    """Exception raised when git is not installed on the system."""
+
+
+class GitCommandError(GitException):
+    """Exception raised when a git command fails."""
+
+
+class GitRepositoryError(GitException):
+    """Exception raised when a git repository is in an invalid state."""
+
+
+def run_git_command(cmd: list[str], git_dir: Path | None = None) -> str:
+    if git_dir is not None:
+        _LOGGER.debug(
+            "Running git command with repository isolation: %s (git_dir=%s)",
+            " ".join(cmd),
+            git_dir,
+        )
+    else:
+        _LOGGER.debug("Running git command: %s", " ".join(cmd))
+
+    # Set up environment for repository isolation if git_dir is provided
+    # Force git to only operate on this specific repository by setting
+    # GIT_DIR and GIT_WORK_TREE. This prevents git from walking up the
+    # directory tree to find parent repositories when the target repo's
+    # .git directory is corrupt. Without this, commands like 'git stash'
+    # could accidentally operate on parent repositories (e.g., the main
+    # ESPHome repo) instead of failing, causing data loss.
+    env: dict[str, str] | None = None
+    cwd: str | None = None
+    if git_dir is not None:
+        env = {
+            **subprocess.os.environ,
+            "GIT_DIR": str(Path(git_dir) / ".git"),
+            "GIT_WORK_TREE": str(git_dir),
+        }
+        cwd = str(git_dir)
 
-def run_git_command(cmd, cwd=None) -> str:
-    _LOGGER.debug("Running git command: %s", " ".join(cmd))
     try:
-        ret = subprocess.run(cmd, cwd=cwd, capture_output=True, check=False)
+        ret = subprocess.run(
+            cmd,
+            cwd=cwd,
+            capture_output=True,
+            check=False,
+            close_fds=False,
+            env=env,
+        )
     except FileNotFoundError as err:
-        raise cv.Invalid(
+        raise GitNotInstalledError(
             "git is not installed but required for external_components.\n"
             "Please see https://git-scm.com/book/en/v2/Getting-Started-Installing-Git for installing git"
         ) from err
@@ -28,8 +80,8 @@ def run_git_command(cmd, cwd=None) -> str:
         err_str = ret.stderr.decode("utf-8")
         lines = [x.strip() for x in err_str.splitlines()]
         if lines[-1].startswith("fatal:"):
-            raise cv.Invalid(lines[-1][len("fatal: ") :])
-        raise cv.Invalid(err_str)
+            raise GitCommandError(lines[-1][len("fatal: ") :])
+        raise GitCommandError(err_str)
 
     return ret.stdout.decode("utf-8").strip()
 
@@ -50,6 +102,7 @@ def clone_or_update(
     username: str = None,
     password: str = None,
     submodules: list[str] | None = None,
+    _recover_broken: bool = True,
 ) -> tuple[Path, Callable[[], None] | None]:
     key = f"{url}@{ref}"
 
@@ -70,51 +123,106 @@ def clone_or_update(
             # We need to fetch the PR branch first, otherwise git will complain
             # about missing objects
             _LOGGER.info("Fetching %s", ref)
-            run_git_command(["git", "fetch", "--", "origin", ref], str(repo_dir))
-            run_git_command(["git", "reset", "--hard", "FETCH_HEAD"], str(repo_dir))
+            run_git_command(["git", "fetch", "--", "origin", ref], git_dir=repo_dir)
+            run_git_command(["git", "reset", "--hard", "FETCH_HEAD"], git_dir=repo_dir)
 
         if submodules is not None:
             _LOGGER.info(
-                "Initialising submodules (%s) for %s", ", ".join(submodules), key
+                "Initializing submodules (%s) for %s", ", ".join(submodules), key
             )
             run_git_command(
-                ["git", "submodule", "update", "--init"] + submodules, str(repo_dir)
+                ["git", "submodule", "update", "--init"] + submodules, git_dir=repo_dir
             )
 
     else:
         # Check refresh needed
+        # Skip refresh if NEVER_REFRESH is specified
+        if refresh == NEVER_REFRESH:
+            _LOGGER.debug("Skipping update for %s (refresh disabled)", key)
+            return repo_dir, None
+
         file_timestamp = Path(repo_dir / ".git" / "FETCH_HEAD")
         # On first clone, FETCH_HEAD does not exists
         if not file_timestamp.exists():
             file_timestamp = Path(repo_dir / ".git" / "HEAD")
         age = datetime.now() - datetime.fromtimestamp(file_timestamp.stat().st_mtime)
         if refresh is None or age.total_seconds() > refresh.total_seconds:
-            old_sha = run_git_command(["git", "rev-parse", "HEAD"], str(repo_dir))
-            _LOGGER.info("Updating %s", key)
-            _LOGGER.debug("Location: %s", repo_dir)
-            # Stash local changes (if any)
-            run_git_command(
-                ["git", "stash", "push", "--include-untracked"], str(repo_dir)
-            )
-            # Fetch remote ref
-            cmd = ["git", "fetch", "--", "origin"]
-            if ref is not None:
-                cmd.append(ref)
-            run_git_command(cmd, str(repo_dir))
-            # Hard reset to FETCH_HEAD (short-lived git ref corresponding to most recent fetch)
-            run_git_command(["git", "reset", "--hard", "FETCH_HEAD"], str(repo_dir))
+            # Try to update the repository, recovering from broken state if needed
+            old_sha: str | None = None
+            try:
+                # First verify the repository is valid by checking HEAD
+                # Use git_dir parameter to prevent git from walking up to parent repos
+                old_sha = run_git_command(
+                    ["git", "rev-parse", "HEAD"], git_dir=repo_dir
+                )
+
+                _LOGGER.info("Updating %s", key)
+                _LOGGER.debug("Location: %s", repo_dir)
+
+                # Stash local changes (if any)
+                # Use git_dir to ensure this only affects the specific repo
+                run_git_command(
+                    ["git", "stash", "push", "--include-untracked"],
+                    git_dir=repo_dir,
+                )
+
+                # Fetch remote ref
+                cmd = ["git", "fetch", "--", "origin"]
+                if ref is not None:
+                    cmd.append(ref)
+                run_git_command(cmd, git_dir=repo_dir)
+
+                # Hard reset to FETCH_HEAD (short-lived git ref corresponding to most recent fetch)
+                run_git_command(
+                    ["git", "reset", "--hard", "FETCH_HEAD"],
+                    git_dir=repo_dir,
+                )
+            except GitException as err:
+                # Repository is in a broken state or update failed
+                # Only attempt recovery once to prevent infinite recursion
+                if not _recover_broken:
+                    _LOGGER.error(
+                        "Repository %s recovery failed, cannot retry (already attempted once)",
+                        key,
+                    )
+                    raise
+
+                _LOGGER.warning(
+                    "Repository %s has issues (%s), attempting recovery",
+                    key,
+                    err,
+                )
+                _LOGGER.info("Removing broken repository at %s", repo_dir)
+                shutil.rmtree(repo_dir)
+                _LOGGER.info("Successfully removed broken repository, re-cloning...")
+
+                # Recursively call clone_or_update to re-clone
+                # Set _recover_broken=False to prevent infinite recursion
+                result = clone_or_update(
+                    url=url,
+                    ref=ref,
+                    refresh=refresh,
+                    domain=domain,
+                    username=username,
+                    password=password,
+                    submodules=submodules,
+                    _recover_broken=False,
+                )
+                _LOGGER.info("Repository %s successfully recovered", key)
+                return result
 
             if submodules is not None:
                 _LOGGER.info(
                     "Updating submodules (%s) for %s", ", ".join(submodules), key
                 )
                 run_git_command(
-                    ["git", "submodule", "update", "--init"] + submodules, str(repo_dir)
+                    ["git", "submodule", "update", "--init"] + submodules,
+                    git_dir=repo_dir,
                 )
 
             def revert():
                 _LOGGER.info("Reverting changes to %s -> %s", key, old_sha)
-                run_git_command(["git", "reset", "--hard", old_sha], str(repo_dir))
+                run_git_command(["git", "reset", "--hard", old_sha], git_dir=repo_dir)
 
             return repo_dir, revert
 
diff --git a/esphome/helpers.py b/esphome/helpers.py
index bf0e3b5cf7..fb7b71775d 100644
--- a/esphome/helpers.py
+++ b/esphome/helpers.py
@@ -1,4 +1,5 @@
-import codecs
+from __future__ import annotations
+
 from contextlib import suppress
 import ipaddress
 import logging
@@ -6,9 +7,28 @@ import os
 from pathlib import Path
 import platform
 import re
+import shutil
 import tempfile
+from typing import TYPE_CHECKING
 from urllib.parse import urlparse
 
+from esphome.const import __version__ as ESPHOME_VERSION
+
+if TYPE_CHECKING:
+    from esphome.address_cache import AddressCache
+
+# Type aliases for socket address information
+AddrInfo = tuple[
+    int,  # family (AF_INET, AF_INET6, etc.)
+    int,  # type (SOCK_STREAM, SOCK_DGRAM, etc.)
+    int,  # proto (IPPROTO_TCP, etc.)
+    str,  # canonname
+    tuple[str, int] | tuple[str, int, int, int],  # sockaddr (IPv4 or IPv6)
+]
+IPv4SockAddr = tuple[str, int]  # (host, port)
+IPv6SockAddr = tuple[str, int, int, int]  # (host, port, flowinfo, scope_id)
+SockAddr = IPv4SockAddr | IPv6SockAddr
+
 _LOGGER = logging.getLogger(__name__)
 
 IS_MACOS = platform.system() == "Darwin"
@@ -96,9 +116,7 @@ def cpp_string_escape(string, encoding="utf-8"):
     def _should_escape(byte: int) -> bool:
         if not 32 <= byte < 127:
             return True
-        if byte in (ord("\\"), ord('"')):
-            return True
-        return False
+        return byte in (ord("\\"), ord('"'))
 
     if isinstance(string, str):
         string = string.encode(encoding)
@@ -114,22 +132,24 @@ def cpp_string_escape(string, encoding="utf-8"):
 def run_system_command(*args):
     import subprocess
 
-    with subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE) as p:
+    with subprocess.Popen(
+        args, stdout=subprocess.PIPE, stderr=subprocess.PIPE, close_fds=False
+    ) as p:
         stdout, stderr = p.communicate()
         rc = p.returncode
         return rc, stdout, stderr
 
 
-def mkdir_p(path):
+def mkdir_p(path: Path):
     if not path:
         # Empty path - means create current dir
         return
     try:
-        os.makedirs(path)
+        path.mkdir(parents=True, exist_ok=True)
     except OSError as err:
         import errno
 
-        if err.errno == errno.EEXIST and os.path.isdir(path):
+        if err.errno == errno.EEXIST and path.is_dir():
             pass
         else:
             from esphome.core import EsphomeError
@@ -145,32 +165,7 @@ def is_ip_address(host):
         return False
 
 
-def _resolve_with_zeroconf(host):
-    from esphome.core import EsphomeError
-    from esphome.zeroconf import EsphomeZeroconf
-
-    try:
-        zc = EsphomeZeroconf()
-    except Exception as err:
-        raise EsphomeError(
-            "Cannot start mDNS sockets, is this a docker container without "
-            "host network mode?"
-        ) from err
-    try:
-        info = zc.resolve_host(f"{host}.")
-    except Exception as err:
-        raise EsphomeError(f"Error resolving mDNS hostname: {err}") from err
-    finally:
-        zc.close()
-    if info is None:
-        raise EsphomeError(
-            "Error resolving address with mDNS: Did not respond. "
-            "Maybe the device is offline."
-        )
-    return info
-
-
-def addr_preference_(res):
+def addr_preference_(res: AddrInfo) -> int:
     # Trivial alternative to RFC6724 sorting. Put sane IPv6 first, then
     # Legacy IP, then IPv6 link-local addresses without an actual link.
     sa = res[4]
@@ -182,10 +177,25 @@ def addr_preference_(res):
     return 1
 
 
-def resolve_ip_address(host, port):
+def _add_ip_addresses_to_addrinfo(
+    addresses: list[str], port: int, res: list[AddrInfo]
+) -> None:
+    """Helper to add IP addresses to addrinfo results with error handling."""
     import socket
 
-    from esphome.core import EsphomeError
+    for addr in addresses:
+        try:
+            res += socket.getaddrinfo(
+                addr, port, proto=socket.IPPROTO_TCP, flags=socket.AI_NUMERICHOST
+            )
+        except OSError:
+            _LOGGER.debug("Failed to parse IP address '%s'", addr)
+
+
+def resolve_ip_address(
+    host: str | list[str], port: int, address_cache: AddressCache | None = None
+) -> list[AddrInfo]:
+    import socket
 
     # There are five cases here. The host argument could be one of:
     #  • a *list* of IP addresses discovered by MQTT,
@@ -193,55 +203,83 @@ def resolve_ip_address(host, port):
     #  • a .local hostname to be resolved by mDNS,
     #  • a normal hostname to be resolved in DNS, or
     #  • A URL from which we should extract the hostname.
-    #
-    # In each of the first three cases, we end up with IP addresses in
-    # string form which need to be converted to a 5-tuple to be used
-    # for the socket connection attempt. The easiest way to construct
-    # those is to pass the IP address string to getaddrinfo(). Which,
-    # coincidentally, is how we do hostname lookups in the other cases
-    # too. So first build a list which contains either IP addresses or
-    # a single hostname, then call getaddrinfo() on each element of
-    # that list.
 
-    errs = []
+    hosts: list[str]
     if isinstance(host, list):
-        addr_list = host
-    elif is_ip_address(host):
-        addr_list = [host]
+        hosts = host
     else:
-        url = urlparse(host)
-        if url.scheme != "":
-            host = url.hostname
+        if not is_ip_address(host):
+            url = urlparse(host)
+            if url.scheme != "":
+                host = url.hostname
+        hosts = [host]
 
-        addr_list = []
-        if host.endswith(".local"):
-            try:
-                _LOGGER.info("Resolving IP address of %s in mDNS", host)
-                addr_list = _resolve_with_zeroconf(host)
-            except EsphomeError as err:
-                errs.append(str(err))
+    res: list[AddrInfo] = []
 
-        # If not mDNS, or if mDNS failed, use normal DNS
-        if not addr_list:
-            addr_list = [host]
+    # Fast path: if all hosts are already IP addresses
+    if all(is_ip_address(h) for h in hosts):
+        _add_ip_addresses_to_addrinfo(hosts, port, res)
+        # Sort by preference
+        res.sort(key=addr_preference_)
+        return res
 
-    # Now we have a list containing either IP addresses or a hostname
-    res = []
-    for addr in addr_list:
-        if not is_ip_address(addr):
-            _LOGGER.info("Resolving IP address of %s", host)
-        try:
-            r = socket.getaddrinfo(addr, port, proto=socket.IPPROTO_TCP)
-        except OSError as err:
-            errs.append(str(err))
-            raise EsphomeError(
-                f"Error resolving IP address: {', '.join(errs)}"
-            ) from err
+    # Process hosts
+    cached_addresses: list[str] = []
+    uncached_hosts: list[str] = []
+    has_cache = address_cache is not None
 
-        res = res + r
+    for h in hosts:
+        if is_ip_address(h):
+            if has_cache:
+                # If we have a cache, treat IPs as cached
+                cached_addresses.append(h)
+            else:
+                # If no cache, pass IPs through to resolver with hostnames
+                uncached_hosts.append(h)
+        elif address_cache and (cached := address_cache.get_addresses(h)):
+            # Found in cache
+            cached_addresses.extend(cached)
+        else:
+            # Not cached, need to resolve
+            if address_cache and address_cache.has_cache():
+                _LOGGER.info("Host %s not in cache, will need to resolve", h)
+            uncached_hosts.append(h)
 
-    # Zeroconf tends to give us link-local IPv6 addresses without specifying
-    # the link. Put those last in the list to be attempted.
+    # Process cached addresses (includes direct IPs and cached lookups)
+    _add_ip_addresses_to_addrinfo(cached_addresses, port, res)
+
+    # If we have uncached hosts (only non-IP hostnames), resolve them
+    if uncached_hosts:
+        from esphome.resolver import AsyncResolver
+
+        resolver = AsyncResolver(uncached_hosts, port)
+        addr_infos = resolver.resolve()
+        # Convert aioesphomeapi AddrInfo to our format
+        for addr_info in addr_infos:
+            sockaddr = addr_info.sockaddr
+            if addr_info.family == socket.AF_INET6:
+                # IPv6
+                sockaddr_tuple = (
+                    sockaddr.address,
+                    sockaddr.port,
+                    sockaddr.flowinfo,
+                    sockaddr.scope_id,
+                )
+            else:
+                # IPv4
+                sockaddr_tuple = (sockaddr.address, sockaddr.port)
+
+            res.append(
+                (
+                    addr_info.family,
+                    addr_info.type,
+                    addr_info.proto,
+                    "",  # canonname
+                    sockaddr_tuple,
+                )
+            )
+
+    # Sort by preference
     res.sort(key=addr_preference_)
     return res
 
@@ -260,23 +298,8 @@ def sort_ip_addresses(address_list: list[str]) -> list[str]:
 
     # First "resolve" all the IP addresses to getaddrinfo() tuples of the form
     # (family, type, proto, canonname, sockaddr)
-    res: list[
-        tuple[
-            int,
-            int,
-            int,
-            str | None,
-            tuple[str, int] | tuple[str, int, int, int],
-        ]
-    ] = []
-    for addr in address_list:
-        # This should always work as these are supposed to be IP addresses
-        try:
-            res += socket.getaddrinfo(
-                addr, 0, proto=socket.IPPROTO_TCP, flags=socket.AI_NUMERICHOST
-            )
-        except OSError:
-            _LOGGER.info("Failed to parse IP address '%s'", addr)
+    res: list[AddrInfo] = []
+    _add_ip_addresses_to_addrinfo(address_list, 0, res)
 
     # Now use that information to sort them.
     res.sort(key=addr_preference_)
@@ -308,16 +331,15 @@ def is_ha_addon():
     return get_bool_env("ESPHOME_IS_HA_ADDON")
 
 
-def walk_files(path):
+def walk_files(path: Path):
     for root, _, files in os.walk(path):
         for name in files:
-            yield os.path.join(root, name)
+            yield Path(root) / name
 
 
-def read_file(path):
+def read_file(path: Path) -> str:
     try:
-        with codecs.open(path, "r", encoding="utf-8") as f_handle:
-            return f_handle.read()
+        return path.read_text(encoding="utf-8")
     except OSError as err:
         from esphome.core import EsphomeError
 
@@ -328,13 +350,15 @@ def read_file(path):
         raise EsphomeError(f"Error reading file {path}: {err}") from err
 
 
-def _write_file(path: Path | str, text: str | bytes):
+def _write_file(
+    path: Path,
+    text: str | bytes,
+    private: bool = False,
+) -> None:
     """Atomically writes `text` to the given path.
 
     Automatically creates all parent directories.
     """
-    if not isinstance(path, Path):
-        path = Path(path)
     data = text
     if isinstance(text, str):
         data = text.encode()
@@ -342,42 +366,54 @@ def _write_file(path: Path | str, text: str | bytes):
     directory = path.parent
     directory.mkdir(exist_ok=True, parents=True)
 
-    tmp_path = None
+    tmp_filename: Path | None = None
+    missing_fchmod = False
     try:
+        # Modern versions of Python tempfile create this file with mode 0o600
         with tempfile.NamedTemporaryFile(
             mode="wb", dir=directory, delete=False
         ) as f_handle:
-            tmp_path = f_handle.name
             f_handle.write(data)
-        # Newer tempfile implementations create the file with mode 0o600
-        os.chmod(tmp_path, 0o644)
-        # If destination exists, will be overwritten
-        os.replace(tmp_path, path)
+            tmp_filename = Path(f_handle.name)
+
+            if not private:
+                try:
+                    os.fchmod(f_handle.fileno(), 0o644)
+                except AttributeError:
+                    # os.fchmod is not available on Windows
+                    missing_fchmod = True
+        shutil.move(tmp_filename, path)
+        if missing_fchmod:
+            path.chmod(0o644)
     finally:
-        if tmp_path is not None and os.path.exists(tmp_path):
+        if tmp_filename and tmp_filename.exists():
             try:
-                os.remove(tmp_path)
+                tmp_filename.unlink()
             except OSError as err:
-                _LOGGER.error("Write file cleanup failed: %s", err)
+                # If we are cleaning up then something else went wrong, so
+                # we should suppress likely follow-on errors in the cleanup
+                _LOGGER.error(
+                    "File replacement cleanup failed for %s while saving %s: %s",
+                    tmp_filename,
+                    path,
+                    err,
+                )
 
 
-def write_file(path: Path | str, text: str):
+def write_file(path: Path, text: str | bytes, private: bool = False) -> None:
     try:
-        _write_file(path, text)
+        _write_file(path, text, private=private)
     except OSError as err:
         from esphome.core import EsphomeError
 
         raise EsphomeError(f"Could not write file at {path}") from err
 
 
-def write_file_if_changed(path: Path | str, text: str) -> bool:
+def write_file_if_changed(path: Path, text: str) -> bool:
     """Write text to the given path, but not if the contents match already.
 
     Returns true if the file was changed.
     """
-    if not isinstance(path, Path):
-        path = Path(path)
-
     src_content = None
     if path.is_file():
         src_content = read_file(path)
@@ -387,12 +423,10 @@ def write_file_if_changed(path: Path | str, text: str) -> bool:
     return True
 
 
-def copy_file_if_changed(src: os.PathLike, dst: os.PathLike) -> None:
-    import shutil
-
+def copy_file_if_changed(src: Path, dst: Path) -> None:
     if file_compare(src, dst):
         return
-    mkdir_p(os.path.dirname(dst))
+    dst.parent.mkdir(parents=True, exist_ok=True)
     try:
         shutil.copyfile(src, dst)
     except OSError as err:
@@ -417,12 +451,12 @@ def list_starts_with(list_, sub):
     return len(sub) <= len(list_) and all(list_[i] == x for i, x in enumerate(sub))
 
 
-def file_compare(path1: os.PathLike, path2: os.PathLike) -> bool:
+def file_compare(path1: Path, path2: Path) -> bool:
     """Return True if the files path1 and path2 have the same contents."""
     import stat
 
     try:
-        stat1, stat2 = os.stat(path1), os.stat(path2)
+        stat1, stat2 = path1.stat(), path2.stat()
     except OSError:
         # File doesn't exist or another error -> not equal
         return False
@@ -439,7 +473,7 @@ def file_compare(path1: os.PathLike, path2: os.PathLike) -> bool:
 
     bufsize = 8 * 1024
     # Read files in blocks until a mismatch is found
-    with open(path1, "rb") as fh1, open(path2, "rb") as fh2:
+    with path1.open("rb") as fh1, path2.open("rb") as fh2:
         while True:
             blob1, blob2 = fh1.read(bufsize), fh2.read(bufsize)
             if blob1 != blob2:
@@ -505,3 +539,20 @@ _DISALLOWED_CHARS = re.compile(r"[^a-zA-Z0-9-_]")
 def sanitize(value):
     """Same behaviour as `helpers.cpp` method `str_sanitize`."""
     return _DISALLOWED_CHARS.sub("_", value)
+
+
+def docs_url(path: str) -> str:
+    """Return the URL to the documentation for a given path."""
+    # Local import to avoid circular import
+    from esphome.config_validation import Version
+
+    version = Version.parse(ESPHOME_VERSION)
+    if version.is_beta:
+        docs_format = "https://beta.esphome.io/{path}"
+    elif version.is_dev:
+        docs_format = "https://next.esphome.io/{path}"
+    else:
+        docs_format = "https://esphome.io/{path}"
+
+    path = path.removeprefix("/")
+    return docs_format.format(path=path)
diff --git a/esphome/idf_component.yml b/esphome/idf_component.yml
index c43b622684..1a6dc8b97d 100644
--- a/esphome/idf_component.yml
+++ b/esphome/idf_component.yml
@@ -2,7 +2,7 @@ dependencies:
   espressif/esp-tflite-micro:
     version: 1.3.3~1
   espressif/esp32-camera:
-    version: 2.0.15
+    version: 2.1.1
   espressif/mdns:
     version: 1.8.2
   espressif/esp_wifi_remote:
@@ -19,3 +19,7 @@ dependencies:
       - if: "target in [esp32h2, esp32p4]"
   zorxx/multipart-parser:
     version: 1.0.1
+  espressif/lan867x:
+    version: "2.0.0"
+    rules:
+      - if: "target in [esp32, esp32p4]"
diff --git a/esphome/loader.py b/esphome/loader.py
index 7b2472521a..387443c032 100644
--- a/esphome/loader.py
+++ b/esphome/loader.py
@@ -82,11 +82,10 @@ class ComponentManifest:
         return getattr(self.module, "CONFLICTS_WITH", [])
 
     @property
-    def auto_load(self) -> list[str]:
-        al = getattr(self.module, "AUTO_LOAD", [])
-        if callable(al):
-            return al()
-        return al
+    def auto_load(
+        self,
+    ) -> list[str] | Callable[[], list[str]] | Callable[[ConfigType], list[str]]:
+        return getattr(self.module, "AUTO_LOAD", [])
 
     @property
     def codeowners(self) -> list[str]:
@@ -192,7 +191,7 @@ def install_custom_components_meta_finder():
     install_meta_finder(custom_components_dir)
 
 
-def _lookup_module(domain, exception):
+def _lookup_module(domain: str, exception: bool) -> ComponentManifest | None:
     if domain in _COMPONENT_CACHE:
         return _COMPONENT_CACHE[domain]
 
@@ -219,16 +218,16 @@ def _lookup_module(domain, exception):
     return manif
 
 
-def get_component(domain, exception=False):
+def get_component(domain: str, exception: bool = False) -> ComponentManifest | None:
     assert "." not in domain
     return _lookup_module(domain, exception)
 
 
-def get_platform(domain, platform):
+def get_platform(domain: str, platform: str) -> ComponentManifest | None:
     full = f"{platform}.{domain}"
     return _lookup_module(full, False)
 
 
-_COMPONENT_CACHE = {}
+_COMPONENT_CACHE: dict[str, ComponentManifest] = {}
 CORE_COMPONENTS_PATH = (Path(__file__).parent / "components").resolve()
 _COMPONENT_CACHE["esphome"] = ComponentManifest(esphome.core.config)
diff --git a/esphome/log.py b/esphome/log.py
index 0e91eb32c2..bfd1875b55 100644
--- a/esphome/log.py
+++ b/esphome/log.py
@@ -37,6 +37,8 @@ class AnsiStyle(Enum):
 
 
 def color(col: AnsiFore, msg: str, reset: bool = True) -> str:
+    if col == AnsiFore.KEEP:
+        return msg
     s = col.value + msg
     if reset and col:
         s += AnsiStyle.RESET_ALL.value
@@ -61,7 +63,7 @@ class ESPHomeLogFormatter(logging.Formatter):
         }.get(record.levelname, "")
         message = f"{prefix}{formatted}{AnsiStyle.RESET_ALL.value}"
         if CORE.dashboard:
-            try:
+            try:  # noqa: SIM105
                 message = message.replace("\033", "\\033")
             except UnicodeEncodeError:
                 pass
diff --git a/esphome/mqtt.py b/esphome/mqtt.py
index a420b5ba7f..f1c631697a 100644
--- a/esphome/mqtt.py
+++ b/esphome/mqtt.py
@@ -1,3 +1,4 @@
+import contextlib
 from datetime import datetime
 import hashlib
 import json
@@ -35,7 +36,7 @@ _LOGGER = logging.getLogger(__name__)
 
 
 def config_from_env():
-    config = {
+    return {
         CONF_MQTT: {
             CONF_USERNAME: get_str_env("ESPHOME_DASHBOARD_MQTT_USERNAME"),
             CONF_PASSWORD: get_str_env("ESPHOME_DASHBOARD_MQTT_PASSWORD"),
@@ -43,7 +44,6 @@ def config_from_env():
             CONF_PORT: get_int_env("ESPHOME_DASHBOARD_MQTT_PORT", 1883),
         },
     }
-    return config
 
 
 def initialize(
@@ -52,10 +52,8 @@ def initialize(
     client = prepare(
         config, subscriptions, on_message, on_connect, username, password, client_id
     )
-    try:
+    with contextlib.suppress(KeyboardInterrupt):
         client.loop_forever()
-    except KeyboardInterrupt:
-        pass
     return 0
 
 
diff --git a/esphome/pins.py b/esphome/pins.py
index 4f9b4859a1..601c05880a 100644
--- a/esphome/pins.py
+++ b/esphome/pins.py
@@ -118,11 +118,11 @@ class PinRegistry(dict):
                         parent_config = fconf.get_config_for_path(parent_path)
                         final_val_fun(pin_config, parent_config)
                     allow_others = pin_config.get(CONF_ALLOW_OTHER_USES, False)
-                    if count != 1 and not allow_others:
+                    if count != 1 and not allow_others and not CORE.testing_mode:
                         raise cv.Invalid(
                             f"Pin {pin_config[CONF_NUMBER]} is used in multiple places"
                         )
-                    if count == 1 and allow_others:
+                    if count == 1 and allow_others and not CORE.testing_mode:
                         raise cv.Invalid(
                             f"Pin {pin_config[CONF_NUMBER]} incorrectly sets {CONF_ALLOW_OTHER_USES}: true"
                         )
diff --git a/esphome/platformio_api.py b/esphome/platformio_api.py
index e34ac028f8..9418c1c7d3 100644
--- a/esphome/platformio_api.py
+++ b/esphome/platformio_api.py
@@ -5,6 +5,7 @@ import os
 from pathlib import Path
 import re
 import subprocess
+from typing import Any
 
 from esphome.const import CONF_COMPILE_PROCESS_LIMIT, CONF_ESPHOME, KEY_CORE
 from esphome.core import CORE, EsphomeError
@@ -18,28 +19,59 @@ def patch_structhash():
     # removed/added. This might have unintended consequences, but this improves compile
     # times greatly when adding/removing components and a simple clean build solves
     # all issues
-    from os import makedirs
-    from os.path import getmtime, isdir, join
-
     from platformio.run import cli, helpers
 
     def patched_clean_build_dir(build_dir, *args):
         from platformio import fs
         from platformio.project.helpers import get_project_dir
 
-        platformio_ini = join(get_project_dir(), "platformio.ini")
+        platformio_ini = Path(get_project_dir()) / "platformio.ini"
+
+        build_dir = Path(build_dir)
 
         # if project's config is modified
-        if isdir(build_dir) and getmtime(platformio_ini) > getmtime(build_dir):
+        if (
+            build_dir.is_dir()
+            and platformio_ini.stat().st_mtime > build_dir.stat().st_mtime
+        ):
             fs.rmtree(build_dir)
 
-        if not isdir(build_dir):
-            makedirs(build_dir)
+        if not build_dir.is_dir():
+            build_dir.mkdir(parents=True)
 
     helpers.clean_build_dir = patched_clean_build_dir
     cli.clean_build_dir = patched_clean_build_dir
 
 
+def patch_file_downloader():
+    """Patch PlatformIO's FileDownloader to retry on PackageException errors."""
+    from platformio.package.download import FileDownloader
+    from platformio.package.exception import PackageException
+
+    original_init = FileDownloader.__init__
+
+    def patched_init(self, *args: Any, **kwargs: Any) -> None:
+        max_retries = 3
+
+        for attempt in range(max_retries):
+            try:
+                return original_init(self, *args, **kwargs)
+            except PackageException as e:
+                if attempt < max_retries - 1:
+                    _LOGGER.warning(
+                        "Package download failed: %s. Retrying... (attempt %d/%d)",
+                        str(e),
+                        attempt + 1,
+                        max_retries,
+                    )
+                else:
+                    # Final attempt - re-raise
+                    raise
+        return None
+
+    FileDownloader.__init__ = patched_init
+
+
 IGNORE_LIB_WARNINGS = f"(?:{'|'.join(['Hash', 'Update'])})"
 FILTER_PLATFORMIO_LINES = [
     r"Verbose mode can be enabled via `-v, --verbose` option.*",
@@ -61,6 +93,7 @@ FILTER_PLATFORMIO_LINES = [
     r"Advanced Memory Usage is available via .*",
     r"Merged .* ELF section",
     r"esptool.py v.*",
+    r"esptool v.*",
     r"Checking size .*",
     r"Retrieving maximum program size .*",
     r"PLATFORM: .*",
@@ -69,14 +102,19 @@ FILTER_PLATFORMIO_LINES = [
     r" - tool-esptool.* \(.*\)",
     r" - toolchain-.* \(.*\)",
     r"Creating BIN file .*",
+    r"Warning! Could not find file \".*.crt\"",
+    r"Warning! Arduino framework as an ESP-IDF component doesn't handle the `variant` field! The default `esp32` variant will be used.",
+    r"Warning: DEPRECATED: 'esptool.py' is deprecated. Please use 'esptool' instead. The '.py' suffix will be removed in a future major release.",
+    r"Warning: esp-idf-size exited with code 2",
+    r"esp_idf_size: error: unrecognized arguments: --ng",
 ]
 
 
 def run_platformio_cli(*args, **kwargs) -> str | int:
     os.environ["PLATFORMIO_FORCE_COLOR"] = "true"
-    os.environ["PLATFORMIO_BUILD_DIR"] = os.path.abspath(CORE.relative_pioenvs_path())
+    os.environ["PLATFORMIO_BUILD_DIR"] = str(CORE.relative_pioenvs_path().absolute())
     os.environ.setdefault(
-        "PLATFORMIO_LIBDEPS_DIR", os.path.abspath(CORE.relative_piolibdeps_path())
+        "PLATFORMIO_LIBDEPS_DIR", str(CORE.relative_piolibdeps_path().absolute())
     )
     # Suppress Python syntax warnings from third-party scripts during compilation
     os.environ.setdefault("PYTHONWARNINGS", "ignore::SyntaxWarning")
@@ -91,11 +129,12 @@ def run_platformio_cli(*args, **kwargs) -> str | int:
     import platformio.__main__
 
     patch_structhash()
+    patch_file_downloader()
     return run_external_command(platformio.__main__.main, *cmd, **kwargs)
 
 
 def run_platformio_cli_run(config, verbose, *args, **kwargs) -> str | int:
-    command = ["run", "-d", CORE.build_path]
+    command = ["run", "-d", str(CORE.build_path)]
     if verbose:
         command += ["-v"]
     command += list(args)
@@ -127,13 +166,15 @@ def _run_idedata(config):
 
 
 def _load_idedata(config):
-    platformio_ini = Path(CORE.relative_build_path("platformio.ini"))
-    temp_idedata = Path(CORE.relative_internal_path("idedata", f"{CORE.name}.json"))
+    platformio_ini = CORE.relative_build_path("platformio.ini")
+    temp_idedata = CORE.relative_internal_path("idedata", f"{CORE.name}.json")
 
     changed = False
-    if not platformio_ini.is_file() or not temp_idedata.is_file():
-        changed = True
-    elif platformio_ini.stat().st_mtime >= temp_idedata.stat().st_mtime:
+    if (
+        not platformio_ini.is_file()
+        or not temp_idedata.is_file()
+        or platformio_ini.stat().st_mtime >= temp_idedata.stat().st_mtime
+    ):
         changed = True
 
     if not changed:
@@ -208,7 +249,7 @@ def _decode_pc(config, addr):
         return
     command = [idedata.addr2line_path, "-pfiaC", "-e", idedata.firmware_elf_path, addr]
     try:
-        translation = subprocess.check_output(command).decode().strip()
+        translation = subprocess.check_output(command, close_fds=False).decode().strip()
     except Exception:  # pylint: disable=broad-except
         _LOGGER.debug("Caught exception for command %s", command, exc_info=1)
         return
@@ -296,7 +337,7 @@ def process_stacktrace(config, line, backtrace_state):
 
 @dataclass
 class FlashImage:
-    path: str
+    path: Path
     offset: str
 
 
@@ -305,17 +346,17 @@ class IDEData:
         self.raw = raw
 
     @property
-    def firmware_elf_path(self):
-        return self.raw["prog_path"]
+    def firmware_elf_path(self) -> Path:
+        return Path(self.raw["prog_path"])
 
     @property
-    def firmware_bin_path(self) -> str:
-        return str(Path(self.firmware_elf_path).with_suffix(".bin"))
+    def firmware_bin_path(self) -> Path:
+        return self.firmware_elf_path.with_suffix(".bin")
 
     @property
     def extra_flash_images(self) -> list[FlashImage]:
         return [
-            FlashImage(path=entry["path"], offset=entry["offset"])
+            FlashImage(path=Path(entry["path"]), offset=entry["offset"])
             for entry in self.raw["extra"]["flash_images"]
         ]
 
diff --git a/esphome/resolver.py b/esphome/resolver.py
new file mode 100644
index 0000000000..99482aa20e
--- /dev/null
+++ b/esphome/resolver.py
@@ -0,0 +1,67 @@
+"""DNS resolver for ESPHome using aioesphomeapi."""
+
+from __future__ import annotations
+
+import asyncio
+import threading
+
+from aioesphomeapi.core import ResolveAPIError, ResolveTimeoutAPIError
+import aioesphomeapi.host_resolver as hr
+
+from esphome.core import EsphomeError
+
+RESOLVE_TIMEOUT = 10.0  # seconds
+
+
+class AsyncResolver(threading.Thread):
+    """Resolver using aioesphomeapi that runs in a thread for faster results.
+
+    This resolver uses aioesphomeapi's async_resolve_host to handle DNS resolution,
+    including proper .local domain fallback. Running in a thread allows us to get
+    the result immediately without waiting for asyncio.run() to complete its
+    cleanup cycle, which can take significant time.
+    """
+
+    def __init__(self, hosts: list[str], port: int) -> None:
+        """Initialize the resolver."""
+        super().__init__(daemon=True)
+        self.hosts = hosts
+        self.port = port
+        self.result: list[hr.AddrInfo] | None = None
+        self.exception: Exception | None = None
+        self.event = threading.Event()
+
+    async def _resolve(self) -> None:
+        """Resolve hostnames to IP addresses."""
+        try:
+            self.result = await hr.async_resolve_host(
+                self.hosts, self.port, timeout=RESOLVE_TIMEOUT
+            )
+        except Exception as e:  # pylint: disable=broad-except
+            # We need to catch all exceptions to ensure the event is set
+            # Otherwise the thread could hang forever
+            self.exception = e
+        finally:
+            self.event.set()
+
+    def run(self) -> None:
+        """Run the DNS resolution."""
+        asyncio.run(self._resolve())
+
+    def resolve(self) -> list[hr.AddrInfo]:
+        """Start the thread and wait for the result."""
+        self.start()
+
+        if not self.event.wait(
+            timeout=RESOLVE_TIMEOUT + 1.0
+        ):  # Give it 1 second more than the resolver timeout
+            raise EsphomeError("Timeout resolving IP address")
+
+        if exc := self.exception:
+            if isinstance(exc, ResolveTimeoutAPIError):
+                raise EsphomeError(f"Timeout resolving IP address: {exc}") from exc
+            if isinstance(exc, ResolveAPIError):
+                raise EsphomeError(f"Error resolving IP address: {exc}") from exc
+            raise exc
+
+        return self.result
diff --git a/esphome/storage_json.py b/esphome/storage_json.py
index b69dc2dd3f..d5423ab1c7 100644
--- a/esphome/storage_json.py
+++ b/esphome/storage_json.py
@@ -1,11 +1,11 @@
 from __future__ import annotations
 
 import binascii
-import codecs
 from datetime import datetime
 import json
 import logging
 import os
+from pathlib import Path
 
 from esphome import const
 from esphome.const import CONF_DISABLED, CONF_MDNS
@@ -16,30 +16,35 @@ from esphome.types import CoreType
 _LOGGER = logging.getLogger(__name__)
 
 
-def storage_path() -> str:
-    return os.path.join(CORE.data_dir, "storage", f"{CORE.config_filename}.json")
+def storage_path() -> Path:
+    return CORE.data_dir / "storage" / f"{CORE.config_filename}.json"
 
 
-def ext_storage_path(config_filename: str) -> str:
-    return os.path.join(CORE.data_dir, "storage", f"{config_filename}.json")
+def ext_storage_path(config_filename: str) -> Path:
+    return CORE.data_dir / "storage" / f"{config_filename}.json"
 
 
-def esphome_storage_path() -> str:
-    return os.path.join(CORE.data_dir, "esphome.json")
+def esphome_storage_path() -> Path:
+    return CORE.data_dir / "esphome.json"
 
 
-def ignored_devices_storage_path() -> str:
-    return os.path.join(CORE.data_dir, "ignored-devices.json")
+def ignored_devices_storage_path() -> Path:
+    return CORE.data_dir / "ignored-devices.json"
 
 
-def trash_storage_path() -> str:
+def trash_storage_path() -> Path:
     return CORE.relative_config_path("trash")
 
 
-def archive_storage_path() -> str:
+def archive_storage_path() -> Path:
     return CORE.relative_config_path("archive")
 
 
+def _to_path_if_not_none(value: str | None) -> Path | None:
+    """Convert a string to Path if it's not None."""
+    return Path(value) if value is not None else None
+
+
 class StorageJSON:
     def __init__(
         self,
@@ -52,8 +57,8 @@ class StorageJSON:
         address: str,
         web_port: int | None,
         target_platform: str,
-        build_path: str | None,
-        firmware_bin_path: str | None,
+        build_path: Path | None,
+        firmware_bin_path: Path | None,
         loaded_integrations: set[str],
         loaded_platforms: set[str],
         no_mdns: bool,
@@ -107,8 +112,8 @@ class StorageJSON:
             "address": self.address,
             "web_port": self.web_port,
             "esp_platform": self.target_platform,
-            "build_path": self.build_path,
-            "firmware_bin_path": self.firmware_bin_path,
+            "build_path": str(self.build_path),
+            "firmware_bin_path": str(self.firmware_bin_path),
             "loaded_integrations": sorted(self.loaded_integrations),
             "loaded_platforms": sorted(self.loaded_platforms),
             "no_mdns": self.no_mdns,
@@ -176,8 +181,8 @@ class StorageJSON:
         )
 
     @staticmethod
-    def _load_impl(path: str) -> StorageJSON | None:
-        with codecs.open(path, "r", encoding="utf-8") as f_handle:
+    def _load_impl(path: Path) -> StorageJSON | None:
+        with path.open("r", encoding="utf-8") as f_handle:
             storage = json.load(f_handle)
         storage_version = storage["storage_version"]
         name = storage.get("name")
@@ -190,8 +195,8 @@ class StorageJSON:
         address = storage.get("address")
         web_port = storage.get("web_port")
         esp_platform = storage.get("esp_platform")
-        build_path = storage.get("build_path")
-        firmware_bin_path = storage.get("firmware_bin_path")
+        build_path = _to_path_if_not_none(storage.get("build_path"))
+        firmware_bin_path = _to_path_if_not_none(storage.get("firmware_bin_path"))
         loaded_integrations = set(storage.get("loaded_integrations", []))
         loaded_platforms = set(storage.get("loaded_platforms", []))
         no_mdns = storage.get("no_mdns", False)
@@ -217,7 +222,7 @@ class StorageJSON:
         )
 
     @staticmethod
-    def load(path: str) -> StorageJSON | None:
+    def load(path: Path) -> StorageJSON | None:
         try:
             return StorageJSON._load_impl(path)
         except Exception:  # pylint: disable=broad-except
@@ -268,7 +273,7 @@ class EsphomeStorageJSON:
 
     @staticmethod
     def _load_impl(path: str) -> EsphomeStorageJSON | None:
-        with codecs.open(path, "r", encoding="utf-8") as f_handle:
+        with Path(path).open("r", encoding="utf-8") as f_handle:
             storage = json.load(f_handle)
         storage_version = storage["storage_version"]
         cookie_secret = storage.get("cookie_secret")
diff --git a/esphome/types.py b/esphome/types.py
index f68f503993..c474d0d076 100644
--- a/esphome/types.py
+++ b/esphome/types.py
@@ -1,6 +1,10 @@
 """This helper module tracks commonly used types in the esphome python codebase."""
 
-from esphome.core import ID, EsphomeCore, Lambda
+import abc
+from collections.abc import Sequence
+from typing import Any, TypedDict
+
+from esphome.core import ID, EsphomeCore, Lambda, TimePeriod
 
 ConfigFragmentType = (
     str
@@ -16,3 +20,39 @@ ConfigFragmentType = (
 ConfigType = dict[str, ConfigFragmentType]
 CoreType = EsphomeCore
 ConfigPathType = str | int
+
+
+class Expression(abc.ABC):
+    __slots__ = ()
+
+    @abc.abstractmethod
+    def __str__(self):
+        """
+        Convert expression into C++ code
+        """
+
+
+SafeExpType = (
+    Expression
+    | bool
+    | str
+    | int
+    | float
+    | TimePeriod
+    | type[bool]
+    | type[int]
+    | type[float]
+    | Sequence[Any]
+)
+
+TemplateArgsType = list[tuple[SafeExpType, str]]
+
+
+class EntityMetadata(TypedDict):
+    """Metadata stored for each entity to help with duplicate detection."""
+
+    name: str
+    device_id: str
+    platform: str
+    entity_id: str
+    component: str
diff --git a/esphome/util.py b/esphome/util.py
index ba26b8adc1..d41800dc20 100644
--- a/esphome/util.py
+++ b/esphome/util.py
@@ -1,19 +1,30 @@
 import collections
+from collections.abc import Callable
 import io
 import logging
-import os
 from pathlib import Path
 import re
 import subprocess
 import sys
+from typing import TYPE_CHECKING, Any
 
 from esphome import const
 
 _LOGGER = logging.getLogger(__name__)
 
+if TYPE_CHECKING:
+    from esphome.config_validation import Schema
+    from esphome.cpp_generator import MockObjClass
+
 
 class RegistryEntry:
-    def __init__(self, name, fun, type_id, schema):
+    def __init__(
+        self,
+        name: str,
+        fun: Callable[..., Any],
+        type_id: "MockObjClass",
+        schema: "Schema",
+    ):
         self.name = name
         self.fun = fun
         self.type_id = type_id
@@ -38,8 +49,8 @@ class Registry(dict[str, RegistryEntry]):
         self.base_schema = base_schema or {}
         self.type_id_key = type_id_key
 
-    def register(self, name, type_id, schema):
-        def decorator(fun):
+    def register(self, name: str, type_id: "MockObjClass", schema: "Schema"):
+        def decorator(fun: Callable[..., Any]):
             self[name] = RegistryEntry(name, fun, type_id, schema)
             return fun
 
@@ -47,8 +58,8 @@ class Registry(dict[str, RegistryEntry]):
 
 
 class SimpleRegistry(dict):
-    def register(self, name, data):
-        def decorator(fun):
+    def register(self, name: str, data: Any):
+        def decorator(fun: Callable[..., Any]):
             self[name] = (fun, data)
             return fun
 
@@ -59,7 +70,7 @@ def safe_print(message="", end="\n"):
     from esphome.core import CORE
 
     if CORE.dashboard:
-        try:
+        try:  # noqa: SIM105
             message = message.replace("\033", "\\033")
         except UnicodeEncodeError:
             pass
@@ -85,7 +96,10 @@ def safe_input(prompt=""):
     return input()
 
 
-def shlex_quote(s):
+def shlex_quote(s: str | Path) -> str:
+    # Convert Path objects to strings
+    if isinstance(s, Path):
+        s = str(s)
     if not s:
         return "''"
     if re.search(r"[^\w@%+=:,./-]", s) is None:
@@ -110,7 +124,7 @@ class RedirectText:
     def __getattr__(self, item):
         return getattr(self._out, item)
 
-    def _write_color_replace(self, s):
+    def _write_color_replace(self, s: str | bytes) -> None:
         from esphome.core import CORE
 
         if CORE.dashboard:
@@ -121,7 +135,7 @@ class RedirectText:
             s = s.replace("\033", "\\033")
         self._out.write(s)
 
-    def write(self, s):
+    def write(self, s: str | bytes) -> int:
         # s is usually a str already (self._out is of type TextIOWrapper)
         # However, s is sometimes also a bytes object in python3. Let's make sure it's a
         # str
@@ -147,6 +161,13 @@ class RedirectText:
                     continue
 
                 self._write_color_replace(line)
+                # Check for flash size error and provide helpful guidance
+                if (
+                    "Error: The program size" in line
+                    and "is greater than maximum allowed" in line
+                    and (help_msg := get_esp32_arduino_flash_error_help())
+                ):
+                    self._write_color_replace(help_msg)
         else:
             self._write_color_replace(s)
 
@@ -216,7 +237,7 @@ def run_external_command(
     return retval
 
 
-def run_external_process(*cmd, **kwargs):
+def run_external_process(*cmd: str, **kwargs: Any) -> int | str:
     full_cmd = " ".join(shlex_quote(x) for x in cmd)
     _LOGGER.debug("Running:  %s", full_cmd)
     filter_lines = kwargs.get("filter_lines")
@@ -231,7 +252,12 @@ def run_external_process(*cmd, **kwargs):
 
     try:
         proc = subprocess.run(
-            cmd, stdout=sub_stdout, stderr=sub_stderr, encoding="utf-8", check=False
+            cmd,
+            stdout=sub_stdout,
+            stderr=sub_stderr,
+            encoding="utf-8",
+            check=False,
+            close_fds=False,
         )
         return proc.stdout if capture_stdout else proc.returncode
     except KeyboardInterrupt:  # pylint: disable=try-except-raise
@@ -259,22 +285,28 @@ class OrderedDict(collections.OrderedDict):
         return dict(self).__repr__()
 
 
-def list_yaml_files(folders):
-    files = filter_yaml_files(
-        [os.path.join(folder, p) for folder in folders for p in os.listdir(folder)]
-    )
-    files.sort()
-    return files
+def list_yaml_files(configs: list[str | Path]) -> list[Path]:
+    files: list[Path] = []
+    for config in configs:
+        config = Path(config)
+        if not config.exists():
+            raise FileNotFoundError(f"Config path '{config}' does not exist!")
+        if config.is_file():
+            files.append(config)
+        else:
+            files.extend(config.glob("*"))
+    files = filter_yaml_files(files)
+    return sorted(files)
 
 
-def filter_yaml_files(files):
+def filter_yaml_files(files: list[Path]) -> list[Path]:
     return [
         f
         for f in files
         if (
-            os.path.splitext(f)[1] in (".yaml", ".yml")
-            and os.path.basename(f) not in ("secrets.yaml", "secrets.yml")
-            and not os.path.basename(f).startswith(".")
+            f.suffix in (".yaml", ".yml")
+            and f.name not in ("secrets.yaml", "secrets.yml")
+            and not f.name.startswith(".")
         )
     ]
 
@@ -309,3 +341,40 @@ def get_serial_ports() -> list[SerialPort]:
 
     result.sort(key=lambda x: x.path)
     return result
+
+
+def get_esp32_arduino_flash_error_help() -> str | None:
+    """Returns helpful message when ESP32 with Arduino runs out of flash space."""
+    from esphome.core import CORE
+
+    if not (CORE.is_esp32 and CORE.using_arduino):
+        return None
+
+    from esphome.log import AnsiFore, color
+
+    return (
+        "\n"
+        + color(
+            AnsiFore.YELLOW,
+            "💡 TIP: Your ESP32 with Arduino framework has run out of flash space.\n",
+        )
+        + "\n"
+        + "To fix this, switch to the ESP-IDF framework which is more memory efficient:\n"
+        + "\n"
+        + "1. In your YAML configuration, modify the framework section:\n"
+        + "\n"
+        + "   esp32:\n"
+        + "     framework:\n"
+        + "       type: esp-idf\n"
+        + "\n"
+        + "2. Clean build files and compile again\n"
+        + "\n"
+        + "Note: ESP-IDF uses less flash space and provides better performance.\n"
+        + "Some Arduino-specific libraries may need alternatives.\n"
+        + "\n"
+        + "For detailed migration instructions, see:\n"
+        + color(
+            AnsiFore.BLUE,
+            "https://esphome.io/guides/esp32_arduino_to_idf.html\n\n",
+        )
+    )
diff --git a/esphome/voluptuous_schema.py b/esphome/voluptuous_schema.py
index 8fb966e3b2..7220fb307f 100644
--- a/esphome/voluptuous_schema.py
+++ b/esphome/voluptuous_schema.py
@@ -225,9 +225,10 @@ class _Schema(vol.Schema):
             return ret
 
         schema = schemas[0]
+        extra_schemas = self._extra_schemas.copy()
+        if isinstance(schema, _Schema):
+            extra_schemas.extend(schema._extra_schemas)
         if isinstance(schema, vol.Schema):
             schema = schema.schema
         ret = super().extend(schema, extra=extra)
-        return _Schema(
-            ret.schema, extra=ret.extra, extra_schemas=self._extra_schemas.copy()
-        )
+        return _Schema(ret.schema, extra=ret.extra, extra_schemas=extra_schemas)
diff --git a/esphome/vscode.py b/esphome/vscode.py
index d8cfe91938..53bb339a8e 100644
--- a/esphome/vscode.py
+++ b/esphome/vscode.py
@@ -2,7 +2,7 @@ from __future__ import annotations
 
 from io import StringIO
 import json
-import os
+from pathlib import Path
 from typing import Any
 
 from esphome.config import Config, _format_vol_invalid, validate_config
@@ -67,25 +67,24 @@ def _read_file_content_from_json_on_stdin() -> str:
     return data["content"]
 
 
-def _print_file_read_event(path: str) -> None:
+def _print_file_read_event(path: Path) -> None:
     """Print a file read event."""
     print(
         json.dumps(
             {
                 "type": "read_file",
-                "path": path,
+                "path": str(path),
             }
         )
     )
 
 
-def _request_and_get_stream_on_stdin(fname: str) -> StringIO:
+def _request_and_get_stream_on_stdin(fname: Path) -> StringIO:
     _print_file_read_event(fname)
-    raw_yaml_stream = StringIO(_read_file_content_from_json_on_stdin())
-    return raw_yaml_stream
+    return StringIO(_read_file_content_from_json_on_stdin())
 
 
-def _vscode_loader(fname: str) -> dict[str, Any]:
+def _vscode_loader(fname: Path) -> dict[str, Any]:
     raw_yaml_stream = _request_and_get_stream_on_stdin(fname)
     # it is required to set the name on StringIO so document on start_mark
     # is set properly. Otherwise it is initialized with ""
@@ -93,7 +92,7 @@ def _vscode_loader(fname: str) -> dict[str, Any]:
     return parse_yaml(fname, raw_yaml_stream, _vscode_loader)
 
 
-def _ace_loader(fname: str) -> dict[str, Any]:
+def _ace_loader(fname: Path) -> dict[str, Any]:
     raw_yaml_stream = _request_and_get_stream_on_stdin(fname)
     return parse_yaml(fname, raw_yaml_stream)
 
@@ -121,10 +120,10 @@ def read_config(args):
             return
         CORE.vscode = True
         if args.ace:  # Running from ESPHome Compiler dashboard, not vscode
-            CORE.config_path = os.path.join(args.configuration, data["file"])
+            CORE.config_path = Path(args.configuration) / data["file"]
             loader = _ace_loader
         else:
-            CORE.config_path = data["file"]
+            CORE.config_path = Path(data["file"])
             loader = _vscode_loader
 
         file_name = CORE.config_path
diff --git a/esphome/wizard.py b/esphome/wizard.py
index 6f7cbd1ff4..97343eea99 100644
--- a/esphome/wizard.py
+++ b/esphome/wizard.py
@@ -1,6 +1,7 @@
-import os
+from pathlib import Path
 import random
 import string
+from typing import Literal, NotRequired, TypedDict, Unpack
 import unicodedata
 
 import voluptuous as vol
@@ -103,11 +104,25 @@ HARDWARE_BASE_CONFIGS = {
 }
 
 
-def sanitize_double_quotes(value):
+def sanitize_double_quotes(value: str) -> str:
     return value.replace("\\", "\\\\").replace('"', '\\"')
 
 
-def wizard_file(**kwargs):
+class WizardFileKwargs(TypedDict):
+    """Keyword arguments for wizard_file function."""
+
+    name: str
+    platform: Literal["ESP8266", "ESP32", "RP2040", "BK72XX", "LN882X", "RTL87XX"]
+    board: str
+    ssid: NotRequired[str]
+    psk: NotRequired[str]
+    password: NotRequired[str]
+    ota_password: NotRequired[str]
+    api_encryption_key: NotRequired[str]
+    friendly_name: NotRequired[str]
+
+
+def wizard_file(**kwargs: Unpack[WizardFileKwargs]) -> str:
     letters = string.ascii_letters + string.digits
     ap_name_base = kwargs["name"].replace("_", " ").title()
     ap_name = f"{ap_name_base} Fallback Hotspot"
@@ -116,10 +131,7 @@ def wizard_file(**kwargs):
     kwargs["fallback_name"] = ap_name
     kwargs["fallback_psk"] = "".join(random.choice(letters) for _ in range(12))
 
-    if kwargs.get("friendly_name"):
-        base = BASE_CONFIG_FRIENDLY
-    else:
-        base = BASE_CONFIG
+    base = BASE_CONFIG_FRIENDLY if kwargs.get("friendly_name") else BASE_CONFIG
 
     config = base.format(**kwargs)
 
@@ -183,7 +195,25 @@ captive_portal:
     return config
 
 
-def wizard_write(path, **kwargs):
+class WizardWriteKwargs(TypedDict):
+    """Keyword arguments for wizard_write function."""
+
+    name: str
+    type: Literal["basic", "empty", "upload"]
+    # Required for "basic" type
+    board: NotRequired[str]
+    platform: NotRequired[str]
+    ssid: NotRequired[str]
+    psk: NotRequired[str]
+    password: NotRequired[str]
+    ota_password: NotRequired[str]
+    api_encryption_key: NotRequired[str]
+    friendly_name: NotRequired[str]
+    # Required for "upload" type
+    file_text: NotRequired[str]
+
+
+def wizard_write(path: Path, **kwargs: Unpack[WizardWriteKwargs]) -> bool:
     from esphome.components.bk72xx import boards as bk72xx_boards
     from esphome.components.esp32 import boards as esp32_boards
     from esphome.components.esp8266 import boards as esp8266_boards
@@ -192,34 +222,47 @@ def wizard_write(path, **kwargs):
     from esphome.components.rtl87xx import boards as rtl87xx_boards
 
     name = kwargs["name"]
-    board = kwargs["board"]
+    if kwargs["type"] == "empty":
+        file_text = ""
+        # Will be updated later after editing the file
+        hardware = "UNKNOWN"
+    elif kwargs["type"] == "upload":
+        file_text = kwargs["file_text"]
+        hardware = "UNKNOWN"
+    else:  # "basic"
+        board = kwargs["board"]
 
-    for key in ("ssid", "psk", "password", "ota_password"):
-        if key in kwargs:
-            kwargs[key] = sanitize_double_quotes(kwargs[key])
+        for key in ("ssid", "psk", "password", "ota_password"):
+            if key in kwargs:
+                kwargs[key] = sanitize_double_quotes(kwargs[key])
+        if "platform" not in kwargs:
+            if board in esp8266_boards.BOARDS:
+                platform = "ESP8266"
+            elif board in esp32_boards.BOARDS:
+                platform = "ESP32"
+            elif board in rp2040_boards.BOARDS:
+                platform = "RP2040"
+            elif board in bk72xx_boards.BOARDS:
+                platform = "BK72XX"
+            elif board in ln882x_boards.BOARDS:
+                platform = "LN882X"
+            elif board in rtl87xx_boards.BOARDS:
+                platform = "RTL87XX"
+            else:
+                safe_print(color(AnsiFore.RED, f'The board "{board}" is unknown.'))
+                return False
+            kwargs["platform"] = platform
+        hardware = kwargs["platform"]
+        file_text = wizard_file(**kwargs)
 
-    if "platform" not in kwargs:
-        if board in esp8266_boards.BOARDS:
-            platform = "ESP8266"
-        elif board in esp32_boards.BOARDS:
-            platform = "ESP32"
-        elif board in rp2040_boards.BOARDS:
-            platform = "RP2040"
-        elif board in bk72xx_boards.BOARDS:
-            platform = "BK72XX"
-        elif board in ln882x_boards.BOARDS:
-            platform = "LN882X"
-        elif board in rtl87xx_boards.BOARDS:
-            platform = "RTL87XX"
-        else:
-            safe_print(color(AnsiFore.RED, f'The board "{board}" is unknown.'))
-            return False
-        kwargs["platform"] = platform
-    hardware = kwargs["platform"]
+    # Check if file already exists to prevent overwriting
+    if path.exists() and path.is_file():
+        safe_print(color(AnsiFore.RED, f'The file "{path}" already exists.'))
+        return False
 
-    write_file(path, wizard_file(**kwargs))
+    write_file(path, file_text)
     storage = StorageJSON.from_wizard(name, name, f"{name}.local", hardware)
-    storage_path = ext_storage_path(os.path.basename(path))
+    storage_path = ext_storage_path(path.name)
     storage.save(storage_path)
 
     return True
@@ -227,14 +270,14 @@ def wizard_write(path, **kwargs):
 
 if get_bool_env(ENV_QUICKWIZARD):
 
-    def sleep(time):
+    def sleep(time: float) -> None:
         pass
 
 else:
     from time import sleep
 
 
-def safe_print_step(step, big):
+def safe_print_step(step: int, big: str) -> None:
     safe_print()
     safe_print()
     safe_print(f"============= STEP {step} =============")
@@ -243,14 +286,14 @@ def safe_print_step(step, big):
     sleep(0.25)
 
 
-def default_input(text, default):
+def default_input(text: str, default: str) -> str:
     safe_print()
     safe_print(f"Press ENTER for default ({default})")
     return safe_input(text.format(default)) or default
 
 
 # From https://stackoverflow.com/a/518232/8924614
-def strip_accents(value):
+def strip_accents(value: str) -> str:
     return "".join(
         c
         for c in unicodedata.normalize("NFD", str(value))
@@ -258,7 +301,7 @@ def strip_accents(value):
     )
 
 
-def wizard(path):
+def wizard(path: Path) -> int:
     from esphome.components.bk72xx import boards as bk72xx_boards
     from esphome.components.esp32 import boards as esp32_boards
     from esphome.components.esp8266 import boards as esp8266_boards
@@ -266,14 +309,14 @@ def wizard(path):
     from esphome.components.rp2040 import boards as rp2040_boards
     from esphome.components.rtl87xx import boards as rtl87xx_boards
 
-    if not path.endswith(".yaml") and not path.endswith(".yml"):
+    if path.suffix not in (".yaml", ".yml"):
         safe_print(
-            f"Please make your configuration file {color(AnsiFore.CYAN, path)} have the extension .yaml or .yml"
+            f"Please make your configuration file {color(AnsiFore.CYAN, str(path))} have the extension .yaml or .yml"
         )
         return 1
-    if os.path.exists(path):
+    if path.exists():
         safe_print(
-            f"Uh oh, it seems like {color(AnsiFore.CYAN, path)} already exists, please delete that file first or chose another configuration file."
+            f"Uh oh, it seems like {color(AnsiFore.CYAN, str(path))} already exists, please delete that file first or chose another configuration file."
         )
         return 2
 
@@ -499,13 +542,14 @@ def wizard(path):
         ssid=ssid,
         psk=psk,
         password=password,
+        type="basic",
     ):
         return 1
 
     safe_print()
     safe_print(
         color(AnsiFore.CYAN, "DONE! I've now written a new configuration file to ")
-        + color(AnsiFore.BOLD_CYAN, path)
+        + color(AnsiFore.BOLD_CYAN, str(path))
     )
     safe_print()
     safe_print("Next steps:")
diff --git a/esphome/writer.py b/esphome/writer.py
index 5438e48570..8eee445cf1 100644
--- a/esphome/writer.py
+++ b/esphome/writer.py
@@ -7,7 +7,6 @@ import re
 from esphome import loader
 from esphome.config import iter_component_configs, iter_components
 from esphome.const import (
-    ENV_NOGITIGNORE,
     HEADER_FILE_EXTENSIONS,
     PLATFORM_ESP32,
     SOURCE_FILE_EXTENSIONS,
@@ -16,8 +15,8 @@ from esphome.const import (
 from esphome.core import CORE, EsphomeError
 from esphome.helpers import (
     copy_file_if_changed,
-    get_bool_env,
-    mkdir_p,
+    get_str_env,
+    is_ha_addon,
     read_file,
     walk_files,
     write_file_if_changed,
@@ -30,8 +29,6 @@ CPP_AUTO_GENERATE_BEGIN = "// ========== AUTO GENERATED CODE BEGIN ==========="
 CPP_AUTO_GENERATE_END = "// =========== AUTO GENERATED CODE END ============"
 CPP_INCLUDE_BEGIN = "// ========== AUTO GENERATED INCLUDE BLOCK BEGIN ==========="
 CPP_INCLUDE_END = "// ========== AUTO GENERATED INCLUDE BLOCK END ==========="
-INI_AUTO_GENERATE_BEGIN = "; ========== AUTO GENERATED CODE BEGIN ==========="
-INI_AUTO_GENERATE_END = "; =========== AUTO GENERATED CODE END ============"
 
 CPP_BASE_FORMAT = (
     """// Auto generated code by esphome
@@ -50,20 +47,6 @@ void loop() {
 """,
 )
 
-INI_BASE_FORMAT = (
-    """; Auto generated code by esphome
-
-[common]
-lib_deps =
-build_flags =
-upload_flags =
-
-""",
-    """
-
-""",
-)
-
 UPLOAD_SPEED_OVERRIDE = {
     "esp210": 57600,
 }
@@ -95,11 +78,11 @@ def get_include_text():
 
 
 def replace_file_content(text, pattern, repl):
-    content_new, count = re.subn(pattern, repl, text, flags=re.M)
+    content_new, count = re.subn(pattern, repl, text, flags=re.MULTILINE)
     return content_new, count
 
 
-def storage_should_clean(old: StorageJSON, new: StorageJSON) -> bool:
+def storage_should_clean(old: StorageJSON | None, new: StorageJSON) -> bool:
     if old is None:
         return True
 
@@ -107,23 +90,22 @@ def storage_should_clean(old: StorageJSON, new: StorageJSON) -> bool:
         return True
     if old.build_path != new.build_path:
         return True
-
-    return False
+    # Check if any components have been removed
+    return bool(old.loaded_integrations - new.loaded_integrations)
 
 
 def storage_should_update_cmake_cache(old: StorageJSON, new: StorageJSON) -> bool:
     if (
         old.loaded_integrations != new.loaded_integrations
         or old.loaded_platforms != new.loaded_platforms
-    ):
-        if new.core_platform == PLATFORM_ESP32:
-            from esphome.components.esp32 import FRAMEWORK_ESP_IDF
+    ) and new.core_platform == PLATFORM_ESP32:
+        from esphome.components.esp32 import FRAMEWORK_ESP_IDF
 
-            return new.framework == FRAMEWORK_ESP_IDF
+        return new.framework == FRAMEWORK_ESP_IDF
     return False
 
 
-def update_storage_json():
+def update_storage_json() -> None:
     path = storage_path()
     old = StorageJSON.load(path)
     new = StorageJSON.from_esphome_core(CORE, old)
@@ -131,7 +113,14 @@ def update_storage_json():
         return
 
     if storage_should_clean(old, new):
-        _LOGGER.info("Core config, version changed, cleaning build files...")
+        if old is not None and old.loaded_integrations - new.loaded_integrations:
+            removed = old.loaded_integrations - new.loaded_integrations
+            _LOGGER.info(
+                "Components removed (%s), cleaning build files...",
+                ", ".join(sorted(removed)),
+            )
+        else:
+            _LOGGER.info("Core config or version changed, cleaning build files...")
         clean_build()
     elif storage_should_update_cmake_cache(old, new):
         _LOGGER.info("Integrations changed, cleaning cmake cache...")
@@ -140,40 +129,6 @@ def update_storage_json():
     new.save(path)
 
 
-def format_ini(data: dict[str, str | list[str]]) -> str:
-    content = ""
-    for key, value in sorted(data.items()):
-        if isinstance(value, list):
-            content += f"{key} =\n"
-            for x in value:
-                content += f"    {x}\n"
-        else:
-            content += f"{key} = {value}\n"
-    return content
-
-
-def get_ini_content():
-    CORE.add_platformio_option(
-        "lib_deps", [x.as_lib_dep for x in CORE.libraries] + ["${common.lib_deps}"]
-    )
-    # Sort to avoid changing build flags order
-    CORE.add_platformio_option("build_flags", sorted(CORE.build_flags))
-
-    # Sort to avoid changing build unflags order
-    CORE.add_platformio_option("build_unflags", sorted(CORE.build_unflags))
-
-    # Add extra script for C++ flags
-    CORE.add_platformio_option("extra_scripts", [f"pre:{CXX_FLAGS_FILE_NAME}"])
-
-    content = "[platformio]\n"
-    content += f"description = ESPHome {__version__}\n"
-
-    content += f"[env:{CORE.name}]\n"
-    content += format_ini(CORE.platformio_options)
-
-    return content
-
-
 def find_begin_end(text, begin_s, end_s):
     begin_index = text.find(begin_s)
     if begin_index == -1:
@@ -201,34 +156,6 @@ def find_begin_end(text, begin_s, end_s):
     return text[:begin_index], text[(end_index + len(end_s)) :]
 
 
-def write_platformio_ini(content):
-    update_storage_json()
-    path = CORE.relative_build_path("platformio.ini")
-
-    if os.path.isfile(path):
-        text = read_file(path)
-        content_format = find_begin_end(
-            text, INI_AUTO_GENERATE_BEGIN, INI_AUTO_GENERATE_END
-        )
-    else:
-        content_format = INI_BASE_FORMAT
-    full_file = f"{content_format[0] + INI_AUTO_GENERATE_BEGIN}\n{content}"
-    full_file += INI_AUTO_GENERATE_END + content_format[1]
-    write_file_if_changed(path, full_file)
-
-
-def write_platformio_project():
-    mkdir_p(CORE.build_path)
-
-    content = get_ini_content()
-    if not get_bool_env(ENV_NOGITIGNORE):
-        write_gitignore()
-    write_platformio_ini(content)
-
-    # Write extra script for C++ specific flags
-    write_cxx_flags_script()
-
-
 DEFINES_H_FORMAT = ESPHOME_H_FORMAT = """\
 #pragma once
 #include "esphome/core/macros.h"
@@ -341,7 +268,7 @@ def generate_version_h():
 
 def write_cpp(code_s):
     path = CORE.relative_src_path("main.cpp")
-    if os.path.isfile(path):
+    if path.is_file():
         text = read_file(path)
         code_format = find_begin_end(
             text, CPP_AUTO_GENERATE_BEGIN, CPP_AUTO_GENERATE_END
@@ -367,24 +294,82 @@ def write_cpp(code_s):
 
 def clean_cmake_cache():
     pioenvs = CORE.relative_pioenvs_path()
-    if os.path.isdir(pioenvs):
-        pioenvs_cmake_path = CORE.relative_pioenvs_path(CORE.name, "CMakeCache.txt")
-        if os.path.isfile(pioenvs_cmake_path):
+    if pioenvs.is_dir():
+        pioenvs_cmake_path = pioenvs / CORE.name / "CMakeCache.txt"
+        if pioenvs_cmake_path.is_file():
             _LOGGER.info("Deleting %s", pioenvs_cmake_path)
-            os.remove(pioenvs_cmake_path)
+            pioenvs_cmake_path.unlink()
 
 
 def clean_build():
     import shutil
 
+    # Allow skipping cache cleaning for integration tests
+    if os.environ.get("ESPHOME_SKIP_CLEAN_BUILD"):
+        _LOGGER.warning("Skipping build cleaning (ESPHOME_SKIP_CLEAN_BUILD set)")
+        return
+
     pioenvs = CORE.relative_pioenvs_path()
-    if os.path.isdir(pioenvs):
+    if pioenvs.is_dir():
         _LOGGER.info("Deleting %s", pioenvs)
         shutil.rmtree(pioenvs)
     piolibdeps = CORE.relative_piolibdeps_path()
-    if os.path.isdir(piolibdeps):
+    if piolibdeps.is_dir():
         _LOGGER.info("Deleting %s", piolibdeps)
         shutil.rmtree(piolibdeps)
+    dependencies_lock = CORE.relative_build_path("dependencies.lock")
+    if dependencies_lock.is_file():
+        _LOGGER.info("Deleting %s", dependencies_lock)
+        dependencies_lock.unlink()
+
+    # Clean PlatformIO cache to resolve CMake compiler detection issues
+    # This helps when toolchain paths change or get corrupted
+    try:
+        from platformio.project.config import ProjectConfig
+    except ImportError:
+        # PlatformIO is not available, skip cache cleaning
+        pass
+    else:
+        config = ProjectConfig.get_instance()
+        cache_dir = Path(config.get("platformio", "cache_dir"))
+        if cache_dir.is_dir():
+            _LOGGER.info("Deleting PlatformIO cache %s", cache_dir)
+            shutil.rmtree(cache_dir)
+
+
+def clean_all(configuration: list[str]):
+    import shutil
+
+    data_dirs = [Path(dir) / ".esphome" for dir in configuration]
+    if is_ha_addon():
+        data_dirs.append(Path("/data"))
+    if "ESPHOME_DATA_DIR" in os.environ:
+        data_dirs.append(Path(get_str_env("ESPHOME_DATA_DIR", None)))
+
+    # Clean build dir
+    for dir in data_dirs:
+        if dir.is_dir():
+            _LOGGER.info("Cleaning %s", dir)
+            # Don't remove storage or .json files which are needed by the dashboard
+            for item in dir.iterdir():
+                if item.is_file() and not item.name.endswith(".json"):
+                    item.unlink()
+                elif item.is_dir() and item.name != "storage":
+                    shutil.rmtree(item)
+
+    # Clean PlatformIO project files
+    try:
+        from platformio.project.config import ProjectConfig
+    except ImportError:
+        # PlatformIO is not available, skip cleaning
+        pass
+    else:
+        config = ProjectConfig.get_instance()
+        for pio_dir in ["cache_dir", "packages_dir", "platforms_dir", "core_dir"]:
+            path = Path(config.get("platformio", pio_dir))
+            if path.is_dir():
+                _LOGGER.info("Deleting PlatformIO %s %s", pio_dir, path)
+                shutil.rmtree(path)
 
 
 GITIGNORE_CONTENT = """# Gitignore settings for ESPHome
@@ -397,23 +382,5 @@ GITIGNORE_CONTENT = """# Gitignore settings for ESPHome
 
 def write_gitignore():
     path = CORE.relative_config_path(".gitignore")
-    if not os.path.isfile(path):
-        with open(file=path, mode="w", encoding="utf-8") as f:
-            f.write(GITIGNORE_CONTENT)
-
-
-CXX_FLAGS_FILE_NAME = "cxx_flags.py"
-CXX_FLAGS_FILE_CONTENTS = """# Auto-generated ESPHome script for C++ specific compiler flags
-Import("env")
-
-# Add C++ specific flags
-"""
-
-
-def write_cxx_flags_script() -> None:
-    path = CORE.relative_build_path(CXX_FLAGS_FILE_NAME)
-    contents = CXX_FLAGS_FILE_CONTENTS
-    if not CORE.is_host:
-        contents += 'env.Append(CXXFLAGS=["-Wno-volatile"])'
-        contents += "\n"
-    write_file_if_changed(path, contents)
+    if not path.is_file():
+        path.write_text(GITIGNORE_CONTENT, encoding="utf-8")
diff --git a/esphome/yaml_util.py b/esphome/yaml_util.py
index e52fc9e788..359b72b48f 100644
--- a/esphome/yaml_util.py
+++ b/esphome/yaml_util.py
@@ -1,7 +1,6 @@
 from __future__ import annotations
 
 from collections.abc import Callable
-import fnmatch
 import functools
 import inspect
 from io import BytesIO, TextIOBase, TextIOWrapper
@@ -9,6 +8,7 @@ from ipaddress import _BaseAddress, _BaseNetwork
 import logging
 import math
 import os
+from pathlib import Path
 from typing import Any
 import uuid
 
@@ -56,9 +56,12 @@ class ESPHomeDataBase:
     def from_node(self, node):
         # pylint: disable=attribute-defined-outside-init
         self._esp_range = DocumentRange.from_marks(node.start_mark, node.end_mark)
-        if isinstance(node, yaml.ScalarNode):
-            if node.style is not None and node.style in "|>":
-                self._content_offset = 1
+        if (
+            isinstance(node, yaml.ScalarNode)
+            and node.style is not None
+            and node.style in "|>"
+        ):
+            self._content_offset = 1
 
     def from_database(self, database):
         # pylint: disable=attribute-defined-outside-init
@@ -66,7 +69,7 @@ class ESPHomeDataBase:
         self._content_offset = database.content_offset
 
 
-class ESPForceValue:
+class ESPLiteralValue:
     pass
 
 
@@ -106,7 +109,9 @@ def _add_data_ref(fn):
 class ESPHomeLoaderMixin:
     """Loader class that keeps track of line numbers."""
 
-    def __init__(self, name: str, yaml_loader: Callable[[str], dict[str, Any]]) -> None:
+    def __init__(
+        self, name: Path, yaml_loader: Callable[[Path], dict[str, Any]]
+    ) -> None:
         """Initialize the loader."""
         self.name = name
         self.yaml_loader = yaml_loader
@@ -251,12 +256,8 @@ class ESPHomeLoaderMixin:
             f"Environment variable '{node.value}' not defined", node.start_mark
         )
 
-    @property
-    def _directory(self) -> str:
-        return os.path.dirname(self.name)
-
-    def _rel_path(self, *args: str) -> str:
-        return os.path.join(self._directory, *args)
+    def _rel_path(self, *args: str) -> Path:
+        return self.name.parent / Path(*args)
 
     @_add_data_ref
     def construct_secret(self, node: yaml.Node) -> str:
@@ -266,8 +267,8 @@ class ESPHomeLoaderMixin:
             if self.name == CORE.config_path:
                 raise e
             try:
-                main_config_dir = os.path.dirname(CORE.config_path)
-                main_secret_yml = os.path.join(main_config_dir, SECRET_YAML)
+                main_config_dir = CORE.config_path.parent
+                main_secret_yml = main_config_dir / SECRET_YAML
                 secrets = self.yaml_loader(main_secret_yml)
             except EsphomeError as er:
                 raise EsphomeError(f"{e}\n{er}") from er
@@ -302,8 +303,7 @@ class ESPHomeLoaderMixin:
         result = self.yaml_loader(self._rel_path(file))
         if not vars:
             vars = {}
-        result = substitute_vars(result, vars)
-        return result
+        return substitute_vars(result, vars)
 
     @_add_data_ref
     def construct_include_dir_list(self, node: yaml.Node) -> list[dict[str, Any]]:
@@ -327,7 +327,7 @@ class ESPHomeLoaderMixin:
         files = filter_yaml_files(_find_files(self._rel_path(node.value), "*.yaml"))
         mapping = OrderedDict()
         for fname in files:
-            filename = os.path.splitext(os.path.basename(fname))[0]
+            filename = fname.stem
             mapping[filename] = self.yaml_loader(fname)
         return mapping
 
@@ -348,9 +348,15 @@ class ESPHomeLoaderMixin:
         return Lambda(str(node.value))
 
     @_add_data_ref
-    def construct_force(self, node: yaml.Node) -> ESPForceValue:
-        obj = self.construct_scalar(node)
-        return add_class_to_obj(obj, ESPForceValue)
+    def construct_literal(self, node: yaml.Node) -> ESPLiteralValue:
+        obj = None
+        if isinstance(node, yaml.ScalarNode):
+            obj = self.construct_scalar(node)
+        elif isinstance(node, yaml.SequenceNode):
+            obj = self.construct_sequence(node)
+        elif isinstance(node, yaml.MappingNode):
+            obj = self.construct_mapping(node)
+        return add_class_to_obj(obj, ESPLiteralValue)
 
     @_add_data_ref
     def construct_extend(self, node: yaml.Node) -> Extend:
@@ -367,8 +373,8 @@ class ESPHomeLoader(ESPHomeLoaderMixin, FastestAvailableSafeLoader):
     def __init__(
         self,
         stream: TextIOBase | BytesIO,
-        name: str,
-        yaml_loader: Callable[[str], dict[str, Any]],
+        name: Path,
+        yaml_loader: Callable[[Path], dict[str, Any]],
     ) -> None:
         FastestAvailableSafeLoader.__init__(self, stream)
         ESPHomeLoaderMixin.__init__(self, name, yaml_loader)
@@ -380,8 +386,8 @@ class ESPHomePurePythonLoader(ESPHomeLoaderMixin, PurePythonLoader):
     def __init__(
         self,
         stream: TextIOBase | BytesIO,
-        name: str,
-        yaml_loader: Callable[[str], dict[str, Any]],
+        name: Path,
+        yaml_loader: Callable[[Path], dict[str, Any]],
     ) -> None:
         PurePythonLoader.__init__(self, stream)
         ESPHomeLoaderMixin.__init__(self, name, yaml_loader)
@@ -407,29 +413,29 @@ for _loader in (ESPHomeLoader, ESPHomePurePythonLoader):
         "!include_dir_merge_named", _loader.construct_include_dir_merge_named
     )
     _loader.add_constructor("!lambda", _loader.construct_lambda)
-    _loader.add_constructor("!force", _loader.construct_force)
+    _loader.add_constructor("!literal", _loader.construct_literal)
     _loader.add_constructor("!extend", _loader.construct_extend)
     _loader.add_constructor("!remove", _loader.construct_remove)
 
 
-def load_yaml(fname: str, clear_secrets: bool = True) -> Any:
+def load_yaml(fname: Path, clear_secrets: bool = True) -> Any:
     if clear_secrets:
         _SECRET_VALUES.clear()
         _SECRET_CACHE.clear()
     return _load_yaml_internal(fname)
 
 
-def _load_yaml_internal(fname: str) -> Any:
+def _load_yaml_internal(fname: Path) -> Any:
     """Load a YAML file."""
     try:
-        with open(fname, encoding="utf-8") as f_handle:
+        with fname.open(encoding="utf-8") as f_handle:
             return parse_yaml(fname, f_handle)
     except (UnicodeDecodeError, OSError) as err:
         raise EsphomeError(f"Error reading file {fname}: {err}") from err
 
 
 def parse_yaml(
-    file_name: str, file_handle: TextIOWrapper, yaml_loader=_load_yaml_internal
+    file_name: Path, file_handle: TextIOWrapper, yaml_loader=_load_yaml_internal
 ) -> Any:
     """Parse a YAML file."""
     try:
@@ -481,9 +487,9 @@ def substitute_vars(config, vars):
 
 def _load_yaml_internal_with_type(
     loader_type: type[ESPHomeLoader] | type[ESPHomePurePythonLoader],
-    fname: str,
+    fname: Path,
     content: TextIOWrapper,
-    yaml_loader: Any,
+    yaml_loader: Callable[[Path], dict[str, Any]],
 ) -> Any:
     """Load a YAML file."""
     loader = loader_type(content, fname, yaml_loader)
@@ -510,13 +516,14 @@ def _is_file_valid(name: str) -> bool:
     return not name.startswith(".")
 
 
-def _find_files(directory, pattern):
+def _find_files(directory: Path, pattern):
     """Recursively load files in a directory."""
-    for root, dirs, files in os.walk(directory, topdown=True):
+    for root, dirs, files in os.walk(directory):
         dirs[:] = [d for d in dirs if _is_file_valid(d)]
-        for basename in files:
-            if _is_file_valid(basename) and fnmatch.fnmatch(basename, pattern):
-                filename = os.path.join(root, basename)
+        for f in files:
+            filename = Path(f)
+            if _is_file_valid(f) and filename.match(pattern):
+                filename = Path(root) / filename
                 yield filename
 
 
@@ -625,3 +632,4 @@ ESPHomeDumper.add_multi_representer(TimePeriod, ESPHomeDumper.represent_stringif
 ESPHomeDumper.add_multi_representer(Lambda, ESPHomeDumper.represent_lambda)
 ESPHomeDumper.add_multi_representer(core.ID, ESPHomeDumper.represent_id)
 ESPHomeDumper.add_multi_representer(uuid.UUID, ESPHomeDumper.represent_stringify)
+ESPHomeDumper.add_multi_representer(Path, ESPHomeDumper.represent_stringify)
diff --git a/esphome/zeroconf.py b/esphome/zeroconf.py
index fa496b3488..dc4ca77eb4 100644
--- a/esphome/zeroconf.py
+++ b/esphome/zeroconf.py
@@ -68,8 +68,11 @@ class DashboardBrowser(AsyncServiceBrowser):
 
 
 class DashboardImportDiscovery:
-    def __init__(self) -> None:
+    def __init__(
+        self, on_update: Callable[[str, DiscoveredImport | None], None] | None = None
+    ) -> None:
         self.import_state: dict[str, DiscoveredImport] = {}
+        self.on_update = on_update
 
     def browser_callback(
         self,
@@ -85,7 +88,9 @@ class DashboardImportDiscovery:
             state_change,
         )
         if state_change == ServiceStateChange.Removed:
-            self.import_state.pop(name, None)
+            removed = self.import_state.pop(name, None)
+            if removed and self.on_update:
+                self.on_update(name, None)
             return
 
         if state_change == ServiceStateChange.Updated and name not in self.import_state:
@@ -139,7 +144,7 @@ class DashboardImportDiscovery:
         if friendly_name is not None:
             friendly_name = friendly_name.decode()
 
-        self.import_state[name] = DiscoveredImport(
+        discovered = DiscoveredImport(
             friendly_name=friendly_name,
             device_name=node_name,
             package_import_url=import_url,
@@ -147,6 +152,10 @@ class DashboardImportDiscovery:
             project_version=project_version,
             network=network,
         )
+        is_new = name not in self.import_state
+        self.import_state[name] = discovered
+        if is_new and self.on_update:
+            self.on_update(name, discovered)
 
     def update_device_mdns(self, node_name: str, version: str):
         storage_path = ext_storage_path(node_name + ".yaml")
diff --git a/netlify.toml b/netlify.toml
index 7783414a91..5f177e6b3a 100644
--- a/netlify.toml
+++ b/netlify.toml
@@ -1,3 +1,4 @@
 [build]
   command = "script/build-api-docs"
   publish = "api-docs"
+  environment = { PYTHON_VERSION = "3.13" }
diff --git a/platformio.ini b/platformio.ini
index 8fcc578103..6b2a8657bb 100644
--- a/platformio.ini
+++ b/platformio.ini
@@ -40,6 +40,7 @@ lib_deps =
     functionpointer/arduino-MLX90393@1.0.2 ; mlx90393
     pavlodn/HaierProtocol@0.9.31           ; haier
     kikuchan98/pngle@1.1.0                 ; online_image
+    https://github.com/esphome/TinyGPSPlus.git#v1.1.0     ; gps
     ; Using the repository directly, otherwise ESP-IDF can't use the library
     https://github.com/bitbank2/JPEGDEC.git#ca1e0f2    ; online_image
     ; This is using the repository until a new release is published to PlatformIO
@@ -71,14 +72,12 @@ lib_deps =
     SPI                                                   ; spi (Arduino built-in)
     Wire                                                  ; i2c (Arduino built-int)
     heman/AsyncMqttClient-esphome@1.0.0                   ; mqtt
-    ESP32Async/ESPAsyncWebServer@3.7.8                    ; web_server_base
     fastled/FastLED@3.9.16                                ; fastled_base
-    mikalhart/TinyGPSPlus@1.1.0                           ; gps
     freekode/TM1651@1.0.1                                 ; tm1651
     glmnet/Dsmr@0.7                                       ; dsmr
     rweather/Crypto@0.4.0                                 ; dsmr
     dudanov/MideaUART@1.1.9                               ; midea
-    tonia/HeatpumpIR@1.0.35                               ; heatpumpir
+    tonia/HeatpumpIR@1.0.37                               ; heatpumpir
 build_flags =
     ${common.build_flags}
     -DUSE_ARDUINO
@@ -107,6 +106,7 @@ lib_deps =
     ESP8266WiFi                           ; wifi (Arduino built-in)
     Update                                ; ota (Arduino built-in)
     ESP32Async/ESPAsyncTCP@2.0.0          ; async_tcp
+    ESP32Async/ESPAsyncWebServer@3.7.8    ; web_server_base
     makuna/NeoPixelBus@2.7.3              ; neopixelbus
     ESP8266HTTPClient                     ; http_request (Arduino built-in)
     ESP8266mDNS                           ; mdns (Arduino built-in)
@@ -125,11 +125,11 @@ extra_scripts = post:esphome/components/esp8266/post_build.py.script
 ; This are common settings for the ESP32 (all variants) using Arduino.
 [common:esp32-arduino]
 extends = common:arduino
-platform = https://github.com/pioarduino/platform-espressif32/releases/download/53.03.13/platform-espressif32.zip
+platform = https://github.com/pioarduino/platform-espressif32/releases/download/55.03.31-1/platform-espressif32.zip
 platform_packages =
-    pioarduino/framework-arduinoespressif32@https://github.com/espressif/arduino-esp32/releases/download/3.1.3/esp32-3.1.3.zip
+    pioarduino/framework-arduinoespressif32@https://github.com/espressif/arduino-esp32/releases/download/3.3.2/esp32-3.3.2.zip
 
-framework = arduino
+framework = arduino, espidf  ; Arduino as an ESP-IDF component
 lib_deps =
     ; order matters with lib-deps; some of the libs in common:arduino.lib_deps
     ; don't declare built-in libraries as dependencies, so they have to be declared first
@@ -138,7 +138,7 @@ lib_deps =
     WiFi                                 ; wifi,web_server_base,ethernet (Arduino built-in)
     Update                               ; ota,web_server_base (Arduino built-in)
     ${common:arduino.lib_deps}
-    ESP32Async/AsyncTCP@3.4.4            ; async_tcp
+    ESP32Async/AsyncTCP@3.4.5            ; async_tcp
     NetworkClientSecure                  ; http_request,nextion (Arduino built-in)
     HTTPClient                           ; http_request,nextion (Arduino built-in)
     ESPmDNS                              ; mdns (Arduino built-in)
@@ -147,7 +147,7 @@ lib_deps =
     makuna/NeoPixelBus@2.8.0             ; neopixelbus
     esphome/ESP32-audioI2S@2.3.0         ; i2s_audio
     droscy/esp_wireguard@0.4.2           ; wireguard
-    esphome/esp-audio-libs@1.1.4         ; audio
+    esphome/esp-audio-libs@2.0.1         ; audio
 
 build_flags =
     ${common:arduino.build_flags}
@@ -161,16 +161,16 @@ extra_scripts = post:esphome/components/esp32/post_build.py.script
 ; This are common settings for the ESP32 (all variants) using IDF.
 [common:esp32-idf]
 extends = common:idf
-platform = https://github.com/pioarduino/platform-espressif32/releases/download/53.03.13/platform-espressif32.zip
+platform = https://github.com/pioarduino/platform-espressif32/releases/download/55.03.31-1/platform-espressif32.zip
 platform_packages =
-    pioarduino/framework-espidf@https://github.com/pioarduino/esp-idf/releases/download/v5.3.2/esp-idf-v5.3.2.zip
+    pioarduino/framework-espidf@https://github.com/pioarduino/esp-idf/releases/download/v5.5.1/esp-idf-v5.5.1.zip
 
 framework = espidf
 lib_deps =
     ${common:idf.lib_deps}
     droscy/esp_wireguard@0.4.2              ; wireguard
     kahrendt/ESPMicroSpeechFeatures@1.1.0   ; micro_wake_word
-    esphome/esp-audio-libs@1.1.4            ; audio
+    esphome/esp-audio-libs@2.0.1            ; audio
 build_flags =
     ${common:idf.build_flags}
     -Wno-nonnull-compare
@@ -180,13 +180,6 @@ build_unflags =
     ${common.build_unflags}
 extra_scripts = post:esphome/components/esp32/post_build.py.script
 
-; This are common settings for the ESP32 using the latest ESP-IDF version.
-[common:esp32-idf-5_3]
-extends = common:esp32-idf
-platform = platformio/espressif32@6.8.0
-platform_packages =
-    platformio/framework-espidf@~3.50300.0
-
 ; These are common settings for the RP2040 using Arduino.
 [common:rp2040-arduino]
 extends = common:arduino
@@ -200,6 +193,7 @@ platform_packages =
 framework = arduino
 lib_deps =
     ${common:arduino.lib_deps}
+    ESP32Async/ESPAsyncWebServer@3.7.8    ; web_server_base
 build_flags =
     ${common:arduino.build_flags}
     -DUSE_RP2040
@@ -214,7 +208,8 @@ platform = libretiny@1.9.1
 framework = arduino
 lib_compat_mode = soft
 lib_deps =
-    droscy/esp_wireguard@0.4.2    ; wireguard
+    ESP32Async/ESPAsyncWebServer@3.7.8    ; web_server_base
+    droscy/esp_wireguard@0.4.2            ; wireguard
 build_flags =
     ${common:arduino.build_flags}
     -DUSE_LIBRETINY
@@ -228,8 +223,8 @@ extends = common
 platform = https://github.com/tomaszduda23/platform-nordicnrf52/archive/refs/tags/v10.3.0-1.zip
 framework = zephyr
 platform_packages =
-    platformio/framework-zephyr @ https://github.com/tomaszduda23/framework-sdk-nrf/archive/refs/tags/v2.6.1-4.zip
-    platformio/toolchain-gccarmnoneeabi@https://github.com/tomaszduda23/toolchain-sdk-ng/archive/refs/tags/v0.16.1-1.zip
+    platformio/framework-zephyr @ https://github.com/tomaszduda23/framework-sdk-nrf/archive/refs/tags/v2.6.1-7.zip
+    platformio/toolchain-gccarmnoneeabi@https://github.com/tomaszduda23/toolchain-sdk-ng/archive/refs/tags/v0.17.4-0.zip
 build_flags =
     ${common.build_flags}
     -DUSE_ZEPHYR
@@ -239,6 +234,7 @@ lib_deps =
     wjtje/qr-code-generator-library@1.7.0  ; qr_code
     pavlodn/HaierProtocol@0.9.31           ; haier
     functionpointer/arduino-MLX90393@1.0.2 ; mlx90393
+    https://github.com/esphome/TinyGPSPlus.git#v1.1.0     ; gps
     https://github.com/Sensirion/arduino-gas-index-algorithm.git#3.2.1 ; Sensirion Gas Index Algorithm Arduino Library
     lvgl/lvgl@8.4.0                                       ; lvgl
 
@@ -280,6 +276,7 @@ build_unflags =
 [env:esp32-arduino-tidy]
 extends = common:esp32-arduino
 board = esp32dev
+board_build.esp-idf.sdkconfig_path = .temp/sdkconfig-esp32-arduino-tidy
 build_flags =
     ${common:esp32-arduino.build_flags}
     ${flags:clangtidy.build_flags}
@@ -298,17 +295,6 @@ build_flags =
 build_unflags =
     ${common.build_unflags}
 
-[env:esp32-idf-5_3]
-extends = common:esp32-idf-5_3
-board = esp32dev
-board_build.esp-idf.sdkconfig_path = .temp/sdkconfig-esp32-idf
-build_flags =
-    ${common:esp32-idf.build_flags}
-    ${flags:runtime.build_flags}
-    -DUSE_ESP32_VARIANT_ESP32
-build_unflags =
-    ${common.build_unflags}
-
 [env:esp32-idf-tidy]
 extends = common:esp32-idf
 board = esp32dev
@@ -353,17 +339,6 @@ build_flags =
 build_unflags =
     ${common.build_unflags}
 
-[env:esp32c3-idf-5_3]
-extends = common:esp32-idf-5_3
-board = esp32-c3-devkitm-1
-board_build.esp-idf.sdkconfig_path = .temp/sdkconfig-esp32c3-idf
-build_flags =
-    ${common:esp32-idf.build_flags}
-    ${flags:runtime.build_flags}
-    -DUSE_ESP32_VARIANT_ESP32C3
-build_unflags =
-    ${common.build_unflags}
-
 [env:esp32c3-idf-tidy]
 extends = common:esp32-idf
 board = esp32-c3-devkitm-1
@@ -385,6 +360,19 @@ build_flags =
     ${common:esp32-idf.build_flags}
     ${flags:runtime.build_flags}
     -DUSE_ESP32_VARIANT_ESP32C6
+build_unflags =
+    ${common.build_unflags}
+
+[env:esp32c6-idf-tidy]
+extends = common:esp32-idf
+board = esp32-c6-devkitc-1
+board_build.esp-idf.sdkconfig_path = .temp/sdkconfig-esp32c6-idf-tidy
+build_flags =
+    ${common:esp32-idf.build_flags}
+    ${flags:clangtidy.build_flags}
+    -DUSE_ESP32_VARIANT_ESP32C6
+build_unflags =
+    ${common.build_unflags}
 
 ;;;;;;;; ESP32-S2 ;;;;;;;;
 
@@ -419,17 +407,6 @@ build_flags =
 build_unflags =
     ${common.build_unflags}
 
-[env:esp32s2-idf-5_3]
-extends = common:esp32-idf-5_3
-board = esp32-s2-kaluga-1
-board_build.esp-idf.sdkconfig_path = .temp/sdkconfig-esp32s2-idf
-build_flags =
-    ${common:esp32-idf.build_flags}
-    ${flags:runtime.build_flags}
-    -DUSE_ESP32_VARIANT_ESP32S2
-build_unflags =
-    ${common.build_unflags}
-
 [env:esp32s2-idf-tidy]
 extends = common:esp32-idf
 board = esp32-s2-kaluga-1
@@ -474,17 +451,6 @@ build_flags =
 build_unflags =
     ${common.build_unflags}
 
-[env:esp32s3-idf-5_3]
-extends = common:esp32-idf-5_3
-board = esp32-s3-devkitc-1
-board_build.esp-idf.sdkconfig_path = .temp/sdkconfig-esp32s3-idf
-build_flags =
-    ${common:esp32-idf.build_flags}
-    ${flags:runtime.build_flags}
-    -DUSE_ESP32_VARIANT_ESP32S3
-build_unflags =
-    ${common.build_unflags}
-
 [env:esp32s3-idf-tidy]
 extends = common:esp32-idf
 board = esp32-s3-devkitc-1
@@ -565,6 +531,8 @@ build_flags =
 build_unflags =
     ${common.build_unflags}
 
+;;;;;;;; Host ;;;;;;;;
+
 [env:host]
 extends = common
 platform = platformio/native
diff --git a/pyproject.toml b/pyproject.toml
index 97b0df9eff..b7b4a48d7e 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name        = "esphome"
-license     = {text = "MIT"}
+license     = "MIT"
 description = "ESPHome is a system to configure your microcontrollers by simple yet powerful configuration files and control them remotely through Home Automation systems."
 readme      = "README.md"
 authors     = [
@@ -15,20 +15,19 @@ classifiers = [
     "Environment :: Console",
     "Intended Audience :: Developers",
     "Intended Audience :: End Users/Desktop",
-    "License :: OSI Approved :: MIT License",
     "Programming Language :: C++",
     "Programming Language :: Python :: 3",
     "Topic :: Home Automation",
 ]
-requires-python = ">=3.10.0"
+requires-python = ">=3.11.0"
 
 dynamic = ["dependencies", "optional-dependencies", "version"]
 
 [project.urls]
 "Documentation"           = "https://esphome.io"
 "Source Code"             = "https://github.com/esphome/esphome"
-"Bug Tracker"             = "https://github.com/esphome/issues/issues"
-"Feature Request Tracker" = "https://github.com/esphome/feature-requests/issues"
+"Bug Tracker"             = "https://github.com/esphome/esphome/issues"
+"Feature Request Tracker" = "https://github.com/orgs/esphome/discussions"
 "Discord"                 = "https://discord.gg/KhAMKrd"
 "Forum"                   = "https://community.home-assistant.io/c/esphome"
 "Twitter"                 = "https://twitter.com/esphome_"
@@ -62,7 +61,7 @@ addopts = [
 ]
 
 [tool.pylint.MAIN]
-py-version = "3.10"
+py-version = "3.11"
 ignore = [
   "api_pb2.py",
 ]
@@ -106,16 +105,20 @@ expected-line-ending-format = "LF"
 
 [tool.ruff]
 required-version = ">=0.5.0"
-target-version = "py310"
+target-version = "py311"
 exclude = ['generated']
 
 [tool.ruff.lint]
 select = [
-  "E", # pycodestyle
-  "F", # pyflakes/autoflake
-  "I", # isort
-  "PL", # pylint
-  "UP", # pyupgrade
+  "E",    # pycodestyle
+  "F",    # pyflakes/autoflake
+  "FURB", # refurb
+  "I",    # isort
+  "PERF", # performance
+  "PL",   # pylint
+  "SIM",  # flake8-simplify
+  "RET",  # flake8-ret
+  "UP",   # pyupgrade
 ]
 
 ignore = [
diff --git a/requirements.txt b/requirements.txt
index 8829208f30..ec7794c75a 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -1,32 +1,28 @@
-async_timeout==5.0.1; python_version <= "3.10"
 cryptography==45.0.1
 voluptuous==0.15.2
-PyYAML==6.0.2
+PyYAML==6.0.3
 paho-mqtt==1.6.1
 colorama==0.4.6
 icmplib==3.0.4
-tornado==6.5.1
+tornado==6.5.2
 tzlocal==5.3.1    # from time
 tzdata>=2021.1  # from time
 pyserial==3.5
 platformio==6.1.18  # When updating platformio, also update /docker/Dockerfile
-esptool==4.9.0
+esptool==5.1.0
 click==8.1.7
-esphome-dashboard==20250514.0
-aioesphomeapi==35.0.1
-zeroconf==0.147.0
+esphome-dashboard==20251013.0
+aioesphomeapi==42.2.0
+zeroconf==0.148.0
 puremagic==1.30
-ruamel.yaml==0.18.14 # dashboard_import
+ruamel.yaml==0.18.15 # dashboard_import
+ruamel.yaml.clib==0.2.14 # dashboard_import
 esphome-glyphsets==0.2.0
-pillow==10.4.0
+pillow==11.3.0
 cairosvg==2.8.2
 freetype-py==2.5.1
 jinja2==3.1.6
 
-# esp-idf requires this, but doesn't bundle it by default
-# https://github.com/espressif/esp-idf/blob/220590d599e134d7a5e7f1e683cc4550349ffbf8/requirements.txt#L24
-kconfiglib==13.7.1
-
 # esp-idf >= 5.0 requires this
 pyparsing >= 3.0
 
diff --git a/requirements_test.txt b/requirements_test.txt
index 67eae63a31..4c60a31d7f 100644
--- a/requirements_test.txt
+++ b/requirements_test.txt
@@ -1,14 +1,14 @@
-pylint==3.3.7
+pylint==4.0.1
 flake8==7.3.0  # also change in .pre-commit-config.yaml when updating
-ruff==0.12.3  # also change in .pre-commit-config.yaml when updating
-pyupgrade==3.20.0  # also change in .pre-commit-config.yaml when updating
+ruff==0.14.1  # also change in .pre-commit-config.yaml when updating
+pyupgrade==3.21.0  # also change in .pre-commit-config.yaml when updating
 pre-commit
 
 # Unit tests
-pytest==8.4.1
-pytest-cov==6.2.1
-pytest-mock==3.14.1
-pytest-asyncio==1.0.0
-pytest-xdist==3.7.0
+pytest==8.4.2
+pytest-cov==7.0.0
+pytest-mock==3.15.1
+pytest-asyncio==1.2.0
+pytest-xdist==3.8.0
 asyncmock==0.4.2
 hypothesis==6.92.1
diff --git a/script/analyze_component_buses.py b/script/analyze_component_buses.py
new file mode 100755
index 0000000000..d0882e22e9
--- /dev/null
+++ b/script/analyze_component_buses.py
@@ -0,0 +1,674 @@
+#!/usr/bin/env python3
+"""Analyze component test files to detect which common bus configs they use.
+
+This script scans component test files and extracts which common bus configurations
+(i2c, spi, uart, etc.) are included via the packages mechanism. This information
+is used to group components that can be tested together.
+
+Components can only be grouped together if they use the EXACT SAME set of common
+bus configurations, ensuring that merged configs are compatible.
+
+Example output:
+{
+    "component1": {
+        "esp32-ard": ["i2c", "uart_19200"],
+        "esp32-idf": ["i2c", "uart_19200"]
+    },
+    "component2": {
+        "esp32-ard": ["spi"],
+        "esp32-idf": ["spi"]
+    }
+}
+"""
+
+from __future__ import annotations
+
+import argparse
+from functools import lru_cache
+import json
+from pathlib import Path
+import re
+import sys
+from typing import Any
+
+# Add esphome to path
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from esphome import yaml_util
+from esphome.config_helpers import Extend, Remove
+
+# Path to common bus configs
+COMMON_BUS_PATH = Path("tests/test_build_components/common")
+
+# Package dependencies - maps packages to the packages they include
+# When a component uses a package on the left, it automatically gets
+# the packages on the right as well
+PACKAGE_DEPENDENCIES = {
+    "modbus": ["uart"],  # modbus packages include uart packages
+    # Add more package dependencies here as needed
+}
+
+# Bus types that can be defined directly in config files
+# Components defining these directly cannot be grouped (they create unique bus IDs)
+DIRECT_BUS_TYPES = (
+    "i2c",
+    "spi",
+    "uart",
+    "modbus",
+    "remote_transmitter",
+    "remote_receiver",
+)
+
+# Signature for components with no bus requirements
+# These components can be merged with any other group
+NO_BUSES_SIGNATURE = "no_buses"
+
+# Prefix for isolated component signatures
+# Isolated components have unique signatures and cannot be merged with others
+ISOLATED_SIGNATURE_PREFIX = "isolated_"
+
+# Base bus components - these ARE the bus implementations and should not
+# be flagged as needing migration since they are the platform/base components
+BASE_BUS_COMPONENTS = {
+    "i2c",
+    "spi",
+    "uart",
+    "modbus",
+    "canbus",
+    "remote_transmitter",
+    "remote_receiver",
+}
+
+# Components that must be tested in isolation (not grouped or batched with others)
+# These have known build issues that prevent grouping
+# NOTE: This should be kept in sync with both test_build_components and split_components_for_ci.py
+ISOLATED_COMPONENTS = {
+    "animation": "Has display lambda in common.yaml that requires existing display platform - breaks when merged without display",
+    "esphome": "Defines devices/areas in esphome: section that are referenced in other sections - breaks when merged",
+    "ethernet": "Defines ethernet: which conflicts with wifi: used by most components",
+    "ethernet_info": "Related to ethernet component which conflicts with wifi",
+    "lvgl": "Defines multiple SDL displays on host platform that conflict when merged with other display configs",
+    "mapping": "Uses dict format for image/display sections incompatible with standard list format - ESPHome merge_config cannot handle",
+    "openthread": "Conflicts with wifi: used by most components",
+    "openthread_info": "Conflicts with wifi: used by most components",
+    "matrix_keypad": "Needs isolation due to keypad",
+    "modbus_controller": "Defines multiple modbus buses for testing client/server functionality - conflicts with package modbus bus",
+    "neopixelbus": "RMT type conflict with ESP32 Arduino/ESP-IDF headers (enum vs struct rmt_channel_t)",
+    "packages": "cannot merge packages",
+}
+
+
+@lru_cache(maxsize=1)
+def get_common_bus_packages() -> frozenset[str]:
+    """Get the list of common bus package names.
+
+    Reads from tests/test_build_components/common/ directory
+    and caches the result. All bus types support component grouping
+    for config validation since --testing-mode bypasses runtime conflicts.
+
+    Returns:
+        Frozenset of common bus package names (i2c, spi, uart, etc.)
+    """
+    if not COMMON_BUS_PATH.exists():
+        return frozenset()
+
+    # List all directories in common/ - these are the bus package names
+    return frozenset(d.name for d in COMMON_BUS_PATH.iterdir() if d.is_dir())
+
+
+def uses_local_file_references(component_dir: Path) -> bool:
+    """Check if a component uses local file references via $component_dir.
+
+    Components that reference local files cannot be grouped because each needs
+    a unique component_dir path pointing to their specific directory.
+
+    Args:
+        component_dir: Path to the component's test directory
+
+    Returns:
+        True if the component uses $component_dir for local file references
+    """
+    common_yaml = component_dir / "common.yaml"
+    if not common_yaml.exists():
+        return False
+
+    try:
+        content = common_yaml.read_text()
+    except Exception:  # pylint: disable=broad-exception-caught
+        return False
+
+    # Pattern to match $component_dir or ${component_dir} references
+    # These indicate local file usage that prevents grouping
+    return bool(re.search(r"\$\{?component_dir\}?", content))
+
+
+def is_platform_component(component_dir: Path) -> bool:
+    """Check if a component is a platform component (abstract base class).
+
+    Platform components have IS_PLATFORM_COMPONENT = True and cannot be
+    instantiated without a platform-specific implementation. These components
+    define abstract methods and cause linker errors if compiled standalone.
+
+    Examples: canbus, mcp23x08_base, mcp23x17_base
+
+    Args:
+        component_dir: Path to the component's test directory
+
+    Returns:
+        True if this is a platform component
+    """
+    # Check in the actual component source, not tests
+    # tests/components/X -> tests/components -> tests -> repo root
+    repo_root = component_dir.parent.parent.parent
+    comp_init = (
+        repo_root / "esphome" / "components" / component_dir.name / "__init__.py"
+    )
+
+    if not comp_init.exists():
+        return False
+
+    try:
+        content = comp_init.read_text()
+        return "IS_PLATFORM_COMPONENT = True" in content
+    except Exception:  # pylint: disable=broad-exception-caught
+        return False
+
+
+def _contains_extend_or_remove(data: Any) -> bool:
+    """Recursively check if data contains Extend or Remove objects.
+
+    Args:
+        data: Parsed YAML data structure
+
+    Returns:
+        True if any Extend or Remove objects are found
+    """
+    if isinstance(data, (Extend, Remove)):
+        return True
+
+    if isinstance(data, dict):
+        for value in data.values():
+            if _contains_extend_or_remove(value):
+                return True
+
+    if isinstance(data, list):
+        for item in data:
+            if _contains_extend_or_remove(item):
+                return True
+
+    return False
+
+
+def analyze_yaml_file(yaml_file: Path) -> dict[str, Any]:
+    """Load a YAML file once and extract all needed information.
+
+    This loads the YAML file a single time and extracts all information needed
+    for component analysis, avoiding multiple file reads.
+
+    Args:
+        yaml_file: Path to the YAML file to analyze
+
+    Returns:
+        Dictionary with keys:
+        - buses: set of common bus package names
+        - has_extend_remove: bool indicating if Extend/Remove objects are present
+        - has_direct_bus_config: bool indicating if buses are defined directly (not via packages)
+        - loaded: bool indicating if file was successfully loaded
+    """
+    result = {
+        "buses": set(),
+        "has_extend_remove": False,
+        "has_direct_bus_config": False,
+        "loaded": False,
+    }
+
+    if not yaml_file.exists():
+        return result
+
+    try:
+        data = yaml_util.load_yaml(yaml_file)
+        result["loaded"] = True
+    except Exception:  # pylint: disable=broad-exception-caught
+        return result
+
+    # Check for Extend/Remove objects
+    result["has_extend_remove"] = _contains_extend_or_remove(data)
+
+    # Check if buses are defined directly (not via packages)
+    # Components that define i2c, spi, uart, or modbus directly in test files
+    # cannot be grouped because they create unique bus IDs
+    if isinstance(data, dict):
+        for bus_type in DIRECT_BUS_TYPES:
+            if bus_type in data:
+                result["has_direct_bus_config"] = True
+                break
+
+    # Extract common bus packages
+    if not isinstance(data, dict) or "packages" not in data:
+        return result
+
+    packages = data["packages"]
+    if not isinstance(packages, dict):
+        return result
+
+    valid_buses = get_common_bus_packages()
+    for pkg_name in packages:
+        if pkg_name not in valid_buses:
+            continue
+        result["buses"].add(pkg_name)
+        # Add any package dependencies (e.g., modbus includes uart)
+        if pkg_name not in PACKAGE_DEPENDENCIES:
+            continue
+        for dep in PACKAGE_DEPENDENCIES[pkg_name]:
+            if dep not in valid_buses:
+                continue
+            result["buses"].add(dep)
+
+    return result
+
+
+def analyze_component(component_dir: Path) -> tuple[dict[str, list[str]], bool, bool]:
+    """Analyze a component directory to find which buses each platform uses.
+
+    Args:
+        component_dir: Path to the component's test directory
+
+    Returns:
+        Tuple of:
+        - Dictionary mapping platform to list of bus configs
+          Example: {"esp32-ard": ["i2c", "spi"], "esp32-idf": ["i2c"]}
+        - Boolean indicating if component uses !extend or !remove
+        - Boolean indicating if component defines buses directly (not via packages)
+    """
+    if not component_dir.is_dir():
+        return {}, False, False
+
+    platform_buses = {}
+    has_extend_remove = False
+    has_direct_bus_config = False
+
+    # Analyze all YAML files in the component directory
+    for yaml_file in component_dir.glob("*.yaml"):
+        analysis = analyze_yaml_file(yaml_file)
+
+        # Track if any file uses extend/remove
+        if analysis["has_extend_remove"]:
+            has_extend_remove = True
+
+        # Track if any file defines buses directly
+        if analysis["has_direct_bus_config"]:
+            has_direct_bus_config = True
+
+        # For test.*.yaml files, extract platform and buses
+        if yaml_file.name.startswith("test.") and yaml_file.suffix == ".yaml":
+            # Extract platform name (e.g., test.esp32-ard.yaml -> esp32-ard)
+            platform = yaml_file.stem.replace("test.", "")
+            # Always add platform, even if it has no buses (empty list)
+            # This allows grouping components that don't use any shared buses
+            platform_buses[platform] = (
+                sorted(analysis["buses"]) if analysis["buses"] else []
+            )
+
+    return platform_buses, has_extend_remove, has_direct_bus_config
+
+
+def analyze_all_components(
+    tests_dir: Path = None,
+) -> tuple[dict[str, dict[str, list[str]]], set[str], set[str]]:
+    """Analyze all component test directories.
+
+    Args:
+        tests_dir: Path to tests/components directory (defaults to auto-detect)
+
+    Returns:
+        Tuple of:
+        - Dictionary mapping component name to platform->buses mapping
+        - Set of component names that cannot be grouped
+        - Set of component names that define buses directly (need migration warning)
+    """
+    if tests_dir is None:
+        tests_dir = Path("tests/components")
+
+    if not tests_dir.exists():
+        print(f"Error: {tests_dir} does not exist", file=sys.stderr)
+        return {}, set(), set()
+
+    components = {}
+    non_groupable = set()
+    direct_bus_components = set()
+
+    for component_dir in sorted(tests_dir.iterdir()):
+        if not component_dir.is_dir():
+            continue
+
+        component_name = component_dir.name
+        platform_buses, has_extend_remove, has_direct_bus_config = analyze_component(
+            component_dir
+        )
+
+        if platform_buses:
+            components[component_name] = platform_buses
+
+        # Note: Components using $component_dir are now groupable because the merge
+        # script rewrites these to absolute paths with component-specific substitutions
+
+        # Check if component is explicitly isolated
+        # These have known issues that prevent grouping with other components
+        if component_name in ISOLATED_COMPONENTS:
+            non_groupable.add(component_name)
+
+        # Check if component is a base bus component
+        # These ARE the bus platform implementations and define buses directly for testing
+        # They cannot be grouped with components that use bus packages (causes ID conflicts)
+        if component_name in BASE_BUS_COMPONENTS:
+            non_groupable.add(component_name)
+
+        # Check if component uses !extend or !remove directives
+        # These rely on specific config structure and cannot be merged with other components
+        # The directives work within a component's own package hierarchy but break when
+        # merging independent components together
+        if has_extend_remove:
+            non_groupable.add(component_name)
+
+        # Check if component defines buses directly in test files
+        # These create unique bus IDs and cause conflicts when merged
+        # Exclude base bus components (i2c, spi, uart, etc.) since they ARE the platform
+        if has_direct_bus_config and component_name not in BASE_BUS_COMPONENTS:
+            non_groupable.add(component_name)
+            direct_bus_components.add(component_name)
+
+    return components, non_groupable, direct_bus_components
+
+
+@lru_cache(maxsize=256)
+def _get_bus_configs(buses: tuple[str, ...]) -> frozenset[tuple[str, str]]:
+    """Map bus type to set of configs for that type.
+
+    Args:
+        buses: Tuple of bus package names (e.g., ("uart_9600", "i2c"))
+
+    Returns:
+        Frozenset of (base_type, full_config) tuples
+        Example: frozenset({("uart", "uart_9600"), ("i2c", "i2c")})
+    """
+    # Split on underscore to get base type: "uart_9600" -> "uart", "i2c" -> "i2c"
+    return frozenset((bus.split("_", 1)[0], bus) for bus in buses)
+
+
+@lru_cache(maxsize=1024)
+def are_buses_compatible(buses1: tuple[str, ...], buses2: tuple[str, ...]) -> bool:
+    """Check if two bus tuples are compatible for merging.
+
+    Two bus lists are compatible if they don't have conflicting configurations
+    for the same bus type. For example:
+    - ("ble", "uart") and ("i2c",) are compatible (different buses)
+    - ("uart_9600",) and ("uart_19200",) are NOT compatible (same bus, different configs)
+    - ("uart_9600",) and ("uart_9600",) are compatible (same bus, same config)
+
+    Args:
+        buses1: First tuple of bus package names
+        buses2: Second tuple of bus package names
+
+    Returns:
+        True if buses can be merged without conflicts
+    """
+    configs1 = _get_bus_configs(buses1)
+    configs2 = _get_bus_configs(buses2)
+
+    # Group configs by base type
+    bus_types1: dict[str, set[str]] = {}
+    for base_type, full_config in configs1:
+        if base_type not in bus_types1:
+            bus_types1[base_type] = set()
+        bus_types1[base_type].add(full_config)
+
+    bus_types2: dict[str, set[str]] = {}
+    for base_type, full_config in configs2:
+        if base_type not in bus_types2:
+            bus_types2[base_type] = set()
+        bus_types2[base_type].add(full_config)
+
+    # Check for conflicts: same bus type with different configs
+    for bus_type, configs in bus_types1.items():
+        if bus_type not in bus_types2:
+            continue  # No conflict - different bus types
+        # Same bus type - check if configs match
+        if configs != bus_types2[bus_type]:
+            return False  # Conflict - same bus type, different configs
+
+    return True  # No conflicts found
+
+
+def merge_compatible_bus_groups(
+    grouped_components: dict[tuple[str, str], list[str]],
+) -> dict[tuple[str, str], list[str]]:
+    """Merge groups with compatible (non-conflicting) buses.
+
+    This function takes groups keyed by (platform, bus_signature) and merges
+    groups that share the same platform and have compatible bus configurations.
+    Two groups can be merged if their buses don't conflict - meaning they don't
+    have different configurations for the same bus type.
+
+    For example:
+    - ["ble"] + ["uart"] = compatible (different buses)
+    - ["uart_9600"] + ["uart_19200"] = incompatible (same bus, different configs)
+    - ["uart_9600"] + ["uart_9600"] = compatible (same bus, same config)
+
+    Args:
+        grouped_components: Dictionary mapping (platform, signature) to list of component names
+
+    Returns:
+        Dictionary with same structure but with compatible groups merged
+    """
+    merged_groups: dict[tuple[str, str], list[str]] = {}
+    processed_keys: set[tuple[str, str]] = set()
+
+    for (platform1, sig1), comps1 in sorted(grouped_components.items()):
+        if (platform1, sig1) in processed_keys:
+            continue
+
+        # Skip NO_BUSES_SIGNATURE - kept separate for flexible batch distribution
+        # These components have no bus requirements and can be added to any batch
+        # as "fillers" for load balancing across CI runners
+        if sig1 == NO_BUSES_SIGNATURE:
+            merged_groups[(platform1, sig1)] = comps1
+            processed_keys.add((platform1, sig1))
+            continue
+
+        # Skip isolated components - they can't be merged with others
+        if sig1.startswith(ISOLATED_SIGNATURE_PREFIX):
+            merged_groups[(platform1, sig1)] = comps1
+            processed_keys.add((platform1, sig1))
+            continue
+
+        # Start with this group's components
+        merged_comps: list[str] = list(comps1)
+        merged_sig: str = sig1
+        processed_keys.add((platform1, sig1))
+
+        # Get buses for this group as tuple for caching
+        buses1: tuple[str, ...] = tuple(sorted(sig1.split("+")))
+
+        # Try to merge with other groups on same platform
+        for (platform2, sig2), comps2 in sorted(grouped_components.items()):
+            if (platform2, sig2) in processed_keys:
+                continue
+            if platform2 != platform1:
+                continue  # Different platforms can't be merged
+            if sig2 == NO_BUSES_SIGNATURE:
+                continue  # Keep separate for flexible batch distribution
+            if sig2.startswith(ISOLATED_SIGNATURE_PREFIX):
+                continue  # Isolated components can't be merged
+
+            # Check if buses are compatible
+            buses2: tuple[str, ...] = tuple(sorted(sig2.split("+")))
+            if are_buses_compatible(buses1, buses2):
+                # Compatible! Merge this group
+                merged_comps.extend(comps2)
+                processed_keys.add((platform2, sig2))
+                # Update merged signature to include all unique buses
+                all_buses: set[str] = set(buses1) | set(buses2)
+                merged_sig = "+".join(sorted(all_buses))
+                buses1 = tuple(sorted(all_buses))  # Update for next iteration
+
+        # Store merged group
+        merged_groups[(platform1, merged_sig)] = merged_comps
+
+    return merged_groups
+
+
+def create_grouping_signature(
+    platform_buses: dict[str, list[str]], platform: str
+) -> str:
+    """Create a signature string for grouping components.
+
+    Components with the same signature can be grouped together for testing.
+    All valid bus types can be grouped since --testing-mode bypasses runtime
+    conflicts during config validation.
+
+    Args:
+        platform_buses: Mapping of platform to list of buses
+        platform: The specific platform to create signature for
+
+    Returns:
+        Signature string (e.g., "i2c" or "uart") or empty if no valid buses
+    """
+    buses = platform_buses.get(platform, [])
+    if not buses:
+        return ""
+
+    # Only include valid bus types in signature
+    common_buses = get_common_bus_packages()
+    valid_buses = [b for b in buses if b in common_buses]
+    if not valid_buses:
+        return ""
+
+    return "+".join(sorted(valid_buses))
+
+
+def group_components_by_signature(
+    components: dict[str, dict[str, list[str]]], platform: str
+) -> dict[str, list[str]]:
+    """Group components by their bus signature for a specific platform.
+
+    Args:
+        components: Component analysis results from analyze_all_components()
+        platform: Platform to group for (e.g., "esp32-ard")
+
+    Returns:
+        Dictionary mapping signature to list of component names
+        Example: {"i2c+uart_19200": ["comp1", "comp2"], "spi": ["comp3"]}
+    """
+    signature_groups: dict[str, list[str]] = {}
+
+    for component_name, platform_buses in components.items():
+        if platform not in platform_buses:
+            continue
+
+        signature = create_grouping_signature(platform_buses, platform)
+        if not signature:
+            continue
+
+        if signature not in signature_groups:
+            signature_groups[signature] = []
+        signature_groups[signature].append(component_name)
+
+    return signature_groups
+
+
+def main() -> None:
+    """Main entry point."""
+    parser = argparse.ArgumentParser(
+        description="Analyze component test files to detect common bus usage"
+    )
+    parser.add_argument(
+        "--components",
+        "-c",
+        nargs="+",
+        help="Specific components to analyze (default: all)",
+    )
+    parser.add_argument(
+        "--platform",
+        "-p",
+        help="Show grouping for a specific platform",
+    )
+    parser.add_argument(
+        "--json",
+        action="store_true",
+        help="Output as JSON",
+    )
+    parser.add_argument(
+        "--group",
+        action="store_true",
+        help="Show component groupings by bus signature",
+    )
+
+    args = parser.parse_args()
+
+    # Analyze components
+    tests_dir = Path("tests/components")
+
+    if args.components:
+        # Analyze only specified components
+        components = {}
+        non_groupable = set()
+        direct_bus_components = set()
+        for comp in args.components:
+            comp_dir = tests_dir / comp
+            platform_buses, has_extend_remove, has_direct_bus_config = (
+                analyze_component(comp_dir)
+            )
+            if platform_buses:
+                components[comp] = platform_buses
+            # Note: Components using $component_dir are now groupable
+            if comp in ISOLATED_COMPONENTS:
+                non_groupable.add(comp)
+            if comp in BASE_BUS_COMPONENTS:
+                non_groupable.add(comp)
+            if has_direct_bus_config and comp not in BASE_BUS_COMPONENTS:
+                non_groupable.add(comp)
+                direct_bus_components.add(comp)
+    else:
+        # Analyze all components
+        components, non_groupable, direct_bus_components = analyze_all_components(
+            tests_dir
+        )
+
+    # Output results
+    if args.group and args.platform:
+        # Show groupings for a specific platform
+        groups = group_components_by_signature(components, args.platform)
+
+        if args.json:
+            print(json.dumps(groups, indent=2))
+        else:
+            print(f"Component groupings for {args.platform}:")
+            print()
+            for signature, comp_list in sorted(groups.items()):
+                print(f"  {signature}:")
+                for comp in sorted(comp_list):
+                    print(f"    - {comp}")
+                print()
+    elif args.json:
+        # JSON output
+        print(json.dumps(components, indent=2))
+    else:
+        # Human-readable output
+        for component, platform_buses in sorted(components.items()):
+            non_groupable_marker = (
+                " [NON-GROUPABLE]" if component in non_groupable else ""
+            )
+            print(f"{component}{non_groupable_marker}:")
+            for platform, buses in sorted(platform_buses.items()):
+                bus_str = ", ".join(buses)
+                print(f"  {platform}: {bus_str}")
+        print()
+        print(f"Total components analyzed: {len(components)}")
+        if non_groupable:
+            print(f"Non-groupable components (use local files): {len(non_groupable)}")
+            for comp in sorted(non_groupable):
+                print(f"  - {comp}")
+
+
+if __name__ == "__main__":
+    main()
diff --git a/script/api_protobuf/api_protobuf.py b/script/api_protobuf/api_protobuf.py
index 3ae1b195e4..4936434fc2 100755
--- a/script/api_protobuf/api_protobuf.py
+++ b/script/api_protobuf/api_protobuf.py
@@ -3,16 +3,15 @@ from __future__ import annotations
 
 from abc import ABC, abstractmethod
 from enum import IntEnum
-import os
 from pathlib import Path
 import re
 from subprocess import call
 import sys
-from textwrap import dedent
 from typing import Any
 
 import aioesphomeapi.api_options_pb2 as pb
 import google.protobuf.descriptor_pb2 as descriptor
+from google.protobuf.descriptor_pb2 import FieldDescriptorProto
 
 
 class WireType(IntEnum):
@@ -62,9 +61,7 @@ def indent_list(text: str, padding: str = "  ") -> list[str]:
     """Indent each line of the given text with the specified padding."""
     lines = []
     for line in text.splitlines():
-        if line == "":
-            p = ""
-        elif line.startswith("#ifdef") or line.startswith("#endif"):
+        if line == "" or line.startswith("#ifdef") or line.startswith("#endif"):
             p = ""
         else:
             p = padding
@@ -76,6 +73,30 @@ def indent(text: str, padding: str = "  ") -> str:
     return "\n".join(indent_list(text, padding))
 
 
+def wrap_with_ifdef(content: str | list[str], ifdef: str | None) -> list[str]:
+    """Wrap content with #ifdef directives if ifdef is provided.
+
+    Args:
+        content: Single string or list of strings to wrap
+        ifdef: The ifdef condition, or None to skip wrapping
+
+    Returns:
+        List of strings with ifdef wrapping if needed
+    """
+    if not ifdef:
+        if isinstance(content, str):
+            return [content]
+        return content
+
+    result = [f"#ifdef {ifdef}"]
+    if isinstance(content, str):
+        result.append(content)
+    else:
+        result.extend(content)
+    result.append("#endif")
+    return result
+
+
 def camel_to_snake(name: str) -> str:
     # https://stackoverflow.com/a/1176023
     s1 = re.sub("(.)([A-Z][a-z]+)", r"\1_\2", name)
@@ -90,8 +111,15 @@ def force_str(force: bool) -> str:
 class TypeInfo(ABC):
     """Base class for all type information."""
 
-    def __init__(self, field: descriptor.FieldDescriptorProto) -> None:
+    def __init__(
+        self,
+        field: descriptor.FieldDescriptorProto,
+        needs_decode: bool = True,
+        needs_encode: bool = True,
+    ) -> None:
         self._field = field
+        self._needs_decode = needs_decode
+        self._needs_encode = needs_encode
 
     @property
     def default_value(self) -> str:
@@ -121,7 +149,7 @@ class TypeInfo(ABC):
     @property
     def repeated(self) -> bool:
         """Check if the field is repeated."""
-        return self._field.label == 3
+        return self._field.label == FieldDescriptorProto.LABEL_REPEATED
 
     @property
     def wire_type(self) -> WireType:
@@ -157,13 +185,7 @@ class TypeInfo(ABC):
         content = self.decode_varint
         if content is None:
             return None
-        return dedent(
-            f"""\
-        case {self.number}: {{
-          this->{self.field_name} = {content};
-          return true;
-        }}"""
-        )
+        return f"case {self.number}: this->{self.field_name} = {content}; break;"
 
     decode_varint = None
 
@@ -172,13 +194,7 @@ class TypeInfo(ABC):
         content = self.decode_length
         if content is None:
             return None
-        return dedent(
-            f"""\
-        case {self.number}: {{
-          this->{self.field_name} = {content};
-          return true;
-        }}"""
-        )
+        return f"case {self.number}: this->{self.field_name} = {content}; break;"
 
     decode_length = None
 
@@ -187,13 +203,7 @@ class TypeInfo(ABC):
         content = self.decode_32bit
         if content is None:
             return None
-        return dedent(
-            f"""\
-        case {self.number}: {{
-          this->{self.field_name} = {content};
-          return true;
-        }}"""
-        )
+        return f"case {self.number}: this->{self.field_name} = {content}; break;"
 
     decode_32bit = None
 
@@ -202,13 +212,7 @@ class TypeInfo(ABC):
         content = self.decode_64bit
         if content is None:
             return None
-        return dedent(
-            f"""\
-        case {self.number}: {{
-          this->{self.field_name} = {content};
-          return true;
-        }}"""
-        )
+        return f"case {self.number}: this->{self.field_name} = {content}; break;"
 
     decode_64bit = None
 
@@ -218,12 +222,26 @@ class TypeInfo(ABC):
 
     encode_func = None
 
+    @classmethod
+    def can_use_dump_field(cls) -> bool:
+        """Whether this type can use the dump_field helper functions.
+
+        Returns True for simple types that have dump_field overloads.
+        Complex types like messages and bytes should return False.
+        """
+        return True
+
+    def dump_field_value(self, value: str) -> str:
+        """Get the value expression to pass to dump_field.
+
+        Most types just pass the value directly, but some (like enums) need a cast.
+        """
+        return value
+
     @property
     def dump_content(self) -> str:
-        o = f'out.append("  {self.name}: ");\n'
-        o += self.dump(f"this->{self.field_name}") + "\n"
-        o += 'out.append("\\n");\n'
-        return o
+        # Default implementation - subclasses can override if they need special handling
+        return f'dump_field(out, "{self.name}", {self.dump_field_value(f"this->{self.field_name}")});'
 
     @abstractmethod
     def dump(self, name: str) -> str:
@@ -257,33 +275,13 @@ class TypeInfo(ABC):
         Args:
             name: Field name
             force: Whether this is for a repeated field
-            base_method: Base method name (e.g., "add_int32_field")
+            base_method: Base method name (e.g., "add_int32")
             value_expr: Optional value expression (defaults to name)
         """
         field_id_size = self.calculate_field_id_size()
-        method = f"{base_method}_repeated" if force else base_method
+        method = f"{base_method}_force" if force else base_method
         value = value_expr if value_expr else name
-        return f"ProtoSize::{method}(total_size, {field_id_size}, {value});"
-
-    def _get_fixed_size_calculation(
-        self, name: str, force: bool, num_bytes: int, zero_check: str
-    ) -> str:
-        """Helper for fixed-size field calculations.
-
-        Args:
-            name: Field name
-            force: Whether this is for a repeated field
-            num_bytes: Number of bytes (4 or 8)
-            zero_check: Expression to check for zero value (e.g., "!= 0.0f")
-        """
-        field_id_size = self.calculate_field_id_size()
-        # Fixed-size repeated fields are handled differently in RepeatedTypeInfo
-        # so we should never get force=True here
-        assert not force, (
-            "Fixed-size repeated fields should be handled by RepeatedTypeInfo"
-        )
-        method = f"add_fixed_field<{num_bytes}>"
-        return f"ProtoSize::{method}(total_size, {field_id_size}, {name} {zero_check});"
+        return f"size.{method}({field_id_size}, {value});"
 
     @abstractmethod
     def get_size_calculation(self, name: str, force: bool = False) -> str:
@@ -313,6 +311,88 @@ class TypeInfo(ABC):
 
 TYPE_INFO: dict[int, TypeInfo] = {}
 
+# Unsupported 64-bit types that would add overhead for embedded systems
+# TYPE_DOUBLE = 1, TYPE_FIXED64 = 6, TYPE_SFIXED64 = 16, TYPE_SINT64 = 18
+UNSUPPORTED_TYPES = {1: "double", 6: "fixed64", 16: "sfixed64", 18: "sint64"}
+
+
+def validate_field_type(field_type: int, field_name: str = "") -> None:
+    """Validate that the field type is supported by ESPHome API.
+
+    Raises ValueError for unsupported 64-bit types.
+    """
+    if field_type in UNSUPPORTED_TYPES:
+        type_name = UNSUPPORTED_TYPES[field_type]
+        field_info = f" (field: {field_name})" if field_name else ""
+        raise ValueError(
+            f"64-bit type '{type_name}'{field_info} is not supported by ESPHome API. "
+            "These types add significant overhead for embedded systems. "
+            "If you need 64-bit support, please add the necessary encoding/decoding "
+            "functions to proto.h/proto.cpp first."
+        )
+
+
+def create_field_type_info(
+    field: descriptor.FieldDescriptorProto,
+    needs_decode: bool = True,
+    needs_encode: bool = True,
+) -> TypeInfo:
+    """Create the appropriate TypeInfo instance for a field, handling repeated fields and custom options."""
+    if field.label == FieldDescriptorProto.LABEL_REPEATED:
+        # Check if this repeated field has fixed_array_with_length_define option
+        if (
+            fixed_size := get_field_opt(field, pb.fixed_array_with_length_define)
+        ) is not None:
+            return FixedArrayWithLengthRepeatedType(field, fixed_size)
+        # Check if this repeated field has fixed_array_size option
+        if (fixed_size := get_field_opt(field, pb.fixed_array_size)) is not None:
+            return FixedArrayRepeatedType(field, fixed_size)
+        # Check if this repeated field has fixed_array_size_define option
+        if (
+            size_define := get_field_opt(field, pb.fixed_array_size_define)
+        ) is not None:
+            return FixedArrayRepeatedType(field, size_define)
+        return RepeatedTypeInfo(field)
+
+    # Check for mutually exclusive options on bytes fields
+    if field.type == 12:
+        has_pointer_to_buffer = get_field_opt(field, pb.pointer_to_buffer, False)
+        fixed_size = get_field_opt(field, pb.fixed_array_size, None)
+
+        if has_pointer_to_buffer and fixed_size is not None:
+            raise ValueError(
+                f"Field '{field.name}' has both pointer_to_buffer and fixed_array_size. "
+                "These options are mutually exclusive. Use pointer_to_buffer for zero-copy "
+                "or fixed_array_size for traditional array storage."
+            )
+
+        if has_pointer_to_buffer:
+            # Zero-copy pointer approach - no size needed, will use size_t for length
+            return PointerToBytesBufferType(field, None)
+
+        if fixed_size is not None:
+            # Traditional fixed array approach with copy
+            return FixedArrayBytesType(field, fixed_size)
+
+    # Check for pointer_to_buffer option on string fields
+    if field.type == 9:
+        has_pointer_to_buffer = get_field_opt(field, pb.pointer_to_buffer, False)
+
+        if has_pointer_to_buffer:
+            # Zero-copy pointer approach for strings
+            return PointerToBytesBufferType(field, None)
+
+    # Special handling for bytes fields
+    if field.type == 12:
+        return BytesType(field, needs_decode, needs_encode)
+
+    # Special handling for string fields
+    if field.type == 9:
+        return StringType(field, needs_decode, needs_encode)
+
+    validate_field_type(field.type, field.name)
+    return TYPE_INFO[field.type](field)
+
 
 def register_type(name: int):
     """Decorator to register a type with a name and number."""
@@ -339,7 +419,8 @@ class DoubleType(TypeInfo):
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_fixed_size_calculation(name, force, 8, "!= 0.0")
+        field_id_size = self.calculate_field_id_size()
+        return f"size.add_double({field_id_size}, {name});"
 
     def get_fixed_size_bytes(self) -> int:
         return 8
@@ -362,7 +443,8 @@ class FloatType(TypeInfo):
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_fixed_size_calculation(name, force, 4, "!= 0.0f")
+        field_id_size = self.calculate_field_id_size()
+        return f"size.add_float({field_id_size}, {name});"
 
     def get_fixed_size_bytes(self) -> int:
         return 4
@@ -385,7 +467,7 @@ class Int64Type(TypeInfo):
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_simple_size_calculation(name, force, "add_int64_field")
+        return self._get_simple_size_calculation(name, force, "add_int64")
 
     def get_estimated_size(self) -> int:
         return self.calculate_field_id_size() + 3  # field ID + 3 bytes typical varint
@@ -405,7 +487,7 @@ class UInt64Type(TypeInfo):
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_simple_size_calculation(name, force, "add_uint64_field")
+        return self._get_simple_size_calculation(name, force, "add_uint64")
 
     def get_estimated_size(self) -> int:
         return self.calculate_field_id_size() + 3  # field ID + 3 bytes typical varint
@@ -425,7 +507,7 @@ class Int32Type(TypeInfo):
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_simple_size_calculation(name, force, "add_int32_field")
+        return self._get_simple_size_calculation(name, force, "add_int32")
 
     def get_estimated_size(self) -> int:
         return self.calculate_field_id_size() + 3  # field ID + 3 bytes typical varint
@@ -445,7 +527,8 @@ class Fixed64Type(TypeInfo):
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_fixed_size_calculation(name, force, 8, "!= 0")
+        field_id_size = self.calculate_field_id_size()
+        return f"size.add_fixed64({field_id_size}, {name});"
 
     def get_fixed_size_bytes(self) -> int:
         return 8
@@ -468,7 +551,8 @@ class Fixed32Type(TypeInfo):
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_fixed_size_calculation(name, force, 4, "!= 0")
+        field_id_size = self.calculate_field_id_size()
+        return f"size.add_fixed32({field_id_size}, {name});"
 
     def get_fixed_size_bytes(self) -> int:
         return 4
@@ -486,11 +570,10 @@ class BoolType(TypeInfo):
     wire_type = WireType.VARINT  # Uses wire type 0
 
     def dump(self, name: str) -> str:
-        o = f"out.append(YESNO({name}));"
-        return o
+        return f"out.append(YESNO({name}));"
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_simple_size_calculation(name, force, "add_bool_field")
+        return self._get_simple_size_calculation(name, force, "add_bool")
 
     def get_estimated_size(self) -> int:
         return self.calculate_field_id_size() + 1  # field ID + 1 byte
@@ -506,12 +589,118 @@ class StringType(TypeInfo):
     encode_func = "encode_string"
     wire_type = WireType.LENGTH_DELIMITED  # Uses wire type 2
 
+    @property
+    def public_content(self) -> list[str]:
+        content: list[str] = []
+
+        # Check if no_zero_copy option is set
+        no_zero_copy = get_field_opt(self._field, pb.no_zero_copy, False)
+
+        # Add std::string storage if message needs decoding OR if no_zero_copy is set
+        if self._needs_decode or no_zero_copy:
+            content.append(f"std::string {self.field_name}{{}};")
+
+        # Only add StringRef if encoding is needed AND no_zero_copy is not set
+        if self._needs_encode and not no_zero_copy:
+            content.extend(
+                [
+                    # Add StringRef field if message needs encoding
+                    f"StringRef {self.field_name}_ref_{{}};",
+                    # Add setter method if message needs encoding
+                    f"void set_{self.field_name}(const StringRef &ref) {{",
+                    f"  this->{self.field_name}_ref_ = ref;",
+                    "}",
+                ]
+            )
+        return content
+
+    @property
+    def encode_content(self) -> str:
+        # Check if no_zero_copy option is set
+        no_zero_copy = get_field_opt(self._field, pb.no_zero_copy, False)
+
+        if no_zero_copy:
+            # Use the std::string directly
+            return f"buffer.encode_string({self.number}, this->{self.field_name});"
+        # Use the StringRef
+        return f"buffer.encode_string({self.number}, this->{self.field_name}_ref_);"
+
     def dump(self, name):
-        o = f'out.append("\'").append({name}).append("\'");'
+        # Check if no_zero_copy option is set
+        no_zero_copy = get_field_opt(self._field, pb.no_zero_copy, False)
+
+        # If name is 'it', this is a repeated field element - always use string
+        if name == "it":
+            return "append_quoted_string(out, StringRef(it));"
+
+        # If no_zero_copy is set, always use std::string
+        if no_zero_copy:
+            return f'out.append("\'").append(this->{self.field_name}).append("\'");'
+
+        # For SOURCE_CLIENT only, always use std::string
+        if not self._needs_encode:
+            return f'out.append("\'").append(this->{self.field_name}).append("\'");'
+
+        # For SOURCE_SERVER, always use StringRef
+        if not self._needs_decode:
+            return f"append_quoted_string(out, this->{self.field_name}_ref_);"
+
+        # For SOURCE_BOTH, check if StringRef is set (sending) or use string (received)
+        return (
+            f"if (!this->{self.field_name}_ref_.empty()) {{"
+            f'  out.append("\'").append(this->{self.field_name}_ref_.c_str()).append("\'");'
+            f"}} else {{"
+            f'  out.append("\'").append(this->{self.field_name}).append("\'");'
+            f"}}"
+        )
+
+    @property
+    def dump_content(self) -> str:
+        # Check if no_zero_copy option is set
+        no_zero_copy = get_field_opt(self._field, pb.no_zero_copy, False)
+
+        # If no_zero_copy is set, always use std::string
+        if no_zero_copy:
+            return f'dump_field(out, "{self.name}", this->{self.field_name});'
+
+        # For SOURCE_CLIENT only, use std::string
+        if not self._needs_encode:
+            return f'dump_field(out, "{self.name}", this->{self.field_name});'
+
+        # For SOURCE_SERVER, use StringRef with _ref_ suffix
+        if not self._needs_decode:
+            return f'dump_field(out, "{self.name}", this->{self.field_name}_ref_);'
+
+        # For SOURCE_BOTH, we need custom logic
+        o = f'out.append("  {self.name}: ");\n'
+        o += self.dump(f"this->{self.field_name}") + "\n"
+        o += 'out.append("\\n");'
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_simple_size_calculation(name, force, "add_string_field")
+        # Check if no_zero_copy option is set
+        no_zero_copy = get_field_opt(self._field, pb.no_zero_copy, False)
+
+        # For SOURCE_CLIENT only messages or no_zero_copy, use the string field directly
+        if not self._needs_encode or no_zero_copy:
+            # For no_zero_copy, we need to use .size() on the string
+            if no_zero_copy and name != "it":
+                field_id_size = self.calculate_field_id_size()
+                return (
+                    f"size.add_length({field_id_size}, this->{self.field_name}.size());"
+                )
+            return self._get_simple_size_calculation(name, force, "add_length")
+
+        # Check if this is being called from a repeated field context
+        # In that case, 'name' will be 'it' and we need to use the repeated version
+        if name == "it":
+            # For repeated fields, we need to use add_length_force which includes field ID
+            field_id_size = self.calculate_field_id_size()
+            return f"size.add_length_force({field_id_size}, it.size());"
+
+        # For messages that need encoding, use the StringRef size
+        field_id_size = self.calculate_field_id_size()
+        return f"size.add_length({field_id_size}, this->{self.field_name}_ref_.size());"
 
     def get_estimated_size(self) -> int:
         return self.calculate_field_id_size() + 8  # field ID + 8 bytes typical string
@@ -519,6 +708,10 @@ class StringType(TypeInfo):
 
 @register_type(11)
 class MessageType(TypeInfo):
+    @classmethod
+    def can_use_dump_field(cls) -> bool:
+        return False
+
     @property
     def cpp_type(self) -> str:
         return self._field.type_name[1:]
@@ -549,22 +742,24 @@ class MessageType(TypeInfo):
     @property
     def decode_length_content(self) -> str:
         # Custom decode that doesn't use templates
-        return dedent(
-            f"""\
-        case {self.number}: {{
-          value.decode_to_message(this->{self.field_name});
-          return true;
-        }}"""
-        )
+        return f"case {self.number}: value.decode_to_message(this->{self.field_name}); break;"
 
     def dump(self, name: str) -> str:
-        o = f"{name}.dump_to(out);"
+        return f"{name}.dump_to(out);"
+
+    @property
+    def dump_content(self) -> str:
+        o = f'out.append("  {self.name}: ");\n'
+        o += f"this->{self.field_name}.dump_to(out);\n"
+        o += 'out.append("\\n");'
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
         return self._get_simple_size_calculation(name, force, "add_message_object")
 
     def get_estimated_size(self) -> int:
+        # For message types, we can't easily estimate the submessage size without
+        # access to the actual message definition. This is just a rough estimate.
         return (
             self.calculate_field_id_size() + 16
         )  # field ID + 16 bytes estimated submessage
@@ -572,29 +767,255 @@ class MessageType(TypeInfo):
 
 @register_type(12)
 class BytesType(TypeInfo):
+    @classmethod
+    def can_use_dump_field(cls) -> bool:
+        return False
+
     cpp_type = "std::string"
     default_value = ""
     reference_type = "std::string &"
     const_reference_type = "const std::string &"
-    decode_length = "value.as_string()"
     encode_func = "encode_bytes"
+    decode_length = "value.as_string()"
     wire_type = WireType.LENGTH_DELIMITED  # Uses wire type 2
 
+    @property
+    def public_content(self) -> list[str]:
+        content: list[str] = []
+        # Add std::string storage if message needs decoding
+        if self._needs_decode:
+            content.append(f"std::string {self.field_name}{{}};")
+
+        if self._needs_encode:
+            content.extend(
+                [
+                    # Add pointer/length fields if message needs encoding
+                    f"const uint8_t* {self.field_name}_ptr_{{nullptr}};",
+                    f"size_t {self.field_name}_len_{{0}};",
+                    # Add setter method if message needs encoding
+                    f"void set_{self.field_name}(const uint8_t* data, size_t len) {{",
+                    f"  this->{self.field_name}_ptr_ = data;",
+                    f"  this->{self.field_name}_len_ = len;",
+                    "}",
+                ]
+            )
+        return content
+
     @property
     def encode_content(self) -> str:
-        return f"buffer.encode_bytes({self.number}, reinterpret_cast(this->{self.field_name}.data()), this->{self.field_name}.size());"
+        return f"buffer.encode_bytes({self.number}, this->{self.field_name}_ptr_, this->{self.field_name}_len_);"
 
     def dump(self, name: str) -> str:
-        o = f"out.append(format_hex_pretty({name}));"
+        ptr_dump = f"format_hex_pretty(this->{self.field_name}_ptr_, this->{self.field_name}_len_)"
+        str_dump = f"format_hex_pretty(reinterpret_cast(this->{self.field_name}.data()), this->{self.field_name}.size())"
+
+        # For SOURCE_CLIENT only, always use std::string
+        if not self._needs_encode:
+            return f"out.append({str_dump});"
+
+        # For SOURCE_SERVER, always use pointer/length
+        if not self._needs_decode:
+            return f"out.append({ptr_dump});"
+
+        # For SOURCE_BOTH, check if pointer is set (sending) or use string (received)
+        return (
+            f"if (this->{self.field_name}_ptr_ != nullptr) {{\n"
+            f"    out.append({ptr_dump});\n"
+            f"  }} else {{\n"
+            f"    out.append({str_dump});\n"
+            f"  }}"
+        )
+
+    @property
+    def dump_content(self) -> str:
+        o = f'out.append("  {self.name}: ");\n'
+        o += self.dump(f"this->{self.field_name}") + "\n"
+        o += 'out.append("\\n");'
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_simple_size_calculation(name, force, "add_string_field")
+        return f"size.add_length({self.calculate_field_id_size()}, this->{self.field_name}_len_);"
 
     def get_estimated_size(self) -> int:
         return self.calculate_field_id_size() + 8  # field ID + 8 bytes typical bytes
 
 
+class PointerToBytesBufferType(TypeInfo):
+    """Type for bytes fields that use pointer_to_buffer option for zero-copy."""
+
+    @classmethod
+    def can_use_dump_field(cls) -> bool:
+        return False
+
+    def __init__(
+        self, field: descriptor.FieldDescriptorProto, size: int | None = None
+    ) -> None:
+        super().__init__(field)
+        # Size is not used for pointer_to_buffer - we always use size_t for length
+        self.array_size = 0
+
+    @property
+    def cpp_type(self) -> str:
+        return "const uint8_t*"
+
+    @property
+    def default_value(self) -> str:
+        return "nullptr"
+
+    @property
+    def reference_type(self) -> str:
+        return "const uint8_t*"
+
+    @property
+    def const_reference_type(self) -> str:
+        return "const uint8_t*"
+
+    @property
+    def public_content(self) -> list[str]:
+        # Use uint16_t for length - max packet size is well below 65535
+        # Add pointer and length fields
+        return [
+            f"const uint8_t* {self.field_name}{{nullptr}};",
+            f"uint16_t {self.field_name}_len{{0}};",
+        ]
+
+    @property
+    def encode_content(self) -> str:
+        return f"buffer.encode_bytes({self.number}, this->{self.field_name}, this->{self.field_name}_len);"
+
+    @property
+    def decode_length_content(self) -> str | None:
+        # Decode directly stores the pointer to avoid allocation
+        return f"""case {self.number}: {{
+      // Use raw data directly to avoid allocation
+      this->{self.field_name} = value.data();
+      this->{self.field_name}_len = value.size();
+      break;
+    }}"""
+
+    @property
+    def decode_length(self) -> str | None:
+        # This is handled in decode_length_content
+        return None
+
+    @property
+    def wire_type(self) -> WireType:
+        """Get the wire type for this bytes field."""
+        return WireType.LENGTH_DELIMITED  # Uses wire type 2
+
+    def dump(self, name: str) -> str:
+        return (
+            f"format_hex_pretty(this->{self.field_name}, this->{self.field_name}_len)"
+        )
+
+    @property
+    def dump_content(self) -> str:
+        # Custom dump that doesn't use dump_field template
+        return (
+            f'out.append("  {self.name}: ");\n'
+            + f"out.append({self.dump(self.field_name)});\n"
+            + 'out.append("\\n");'
+        )
+
+    def get_size_calculation(self, name: str, force: bool = False) -> str:
+        return f"size.add_length({self.number}, this->{self.field_name}_len);"
+
+    def get_estimated_size(self) -> int:
+        # field ID + length varint + typical data (assume small for pointer fields)
+        return self.calculate_field_id_size() + 2 + 16
+
+
+class FixedArrayBytesType(TypeInfo):
+    """Special type for fixed-size byte arrays."""
+
+    @classmethod
+    def can_use_dump_field(cls) -> bool:
+        return False
+
+    def __init__(self, field: descriptor.FieldDescriptorProto, size: int) -> None:
+        super().__init__(field)
+        self.array_size = size
+
+    @property
+    def cpp_type(self) -> str:
+        return "uint8_t"
+
+    @property
+    def default_value(self) -> str:
+        return "{}"
+
+    @property
+    def reference_type(self) -> str:
+        return f"uint8_t (&)[{self.array_size}]"
+
+    @property
+    def const_reference_type(self) -> str:
+        return f"const uint8_t (&)[{self.array_size}]"
+
+    @property
+    def public_content(self) -> list[str]:
+        len_type = (
+            "uint8_t"
+            if self.array_size <= 255
+            else "uint16_t"
+            if self.array_size <= 65535
+            else "size_t"
+        )
+        # Add both the array and length fields
+        return [
+            f"uint8_t {self.field_name}[{self.array_size}]{{}};",
+            f"{len_type} {self.field_name}_len{{0}};",
+        ]
+
+    @property
+    def decode_length_content(self) -> str:
+        o = f"case {self.number}: {{\n"
+        o += "  const std::string &data_str = value.as_string();\n"
+        o += f"  this->{self.field_name}_len = data_str.size();\n"
+        o += f"  if (this->{self.field_name}_len > {self.array_size}) {{\n"
+        o += f"    this->{self.field_name}_len = {self.array_size};\n"
+        o += "  }\n"
+        o += f"  memcpy(this->{self.field_name}, data_str.data(), this->{self.field_name}_len);\n"
+        o += "  break;\n"
+        o += "}"
+        return o
+
+    @property
+    def encode_content(self) -> str:
+        return f"buffer.encode_bytes({self.number}, this->{self.field_name}, this->{self.field_name}_len);"
+
+    def dump(self, name: str) -> str:
+        return f"out.append(format_hex_pretty({name}, {name}_len));"
+
+    @property
+    def dump_content(self) -> str:
+        o = f'out.append("  {self.name}: ");\n'
+        o += f"out.append(format_hex_pretty(this->{self.field_name}, this->{self.field_name}_len));\n"
+        o += 'out.append("\\n");'
+        return o
+
+    def get_size_calculation(self, name: str, force: bool = False) -> str:
+        # Use the actual length stored in the _len field
+        length_field = f"this->{self.field_name}_len"
+        field_id_size = self.calculate_field_id_size()
+
+        if force:
+            # For repeated fields, always calculate size (no zero check)
+            return f"size.add_length_force({field_id_size}, {length_field});"
+        # For non-repeated fields, add_length already checks for zero
+        return f"size.add_length({field_id_size}, {length_field});"
+
+    def get_estimated_size(self) -> int:
+        # Estimate based on typical BLE advertisement size
+        return (
+            self.calculate_field_id_size() + 1 + 31
+        )  # field ID + length byte + typical 31 bytes
+
+    @property
+    def wire_type(self) -> WireType:
+        return WireType.LENGTH_DELIMITED
+
+
 @register_type(13)
 class UInt32Type(TypeInfo):
     cpp_type = "uint32_t"
@@ -609,7 +1030,7 @@ class UInt32Type(TypeInfo):
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_simple_size_calculation(name, force, "add_uint32_field")
+        return self._get_simple_size_calculation(name, force, "add_uint32")
 
     def get_estimated_size(self) -> int:
         return self.calculate_field_id_size() + 3  # field ID + 3 bytes typical varint
@@ -637,12 +1058,15 @@ class EnumType(TypeInfo):
         return f"buffer.{self.encode_func}({self.number}, static_cast(this->{self.field_name}));"
 
     def dump(self, name: str) -> str:
-        o = f"out.append(proto_enum_to_string<{self.cpp_type}>({name}));"
-        return o
+        return f"out.append(proto_enum_to_string<{self.cpp_type}>({name}));"
+
+    def dump_field_value(self, value: str) -> str:
+        # Enums need explicit cast for the template
+        return f"static_cast<{self.cpp_type}>({value})"
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
         return self._get_simple_size_calculation(
-            name, force, "add_enum_field", f"static_cast({name})"
+            name, force, "add_uint32", f"static_cast({name})"
         )
 
     def get_estimated_size(self) -> int:
@@ -663,7 +1087,8 @@ class SFixed32Type(TypeInfo):
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_fixed_size_calculation(name, force, 4, "!= 0")
+        field_id_size = self.calculate_field_id_size()
+        return f"size.add_sfixed32({field_id_size}, {name});"
 
     def get_fixed_size_bytes(self) -> int:
         return 4
@@ -686,7 +1111,8 @@ class SFixed64Type(TypeInfo):
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_fixed_size_calculation(name, force, 8, "!= 0")
+        field_id_size = self.calculate_field_id_size()
+        return f"size.add_sfixed64({field_id_size}, {name});"
 
     def get_fixed_size_bytes(self) -> int:
         return 8
@@ -709,7 +1135,7 @@ class SInt32Type(TypeInfo):
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_simple_size_calculation(name, force, "add_sint32_field")
+        return self._get_simple_size_calculation(name, force, "add_sint32")
 
     def get_estimated_size(self) -> int:
         return self.calculate_field_id_size() + 3  # field ID + 3 bytes typical varint
@@ -729,19 +1155,294 @@ class SInt64Type(TypeInfo):
         return o
 
     def get_size_calculation(self, name: str, force: bool = False) -> str:
-        return self._get_simple_size_calculation(name, force, "add_sint64_field")
+        return self._get_simple_size_calculation(name, force, "add_sint64")
 
     def get_estimated_size(self) -> int:
         return self.calculate_field_id_size() + 3  # field ID + 3 bytes typical varint
 
 
+def _generate_array_dump_content(
+    ti, field_name: str, name: str, is_bool: bool = False
+) -> str:
+    """Generate dump content for array types (repeated or fixed array).
+
+    Shared helper to avoid code duplication between RepeatedTypeInfo and FixedArrayRepeatedType.
+    """
+    o = f"for (const auto {'' if is_bool else '&'}it : {field_name}) {{\n"
+    # Check if underlying type can use dump_field
+    if ti.can_use_dump_field():
+        # For types that have dump_field overloads, use them with extra indent
+        # std::vector iterators return proxy objects, need explicit cast
+        value_expr = "static_cast(it)" if is_bool else ti.dump_field_value("it")
+        o += f'  dump_field(out, "{name}", {value_expr}, 4);\n'
+    else:
+        # For complex types (messages, bytes), use the old pattern
+        o += f'  out.append("  {name}: ");\n'
+        o += indent(ti.dump("it")) + "\n"
+        o += '  out.append("\\n");\n'
+    o += "}"
+    return o
+
+
+class FixedArrayRepeatedType(TypeInfo):
+    """Special type for fixed-size repeated fields using std::array.
+
+    Fixed arrays are only supported for encoding (SOURCE_SERVER) since we cannot
+    control how many items we receive when decoding.
+    """
+
+    def __init__(self, field: descriptor.FieldDescriptorProto, size: int | str) -> None:
+        super().__init__(field)
+        self.array_size = size
+        self.is_define = isinstance(size, str)
+        # Check if we should skip encoding when all elements are zero
+        # Use getattr to handle older versions of api_options_pb2
+        self.skip_zero = get_field_opt(
+            field, getattr(pb, "fixed_array_skip_zero", None), False
+        )
+        # Create the element type info
+        validate_field_type(field.type, field.name)
+        self._ti: TypeInfo = TYPE_INFO[field.type](field)
+
+    def _encode_element(self, element: str) -> str:
+        """Helper to generate encode statement for a single element."""
+        if isinstance(self._ti, EnumType):
+            return f"buffer.{self._ti.encode_func}({self.number}, static_cast({element}), true);"
+        return f"buffer.{self._ti.encode_func}({self.number}, {element}, true);"
+
+    @property
+    def cpp_type(self) -> str:
+        return f"std::array<{self._ti.cpp_type}, {self.array_size}>"
+
+    @property
+    def reference_type(self) -> str:
+        return f"{self.cpp_type} &"
+
+    @property
+    def const_reference_type(self) -> str:
+        return f"const {self.cpp_type} &"
+
+    @property
+    def wire_type(self) -> WireType:
+        """Get the wire type for this fixed array field."""
+        return self._ti.wire_type
+
+    @property
+    def public_content(self) -> list[str]:
+        # Just the array member, no index needed since we don't decode
+        return [f"{self.cpp_type} {self.field_name}{{}};"]
+
+    # No decode methods needed - fixed arrays don't support decoding
+    # The base class TypeInfo already returns None for all decode properties
+
+    @property
+    def encode_content(self) -> str:
+        # If skip_zero is enabled, wrap encoding in a zero check
+        if self.skip_zero:
+            if self.is_define:
+                # When using a define, we need to use a loop-based approach
+                o = f"for (const auto &it : this->{self.field_name}) {{\n"
+                o += "  if (it != 0) {\n"
+                o += f"    {self._encode_element('it')}\n"
+                o += "  }\n"
+                o += "}"
+                return o
+            # Build the condition to check if at least one element is non-zero
+            non_zero_checks = " || ".join(
+                [f"this->{self.field_name}[{i}] != 0" for i in range(self.array_size)]
+            )
+            encode_lines = [
+                f"  {self._encode_element(f'this->{self.field_name}[{i}]')}"
+                for i in range(self.array_size)
+            ]
+            return f"if ({non_zero_checks}) {{\n" + "\n".join(encode_lines) + "\n}"
+
+        # When using a define, always use loop-based approach
+        if self.is_define:
+            o = f"for (const auto &it : this->{self.field_name}) {{\n"
+            o += f"  {self._encode_element('it')}\n"
+            o += "}"
+            return o
+
+        # Unroll small arrays for efficiency
+        if self.array_size == 1:
+            return self._encode_element(f"this->{self.field_name}[0]")
+        if self.array_size == 2:
+            return (
+                self._encode_element(f"this->{self.field_name}[0]")
+                + "\n  "
+                + self._encode_element(f"this->{self.field_name}[1]")
+            )
+
+        # Use loops for larger arrays
+        o = f"for (const auto &it : this->{self.field_name}) {{\n"
+        o += f"  {self._encode_element('it')}\n"
+        o += "}"
+        return o
+
+    @property
+    def dump_content(self) -> str:
+        return _generate_array_dump_content(
+            self._ti, f"this->{self.field_name}", self.name, is_bool=False
+        )
+
+    def dump(self, name: str) -> str:
+        # This is used when dumping the array itself (not its elements)
+        # Since dump_content handles the iteration, this is not used directly
+        return ""
+
+    def get_size_calculation(self, name: str, force: bool = False) -> str:
+        # If skip_zero is enabled, wrap size calculation in a zero check
+        if self.skip_zero:
+            if self.is_define:
+                # When using a define, we need to use a loop-based approach
+                o = f"for (const auto &it : {name}) {{\n"
+                o += "  if (it != 0) {\n"
+                o += f"    {self._ti.get_size_calculation('it', True)}\n"
+                o += "  }\n"
+                o += "}"
+                return o
+            # Build the condition to check if at least one element is non-zero
+            non_zero_checks = " || ".join(
+                [f"{name}[{i}] != 0" for i in range(self.array_size)]
+            )
+            size_lines = [
+                f"  {self._ti.get_size_calculation(f'{name}[{i}]', True)}"
+                for i in range(self.array_size)
+            ]
+            return f"if ({non_zero_checks}) {{\n" + "\n".join(size_lines) + "\n}"
+
+        # When using a define, always use loop-based approach
+        if self.is_define:
+            o = f"for (const auto &it : {name}) {{\n"
+            o += f"  {self._ti.get_size_calculation('it', True)}\n"
+            o += "}"
+            return o
+
+        # For fixed arrays, we always encode all elements
+
+        # Special case for single-element arrays - no loop needed
+        if self.array_size == 1:
+            return self._ti.get_size_calculation(f"{name}[0]", True)
+
+        # Special case for 2-element arrays - unroll the calculation
+        if self.array_size == 2:
+            return (
+                self._ti.get_size_calculation(f"{name}[0]", True)
+                + "\n  "
+                + self._ti.get_size_calculation(f"{name}[1]", True)
+            )
+
+        # Use loops for larger arrays
+        o = f"for (const auto &it : {name}) {{\n"
+        o += f"  {self._ti.get_size_calculation('it', True)}\n"
+        o += "}"
+        return o
+
+    def get_estimated_size(self) -> int:
+        # For fixed arrays, estimate underlying type size * array size
+        underlying_size = self._ti.get_estimated_size()
+        if self.is_define:
+            # When using a define, we don't know the actual size so just guess 3
+            # This is only used for documentation and never actually used since
+            # fixed arrays are only for SOURCE_SERVER (encode-only) messages
+            return underlying_size * 3
+        return underlying_size * self.array_size
+
+
+class FixedArrayWithLengthRepeatedType(FixedArrayRepeatedType):
+    """Special type for fixed-size repeated fields with variable length tracking.
+
+    Similar to FixedArrayRepeatedType but generates an additional length field
+    to track how many elements are actually in use. Only encodes/sends elements
+    up to the current length.
+
+    Fixed arrays with length are only supported for encoding (SOURCE_SERVER) since
+    we cannot control how many items we receive when decoding.
+    """
+
+    @property
+    def public_content(self) -> list[str]:
+        # Return both the array and the length field
+        return [
+            f"{self.cpp_type} {self.field_name}{{}};",
+            f"uint16_t {self.field_name}_len{{0}};",
+        ]
+
+    @property
+    def encode_content(self) -> str:
+        # Always use a loop up to the current length
+        o = f"for (uint16_t i = 0; i < this->{self.field_name}_len; i++) {{\n"
+        o += f"  {self._encode_element(f'this->{self.field_name}[i]')}\n"
+        o += "}"
+        return o
+
+    @property
+    def dump_content(self) -> str:
+        # Dump only the active elements
+        o = f"for (uint16_t i = 0; i < this->{self.field_name}_len; i++) {{\n"
+        # Check if underlying type can use dump_field
+        if self._ti.can_use_dump_field():
+            o += f'  dump_field(out, "{self.name}", {self._ti.dump_field_value(f"this->{self.field_name}[i]")}, 4);\n'
+        else:
+            o += f'  out.append("  {self.name}: ");\n'
+            o += indent(self._ti.dump(f"this->{self.field_name}[i]")) + "\n"
+            o += '  out.append("\\n");\n'
+        o += "}"
+        return o
+
+    def get_size_calculation(self, name: str, force: bool = False) -> str:
+        # Calculate size only for active elements
+        o = f"for (uint16_t i = 0; i < {name}_len; i++) {{\n"
+        o += f"  {self._ti.get_size_calculation(f'{name}[i]', True)}\n"
+        o += "}"
+        return o
+
+    def get_estimated_size(self) -> int:
+        # For fixed arrays with length, estimate based on typical usage
+        # Assume on average half the array is used
+        underlying_size = self._ti.get_estimated_size()
+        if self.is_define:
+            # When using a define, estimate 8 elements as typical
+            return underlying_size * 8
+        return underlying_size * (
+            self.array_size // 2 if self.array_size > 2 else self.array_size
+        )
+
+
 class RepeatedTypeInfo(TypeInfo):
     def __init__(self, field: descriptor.FieldDescriptorProto) -> None:
         super().__init__(field)
+        # Check if this is a pointer field by looking for container_pointer option
+        self._container_type = get_field_opt(field, pb.container_pointer, "")
+        self._use_pointer = bool(self._container_type)
+        # Check if this should use FixedVector instead of std::vector
+        self._use_fixed_vector = get_field_opt(field, pb.fixed_vector, False)
+
+        # For repeated fields, we need to get the base type info
+        # but we can't call create_field_type_info as it would cause recursion
+        # So we extract just the type creation logic
+        if (
+            field.type == 12
+            and (fixed_size := get_field_opt(field, pb.fixed_array_size)) is not None
+        ):
+            self._ti: TypeInfo = FixedArrayBytesType(field, fixed_size)
+            return
+
+        validate_field_type(field.type, field.name)
         self._ti: TypeInfo = TYPE_INFO[field.type](field)
 
     @property
     def cpp_type(self) -> str:
+        if self._use_pointer and self._container_type:
+            # For pointer fields, use the specified container type
+            # If the container type already includes the element type (e.g., std::set)
+            # use it as-is, otherwise append the element type
+            if "<" in self._container_type and ">" in self._container_type:
+                return f"const {self._container_type}*"
+            return f"const {self._container_type}<{self._ti.cpp_type}>*"
+        if self._use_fixed_vector:
+            return f"FixedVector<{self._ti.cpp_type}>"
         return f"std::vector<{self._ti.cpp_type}>"
 
     @property
@@ -762,64 +1463,53 @@ class RepeatedTypeInfo(TypeInfo):
 
     @property
     def decode_varint_content(self) -> str:
+        # Pointer fields don't support decoding
+        if self._use_pointer:
+            return None
         content = self._ti.decode_varint
         if content is None:
             return None
-        return dedent(
-            f"""\
-        case {self.number}: {{
-          this->{self.field_name}.push_back({content});
-          return true;
-        }}"""
+        return (
+            f"case {self.number}: this->{self.field_name}.push_back({content}); break;"
         )
 
     @property
     def decode_length_content(self) -> str:
+        # Pointer fields don't support decoding
+        if self._use_pointer:
+            return None
         content = self._ti.decode_length
         if content is None and isinstance(self._ti, MessageType):
             # Special handling for non-template message decoding
-            return dedent(
-                f"""\
-        case {self.number}: {{
-          this->{self.field_name}.emplace_back();
-          value.decode_to_message(this->{self.field_name}.back());
-          return true;
-        }}"""
-            )
+            return f"case {self.number}: this->{self.field_name}.emplace_back(); value.decode_to_message(this->{self.field_name}.back()); break;"
         if content is None:
             return None
-        return dedent(
-            f"""\
-        case {self.number}: {{
-          this->{self.field_name}.push_back({content});
-          return true;
-        }}"""
+        return (
+            f"case {self.number}: this->{self.field_name}.push_back({content}); break;"
         )
 
     @property
     def decode_32bit_content(self) -> str:
+        # Pointer fields don't support decoding
+        if self._use_pointer:
+            return None
         content = self._ti.decode_32bit
         if content is None:
             return None
-        return dedent(
-            f"""\
-        case {self.number}: {{
-          this->{self.field_name}.push_back({content});
-          return true;
-        }}"""
+        return (
+            f"case {self.number}: this->{self.field_name}.push_back({content}); break;"
         )
 
     @property
     def decode_64bit_content(self) -> str:
+        # Pointer fields don't support decoding
+        if self._use_pointer:
+            return None
         content = self._ti.decode_64bit
         if content is None:
             return None
-        return dedent(
-            f"""\
-        case {self.number}: {{
-          this->{self.field_name}.push_back({content});
-          return true;
-        }}"""
+        return (
+            f"case {self.number}: this->{self.field_name}.push_back({content}); break;"
         )
 
     @property
@@ -829,6 +1519,15 @@ class RepeatedTypeInfo(TypeInfo):
 
     @property
     def encode_content(self) -> str:
+        if self._use_pointer:
+            # For pointer fields, just dereference (pointer should never be null in our use case)
+            o = f"for (const auto &it : *this->{self.field_name}) {{\n"
+            if isinstance(self._ti, EnumType):
+                o += f"  buffer.{self._ti.encode_func}({self.number}, static_cast(it), true);\n"
+            else:
+                o += f"  buffer.{self._ti.encode_func}({self.number}, it, true);\n"
+            o += "}"
+            return o
         o = f"for (auto {'' if self._ti_is_bool else '&'}it : this->{self.field_name}) {{\n"
         if isinstance(self._ti, EnumType):
             o += f"  buffer.{self._ti.encode_func}({self.number}, static_cast(it), true);\n"
@@ -839,12 +1538,14 @@ class RepeatedTypeInfo(TypeInfo):
 
     @property
     def dump_content(self) -> str:
-        o = f"for (const auto {'' if self._ti_is_bool else '&'}it : this->{self.field_name}) {{\n"
-        o += f'  out.append("  {self.name}: ");\n'
-        o += indent(self._ti.dump("it")) + "\n"
-        o += '  out.append("\\n");\n'
-        o += "}\n"
-        return o
+        if self._use_pointer:
+            # For pointer fields, dereference and use the existing helper
+            return _generate_array_dump_content(
+                self._ti, f"*this->{self.field_name}", self.name, is_bool=False
+            )
+        return _generate_array_dump_content(
+            self._ti, f"this->{self.field_name}", self.name, is_bool=self._ti_is_bool
+        )
 
     def dump(self, _: str):
         pass
@@ -852,28 +1553,34 @@ class RepeatedTypeInfo(TypeInfo):
     def get_size_calculation(self, name: str, force: bool = False) -> str:
         # For repeated fields, we always need to pass force=True to the underlying type's calculation
         # This is because the encode method always sets force=true for repeated fields
+
+        # Handle message types separately as they use a dedicated helper
         if isinstance(self._ti, MessageType):
-            # For repeated messages, use the dedicated helper that handles iteration internally
             field_id_size = self._ti.calculate_field_id_size()
-            o = f"ProtoSize::add_repeated_message(total_size, {field_id_size}, {name});"
-            return o
+            container = f"*{name}" if self._use_pointer else name
+            return f"size.add_repeated_message({field_id_size}, {container});"
 
-        # For other repeated types, use the underlying type's size calculation with force=True
-        o = f"if (!{name}.empty()) {{\n"
+        # For non-message types, generate size calculation with iteration
+        container_ref = f"*{name}" if self._use_pointer else name
+        empty_check = f"{name}->empty()" if self._use_pointer else f"{name}.empty()"
 
-        # Check if this is a fixed-size type by seeing if it has a fixed byte count
+        o = f"if (!{empty_check}) {{\n"
+
+        # Check if this is a fixed-size type
         num_bytes = self._ti.get_fixed_size_bytes()
         if num_bytes is not None:
-            # Fixed types have constant size per element, so we can multiply
+            # Fixed types have constant size per element
             field_id_size = self._ti.calculate_field_id_size()
-            # Pre-calculate the total bytes per element
             bytes_per_element = field_id_size + num_bytes
-            o += f"  total_size += {name}.size() * {bytes_per_element};\n"
+            size_expr = f"{name}->size()" if self._use_pointer else f"{name}.size()"
+            o += f"  size.add_precalculated_size({size_expr} * {bytes_per_element});\n"
         else:
             # Other types need the actual value
-            o += f"  for (const auto {'' if self._ti_is_bool else '&'}it : {name}) {{\n"
+            auto_ref = "" if self._ti_is_bool else "&"
+            o += f"  for (const auto {auto_ref}it : {container_ref}) {{\n"
             o += f"    {self._ti.get_size_calculation('it', True)}\n"
             o += "  }\n"
+
         o += "}"
         return o
 
@@ -889,14 +1596,15 @@ class RepeatedTypeInfo(TypeInfo):
 
 def build_type_usage_map(
     file_desc: descriptor.FileDescriptorProto,
-) -> tuple[dict[str, str | None], dict[str, str | None]]:
+) -> tuple[dict[str, str | None], dict[str, str | None], dict[str, int], set[str]]:
     """Build mappings for both enums and messages to their ifdefs based on usage.
 
     Returns:
-        tuple: (enum_ifdef_map, message_ifdef_map)
+        tuple: (enum_ifdef_map, message_ifdef_map, message_source_map, used_messages)
     """
     enum_ifdef_map: dict[str, str | None] = {}
     message_ifdef_map: dict[str, str | None] = {}
+    message_source_map: dict[str, int] = {}
 
     # Build maps of which types are used by which messages
     enum_usage: dict[
@@ -905,6 +1613,7 @@ def build_type_usage_map(
     message_usage: dict[
         str, set[str]
     ] = {}  # message_name -> set of message names that use it
+    used_messages: set[str] = set()  # Track which messages are actually used
 
     # Build message name to ifdef mapping for quick lookup
     message_to_ifdef: dict[str, str | None] = {
@@ -912,18 +1621,35 @@ def build_type_usage_map(
     }
 
     # Analyze field usage
+    # Also track field_ifdef for message types
+    message_field_ifdefs: dict[
+        str, set[str | None]
+    ] = {}  # message_name -> set of field_ifdefs that use it
+
     for message in file_desc.message_type:
+        # Skip deprecated messages entirely
+        if message.options.deprecated:
+            continue
+
         for field in message.field:
+            # Skip deprecated fields when tracking enum usage
+            if field.options.deprecated:
+                continue
+
             type_name = field.type_name.split(".")[-1] if field.type_name else None
             if not type_name:
                 continue
 
-            # Track enum usage
+            # Track enum usage (only from non-deprecated fields)
             if field.type == 14:  # TYPE_ENUM
                 enum_usage.setdefault(type_name, set()).add(message.name)
             # Track message usage
             elif field.type == 11:  # TYPE_MESSAGE
                 message_usage.setdefault(type_name, set()).add(message.name)
+                # Also track the field_ifdef if present
+                field_ifdef = get_field_opt(field, pb.field_ifdef)
+                message_field_ifdefs.setdefault(type_name, set()).add(field_ifdef)
+                used_messages.add(type_name)
 
     # Helper to get unique ifdef from a set of messages
     def get_unique_ifdef(message_names: set[str]) -> str | None:
@@ -949,9 +1675,16 @@ def build_type_usage_map(
             message_ifdef_map[message.name] = explicit_ifdef
         elif message.name in message_usage:
             # Inherit ifdef if all parent messages have the same one
-            message_ifdef_map[message.name] = get_unique_ifdef(
-                message_usage[message.name]
-            )
+            if parent_ifdef := get_unique_ifdef(message_usage[message.name]):
+                message_ifdef_map[message.name] = parent_ifdef
+            elif message.name in message_field_ifdefs:
+                # If no parent message ifdef, check if all fields using this message have the same field_ifdef
+                field_ifdefs = message_field_ifdefs[message.name] - {None}
+                message_ifdef_map[message.name] = (
+                    field_ifdefs.pop() if len(field_ifdefs) == 1 else None
+                )
+            else:
+                message_ifdef_map[message.name] = None
         else:
             message_ifdef_map[message.name] = None
 
@@ -983,7 +1716,55 @@ def build_type_usage_map(
                 message_ifdef_map[message.name] = parent_ifdefs.pop()
                 changed = True
 
-    return enum_ifdef_map, message_ifdef_map
+    # Build message source map
+    # First pass: Get explicit sources for messages with source option or id
+    for msg in file_desc.message_type:
+        # Skip deprecated messages
+        if msg.options.deprecated:
+            continue
+
+        if msg.options.HasExtension(pb.source):
+            # Explicit source option takes precedence
+            message_source_map[msg.name] = get_opt(msg, pb.source, SOURCE_BOTH)
+        elif msg.options.HasExtension(pb.id):
+            # Service messages (with id) default to SOURCE_BOTH
+            message_source_map[msg.name] = SOURCE_BOTH
+            # Service messages are always used
+            used_messages.add(msg.name)
+
+    # Second pass: Determine sources for embedded messages based on their usage
+    for msg in file_desc.message_type:
+        if msg.name in message_source_map:
+            continue  # Already has explicit source
+
+        if msg.name in message_usage:
+            # Get sources from all parent messages that use this one
+            parent_sources = {
+                message_source_map[parent]
+                for parent in message_usage[msg.name]
+                if parent in message_source_map
+            }
+
+            # Combine parent sources
+            if not parent_sources:
+                # No parent has explicit source, default to encode-only
+                message_source_map[msg.name] = SOURCE_SERVER
+            elif len(parent_sources) > 1:
+                # Multiple different sources or SOURCE_BOTH present
+                message_source_map[msg.name] = SOURCE_BOTH
+            else:
+                # Inherit single parent source
+                message_source_map[msg.name] = parent_sources.pop()
+        else:
+            # Not used by any message and no explicit source - default to encode-only
+            message_source_map[msg.name] = SOURCE_SERVER
+
+    return (
+        enum_ifdef_map,
+        message_ifdef_map,
+        message_source_map,
+        used_messages,
+    )
 
 
 def build_enum_type(desc, enum_ifdef_map) -> tuple[str, str, str]:
@@ -1025,10 +1806,11 @@ def calculate_message_estimated_size(desc: descriptor.DescriptorProto) -> int:
     total_size = 0
 
     for field in desc.field:
-        if field.label == 3:  # repeated
-            ti = RepeatedTypeInfo(field)
-        else:
-            ti = TYPE_INFO[field.type](field)
+        # Skip deprecated fields
+        if field.options.deprecated:
+            continue
+
+        ti = create_field_type_info(field)
 
         # Add estimated size for this field
         total_size += ti.get_estimated_size()
@@ -1038,7 +1820,8 @@ def calculate_message_estimated_size(desc: descriptor.DescriptorProto) -> int:
 
 def build_message_type(
     desc: descriptor.DescriptorProto,
-    base_class_fields: dict[str, list[descriptor.FieldDescriptorProto]] = None,
+    base_class_fields: dict[str, list[descriptor.FieldDescriptorProto]],
+    message_source_map: dict[str, int],
 ) -> tuple[str, str, str]:
     public_content: list[str] = []
     protected_content: list[str] = []
@@ -1060,7 +1843,7 @@ def build_message_type(
     message_id: int | None = get_opt(desc, pb.id)
 
     # Get source direction to determine if we need decode/encode methods
-    source: int = get_opt(desc, pb.source, SOURCE_BOTH)
+    source = message_source_map[desc.name]
     needs_decode = source in (SOURCE_BOTH, SOURCE_CLIENT)
     needs_encode = source in (SOURCE_BOTH, SOURCE_SERVER)
 
@@ -1077,13 +1860,16 @@ def build_message_type(
 
         # Add estimated size constant
         estimated_size = calculate_message_estimated_size(desc)
-        # Validate that estimated_size fits in uint8_t
-        if estimated_size > 255:
-            raise ValueError(
-                f"Estimated size {estimated_size} for {desc.name} exceeds uint8_t maximum (255)"
-            )
+        # Use a type appropriate for estimated_size
+        estimated_size_type = (
+            "uint8_t"
+            if estimated_size <= 255
+            else "uint16_t"
+            if estimated_size <= 65535
+            else "size_t"
+        )
         public_content.append(
-            f"static constexpr uint8_t ESTIMATED_SIZE = {estimated_size};"
+            f"static constexpr {estimated_size_type} ESTIMATED_SIZE = {estimated_size};"
         )
 
         # Add message_name method inline in header
@@ -1094,87 +1880,181 @@ def build_message_type(
         )
         public_content.append("#endif")
 
+    # Collect fixed_vector fields for custom decode generation
+    fixed_vector_fields = []
+
     for field in desc.field:
-        if field.label == 3:
-            ti = RepeatedTypeInfo(field)
-        else:
-            ti = TYPE_INFO[field.type](field)
+        # Skip deprecated fields completely
+        if field.options.deprecated:
+            continue
+
+        # Validate that fixed_array_size is only used in encode-only messages
+        if (
+            needs_decode
+            and field.label == FieldDescriptorProto.LABEL_REPEATED
+            and get_field_opt(field, pb.fixed_array_size) is not None
+        ):
+            raise ValueError(
+                f"Message '{desc.name}' uses fixed_array_size on field '{field.name}' "
+                f"but has source={SOURCE_NAMES[source]}. "
+                f"Fixed arrays are only supported for SOURCE_SERVER (encode-only) messages "
+                f"since we cannot trust or control the number of items received from clients."
+            )
+
+        # Validate that fixed_array_with_length_define is only used in encode-only messages
+        if (
+            needs_decode
+            and field.label == FieldDescriptorProto.LABEL_REPEATED
+            and get_field_opt(field, pb.fixed_array_with_length_define) is not None
+        ):
+            raise ValueError(
+                f"Message '{desc.name}' uses fixed_array_with_length_define on field '{field.name}' "
+                f"but has source={SOURCE_NAMES[source]}. "
+                f"Fixed arrays with length are only supported for SOURCE_SERVER (encode-only) messages "
+                f"since we cannot trust or control the number of items received from clients."
+            )
+
+        # Collect fixed_vector repeated fields for custom decode generation
+        if (
+            needs_decode
+            and field.label == FieldDescriptorProto.LABEL_REPEATED
+            and get_field_opt(field, pb.fixed_vector, False)
+        ):
+            fixed_vector_fields.append((field.name, field.number))
+
+        ti = create_field_type_info(field, needs_decode, needs_encode)
 
         # Skip field declarations for fields that are in the base class
         # but include their encode/decode logic
         if field.name not in common_field_names:
-            protected_content.extend(ti.protected_content)
-            public_content.extend(ti.public_content)
+            # Check for field_ifdef option
+            field_ifdef = None
+            if field.options.HasExtension(pb.field_ifdef):
+                field_ifdef = field.options.Extensions[pb.field_ifdef]
+
+            if ti.protected_content:
+                protected_content.extend(
+                    wrap_with_ifdef(ti.protected_content, field_ifdef)
+                )
+            if ti.public_content:
+                public_content.extend(wrap_with_ifdef(ti.public_content, field_ifdef))
 
         # Only collect encode logic if this message needs it
         if needs_encode:
-            encode.append(ti.encode_content)
-            size_calc.append(ti.get_size_calculation(f"this->{ti.field_name}"))
+            # Check for field_ifdef option
+            field_ifdef = None
+            if field.options.HasExtension(pb.field_ifdef):
+                field_ifdef = field.options.Extensions[pb.field_ifdef]
+
+            encode.extend(wrap_with_ifdef(ti.encode_content, field_ifdef))
+            size_calc.extend(
+                wrap_with_ifdef(
+                    ti.get_size_calculation(f"this->{ti.field_name}"), field_ifdef
+                )
+            )
 
         # Only collect decode methods if this message needs them
         if needs_decode:
+            # Check for field_ifdef option for decode as well
+            field_ifdef = None
+            if field.options.HasExtension(pb.field_ifdef):
+                field_ifdef = field.options.Extensions[pb.field_ifdef]
+
             if ti.decode_varint_content:
-                decode_varint.append(ti.decode_varint_content)
+                decode_varint.extend(
+                    wrap_with_ifdef(ti.decode_varint_content, field_ifdef)
+                )
             if ti.decode_length_content:
-                decode_length.append(ti.decode_length_content)
+                decode_length.extend(
+                    wrap_with_ifdef(ti.decode_length_content, field_ifdef)
+                )
             if ti.decode_32bit_content:
-                decode_32bit.append(ti.decode_32bit_content)
+                decode_32bit.extend(
+                    wrap_with_ifdef(ti.decode_32bit_content, field_ifdef)
+                )
             if ti.decode_64bit_content:
-                decode_64bit.append(ti.decode_64bit_content)
+                decode_64bit.extend(
+                    wrap_with_ifdef(ti.decode_64bit_content, field_ifdef)
+                )
         if ti.dump_content:
-            dump.append(ti.dump_content)
+            # Check for field_ifdef option for dump as well
+            field_ifdef = None
+            if field.options.HasExtension(pb.field_ifdef):
+                field_ifdef = field.options.Extensions[pb.field_ifdef]
+
+            dump.extend(wrap_with_ifdef(ti.dump_content, field_ifdef))
 
     cpp = ""
     if decode_varint:
-        decode_varint.append("default:\n  return false;")
         o = f"bool {desc.name}::decode_varint(uint32_t field_id, ProtoVarInt value) {{\n"
         o += "  switch (field_id) {\n"
         o += indent("\n".join(decode_varint), "    ") + "\n"
+        o += "    default: return false;\n"
         o += "  }\n"
+        o += "  return true;\n"
         o += "}\n"
         cpp += o
         prot = "bool decode_varint(uint32_t field_id, ProtoVarInt value) override;"
         protected_content.insert(0, prot)
     if decode_length:
-        decode_length.append("default:\n  return false;")
         o = f"bool {desc.name}::decode_length(uint32_t field_id, ProtoLengthDelimited value) {{\n"
         o += "  switch (field_id) {\n"
         o += indent("\n".join(decode_length), "    ") + "\n"
+        o += "    default: return false;\n"
         o += "  }\n"
+        o += "  return true;\n"
         o += "}\n"
         cpp += o
         prot = "bool decode_length(uint32_t field_id, ProtoLengthDelimited value) override;"
         protected_content.insert(0, prot)
     if decode_32bit:
-        decode_32bit.append("default:\n  return false;")
         o = f"bool {desc.name}::decode_32bit(uint32_t field_id, Proto32Bit value) {{\n"
         o += "  switch (field_id) {\n"
         o += indent("\n".join(decode_32bit), "    ") + "\n"
+        o += "    default: return false;\n"
         o += "  }\n"
+        o += "  return true;\n"
         o += "}\n"
         cpp += o
         prot = "bool decode_32bit(uint32_t field_id, Proto32Bit value) override;"
         protected_content.insert(0, prot)
     if decode_64bit:
-        decode_64bit.append("default:\n  return false;")
         o = f"bool {desc.name}::decode_64bit(uint32_t field_id, Proto64Bit value) {{\n"
         o += "  switch (field_id) {\n"
         o += indent("\n".join(decode_64bit), "    ") + "\n"
+        o += "    default: return false;\n"
         o += "  }\n"
+        o += "  return true;\n"
         o += "}\n"
         cpp += o
         prot = "bool decode_64bit(uint32_t field_id, Proto64Bit value) override;"
         protected_content.insert(0, prot)
 
+    # Generate custom decode() override for messages with FixedVector fields
+    if fixed_vector_fields:
+        # Generate the decode() implementation in cpp
+        o = f"void {desc.name}::decode(const uint8_t *buffer, size_t length) {{\n"
+        # Count and init each FixedVector field
+        for field_name, field_number in fixed_vector_fields:
+            o += f"  uint32_t count_{field_name} = ProtoDecodableMessage::count_repeated_field(buffer, length, {field_number});\n"
+            o += f"  this->{field_name}.init(count_{field_name});\n"
+        # Call parent decode to populate the fields
+        o += "  ProtoDecodableMessage::decode(buffer, length);\n"
+        o += "}\n"
+        cpp += o
+        # Generate the decode() declaration in header (public method)
+        prot = "void decode(const uint8_t *buffer, size_t length) override;"
+        public_content.append(prot)
+
     # Only generate encode method if this message needs encoding and has fields
     if needs_encode and encode:
         o = f"void {desc.name}::encode(ProtoWriteBuffer buffer) const {{"
         if len(encode) == 1 and len(encode[0]) + len(o) + 3 < 120:
-            o += f" {encode[0]} "
+            o += f" {encode[0]} }}\n"
         else:
             o += "\n"
             o += indent("\n".join(encode)) + "\n"
-        o += "}\n"
+            o += "}\n"
         cpp += o
         prot = "void encode(ProtoWriteBuffer buffer) const override;"
         public_content.append(prot)
@@ -1182,17 +2062,17 @@ def build_message_type(
 
     # Add calculate_size method only if this message needs encoding and has fields
     if needs_encode and size_calc:
-        o = f"void {desc.name}::calculate_size(uint32_t &total_size) const {{"
+        o = f"void {desc.name}::calculate_size(ProtoSize &size) const {{"
         # For a single field, just inline it for simplicity
         if len(size_calc) == 1 and len(size_calc[0]) + len(o) + 3 < 120:
-            o += f" {size_calc[0]} "
+            o += f" {size_calc[0]} }}\n"
         else:
             # For multiple fields
             o += "\n"
             o += indent("\n".join(size_calc)) + "\n"
-        o += "}\n"
+            o += "}\n"
         cpp += o
-        prot = "void calculate_size(uint32_t &total_size) const override;"
+        prot = "void calculate_size(ProtoSize &size) const override;"
         public_content.append(prot)
     # If no fields to calculate size for or message doesn't need encoding, the default implementation in ProtoMessage will be used
 
@@ -1209,10 +2089,8 @@ def build_message_type(
             dump_impl += f" {dump[0]} "
         else:
             dump_impl += "\n"
-            dump_impl += "  __attribute__((unused)) char buffer[64];\n"
-            dump_impl += f'  out.append("{desc.name} {{\\n");\n'
+            dump_impl += f'  MessageDumpHelper helper(out, "{desc.name}");\n'
             dump_impl += indent("\n".join(dump)) + "\n"
-            dump_impl += '  out.append("}");\n'
     else:
         o2 = f'out.append("{desc.name} {{}}");'
         if len(dump_impl) + len(o2) + 3 < 120:
@@ -1223,9 +2101,16 @@ def build_message_type(
     dump_impl += "}\n"
 
     if base_class:
-        out = f"class {desc.name} : public {base_class} {{\n"
+        out = f"class {desc.name} final : public {base_class} {{\n"
     else:
-        out = f"class {desc.name} : public ProtoMessage {{\n"
+        # Check if message has any non-deprecated fields
+        has_fields = any(not field.options.deprecated for field in desc.field)
+        # Determine inheritance based on whether the message needs decoding and has fields
+        if needs_decode and has_fields:
+            base_class = "ProtoDecodableMessage"
+        else:
+            base_class = "ProtoMessage"
+        out = f"class {desc.name} final : public {base_class} {{\n"
     out += " public:\n"
     out += indent("\n".join(public_content)) + "\n"
     out += "\n"
@@ -1245,6 +2130,12 @@ SOURCE_BOTH = 0
 SOURCE_SERVER = 1
 SOURCE_CLIENT = 2
 
+SOURCE_NAMES = {
+    SOURCE_BOTH: "SOURCE_BOTH",
+    SOURCE_SERVER: "SOURCE_SERVER",
+    SOURCE_CLIENT: "SOURCE_CLIENT",
+}
+
 RECEIVE_CASES: dict[int, tuple[str, str | None]] = {}
 
 ifdefs: dict[str, str] = {}
@@ -1261,6 +2152,17 @@ def get_opt(
     return desc.options.Extensions[opt]
 
 
+def get_field_opt(
+    field: descriptor.FieldDescriptorProto,
+    opt: descriptor.FieldOptions,
+    default: Any = None,
+) -> Any:
+    """Get the option from a field descriptor."""
+    if not field.options.HasExtension(opt):
+        return default
+    return field.options.Extensions[opt]
+
+
 def get_base_class(desc: descriptor.DescriptorProto) -> str | None:
     """Get the base_class option from a message descriptor."""
     if not desc.options.HasExtension(pb.base_class):
@@ -1291,8 +2193,10 @@ def find_common_fields(
     if not messages:
         return []
 
-    # Start with fields from the first message
-    first_msg_fields = {field.name: field for field in messages[0].field}
+    # Start with fields from the first message (excluding deprecated fields)
+    first_msg_fields = {
+        field.name: field for field in messages[0].field if not field.options.deprecated
+    }
     common_fields = []
 
     # Check each field to see if it exists in all messages with same type
@@ -1303,6 +2207,9 @@ def find_common_fields(
         for msg in messages[1:]:
             found = False
             for other_field in msg.field:
+                # Skip deprecated fields
+                if other_field.options.deprecated:
+                    continue
                 if (
                     other_field.name == field_name
                     and other_field.type == field.type
@@ -1323,28 +2230,65 @@ def find_common_fields(
     return common_fields
 
 
+def get_common_field_ifdef(
+    field_name: str, messages: list[descriptor.DescriptorProto]
+) -> str | None:
+    """Get the field_ifdef option if it's consistent across all messages.
+
+    Args:
+        field_name: Name of the field to check
+        messages: List of messages that contain this field
+
+    Returns:
+        The field_ifdef string if all messages have the same value, None otherwise
+    """
+    field_ifdefs = {
+        get_field_opt(field, pb.field_ifdef)
+        for msg in messages
+        if (field := next((f for f in msg.field if f.name == field_name), None))
+    }
+
+    # Return the ifdef only if all messages agree on the same value
+    return field_ifdefs.pop() if len(field_ifdefs) == 1 else None
+
+
 def build_base_class(
     base_class_name: str,
     common_fields: list[descriptor.FieldDescriptorProto],
+    messages: list[descriptor.DescriptorProto],
+    message_source_map: dict[str, int],
 ) -> tuple[str, str, str]:
     """Build the base class definition and implementation."""
     public_content = []
     protected_content = []
 
+    # Determine if any message using this base class needs decoding/encoding
+    needs_decode = any(
+        message_source_map.get(msg.name, SOURCE_BOTH) in (SOURCE_BOTH, SOURCE_CLIENT)
+        for msg in messages
+    )
+    needs_encode = any(
+        message_source_map.get(msg.name, SOURCE_BOTH) in (SOURCE_BOTH, SOURCE_SERVER)
+        for msg in messages
+    )
+
     # For base classes, we only declare the fields but don't handle encode/decode
     # The derived classes will handle encoding/decoding with their specific field numbers
     for field in common_fields:
-        if field.label == 3:  # repeated
-            ti = RepeatedTypeInfo(field)
-        else:
-            ti = TYPE_INFO[field.type](field)
+        ti = create_field_type_info(field, needs_decode, needs_encode)
+
+        # Get field_ifdef if it's consistent across all messages
+        field_ifdef = get_common_field_ifdef(field.name, messages)
 
         # Only add field declarations, not encode/decode logic
-        protected_content.extend(ti.protected_content)
-        public_content.extend(ti.public_content)
+        if ti.protected_content:
+            protected_content.extend(wrap_with_ifdef(ti.protected_content, field_ifdef))
+        if ti.public_content:
+            public_content.extend(wrap_with_ifdef(ti.public_content, field_ifdef))
 
     # Build header
-    out = f"class {base_class_name} : public ProtoMessage {{\n"
+    parent_class = "ProtoDecodableMessage" if needs_decode else "ProtoMessage"
+    out = f"class {base_class_name} : public {parent_class} {{\n"
     out += " public:\n"
 
     # Add destructor with override
@@ -1370,6 +2314,7 @@ def build_base_class(
 
 def generate_base_classes(
     base_class_groups: dict[str, list[descriptor.DescriptorProto]],
+    message_source_map: dict[str, int],
 ) -> tuple[str, str, str]:
     """Generate all base classes."""
     all_headers = []
@@ -1382,7 +2327,9 @@ def generate_base_classes(
 
         if common_fields:
             # Generate base class
-            header, cpp, dump_cpp = build_base_class(base_class_name, common_fields)
+            header, cpp, dump_cpp = build_base_class(
+                base_class_name, common_fields, messages, message_source_map
+            )
             all_headers.append(header)
             all_cpp.append(cpp)
             all_dump_cpp.append(dump_cpp)
@@ -1392,14 +2339,19 @@ def generate_base_classes(
 
 def build_service_message_type(
     mt: descriptor.DescriptorProto,
+    message_source_map: dict[str, int],
 ) -> tuple[str, str] | None:
     """Builds the service message type."""
+    # Skip deprecated messages
+    if mt.options.deprecated:
+        return None
+
     snake = camel_to_snake(mt.name)
     id_: int | None = get_opt(mt, pb.id)
     if id_ is None:
         return None
 
-    source: int = get_opt(mt, pb.source, 0)
+    source: int = message_source_map.get(mt.name, SOURCE_BOTH)
 
     ifdef: str | None = get_opt(mt, pb.ifdef)
     log: bool = get_opt(mt, pb.log, True)
@@ -1423,15 +2375,22 @@ def build_service_message_type(
         hout += f"virtual void {func}(const {mt.name} &value){{}};\n"
         case = ""
         case += f"{mt.name} msg;\n"
-        case += "msg.decode(msg_data, msg_size);\n"
+        # Check if this message has any fields (excluding deprecated ones)
+        has_fields = any(not field.options.deprecated for field in mt.field)
+        if has_fields:
+            # Normal case: decode the message
+            case += "msg.decode(msg_data, msg_size);\n"
+        else:
+            # Empty message optimization: skip decode since there are no fields
+            case += "// Empty message: no decode needed\n"
         if log:
             case += "#ifdef HAS_PROTO_MESSAGE_DUMP\n"
             case += f'ESP_LOGVV(TAG, "{func}: %s", msg.dump().c_str());\n'
             case += "#endif\n"
         case += f"this->{func}(msg);\n"
         case += "break;"
-        # Store the ifdef with the case for later use
-        RECEIVE_CASES[id_] = (case, ifdef)
+        # Store the message name and ifdef with the case for later use
+        RECEIVE_CASES[id_] = (case, ifdef, mt.name)
 
         # Only close ifdef if we opened it
         if ifdef is not None:
@@ -1452,16 +2411,20 @@ def main() -> None:
     d = descriptor.FileDescriptorSet.FromString(proto_content)
 
     file = d.file[0]
+
     content = FILE_HEADER
     content += """\
 #pragma once
 
 #include "esphome/core/defines.h"
+#include "esphome/core/string_ref.h"
 
 #include "proto.h"
+#include "api_pb2_includes.h"
+"""
 
-namespace esphome {
-namespace api {
+    content += """
+namespace esphome::api {
 
 """
 
@@ -1470,9 +2433,9 @@ namespace api {
     #include "api_pb2.h"
     #include "esphome/core/log.h"
     #include "esphome/core/helpers.h"
+    #include 
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 """
 
@@ -1486,20 +2449,111 @@ namespace api {
 
 #ifdef HAS_PROTO_MESSAGE_DUMP
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
+
+// Helper function to append a quoted string, handling empty StringRef
+static inline void append_quoted_string(std::string &out, const StringRef &ref) {
+  out.append("'");
+  if (!ref.empty()) {
+    out.append(ref.c_str());
+  }
+  out.append("'");
+}
+
+// Common helpers for dump_field functions
+static inline void append_field_prefix(std::string &out, const char *field_name, int indent) {
+  out.append(indent, ' ').append(field_name).append(": ");
+}
+
+static inline void append_with_newline(std::string &out, const char *str) {
+  out.append(str);
+  out.append("\\n");
+}
+
+// RAII helper for message dump formatting
+class MessageDumpHelper {
+ public:
+  MessageDumpHelper(std::string &out, const char *message_name) : out_(out) {
+    out_.append(message_name);
+    out_.append(" {\\n");
+  }
+  ~MessageDumpHelper() { out_.append(" }"); }
+
+ private:
+  std::string &out_;
+};
+
+// Helper functions to reduce code duplication in dump methods
+static void dump_field(std::string &out, const char *field_name, int32_t value, int indent = 2) {
+  char buffer[64];
+  append_field_prefix(out, field_name, indent);
+  snprintf(buffer, 64, "%" PRId32, value);
+  append_with_newline(out, buffer);
+}
+
+static void dump_field(std::string &out, const char *field_name, uint32_t value, int indent = 2) {
+  char buffer[64];
+  append_field_prefix(out, field_name, indent);
+  snprintf(buffer, 64, "%" PRIu32, value);
+  append_with_newline(out, buffer);
+}
+
+static void dump_field(std::string &out, const char *field_name, float value, int indent = 2) {
+  char buffer[64];
+  append_field_prefix(out, field_name, indent);
+  snprintf(buffer, 64, "%g", value);
+  append_with_newline(out, buffer);
+}
+
+static void dump_field(std::string &out, const char *field_name, uint64_t value, int indent = 2) {
+  char buffer[64];
+  append_field_prefix(out, field_name, indent);
+  snprintf(buffer, 64, "%llu", value);
+  append_with_newline(out, buffer);
+}
+
+static void dump_field(std::string &out, const char *field_name, bool value, int indent = 2) {
+  append_field_prefix(out, field_name, indent);
+  out.append(YESNO(value));
+  out.append("\\n");
+}
+
+static void dump_field(std::string &out, const char *field_name, const std::string &value, int indent = 2) {
+  append_field_prefix(out, field_name, indent);
+  out.append("'").append(value).append("'");
+  out.append("\\n");
+}
+
+static void dump_field(std::string &out, const char *field_name, StringRef value, int indent = 2) {
+  append_field_prefix(out, field_name, indent);
+  append_quoted_string(out, value);
+  out.append("\\n");
+}
+
+template
+static void dump_field(std::string &out, const char *field_name, T value, int indent = 2) {
+  append_field_prefix(out, field_name, indent);
+  out.append(proto_enum_to_string(value));
+  out.append("\\n");
+}
 
 """
 
     content += "namespace enums {\n\n"
 
     # Build dynamic ifdef mappings for both enums and messages
-    enum_ifdef_map, message_ifdef_map = build_type_usage_map(file)
+    enum_ifdef_map, message_ifdef_map, message_source_map, used_messages = (
+        build_type_usage_map(file)
+    )
 
     # Simple grouping of enums by ifdef
     current_ifdef = None
 
     for enum in file.enum_type:
+        # Skip deprecated enums
+        if enum.options.deprecated:
+            continue
+
         s, c, dc = build_enum_type(enum, enum_ifdef_map)
         enum_ifdef = enum_ifdef_map.get(enum.name)
 
@@ -1538,7 +2592,9 @@ namespace api {
 
     # Generate base classes
     if base_class_fields:
-        base_headers, base_cpp, base_dump_cpp = generate_base_classes(base_class_groups)
+        base_headers, base_cpp, base_dump_cpp = generate_base_classes(
+            base_class_groups, message_source_map
+        )
         content += base_headers
         cpp += base_cpp
         dump_cpp += base_dump_cpp
@@ -1548,7 +2604,15 @@ namespace api {
     current_ifdef = None
 
     for m in mt:
-        s, c, dc = build_message_type(m, base_class_fields)
+        # Skip deprecated messages
+        if m.options.deprecated:
+            continue
+
+        # Skip messages that aren't used (unless they have an ID/service message)
+        if m.name not in used_messages and not m.options.HasExtension(pb.id):
+            continue
+
+        s, c, dc = build_message_type(m, base_class_fields, message_source_map)
         msg_ifdef = message_ifdef_map.get(m.name)
 
         # Handle ifdef changes
@@ -1577,19 +2641,16 @@ namespace api {
 
     content += """\
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 """
     cpp += """\
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 """
 
     dump_cpp += """\
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 
 #endif  // HAS_PROTO_MESSAGE_DUMP
 """
@@ -1611,8 +2672,7 @@ namespace api {
 
 #include "api_pb2.h"
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 """
 
@@ -1621,8 +2681,7 @@ namespace api {
 #include "api_pb2_service.h"
 #include "esphome/core/log.h"
 
-namespace esphome {
-namespace api {
+namespace esphome::api {
 
 static const char *const TAG = "api.service";
 
@@ -1640,13 +2699,12 @@ static const char *const TAG = "api.service";
     hpp += " public:\n"
     hpp += "#endif\n\n"
 
-    # Add generic send_message method
-    hpp += "  template\n"
-    hpp += "  bool send_message(const T &msg) {\n"
+    # Add non-template send_message method
+    hpp += "  bool send_message(const ProtoMessage &msg, uint8_t message_type) {\n"
     hpp += "#ifdef HAS_PROTO_MESSAGE_DUMP\n"
     hpp += "    this->log_send_message_(msg.message_name(), msg.dump());\n"
     hpp += "#endif\n"
-    hpp += "    return this->send_message_(msg, T::MESSAGE_TYPE);\n"
+    hpp += "    return this->send_message_(msg, message_type);\n"
     hpp += "  }\n\n"
 
     # Add logging helper method implementation to cpp
@@ -1657,7 +2715,7 @@ static const char *const TAG = "api.service";
     cpp += "#endif\n\n"
 
     for mt in file.message_type:
-        obj = build_service_message_type(mt)
+        obj = build_service_message_type(mt, message_source_map)
         if obj is None:
             continue
         hout, cout = obj
@@ -1670,10 +2728,11 @@ static const char *const TAG = "api.service";
     hpp += "  void read_message(uint32_t msg_size, uint32_t msg_type, uint8_t *msg_data) override;\n"
     out = f"void {class_name}::read_message(uint32_t msg_size, uint32_t msg_type, uint8_t *msg_data) {{\n"
     out += "  switch (msg_type) {\n"
-    for i, (case, ifdef) in cases:
+    for i, (case, ifdef, message_name) in cases:
         if ifdef is not None:
             out += f"#ifdef {ifdef}\n"
-        c = f"    case {i}: {{\n"
+
+        c = f"    case {message_name}::MESSAGE_TYPE: {{\n"
         c += indent(case, "      ") + "\n"
         c += "    }"
         out += c + "\n"
@@ -1694,6 +2753,10 @@ static const char *const TAG = "api.service";
     hpp_protected = ""
     cpp += "\n"
 
+    # Build a mapping of message input types to their authentication requirements
+    message_auth_map: dict[str, bool] = {}
+    message_conn_map: dict[str, bool] = {}
+
     m = serv.method[0]
     for m in serv.method:
         func = m.name
@@ -1705,7 +2768,11 @@ static const char *const TAG = "api.service";
         needs_conn = get_opt(m, pb.needs_setup_connection, True)
         needs_auth = get_opt(m, pb.needs_authentication, True)
 
-        ifdef = message_ifdef_map.get(inp, ifdefs.get(inp, None))
+        # Store authentication requirements for message types
+        message_auth_map[inp] = needs_auth
+        message_conn_map[inp] = needs_conn
+
+        ifdef = message_ifdef_map.get(inp, ifdefs.get(inp))
 
         if ifdef is not None:
             hpp += f"#ifdef {ifdef}\n"
@@ -1713,41 +2780,23 @@ static const char *const TAG = "api.service";
             cpp += f"#ifdef {ifdef}\n"
 
         hpp_protected += f"  void {on_func}(const {inp} &msg) override;\n"
-        hpp += f"  virtual {ret} {func}(const {inp} &msg) = 0;\n"
+
+        # For non-void methods, generate a send_ method instead of return-by-value
+        if is_void:
+            hpp += f"  virtual void {func}(const {inp} &msg) = 0;\n"
+        else:
+            hpp += f"  virtual bool send_{func}_response(const {inp} &msg) = 0;\n"
+
         cpp += f"void {class_name}::{on_func}(const {inp} &msg) {{\n"
 
-        # Start with authentication/connection check if needed
-        if needs_auth or needs_conn:
-            # Determine which check to use
-            if needs_auth:
-                check_func = "this->check_authenticated_()"
-            else:
-                check_func = "this->check_connection_setup_()"
-
-            body = f"if ({check_func}) {{\n"
-
-            # Add the actual handler code, indented
-            handler_body = ""
-            if is_void:
-                handler_body = f"this->{func}(msg);\n"
-            else:
-                handler_body = f"{ret} ret = this->{func}(msg);\n"
-                handler_body += "if (!this->send_message(ret)) {\n"
-                handler_body += "  this->on_fatal_error();\n"
-                handler_body += "}\n"
-
-            body += indent(handler_body) + "\n"
-            body += "}\n"
+        # No authentication check here - it's done in read_message
+        body = ""
+        if is_void:
+            body += f"this->{func}(msg);\n"
         else:
-            # No auth check needed, just call the handler
-            body = ""
-            if is_void:
-                body += f"this->{func}(msg);\n"
-            else:
-                body += f"{ret} ret = this->{func}(msg);\n"
-                body += "if (!this->send_message(ret)) {\n"
-                body += "  this->on_fatal_error();\n"
-                body += "}\n"
+            body += f"if (!this->send_{func}_response(msg)) {{\n"
+            body += "  this->on_fatal_error();\n"
+            body += "}\n"
 
         cpp += indent(body) + "\n" + "}\n"
 
@@ -1756,19 +2805,76 @@ static const char *const TAG = "api.service";
             hpp_protected += "#endif\n"
             cpp += "#endif\n"
 
+    # Generate optimized read_message with authentication checking
+    # Categorize messages by their authentication requirements
+    no_conn_ids: set[int] = set()
+    conn_only_ids: set[int] = set()
+
+    for id_, (_, _, case_msg_name) in cases:
+        if case_msg_name in message_auth_map:
+            needs_auth = message_auth_map[case_msg_name]
+            needs_conn = message_conn_map[case_msg_name]
+
+            if not needs_conn:
+                no_conn_ids.add(id_)
+            elif not needs_auth:
+                conn_only_ids.add(id_)
+
+    # Generate override if we have messages that skip checks
+    if no_conn_ids or conn_only_ids:
+        # Helper to generate case statements with ifdefs
+        def generate_cases(ids: set[int], comment: str) -> str:
+            result = ""
+            for id_ in sorted(ids):
+                _, ifdef, msg_name = RECEIVE_CASES[id_]
+                if ifdef:
+                    result += f"#ifdef {ifdef}\n"
+                result += f"    case {msg_name}::MESSAGE_TYPE:  {comment}\n"
+                if ifdef:
+                    result += "#endif\n"
+            return result
+
+        hpp_protected += "  void read_message(uint32_t msg_size, uint32_t msg_type, uint8_t *msg_data) override;\n"
+
+        cpp += f"\nvoid {class_name}::read_message(uint32_t msg_size, uint32_t msg_type, uint8_t *msg_data) {{\n"
+        cpp += "  // Check authentication/connection requirements for messages\n"
+        cpp += "  switch (msg_type) {\n"
+
+        # Messages that don't need any checks
+        if no_conn_ids:
+            cpp += generate_cases(no_conn_ids, "// No setup required")
+            cpp += "      break;  // Skip all checks for these messages\n"
+
+        # Messages that only need connection setup
+        if conn_only_ids:
+            cpp += generate_cases(conn_only_ids, "// Connection setup only")
+            cpp += "      if (!this->check_connection_setup_()) {\n"
+            cpp += "        return;  // Connection not setup\n"
+            cpp += "      }\n"
+            cpp += "      break;\n"
+
+        cpp += "    default:\n"
+        cpp += "      // All other messages require authentication (which includes connection check)\n"
+        cpp += "      if (!this->check_authenticated_()) {\n"
+        cpp += "        return;  // Authentication failed\n"
+        cpp += "      }\n"
+        cpp += "      break;\n"
+        cpp += "  }\n\n"
+        cpp += "  // Call base implementation to process the message\n"
+        cpp += f"  {class_name}Base::read_message(msg_size, msg_type, msg_data);\n"
+        cpp += "}\n"
+
     hpp += " protected:\n"
     hpp += hpp_protected
     hpp += "};\n"
 
     hpp += """\
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 """
     cpp += """\
 
-}  // namespace api
-}  // namespace esphome
+}  // namespace esphome::api
 """
 
     with open(root / "api_pb2_service.h", "w", encoding="utf-8") as f:
@@ -1783,8 +2889,8 @@ static const char *const TAG = "api.service";
         import clang_format
 
         def exec_clang_format(path: Path) -> None:
-            clang_format_path = os.path.join(
-                os.path.dirname(clang_format.__file__), "data", "bin", "clang-format"
+            clang_format_path = (
+                Path(clang_format.__file__).parent / "data" / "bin" / "clang-format"
             )
             call([clang_format_path, "-i", path])
 
diff --git a/script/build_codeowners.py b/script/build_codeowners.py
index 523fe8ac7f..10ca1295b7 100755
--- a/script/build_codeowners.py
+++ b/script/build_codeowners.py
@@ -31,6 +31,7 @@ BASE = """
 pyproject.toml @esphome/core
 esphome/*.py @esphome/core
 esphome/core/* @esphome/core
+.github/** @esphome/core
 
 # Integrations
 """.strip()
@@ -38,7 +39,7 @@ esphome/core/* @esphome/core
 parts = [BASE]
 
 # Fake some directory so that get_component works
-CORE.config_path = str(root)
+CORE.config_path = root
 CORE.data[KEY_CORE] = {KEY_TARGET_FRAMEWORK: None, KEY_TARGET_PLATFORM: None}
 
 codeowners = defaultdict(list)
@@ -81,7 +82,7 @@ for path in components_dir.iterdir():
 
 
 for path, owners in sorted(codeowners.items()):
-    owners = sorted(set(owners))
+    owners = sorted(set(owners), key=str.casefold)
     if not owners:
         continue
     for owner in owners:
diff --git a/script/build_language_schema.py b/script/build_language_schema.py
index 4473ec1b5a..1ffe3c2873 100755
--- a/script/build_language_schema.py
+++ b/script/build_language_schema.py
@@ -1,9 +1,9 @@
 #!/usr/bin/env python3
 import argparse
-import glob
 import inspect
 import json
 import os
+from pathlib import Path
 import re
 
 import voluptuous as vol
@@ -70,12 +70,14 @@ def get_component_names():
     component_names = ["esphome", "sensor", "esp32", "esp8266"]
     skip_components = []
 
-    for d in os.listdir(CORE_COMPONENTS_PATH):
-        if not d.startswith("__") and os.path.isdir(
-            os.path.join(CORE_COMPONENTS_PATH, d)
+    for d in CORE_COMPONENTS_PATH.iterdir():
+        if (
+            not d.name.startswith("__")
+            and d.is_dir()
+            and d.name not in component_names
+            and d.name not in skip_components
         ):
-            if d not in component_names and d not in skip_components:
-                component_names.append(d)
+            component_names.append(d.name)
 
     return sorted(component_names)
 
@@ -119,7 +121,7 @@ from esphome.util import Registry  # noqa: E402
 
 
 def write_file(name, obj):
-    full_path = os.path.join(args.output_path, name + ".json")
+    full_path = Path(args.output_path) / f"{name}.json"
     if JSON_DUMP_PRETTY:
         json_str = json.dumps(obj, indent=2)
     else:
@@ -129,9 +131,10 @@ def write_file(name, obj):
 
 
 def delete_extra_files(keep_names):
-    for d in os.listdir(args.output_path):
-        if d.endswith(".json") and d[:-5] not in keep_names:
-            os.remove(os.path.join(args.output_path, d))
+    output_path = Path(args.output_path)
+    for d in output_path.iterdir():
+        if d.suffix == ".json" and d.stem not in keep_names:
+            d.unlink()
             print(f"Deleted {d}")
 
 
@@ -139,11 +142,10 @@ def register_module_schemas(key, module, manifest=None):
     for name, schema in module_schemas(module):
         register_known_schema(key, name, schema)
 
-    if manifest:
+    if manifest and manifest.multi_conf and S_CONFIG_SCHEMA in output[key][S_SCHEMAS]:
         # Multi conf should allow list of components
         # not sure about 2nd part of the if, might be useless config (e.g. as3935)
-        if manifest.multi_conf and S_CONFIG_SCHEMA in output[key][S_SCHEMAS]:
-            output[key][S_SCHEMAS][S_CONFIG_SCHEMA]["is_list"] = True
+        output[key][S_SCHEMAS][S_CONFIG_SCHEMA]["is_list"] = True
 
 
 def register_known_schema(module, name, schema):
@@ -230,7 +232,7 @@ def add_module_registries(domain, module):
                 reg_type = attr_name.partition("_")[0].lower()
                 found_registries[repr(attr_obj)] = f"{domain}.{reg_type}"
 
-            for name in attr_obj.keys():
+            for name in attr_obj:
                 if "." not in name:
                     reg_entry_name = name
                 else:
@@ -366,13 +368,11 @@ def get_logger_tags():
         "scheduler",
         "api.service",
     ]
-    for x in os.walk(CORE_COMPONENTS_PATH):
-        for y in glob.glob(os.path.join(x[0], "*.cpp")):
-            with open(y, encoding="utf-8") as file:
-                data = file.read()
-                match = pattern.search(data)
-                if match:
-                    tags.append(match.group(1))
+    for file in CORE_COMPONENTS_PATH.rglob("*.cpp"):
+        data = file.read_text()
+        match = pattern.search(data)
+        if match:
+            tags.append(match.group(1))
     return tags
 
 
@@ -411,11 +411,16 @@ def add_referenced_recursive(referenced_schemas, config_var, path, eat_schema=Fa
 
             s1 = get_str_path_schema(k)
             p = k.split(".")
-            if len(p) == 3 and path[0] == f"{p[0]}.{p[1]}":
-                # special case for schema inside platforms
-                add_referenced_recursive(
-                    referenced_schemas, s1, [path[0], "schemas", p[2]]
-                )
+            if len(p) == 3:
+                if path[0] == f"{p[0]}.{p[1]}":
+                    # special case for schema inside platforms
+                    add_referenced_recursive(
+                        referenced_schemas, s1, [path[0], "schemas", p[2]]
+                    )
+                else:
+                    add_referenced_recursive(
+                        referenced_schemas, s1, [f"{p[0]}.{p[1]}", "schemas", p[2]]
+                    )
             else:
                 add_referenced_recursive(
                     referenced_schemas, s1, [p[0], "schemas", p[1]]
@@ -438,8 +443,7 @@ def get_str_path_schema(strPath):
     if len(parts) > 2:
         parts[0] += "." + parts[1]
         parts[1] = parts[2]
-    s1 = output.get(parts[0], {}).get(S_SCHEMAS, {}).get(parts[1], {})
-    return s1
+    return output.get(parts[0], {}).get(S_SCHEMAS, {}).get(parts[1], {})
 
 
 def pop_str_path_schema(strPath):
@@ -695,7 +699,7 @@ def is_convertible_schema(schema):
     if repr(schema) in ejs.registry_schemas:
         return True
     if isinstance(schema, dict):
-        for k in schema.keys():
+        for k in schema:
             if isinstance(k, (cv.Required, cv.Optional)):
                 return True
     return False
@@ -813,7 +817,7 @@ def convert(schema, config_var, path):
         elif schema_type == "automation":
             extra_schema = None
             config_var[S_TYPE] = "trigger"
-            if automation.AUTOMATION_SCHEMA == ejs.extended_schemas[repr(data)][0]:
+            if ejs.extended_schemas[repr(data)][0] == automation.AUTOMATION_SCHEMA:
                 extra_schema = ejs.extended_schemas[repr(data)][1]
             if (
                 extra_schema is not None and len(extra_schema) > 1
@@ -921,9 +925,8 @@ def convert(schema, config_var, path):
             config = convert_config(schema_type, path + "/type_" + schema_key)
             types[schema_key] = config["schema"]
 
-    elif DUMP_UNKNOWN:
-        if S_TYPE not in config_var:
-            config_var["unknown"] = repr_schema
+    elif DUMP_UNKNOWN and S_TYPE not in config_var:
+        config_var["unknown"] = repr_schema
 
     if DUMP_PATH:
         config_var["path"] = path
diff --git a/script/ci-custom.py b/script/ci-custom.py
index 1310a93230..6b01623d92 100755
--- a/script/ci-custom.py
+++ b/script/ci-custom.py
@@ -6,6 +6,7 @@ import collections
 import fnmatch
 import functools
 import os.path
+from pathlib import Path
 import re
 import sys
 import time
@@ -75,12 +76,12 @@ ignore_types = (
 LINT_FILE_CHECKS = []
 LINT_CONTENT_CHECKS = []
 LINT_POST_CHECKS = []
-EXECUTABLE_BIT = {}
+EXECUTABLE_BIT: dict[str, int] = {}
 
-errors = collections.defaultdict(list)
+errors: collections.defaultdict[Path, list] = collections.defaultdict(list)
 
 
-def add_errors(fname, errs):
+def add_errors(fname: Path, errs: list[tuple[int, int, str] | None]) -> None:
     if not isinstance(errs, list):
         errs = [errs]
     for err in errs:
@@ -197,7 +198,7 @@ def lint_content_find_check(find, only_first=False, **kwargs):
                 find_ = find(fname, content)
             errs = []
             for line, col in find_all(content, find_):
-                err = func(fname)
+                err = func(fname, line, col, content)
                 errs.append((line + 1, col + 1, err))
                 if only_first:
                     break
@@ -241,10 +242,13 @@ def lint_ext_check(fname):
         "docker/ha-addon-rootfs/**",
         "docker/*.py",
         "script/*",
+        "CLAUDE.md",
+        "GEMINI.md",
+        ".github/copilot-instructions.md",
     ]
 )
-def lint_executable_bit(fname):
-    ex = EXECUTABLE_BIT[fname]
+def lint_executable_bit(fname: Path) -> str | None:
+    ex = EXECUTABLE_BIT[str(fname)]
     if ex != 100644:
         return (
             f"File has invalid executable bit {ex}. If running from a windows machine please "
@@ -261,12 +265,12 @@ def lint_executable_bit(fname):
         "esphome/dashboard/static/ext-searchbox.js",
     ],
 )
-def lint_tabs(fname):
+def lint_tabs(fname, line, col, content):
     return "File contains tab character. Please convert tabs to spaces."
 
 
 @lint_content_find_check("\r", only_first=True)
-def lint_newline(fname):
+def lint_newline(fname, line, col, content):
     return "File contains Windows newline. Please set your editor to Unix newline mode."
 
 
@@ -497,19 +501,20 @@ def lint_constants_usage():
             continue
         errs.append(
             f"Constant {highlight(constant)} is defined in {len(uses)} files. Please move all definitions of the "
-            f"constant to const.py (Uses: {', '.join(uses)})"
+            f"constant to const.py (Uses: {', '.join(str(u) for u in uses)}) in a separate PR. "
+            "See https://developers.esphome.io/contributing/code/#python"
         )
     return errs
 
 
-def relative_cpp_search_text(fname, content):
-    parts = fname.split("/")
+def relative_cpp_search_text(fname: Path, content) -> str:
+    parts = fname.parts
     integration = parts[2]
     return f'#include "esphome/components/{integration}'
 
 
 @lint_content_find_check(relative_cpp_search_text, include=["esphome/components/*.cpp"])
-def lint_relative_cpp_import(fname):
+def lint_relative_cpp_import(fname, line, col, content):
     return (
         "Component contains absolute import - Components must always use "
         "relative imports.\n"
@@ -520,12 +525,26 @@ def lint_relative_cpp_import(fname):
     )
 
 
-def relative_py_search_text(fname, content):
-    parts = fname.split("/")
+def relative_py_search_text(fname: Path, content: str) -> str:
+    parts = fname.parts
     integration = parts[2]
     return f"esphome.components.{integration}"
 
 
+def convert_path_to_relative(abspath, current):
+    """Convert an absolute path to a relative import path."""
+    if abspath == current:
+        return "."
+    absparts = abspath.split(".")
+    curparts = current.split(".")
+    uplen = len(curparts)
+    while absparts and curparts and absparts[0] == curparts[0]:
+        absparts.pop(0)
+        curparts.pop(0)
+        uplen -= 1
+    return "." * uplen + ".".join(absparts)
+
+
 @lint_content_find_check(
     relative_py_search_text,
     include=["esphome/components/*.py"],
@@ -534,14 +553,19 @@ def relative_py_search_text(fname, content):
         "esphome/components/web_server/__init__.py",
     ],
 )
-def lint_relative_py_import(fname):
+def lint_relative_py_import(fname, line, col, content):
+    import_line = content.splitlines()[line]
+    abspath = import_line[col:].split(" ")[0]
+    current = fname.removesuffix(".py").replace(os.path.sep, ".")
+    replacement = convert_path_to_relative(abspath, current)
+    newline = import_line.replace(abspath, replacement)
     return (
         "Component contains absolute import - Components must always use "
         "relative imports within the integration.\n"
         "Change:\n"
-        '  from esphome.components.abc import abc_ns"\n'
+        f"    {import_line}\n"
         "to:\n"
-        "  from . import abc_ns\n\n"
+        f"    {newline}\n"
     )
 
 
@@ -568,22 +592,22 @@ def lint_relative_py_import(fname):
         "esphome/components/http_request/httplib.h",
     ],
 )
-def lint_namespace(fname, content):
-    expected_name = re.match(
-        r"^esphome/components/([^/]+)/.*", fname.replace(os.path.sep, "/")
-    ).group(1)
-    search = f"namespace {expected_name}"
-    if search in content:
+def lint_namespace(fname: Path, content: str) -> str | None:
+    expected_name = fname.parts[2]
+    # Check for both old style and C++17 nested namespace syntax
+    search_old = f"namespace {expected_name}"
+    search_new = f"namespace esphome::{expected_name}"
+    if search_old in content or search_new in content:
         return None
     return (
         "Invalid namespace found in C++ file. All integration C++ files should put all "
         "functions in a separate namespace that matches the integration's name. "
-        f"Please make sure the file contains {highlight(search)}"
+        f"Please make sure the file contains {highlight(search_old)} or {highlight(search_new)}"
     )
 
 
 @lint_content_find_check('"esphome.h"', include=cpp_include, exclude=["tests/custom.h"])
-def lint_esphome_h(fname):
+def lint_esphome_h(fname, line, col, content):
     return (
         "File contains reference to 'esphome.h' - This file is "
         "auto-generated and should only be used for *custom* "
@@ -674,7 +698,7 @@ def lint_trailing_whitespace(fname, match):
         "tests/custom.h",
     ],
 )
-def lint_log_in_header(fname):
+def lint_log_in_header(fname, line, col, content):
     return (
         "Found reference to ESP_LOG in header file. Using ESP_LOG* in header files "
         "is currently not possible - please move the definition to a source file (.cpp)"
@@ -708,9 +732,9 @@ def main():
     files.sort()
 
     for fname in files:
-        _, ext = os.path.splitext(fname)
+        fname = Path(fname)
         run_checks(LINT_FILE_CHECKS, fname, fname)
-        if ext in ignore_types:
+        if fname.suffix in ignore_types:
             continue
         try:
             with codecs.open(fname, "r", encoding="utf-8") as f_handle:
diff --git a/script/clang-format b/script/clang-format
index b1e84a56b7..028d752c55 100755
--- a/script/clang-format
+++ b/script/clang-format
@@ -31,7 +31,11 @@ def run_format(executable, args, queue, lock, failed_files):
         invocation.append(path)
 
         proc = subprocess.run(
-            invocation, capture_output=True, encoding="utf-8", check=False
+            invocation,
+            capture_output=True,
+            encoding="utf-8",
+            check=False,
+            close_fds=False,
         )
         if proc.returncode != 0:
             with lock:
@@ -66,9 +70,10 @@ def main():
     )
     args = parser.parse_args()
 
-    files = []
-    for path in git_ls_files(["*.cpp", "*.h", "*.tcc"]):
-        files.append(os.path.relpath(path, os.getcwd()))
+    cwd = os.getcwd()
+    files = [
+        os.path.relpath(path, cwd) for path in git_ls_files(["*.cpp", "*.h", "*.tcc"])
+    ]
 
     if args.files:
         # Match against files specified on command-line
diff --git a/script/clang-tidy b/script/clang-tidy
index b5905e0e4e..142b616119 100755
--- a/script/clang-tidy
+++ b/script/clang-tidy
@@ -105,7 +105,7 @@ def clang_options(idedata):
     # idedata contains include directories for all toolchains of this platform, only use those from the one in use
     toolchain_dir = os.path.normpath(f"{idedata['cxx_path']}/../../")
     for directory in idedata["includes"]["toolchain"]:
-        if directory.startswith(toolchain_dir):
+        if directory.startswith(toolchain_dir) and "picolibc" not in directory:
             cmd.extend(["-isystem", directory])
 
     # add library include directories using -isystem to suppress their errors
@@ -158,7 +158,11 @@ def run_tidy(executable, args, options, tmpdir, path_queue, lock, failed_files):
         invocation.extend(options)
 
         proc = subprocess.run(
-            invocation, capture_output=True, encoding="utf-8", check=False
+            invocation,
+            capture_output=True,
+            encoding="utf-8",
+            check=False,
+            close_fds=False,
         )
         if proc.returncode != 0:
             with lock:
@@ -205,7 +209,12 @@ def main():
     parser.add_argument(
         "-c", "--changed", action="store_true", help="only run on changed files"
     )
-    parser.add_argument("-g", "--grep", help="only run on files containing value")
+    parser.add_argument(
+        "-g",
+        "--grep",
+        action="append",
+        help="only run on files containing value",
+    )
     parser.add_argument(
         "--split-num", type=int, help="split the files into X jobs.", default=None
     )
@@ -219,9 +228,8 @@ def main():
     )
     args = parser.parse_args()
 
-    files = []
-    for path in git_ls_files(["*.cpp"]):
-        files.append(os.path.relpath(path, os.getcwd()))
+    cwd = os.getcwd()
+    files = [os.path.relpath(path, cwd) for path in git_ls_files(["*.cpp"])]
 
     # Print initial file count if it's large
     if len(files) > 50:
@@ -316,9 +324,11 @@ def main():
         print("Applying fixes ...")
         try:
             try:
-                subprocess.call(["clang-apply-replacements-18", tmpdir])
+                subprocess.call(
+                    ["clang-apply-replacements-18", tmpdir], close_fds=False
+                )
             except FileNotFoundError:
-                subprocess.call(["clang-apply-replacements", tmpdir])
+                subprocess.call(["clang-apply-replacements", tmpdir], close_fds=False)
         except FileNotFoundError:
             print(
                 "Error please install clang-apply-replacements-18 or clang-apply-replacements.\n",
diff --git a/script/clang_tidy_hash.py b/script/clang_tidy_hash.py
index 19eb2a825e..d0d8438437 100755
--- a/script/clang_tidy_hash.py
+++ b/script/clang_tidy_hash.py
@@ -48,9 +48,10 @@ def parse_requirement_line(line: str) -> tuple[str, str] | None:
     return None
 
 
-def get_clang_tidy_version_from_requirements() -> str:
+def get_clang_tidy_version_from_requirements(repo_root: Path | None = None) -> str:
     """Get clang-tidy version from requirements_dev.txt"""
-    requirements_path = Path(__file__).parent.parent / "requirements_dev.txt"
+    repo_root = _ensure_repo_root(repo_root)
+    requirements_path = repo_root / "requirements_dev.txt"
     lines = read_file_lines(requirements_path)
 
     for line in lines:
@@ -68,30 +69,49 @@ def read_file_bytes(path: Path) -> bytes:
         return f.read()
 
 
-def calculate_clang_tidy_hash() -> str:
+def get_repo_root() -> Path:
+    """Get the repository root directory."""
+    return Path(__file__).parent.parent
+
+
+def _ensure_repo_root(repo_root: Path | None) -> Path:
+    """Ensure repo_root is a Path, using default if None."""
+    return repo_root if repo_root is not None else get_repo_root()
+
+
+def calculate_clang_tidy_hash(repo_root: Path | None = None) -> str:
     """Calculate hash of clang-tidy configuration and version"""
+    repo_root = _ensure_repo_root(repo_root)
+
     hasher = hashlib.sha256()
 
     # Hash .clang-tidy file
-    clang_tidy_path = Path(__file__).parent.parent / ".clang-tidy"
+    clang_tidy_path = repo_root / ".clang-tidy"
     content = read_file_bytes(clang_tidy_path)
     hasher.update(content)
 
     # Hash clang-tidy version from requirements_dev.txt
-    version = get_clang_tidy_version_from_requirements()
+    version = get_clang_tidy_version_from_requirements(repo_root)
     hasher.update(version.encode())
 
     # Hash the entire platformio.ini file
-    platformio_path = Path(__file__).parent.parent / "platformio.ini"
+    platformio_path = repo_root / "platformio.ini"
     platformio_content = read_file_bytes(platformio_path)
     hasher.update(platformio_content)
 
+    # Hash sdkconfig.defaults file
+    sdkconfig_path = repo_root / "sdkconfig.defaults"
+    if sdkconfig_path.exists():
+        sdkconfig_content = read_file_bytes(sdkconfig_path)
+        hasher.update(sdkconfig_content)
+
     return hasher.hexdigest()
 
 
-def read_stored_hash() -> str | None:
+def read_stored_hash(repo_root: Path | None = None) -> str | None:
     """Read the stored hash from file"""
-    hash_file = Path(__file__).parent.parent / ".clang-tidy.hash"
+    repo_root = _ensure_repo_root(repo_root)
+    hash_file = repo_root / ".clang-tidy.hash"
     if hash_file.exists():
         lines = read_file_lines(hash_file)
         return lines[0].strip() if lines else None
@@ -104,9 +124,10 @@ def write_file_content(path: Path, content: str) -> None:
         f.write(content)
 
 
-def write_hash(hash_value: str) -> None:
+def write_hash(hash_value: str, repo_root: Path | None = None) -> None:
     """Write hash to file"""
-    hash_file = Path(__file__).parent.parent / ".clang-tidy.hash"
+    repo_root = _ensure_repo_root(repo_root)
+    hash_file = repo_root / ".clang-tidy.hash"
     # Strip any trailing newlines to ensure consistent formatting
     write_file_content(hash_file, hash_value.strip() + "\n")
 
@@ -134,8 +155,28 @@ def main() -> None:
     stored_hash = read_stored_hash()
 
     if args.check:
-        # Exit 0 if full scan needed (hash changed or no hash file)
-        sys.exit(0 if current_hash != stored_hash else 1)
+        # Check if hash changed OR if .clang-tidy.hash was updated in this PR
+        # This is used in CI to determine if a full clang-tidy scan is needed
+        hash_changed = current_hash != stored_hash
+
+        # Lazy import to avoid requiring dependencies that aren't needed for other modes
+        from helpers import changed_files  # noqa: E402
+
+        hash_file_updated = ".clang-tidy.hash" in changed_files()
+
+        # Exit 0 if full scan needed
+        sys.exit(0 if (hash_changed or hash_file_updated) else 1)
+
+    elif args.verify:
+        # Verify that hash file is up to date with current configuration
+        # This is used in pre-commit and CI checks to ensure hash was updated
+        if current_hash != stored_hash:
+            print("ERROR: Clang-tidy configuration has changed but hash not updated!")
+            print(f"Expected: {current_hash}")
+            print(f"Found: {stored_hash}")
+            print("\nPlease run: script/clang_tidy_hash.py --update")
+            sys.exit(1)
+        print("Hash verification passed")
 
     elif args.update:
         write_hash(current_hash)
@@ -151,15 +192,6 @@ def main() -> None:
             print("Clang-tidy hash unchanged")
             sys.exit(0)
 
-    elif args.verify:
-        if current_hash != stored_hash:
-            print("ERROR: Clang-tidy configuration has changed but hash not updated!")
-            print(f"Expected: {current_hash}")
-            print(f"Found: {stored_hash}")
-            print("\nPlease run: script/clang_tidy_hash.py --update")
-            sys.exit(1)
-        print("Hash verification passed")
-
     else:
         print(f"Current hash: {current_hash}")
         print(f"Stored hash: {stored_hash}")
diff --git a/script/determine-jobs.py b/script/determine-jobs.py
index fc5c397c65..b000ecee3b 100755
--- a/script/determine-jobs.py
+++ b/script/determine-jobs.py
@@ -31,6 +31,7 @@ Options:
 from __future__ import annotations
 
 import argparse
+from functools import cache
 import json
 import os
 from pathlib import Path
@@ -45,7 +46,6 @@ from helpers import (
     changed_files,
     get_all_dependencies,
     get_components_from_integration_fixtures,
-    parse_list_components_output,
     root_path,
 )
 
@@ -137,6 +137,10 @@ def should_run_clang_tidy(branch: str | None = None) -> bool:
        - This ensures all C++ code is checked, including tests, examples, etc.
        - Examples: esphome/core/component.cpp, tests/custom/my_component.h
 
+    3. The .clang-tidy.hash file itself changed
+       - This indicates the configuration has been updated and clang-tidy should run
+       - Ensures that PRs updating the clang-tidy configuration are properly validated
+
     If the hash check fails for any reason, clang-tidy runs as a safety measure to ensure
     code quality is maintained.
 
@@ -160,6 +164,12 @@ def should_run_clang_tidy(branch: str | None = None) -> bool:
         # If hash check fails, run clang-tidy to be safe
         return True
 
+    # Check if .clang-tidy.hash file itself was changed
+    # This handles the case where the hash was properly updated in the PR
+    files = changed_files(branch)
+    if ".clang-tidy.hash" in files:
+        return True
+
     return _any_changed_file_endswith(branch, CPP_FILE_EXTENSIONS)
 
 
@@ -202,6 +212,24 @@ def _any_changed_file_endswith(branch: str | None, extensions: tuple[str, ...])
     return any(file.endswith(extensions) for file in changed_files(branch))
 
 
+@cache
+def _component_has_tests(component: str) -> bool:
+    """Check if a component has test files.
+
+    Cached to avoid repeated filesystem operations for the same component.
+
+    Args:
+        component: Component name to check
+
+    Returns:
+        True if the component has test YAML files
+    """
+    tests_dir = Path(root_path) / "tests" / "components" / component
+    if not tests_dir.exists():
+        return False
+    return any(tests_dir.glob("test.*.yaml"))
+
+
 def main() -> None:
     """Main function that determines which CI jobs to run."""
     parser = argparse.ArgumentParser(
@@ -218,14 +246,38 @@ def main() -> None:
     run_clang_format = should_run_clang_format(args.branch)
     run_python_linters = should_run_python_linters(args.branch)
 
-    # Get changed components using list-components.py for exact compatibility
+    # Get both directly changed and all changed components (with dependencies) in one call
     script_path = Path(__file__).parent / "list-components.py"
-    cmd = [sys.executable, str(script_path), "--changed"]
+    cmd = [sys.executable, str(script_path), "--changed-with-deps"]
     if args.branch:
         cmd.extend(["-b", args.branch])
 
     result = subprocess.run(cmd, capture_output=True, text=True, check=True)
-    changed_components = parse_list_components_output(result.stdout)
+    component_data = json.loads(result.stdout)
+    directly_changed_components = component_data["directly_changed"]
+    changed_components = component_data["all_changed"]
+
+    # Filter to only components that have test files
+    # Components without tests shouldn't generate CI test jobs
+    changed_components_with_tests = [
+        component for component in changed_components if _component_has_tests(component)
+    ]
+
+    # Get directly changed components with tests (for isolated testing)
+    # These will be tested WITHOUT --testing-mode in CI to enable full validation
+    # (pin conflicts, etc.) since they contain the actual changes being reviewed
+    directly_changed_with_tests = [
+        component
+        for component in directly_changed_components
+        if _component_has_tests(component)
+    ]
+
+    # Get dependency-only components (for grouped testing)
+    dependency_only_components = [
+        component
+        for component in changed_components_with_tests
+        if component not in directly_changed_components
+    ]
 
     # Build output
     output: dict[str, Any] = {
@@ -234,7 +286,12 @@ def main() -> None:
         "clang_format": run_clang_format,
         "python_linters": run_python_linters,
         "changed_components": changed_components,
-        "component_test_count": len(changed_components),
+        "changed_components_with_tests": changed_components_with_tests,
+        "directly_changed_components_with_tests": directly_changed_with_tests,
+        "dependency_only_components_with_tests": dependency_only_components,
+        "component_test_count": len(changed_components_with_tests),
+        "directly_changed_count": len(directly_changed_with_tests),
+        "dependency_only_count": len(dependency_only_components),
     }
 
     # Output as JSON
diff --git a/script/generate-esp32-boards.py b/script/generate-esp32-boards.py
index 83d0f0c3e0..81b78b04be 100755
--- a/script/generate-esp32-boards.py
+++ b/script/generate-esp32-boards.py
@@ -1,14 +1,19 @@
 #!/usr/bin/env python3
 
+import argparse
 import json
-import os
+from pathlib import Path
 import subprocess
+import sys
 import tempfile
 
-from esphome.components.esp32 import ESP_IDF_PLATFORM_VERSION as ver
+from esphome.components.esp32 import PLATFORM_VERSION_LOOKUP
+from esphome.helpers import write_file_if_changed
 
+ver = PLATFORM_VERSION_LOOKUP["recommended"]
 version_str = f"{ver.major}.{ver.minor:02d}.{ver.patch:02d}"
-print(f"ESP32 Platform Version: {version_str}")
+root = Path(__file__).parent.parent
+boards_file_path = root / "esphome" / "components" / "esp32" / "boards.py"
 
 
 def get_boards():
@@ -17,6 +22,9 @@ def get_boards():
             [
                 "git",
                 "clone",
+                "-q",
+                "-c",
+                "advice.detachedHead=false",
                 "--depth",
                 "1",
                 "--branch",
@@ -26,16 +34,14 @@ def get_boards():
             ],
             check=True,
         )
-        boards_file = os.path.join(tempdir, "boards")
+        boards_directory = Path(tempdir) / "boards"
         boards = {}
-        for fname in os.listdir(boards_file):
-            if not fname.endswith(".json"):
-                continue
-            with open(os.path.join(boards_file, fname), encoding="utf-8") as f:
+        for fname in boards_directory.glob("*.json"):
+            with fname.open(encoding="utf-8") as f:
                 board_info = json.load(f)
                 mcu = board_info["build"]["mcu"]
                 name = board_info["name"]
-                board = fname[:-5]
+                board = fname.stem
                 variant = mcu.upper()
                 boards[board] = {
                     "name": name,
@@ -47,31 +53,47 @@ def get_boards():
 TEMPLATE = """    "%s": {
         "name": "%s",
         "variant": %s,
-    },
-"""
+    },"""
 
 
-def main():
+def main(check: bool):
     boards = get_boards()
     # open boards.py, delete existing BOARDS variable and write the new boards dict
-    boards_file_path = os.path.join(
-        os.path.dirname(__file__), "..", "esphome", "components", "esp32", "boards.py"
-    )
-    with open(boards_file_path, encoding="UTF-8") as f:
-        lines = f.readlines()
+    existing_content = boards_file_path.read_text(encoding="UTF-8")
 
-    with open(boards_file_path, "w", encoding="UTF-8") as f:
-        for line in lines:
-            if line.startswith("BOARDS = {"):
-                f.write("BOARDS = {\n")
-                for board, info in sorted(boards.items()):
-                    f.write(TEMPLATE % (board, info["name"], info["variant"]))
-                f.write("}\n")
-                break
+    parts: list[str] = []
+    for line in existing_content.splitlines():
+        if line == "BOARDS = {":
+            parts.append(line)
+            parts.extend(
+                TEMPLATE % (board, info["name"], info["variant"])
+                for board, info in sorted(boards.items())
+            )
+            parts.append("}")
+            parts.append("# DO NOT ADD ANYTHING BELOW THIS LINE")
+            break
 
-            f.write(line)
+        parts.append(line)
+
+    parts.append("")
+    content = "\n".join(parts)
+
+    if check:
+        if existing_content != content:
+            print("boards.py file is not up to date.")
+            print("Please run `script/generate-esp32-boards.py`")
+            sys.exit(1)
+        print("boards.py file is up to date")
+    elif write_file_if_changed(boards_file_path, content):
+        print("ESP32 boards updated successfully.")
 
 
 if __name__ == "__main__":
-    main()
-    print("ESP32 boards updated successfully.")
+    parser = argparse.ArgumentParser()
+    parser.add_argument(
+        "--check",
+        help="Check if the boards.py file is up to date.",
+        action="store_true",
+    )
+    args = parser.parse_args()
+    main(args.check)
diff --git a/script/helpers.py b/script/helpers.py
index ff63bbc5b6..61306b9489 100644
--- a/script/helpers.py
+++ b/script/helpers.py
@@ -52,10 +52,10 @@ def styled(color: str | tuple[str, ...], msg: str, reset: bool = True) -> str:
     return prefix + msg + suffix
 
 
-def print_error_for_file(file: str, body: str | None) -> None:
+def print_error_for_file(file: str | Path, body: str | None) -> None:
     print(
         styled(colorama.Fore.GREEN, "### File ")
-        + styled((colorama.Fore.GREEN, colorama.Style.BRIGHT), file)
+        + styled((colorama.Fore.GREEN, colorama.Style.BRIGHT), str(file))
     )
     print()
     if body is not None:
@@ -139,9 +139,24 @@ def _get_changed_files_github_actions() -> list[str] | None:
     if event_name == "pull_request":
         pr_number = _get_pr_number_from_github_env()
         if pr_number:
-            # Use GitHub CLI to get changed files directly
+            # Try gh pr diff first (faster for small PRs)
             cmd = ["gh", "pr", "diff", pr_number, "--name-only"]
-            return _get_changed_files_from_command(cmd)
+            try:
+                return _get_changed_files_from_command(cmd)
+            except Exception as e:
+                # If it fails due to the 300 file limit, use the API method
+                if "maximum" in str(e) and "files" in str(e):
+                    cmd = [
+                        "gh",
+                        "api",
+                        f"repos/esphome/esphome/pulls/{pr_number}/files",
+                        "--paginate",
+                        "--jq",
+                        ".[].filename",
+                    ]
+                    return _get_changed_files_from_command(cmd)
+                # Re-raise for other errors
+                raise
 
     # For pushes (including squash-and-merge)
     elif event_name == "push":
@@ -338,12 +353,12 @@ def filter_changed(files: list[str]) -> list[str]:
     return files
 
 
-def filter_grep(files: list[str], value: str) -> list[str]:
+def filter_grep(files: list[str], value: list[str]) -> list[str]:
     matched = []
     for file in files:
         with open(file, encoding="utf-8") as handle:
             contents = handle.read()
-        if value in contents:
+        if any(v in contents for v in value):
             matched.append(file)
     return matched
 
@@ -365,9 +380,11 @@ def load_idedata(environment: str) -> dict[str, Any]:
     platformio_ini = Path(root_path) / "platformio.ini"
     temp_idedata = Path(temp_folder) / f"idedata-{environment}.json"
     changed = False
-    if not platformio_ini.is_file() or not temp_idedata.is_file():
-        changed = True
-    elif platformio_ini.stat().st_mtime >= temp_idedata.stat().st_mtime:
+    if (
+        not platformio_ini.is_file()
+        or not temp_idedata.is_file()
+        or platformio_ini.stat().st_mtime >= temp_idedata.stat().st_mtime
+    ):
         changed = True
 
     if "idf" in environment:
@@ -496,7 +513,7 @@ def get_all_dependencies(component_names: set[str]) -> set[str]:
 
     # Set up fake config path for component loading
     root = Path(__file__).parent.parent
-    CORE.config_path = str(root)
+    CORE.config_path = root
     CORE.data[KEY_CORE] = {}
 
     # Keep finding dependencies until no new ones are found
@@ -512,7 +529,16 @@ def get_all_dependencies(component_names: set[str]) -> set[str]:
             new_components.update(dep.split(".")[0] for dep in comp.dependencies)
 
             # Add auto_load components
-            new_components.update(comp.auto_load)
+            auto_load = comp.auto_load
+            if callable(auto_load):
+                import inspect
+
+                if inspect.signature(auto_load).parameters:
+                    auto_load = auto_load(None)
+                else:
+                    auto_load = auto_load()
+
+            new_components.update(auto_load)
 
         # Check if we found any new components
         new_components -= all_components
@@ -530,27 +556,26 @@ def get_components_from_integration_fixtures() -> set[str]:
     Returns:
         Set of component names used in integration test fixtures
     """
-    import yaml
+    from esphome import yaml_util
 
     components: set[str] = set()
     fixtures_dir = Path(__file__).parent.parent / "tests" / "integration" / "fixtures"
 
     for yaml_file in fixtures_dir.glob("*.yaml"):
-        with open(yaml_file) as f:
-            config: dict[str, any] | None = yaml.safe_load(f)
-            if not config:
+        config: dict[str, any] | None = yaml_util.load_yaml(yaml_file)
+        if not config:
+            continue
+
+        # Add all top-level component keys
+        components.update(config.keys())
+
+        # Add platform components (e.g., output.template)
+        for value in config.values():
+            if not isinstance(value, list):
                 continue
 
-            # Add all top-level component keys
-            components.update(config.keys())
-
-            # Add platform components (e.g., output.template)
-            for value in config.values():
-                if not isinstance(value, list):
-                    continue
-
-                for item in value:
-                    if isinstance(item, dict) and "platform" in item:
-                        components.add(item["platform"])
+            for item in value:
+                if isinstance(item, dict) and "platform" in item:
+                    components.add(item["platform"])
 
     return components
diff --git a/script/helpers_zephyr.py b/script/helpers_zephyr.py
index c3ba149005..f72b335e64 100644
--- a/script/helpers_zephyr.py
+++ b/script/helpers_zephyr.py
@@ -25,6 +25,8 @@ int main() { return 0;}
     Path(zephyr_dir / "prj.conf").write_text(
         """
 CONFIG_NEWLIB_LIBC=y
+CONFIG_BT=y
+CONFIG_ADC=y
 """,
         encoding="utf-8",
     )
@@ -41,12 +43,12 @@ CONFIG_NEWLIB_LIBC=y
         return include_paths
 
     def extract_defines(command):
-        defines = []
         define_pattern = re.compile(r"-D\s*([^\s]+)")
-        for match in define_pattern.findall(command):
-            if match not in ("_ASMLANGUAGE"):
-                defines.append(match)
-        return defines
+        return [
+            match
+            for match in define_pattern.findall(command)
+            if match not in ("_ASMLANGUAGE")
+        ]
 
     def find_cxx_path(commands):
         for entry in commands:
@@ -55,6 +57,7 @@ CONFIG_NEWLIB_LIBC=y
             if not cxx_path.endswith("++"):
                 continue
             return cxx_path
+        return None
 
     def get_builtin_include_paths(compiler):
         result = subprocess.run(
@@ -78,14 +81,14 @@ CONFIG_NEWLIB_LIBC=y
         return include_paths
 
     def extract_cxx_flags(command):
-        flags = []
         # Extracts CXXFLAGS from the command string, excluding includes and defines.
         flag_pattern = re.compile(
             r"(-O[0-3s]|-g|-std=[^\s]+|-Wall|-Wextra|-Werror|--[^\s]+|-f[^\s]+|-m[^\s]+|-imacros\s*[^\s]+)"
         )
-        for match in flag_pattern.findall(command):
-            flags.append(match.replace("-imacros ", "-imacros"))
-        return flags
+        return [
+            match.replace("-imacros ", "-imacros")
+            for match in flag_pattern.findall(command)
+        ]
 
     def transform_to_idedata_format(compile_commands):
         cxx_path = find_cxx_path(compile_commands)
diff --git a/script/lint-python b/script/lint-python
index 2c25e4aee0..18281c711e 100755
--- a/script/lint-python
+++ b/script/lint-python
@@ -137,7 +137,7 @@ def main():
     print()
     print("Running pyupgrade...")
     print()
-    PYUPGRADE_TARGET = "--py310-plus"
+    PYUPGRADE_TARGET = "--py311-plus"
     for files in filesets:
         cmd = ["pyupgrade", PYUPGRADE_TARGET] + files
         log = get_err(*cmd)
diff --git a/script/list-components.py b/script/list-components.py
index 66212f44e7..9abb2bc345 100755
--- a/script/list-components.py
+++ b/script/list-components.py
@@ -1,5 +1,6 @@
 #!/usr/bin/env python3
 import argparse
+from collections.abc import Callable
 from pathlib import Path
 import sys
 
@@ -13,7 +14,7 @@ from esphome.const import (
     PLATFORM_ESP8266,
 )
 from esphome.core import CORE
-from esphome.loader import get_component, get_platform
+from esphome.loader import ComponentManifest, get_component, get_platform
 
 
 def filter_component_files(str):
@@ -45,12 +46,35 @@ def add_item_to_components_graph(components_graph, parent, child):
             components_graph[parent].append(child)
 
 
+def resolve_auto_load(
+    auto_load: list[str] | Callable[[], list[str]] | Callable[[dict | None], list[str]],
+    config: dict | None = None,
+) -> list[str]:
+    """Resolve AUTO_LOAD to a list, handling callables with or without config parameter.
+
+    Args:
+        auto_load: The AUTO_LOAD value (list or callable)
+        config: Optional config to pass to callable AUTO_LOAD functions
+
+    Returns:
+        List of component names to auto-load
+    """
+    if not callable(auto_load):
+        return auto_load
+
+    import inspect
+
+    if inspect.signature(auto_load).parameters:
+        return auto_load(config)
+    return auto_load()
+
+
 def create_components_graph():
     # The root directory of the repo
     root = Path(__file__).parent.parent
     components_dir = root / "esphome" / "components"
     # Fake some directory so that get_component works
-    CORE.config_path = str(root)
+    CORE.config_path = root
     # Various configuration to capture different outcomes used by `AUTO_LOAD` function.
     TARGET_CONFIGURATIONS = [
         {KEY_TARGET_FRAMEWORK: None, KEY_TARGET_PLATFORM: None},
@@ -63,7 +87,7 @@ def create_components_graph():
 
     components_graph = {}
     platforms = []
-    components = []
+    components: list[tuple[ComponentManifest, str, Path]] = []
 
     for path in components_dir.iterdir():
         if not path.is_dir():
@@ -92,8 +116,8 @@ def create_components_graph():
 
         for target_config in TARGET_CONFIGURATIONS:
             CORE.data[KEY_CORE] = target_config
-            for auto_load in comp.auto_load:
-                add_item_to_components_graph(components_graph, auto_load, name)
+            for item in resolve_auto_load(comp.auto_load, config=None):
+                add_item_to_components_graph(components_graph, item, name)
         # restore config
         CORE.data[KEY_CORE] = TARGET_CONFIGURATIONS[0]
 
@@ -114,8 +138,8 @@ def create_components_graph():
 
             for target_config in TARGET_CONFIGURATIONS:
                 CORE.data[KEY_CORE] = target_config
-                for auto_load in platform.auto_load:
-                    add_item_to_components_graph(components_graph, auto_load, name)
+                for item in resolve_auto_load(platform.auto_load, config={}):
+                    add_item_to_components_graph(components_graph, item, name)
             # restore config
             CORE.data[KEY_CORE] = TARGET_CONFIGURATIONS[0]
 
@@ -161,33 +185,89 @@ def main():
         "-c",
         "--changed",
         action="store_true",
-        help="List all components required for testing based on changes",
+        help="List all components with dependencies (used by clang-tidy). "
+        "When base test infrastructure changes, returns ALL components.",
+    )
+    parser.add_argument(
+        "--changed-direct",
+        action="store_true",
+        help="List only directly changed components, ignoring infrastructure changes "
+        "(used by CI for isolation decisions)",
+    )
+    parser.add_argument(
+        "--changed-with-deps",
+        action="store_true",
+        help="Output JSON with both directly changed and all changed components "
+        "(with dependencies), ignoring infrastructure changes (used by CI for test determination)",
     )
     parser.add_argument(
         "-b", "--branch", help="Branch to compare changed files against"
     )
     args = parser.parse_args()
 
-    if args.branch and not args.changed:
-        parser.error("--branch requires --changed")
+    if args.branch and not (
+        args.changed or args.changed_direct or args.changed_with_deps
+    ):
+        parser.error(
+            "--branch requires --changed, --changed-direct, or --changed-with-deps"
+        )
 
-    if args.changed:
-        # When --changed is passed, only get the changed files
+    if args.changed or args.changed_direct or args.changed_with_deps:
+        # When --changed* is passed, only get the changed files
         changed = changed_files(args.branch)
 
-        # If any base test file(s) changed, there's no need to filter out components
-        if any("tests/test_build_components" in file for file in changed):
-            # Need to get all component files
+        # If any base test file(s) changed, we need to check all components
+        # BUT only for --changed (used by clang-tidy for comprehensive checking)
+        # NOT for --changed-direct or --changed-with-deps (used by CI for targeted testing)
+        #
+        # Flag usage:
+        # - --changed: Used by clang-tidy (script/helpers.py get_changed_components)
+        #   Returns: All components with dependencies when base test files change
+        #   Reason: Test infrastructure changes may affect any component
+        #
+        # - --changed-direct: Used by CI isolation (script/determine-jobs.py)
+        #   Returns: Only components with actual code changes (not infrastructure)
+        #   Reason: Only directly changed components need isolated testing
+        #
+        # - --changed-with-deps: Used by CI test determination (script/determine-jobs.py)
+        #   Returns: Components with code changes + their dependencies (not infrastructure)
+        #   Reason: CI needs to test changed components and their dependents
+        base_test_changed = any(
+            "tests/test_build_components" in file for file in changed
+        )
+
+        if base_test_changed and not args.changed_direct and not args.changed_with_deps:
+            # Base test infrastructure changed - load all component files
+            # This is for --changed (clang-tidy) which needs comprehensive checking
             files = get_all_component_files()
         else:
-            # Only look at changed component files
+            # Only look at changed component files (ignore infrastructure changes)
+            # For --changed-direct: only actual component code changes matter (for isolation)
+            # For --changed-with-deps: only actual component code changes matter (for testing)
             files = [f for f in changed if filter_component_files(f)]
     else:
         # Get all component files
         files = get_all_component_files()
 
-    for c in get_components(files, args.changed):
-        print(c)
+    if args.changed_with_deps:
+        # Return JSON with both directly changed and all changed components
+        import json
+
+        directly_changed = get_components(files, False)
+        all_changed = get_components(files, True)
+        output = {
+            "directly_changed": directly_changed,
+            "all_changed": all_changed,
+        }
+        print(json.dumps(output))
+    elif args.changed_direct:
+        # Return only directly changed components (without dependencies)
+        for c in get_components(files, False):
+            print(c)
+    else:
+        # Return all changed components (with dependencies) - default behavior
+        for c in get_components(files, args.changed):
+            print(c)
 
 
 if __name__ == "__main__":
diff --git a/script/merge_component_configs.py b/script/merge_component_configs.py
new file mode 100755
index 0000000000..59774edba9
--- /dev/null
+++ b/script/merge_component_configs.py
@@ -0,0 +1,443 @@
+#!/usr/bin/env python3
+"""Merge multiple component test configurations into a single test file.
+
+This script combines multiple component test files that use the same common bus
+configurations into a single merged test file. This allows testing multiple
+compatible components together, reducing CI build time.
+
+The merger handles:
+- Component-specific substitutions (prefixing to avoid conflicts)
+- Multiple instances of component configurations
+- Shared common bus packages (included only once)
+- Platform-specific configurations
+- Uses ESPHome's built-in merge_config for proper YAML merging
+"""
+
+from __future__ import annotations
+
+import argparse
+from functools import lru_cache
+from pathlib import Path
+import re
+import sys
+from typing import Any
+
+# Add esphome to path so we can import from it
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from esphome import yaml_util
+from esphome.config_helpers import merge_config
+from script.analyze_component_buses import PACKAGE_DEPENDENCIES, get_common_bus_packages
+
+# Prefix for dependency markers in package tracking
+# Used to mark packages that are included transitively (e.g., uart via modbus)
+DEPENDENCY_MARKER_PREFIX = "_dep_"
+
+
+def load_yaml_file(yaml_file: Path) -> dict:
+    """Load YAML file using ESPHome's YAML loader.
+
+    Args:
+        yaml_file: Path to the YAML file
+
+    Returns:
+        Parsed YAML as dictionary
+    """
+    if not yaml_file.exists():
+        raise FileNotFoundError(f"YAML file not found: {yaml_file}")
+
+    return yaml_util.load_yaml(yaml_file)
+
+
+@lru_cache(maxsize=256)
+def get_component_packages(
+    component_name: str, platform: str, tests_dir_str: str
+) -> dict:
+    """Get packages dict from a component's test file with caching.
+
+    This function is cached to avoid re-loading and re-parsing the same file
+    multiple times when extracting packages during cross-bus merging.
+
+    Args:
+        component_name: Name of the component
+        platform: Platform name (e.g., "esp32-idf")
+        tests_dir_str: String path to tests/components directory (must be string for cache hashability)
+
+    Returns:
+        Dictionary with 'packages' key containing the raw packages dict from the YAML,
+        or empty dict if no packages section exists
+    """
+    tests_dir = Path(tests_dir_str)
+    test_file = tests_dir / component_name / f"test.{platform}.yaml"
+    comp_data = load_yaml_file(test_file)
+
+    if "packages" not in comp_data or not isinstance(comp_data["packages"], dict):
+        return {}
+
+    return comp_data["packages"]
+
+
+def extract_packages_from_yaml(data: dict) -> dict[str, str]:
+    """Extract COMMON BUS package includes from parsed YAML.
+
+    Only extracts packages that are from test_build_components/common/,
+    ignoring component-specific packages.
+
+    Args:
+        data: Parsed YAML dictionary
+
+    Returns:
+        Dictionary mapping package name to include path (as string representation)
+        Only includes common bus packages (i2c, spi, uart, etc.)
+    """
+    if "packages" not in data:
+        return {}
+
+    packages_value = data["packages"]
+    if not isinstance(packages_value, dict):
+        # List format doesn't include common bus packages (those use dict format)
+        return {}
+
+    # Get common bus package names (cached)
+    common_bus_packages = get_common_bus_packages()
+    packages = {}
+
+    # Dictionary format: packages: {name: value}
+    for name, value in packages_value.items():
+        # Only include common bus packages, ignore component-specific ones
+        if name not in common_bus_packages:
+            continue
+        packages[name] = str(value)
+        # Also track package dependencies (e.g., modbus includes uart)
+        if name not in PACKAGE_DEPENDENCIES:
+            continue
+        for dep in PACKAGE_DEPENDENCIES[name]:
+            if dep not in common_bus_packages:
+                continue
+            # Mark as included via dependency
+            packages[f"{DEPENDENCY_MARKER_PREFIX}{dep}"] = f"(included via {name})"
+
+    return packages
+
+
+def prefix_substitutions_in_dict(
+    data: Any, prefix: str, exclude: set[str] | None = None
+) -> Any:
+    """Recursively prefix all substitution references in a data structure.
+
+    Args:
+        data: YAML data structure (dict, list, or scalar)
+        prefix: Prefix to add to substitution names
+        exclude: Set of substitution names to exclude from prefixing
+
+    Returns:
+        Data structure with prefixed substitution references
+    """
+    if exclude is None:
+        exclude = set()
+
+    def replace_sub(text: str) -> str:
+        """Replace substitution references in a string."""
+
+        def replace_match(match):
+            sub_name = match.group(1)
+            if sub_name in exclude:
+                return match.group(0)
+            # Always use braced format in output for consistency
+            return f"${{{prefix}_{sub_name}}}"
+
+        # Match both ${substitution} and $substitution formats
+        return re.sub(r"\$\{?(\w+)\}?", replace_match, text)
+
+    if isinstance(data, dict):
+        result = {}
+        for key, value in data.items():
+            result[key] = prefix_substitutions_in_dict(value, prefix, exclude)
+        return result
+    if isinstance(data, list):
+        return [prefix_substitutions_in_dict(item, prefix, exclude) for item in data]
+    if isinstance(data, str):
+        return replace_sub(data)
+    return data
+
+
+def deduplicate_by_id(data: dict) -> dict:
+    """Deduplicate list items with the same ID.
+
+    Keeps only the first occurrence of each ID. If items with the same ID
+    are identical, this silently deduplicates. If they differ, the first
+    one is kept (ESPHome's validation will catch if this causes issues).
+
+    Args:
+        data: Parsed config dictionary
+
+    Returns:
+        Config with deduplicated lists
+    """
+    if not isinstance(data, dict):
+        return data
+
+    result = {}
+    for key, value in data.items():
+        if isinstance(value, list):
+            # Check for items with 'id' field
+            seen_ids = set()
+            deduped_list = []
+
+            for item in value:
+                if isinstance(item, dict) and "id" in item:
+                    item_id = item["id"]
+                    if item_id not in seen_ids:
+                        seen_ids.add(item_id)
+                        deduped_list.append(item)
+                    # else: skip duplicate ID (keep first occurrence)
+                else:
+                    # No ID, just add it
+                    deduped_list.append(item)
+
+            result[key] = deduped_list
+        elif isinstance(value, dict):
+            # Recursively deduplicate nested dicts
+            result[key] = deduplicate_by_id(value)
+        else:
+            result[key] = value
+
+    return result
+
+
+def merge_component_configs(
+    component_names: list[str],
+    platform: str,
+    tests_dir: Path,
+    output_file: Path,
+) -> None:
+    """Merge multiple component test configs into a single file.
+
+    Args:
+        component_names: List of component names to merge
+        platform: Platform to merge for (e.g., "esp32-ard")
+        tests_dir: Path to tests/components directory
+        output_file: Path to output merged config file
+    """
+    if not component_names:
+        raise ValueError("No components specified")
+
+    # Track packages to ensure they're identical
+    all_packages = None
+
+    # Start with empty config
+    merged_config_data = {}
+
+    # Convert tests_dir to string for caching
+    tests_dir_str = str(tests_dir)
+
+    # Process each component
+    for comp_name in component_names:
+        comp_dir = tests_dir / comp_name
+        test_file = comp_dir / f"test.{platform}.yaml"
+
+        if not test_file.exists():
+            raise FileNotFoundError(f"Test file not found: {test_file}")
+
+        # Load the component's test file
+        comp_data = load_yaml_file(test_file)
+
+        # Merge packages from all components (cross-bus merging)
+        # Components can have different packages (e.g., one with ble, another with uart)
+        # as long as they don't conflict (checked by are_buses_compatible before calling this)
+        comp_packages = extract_packages_from_yaml(comp_data)
+
+        if all_packages is None:
+            # First component - initialize package dict
+            all_packages = comp_packages if comp_packages else {}
+        elif comp_packages:
+            # Merge packages - combine all unique package types
+            # If both have the same package type, verify they're identical
+            for pkg_name, pkg_config in comp_packages.items():
+                if pkg_name in all_packages:
+                    # Same package type - verify config matches
+                    if all_packages[pkg_name] != pkg_config:
+                        raise ValueError(
+                            f"Component {comp_name} has conflicting config for package '{pkg_name}'. "
+                            f"Expected: {all_packages[pkg_name]}, Got: {pkg_config}. "
+                            f"Components with conflicting bus configs cannot be merged."
+                        )
+                else:
+                    # New package type - add it
+                    all_packages[pkg_name] = pkg_config
+
+        # Handle $component_dir by replacing with absolute path
+        # This allows components that use local file references to be grouped
+        comp_abs_dir = str(comp_dir.absolute())
+
+        # Save top-level substitutions BEFORE expanding packages
+        # In ESPHome, top-level substitutions override package substitutions
+        top_level_subs = (
+            comp_data["substitutions"].copy()
+            if "substitutions" in comp_data and comp_data["substitutions"] is not None
+            else {}
+        )
+
+        # Expand packages - but we'll restore substitution priority after
+        if "packages" in comp_data:
+            packages_value = comp_data["packages"]
+
+            if isinstance(packages_value, dict):
+                # Dict format - check each package
+                common_bus_packages = get_common_bus_packages()
+                for pkg_name, pkg_value in list(packages_value.items()):
+                    if pkg_name in common_bus_packages:
+                        continue
+                    if not isinstance(pkg_value, dict):
+                        continue
+                    # Component-specific package - expand its content into top level
+                    comp_data = merge_config(comp_data, pkg_value)
+            elif isinstance(packages_value, list):
+                # List format - expand all package includes
+                for pkg_value in packages_value:
+                    if not isinstance(pkg_value, dict):
+                        continue
+                    comp_data = merge_config(comp_data, pkg_value)
+
+            # Remove all packages (common will be re-added at the end)
+            del comp_data["packages"]
+
+        # Restore top-level substitution priority
+        # Top-level substitutions override any from packages
+        if "substitutions" not in comp_data or comp_data["substitutions"] is None:
+            comp_data["substitutions"] = {}
+
+        # Merge: package subs as base, top-level subs override
+        comp_data["substitutions"].update(top_level_subs)
+
+        # Now prefix the final merged substitutions
+        comp_data["substitutions"] = {
+            f"{comp_name}_{sub_name}": sub_value
+            for sub_name, sub_value in comp_data["substitutions"].items()
+        }
+
+        # Add component_dir substitution with absolute path for this component
+        comp_data["substitutions"][f"{comp_name}_component_dir"] = comp_abs_dir
+
+        # Prefix substitution references throughout the config
+        comp_data = prefix_substitutions_in_dict(comp_data, comp_name)
+
+        # Use ESPHome's merge_config to merge this component into the result
+        # merge_config handles list merging with ID-based deduplication automatically
+        merged_config_data = merge_config(merged_config_data, comp_data)
+
+    # Add merged packages back (union of all component packages)
+    # IMPORTANT: Only include common bus packages (spi, i2c, uart, etc.)
+    # Do NOT re-add component-specific packages as they contain unprefixed $component_dir refs
+    if all_packages:
+        # Build packages dict from merged all_packages
+        # all_packages is a dict mapping package_name -> str(package_value)
+        # We need to reconstruct the actual package values by loading them from any component
+        # Since packages with the same name must have identical configs (verified above),
+        # we can load the package value from the first component that has each package
+        common_bus_packages = get_common_bus_packages()
+        merged_packages: dict[str, Any] = {}
+
+        # Collect packages that are included as dependencies
+        # If modbus is present, uart is included via modbus.packages.uart
+        packages_to_skip: set[str] = set()
+        for pkg_name in all_packages:
+            if pkg_name.startswith(DEPENDENCY_MARKER_PREFIX):
+                # Extract the actual package name (remove _dep_ prefix)
+                dep_name = pkg_name[len(DEPENDENCY_MARKER_PREFIX) :]
+                packages_to_skip.add(dep_name)
+
+        for pkg_name in all_packages:
+            # Skip dependency markers
+            if pkg_name.startswith(DEPENDENCY_MARKER_PREFIX):
+                continue
+            # Skip non-common-bus packages
+            if pkg_name not in common_bus_packages:
+                continue
+            # Skip packages that are included as dependencies of other packages
+            # This prevents duplicate definitions (e.g., uart via modbus + uart separately)
+            if pkg_name in packages_to_skip:
+                continue
+
+            # Find a component that has this package and extract its value
+            # Uses cached lookup to avoid re-loading the same files
+            for comp_name in component_names:
+                comp_packages = get_component_packages(
+                    comp_name, platform, tests_dir_str
+                )
+                if pkg_name in comp_packages:
+                    merged_packages[pkg_name] = comp_packages[pkg_name]
+                    break
+
+        if merged_packages:
+            merged_config_data["packages"] = merged_packages
+
+    # Deduplicate items with same ID (keeps first occurrence)
+    merged_config_data = deduplicate_by_id(merged_config_data)
+
+    # Remove esphome section since it will be provided by the wrapper file
+    # The wrapper file includes this merged config via packages and provides
+    # the proper esphome: section with name, platform, etc.
+    if "esphome" in merged_config_data:
+        del merged_config_data["esphome"]
+
+    # Write merged config
+    output_file.parent.mkdir(parents=True, exist_ok=True)
+    yaml_content = yaml_util.dump(merged_config_data)
+    output_file.write_text(yaml_content)
+
+    print(f"Successfully merged {len(component_names)} components into {output_file}")
+
+
+def main() -> None:
+    """Main entry point."""
+    parser = argparse.ArgumentParser(
+        description="Merge multiple component test configs into a single file"
+    )
+    parser.add_argument(
+        "--components",
+        "-c",
+        required=True,
+        help="Comma-separated list of component names to merge",
+    )
+    parser.add_argument(
+        "--platform",
+        "-p",
+        required=True,
+        help="Platform to merge for (e.g., esp32-ard)",
+    )
+    parser.add_argument(
+        "--output",
+        "-o",
+        required=True,
+        type=Path,
+        help="Output file path for merged config",
+    )
+    parser.add_argument(
+        "--tests-dir",
+        type=Path,
+        default=Path("tests/components"),
+        help="Path to tests/components directory",
+    )
+
+    args = parser.parse_args()
+
+    component_names = [c.strip() for c in args.components.split(",")]
+
+    try:
+        merge_component_configs(
+            component_names=component_names,
+            platform=args.platform,
+            tests_dir=args.tests_dir,
+            output_file=args.output,
+        )
+    except Exception as e:
+        print(f"Error merging configs: {e}", file=sys.stderr)
+        import traceback
+
+        traceback.print_exc()
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()
diff --git a/script/platformio_install_deps.py b/script/platformio_install_deps.py
index ed133ecb47..8f7261efc3 100755
--- a/script/platformio_install_deps.py
+++ b/script/platformio_install_deps.py
@@ -55,4 +55,6 @@ for section in config.sections():
             tools.append("-t")
             tools.append(tool)
 
-subprocess.check_call(["platformio", "pkg", "install", "-g", *libs, *platforms, *tools])
+subprocess.check_call(
+    ["platformio", "pkg", "install", "-g", *libs, *platforms, *tools], close_fds=False
+)
diff --git a/script/run-in-env.py b/script/run-in-env.py
index d9bd01a62f..886e65db27 100755
--- a/script/run-in-env.py
+++ b/script/run-in-env.py
@@ -13,7 +13,7 @@ def find_and_activate_virtualenv():
     try:
         # Get the top-level directory of the git repository
         my_path = subprocess.check_output(
-            ["git", "rev-parse", "--show-toplevel"], text=True
+            ["git", "rev-parse", "--show-toplevel"], text=True, close_fds=False
         ).strip()
     except subprocess.CalledProcessError:
         print(
@@ -44,7 +44,7 @@ def find_and_activate_virtualenv():
 def run_command():
     # Execute the remaining arguments in the new environment
     if len(sys.argv) > 1:
-        subprocess.run(sys.argv[1:], check=False)
+        subprocess.run(sys.argv[1:], check=False, close_fds=False)
     else:
         print(
             "No command provided to run in the virtual environment.",
diff --git a/script/setup b/script/setup
index b17d3235a7..8cad7017ff 100755
--- a/script/setup
+++ b/script/setup
@@ -6,7 +6,7 @@ set -e
 cd "$(dirname "$0")/.."
 if [ ! -n "$VIRTUAL_ENV" ]; then
   if [ -x "$(command -v uv)" ]; then
-    uv venv venv
+    uv venv --seed venv
   else
     python3 -m venv venv
   fi
@@ -22,8 +22,6 @@ uv pip install -e ".[dev,test]" --config-settings editable_mode=compat
 
 pre-commit install
 
-script/platformio_install_deps.py platformio.ini --libraries --tools --platforms
-
 mkdir -p .temp
 
 echo
diff --git a/script/setup.bat b/script/setup.bat
index f89d5aea1a..003ea31b36 100644
--- a/script/setup.bat
+++ b/script/setup.bat
@@ -19,8 +19,6 @@ pip3 install -e ".[dev,test]" --config-settings editable_mode=compat
 
 pre-commit install
 
-python script/platformio_install_deps.py platformio.ini --libraries --tools --platforms
-
 echo .
 echo .
 echo Virtual environment created. Run 'venv/Scripts/activate' to use it.
diff --git a/script/split_components_for_ci.py b/script/split_components_for_ci.py
new file mode 100755
index 0000000000..dff46d3619
--- /dev/null
+++ b/script/split_components_for_ci.py
@@ -0,0 +1,384 @@
+#!/usr/bin/env python3
+"""Split components into batches with intelligent grouping.
+
+This script analyzes components to identify which ones share common bus configurations
+and intelligently groups them into batches to maximize the efficiency of the
+component grouping system in CI.
+
+Components with the same bus signature are placed in the same batch whenever possible,
+allowing the test_build_components.py script to merge them into single builds.
+"""
+
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+import json
+from pathlib import Path
+import sys
+
+# Add esphome to path
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from script.analyze_component_buses import (
+    ISOLATED_COMPONENTS,
+    ISOLATED_SIGNATURE_PREFIX,
+    NO_BUSES_SIGNATURE,
+    analyze_all_components,
+    create_grouping_signature,
+    merge_compatible_bus_groups,
+)
+
+# Weighting for batch creation
+# Isolated components can't be grouped/merged, so they count as 10x
+# Groupable components can be merged into single builds, so they count as 1x
+ISOLATED_WEIGHT = 10
+GROUPABLE_WEIGHT = 1
+
+# Platform used for batching (platform-agnostic batching)
+# Batches are split across CI runners and each runner tests all platforms
+ALL_PLATFORMS = "all"
+
+
+def has_test_files(component_name: str, tests_dir: Path) -> bool:
+    """Check if a component has test files.
+
+    Args:
+        component_name: Name of the component
+        tests_dir: Path to tests/components directory
+
+    Returns:
+        True if the component has test.*.yaml files
+    """
+    component_dir = tests_dir / component_name
+    if not component_dir.exists() or not component_dir.is_dir():
+        return False
+
+    # Check for test.*.yaml files
+    return any(component_dir.glob("test.*.yaml"))
+
+
+def create_intelligent_batches(
+    components: list[str],
+    tests_dir: Path,
+    batch_size: int = 40,
+    directly_changed: set[str] | None = None,
+) -> tuple[list[list[str]], dict[tuple[str, str], list[str]]]:
+    """Create batches optimized for component grouping.
+
+    Args:
+        components: List of component names to batch
+        tests_dir: Path to tests/components directory
+        batch_size: Target size for each batch
+        directly_changed: Set of directly changed components (for logging only)
+
+    Returns:
+        Tuple of (batches, signature_groups) where:
+        - batches: List of component batches (lists of component names)
+        - signature_groups: Dict mapping (platform, signature) to component lists
+    """
+    # Filter out components without test files
+    # Platform components like 'climate' and 'climate_ir' don't have test files
+    components_with_tests = [
+        comp for comp in components if has_test_files(comp, tests_dir)
+    ]
+
+    # Log filtered components to stderr for debugging
+    if len(components_with_tests) < len(components):
+        filtered_out = set(components) - set(components_with_tests)
+        print(
+            f"Note: Filtered {len(filtered_out)} components without test files: "
+            f"{', '.join(sorted(filtered_out))}",
+            file=sys.stderr,
+        )
+
+    # Analyze all components to get their bus signatures
+    component_buses, non_groupable, _direct_bus_components = analyze_all_components(
+        tests_dir
+    )
+
+    # Group components by their bus signature ONLY (ignore platform)
+    # All platforms will be tested by test_build_components.py for each batch
+    # Key: (platform, signature), Value: list of components
+    # We use ALL_PLATFORMS since batching is platform-agnostic
+    signature_groups: dict[tuple[str, str], list[str]] = defaultdict(list)
+
+    for component in components_with_tests:
+        # Components that can't be grouped get unique signatures
+        # This includes:
+        # - Manually curated ISOLATED_COMPONENTS
+        # - Automatically detected non_groupable components
+        # - Directly changed components (passed via --isolate in CI)
+        # These can share a batch/runner but won't be grouped/merged
+        is_isolated = (
+            component in ISOLATED_COMPONENTS
+            or component in non_groupable
+            or (directly_changed and component in directly_changed)
+        )
+        if is_isolated:
+            signature_groups[
+                (ALL_PLATFORMS, f"{ISOLATED_SIGNATURE_PREFIX}{component}")
+            ].append(component)
+            continue
+
+        # Get signature from any platform (they should all have the same buses)
+        # Components not in component_buses were filtered out by has_test_files check
+        comp_platforms = component_buses[component]
+        for platform, buses in comp_platforms.items():
+            if buses:
+                signature = create_grouping_signature({platform: buses}, platform)
+                # Group by signature only - platform doesn't matter for batching
+                # Use ALL_PLATFORMS since we're batching across all platforms
+                signature_groups[(ALL_PLATFORMS, signature)].append(component)
+                break  # Only use first platform for grouping
+        else:
+            # No buses found for any platform - can be grouped together
+            signature_groups[(ALL_PLATFORMS, NO_BUSES_SIGNATURE)].append(component)
+
+    # Merge compatible bus groups (cross-bus optimization)
+    # This allows components with different buses (ble + uart) to be batched together
+    # improving the efficiency of test_build_components.py grouping
+    signature_groups = merge_compatible_bus_groups(signature_groups)
+
+    # Create batches by keeping signature groups together
+    # Components with the same signature stay in the same batches
+    batches = []
+
+    # Sort signature groups to prioritize groupable components
+    # 1. Put "isolated_*" signatures last (can't be grouped with others)
+    # 2. Sort groupable signatures by size (largest first)
+    # 3. "no_buses" components CAN be grouped together
+    def sort_key(item):
+        (_platform, signature), components = item
+        is_isolated = signature.startswith(ISOLATED_SIGNATURE_PREFIX)
+        # Put "isolated_*" last (1), groupable first (0)
+        # Within each category, sort by size (largest first)
+        return (is_isolated, -len(components))
+
+    sorted_groups = sorted(signature_groups.items(), key=sort_key)
+
+    # Strategy: Create batches using weighted sizes
+    # - Isolated components count as 10x (since they can't be grouped/merged)
+    # - Groupable components count as 1x (can be merged into single builds)
+    # - This distributes isolated components across more runners
+    # - Ensures each runner has a good mix of groupable vs isolated components
+
+    current_batch = []
+    current_weight = 0
+
+    for (_platform, signature), group_components in sorted_groups:
+        is_isolated = signature.startswith(ISOLATED_SIGNATURE_PREFIX)
+        weight_per_component = ISOLATED_WEIGHT if is_isolated else GROUPABLE_WEIGHT
+
+        for component in group_components:
+            # Check if adding this component would exceed the batch size
+            if current_weight + weight_per_component > batch_size and current_batch:
+                # Start a new batch
+                batches.append(current_batch)
+                current_batch = []
+                current_weight = 0
+
+            # Add component to current batch
+            current_batch.append(component)
+            current_weight += weight_per_component
+
+    # Don't forget the last batch
+    if current_batch:
+        batches.append(current_batch)
+
+    return batches, signature_groups
+
+
+def main() -> int:
+    """Main entry point."""
+    parser = argparse.ArgumentParser(
+        description="Split components into intelligent batches for CI testing"
+    )
+    parser.add_argument(
+        "--components",
+        "-c",
+        required=True,
+        help="JSON array of component names",
+    )
+    parser.add_argument(
+        "--batch-size",
+        "-b",
+        type=int,
+        default=40,
+        help="Target batch size (default: 40, weighted)",
+    )
+    parser.add_argument(
+        "--tests-dir",
+        type=Path,
+        default=Path("tests/components"),
+        help="Path to tests/components directory",
+    )
+    parser.add_argument(
+        "--directly-changed",
+        help="JSON array of directly changed component names (for logging only)",
+    )
+    parser.add_argument(
+        "--output",
+        "-o",
+        choices=["json", "github"],
+        default="github",
+        help="Output format (json or github for GitHub Actions)",
+    )
+
+    args = parser.parse_args()
+
+    # Parse component list from JSON
+    try:
+        components = json.loads(args.components)
+    except json.JSONDecodeError as e:
+        print(f"Error parsing components JSON: {e}", file=sys.stderr)
+        return 1
+
+    if not isinstance(components, list):
+        print("Components must be a JSON array", file=sys.stderr)
+        return 1
+
+    # Parse directly changed components list from JSON (if provided)
+    directly_changed = None
+    if args.directly_changed:
+        try:
+            directly_changed = set(json.loads(args.directly_changed))
+        except json.JSONDecodeError as e:
+            print(f"Error parsing directly-changed JSON: {e}", file=sys.stderr)
+            return 1
+
+    # Create intelligent batches
+    batches, signature_groups = create_intelligent_batches(
+        components=components,
+        tests_dir=args.tests_dir,
+        batch_size=args.batch_size,
+        directly_changed=directly_changed,
+    )
+
+    # Convert batches to space-separated strings for CI
+    batch_strings = [" ".join(batch) for batch in batches]
+
+    if args.output == "json":
+        # Output as JSON array
+        print(json.dumps(batch_strings))
+    else:
+        # Output for GitHub Actions (set output)
+        output_json = json.dumps(batch_strings)
+        print(f"components={output_json}")
+
+    # Print summary to stderr so it shows in CI logs
+    # Count actual components being batched
+    actual_components = sum(len(batch.split()) for batch in batch_strings)
+
+    # Re-analyze to get isolated component counts for summary
+    _, non_groupable, _ = analyze_all_components(args.tests_dir)
+
+    # Show grouping details
+    print("\n=== Component Grouping Details ===", file=sys.stderr)
+    # Sort groups by signature for readability
+    groupable_groups = []
+    isolated_groups = []
+    for (platform, signature), group_comps in sorted(signature_groups.items()):
+        if signature.startswith(ISOLATED_SIGNATURE_PREFIX):
+            isolated_groups.append((signature, group_comps))
+        else:
+            groupable_groups.append((signature, group_comps))
+
+    if groupable_groups:
+        print(
+            f"\nGroupable signatures ({len(groupable_groups)} merged groups after cross-bus optimization):",
+            file=sys.stderr,
+        )
+        for signature, group_comps in sorted(
+            groupable_groups, key=lambda x: (-len(x[1]), x[0])
+        ):
+            # Check if this is a merged signature (contains +)
+            is_merged = "+" in signature and signature != NO_BUSES_SIGNATURE
+            # Special handling for no_buses components
+            if signature == NO_BUSES_SIGNATURE:
+                print(
+                    f"  [{signature}]: {len(group_comps)} components (used as fillers across batches)",
+                    file=sys.stderr,
+                )
+            else:
+                merge_indicator = " [MERGED]" if is_merged else ""
+                print(
+                    f"  [{signature}]{merge_indicator}: {len(group_comps)} components",
+                    file=sys.stderr,
+                )
+            # Show first few components as examples
+            examples = ", ".join(sorted(group_comps)[:8])
+            if len(group_comps) > 8:
+                examples += f", ... (+{len(group_comps) - 8} more)"
+            print(f"    → {examples}", file=sys.stderr)
+
+    if isolated_groups:
+        print(
+            f"\nIsolated components ({len(isolated_groups)} components - tested individually):",
+            file=sys.stderr,
+        )
+        isolated_names = sorted(
+            [comp for _, comps in isolated_groups for comp in comps]
+        )
+        # Group isolated components for compact display
+        for i in range(0, len(isolated_names), 10):
+            chunk = isolated_names[i : i + 10]
+            print(f"  {', '.join(chunk)}", file=sys.stderr)
+
+    # Count isolated vs groupable components
+    all_batched_components = [comp for batch in batches for comp in batch]
+    isolated_count = sum(
+        1
+        for comp in all_batched_components
+        if comp in ISOLATED_COMPONENTS
+        or comp in non_groupable
+        or (directly_changed and comp in directly_changed)
+    )
+    groupable_count = actual_components - isolated_count
+
+    print("\n=== Intelligent Batch Summary ===", file=sys.stderr)
+    print(f"Total components requested: {len(components)}", file=sys.stderr)
+    print(f"Components with test files: {actual_components}", file=sys.stderr)
+
+    # Show breakdown of directly changed vs dependencies
+    if directly_changed:
+        direct_count = sum(
+            1 for comp in all_batched_components if comp in directly_changed
+        )
+        dep_count = actual_components - direct_count
+        direct_comps = [
+            comp for comp in all_batched_components if comp in directly_changed
+        ]
+        dep_comps = [
+            comp for comp in all_batched_components if comp not in directly_changed
+        ]
+        print(
+            f"  - Direct changes: {direct_count} ({', '.join(sorted(direct_comps))})",
+            file=sys.stderr,
+        )
+        print(
+            f"  - Dependencies: {dep_count} ({', '.join(sorted(dep_comps))})",
+            file=sys.stderr,
+        )
+
+    print(f"  - Groupable (weight=1): {groupable_count}", file=sys.stderr)
+    print(f"  - Isolated (weight=10): {isolated_count}", file=sys.stderr)
+    if actual_components < len(components):
+        print(
+            f"Components skipped (no test files): {len(components) - actual_components}",
+            file=sys.stderr,
+        )
+    print(f"Number of batches: {len(batches)}", file=sys.stderr)
+    print(f"Batch size target (weighted): {args.batch_size}", file=sys.stderr)
+    if len(batches) > 0:
+        print(
+            f"Average components per batch: {actual_components / len(batches):.1f}",
+            file=sys.stderr,
+        )
+    print(file=sys.stderr)
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/script/test_build_components b/script/test_build_components
deleted file mode 100755
index 3796280176..0000000000
--- a/script/test_build_components
+++ /dev/null
@@ -1,106 +0,0 @@
-#!/usr/bin/env bash
-
-set -e
-
-help() {
-  echo "Usage: $0 [-e ] [-c ] [-t ]" 1>&2
-  echo 1>&2
-  echo "  - e - Parameter for esphome command. Default compile. Common alternative is config." 1>&2
-  echo "  - c - Component folder name to test. Default *. E.g. '-c logger'." 1>&2
-  echo "  - t - Target name to test. Put '-t list' to display all possibilities. E.g. '-t esp32-s2-idf-51'." 1>&2
-  exit 1
-}
-
-# Parse parameter:
-# - `e` - Parameter for `esphome` command. Default `compile`. Common alternative is `config`.
-# - `c` - Component folder name to test. Default `*`.
-esphome_command="compile"
-target_component="*"
-while getopts e:c:t: flag
-do
-    case $flag in
-        e) esphome_command=${OPTARG};;
-        c) target_component=${OPTARG};;
-        t) requested_target_platform=${OPTARG};;
-        \?) help;;
-    esac
-done
-
-cd "$(dirname "$0")/.."
-
-if ! [ -d "./tests/test_build_components/build" ]; then
-  mkdir ./tests/test_build_components/build
-fi
-
-start_esphome() {
-  if [ -n "$requested_target_platform" ] && [ "$requested_target_platform" != "$target_platform_with_version" ]; then
-    echo "Skipping $target_platform_with_version"
-    return
-  fi
-  # create dynamic yaml file in `build` folder.
-  # `./tests/test_build_components/build/[target_component].[test_name].[target_platform_with_version].yaml`
-  component_test_file="./tests/test_build_components/build/$target_component.$test_name.$target_platform_with_version.yaml"
-
-  cp $target_platform_file $component_test_file
-  if [[ "$OSTYPE" == "darwin"* ]]; then
-    # macOS sed is...different
-    sed -i '' "s!\$component_test_file!../../.$f!g" $component_test_file
-  else
-    sed -i "s!\$component_test_file!../../.$f!g" $component_test_file
-  fi
-
-  # Start esphome process
-  echo "> [$target_component] [$test_name] [$target_platform_with_version]"
-  set -x
-  # TODO: Validate escape of Command line substitution value
-  python3 -m esphome -s component_name $target_component -s component_dir ../../components/$target_component -s test_name $test_name -s target_platform $target_platform $esphome_command $component_test_file
-  { set +x; } 2>/dev/null
-}
-
-# Find all test yaml files.
-# - `./tests/components/[target_component]/[test_name].[target_platform].yaml`
-# - `./tests/components/[target_component]/[test_name].all.yaml`
-for f in ./tests/components/$target_component/*.*.yaml; do
-  [ -f "$f" ] || continue
-  IFS='/' read -r -a folder_name <<< "$f"
-  target_component="${folder_name[3]}"
-
-  IFS='.' read -r -a file_name <<< "${folder_name[4]}"
-  test_name="${file_name[0]}"
-  target_platform="${file_name[1]}"
-  file_name_parts=${#file_name[@]}
-
-  if [ "$target_platform" = "all" ] || [ $file_name_parts = 2 ]; then
-    # Test has *not* defined a specific target platform. Need to run tests for all possible target platforms.
-
-    for target_platform_file in ./tests/test_build_components/build_components_base.*.yaml; do
-      IFS='/' read -r -a folder_name <<< "$target_platform_file"
-      IFS='.' read -r -a file_name <<< "${folder_name[3]}"
-      target_platform="${file_name[1]}"
-
-      start_esphome
-    done
-
-  else
-    # Test has defined a specific target platform.
-
-    # Validate we have a base test yaml for selected platform.
-    # The target_platform is sourced from the following location.
-    # 1. `./tests/test_build_components/build_components_base.[target_platform].yaml`
-    # 2. `./tests/test_build_components/build_components_base.[target_platform]-ard.yaml`
-    target_platform_file="./tests/test_build_components/build_components_base.$target_platform.yaml"
-    if ! [ -f "$target_platform_file" ]; then
-      echo "No base test file [./tests/test_build_components/build_components_base.$target_platform.yaml] for component test [$f] found."
-      exit 1
-    fi
-
-    for target_platform_file in ./tests/test_build_components/build_components_base.$target_platform*.yaml; do
-      # trim off "./tests/test_build_components/build_components_base." prefix
-      target_platform_with_version=${target_platform_file:52}
-      # ...now remove suffix starting with "." leaving just the test target hardware and software platform (possibly with version)
-      # For example: "esp32-s3-idf-50"
-      target_platform_with_version=${target_platform_with_version%.*}
-      start_esphome
-    done
-  fi
-done
diff --git a/script/test_build_components b/script/test_build_components
new file mode 120000
index 0000000000..832a4a72c6
--- /dev/null
+++ b/script/test_build_components
@@ -0,0 +1 @@
+test_build_components.py
\ No newline at end of file
diff --git a/script/test_build_components.py b/script/test_build_components.py
new file mode 100755
index 0000000000..df092c091d
--- /dev/null
+++ b/script/test_build_components.py
@@ -0,0 +1,1147 @@
+#!/usr/bin/env python3
+"""Test ESPHome component builds with intelligent grouping.
+
+This script replaces the bash test_build_components script with Python,
+adding support for intelligent component grouping based on shared bus
+configurations to reduce CI build time.
+
+Features:
+- Analyzes components for shared common bus configs
+- Groups compatible components together
+- Merges configs for grouped components
+- Uses --testing-mode for grouped tests
+- Maintains backward compatibility with single component testing
+"""
+
+from __future__ import annotations
+
+import argparse
+from collections import defaultdict
+from dataclasses import dataclass
+import hashlib
+import os
+from pathlib import Path
+import subprocess
+import sys
+import time
+
+# Add esphome to path
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+# pylint: disable=wrong-import-position
+from script.analyze_component_buses import (
+    BASE_BUS_COMPONENTS,
+    ISOLATED_COMPONENTS,
+    NO_BUSES_SIGNATURE,
+    analyze_all_components,
+    create_grouping_signature,
+    is_platform_component,
+    merge_compatible_bus_groups,
+    uses_local_file_references,
+)
+from script.merge_component_configs import merge_component_configs
+
+
+@dataclass
+class TestResult:
+    """Store information about a single test run."""
+
+    test_id: str
+    components: list[str]
+    platform: str
+    success: bool
+    duration: float
+    command: str = ""
+    test_type: str = "compile"  # "config" or "compile"
+
+
+def show_disk_space_if_ci(esphome_command: str) -> None:
+    """Show disk space usage if running in CI during compile.
+
+    Only shows output during compilation (not config validation) since
+    disk space is only relevant when actually building firmware.
+
+    Args:
+        esphome_command: The esphome command being run (config/compile/clean)
+    """
+    # Only show disk space during compilation in CI
+    # Config validation doesn't build anything so disk space isn't relevant
+    if not os.environ.get("GITHUB_ACTIONS"):
+        return
+    if esphome_command != "compile":
+        return
+
+    print("\n" + "=" * 80)
+    print("Disk Space After Build:")
+    print("=" * 80)
+    # Use sys.stdout.flush() to ensure output appears immediately
+    sys.stdout.flush()
+    subprocess.run(["df", "-h"], check=False, stdout=sys.stdout, stderr=sys.stderr)
+    print("=" * 80 + "\n")
+    sys.stdout.flush()
+
+
+def find_component_tests(
+    components_dir: Path, component_pattern: str = "*"
+) -> dict[str, list[Path]]:
+    """Find all component test files.
+
+    Args:
+        components_dir: Path to tests/components directory
+        component_pattern: Glob pattern for component names
+
+    Returns:
+        Dictionary mapping component name to list of test files
+    """
+    component_tests = defaultdict(list)
+
+    for comp_dir in components_dir.glob(component_pattern):
+        if not comp_dir.is_dir():
+            continue
+
+        # Find test files matching test.*.yaml or test-*.yaml patterns
+        for test_file in comp_dir.glob("test[.-]*.yaml"):
+            component_tests[comp_dir.name].append(test_file)
+
+    return dict(component_tests)
+
+
+def parse_test_filename(test_file: Path) -> tuple[str, str]:
+    """Parse test filename to extract test name and platform.
+
+    Args:
+        test_file: Path to test file
+
+    Returns:
+        Tuple of (test_name, platform)
+    """
+    parts = test_file.stem.split(".")
+    if len(parts) == 2:
+        return parts[0], parts[1]  # test, platform
+    return parts[0], "all"
+
+
+def get_platform_base_files(base_dir: Path) -> dict[str, list[Path]]:
+    """Get all platform base files.
+
+    Args:
+        base_dir: Path to test_build_components directory
+
+    Returns:
+        Dictionary mapping platform to list of base files (for version variants)
+    """
+    platform_files = defaultdict(list)
+
+    for base_file in base_dir.glob("build_components_base.*.yaml"):
+        # Extract platform from filename
+        # e.g., build_components_base.esp32-idf.yaml -> esp32-idf
+        # or build_components_base.esp32-idf-50.yaml -> esp32-idf
+        filename = base_file.stem
+        parts = filename.replace("build_components_base.", "").split("-")
+
+        # Platform is everything before version number (if present)
+        # Check if last part is a number (version)
+        platform = "-".join(parts[:-1]) if parts[-1].isdigit() else "-".join(parts)
+
+        platform_files[platform].append(base_file)
+
+    return dict(platform_files)
+
+
+def group_components_by_platform(
+    failed_results: list[TestResult],
+) -> dict[tuple[str, str], list[str]]:
+    """Group failed components by platform and test type for simplified reproduction commands.
+
+    Args:
+        failed_results: List of failed test results
+
+    Returns:
+        Dictionary mapping (platform, test_type) to list of component names
+    """
+    platform_components: dict[tuple[str, str], list[str]] = {}
+    for result in failed_results:
+        key = (result.platform, result.test_type)
+        if key not in platform_components:
+            platform_components[key] = []
+        platform_components[key].extend(result.components)
+
+    # Remove duplicates and sort for each platform
+    return {
+        key: sorted(set(components)) for key, components in platform_components.items()
+    }
+
+
+def format_github_summary(test_results: list[TestResult]) -> str:
+    """Format test results as GitHub Actions job summary markdown.
+
+    Args:
+        test_results: List of all test results
+
+    Returns:
+        Markdown formatted summary string
+    """
+    # Separate results into passed and failed
+    passed_results = [r for r in test_results if r.success]
+    failed_results = [r for r in test_results if not r.success]
+
+    lines = []
+
+    # Header with emoji based on success/failure
+    if failed_results:
+        lines.append("## :x: Component Tests Failed\n")
+    else:
+        lines.append("## :white_check_mark: Component Tests Passed\n")
+
+    # Summary statistics
+    total_time = sum(r.duration for r in test_results)
+    # Determine test type from results (all should be the same)
+    test_type = test_results[0].test_type if test_results else "unknown"
+    lines.append(
+        f"**Results:** {len(passed_results)} passed, {len(failed_results)} failed\n"
+    )
+    lines.append(f"**Total time:** {total_time:.1f}s\n")
+    lines.append(f"**Test type:** `{test_type}`\n")
+
+    # Show failed tests if any
+    if failed_results:
+        lines.append("### Failed Tests\n")
+        lines.append("| Test | Components | Platform | Duration |\n")
+        lines.append("|------|-----------|----------|----------|\n")
+        for result in failed_results:
+            components_str = ", ".join(result.components)
+            lines.append(
+                f"| `{result.test_id}` | {components_str} | {result.platform} | {result.duration:.1f}s |\n"
+            )
+        lines.append("\n")
+
+        # Show simplified commands to reproduce failures
+        # Group all failed components by platform for a single command per platform
+        lines.append("
\n")
+        lines.append("Commands to reproduce failures\n\n")
+        lines.append("```bash\n")
+
+        # Generate one command per platform and test type
+        platform_components = group_components_by_platform(failed_results)
+        for platform, test_type in sorted(platform_components.keys()):
+            components_csv = ",".join(platform_components[(platform, test_type)])
+            lines.append(
+                f"script/test_build_components.py -c {components_csv} -t {platform} -e {test_type}\n"
+            )
+
+        lines.append("```\n")
+        lines.append("
\n")
+
+    # Show passed tests
+    if passed_results:
+        lines.append("### Passed Tests\n\n")
+        lines.append(f"{len(passed_results)} tests passed successfully\n")
+
+        # Separate grouped and individual tests
+        grouped_results = [r for r in passed_results if len(r.components) > 1]
+        individual_results = [r for r in passed_results if len(r.components) == 1]
+
+        if grouped_results:
+            lines.append("#### Grouped Tests\n")
+            lines.append("| Components | Platform | Count | Duration |\n")
+            lines.append("|-----------|----------|-------|----------|\n")
+            for result in grouped_results:
+                components_str = ", ".join(result.components)
+                lines.append(
+                    f"| {components_str} | {result.platform} | {len(result.components)} | {result.duration:.1f}s |\n"
+                )
+            lines.append("\n")
+
+        if individual_results:
+            lines.append("#### Individual Tests\n")
+            # Show first 10 individual tests with timing
+            if len(individual_results) <= 10:
+                lines.extend(
+                    f"- `{result.test_id}` - {result.duration:.1f}s\n"
+                    for result in individual_results
+                )
+            else:
+                lines.extend(
+                    f"- `{result.test_id}` - {result.duration:.1f}s\n"
+                    for result in individual_results[:10]
+                )
+                lines.append(f"\n...and {len(individual_results) - 10} more\n")
+            lines.append("\n")
+
+    return "".join(lines)
+
+
+def write_github_summary(test_results: list[TestResult]) -> None:
+    """Write GitHub Actions job summary with test results and timing.
+
+    Args:
+        test_results: List of all test results
+    """
+    summary_content = format_github_summary(test_results)
+    with open(os.environ["GITHUB_STEP_SUMMARY"], "a", encoding="utf-8") as f:
+        f.write(summary_content)
+
+
+def extract_platform_with_version(base_file: Path) -> str:
+    """Extract platform with version from base filename.
+
+    Args:
+        base_file: Path to base file
+
+    Returns:
+        Platform with version (e.g., "esp32-idf-50" or "esp32-idf")
+    """
+    # Remove "build_components_base." prefix and ".yaml" suffix
+    return base_file.stem.replace("build_components_base.", "")
+
+
+def run_esphome_test(
+    component: str,
+    test_file: Path,
+    platform: str,
+    platform_with_version: str,
+    base_file: Path,
+    build_dir: Path,
+    esphome_command: str,
+    continue_on_fail: bool,
+    use_testing_mode: bool = False,
+) -> TestResult:
+    """Run esphome test for a single component.
+
+    Args:
+        component: Component name
+        test_file: Path to component test file
+        platform: Platform name (e.g., "esp32-idf")
+        platform_with_version: Platform with version (e.g., "esp32-idf-50")
+        base_file: Path to platform base file
+        build_dir: Path to build directory
+        esphome_command: ESPHome command (config/compile)
+        continue_on_fail: Whether to continue on failure
+        use_testing_mode: Whether to use --testing-mode flag
+
+    Returns:
+        TestResult object with test details and timing
+    """
+    test_name = test_file.stem.split(".")[0]
+
+    # Create dynamic test file in build directory
+    output_file = build_dir / f"{component}.{test_name}.{platform_with_version}.yaml"
+
+    # Copy base file and substitute component test file reference
+    base_content = base_file.read_text()
+    # Get relative path from build dir to test file
+    repo_root = Path(__file__).parent.parent
+    component_test_ref = f"../../{test_file.relative_to(repo_root / 'tests')}"
+    output_content = base_content.replace("$component_test_file", component_test_ref)
+    output_file.write_text(output_content)
+
+    # Build esphome command
+    cmd = [
+        sys.executable,
+        "-m",
+        "esphome",
+    ]
+
+    # Add --testing-mode if needed (must be before subcommand)
+    if use_testing_mode:
+        cmd.append("--testing-mode")
+
+    # Add substitutions
+    cmd.extend(
+        [
+            "-s",
+            "component_name",
+            component,
+            "-s",
+            "component_dir",
+            f"../../components/{component}",
+            "-s",
+            "test_name",
+            test_name,
+            "-s",
+            "target_platform",
+            platform,
+        ]
+    )
+
+    # Add command and config file
+    cmd.extend([esphome_command, str(output_file)])
+
+    # Build command string for display/logging
+    cmd_str = " ".join(cmd)
+
+    # Run command
+    print(f"> [{component}] [{test_name}] [{platform_with_version}]")
+    if use_testing_mode:
+        print("  (using --testing-mode)")
+
+    start_time = time.time()
+    test_id = f"{component}.{test_name}.{platform_with_version}"
+
+    try:
+        result = subprocess.run(cmd, check=False)
+        success = result.returncode == 0
+        duration = time.time() - start_time
+
+        # Show disk space after build in CI during compile
+        show_disk_space_if_ci(esphome_command)
+
+        if not success and not continue_on_fail:
+            # Print command immediately for failed tests
+            print(f"\n{'=' * 80}")
+            print("FAILED - Command to reproduce:")
+            print(f"{'=' * 80}")
+            print(cmd_str)
+            print()
+            raise subprocess.CalledProcessError(result.returncode, cmd)
+
+        return TestResult(
+            test_id=test_id,
+            components=[component],
+            platform=platform_with_version,
+            success=success,
+            duration=duration,
+            command=cmd_str,
+            test_type=esphome_command,
+        )
+    except subprocess.CalledProcessError:
+        duration = time.time() - start_time
+        # Re-raise if we're not continuing on fail
+        if not continue_on_fail:
+            raise
+        return TestResult(
+            test_id=test_id,
+            components=[component],
+            platform=platform_with_version,
+            success=False,
+            duration=duration,
+            command=cmd_str,
+            test_type=esphome_command,
+        )
+
+
+def run_grouped_test(
+    components: list[str],
+    platform: str,
+    platform_with_version: str,
+    base_file: Path,
+    build_dir: Path,
+    tests_dir: Path,
+    esphome_command: str,
+    continue_on_fail: bool,
+) -> TestResult:
+    """Run esphome test for a group of components with shared bus configs.
+
+    Args:
+        components: List of component names to test together
+        platform: Platform name (e.g., "esp32-idf")
+        platform_with_version: Platform with version (e.g., "esp32-idf-50")
+        base_file: Path to platform base file
+        build_dir: Path to build directory
+        tests_dir: Path to tests/components directory
+        esphome_command: ESPHome command (config/compile)
+        continue_on_fail: Whether to continue on failure
+
+    Returns:
+        TestResult object with test details and timing
+    """
+    # Create merged config
+    group_name = "_".join(components[:3])  # Use first 3 components for name
+    if len(components) > 3:
+        group_name += f"_plus_{len(components) - 3}"
+
+    # Create unique device name by hashing sorted component list + platform
+    # This prevents conflicts when different component groups are tested
+    sorted_components = sorted(components)
+    hash_input = "_".join(sorted_components) + "_" + platform
+    group_hash = hashlib.md5(hash_input.encode()).hexdigest()[:8]
+    device_name = f"comptest{platform.replace('-', '')}{group_hash}"
+
+    merged_config_file = build_dir / f"merged_{group_name}.{platform_with_version}.yaml"
+
+    try:
+        merge_component_configs(
+            component_names=components,
+            platform=platform_with_version,
+            tests_dir=tests_dir,
+            output_file=merged_config_file,
+        )
+    except Exception as e:  # pylint: disable=broad-exception-caught
+        print(f"Error merging configs for {components}: {e}")
+        if not continue_on_fail:
+            raise
+        # Return TestResult for merge failure
+        test_id = f"GROUPED[{','.join(components)}].{platform_with_version}"
+        return TestResult(
+            test_id=test_id,
+            components=components,
+            platform=platform_with_version,
+            success=False,
+            duration=0.0,
+            command=f"# Failed during config merge: {e}",
+            test_type=esphome_command,
+        )
+
+    # Create test file that includes merged config
+    output_file = build_dir / f"test_{group_name}.{platform_with_version}.yaml"
+    base_content = base_file.read_text()
+    merged_ref = merged_config_file.name
+    output_content = base_content.replace("$component_test_file", merged_ref)
+    output_file.write_text(output_content)
+
+    # Build esphome command with --testing-mode
+    cmd = [
+        sys.executable,
+        "-m",
+        "esphome",
+        "--testing-mode",  # Required for grouped tests
+        "-s",
+        "component_name",
+        device_name,  # Use unique hash-based device name
+        "-s",
+        "component_dir",
+        "../../components",
+        "-s",
+        "test_name",
+        "merged",
+        "-s",
+        "target_platform",
+        platform,
+        esphome_command,
+        str(output_file),
+    ]
+
+    # Build command string for display/logging
+    cmd_str = " ".join(cmd)
+
+    # Run command
+    components_str = ", ".join(components)
+    print(f"> [GROUPED: {components_str}] [{platform_with_version}]")
+    print("  (using --testing-mode)")
+
+    start_time = time.time()
+    test_id = f"GROUPED[{','.join(components)}].{platform_with_version}"
+
+    try:
+        result = subprocess.run(cmd, check=False)
+        success = result.returncode == 0
+        duration = time.time() - start_time
+
+        # Show disk space after build in CI during compile
+        show_disk_space_if_ci(esphome_command)
+
+        if not success and not continue_on_fail:
+            # Print command immediately for failed tests
+            print(f"\n{'=' * 80}")
+            print("FAILED - Command to reproduce:")
+            print(f"{'=' * 80}")
+            print(cmd_str)
+            print()
+            raise subprocess.CalledProcessError(result.returncode, cmd)
+
+        return TestResult(
+            test_id=test_id,
+            components=components,
+            platform=platform_with_version,
+            success=success,
+            duration=duration,
+            command=cmd_str,
+            test_type=esphome_command,
+        )
+    except subprocess.CalledProcessError:
+        duration = time.time() - start_time
+        # Re-raise if we're not continuing on fail
+        if not continue_on_fail:
+            raise
+        return TestResult(
+            test_id=test_id,
+            components=components,
+            platform=platform_with_version,
+            success=False,
+            duration=duration,
+            command=cmd_str,
+            test_type=esphome_command,
+        )
+
+
+def run_grouped_component_tests(
+    all_tests: dict[str, list[Path]],
+    platform_filter: str | None,
+    platform_bases: dict[str, list[Path]],
+    tests_dir: Path,
+    build_dir: Path,
+    esphome_command: str,
+    continue_on_fail: bool,
+    additional_isolated: set[str] | None = None,
+) -> tuple[set[tuple[str, str]], list[TestResult]]:
+    """Run grouped component tests.
+
+    Args:
+        all_tests: Dictionary mapping component names to test files
+        platform_filter: Optional platform to filter by
+        platform_bases: Platform base files mapping
+        tests_dir: Path to tests/components directory
+        build_dir: Path to build directory
+        esphome_command: ESPHome command (config/compile)
+        continue_on_fail: Whether to continue on failure
+        additional_isolated: Additional components to treat as isolated (not grouped)
+
+    Returns:
+        Tuple of (tested_components, test_results)
+    """
+    tested_components = set()
+    test_results = []
+
+    # Group components by platform and bus signature
+    grouped_components: dict[tuple[str, str], list[str]] = defaultdict(list)
+    print("\n" + "=" * 80)
+    print("Analyzing components for intelligent grouping...")
+    print("=" * 80)
+    component_buses, non_groupable, direct_bus_components = analyze_all_components(
+        tests_dir
+    )
+
+    # Track why components can't be grouped (for detailed output)
+    non_groupable_reasons = {}
+
+    # Merge additional isolated components with predefined ones
+    # ISOLATED COMPONENTS are tested individually WITHOUT --testing-mode
+    # This is critical because:
+    # - Grouped tests use --testing-mode which disables pin conflict checks and other validation
+    # - These checks are disabled to allow config merging (multiple components in one build)
+    # - For directly changed components (via --isolate), we need full validation to catch issues
+    # - Dependencies are safe to group since they weren't modified in the PR
+    all_isolated = set(ISOLATED_COMPONENTS.keys())
+    if additional_isolated:
+        all_isolated.update(additional_isolated)
+
+    # Group by (platform, bus_signature)
+    for component, platforms in component_buses.items():
+        if component not in all_tests:
+            continue
+
+        # Skip components that must be tested in isolation
+        # These are shown separately and should not be in non_groupable_reasons
+        if component in all_isolated:
+            continue
+
+        # Skip base bus components (these test the bus platforms themselves)
+        if component in BASE_BUS_COMPONENTS:
+            continue
+
+        # Skip components that use local file references or direct bus configs
+        if component in non_groupable:
+            # Track the reason (using pre-calculated results to avoid expensive re-analysis)
+            if component not in non_groupable_reasons:
+                if component in direct_bus_components:
+                    non_groupable_reasons[component] = (
+                        "Defines buses directly (not via packages) - NEEDS MIGRATION"
+                    )
+                elif uses_local_file_references(tests_dir / component):
+                    non_groupable_reasons[component] = (
+                        "Uses local file references ($component_dir)"
+                    )
+                elif is_platform_component(tests_dir / component):
+                    non_groupable_reasons[component] = (
+                        "Platform component (abstract base class)"
+                    )
+                else:
+                    non_groupable_reasons[component] = (
+                        "Uses !extend or !remove directives"
+                    )
+            continue
+
+        for platform, buses in platforms.items():
+            # Skip if platform doesn't match filter
+            if platform_filter and not platform.startswith(platform_filter):
+                continue
+
+            # Create signature for this component's bus configuration
+            # Components with no buses get NO_BUSES_SIGNATURE so they can be grouped together
+            if buses:
+                signature = create_grouping_signature({platform: buses}, platform)
+            else:
+                signature = NO_BUSES_SIGNATURE
+
+            # Add to grouped components (including those with no buses)
+            if signature:
+                grouped_components[(platform, signature)].append(component)
+
+    # Merge groups with compatible buses (cross-bus grouping optimization)
+    # This allows mixing components with different buses (e.g., ble + uart)
+    # as long as they don't have conflicting configurations for the same bus type
+    grouped_components = merge_compatible_bus_groups(grouped_components)
+
+    # Print detailed grouping plan
+    print("\nGrouping Plan:")
+    print("-" * 80)
+
+    # Show isolated components (must test individually due to known issues or direct changes)
+    isolated_in_tests = [c for c in all_isolated if c in all_tests]
+    if isolated_in_tests:
+        predefined_isolated = [c for c in isolated_in_tests if c in ISOLATED_COMPONENTS]
+        additional_in_tests = [
+            c for c in isolated_in_tests if c in (additional_isolated or set())
+        ]
+
+        if predefined_isolated:
+            print(
+                f"\n⚠ {len(predefined_isolated)} components must be tested in isolation (known build issues):"
+            )
+            for comp in sorted(predefined_isolated):
+                reason = ISOLATED_COMPONENTS[comp]
+                print(f"  - {comp}: {reason}")
+
+        if additional_in_tests:
+            print(
+                f"\n✓ {len(additional_in_tests)} components tested in isolation (directly changed in PR):"
+            )
+            for comp in sorted(additional_in_tests):
+                print(f"  - {comp}")
+
+    # Show base bus components (test the bus platform implementations)
+    base_bus_in_tests = [c for c in BASE_BUS_COMPONENTS if c in all_tests]
+    if base_bus_in_tests:
+        print(
+            f"\n○ {len(base_bus_in_tests)} base bus platform components (tested individually):"
+        )
+        for comp in sorted(base_bus_in_tests):
+            print(f"  - {comp}")
+
+    # Show excluded components with detailed reasons
+    if non_groupable_reasons:
+        excluded_in_tests = [c for c in non_groupable_reasons if c in all_tests]
+        if excluded_in_tests:
+            print(
+                f"\n⚠ {len(excluded_in_tests)} components excluded from grouping (each needs individual build):"
+            )
+            # Group by reason to show summary
+            direct_bus = [
+                c
+                for c in excluded_in_tests
+                if "NEEDS MIGRATION" in non_groupable_reasons.get(c, "")
+            ]
+            if direct_bus:
+                print(
+                    f"\n  ⚠⚠⚠ {len(direct_bus)} DEFINE BUSES DIRECTLY - NEED MIGRATION TO PACKAGES:"
+                )
+                for comp in sorted(direct_bus):
+                    print(f"    - {comp}")
+
+            other_reasons = [
+                c
+                for c in excluded_in_tests
+                if "NEEDS MIGRATION" not in non_groupable_reasons.get(c, "")
+            ]
+            if other_reasons and len(other_reasons) <= 10:
+                print("\n  Other non-groupable components:")
+                for comp in sorted(other_reasons):
+                    reason = non_groupable_reasons[comp]
+                    print(f"    - {comp}: {reason}")
+            elif other_reasons:
+                print(
+                    f"\n  Other non-groupable components: {len(other_reasons)} components"
+                )
+
+    # Distribute no_buses components into other groups to maximize efficiency
+    # Components with no buses can merge with any bus group since they have no conflicting requirements
+    no_buses_by_platform: dict[str, list[str]] = {}
+    for (platform, signature), components in list(grouped_components.items()):
+        if signature == NO_BUSES_SIGNATURE:
+            no_buses_by_platform[platform] = components
+            # Remove from grouped_components - we'll distribute them
+            del grouped_components[(platform, signature)]
+
+    # Distribute no_buses components into existing groups for each platform
+    for platform, no_buses_comps in no_buses_by_platform.items():
+        # Find all non-empty groups for this platform (excluding no_buses)
+        platform_groups = [
+            (sig, comps)
+            for (plat, sig), comps in grouped_components.items()
+            if plat == platform and sig != NO_BUSES_SIGNATURE
+        ]
+
+        if platform_groups:
+            # Distribute no_buses components round-robin across existing groups
+            for i, comp in enumerate(no_buses_comps):
+                sig, _ = platform_groups[i % len(platform_groups)]
+                grouped_components[(platform, sig)].append(comp)
+        else:
+            # No other groups for this platform - keep no_buses components together
+            grouped_components[(platform, NO_BUSES_SIGNATURE)] = no_buses_comps
+
+    groups_to_test = []
+    individual_tests = set()  # Use set to avoid duplicates
+
+    for (platform, signature), components in sorted(grouped_components.items()):
+        if len(components) > 1:
+            groups_to_test.append((platform, signature, components))
+        # Note: Don't add single-component groups to individual_tests here
+        # They'll be added below when we check for ungrouped components
+
+    # Add components that weren't grouped on any platform
+    for component in all_tests:
+        if component not in [c for _, _, comps in groups_to_test for c in comps]:
+            individual_tests.add(component)
+
+    if groups_to_test:
+        print(f"\n✓ {len(groups_to_test)} groups will be tested together:")
+        for platform, signature, components in groups_to_test:
+            component_list = ", ".join(sorted(components))
+            print(f"  [{platform}] [{signature}]: {component_list}")
+            print(
+                f"    → {len(components)} components in 1 build (saves {len(components) - 1} builds)"
+            )
+
+    if individual_tests:
+        print(f"\n○ {len(individual_tests)} components will be tested individually:")
+        sorted_individual = sorted(individual_tests)
+        for comp in sorted_individual[:10]:
+            print(f"  - {comp}")
+        if len(individual_tests) > 10:
+            print(f"  ... and {len(individual_tests) - 10} more")
+
+    # Calculate actual build counts based on test files, not component counts
+    # Without grouping: every test file would be built separately
+    total_test_files = sum(len(test_files) for test_files in all_tests.values())
+
+    # With grouping:
+    # - 1 build per group (regardless of how many components)
+    # - Individual components still need all their platform builds
+    individual_test_file_count = sum(
+        len(all_tests[comp]) for comp in individual_tests if comp in all_tests
+    )
+
+    total_grouped_components = sum(len(comps) for _, _, comps in groups_to_test)
+    total_builds_with_grouping = len(groups_to_test) + individual_test_file_count
+    builds_saved = total_test_files - total_builds_with_grouping
+
+    print(f"\n{'=' * 80}")
+    print(
+        f"Summary: {total_builds_with_grouping} builds total (vs {total_test_files} without grouping)"
+    )
+    print(
+        f"  • {len(groups_to_test)} grouped builds ({total_grouped_components} components)"
+    )
+    print(
+        f"  • {individual_test_file_count} individual builds ({len(individual_tests)} components)"
+    )
+    if total_test_files > 0:
+        reduction_pct = (builds_saved / total_test_files) * 100
+        print(f"  • Saves {builds_saved} builds ({reduction_pct:.1f}% reduction)")
+    print("=" * 80 + "\n")
+
+    # Execute grouped tests
+    for (platform, signature), components in grouped_components.items():
+        # Only group if we have multiple components with same signature
+        if len(components) <= 1:
+            continue
+
+        # Filter out components not in our test list
+        components_to_group = [c for c in components if c in all_tests]
+        if len(components_to_group) <= 1:
+            continue
+
+        # Get platform base files
+        if platform not in platform_bases:
+            continue
+
+        for base_file in platform_bases[platform]:
+            platform_with_version = extract_platform_with_version(base_file)
+
+            # Skip if platform filter doesn't match
+            if platform_filter and platform != platform_filter:
+                continue
+            if (
+                platform_filter
+                and platform_with_version != platform_filter
+                and not platform_with_version.startswith(f"{platform_filter}-")
+            ):
+                continue
+
+            # Run grouped test
+            test_result = run_grouped_test(
+                components=components_to_group,
+                platform=platform,
+                platform_with_version=platform_with_version,
+                base_file=base_file,
+                build_dir=build_dir,
+                tests_dir=tests_dir,
+                esphome_command=esphome_command,
+                continue_on_fail=continue_on_fail,
+            )
+
+            # Mark all components as tested
+            for comp in components_to_group:
+                tested_components.add((comp, platform_with_version))
+
+            # Store test result
+            test_results.append(test_result)
+
+    return tested_components, test_results
+
+
+def run_individual_component_test(
+    component: str,
+    test_file: Path,
+    platform: str,
+    platform_with_version: str,
+    base_file: Path,
+    build_dir: Path,
+    esphome_command: str,
+    continue_on_fail: bool,
+    tested_components: set[tuple[str, str]],
+    test_results: list[TestResult],
+) -> None:
+    """Run an individual component test if not already tested in a group.
+
+    Args:
+        component: Component name
+        test_file: Test file path
+        platform: Platform name
+        platform_with_version: Platform with version
+        base_file: Base file for platform
+        build_dir: Build directory
+        esphome_command: ESPHome command
+        continue_on_fail: Whether to continue on failure
+        tested_components: Set of already tested components
+        test_results: List to append test results
+    """
+    # Skip if already tested in a group
+    if (component, platform_with_version) in tested_components:
+        return
+
+    test_result = run_esphome_test(
+        component=component,
+        test_file=test_file,
+        platform=platform,
+        platform_with_version=platform_with_version,
+        base_file=base_file,
+        build_dir=build_dir,
+        esphome_command=esphome_command,
+        continue_on_fail=continue_on_fail,
+    )
+    test_results.append(test_result)
+
+
+def test_components(
+    component_patterns: list[str],
+    platform_filter: str | None,
+    esphome_command: str,
+    continue_on_fail: bool,
+    enable_grouping: bool = True,
+    isolated_components: set[str] | None = None,
+) -> int:
+    """Test components with optional intelligent grouping.
+
+    Args:
+        component_patterns: List of component name patterns
+        platform_filter: Optional platform to filter by
+        esphome_command: ESPHome command (config/compile)
+        continue_on_fail: Whether to continue on failure
+        enable_grouping: Whether to enable component grouping
+        isolated_components: Set of component names to test in isolation (not grouped).
+            These are tested WITHOUT --testing-mode to enable full validation
+            (pin conflicts, etc). This is used in CI for directly changed components
+            to catch issues that would be missed with --testing-mode.
+
+    Returns:
+        Exit code (0 for success, 1 for failure)
+    """
+    # Setup paths
+    repo_root = Path(__file__).parent.parent
+    tests_dir = repo_root / "tests" / "components"
+    build_components_dir = repo_root / "tests" / "test_build_components"
+    build_dir = build_components_dir / "build"
+    build_dir.mkdir(parents=True, exist_ok=True)
+
+    # Get platform base files
+    platform_bases = get_platform_base_files(build_components_dir)
+
+    # Find all component tests
+    all_tests = {}
+    for pattern in component_patterns:
+        all_tests.update(find_component_tests(tests_dir, pattern))
+
+    if not all_tests:
+        print(f"No components found matching: {component_patterns}")
+        return 1
+
+    print(f"Found {len(all_tests)} components to test")
+
+    # Run tests
+    test_results = []
+    tested_components = set()  # Track which components were tested in groups
+
+    # First, run grouped tests if grouping is enabled
+    if enable_grouping:
+        tested_components, grouped_results = run_grouped_component_tests(
+            all_tests=all_tests,
+            platform_filter=platform_filter,
+            platform_bases=platform_bases,
+            tests_dir=tests_dir,
+            build_dir=build_dir,
+            esphome_command=esphome_command,
+            continue_on_fail=continue_on_fail,
+            additional_isolated=isolated_components,
+        )
+        test_results.extend(grouped_results)
+
+    # Then run individual tests for components not in groups
+    for component, test_files in sorted(all_tests.items()):
+        for test_file in test_files:
+            test_name, platform = parse_test_filename(test_file)
+
+            # Handle "all" platform tests
+            if platform == "all":
+                # Run for all platforms
+                for plat, base_files in platform_bases.items():
+                    if platform_filter and plat != platform_filter:
+                        continue
+
+                    for base_file in base_files:
+                        platform_with_version = extract_platform_with_version(base_file)
+                        run_individual_component_test(
+                            component=component,
+                            test_file=test_file,
+                            platform=plat,
+                            platform_with_version=platform_with_version,
+                            base_file=base_file,
+                            build_dir=build_dir,
+                            esphome_command=esphome_command,
+                            continue_on_fail=continue_on_fail,
+                            tested_components=tested_components,
+                            test_results=test_results,
+                        )
+            else:
+                # Platform-specific test
+                if platform_filter and platform != platform_filter:
+                    continue
+
+                if platform not in platform_bases:
+                    print(f"No base file for platform: {platform}")
+                    continue
+
+                for base_file in platform_bases[platform]:
+                    platform_with_version = extract_platform_with_version(base_file)
+
+                    # Skip if requested platform doesn't match
+                    if (
+                        platform_filter
+                        and platform_with_version != platform_filter
+                        and not platform_with_version.startswith(f"{platform_filter}-")
+                    ):
+                        continue
+
+                    run_individual_component_test(
+                        component=component,
+                        test_file=test_file,
+                        platform=platform,
+                        platform_with_version=platform_with_version,
+                        base_file=base_file,
+                        build_dir=build_dir,
+                        esphome_command=esphome_command,
+                        continue_on_fail=continue_on_fail,
+                        tested_components=tested_components,
+                        test_results=test_results,
+                    )
+
+    # Separate results into passed and failed
+    passed_results = [r for r in test_results if r.success]
+    failed_results = [r for r in test_results if not r.success]
+
+    # Print summary
+    print("\n" + "=" * 80)
+    print(f"Test Summary: {len(passed_results)} passed, {len(failed_results)} failed")
+    print("=" * 80)
+
+    if failed_results:
+        print("\nFailed tests:")
+        for result in failed_results:
+            print(f"  - {result.test_id}")
+
+        # Print simplified commands grouped by platform and test type for easy copy-paste
+        print("\n" + "=" * 80)
+        print("Commands to reproduce failures (copy-paste to reproduce locally):")
+        print("=" * 80)
+        platform_components = group_components_by_platform(failed_results)
+        for platform, test_type in sorted(platform_components.keys()):
+            components_csv = ",".join(platform_components[(platform, test_type)])
+            print(
+                f"script/test_build_components.py -c {components_csv} -t {platform} -e {test_type}"
+            )
+        print()
+
+    # Write GitHub Actions job summary if in CI
+    if os.environ.get("GITHUB_STEP_SUMMARY"):
+        write_github_summary(test_results)
+
+    if failed_results:
+        return 1
+
+    return 0
+
+
+def main() -> int:
+    """Main entry point."""
+    parser = argparse.ArgumentParser(
+        description="Test ESPHome component builds with intelligent grouping"
+    )
+    parser.add_argument(
+        "-e",
+        "--esphome-command",
+        default="compile",
+        choices=["config", "compile", "clean"],
+        help="ESPHome command to run (default: compile)",
+    )
+    parser.add_argument(
+        "-c",
+        "--components",
+        default="*",
+        help="Component pattern(s) to test (default: *). Comma-separated.",
+    )
+    parser.add_argument(
+        "-t",
+        "--target",
+        help="Target platform to test (e.g., esp32-idf)",
+    )
+    parser.add_argument(
+        "-f",
+        "--continue-on-fail",
+        action="store_true",
+        help="Continue testing even if a test fails",
+    )
+    parser.add_argument(
+        "--no-grouping",
+        action="store_true",
+        help="Disable component grouping (test each component individually)",
+    )
+    parser.add_argument(
+        "--isolate",
+        help="Comma-separated list of components to test in isolation (not grouped with others). "
+        "These are tested WITHOUT --testing-mode to enable full validation. "
+        "Used in CI for directly changed components to catch pin conflicts and other issues.",
+    )
+
+    args = parser.parse_args()
+
+    # Parse component patterns
+    component_patterns = [p.strip() for p in args.components.split(",")]
+
+    # Parse isolated components
+    isolated_components = None
+    if args.isolate:
+        isolated_components = {c.strip() for c in args.isolate.split(",") if c.strip()}
+
+    return test_components(
+        component_patterns=component_patterns,
+        platform_filter=args.target,
+        esphome_command=args.esphome_command,
+        continue_on_fail=args.continue_on_fail,
+        enable_grouping=not args.no_grouping,
+        isolated_components=isolated_components,
+    )
+
+
+if __name__ == "__main__":
+    sys.exit(main())
diff --git a/script/test_component_grouping.py b/script/test_component_grouping.py
new file mode 100755
index 0000000000..a2cee6e888
--- /dev/null
+++ b/script/test_component_grouping.py
@@ -0,0 +1,227 @@
+#!/usr/bin/env python3
+"""Test component grouping by finding and testing groups of components.
+
+This script analyzes components, finds groups that can be tested together,
+and runs test builds for those groups.
+"""
+
+from __future__ import annotations
+
+import argparse
+from pathlib import Path
+import subprocess
+import sys
+
+# Add esphome to path
+sys.path.insert(0, str(Path(__file__).parent.parent))
+
+from script.analyze_component_buses import (
+    analyze_all_components,
+    group_components_by_signature,
+)
+
+
+def test_component_group(
+    components: list[str],
+    platform: str,
+    esphome_command: str = "compile",
+    dry_run: bool = False,
+) -> bool:
+    """Test a group of components together.
+
+    Args:
+        components: List of component names to test together
+        platform: Platform to test on (e.g., "esp32-idf")
+        esphome_command: ESPHome command to run (config/compile/clean)
+        dry_run: If True, only print the command without running it
+
+    Returns:
+        True if test passed, False otherwise
+    """
+    components_str = ",".join(components)
+    cmd = [
+        "./script/test_build_components",
+        "-c",
+        components_str,
+        "-t",
+        platform,
+        "-e",
+        esphome_command,
+    ]
+
+    print(f"\n{'=' * 80}")
+    print(f"Testing {len(components)} components on {platform}:")
+    for comp in components:
+        print(f"  - {comp}")
+    print(f"{'=' * 80}")
+    print(f"Command: {' '.join(cmd)}\n")
+
+    if dry_run:
+        print("[DRY RUN] Skipping actual test")
+        return True
+
+    try:
+        result = subprocess.run(cmd, check=False)
+        return result.returncode == 0
+    except Exception as e:
+        print(f"Error running test: {e}")
+        return False
+
+
+def main() -> None:
+    """Main entry point."""
+    parser = argparse.ArgumentParser(
+        description="Test component grouping by finding and testing groups"
+    )
+    parser.add_argument(
+        "--platform",
+        "-p",
+        default="esp32-idf",
+        help="Platform to test (default: esp32-idf)",
+    )
+    parser.add_argument(
+        "-e",
+        "--esphome-command",
+        default="compile",
+        choices=["config", "compile", "clean"],
+        help="ESPHome command to run (default: compile)",
+    )
+    parser.add_argument(
+        "--all",
+        action="store_true",
+        help="Test all components (sets --min-size=1, --max-size=10000, --max-groups=10000)",
+    )
+    parser.add_argument(
+        "--min-size",
+        type=int,
+        default=3,
+        help="Minimum group size to test (default: 3)",
+    )
+    parser.add_argument(
+        "--max-size",
+        type=int,
+        default=10,
+        help="Maximum group size to test (default: 10)",
+    )
+    parser.add_argument(
+        "--max-groups",
+        type=int,
+        default=5,
+        help="Maximum number of groups to test (default: 5)",
+    )
+    parser.add_argument(
+        "--signature",
+        "-s",
+        help="Only test groups with this bus signature (e.g., 'spi', 'i2c', 'uart')",
+    )
+    parser.add_argument(
+        "--dry-run",
+        action="store_true",
+        help="Print commands without running them",
+    )
+
+    args = parser.parse_args()
+
+    # If --all is specified, test all components without grouping
+    if args.all:
+        # Get all components from tests/components directory
+        components_dir = Path("tests/components")
+        all_components = sorted(
+            [d.name for d in components_dir.iterdir() if d.is_dir()]
+        )
+
+        if not all_components:
+            print(f"\nNo components found in {components_dir}")
+            return
+
+        print(f"\nTesting all {len(all_components)} components together")
+
+        success = test_component_group(
+            all_components, args.platform, args.esphome_command, args.dry_run
+        )
+
+        # Print summary
+        print(f"\n{'=' * 80}")
+        print("TEST SUMMARY")
+        print(f"{'=' * 80}")
+        status = "✅ PASS" if success else "❌ FAIL"
+        print(f"{status} All components: {len(all_components)} components")
+
+        if not args.dry_run and not success:
+            sys.exit(1)
+        return
+
+    print("Analyzing all components...")
+    components, non_groupable, _ = analyze_all_components(Path("tests/components"))
+
+    print(f"Found {len(components)} components, {len(non_groupable)} non-groupable")
+
+    # Group components by signature for the platform
+    groups = group_components_by_signature(components, args.platform)
+
+    # Filter and sort groups
+    filtered_groups = []
+    for signature, comp_list in groups.items():
+        # Filter by signature if specified
+        if args.signature and signature != args.signature:
+            continue
+
+        # Remove non-groupable components
+        comp_list = [c for c in comp_list if c not in non_groupable]
+
+        # Filter by minimum size
+        if len(comp_list) < args.min_size:
+            continue
+
+        # If group is larger than max_size, we'll take a subset later
+        filtered_groups.append((signature, comp_list))
+
+    # Sort by group size (largest first)
+    filtered_groups.sort(key=lambda x: len(x[1]), reverse=True)
+
+    # Limit number of groups
+    filtered_groups = filtered_groups[: args.max_groups]
+
+    if not filtered_groups:
+        print("\nNo groups found matching criteria:")
+        print(f"  - Platform: {args.platform}")
+        print(f"  - Size: {args.min_size}-{args.max_size}")
+        if args.signature:
+            print(f"  - Signature: {args.signature}")
+        return
+
+    print(f"\nFound {len(filtered_groups)} groups to test:")
+    for signature, comp_list in filtered_groups:
+        print(f"  [{signature}]: {len(comp_list)} components")
+
+    # Test each group
+    results = []
+    for signature, comp_list in filtered_groups:
+        # Limit to max_size if group is larger
+        if len(comp_list) > args.max_size:
+            comp_list = comp_list[: args.max_size]
+
+        success = test_component_group(
+            comp_list, args.platform, args.esphome_command, args.dry_run
+        )
+        results.append((signature, comp_list, success))
+
+        if not args.dry_run and not success:
+            print(f"\n❌ FAILED: {signature} group")
+            break
+
+    # Print summary
+    print(f"\n{'=' * 80}")
+    print("TEST SUMMARY")
+    print(f"{'=' * 80}")
+    for signature, comp_list, success in results:
+        status = "✅ PASS" if success else "❌ FAIL"
+        print(f"{status} [{signature}]: {len(comp_list)} components")
+
+    # Exit with error if any tests failed
+    if not args.dry_run and any(not success for _, _, success in results):
+        sys.exit(1)
+
+
+if __name__ == "__main__":
+    main()
diff --git a/sdkconfig.defaults b/sdkconfig.defaults
index 72ca3f6e9c..322efb701a 100644
--- a/sdkconfig.defaults
+++ b/sdkconfig.defaults
@@ -13,6 +13,7 @@ CONFIG_ESP_TASK_WDT=y
 CONFIG_ESP_TASK_WDT_PANIC=y
 CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU0=n
 CONFIG_ESP_TASK_WDT_CHECK_IDLE_TASK_CPU1=n
+CONFIG_AUTOSTART_ARDUINO=y
 
 # esp32_ble
 CONFIG_BT_ENABLED=y
diff --git a/tests/__init__.py b/tests/__init__.py
new file mode 100644
index 0000000000..d49aac4bab
--- /dev/null
+++ b/tests/__init__.py
@@ -0,0 +1 @@
+"""ESPHome tests package."""
diff --git a/tests/component_tests/__init__.py b/tests/component_tests/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/component_tests/binary_sensor/__init__.py b/tests/component_tests/binary_sensor/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/component_tests/button/__init__.py b/tests/component_tests/button/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/component_tests/config_validation/test_config.py b/tests/component_tests/config_validation/test_config.py
new file mode 100644
index 0000000000..1a9b9bc1f3
--- /dev/null
+++ b/tests/component_tests/config_validation/test_config.py
@@ -0,0 +1,51 @@
+"""
+Test schema.extend functionality in esphome.config_validation.
+"""
+
+from typing import Any
+
+import esphome.config_validation as cv
+
+
+def test_config_extend() -> None:
+    """Test that schema.extend correctly merges schemas with extras."""
+
+    def func1(data: dict[str, Any]) -> dict[str, Any]:
+        data["extra_1"] = "value1"
+        return data
+
+    def func2(data: dict[str, Any]) -> dict[str, Any]:
+        data["extra_2"] = "value2"
+        return data
+
+    schema1 = cv.Schema(
+        {
+            cv.Required("key1"): cv.string,
+        }
+    )
+    schema1.add_extra(func1)
+    schema2 = cv.Schema(
+        {
+            cv.Required("key2"): cv.string,
+        }
+    )
+    schema2.add_extra(func2)
+    extended_schema = schema1.extend(schema2)
+    config = {
+        "key1": "initial_value1",
+        "key2": "initial_value2",
+    }
+    validated = extended_schema(config)
+    assert validated["key1"] == "initial_value1"
+    assert validated["key2"] == "initial_value2"
+    assert validated["extra_1"] == "value1"
+    assert validated["extra_2"] == "value2"
+
+    # Check the opposite order of extension
+    extended_schema = schema2.extend(schema1)
+
+    validated = extended_schema(config)
+    assert validated["key1"] == "initial_value1"
+    assert validated["key2"] == "initial_value2"
+    assert validated["extra_1"] == "value1"
+    assert validated["extra_2"] == "value2"
diff --git a/tests/component_tests/conftest.py b/tests/component_tests/conftest.py
index b1e0eaa200..0641e698e9 100644
--- a/tests/component_tests/conftest.py
+++ b/tests/component_tests/conftest.py
@@ -5,18 +5,32 @@ from __future__ import annotations
 from collections.abc import Callable, Generator
 from pathlib import Path
 import sys
+from typing import Any
+from unittest import mock
 
 import pytest
 
+from esphome import config, final_validate
+from esphome.const import (
+    KEY_CORE,
+    KEY_TARGET_FRAMEWORK,
+    KEY_TARGET_PLATFORM,
+    PlatformFramework,
+)
+from esphome.types import ConfigType
+from esphome.util import OrderedDict
+
 # Add package root to python path
 here = Path(__file__).parent
 package_root = here.parent.parent
 sys.path.insert(0, package_root.as_posix())
 
 from esphome.__main__ import generate_cpp_contents  # noqa: E402
-from esphome.config import read_config  # noqa: E402
+from esphome.config import Config, read_config  # noqa: E402
 from esphome.core import CORE  # noqa: E402
 
+from .types import SetCoreConfigCallable  # noqa: E402
+
 
 @pytest.fixture(autouse=True)
 def config_path(request: pytest.FixtureRequest) -> Generator[None]:
@@ -28,14 +42,68 @@ def config_path(request: pytest.FixtureRequest) -> Generator[None]:
     if config_dir.exists():
         # Set config_path to a dummy yaml file in the config directory
         # This ensures CORE.config_dir points to the config directory
-        CORE.config_path = str(config_dir / "dummy.yaml")
+        CORE.config_path = config_dir / "dummy.yaml"
     else:
-        CORE.config_path = str(Path(request.fspath).parent / "dummy.yaml")
+        CORE.config_path = Path(request.fspath).parent / "dummy.yaml"
 
     yield
     CORE.config_path = original_path
 
 
+@pytest.fixture(autouse=True)
+def reset_core() -> Generator[None]:
+    """Reset CORE after each test."""
+    yield
+    CORE.reset()
+
+
+@pytest.fixture
+def set_core_config() -> Generator[SetCoreConfigCallable]:
+    """Fixture to set up the core configuration for tests."""
+
+    def setter(
+        platform_framework: PlatformFramework,
+        /,
+        *,
+        core_data: ConfigType | None = None,
+        platform_data: ConfigType | None = None,
+        full_config: dict[str, ConfigType] | None = None,
+    ) -> None:
+        platform, framework = platform_framework.value
+
+        # Set base core configuration
+        CORE.data[KEY_CORE] = {
+            KEY_TARGET_PLATFORM: platform.value,
+            KEY_TARGET_FRAMEWORK: framework.value,
+        }
+
+        # Update with any additional core data
+        if core_data:
+            CORE.data[KEY_CORE].update(core_data)
+
+        # Set platform-specific data
+        if platform_data:
+            CORE.data[platform.value] = platform_data
+
+        config.path_context.set([])
+        final_validate.full_config.set(full_config or Config())
+
+    yield setter
+
+
+@pytest.fixture
+def set_component_config() -> Callable[[str, Any], None]:
+    """
+    Fixture to set a component configuration in the mock config.
+    This must be used after the core configuration has been set up.
+    """
+
+    def setter(name: str, value: Any) -> None:
+        final_validate.full_config.get()[name] = value
+
+    return setter
+
+
 @pytest.fixture
 def component_fixture_path(request: pytest.FixtureRequest) -> Callable[[str], Path]:
     """Return a function to get absolute paths relative to the component's fixtures directory."""
@@ -60,14 +128,38 @@ def component_config_path(request: pytest.FixtureRequest) -> Callable[[str], Pat
 
 @pytest.fixture
 def generate_main() -> Generator[Callable[[str | Path], str]]:
-    """Generates the C++ main.cpp file and returns it in string form."""
+    """Generates the C++ main.cpp from a given yaml file and returns it in string form."""
 
     def generator(path: str | Path) -> str:
-        CORE.config_path = str(path)
+        CORE.config_path = Path(path)
         CORE.config = read_config({})
         generate_cpp_contents(CORE.config)
         return CORE.cpp_main_section
 
     yield generator
 
-    CORE.reset()
+
+@pytest.fixture
+def mock_clone_or_update() -> Generator[Any]:
+    """Mock git.clone_or_update for testing."""
+    with mock.patch("esphome.git.clone_or_update") as mock_func:
+        # Default return value
+        mock_func.return_value = (Path("/tmp/test"), None)
+        yield mock_func
+
+
+@pytest.fixture
+def mock_load_yaml() -> Generator[Any]:
+    """Mock yaml_util.load_yaml for testing."""
+
+    with mock.patch("esphome.yaml_util.load_yaml") as mock_func:
+        # Default return value
+        mock_func.return_value = OrderedDict({"sensor": []})
+        yield mock_func
+
+
+@pytest.fixture
+def mock_install_meta_finder() -> Generator[Any]:
+    """Mock loader.install_meta_finder for testing."""
+    with mock.patch("esphome.loader.install_meta_finder") as mock_func:
+        yield mock_func
diff --git a/tests/component_tests/deep_sleep/__init__.py b/tests/component_tests/deep_sleep/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/component_tests/esp32/test_esp32.py b/tests/component_tests/esp32/test_esp32.py
new file mode 100644
index 0000000000..91e96f24d6
--- /dev/null
+++ b/tests/component_tests/esp32/test_esp32.py
@@ -0,0 +1,111 @@
+"""
+Test ESP32 configuration
+"""
+
+from typing import Any
+
+import pytest
+
+from esphome.components.esp32 import VARIANTS
+import esphome.config_validation as cv
+from esphome.const import CONF_ESPHOME, PlatformFramework
+from tests.component_tests.types import SetCoreConfigCallable
+
+
+def test_esp32_config(
+    set_core_config: SetCoreConfigCallable,
+) -> None:
+    set_core_config(PlatformFramework.ESP32_IDF)
+
+    from esphome.components.esp32 import CONFIG_SCHEMA
+    from esphome.components.esp32.const import VARIANT_ESP32, VARIANT_FRIENDLY
+
+    # Example ESP32 configuration
+    config = {
+        "board": "esp32dev",
+        "variant": VARIANT_ESP32,
+        "cpu_frequency": "240MHz",
+        "flash_size": "4MB",
+        "framework": {
+            "type": "esp-idf",
+        },
+    }
+
+    # Check if the variant is valid
+    config = CONFIG_SCHEMA(config)
+    assert config["variant"] == VARIANT_ESP32
+
+    # Check that defining a variant sets the board name correctly
+    for variant in VARIANTS:
+        config = CONFIG_SCHEMA(
+            {
+                "variant": variant,
+            }
+        )
+        assert VARIANT_FRIENDLY[variant].lower() in config["board"]
+
+
+@pytest.mark.parametrize(
+    ("config", "error_match"),
+    [
+        pytest.param(
+            {"flash_size": "4MB"},
+            r"This board is unknown, if you are sure you want to compile with this board selection, override with option 'variant' @ data\['board'\]",
+            id="unknown_board_config",
+        ),
+        pytest.param(
+            {"variant": "esp32xx"},
+            r"Unknown value 'ESP32XX', did you mean 'ESP32', 'ESP32S3', 'ESP32S2'\? for dictionary value @ data\['variant'\]",
+            id="unknown_variant_config",
+        ),
+        pytest.param(
+            {"variant": "esp32s3", "board": "esp32dev"},
+            r"Option 'variant' does not match selected board. @ data\['variant'\]",
+            id="mismatched_board_variant_config",
+        ),
+        pytest.param(
+            {
+                "variant": "esp32s2",
+                "framework": {
+                    "type": "esp-idf",
+                    "advanced": {"execute_from_psram": True},
+                },
+            },
+            r"'execute_from_psram' is only supported on ESP32S3 variant @ data\['framework'\]\['advanced'\]\['execute_from_psram'\]",
+            id="execute_from_psram_invalid_for_variant_config",
+        ),
+        pytest.param(
+            {
+                "variant": "esp32s3",
+                "framework": {
+                    "type": "esp-idf",
+                    "advanced": {"execute_from_psram": True},
+                },
+            },
+            r"'execute_from_psram' requires PSRAM to be configured @ data\['framework'\]\['advanced'\]\['execute_from_psram'\]",
+            id="execute_from_psram_requires_psram_config",
+        ),
+        pytest.param(
+            {
+                "variant": "esp32s3",
+                "framework": {
+                    "type": "esp-idf",
+                    "advanced": {"ignore_efuse_mac_crc": True},
+                },
+            },
+            r"'ignore_efuse_mac_crc' is not supported on ESP32S3 @ data\['framework'\]\['advanced'\]\['ignore_efuse_mac_crc'\]",
+            id="ignore_efuse_mac_crc_only_on_esp32",
+        ),
+    ],
+)
+def test_esp32_configuration_errors(
+    config: Any,
+    error_match: str,
+    set_core_config: SetCoreConfigCallable,
+) -> None:
+    set_core_config(PlatformFramework.ESP32_IDF, full_config={CONF_ESPHOME: {}})
+    """Test detection of invalid configuration."""
+    from esphome.components.esp32 import CONFIG_SCHEMA, FINAL_VALIDATE_SCHEMA
+
+    with pytest.raises(cv.Invalid, match=error_match):
+        FINAL_VALIDATE_SCHEMA(CONFIG_SCHEMA(config))
diff --git a/tests/component_tests/external_components/test_init.py b/tests/component_tests/external_components/test_init.py
new file mode 100644
index 0000000000..905c0afa8b
--- /dev/null
+++ b/tests/component_tests/external_components/test_init.py
@@ -0,0 +1,134 @@
+"""Tests for the external_components skip_update functionality."""
+
+from pathlib import Path
+from typing import Any
+from unittest.mock import MagicMock
+
+from esphome.components.external_components import do_external_components_pass
+from esphome.const import (
+    CONF_EXTERNAL_COMPONENTS,
+    CONF_REFRESH,
+    CONF_SOURCE,
+    CONF_URL,
+    TYPE_GIT,
+)
+
+
+def test_external_components_skip_update_true(
+    tmp_path: Path, mock_clone_or_update: MagicMock, mock_install_meta_finder: MagicMock
+) -> None:
+    """Test that external components don't update when skip_update=True."""
+    # Create a components directory structure
+    components_dir = tmp_path / "components"
+    components_dir.mkdir()
+
+    # Create a test component
+    test_component_dir = components_dir / "test_component"
+    test_component_dir.mkdir()
+    (test_component_dir / "__init__.py").write_text("# Test component")
+
+    # Set up mock to return our tmp_path
+    mock_clone_or_update.return_value = (tmp_path, None)
+
+    config: dict[str, Any] = {
+        CONF_EXTERNAL_COMPONENTS: [
+            {
+                CONF_SOURCE: {
+                    "type": TYPE_GIT,
+                    CONF_URL: "https://github.com/test/components",
+                },
+                CONF_REFRESH: "1d",
+                "components": "all",
+            }
+        ]
+    }
+
+    # Call with skip_update=True
+    do_external_components_pass(config, skip_update=True)
+
+    # Verify clone_or_update was called with NEVER_REFRESH
+    mock_clone_or_update.assert_called_once()
+    call_args = mock_clone_or_update.call_args
+    from esphome import git
+
+    assert call_args.kwargs["refresh"] == git.NEVER_REFRESH
+
+
+def test_external_components_skip_update_false(
+    tmp_path: Path, mock_clone_or_update: MagicMock, mock_install_meta_finder: MagicMock
+) -> None:
+    """Test that external components update when skip_update=False."""
+    # Create a components directory structure
+    components_dir = tmp_path / "components"
+    components_dir.mkdir()
+
+    # Create a test component
+    test_component_dir = components_dir / "test_component"
+    test_component_dir.mkdir()
+    (test_component_dir / "__init__.py").write_text("# Test component")
+
+    # Set up mock to return our tmp_path
+    mock_clone_or_update.return_value = (tmp_path, None)
+
+    config: dict[str, Any] = {
+        CONF_EXTERNAL_COMPONENTS: [
+            {
+                CONF_SOURCE: {
+                    "type": TYPE_GIT,
+                    CONF_URL: "https://github.com/test/components",
+                },
+                CONF_REFRESH: "1d",
+                "components": "all",
+            }
+        ]
+    }
+
+    # Call with skip_update=False
+    do_external_components_pass(config, skip_update=False)
+
+    # Verify clone_or_update was called with actual refresh value
+    mock_clone_or_update.assert_called_once()
+    call_args = mock_clone_or_update.call_args
+    from esphome.core import TimePeriodSeconds
+
+    assert call_args.kwargs["refresh"] == TimePeriodSeconds(days=1)
+
+
+def test_external_components_default_no_skip(
+    tmp_path: Path, mock_clone_or_update: MagicMock, mock_install_meta_finder: MagicMock
+) -> None:
+    """Test that external components update by default when skip_update not specified."""
+    # Create a components directory structure
+    components_dir = tmp_path / "components"
+    components_dir.mkdir()
+
+    # Create a test component
+    test_component_dir = components_dir / "test_component"
+    test_component_dir.mkdir()
+    (test_component_dir / "__init__.py").write_text("# Test component")
+
+    # Set up mock to return our tmp_path
+    mock_clone_or_update.return_value = (tmp_path, None)
+
+    config: dict[str, Any] = {
+        CONF_EXTERNAL_COMPONENTS: [
+            {
+                CONF_SOURCE: {
+                    "type": TYPE_GIT,
+                    CONF_URL: "https://github.com/test/components",
+                },
+                CONF_REFRESH: "1d",
+                "components": "all",
+            }
+        ]
+    }
+
+    # Call without skip_update parameter
+    do_external_components_pass(config)
+
+    # Verify clone_or_update was called with actual refresh value
+    mock_clone_or_update.assert_called_once()
+    call_args = mock_clone_or_update.call_args
+    from esphome.core import TimePeriodSeconds
+
+    assert call_args.kwargs["refresh"] == TimePeriodSeconds(days=1)
diff --git a/tests/component_tests/image/__init__.py b/tests/component_tests/image/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/component_tests/image/config/image_test.yaml b/tests/component_tests/image/config/image_test.yaml
index 3ff1260bd0..c34e0993a5 100644
--- a/tests/component_tests/image/config/image_test.yaml
+++ b/tests/component_tests/image/config/image_test.yaml
@@ -5,10 +5,12 @@ esp32:
   board: esp32s3box
 
 image:
-  - file: image.png
-    byte_order: little_endian
-    id: cat_img
+  defaults:
     type: rgb565
+    byte_order: little_endian
+  images:
+    - file: image.png
+      id: cat_img
 
 spi:
   mosi_pin: 6
diff --git a/tests/component_tests/image/test_init.py b/tests/component_tests/image/test_init.py
index d8a883d32f..f0b132cef8 100644
--- a/tests/component_tests/image/test_init.py
+++ b/tests/component_tests/image/test_init.py
@@ -9,7 +9,8 @@ from typing import Any
 import pytest
 
 from esphome import config_validation as cv
-from esphome.components.image import CONFIG_SCHEMA
+from esphome.components.image import CONF_TRANSPARENCY, CONFIG_SCHEMA
+from esphome.const import CONF_ID, CONF_RAW_DATA_ID, CONF_TYPE
 
 
 @pytest.mark.parametrize(
@@ -22,12 +23,12 @@ from esphome.components.image import CONFIG_SCHEMA
         ),
         pytest.param(
             {"id": "image_id", "type": "rgb565"},
-            r"required key not provided @ data\[0\]\['file'\]",
+            r"required key not provided @ data\['file'\]",
             id="missing_file",
         ),
         pytest.param(
             {"file": "image.png", "type": "rgb565"},
-            r"required key not provided @ data\[0\]\['id'\]",
+            r"required key not provided @ data\['id'\]",
             id="missing_id",
         ),
         pytest.param(
@@ -160,13 +161,66 @@ def test_image_configuration_errors(
             },
             id="type_based_organization",
         ),
+        pytest.param(
+            {
+                "defaults": {
+                    "type": "binary",
+                    "transparency": "chroma_key",
+                    "byte_order": "little_endian",
+                    "dither": "FloydSteinberg",
+                    "resize": "100x100",
+                    "invert_alpha": False,
+                },
+                "rgb565": {
+                    "alpha_channel": [
+                        {
+                            "id": "image_id",
+                            "file": "image.png",
+                            "transparency": "alpha_channel",
+                            "dither": "none",
+                        }
+                    ]
+                },
+                "binary": [
+                    {
+                        "id": "image_id",
+                        "file": "image.png",
+                        "transparency": "opaque",
+                    }
+                ],
+            },
+            id="type_based_with_defaults",
+        ),
+        pytest.param(
+            {
+                "defaults": {
+                    "type": "rgb565",
+                    "transparency": "alpha_channel",
+                },
+                "binary": {
+                    "opaque": [
+                        {
+                            "id": "image_id",
+                            "file": "image.png",
+                        }
+                    ],
+                },
+            },
+            id="binary_with_defaults",
+        ),
     ],
 )
 def test_image_configuration_success(
     config: dict[str, Any] | list[dict[str, Any]],
 ) -> None:
     """Test successful configuration validation."""
-    CONFIG_SCHEMA(config)
+    result = CONFIG_SCHEMA(config)
+    # All valid configurations should return a list of images
+    assert isinstance(result, list)
+    for key in (CONF_TYPE, CONF_ID, CONF_TRANSPARENCY, CONF_RAW_DATA_ID):
+        assert all(key in x for x in result), (
+            f"Missing key {key} in image configuration"
+        )
 
 
 def test_image_generation(
diff --git a/tests/component_tests/mipi_dsi/fixtures/mipi_dsi.yaml b/tests/component_tests/mipi_dsi/fixtures/mipi_dsi.yaml
new file mode 100644
index 0000000000..7d1fc84121
--- /dev/null
+++ b/tests/component_tests/mipi_dsi/fixtures/mipi_dsi.yaml
@@ -0,0 +1,28 @@
+esphome:
+  name: p4-test
+
+esp32:
+  board: esp32-p4-evboard
+  framework:
+    type: esp-idf
+
+psram:
+  speed: 200MHz
+
+esp_ldo:
+  - channel: 3
+    voltage: 2.5V
+
+display:
+  - platform: mipi_dsi
+    model: WAVESHARE-P4-NANO-10.1
+
+i2c:
+  sda: GPIO7
+  scl: GPIO8
+  scan: true
+  frequency: 400kHz
+
+#light:
+  #- platform: mipi_dsi
+    #id: backlight_id
diff --git a/tests/component_tests/mipi_dsi/test_mipi_dsi_config.py b/tests/component_tests/mipi_dsi/test_mipi_dsi_config.py
new file mode 100644
index 0000000000..f8a9af0279
--- /dev/null
+++ b/tests/component_tests/mipi_dsi/test_mipi_dsi_config.py
@@ -0,0 +1,127 @@
+"""Tests for mpi_dsi configuration validation."""
+
+from collections.abc import Callable
+from pathlib import Path
+
+import pytest
+
+from esphome import config_validation as cv
+from esphome.components.esp32 import KEY_BOARD, VARIANT_ESP32P4
+from esphome.const import (
+    CONF_DIMENSIONS,
+    CONF_HEIGHT,
+    CONF_INIT_SEQUENCE,
+    CONF_WIDTH,
+    KEY_VARIANT,
+    PlatformFramework,
+)
+from tests.component_tests.types import SetCoreConfigCallable
+
+
+def test_configuration_errors(set_core_config: SetCoreConfigCallable) -> None:
+    """Test detection of invalid configuration"""
+    set_core_config(
+        PlatformFramework.ESP32_IDF,
+        platform_data={KEY_BOARD: "esp32-p4-evboard", KEY_VARIANT: VARIANT_ESP32P4},
+    )
+
+    from esphome.components.mipi_dsi.display import CONFIG_SCHEMA
+
+    with pytest.raises(cv.Invalid, match="expected a dictionary"):
+        CONFIG_SCHEMA("a string")
+
+    with pytest.raises(
+        cv.Invalid, match=r"required key not provided @ data\['model'\]"
+    ):
+        CONFIG_SCHEMA({"id": "display_id"})
+
+    with pytest.raises(
+        cv.Invalid,
+        match=r"string value is None for dictionary value @ data\['lane_bit_rate'\]",
+    ):
+        CONFIG_SCHEMA(
+            {"id": "display_id", "model": "custom", "init_sequence": [[0x36, 0x01]]}
+        )
+
+    with pytest.raises(
+        cv.Invalid, match=r"required key not provided @ data\['dimensions'\]"
+    ):
+        CONFIG_SCHEMA(
+            {
+                "id": "display_id",
+                "model": "custom",
+                "init_sequence": [[0x36, 0x01]],
+                "lane_bit_rate": "1.5Gbps",
+            }
+        )
+
+    with pytest.raises(
+        cv.Invalid, match=r"required key not provided @ data\['init_sequence'\]"
+    ):
+        CONFIG_SCHEMA(
+            {
+                "model": "custom",
+                "lane_bit_rate": "1.5Gbps",
+                "dimensions": {"width": 320, "height": 240},
+            }
+        )
+
+
+def test_configuration_success(set_core_config: SetCoreConfigCallable) -> None:
+    """Test successful configuration validation."""
+    set_core_config(
+        PlatformFramework.ESP32_IDF,
+        platform_data={KEY_BOARD: "esp32-p4-evboard", KEY_VARIANT: VARIANT_ESP32P4},
+    )
+
+    from esphome.components.mipi_dsi.display import CONFIG_SCHEMA, MODELS
+
+    # Custom model with all options
+    CONFIG_SCHEMA(
+        {
+            "model": "custom",
+            "pixel_mode": "16bit",
+            "id": "display_id",
+            "byte_order": "little_endian",
+            "color_order": "rgb",
+            "reset_pin": 12,
+            "init_sequence": [[0xA0, 0x01]],
+            "dimensions": {
+                "width": 320,
+                "height": 240,
+            },
+            "invert_colors": True,
+            "transform": {"mirror_x": True, "mirror_y": True},
+            "pclk_frequency": "40MHz",
+            "lane_bit_rate": "1.5Gbps",
+            "lanes": 2,
+            "use_axis_flips": True,
+        }
+    )
+
+    # Test all models, providing default values where necessary
+    for name, model in MODELS.items():
+        config = {"model": name}
+        if model.initsequence is None:
+            config[CONF_INIT_SEQUENCE] = [[0xA0, 0x01]]
+        if not model.get_default(CONF_DIMENSIONS):
+            config[CONF_DIMENSIONS] = {CONF_WIDTH: 400, CONF_HEIGHT: 300}
+        if not model.get_default("lane_bit_rate"):
+            config["lane_bit_rate"] = "1.5Gbps"
+        CONFIG_SCHEMA(config)
+
+
+def test_code_generation(
+    generate_main: Callable[[str | Path], str],
+    component_fixture_path: Callable[[str], Path],
+) -> None:
+    """Test code generation for display."""
+
+    main_cpp = generate_main(component_fixture_path("mipi_dsi.yaml"))
+    assert (
+        "mipi_dsi_mipi_dsi_id = new mipi_dsi::MIPI_DSI(800, 1280, display::COLOR_BITNESS_565, 16);"
+        in main_cpp
+    )
+    assert "set_init_sequence({224, 1, 0, 225, 1, 147, 226, 1," in main_cpp
+    assert "mipi_dsi_mipi_dsi_id->set_lane_bit_rate(1500);" in main_cpp
+    # assert "backlight_id = new light::LightState(mipi_dsi_dsibacklight_id);" in main_cpp
diff --git a/tests/component_tests/mipi_spi/__init__.py b/tests/component_tests/mipi_spi/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/component_tests/mipi_spi/conftest.py b/tests/component_tests/mipi_spi/conftest.py
new file mode 100644
index 0000000000..c3070c7965
--- /dev/null
+++ b/tests/component_tests/mipi_spi/conftest.py
@@ -0,0 +1,43 @@
+"""Tests for mpip_spi configuration validation."""
+
+from collections.abc import Callable, Generator
+
+import pytest
+
+from esphome import config_validation as cv
+from esphome.components.esp32 import KEY_ESP32, KEY_VARIANT, VARIANTS
+from esphome.components.esp32.gpio import validate_gpio_pin
+from esphome.const import CONF_INPUT, CONF_OUTPUT
+from esphome.core import CORE
+from esphome.pins import gpio_pin_schema
+
+
+@pytest.fixture
+def choose_variant_with_pins() -> Generator[Callable[[list], None]]:
+    """
+    Set the ESP32 variant for the given model based on pins. For ESP32 only since the other platforms
+    do not have variants.
+    """
+
+    def chooser(pins: list) -> None:
+        for v in VARIANTS:
+            try:
+                CORE.data[KEY_ESP32][KEY_VARIANT] = v
+                for pin in pins:
+                    if pin is not None:
+                        pin = gpio_pin_schema(
+                            {
+                                CONF_INPUT: True,
+                                CONF_OUTPUT: True,
+                            },
+                            internal=True,
+                        )(pin)
+                        validate_gpio_pin(pin)
+                return
+            except cv.Invalid:
+                continue
+        raise cv.Invalid(
+            f"No compatible variant found for pins: {', '.join(map(str, pins))}"
+        )
+
+    yield chooser
diff --git a/tests/component_tests/mipi_spi/fixtures/lvgl.yaml b/tests/component_tests/mipi_spi/fixtures/lvgl.yaml
new file mode 100644
index 0000000000..bc800e1090
--- /dev/null
+++ b/tests/component_tests/mipi_spi/fixtures/lvgl.yaml
@@ -0,0 +1,25 @@
+esphome:
+  name: c3-7735
+
+esp32:
+  board: lolin_c3_mini
+
+spi:
+  mosi_pin:
+    number: GPIO2
+    ignore_strapping_warning: true
+  clk_pin: GPIO1
+
+display:
+  - platform: mipi_spi
+    data_rate: 20MHz
+    model: st7735
+    cs_pin:
+      number: GPIO8
+      ignore_strapping_warning: true
+    dc_pin:
+      number: GPIO3
+    reset_pin:
+      number: GPIO4
+
+lvgl:
diff --git a/tests/component_tests/mipi_spi/fixtures/native.yaml b/tests/component_tests/mipi_spi/fixtures/native.yaml
new file mode 100644
index 0000000000..6962ac25c7
--- /dev/null
+++ b/tests/component_tests/mipi_spi/fixtures/native.yaml
@@ -0,0 +1,20 @@
+esphome:
+  name: jc3636w518
+
+esp32:
+  board: esp32-s3-devkitc-1
+  framework:
+    type: esp-idf
+
+psram:
+  mode: octal
+
+spi:
+  id: display_qspi
+  type: quad
+  clk_pin: 9
+  data_pins: [11, 12, 13, 14]
+
+display:
+  - platform: mipi_spi
+    model: jc3636w518
diff --git a/tests/component_tests/mipi_spi/test_init.py b/tests/component_tests/mipi_spi/test_init.py
new file mode 100644
index 0000000000..e68f6fbfba
--- /dev/null
+++ b/tests/component_tests/mipi_spi/test_init.py
@@ -0,0 +1,360 @@
+"""Tests for mpip_spi configuration validation."""
+
+from collections.abc import Callable
+from pathlib import Path
+from typing import Any
+
+import pytest
+
+from esphome import config_validation as cv
+from esphome.components.esp32 import (
+    KEY_BOARD,
+    KEY_VARIANT,
+    VARIANT_ESP32,
+    VARIANT_ESP32S3,
+)
+from esphome.components.mipi import CONF_NATIVE_HEIGHT
+from esphome.components.mipi_spi.display import (
+    CONF_BUS_MODE,
+    CONFIG_SCHEMA,
+    FINAL_VALIDATE_SCHEMA,
+    MODELS,
+    dimension_schema,
+)
+from esphome.const import (
+    CONF_DC_PIN,
+    CONF_DIMENSIONS,
+    CONF_HEIGHT,
+    CONF_INIT_SEQUENCE,
+    CONF_WIDTH,
+    PlatformFramework,
+)
+from esphome.types import ConfigType
+from tests.component_tests.types import SetCoreConfigCallable
+
+
+def run_schema_validation(config: ConfigType) -> None:
+    """Run schema validation on a configuration."""
+    FINAL_VALIDATE_SCHEMA(CONFIG_SCHEMA(config))
+
+
+@pytest.mark.parametrize(
+    ("config", "error_match"),
+    [
+        pytest.param(
+            "a string",
+            "expected a dictionary",
+            id="invalid_string_config",
+        ),
+        pytest.param(
+            {"id": "display_id"},
+            r"required key not provided @ data\['model'\]",
+            id="missing_model",
+        ),
+        pytest.param(
+            {"id": "display_id", "model": "custom", "init_sequence": [[0x36, 0x01]]},
+            r"required key not provided @ data\['dimensions'\]",
+            id="missing_dimensions",
+        ),
+        pytest.param(
+            {
+                "model": "custom",
+                "dc_pin": 18,
+                "dimensions": {"width": 320, "height": 240},
+            },
+            r"required key not provided @ data\['init_sequence'\]",
+            id="missing_init_sequence",
+        ),
+        pytest.param(
+            {
+                "id": "display_id",
+                "model": "custom",
+                "dimensions": {"width": 260, "height": 260},
+                "draw_rounding": 13,
+                "init_sequence": [[0xA0, 0x01]],
+            },
+            r"value must be a power of two for dictionary value @ data\['draw_rounding'\]",
+            id="invalid_draw_rounding",
+        ),
+    ],
+)
+def test_basic_configuration_errors(
+    config: str | ConfigType,
+    error_match: str,
+    set_core_config: SetCoreConfigCallable,
+) -> None:
+    """Test basic configuration validation errors"""
+
+    set_core_config(
+        PlatformFramework.ESP32_IDF,
+        platform_data={KEY_BOARD: "esp32dev", KEY_VARIANT: VARIANT_ESP32},
+    )
+
+    with pytest.raises(cv.Invalid, match=error_match):
+        run_schema_validation(config)
+
+
+@pytest.mark.parametrize(
+    ("rounding", "config", "error_match"),
+    [
+        pytest.param(
+            4,
+            {"width": 320},
+            r"required key not provided @ data\['height'\]",
+            id="missing_height",
+        ),
+        pytest.param(
+            32,
+            {"width": 320, "height": 111},
+            "Dimensions and offsets must be divisible by 32",
+            id="dimensions_not_divisible",
+        ),
+    ],
+)
+def test_dimension_validation(
+    rounding: int,
+    config: ConfigType,
+    error_match: str,
+    set_core_config: SetCoreConfigCallable,
+) -> None:
+    """Test dimension-related validation errors"""
+
+    set_core_config(
+        PlatformFramework.ESP32_IDF,
+        platform_data={KEY_BOARD: "esp32dev", KEY_VARIANT: VARIANT_ESP32},
+    )
+
+    with pytest.raises(cv.Invalid, match=error_match):
+        dimension_schema(rounding)(config)
+
+
+@pytest.mark.parametrize(
+    ("config", "error_match"),
+    [
+        pytest.param(
+            {
+                "model": "JC3248W535",
+                "transform": {"mirror_x": False, "mirror_y": True, "swap_xy": True},
+            },
+            "Axis swapping not supported by this model",
+            id="axis_swapping_not_supported",
+        ),
+        pytest.param(
+            {
+                "model": "custom",
+                "dimensions": {"width": 320, "height": 240},
+                "transform": {"mirror_x": False, "mirror_y": True, "swap_xy": False},
+                "init_sequence": [[0x36, 0x01]],
+            },
+            r"transform is not supported when MADCTL \(0X36\) is in the init sequence",
+            id="transform_with_madctl",
+        ),
+        pytest.param(
+            {
+                "model": "custom",
+                "dimensions": {"width": 320, "height": 240},
+                "init_sequence": [[0x3A, 0x01]],
+            },
+            r"PIXFMT \(0X3A\) should not be in the init sequence, it will be set automatically",
+            id="pixfmt_in_init_sequence",
+        ),
+    ],
+)
+def test_transform_and_init_sequence_errors(
+    config: ConfigType,
+    error_match: str,
+    set_core_config: SetCoreConfigCallable,
+) -> None:
+    """Test transform and init sequence validation errors"""
+
+    set_core_config(
+        PlatformFramework.ESP32_IDF,
+        platform_data={KEY_BOARD: "esp32dev", KEY_VARIANT: VARIANT_ESP32},
+    )
+
+    with pytest.raises(cv.Invalid, match=error_match):
+        run_schema_validation(config)
+
+
+@pytest.mark.parametrize(
+    ("config", "error_match"),
+    [
+        pytest.param(
+            {"model": "t4-s3", "dc_pin": 18},
+            "DC pin is not supported in quad mode",
+            id="dc_pin_not_supported_quad_mode",
+        ),
+        pytest.param(
+            {"model": "t4-s3", "color_depth": 18},
+            "Unknown value '18', valid options are '16', '16bit",
+            id="invalid_color_depth_t4_s3",
+        ),
+        pytest.param(
+            {"model": "t-embed", "color_depth": 24},
+            "Unknown value '24', valid options are '16', '8",
+            id="invalid_color_depth_t_embed",
+        ),
+        pytest.param(
+            {"model": "ili9488"},
+            "DC pin is required in single mode",
+            id="dc_pin_required_single_mode",
+        ),
+        pytest.param(
+            {"model": "wt32-sc01-plus", "brightness": 128},
+            r"extra keys not allowed @ data\['brightness'\]",
+            id="brightness_not_supported",
+        ),
+        pytest.param(
+            {"model": "T-DISPLAY-S3-PRO"},
+            "PSRAM is required for this display",
+            id="psram_required",
+        ),
+    ],
+)
+def test_esp32s3_specific_errors(
+    config: ConfigType,
+    error_match: str,
+    set_core_config: SetCoreConfigCallable,
+) -> None:
+    """Test ESP32-S3 specific configuration errors"""
+
+    set_core_config(
+        PlatformFramework.ESP32_IDF,
+        platform_data={KEY_BOARD: "esp32dev", KEY_VARIANT: VARIANT_ESP32S3},
+    )
+
+    with pytest.raises(cv.Invalid, match=error_match):
+        run_schema_validation(config)
+
+
+def test_framework_specific_errors(
+    set_core_config: SetCoreConfigCallable,
+) -> None:
+    """Test framework-specific configuration errors"""
+
+    set_core_config(
+        PlatformFramework.ESP32_ARDUINO,
+        platform_data={KEY_BOARD: "esp32dev", KEY_VARIANT: VARIANT_ESP32},
+    )
+
+    with pytest.raises(
+        cv.Invalid,
+        match=r"This feature is only available with framework\(s\) esp-idf",
+    ):
+        run_schema_validation({"model": "wt32-sc01-plus"})
+
+
+def test_custom_model_with_all_options(
+    set_core_config: SetCoreConfigCallable,
+) -> None:
+    """Test custom model configuration with all available options."""
+    set_core_config(
+        PlatformFramework.ESP32_IDF,
+        platform_data={KEY_BOARD: "esp32dev", KEY_VARIANT: VARIANT_ESP32S3},
+    )
+
+    run_schema_validation(
+        {
+            "model": "custom",
+            "pixel_mode": "18bit",
+            "color_depth": 8,
+            "id": "display_id",
+            "byte_order": "little_endian",
+            "bus_mode": "single",
+            "color_order": "rgb",
+            "dc_pin": 11,
+            "reset_pin": 12,
+            "enable_pin": 13,
+            "cs_pin": 14,
+            "init_sequence": [[0xA0, 0x01]],
+            "dimensions": {
+                "width": 320,
+                "height": 240,
+                "offset_width": 32,
+                "offset_height": 32,
+            },
+            "invert_colors": True,
+            "transform": {"mirror_x": True, "mirror_y": True, "swap_xy": False},
+            "spi_mode": "mode0",
+            "data_rate": "40MHz",
+            "use_axis_flips": True,
+            "draw_rounding": 4,
+            "spi_16": True,
+            "buffer_size": 0.25,
+        }
+    )
+
+
+def test_all_predefined_models(
+    set_core_config: SetCoreConfigCallable,
+    set_component_config: Callable[[str, Any], None],
+    choose_variant_with_pins: Callable[[list], None],
+) -> None:
+    """Test all predefined display models validate successfully with appropriate defaults."""
+    set_core_config(
+        PlatformFramework.ESP32_IDF,
+        platform_data={KEY_BOARD: "esp32dev", KEY_VARIANT: VARIANT_ESP32S3},
+    )
+
+    # Enable PSRAM which is required for some models
+    set_component_config("psram", True)
+
+    # Test all models, providing default values where necessary
+    for name, model in MODELS.items():
+        config = {"model": name}
+
+        # Get the pins required by this model and find a compatible variant
+        pins = [
+            pin
+            for pin in [
+                model.get_default(pin, None)
+                for pin in ("dc_pin", "reset_pin", "cs_pin")
+            ]
+            if pin is not None
+        ]
+        choose_variant_with_pins(pins)
+
+        # Add required fields that don't have defaults
+        if (
+            not model.get_default(CONF_DC_PIN)
+            and model.get_default(CONF_BUS_MODE) != "quad"
+        ):
+            config[CONF_DC_PIN] = 14
+        if not model.get_default(CONF_NATIVE_HEIGHT):
+            config[CONF_DIMENSIONS] = {CONF_HEIGHT: 240, CONF_WIDTH: 320}
+        if model.initsequence is None:
+            config[CONF_INIT_SEQUENCE] = [[0xA0, 0x01]]
+
+        run_schema_validation(config)
+
+
+def test_native_generation(
+    generate_main: Callable[[str | Path], str],
+    component_fixture_path: Callable[[str], Path],
+) -> None:
+    """Test code generation for display."""
+
+    main_cpp = generate_main(component_fixture_path("native.yaml"))
+    assert (
+        "mipi_spi::MipiSpiBuffer()"
+        in main_cpp
+    )
+    assert "set_init_sequence({240, 1, 8, 242" in main_cpp
+    assert "show_test_card();" in main_cpp
+    assert "set_write_only(true);" in main_cpp
+
+
+def test_lvgl_generation(
+    generate_main: Callable[[str | Path], str],
+    component_fixture_path: Callable[[str], Path],
+) -> None:
+    """Test LVGL generation configuration."""
+
+    main_cpp = generate_main(component_fixture_path("lvgl.yaml"))
+    assert (
+        "mipi_spi::MipiSpi();"
+        in main_cpp
+    )
+    assert "set_init_sequence({1, 0, 10, 255, 177" in main_cpp
+    assert "show_test_card();" not in main_cpp
+    assert "set_auto_clear(false);" in main_cpp
diff --git a/tests/component_tests/ota/__init__.py b/tests/component_tests/ota/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/component_tests/packages/__init__.py b/tests/component_tests/packages/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/component_tests/packages/test_init.py b/tests/component_tests/packages/test_init.py
new file mode 100644
index 0000000000..779244e2ed
--- /dev/null
+++ b/tests/component_tests/packages/test_init.py
@@ -0,0 +1,114 @@
+"""Tests for the packages component skip_update functionality."""
+
+from pathlib import Path
+from typing import Any
+from unittest.mock import MagicMock
+
+from esphome.components.packages import do_packages_pass
+from esphome.const import CONF_FILES, CONF_PACKAGES, CONF_REFRESH, CONF_URL
+from esphome.util import OrderedDict
+
+
+def test_packages_skip_update_true(
+    tmp_path: Path, mock_clone_or_update: MagicMock, mock_load_yaml: MagicMock
+) -> None:
+    """Test that packages don't update when skip_update=True."""
+    # Set up mock to return our tmp_path
+    mock_clone_or_update.return_value = (tmp_path, None)
+
+    # Create the test yaml file
+    test_file = tmp_path / "test.yaml"
+    test_file.write_text("sensor: []")
+
+    # Set mock_load_yaml to return some valid config
+    mock_load_yaml.return_value = OrderedDict({"sensor": []})
+
+    config: dict[str, Any] = {
+        CONF_PACKAGES: {
+            "test_package": {
+                CONF_URL: "https://github.com/test/repo",
+                CONF_FILES: ["test.yaml"],
+                CONF_REFRESH: "1d",
+            }
+        }
+    }
+
+    # Call with skip_update=True
+    do_packages_pass(config, skip_update=True)
+
+    # Verify clone_or_update was called with NEVER_REFRESH
+    mock_clone_or_update.assert_called_once()
+    call_args = mock_clone_or_update.call_args
+    from esphome import git
+
+    assert call_args.kwargs["refresh"] == git.NEVER_REFRESH
+
+
+def test_packages_skip_update_false(
+    tmp_path: Path, mock_clone_or_update: MagicMock, mock_load_yaml: MagicMock
+) -> None:
+    """Test that packages update when skip_update=False."""
+    # Set up mock to return our tmp_path
+    mock_clone_or_update.return_value = (tmp_path, None)
+
+    # Create the test yaml file
+    test_file = tmp_path / "test.yaml"
+    test_file.write_text("sensor: []")
+
+    # Set mock_load_yaml to return some valid config
+    mock_load_yaml.return_value = OrderedDict({"sensor": []})
+
+    config: dict[str, Any] = {
+        CONF_PACKAGES: {
+            "test_package": {
+                CONF_URL: "https://github.com/test/repo",
+                CONF_FILES: ["test.yaml"],
+                CONF_REFRESH: "1d",
+            }
+        }
+    }
+
+    # Call with skip_update=False (default)
+    do_packages_pass(config, skip_update=False)
+
+    # Verify clone_or_update was called with actual refresh value
+    mock_clone_or_update.assert_called_once()
+    call_args = mock_clone_or_update.call_args
+    from esphome.core import TimePeriodSeconds
+
+    assert call_args.kwargs["refresh"] == TimePeriodSeconds(days=1)
+
+
+def test_packages_default_no_skip(
+    tmp_path: Path, mock_clone_or_update: MagicMock, mock_load_yaml: MagicMock
+) -> None:
+    """Test that packages update by default when skip_update not specified."""
+    # Set up mock to return our tmp_path
+    mock_clone_or_update.return_value = (tmp_path, None)
+
+    # Create the test yaml file
+    test_file = tmp_path / "test.yaml"
+    test_file.write_text("sensor: []")
+
+    # Set mock_load_yaml to return some valid config
+    mock_load_yaml.return_value = OrderedDict({"sensor": []})
+
+    config: dict[str, Any] = {
+        CONF_PACKAGES: {
+            "test_package": {
+                CONF_URL: "https://github.com/test/repo",
+                CONF_FILES: ["test.yaml"],
+                CONF_REFRESH: "1d",
+            }
+        }
+    }
+
+    # Call without skip_update parameter
+    do_packages_pass(config)
+
+    # Verify clone_or_update was called with actual refresh value
+    mock_clone_or_update.assert_called_once()
+    call_args = mock_clone_or_update.call_args
+    from esphome.core import TimePeriodSeconds
+
+    assert call_args.kwargs["refresh"] == TimePeriodSeconds(days=1)
diff --git a/tests/component_tests/psram/test_psram.py b/tests/component_tests/psram/test_psram.py
new file mode 100644
index 0000000000..3e40a8d192
--- /dev/null
+++ b/tests/component_tests/psram/test_psram.py
@@ -0,0 +1,194 @@
+"""Tests for PSRAM component."""
+
+from typing import Any
+
+import pytest
+
+from esphome.components.esp32.const import (
+    KEY_VARIANT,
+    VARIANT_ESP32,
+    VARIANT_ESP32C2,
+    VARIANT_ESP32C3,
+    VARIANT_ESP32C5,
+    VARIANT_ESP32C6,
+    VARIANT_ESP32H2,
+    VARIANT_ESP32P4,
+    VARIANT_ESP32S2,
+    VARIANT_ESP32S3,
+)
+import esphome.config_validation as cv
+from esphome.const import CONF_ESPHOME, PlatformFramework
+from tests.component_tests.types import SetCoreConfigCallable
+
+UNSUPPORTED_PSRAM_VARIANTS = [
+    VARIANT_ESP32C2,
+    VARIANT_ESP32C3,
+    VARIANT_ESP32C5,
+    VARIANT_ESP32C6,
+    VARIANT_ESP32H2,
+]
+
+SUPPORTED_PSRAM_VARIANTS = [
+    VARIANT_ESP32,
+    VARIANT_ESP32S2,
+    VARIANT_ESP32S3,
+    VARIANT_ESP32P4,
+]
+
+
+@pytest.mark.parametrize(
+    ("config", "error_match"),
+    [
+        pytest.param(
+            {},
+            r"PSRAM is not supported on this chip",
+            id="psram_not_supported",
+        ),
+    ],
+)
+@pytest.mark.parametrize("variant", UNSUPPORTED_PSRAM_VARIANTS)
+def test_psram_configuration_errors_unsupported_variants(
+    config: Any,
+    error_match: str,
+    variant: str,
+    set_core_config: SetCoreConfigCallable,
+) -> None:
+    set_core_config(
+        PlatformFramework.ESP32_IDF,
+        platform_data={KEY_VARIANT: variant},
+        full_config={CONF_ESPHOME: {}},
+    )
+    """Test detection of invalid PSRAM configuration on unsupported variants."""
+    from esphome.components.psram import CONFIG_SCHEMA
+
+    with pytest.raises(cv.Invalid, match=error_match):
+        CONFIG_SCHEMA(config)
+
+
+@pytest.mark.parametrize("variant", SUPPORTED_PSRAM_VARIANTS)
+def test_psram_configuration_valid_supported_variants(
+    variant: str,
+    set_core_config: SetCoreConfigCallable,
+) -> None:
+    set_core_config(
+        PlatformFramework.ESP32_IDF,
+        platform_data={KEY_VARIANT: variant},
+        full_config={
+            CONF_ESPHOME: {},
+            "esp32": {
+                "variant": variant,
+                "cpu_frequency": "160MHz",
+                "framework": {"type": "esp-idf"},
+            },
+        },
+    )
+    """Test that PSRAM configuration is valid on supported variants."""
+    from esphome.components.psram import CONFIG_SCHEMA, FINAL_VALIDATE_SCHEMA
+
+    # This should not raise an exception
+    config = CONFIG_SCHEMA({})
+    FINAL_VALIDATE_SCHEMA(config)
+
+
+def _setup_psram_final_validation_test(
+    esp32_config: dict,
+    set_core_config: SetCoreConfigCallable,
+    set_component_config: Any,
+) -> str:
+    """Helper function to set up ESP32 configuration for PSRAM final validation tests."""
+    # Use ESP32S3 for schema validation to allow all options, then override for final validation
+    schema_variant = "ESP32S3"
+    final_variant = esp32_config.get("variant", "ESP32S3")
+    full_esp32_config = {
+        "variant": final_variant,
+        "cpu_frequency": esp32_config.get("cpu_frequency", "240MHz"),
+        "framework": {"type": "esp-idf"},
+    }
+
+    set_core_config(
+        PlatformFramework.ESP32_IDF,
+        platform_data={KEY_VARIANT: schema_variant},
+        full_config={
+            CONF_ESPHOME: {},
+            "esp32": full_esp32_config,
+        },
+    )
+    set_component_config("esp32", full_esp32_config)
+
+    return final_variant
+
+
+@pytest.mark.parametrize(
+    ("config", "esp32_config", "expect_error", "error_match"),
+    [
+        pytest.param(
+            {"speed": "120MHz"},
+            {"cpu_frequency": "160MHz"},
+            True,
+            r"PSRAM 120MHz requires 240MHz CPU frequency",
+            id="120mhz_requires_240mhz_cpu",
+        ),
+        pytest.param(
+            {"mode": "octal"},
+            {"variant": "ESP32"},
+            True,
+            r"Octal PSRAM is only supported on ESP32-S3",
+            id="octal_mode_only_esp32s3",
+        ),
+        pytest.param(
+            {"mode": "quad", "enable_ecc": True},
+            {},
+            True,
+            r"ECC is only available in octal mode",
+            id="ecc_only_in_octal_mode",
+        ),
+        pytest.param(
+            {"speed": "120MHZ"},
+            {"cpu_frequency": "240MHZ"},
+            False,
+            None,
+            id="120mhz_with_240mhz_cpu",
+        ),
+        pytest.param(
+            {"mode": "octal"},
+            {"variant": "ESP32S3"},
+            False,
+            None,
+            id="octal_mode_on_esp32s3",
+        ),
+        pytest.param(
+            {"mode": "octal", "enable_ecc": True},
+            {"variant": "ESP32S3"},
+            False,
+            None,
+            id="ecc_in_octal_mode",
+        ),
+    ],
+)
+def test_psram_final_validation(
+    config: Any,
+    esp32_config: dict,
+    expect_error: bool,
+    error_match: str | None,
+    set_core_config: SetCoreConfigCallable,
+    set_component_config: Any,
+) -> None:
+    """Test PSRAM final validation for both error and valid cases."""
+    from esphome.components.psram import CONFIG_SCHEMA, FINAL_VALIDATE_SCHEMA
+    from esphome.core import CORE
+
+    final_variant = _setup_psram_final_validation_test(
+        esp32_config, set_core_config, set_component_config
+    )
+
+    validated_config = CONFIG_SCHEMA(config)
+
+    # Update CORE variant for final validation
+    CORE.data["esp32"][KEY_VARIANT] = final_variant
+
+    if expect_error:
+        with pytest.raises(cv.Invalid, match=error_match):
+            FINAL_VALIDATE_SCHEMA(validated_config)
+    else:
+        # This should not raise an exception
+        FINAL_VALIDATE_SCHEMA(validated_config)
diff --git a/tests/component_tests/sensor/__init__.py b/tests/component_tests/sensor/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/component_tests/text/__init__.py b/tests/component_tests/text/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/component_tests/text/test_text.yaml b/tests/component_tests/text/test_text.yaml
index d81c909f9d..9b05d59349 100644
--- a/tests/component_tests/text/test_text.yaml
+++ b/tests/component_tests/text/test_text.yaml
@@ -4,6 +4,8 @@ esphome:
 esp32:
   board: esp32dev
 
+logger:
+
 text:
   - platform: template
     name: "test 1 text"
diff --git a/tests/component_tests/text_sensor/__init__.py b/tests/component_tests/text_sensor/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/component_tests/types.py b/tests/component_tests/types.py
new file mode 100644
index 0000000000..ee9d317339
--- /dev/null
+++ b/tests/component_tests/types.py
@@ -0,0 +1,22 @@
+"""Type definitions for component tests."""
+
+from __future__ import annotations
+
+from typing import Protocol
+
+from esphome.const import PlatformFramework
+from esphome.types import ConfigType
+
+
+class SetCoreConfigCallable(Protocol):
+    """Protocol for the set_core_config fixture setter function."""
+
+    def __call__(  # noqa: E704
+        self,
+        platform_framework: PlatformFramework,
+        /,
+        *,
+        core_data: ConfigType | None = None,
+        platform_data: ConfigType | None = None,
+        full_config: dict[str, ConfigType] | None = None,
+    ) -> None: ...
diff --git a/tests/component_tests/web_server/__init__.py b/tests/component_tests/web_server/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/component_tests/web_server/test_ota_migration.py b/tests/component_tests/web_server/test_ota_migration.py
index 7f34ec75f6..da25bab0e8 100644
--- a/tests/component_tests/web_server/test_ota_migration.py
+++ b/tests/component_tests/web_server/test_ota_migration.py
@@ -8,31 +8,31 @@ from esphome.types import ConfigType
 
 def test_web_server_ota_true_fails_validation() -> None:
     """Test that web_server with ota: true fails validation with helpful message."""
-    from esphome.components.web_server import validate_ota_removed
+    from esphome.components.web_server import validate_ota
 
     # Config with ota: true should fail
     config: ConfigType = {"ota": True}
 
     with pytest.raises(cv.Invalid) as exc_info:
-        validate_ota_removed(config)
+        validate_ota(config)
 
     # Check error message contains migration instructions
     error_msg = str(exc_info.value)
-    assert "has been removed from 'web_server'" in error_msg
+    assert "only accepts 'false' to disable OTA" in error_msg
     assert "platform: web_server" in error_msg
     assert "ota:" in error_msg
 
 
 def test_web_server_ota_false_passes_validation() -> None:
     """Test that web_server with ota: false passes validation."""
-    from esphome.components.web_server import validate_ota_removed
+    from esphome.components.web_server import validate_ota
 
     # Config with ota: false should pass
     config: ConfigType = {"ota": False}
-    result = validate_ota_removed(config)
+    result = validate_ota(config)
     assert result == config
 
     # Config without ota should also pass
     config: ConfigType = {}
-    result = validate_ota_removed(config)
+    result = validate_ota(config)
     assert result == config
diff --git a/tests/components/a01nyub/common.yaml b/tests/components/a01nyub/common.yaml
index 0717acfff7..fc0b2d3bfb 100644
--- a/tests/components/a01nyub/common.yaml
+++ b/tests/components/a01nyub/common.yaml
@@ -1,11 +1,4 @@
-uart:
-  - id: uart_a01nyub
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: a01nyub
     id: a01nyub_sensor
     name: a01nyub Distance
-    uart_id: uart_a01nyub
diff --git a/tests/components/a01nyub/test.esp32-ard.yaml b/tests/components/a01nyub/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/a01nyub/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/a01nyub/test.esp32-c3-ard.yaml b/tests/components/a01nyub/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/a01nyub/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/a01nyub/test.esp32-c3-idf.yaml b/tests/components/a01nyub/test.esp32-c3-idf.yaml
index b516342f3b..2cda8deaf9 100644
--- a/tests/components/a01nyub/test.esp32-c3-idf.yaml
+++ b/tests/components/a01nyub/test.esp32-c3-idf.yaml
@@ -1,3 +1,6 @@
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
+
 substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
diff --git a/tests/components/a01nyub/test.esp32-idf.yaml b/tests/components/a01nyub/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/a01nyub/test.esp32-idf.yaml
+++ b/tests/components/a01nyub/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/a01nyub/test.esp8266-ard.yaml b/tests/components/a01nyub/test.esp8266-ard.yaml
index b516342f3b..5a05efa259 100644
--- a/tests/components/a01nyub/test.esp8266-ard.yaml
+++ b/tests/components/a01nyub/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/a01nyub/test.rp2040-ard.yaml b/tests/components/a01nyub/test.rp2040-ard.yaml
index b516342f3b..f1df2daf83 100644
--- a/tests/components/a01nyub/test.rp2040-ard.yaml
+++ b/tests/components/a01nyub/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/a02yyuw/common.yaml b/tests/components/a02yyuw/common.yaml
index b2e5927ff4..4de8a6eb67 100644
--- a/tests/components/a02yyuw/common.yaml
+++ b/tests/components/a02yyuw/common.yaml
@@ -1,11 +1,4 @@
-uart:
-  - id: uart_a02yyuw
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: a02yyuw
     id: a02yyuw_sensor
     name: a02yyuw Distance
-    uart_id: uart_a02yyuw
diff --git a/tests/components/a02yyuw/test.esp32-ard.yaml b/tests/components/a02yyuw/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/a02yyuw/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/a02yyuw/test.esp32-c3-ard.yaml b/tests/components/a02yyuw/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/a02yyuw/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/a02yyuw/test.esp32-c3-idf.yaml b/tests/components/a02yyuw/test.esp32-c3-idf.yaml
index b516342f3b..2cda8deaf9 100644
--- a/tests/components/a02yyuw/test.esp32-c3-idf.yaml
+++ b/tests/components/a02yyuw/test.esp32-c3-idf.yaml
@@ -1,3 +1,6 @@
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
+
 substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
diff --git a/tests/components/a02yyuw/test.esp32-idf.yaml b/tests/components/a02yyuw/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/a02yyuw/test.esp32-idf.yaml
+++ b/tests/components/a02yyuw/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/a02yyuw/test.esp8266-ard.yaml b/tests/components/a02yyuw/test.esp8266-ard.yaml
index b516342f3b..5a05efa259 100644
--- a/tests/components/a02yyuw/test.esp8266-ard.yaml
+++ b/tests/components/a02yyuw/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/a02yyuw/test.rp2040-ard.yaml b/tests/components/a02yyuw/test.rp2040-ard.yaml
index b516342f3b..f1df2daf83 100644
--- a/tests/components/a02yyuw/test.rp2040-ard.yaml
+++ b/tests/components/a02yyuw/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/a4988/test.esp32-ard.yaml b/tests/components/a4988/test.esp32-ard.yaml
deleted file mode 100644
index 1ca8c0c084..0000000000
--- a/tests/components/a4988/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  step_pin: GPIO22
-  dir_pin: GPIO23
-  sleep_pin: GPIO25
-
-<<: !include common.yaml
diff --git a/tests/components/a4988/test.esp32-c3-ard.yaml b/tests/components/a4988/test.esp32-c3-ard.yaml
deleted file mode 100644
index 25caba75b5..0000000000
--- a/tests/components/a4988/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  step_pin: GPIO2
-  dir_pin: GPIO3
-  sleep_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/a4988/test.esp32-idf.yaml b/tests/components/a4988/test.esp32-idf.yaml
index 1ca8c0c084..7d46b048e1 100644
--- a/tests/components/a4988/test.esp32-idf.yaml
+++ b/tests/components/a4988/test.esp32-idf.yaml
@@ -1,6 +1,6 @@
 substitutions:
   step_pin: GPIO22
-  dir_pin: GPIO23
+  dir_pin: GPIO4
   sleep_pin: GPIO25
 
 <<: !include common.yaml
diff --git a/tests/components/a4988/test.esp8266-ard.yaml b/tests/components/a4988/test.esp8266-ard.yaml
index 22b5677d27..5b1b1293be 100644
--- a/tests/components/a4988/test.esp8266-ard.yaml
+++ b/tests/components/a4988/test.esp8266-ard.yaml
@@ -1,6 +1,6 @@
 substitutions:
   step_pin: GPIO1
   dir_pin: GPIO2
-  sleep_pin: GPIO5
+  sleep_pin: GPIO0
 
 <<: !include common.yaml
diff --git a/tests/components/absolute_humidity/common.yaml b/tests/components/absolute_humidity/common.yaml
index 87a99f5206..026f88654f 100644
--- a/tests/components/absolute_humidity/common.yaml
+++ b/tests/components/absolute_humidity/common.yaml
@@ -8,14 +8,12 @@ sensor:
     lambda: |-
       if (millis() > 10000) {
         return 0.6;
-      } else {
-        return 0.0;
       }
+      return 0.0;
   - platform: template
     id: template_temperature
     lambda: |-
       if (millis() > 10000) {
         return 42.0;
-      } else {
-        return 0.0;
       }
+      return 0.0;
diff --git a/tests/components/ac_dimmer/test.esp32-ard.yaml b/tests/components/ac_dimmer/test.esp32-ard.yaml
index 3ec069f430..eaa4901f03 100644
--- a/tests/components/ac_dimmer/test.esp32-ard.yaml
+++ b/tests/components/ac_dimmer/test.esp32-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  gate_pin: GPIO18
-  zero_cross_pin: GPIO19
+  gate_pin: GPIO4
+  zero_cross_pin: GPIO5
 
 <<: !include common.yaml
diff --git a/tests/components/ac_dimmer/test.esp32-c3-ard.yaml b/tests/components/ac_dimmer/test.esp32-c3-ard.yaml
deleted file mode 100644
index 5d2d42b713..0000000000
--- a/tests/components/ac_dimmer/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  gate_pin: GPIO5
-  zero_cross_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ac_dimmer/test.esp8266-ard.yaml b/tests/components/ac_dimmer/test.esp8266-ard.yaml
index 5d2d42b713..2f50b04956 100644
--- a/tests/components/ac_dimmer/test.esp8266-ard.yaml
+++ b/tests/components/ac_dimmer/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  gate_pin: GPIO5
-  zero_cross_pin: GPIO4
+  gate_pin: GPIO0
+  zero_cross_pin: GPIO2
 
 <<: !include common.yaml
diff --git a/tests/components/adc/test.bk72xx-ard.yaml b/tests/components/adc/test.bk72xx-ard.yaml
index 491d0af3b9..0645333a81 100644
--- a/tests/components/adc/test.bk72xx-ard.yaml
+++ b/tests/components/adc/test.bk72xx-ard.yaml
@@ -1,4 +1,11 @@
 sensor:
-  - platform: adc
+  - id: my_sensor
+    platform: adc
     pin: P23
-    name: Basic ADC Test
+    name: ADC Test sensor
+    update_interval: "1:01"
+    unit_of_measurement: "°C"
+    icon: "mdi:water-percent"
+    accuracy_decimals: 5
+    setup_priority: -100
+    force_update: true
diff --git a/tests/components/adc/test.esp32-c3-ard.yaml b/tests/components/adc/test.esp32-c3-ard.yaml
deleted file mode 100644
index e74477c582..0000000000
--- a/tests/components/adc/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-sensor:
-  - platform: adc
-    id: my_sensor
-    pin: 4
-    attenuation: 12db
diff --git a/tests/components/adc/test.esp32-ard.yaml b/tests/components/adc/test.esp32-c3-idf.yaml
similarity index 71%
rename from tests/components/adc/test.esp32-ard.yaml
rename to tests/components/adc/test.esp32-c3-idf.yaml
index 923fd0d706..e764f0fe21 100644
--- a/tests/components/adc/test.esp32-ard.yaml
+++ b/tests/components/adc/test.esp32-c3-idf.yaml
@@ -1,7 +1,8 @@
 sensor:
-  - platform: adc
-    pin: A0
-    name: Living Room Brightness
+  - id: my_sensor
+    platform: adc
+    pin: GPIO1
+    name: ADC Test sensor
     update_interval: "1:01"
     attenuation: 2.5db
     unit_of_measurement: "°C"
diff --git a/tests/components/adc/test.esp32-idf.yaml b/tests/components/adc/test.esp32-idf.yaml
index 923fd0d706..ff1e3bb919 100644
--- a/tests/components/adc/test.esp32-idf.yaml
+++ b/tests/components/adc/test.esp32-idf.yaml
@@ -1,7 +1,8 @@
 sensor:
-  - platform: adc
+  - id: my_sensor
+    platform: adc
     pin: A0
-    name: Living Room Brightness
+    name: ADC Test sensor
     update_interval: "1:01"
     attenuation: 2.5db
     unit_of_measurement: "°C"
diff --git a/tests/components/adc/test.esp32-p4-idf.yaml b/tests/components/adc/test.esp32-p4-idf.yaml
new file mode 100644
index 0000000000..b77dc299c2
--- /dev/null
+++ b/tests/components/adc/test.esp32-p4-idf.yaml
@@ -0,0 +1,12 @@
+sensor:
+  - id: my_sensor
+    platform: adc
+    pin: GPIO16
+    name: ADC Test sensor
+    update_interval: "1:01"
+    attenuation: 2.5db
+    unit_of_measurement: "°C"
+    icon: "mdi:water-percent"
+    accuracy_decimals: 5
+    setup_priority: -100
+    force_update: true
diff --git a/tests/components/adc/test.esp32-s2-ard.yaml b/tests/components/adc/test.esp32-s2-ard.yaml
deleted file mode 100644
index e1a6bc22e5..0000000000
--- a/tests/components/adc/test.esp32-s2-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-sensor:
-  - platform: adc
-    id: my_sensor
-    pin: 1
-    attenuation: 12db
diff --git a/tests/components/adc/test.esp32-s2-idf.yaml b/tests/components/adc/test.esp32-s2-idf.yaml
new file mode 100644
index 0000000000..e764f0fe21
--- /dev/null
+++ b/tests/components/adc/test.esp32-s2-idf.yaml
@@ -0,0 +1,12 @@
+sensor:
+  - id: my_sensor
+    platform: adc
+    pin: GPIO1
+    name: ADC Test sensor
+    update_interval: "1:01"
+    attenuation: 2.5db
+    unit_of_measurement: "°C"
+    icon: "mdi:water-percent"
+    accuracy_decimals: 5
+    setup_priority: -100
+    force_update: true
diff --git a/tests/components/adc/test.esp32-s3-ard.yaml b/tests/components/adc/test.esp32-s3-ard.yaml
deleted file mode 100644
index e1a6bc22e5..0000000000
--- a/tests/components/adc/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-sensor:
-  - platform: adc
-    id: my_sensor
-    pin: 1
-    attenuation: 12db
diff --git a/tests/components/adc/test.esp32-s3-idf.yaml b/tests/components/adc/test.esp32-s3-idf.yaml
new file mode 100644
index 0000000000..e764f0fe21
--- /dev/null
+++ b/tests/components/adc/test.esp32-s3-idf.yaml
@@ -0,0 +1,12 @@
+sensor:
+  - id: my_sensor
+    platform: adc
+    pin: GPIO1
+    name: ADC Test sensor
+    update_interval: "1:01"
+    attenuation: 2.5db
+    unit_of_measurement: "°C"
+    icon: "mdi:water-percent"
+    accuracy_decimals: 5
+    setup_priority: -100
+    force_update: true
diff --git a/tests/components/adc/test.esp8266-ard.yaml b/tests/components/adc/test.esp8266-ard.yaml
index 1ef79c7ca1..4cc865bb5d 100644
--- a/tests/components/adc/test.esp8266-ard.yaml
+++ b/tests/components/adc/test.esp8266-ard.yaml
@@ -1,4 +1,11 @@
 sensor:
-  - platform: adc
-    id: my_sensor
+  - id: my_sensor
+    platform: adc
     pin: VCC
+    name: ADC Test sensor
+    update_interval: "1:01"
+    unit_of_measurement: "°C"
+    icon: "mdi:water-percent"
+    accuracy_decimals: 5
+    setup_priority: -100
+    force_update: true
diff --git a/tests/components/adc/test.ln882x-ard.yaml b/tests/components/adc/test.ln882x-ard.yaml
index 92c76ca9b3..face38b647 100644
--- a/tests/components/adc/test.ln882x-ard.yaml
+++ b/tests/components/adc/test.ln882x-ard.yaml
@@ -1,4 +1,11 @@
 sensor:
-  - platform: adc
-    pin: PA0
-    name: Basic ADC Test
+  - id: my_sensor
+    platform: adc
+    pin: A5
+    name: ADC Test sensor
+    update_interval: "1:01"
+    unit_of_measurement: "°C"
+    icon: "mdi:water-percent"
+    accuracy_decimals: 5
+    setup_priority: -100
+    force_update: true
diff --git a/tests/components/adc/test.nrf52-adafruit.yaml b/tests/components/adc/test.nrf52-adafruit.yaml
new file mode 100644
index 0000000000..4be5b4b5c2
--- /dev/null
+++ b/tests/components/adc/test.nrf52-adafruit.yaml
@@ -0,0 +1,23 @@
+sensor:
+  - platform: adc
+    pin: VDDHDIV5
+    name: "VDDH Voltage"
+    update_interval: 5sec
+    filters:
+      - multiply: 5
+  - platform: adc
+    pin: VDD
+    name: "VDD Voltage"
+    update_interval: 5sec
+  - platform: adc
+    pin: AIN0
+    name: "AIN0 Voltage"
+    update_interval: 5sec
+  - platform: adc
+    pin: P0.03
+    name: "AIN1 Voltage"
+    update_interval: 5sec
+  - platform: adc
+    name: "AIN2 Voltage"
+    update_interval: 5sec
+    pin: 4
diff --git a/tests/components/adc/test.nrf52-mcumgr.yaml b/tests/components/adc/test.nrf52-mcumgr.yaml
new file mode 100644
index 0000000000..4be5b4b5c2
--- /dev/null
+++ b/tests/components/adc/test.nrf52-mcumgr.yaml
@@ -0,0 +1,23 @@
+sensor:
+  - platform: adc
+    pin: VDDHDIV5
+    name: "VDDH Voltage"
+    update_interval: 5sec
+    filters:
+      - multiply: 5
+  - platform: adc
+    pin: VDD
+    name: "VDD Voltage"
+    update_interval: 5sec
+  - platform: adc
+    pin: AIN0
+    name: "AIN0 Voltage"
+    update_interval: 5sec
+  - platform: adc
+    pin: P0.03
+    name: "AIN1 Voltage"
+    update_interval: 5sec
+  - platform: adc
+    name: "AIN2 Voltage"
+    update_interval: 5sec
+    pin: 4
diff --git a/tests/components/adc/test.rp2040-ard.yaml b/tests/components/adc/test.rp2040-ard.yaml
index 200b802a4d..4cc865bb5d 100644
--- a/tests/components/adc/test.rp2040-ard.yaml
+++ b/tests/components/adc/test.rp2040-ard.yaml
@@ -1,4 +1,11 @@
 sensor:
-  - platform: adc
+  - id: my_sensor
+    platform: adc
     pin: VCC
-    name: VSYS
+    name: ADC Test sensor
+    update_interval: "1:01"
+    unit_of_measurement: "°C"
+    icon: "mdi:water-percent"
+    accuracy_decimals: 5
+    setup_priority: -100
+    force_update: true
diff --git a/tests/components/adc128s102/common.yaml b/tests/components/adc128s102/common.yaml
index 5f1638a7e2..b909310bdf 100644
--- a/tests/components/adc128s102/common.yaml
+++ b/tests/components/adc128s102/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_adc128s102
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 adc128s102:
   cs_pin: ${cs_pin}
   id: adc128s102_adc
diff --git a/tests/components/adc128s102/test.esp32-ard.yaml b/tests/components/adc128s102/test.esp32-ard.yaml
deleted file mode 100644
index aba72f0614..0000000000
--- a/tests/components/adc128s102/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO12
-
-<<: !include common.yaml
diff --git a/tests/components/adc128s102/test.esp32-c3-ard.yaml b/tests/components/adc128s102/test.esp32-c3-ard.yaml
deleted file mode 100644
index 24da4b5452..0000000000
--- a/tests/components/adc128s102/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/adc128s102/test.esp32-c3-idf.yaml b/tests/components/adc128s102/test.esp32-c3-idf.yaml
index 24da4b5452..a60568a736 100644
--- a/tests/components/adc128s102/test.esp32-c3-idf.yaml
+++ b/tests/components/adc128s102/test.esp32-c3-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO2
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/adc128s102/test.esp32-idf.yaml b/tests/components/adc128s102/test.esp32-idf.yaml
index aba72f0614..9bb524aa65 100644
--- a/tests/components/adc128s102/test.esp32-idf.yaml
+++ b/tests/components/adc128s102/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO12
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/adc128s102/test.esp8266-ard.yaml b/tests/components/adc128s102/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/adc128s102/test.esp8266-ard.yaml
+++ b/tests/components/adc128s102/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/adc128s102/test.rp2040-ard.yaml b/tests/components/adc128s102/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/adc128s102/test.rp2040-ard.yaml
+++ b/tests/components/adc128s102/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/addressable_light/esp32_rmt_led_strip.esp32-ard.yaml b/tests/components/addressable_light/esp32_rmt_led_strip.esp32-ard.yaml
deleted file mode 100644
index d93c554dae..0000000000
--- a/tests/components/addressable_light/esp32_rmt_led_strip.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common-ard-esp32_rmt_led_strip.yaml
diff --git a/tests/components/addressable_light/esp32_rmt_led_strip.esp32-c3-ard.yaml b/tests/components/addressable_light/esp32_rmt_led_strip.esp32-c3-ard.yaml
deleted file mode 100644
index d93c554dae..0000000000
--- a/tests/components/addressable_light/esp32_rmt_led_strip.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common-ard-esp32_rmt_led_strip.yaml
diff --git a/tests/components/addressable_light/fastled_clockless.esp32-ard.yaml b/tests/components/addressable_light/fastled_clockless.esp32-ard.yaml
deleted file mode 100644
index 78eb5d7fdb..0000000000
--- a/tests/components/addressable_light/fastled_clockless.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common-ard-fastled.yaml
diff --git a/tests/components/ade7880/common.yaml b/tests/components/ade7880/common.yaml
index 48c22c8485..0b0b560282 100644
--- a/tests/components/ade7880/common.yaml
+++ b/tests/components/ade7880/common.yaml
@@ -1,23 +1,18 @@
-i2c:
-  - id: i2c_ade7880
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ade7880
-    i2c_id: i2c_ade7880
+    i2c_id: i2c_bus
     irq0_pin: ${irq0_pin}
     irq1_pin: ${irq1_pin}
     reset_pin: ${reset_pin}
     frequency: 60Hz
     phase_a:
       name: Channel A
-      voltage: Channel A Voltage
-      current: Channel A Current
-      active_power: Channel A Active Power
-      power_factor: Channel A Power Factor
-      forward_active_energy: Channel A Forward Active Energy
-      reverse_active_energy: Channel A Reverse Active Energy
+      voltage: Voltage
+      current: Current
+      active_power: Active Power
+      power_factor: Power Factor
+      forward_active_energy: Forward Active Energy
+      reverse_active_energy: Reverse Active Energy
       calibration:
         current_gain: 3116628
         voltage_gain: -757178
@@ -25,12 +20,12 @@ sensor:
         phase_angle: 188
     phase_b:
       name: Channel B
-      voltage: Channel B Voltage
-      current: Channel B Current
-      active_power: Channel B Active Power
-      power_factor: Channel B Power Factor
-      forward_active_energy: Channel B Forward Active Energy
-      reverse_active_energy: Channel B Reverse Active Energy
+      voltage: Voltage
+      current: Current
+      active_power: Active Power
+      power_factor: Power Factor
+      forward_active_energy: Forward Active Energy
+      reverse_active_energy: Reverse Active Energy
       calibration:
         current_gain: 3133655
         voltage_gain: -755235
@@ -38,12 +33,12 @@ sensor:
         phase_angle: 188
     phase_c:
       name: Channel C
-      voltage: Channel C Voltage
-      current: Channel C Current
-      active_power: Channel C Active Power
-      power_factor: Channel C Power Factor
-      forward_active_energy: Channel C Forward Active Energy
-      reverse_active_energy: Channel C Reverse Active Energy
+      voltage: Voltage
+      current: Current
+      active_power: Active Power
+      power_factor: Power Factor
+      forward_active_energy: Forward Active Energy
+      reverse_active_energy: Reverse Active Energy
       calibration:
         current_gain: 3111158
         voltage_gain: -743813
@@ -51,6 +46,6 @@ sensor:
         phase_angle: 180
     neutral:
       name: Neutral
-      current: Neutral Current
+      current: Current
       calibration:
         current_gain: 3189
diff --git a/tests/components/ade7880/test.esp32-ard.yaml b/tests/components/ade7880/test.esp32-ard.yaml
deleted file mode 100644
index 685b49ff32..0000000000
--- a/tests/components/ade7880/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  irq0_pin: GPIO13
-  irq1_pin: GPIO15
-  reset_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/ade7880/test.esp32-c3-ard.yaml b/tests/components/ade7880/test.esp32-c3-ard.yaml
deleted file mode 100644
index 87db3e9427..0000000000
--- a/tests/components/ade7880/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  irq0_pin: GPIO6
-  irq1_pin: GPIO7
-  reset_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/ade7880/test.esp32-c3-idf.yaml b/tests/components/ade7880/test.esp32-c3-idf.yaml
index 87db3e9427..7d5b41fc5a 100644
--- a/tests/components/ade7880/test.esp32-c3-idf.yaml
+++ b/tests/components/ade7880/test.esp32-c3-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq0_pin: GPIO6
   irq1_pin: GPIO7
-  reset_pin: GPIO10
+  reset_pin: GPIO9
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ade7880/test.esp32-idf.yaml b/tests/components/ade7880/test.esp32-idf.yaml
index 685b49ff32..9db2e50049 100644
--- a/tests/components/ade7880/test.esp32-idf.yaml
+++ b/tests/components/ade7880/test.esp32-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq0_pin: GPIO13
   irq1_pin: GPIO15
-  reset_pin: GPIO16
+  reset_pin: GPIO12
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ade7880/test.esp8266-ard.yaml b/tests/components/ade7880/test.esp8266-ard.yaml
index 685b49ff32..8b5e47f0b5 100644
--- a/tests/components/ade7880/test.esp8266-ard.yaml
+++ b/tests/components/ade7880/test.esp8266-ard.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  irq0_pin: GPIO13
+  irq0_pin: GPIO0
   irq1_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ade7880/test.rp2040-ard.yaml b/tests/components/ade7880/test.rp2040-ard.yaml
index 685b49ff32..f531f852ae 100644
--- a/tests/components/ade7880/test.rp2040-ard.yaml
+++ b/tests/components/ade7880/test.rp2040-ard.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq0_pin: GPIO13
   irq1_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ade7953_i2c/common.yaml b/tests/components/ade7953_i2c/common.yaml
index a2d163567d..5253759888 100644
--- a/tests/components/ade7953_i2c/common.yaml
+++ b/tests/components/ade7953_i2c/common.yaml
@@ -1,20 +1,16 @@
-i2c:
-  - id: i2c_ade7953
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ade7953_i2c
+    i2c_id: i2c_bus
     irq_pin: ${irq_pin}
     voltage:
       name: ADE7953 Voltage
-      id: ade7953_voltage
+      id: ade7953_i2c_voltage
     current_a:
       name: ADE7953 Current A
-      id: ade7953_current_a
+      id: ade7953_i2c_current_a
     current_b:
       name: ADE7953 Current B
-      id: ade7953_current_b
+      id: ade7953_i2c_current_b
     power_factor_a:
       name: ADE7953 Power Factor A
     power_factor_b:
diff --git a/tests/components/ade7953_i2c/test.esp32-ard.yaml b/tests/components/ade7953_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 2c57d412f6..0000000000
--- a/tests/components/ade7953_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  irq_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/ade7953_i2c/test.esp32-c3-ard.yaml b/tests/components/ade7953_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index 799acabd5a..0000000000
--- a/tests/components/ade7953_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  irq_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/ade7953_i2c/test.esp32-c3-idf.yaml b/tests/components/ade7953_i2c/test.esp32-c3-idf.yaml
index 799acabd5a..59296a1e6e 100644
--- a/tests/components/ade7953_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/ade7953_i2c/test.esp32-c3-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ade7953_i2c/test.esp32-idf.yaml b/tests/components/ade7953_i2c/test.esp32-idf.yaml
index 2c57d412f6..49629536e7 100644
--- a/tests/components/ade7953_i2c/test.esp32-idf.yaml
+++ b/tests/components/ade7953_i2c/test.esp32-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   irq_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ade7953_i2c/test.esp8266-ard.yaml b/tests/components/ade7953_i2c/test.esp8266-ard.yaml
index c8e6a43f44..dc7609ab37 100644
--- a/tests/components/ade7953_i2c/test.esp8266-ard.yaml
+++ b/tests/components/ade7953_i2c/test.esp8266-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ade7953_i2c/test.rp2040-ard.yaml b/tests/components/ade7953_i2c/test.rp2040-ard.yaml
index 799acabd5a..b80562ad22 100644
--- a/tests/components/ade7953_i2c/test.rp2040-ard.yaml
+++ b/tests/components/ade7953_i2c/test.rp2040-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ade7953_spi/common.yaml b/tests/components/ade7953_spi/common.yaml
index 706f31f22c..f66ab697a1 100644
--- a/tests/components/ade7953_spi/common.yaml
+++ b/tests/components/ade7953_spi/common.yaml
@@ -1,22 +1,16 @@
-spi:
-  - id: spi_ade7953
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: ade7953_spi
     cs_pin: ${cs_pin}
     irq_pin: ${irq_pin}
     voltage:
       name: ADE7953 Voltage
-      id: ade7953_voltage
+      id: ade7953_spi_voltage
     current_a:
       name: ADE7953 Current A
-      id: ade7953_current_a
+      id: ade7953_spi_current_a
     current_b:
       name: ADE7953 Current B
-      id: ade7953_current_b
+      id: ade7953_spi_current_b
     power_factor_a:
       name: ADE7953 Power Factor A
     power_factor_b:
diff --git a/tests/components/ade7953_spi/test.esp32-ard.yaml b/tests/components/ade7953_spi/test.esp32-ard.yaml
deleted file mode 100644
index e00f522dd4..0000000000
--- a/tests/components/ade7953_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  irq_pin: GPIO13
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/ade7953_spi/test.esp32-c3-ard.yaml b/tests/components/ade7953_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index fcf35f528e..0000000000
--- a/tests/components/ade7953_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  irq_pin: GPIO9
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/ade7953_spi/test.esp32-c3-idf.yaml b/tests/components/ade7953_spi/test.esp32-c3-idf.yaml
index fcf35f528e..5e7e2dc82c 100644
--- a/tests/components/ade7953_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/ade7953_spi/test.esp32-c3-idf.yaml
@@ -1,8 +1,7 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   irq_pin: GPIO9
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ade7953_spi/test.esp32-idf.yaml b/tests/components/ade7953_spi/test.esp32-idf.yaml
index e00f522dd4..19791e24b7 100644
--- a/tests/components/ade7953_spi/test.esp32-idf.yaml
+++ b/tests/components/ade7953_spi/test.esp32-idf.yaml
@@ -1,8 +1,8 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   irq_pin: GPIO13
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ade7953_spi/test.esp8266-ard.yaml b/tests/components/ade7953_spi/test.esp8266-ard.yaml
index b90e661ec0..8475dc4c50 100644
--- a/tests/components/ade7953_spi/test.esp8266-ard.yaml
+++ b/tests/components/ade7953_spi/test.esp8266-ard.yaml
@@ -1,8 +1,11 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   irq_pin: GPIO5
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ade7953_spi/test.rp2040-ard.yaml b/tests/components/ade7953_spi/test.rp2040-ard.yaml
index 8f5941e1b2..7c4a74a236 100644
--- a/tests/components/ade7953_spi/test.rp2040-ard.yaml
+++ b/tests/components/ade7953_spi/test.rp2040-ard.yaml
@@ -5,4 +5,7 @@ substitutions:
   irq_pin: GPIO5
   cs_pin: GPIO6
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ads1115/common.yaml b/tests/components/ads1115/common.yaml
index 297877d2d8..4724dc5a14 100644
--- a/tests/components/ads1115/common.yaml
+++ b/tests/components/ads1115/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_ads1115
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 ads1115:
+  i2c_id: i2c_bus
   address: 0x48
 
 sensor:
diff --git a/tests/components/ads1115/test.esp32-ard.yaml b/tests/components/ads1115/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ads1115/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ads1115/test.esp32-c3-ard.yaml b/tests/components/ads1115/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ads1115/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ads1115/test.esp32-c3-idf.yaml b/tests/components/ads1115/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ads1115/test.esp32-c3-idf.yaml
+++ b/tests/components/ads1115/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ads1115/test.esp32-idf.yaml b/tests/components/ads1115/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ads1115/test.esp32-idf.yaml
+++ b/tests/components/ads1115/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ads1115/test.esp8266-ard.yaml b/tests/components/ads1115/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ads1115/test.esp8266-ard.yaml
+++ b/tests/components/ads1115/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ads1115/test.rp2040-ard.yaml b/tests/components/ads1115/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ads1115/test.rp2040-ard.yaml
+++ b/tests/components/ads1115/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ags10/common.yaml b/tests/components/ags10/common.yaml
index 0c4c3513cf..0551871e59 100644
--- a/tests/components/ags10/common.yaml
+++ b/tests/components/ags10/common.yaml
@@ -1,9 +1,3 @@
-i2c:
-  - id: i2c_ags10
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-    frequency: 10kHz
-
 sensor:
   - platform: ags10
     id: ags10_1
diff --git a/tests/components/ags10/test.esp32-ard.yaml b/tests/components/ags10/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ags10/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ags10/test.esp32-c3-ard.yaml b/tests/components/ags10/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ags10/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ags10/test.esp32-c3-idf.yaml b/tests/components/ags10/test.esp32-c3-idf.yaml
index ee2c29ca4e..72703301a1 100644
--- a/tests/components/ags10/test.esp32-c3-idf.yaml
+++ b/tests/components/ags10/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c_low_freq: !include ../../test_build_components/common/i2c_low_freq/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ags10/test.esp32-idf.yaml b/tests/components/ags10/test.esp32-idf.yaml
index 63c3bd6afd..7a5d01898a 100644
--- a/tests/components/ags10/test.esp32-idf.yaml
+++ b/tests/components/ags10/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c_low_freq: !include ../../test_build_components/common/i2c_low_freq/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ags10/test.esp8266-ard.yaml b/tests/components/ags10/test.esp8266-ard.yaml
index ee2c29ca4e..9e23bb3778 100644
--- a/tests/components/ags10/test.esp8266-ard.yaml
+++ b/tests/components/ags10/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c_low_freq: !include ../../test_build_components/common/i2c_low_freq/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/aht10/common.yaml b/tests/components/aht10/common.yaml
index 721af09bb4..d7c3f9364f 100644
--- a/tests/components/aht10/common.yaml
+++ b/tests/components/aht10/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_aht10
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: aht10
+    i2c_id: i2c_bus
     temperature:
       name: Temperature
     humidity:
diff --git a/tests/components/aht10/test.esp32-ard.yaml b/tests/components/aht10/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/aht10/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/aht10/test.esp32-c3-ard.yaml b/tests/components/aht10/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/aht10/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/aht10/test.esp32-c3-idf.yaml b/tests/components/aht10/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/aht10/test.esp32-c3-idf.yaml
+++ b/tests/components/aht10/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/aht10/test.esp32-idf.yaml b/tests/components/aht10/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/aht10/test.esp32-idf.yaml
+++ b/tests/components/aht10/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/aht10/test.esp8266-ard.yaml b/tests/components/aht10/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/aht10/test.esp8266-ard.yaml
+++ b/tests/components/aht10/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/aht10/test.rp2040-ard.yaml b/tests/components/aht10/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/aht10/test.rp2040-ard.yaml
+++ b/tests/components/aht10/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/aic3204/common.yaml b/tests/components/aic3204/common.yaml
index 6e939bd260..5f175faee3 100644
--- a/tests/components/aic3204/common.yaml
+++ b/tests/components/aic3204/common.yaml
@@ -6,10 +6,6 @@ esphome:
       - audio_dac.set_volume:
           volume: 50%
 
-i2c:
-  - id: i2c_aic3204
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 audio_dac:
   - platform: aic3204
+    i2c_id: i2c_bus
diff --git a/tests/components/aic3204/test.esp32-ard.yaml b/tests/components/aic3204/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/aic3204/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/aic3204/test.esp32-c3-ard.yaml b/tests/components/aic3204/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/aic3204/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/aic3204/test.esp32-c3-idf.yaml b/tests/components/aic3204/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/aic3204/test.esp32-c3-idf.yaml
+++ b/tests/components/aic3204/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/aic3204/test.esp32-idf.yaml b/tests/components/aic3204/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/aic3204/test.esp32-idf.yaml
+++ b/tests/components/aic3204/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/aic3204/test.esp8266-ard.yaml b/tests/components/aic3204/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/aic3204/test.esp8266-ard.yaml
+++ b/tests/components/aic3204/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/airthings_wave_mini/test.esp32-c3-idf.yaml b/tests/components/airthings_wave_mini/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/airthings_wave_mini/test.esp32-c3-idf.yaml
+++ b/tests/components/airthings_wave_mini/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/airthings_wave_mini/test.esp32-idf.yaml b/tests/components/airthings_wave_mini/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/airthings_wave_mini/test.esp32-idf.yaml
+++ b/tests/components/airthings_wave_mini/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/airthings_wave_plus/test.esp32-c3-idf.yaml b/tests/components/airthings_wave_plus/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/airthings_wave_plus/test.esp32-c3-idf.yaml
+++ b/tests/components/airthings_wave_plus/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/airthings_wave_plus/test.esp32-idf.yaml b/tests/components/airthings_wave_plus/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/airthings_wave_plus/test.esp32-idf.yaml
+++ b/tests/components/airthings_wave_plus/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/absolute_humidity/test.esp32-ard.yaml b/tests/components/alarm_control_panel/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/absolute_humidity/test.esp32-ard.yaml
rename to tests/components/alarm_control_panel/test.nrf52-adafruit.yaml
diff --git a/tests/components/absolute_humidity/test.esp32-c3-ard.yaml b/tests/components/alarm_control_panel/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/absolute_humidity/test.esp32-c3-ard.yaml
rename to tests/components/alarm_control_panel/test.nrf52-mcumgr.yaml
diff --git a/tests/components/alpha3/test.esp32-c3-idf.yaml b/tests/components/alpha3/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/alpha3/test.esp32-c3-idf.yaml
+++ b/tests/components/alpha3/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/alpha3/test.esp32-idf.yaml b/tests/components/alpha3/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/alpha3/test.esp32-idf.yaml
+++ b/tests/components/alpha3/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/am2315c/common.yaml b/tests/components/am2315c/common.yaml
index ab4656c17d..362fe19e4d 100644
--- a/tests/components/am2315c/common.yaml
+++ b/tests/components/am2315c/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_am2315c
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: am2315c
+    i2c_id: i2c_bus
     temperature:
       name: Temperature
     humidity:
diff --git a/tests/components/am2315c/test.esp32-ard.yaml b/tests/components/am2315c/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/am2315c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/am2315c/test.esp32-c3-ard.yaml b/tests/components/am2315c/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/am2315c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/am2315c/test.esp32-c3-idf.yaml b/tests/components/am2315c/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/am2315c/test.esp32-c3-idf.yaml
+++ b/tests/components/am2315c/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/am2315c/test.esp32-idf.yaml b/tests/components/am2315c/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/am2315c/test.esp32-idf.yaml
+++ b/tests/components/am2315c/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/am2315c/test.esp8266-ard.yaml b/tests/components/am2315c/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/am2315c/test.esp8266-ard.yaml
+++ b/tests/components/am2315c/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/am2315c/test.rp2040-ard.yaml b/tests/components/am2315c/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/am2315c/test.rp2040-ard.yaml
+++ b/tests/components/am2315c/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/am2320/common.yaml b/tests/components/am2320/common.yaml
index c0982b8818..d67ca3e564 100644
--- a/tests/components/am2320/common.yaml
+++ b/tests/components/am2320/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_am2320
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: am2320
+    i2c_id: i2c_bus
     temperature:
       name: Temperature
     humidity:
diff --git a/tests/components/am2320/test.esp32-ard.yaml b/tests/components/am2320/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/am2320/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/am2320/test.esp32-c3-ard.yaml b/tests/components/am2320/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/am2320/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/am2320/test.esp32-c3-idf.yaml b/tests/components/am2320/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/am2320/test.esp32-c3-idf.yaml
+++ b/tests/components/am2320/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/am2320/test.esp32-idf.yaml b/tests/components/am2320/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/am2320/test.esp32-idf.yaml
+++ b/tests/components/am2320/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/am2320/test.esp8266-ard.yaml b/tests/components/am2320/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/am2320/test.esp8266-ard.yaml
+++ b/tests/components/am2320/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/am2320/test.rp2040-ard.yaml b/tests/components/am2320/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/am2320/test.rp2040-ard.yaml
+++ b/tests/components/am2320/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/am43/test.esp32-c3-idf.yaml b/tests/components/am43/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/am43/test.esp32-c3-idf.yaml
+++ b/tests/components/am43/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/am43/test.esp32-idf.yaml b/tests/components/am43/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/am43/test.esp32-idf.yaml
+++ b/tests/components/am43/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/analog_threshold/common.yaml b/tests/components/analog_threshold/common.yaml
index 44d79756b5..26c401b92a 100644
--- a/tests/components/analog_threshold/common.yaml
+++ b/tests/components/analog_threshold/common.yaml
@@ -5,9 +5,8 @@ sensor:
     lambda: |-
       if (millis() > 10000) {
         return 42.0;
-      } else {
-        return 0.0;
       }
+      return 0.0;
     update_interval: 15s
 
 binary_sensor:
diff --git a/tests/components/airthings_wave_mini/test.esp32-ard.yaml b/tests/components/analog_threshold/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/airthings_wave_mini/test.esp32-ard.yaml
rename to tests/components/analog_threshold/test.nrf52-adafruit.yaml
diff --git a/tests/components/airthings_wave_mini/test.esp32-c3-ard.yaml b/tests/components/analog_threshold/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/airthings_wave_mini/test.esp32-c3-ard.yaml
rename to tests/components/analog_threshold/test.nrf52-mcumgr.yaml
diff --git a/tests/components/animation/test.esp32-ard.yaml b/tests/components/animation/test.esp32-ard.yaml
deleted file mode 100644
index 7d9fe45bff..0000000000
--- a/tests/components/animation/test.esp32-ard.yaml
+++ /dev/null
@@ -1,17 +0,0 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 16
-    mosi_pin: 17
-    miso_pin: 15
-
-display:
-  - platform: ili9xxx
-    id: main_lcd
-    model: ili9342
-    cs_pin: 12
-    dc_pin: 13
-    reset_pin: 21
-    invert_colors: false
-
-packages:
-  animation: !include common.yaml
diff --git a/tests/components/animation/test.esp32-c3-ard.yaml b/tests/components/animation/test.esp32-c3-ard.yaml
deleted file mode 100644
index 18aa2a5b06..0000000000
--- a/tests/components/animation/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,17 +0,0 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 6
-    mosi_pin: 7
-    miso_pin: 5
-
-display:
-  - platform: ili9xxx
-    id: main_lcd
-    model: ili9342
-    cs_pin: 8
-    dc_pin: 9
-    reset_pin: 10
-    invert_colors: false
-
-packages:
-  animation: !include common.yaml
diff --git a/tests/components/animation/test.esp32-c3-idf.yaml b/tests/components/animation/test.esp32-c3-idf.yaml
index 18aa2a5b06..a08a683333 100644
--- a/tests/components/animation/test.esp32-c3-idf.yaml
+++ b/tests/components/animation/test.esp32-c3-idf.yaml
@@ -1,17 +1,13 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 6
-    mosi_pin: 7
-    miso_pin: 5
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
+  animation: !include common.yaml
 
 display:
   - platform: ili9xxx
     id: main_lcd
+    spi_id: spi_bus
     model: ili9342
     cs_pin: 8
     dc_pin: 9
     reset_pin: 10
     invert_colors: false
-
-packages:
-  animation: !include common.yaml
diff --git a/tests/components/animation/test.esp32-idf.yaml b/tests/components/animation/test.esp32-idf.yaml
index 7d9fe45bff..c28e9584dd 100644
--- a/tests/components/animation/test.esp32-idf.yaml
+++ b/tests/components/animation/test.esp32-idf.yaml
@@ -1,17 +1,13 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 16
-    mosi_pin: 17
-    miso_pin: 15
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+  animation: !include common.yaml
 
 display:
   - platform: ili9xxx
     id: main_lcd
+    spi_id: spi_bus
     model: ili9342
     cs_pin: 12
     dc_pin: 13
     reset_pin: 21
     invert_colors: false
-
-packages:
-  animation: !include common.yaml
diff --git a/tests/components/animation/test.esp8266-ard.yaml b/tests/components/animation/test.esp8266-ard.yaml
index 9548c7fbeb..11a7117d91 100644
--- a/tests/components/animation/test.esp8266-ard.yaml
+++ b/tests/components/animation/test.esp8266-ard.yaml
@@ -1,17 +1,13 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 14
-    mosi_pin: 13
-    miso_pin: 12
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+  animation: !include common.yaml
 
 display:
   - platform: ili9xxx
     id: main_lcd
+    spi_id: spi_bus
     model: ili9342
     cs_pin: 5
     dc_pin: 15
     reset_pin: 16
     invert_colors: false
-
-packages:
-  animation: !include common.yaml
diff --git a/tests/components/animation/test.rp2040-ard.yaml b/tests/components/animation/test.rp2040-ard.yaml
index efb3f2907c..32fb4efb04 100644
--- a/tests/components/animation/test.rp2040-ard.yaml
+++ b/tests/components/animation/test.rp2040-ard.yaml
@@ -1,17 +1,13 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 2
-    mosi_pin: 3
-    miso_pin: 4
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+  animation: !include common.yaml
 
 display:
   - platform: ili9xxx
     id: main_lcd
+    spi_id: spi_bus
     model: ili9342
     cs_pin: 20
     dc_pin: 21
     reset_pin: 22
     invert_colors: false
-
-packages:
-  animation: !include common.yaml
diff --git a/tests/components/anova/test.esp32-c3-idf.yaml b/tests/components/anova/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/anova/test.esp32-c3-idf.yaml
+++ b/tests/components/anova/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/anova/test.esp32-idf.yaml b/tests/components/anova/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/anova/test.esp32-idf.yaml
+++ b/tests/components/anova/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/apds9306/common.yaml b/tests/components/apds9306/common.yaml
index b3828e62ff..dc34f47645 100644
--- a/tests/components/apds9306/common.yaml
+++ b/tests/components/apds9306/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_apds9306
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: apds9306
+    i2c_id: i2c_bus
     name: "APDS9306 Light Level"
     gain: 3
     bit_width: 16
diff --git a/tests/components/apds9306/test.esp32-ard.yaml b/tests/components/apds9306/test.esp32-ard.yaml
deleted file mode 100644
index 3b761d3fc1..0000000000
--- a/tests/components/apds9306/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
-
-<<: !include common.yaml
diff --git a/tests/components/apds9306/test.esp32-c3-ard.yaml b/tests/components/apds9306/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/apds9306/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/apds9306/test.esp32-c3-idf.yaml b/tests/components/apds9306/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/apds9306/test.esp32-c3-idf.yaml
+++ b/tests/components/apds9306/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/apds9306/test.esp32-idf.yaml b/tests/components/apds9306/test.esp32-idf.yaml
index 3b761d3fc1..b47e39c389 100644
--- a/tests/components/apds9306/test.esp32-idf.yaml
+++ b/tests/components/apds9306/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/apds9306/test.esp8266-ard.yaml b/tests/components/apds9306/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/apds9306/test.esp8266-ard.yaml
+++ b/tests/components/apds9306/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/apds9306/test.rp2040-ard.yaml b/tests/components/apds9306/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/apds9306/test.rp2040-ard.yaml
+++ b/tests/components/apds9306/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/apds9960/common.yaml b/tests/components/apds9960/common.yaml
index de7706648a..c14212d263 100644
--- a/tests/components/apds9960/common.yaml
+++ b/tests/components/apds9960/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_apds9960
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 apds9960:
+  i2c_id: i2c_bus
   address: 0x20
   update_interval: 60s
 
diff --git a/tests/components/apds9960/test.esp32-ard.yaml b/tests/components/apds9960/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/apds9960/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/apds9960/test.esp32-c3-ard.yaml b/tests/components/apds9960/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/apds9960/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/apds9960/test.esp32-c3-idf.yaml b/tests/components/apds9960/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/apds9960/test.esp32-c3-idf.yaml
+++ b/tests/components/apds9960/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/apds9960/test.esp32-idf.yaml b/tests/components/apds9960/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/apds9960/test.esp32-idf.yaml
+++ b/tests/components/apds9960/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/apds9960/test.esp8266-ard.yaml b/tests/components/apds9960/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/apds9960/test.esp8266-ard.yaml
+++ b/tests/components/apds9960/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/apds9960/test.rp2040-ard.yaml b/tests/components/apds9960/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/apds9960/test.rp2040-ard.yaml
+++ b/tests/components/apds9960/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/api/common-base.yaml b/tests/components/api/common-base.yaml
new file mode 100644
index 0000000000..6483d5a997
--- /dev/null
+++ b/tests/components/api/common-base.yaml
@@ -0,0 +1,89 @@
+esphome:
+  on_boot:
+    then:
+      - homeassistant.event:
+          event: esphome.button_pressed
+          data:
+            message: Button was pressed
+      - homeassistant.action:
+          action: notify.html5
+          data:
+            message: Button was pressed
+      - homeassistant.tag_scanned: pulse
+      - homeassistant.action:
+          action: weather.get_forecasts
+          data:
+            entity_id: weather.forecast_home
+            type: hourly
+          capture_response: true
+          on_success:
+            - lambda: |-
+                JsonObjectConst next_hour = response["response"]["weather.forecast_home"]["forecast"][0];
+                float next_temperature = next_hour["temperature"].as();
+                ESP_LOGD("main", "Next hour temperature: %f", next_temperature);
+          on_error:
+            - lambda: |-
+                ESP_LOGE("main", "Action failed with error: %s", error.c_str());
+      - homeassistant.action:
+          action: weather.get_forecasts
+          data:
+            entity_id: weather.forecast_home
+            type: hourly
+          capture_response: true
+          response_template: "{{ response['weather.forecast_home']['forecast'][0]['temperature'] }}"
+          on_success:
+            - lambda: |-
+                float temperature = response["response"].as();
+                ESP_LOGD("main", "Next hour temperature: %f", temperature);
+      - homeassistant.action:
+          action: light.toggle
+          data:
+            entity_id: light.demo_light
+          on_success:
+            - logger.log: "Toggled demo light"
+          on_error:
+            - logger.log: "Failed to toggle demo light"
+
+api:
+  port: 8000
+  reboot_timeout: 0min
+  actions:
+    - action: hello_world
+      variables:
+        name: string
+      then:
+        - logger.log:
+            format: Hello World %s!
+            args:
+              - name.c_str()
+    - action: empty_action
+      then:
+        - logger.log: Action Called
+    - action: all_types
+      variables:
+        bool_: bool
+        int_: int
+        float_: float
+        string_: string
+      then:
+        - logger.log: Something happened
+    - action: array_types
+      variables:
+        bool_arr: bool[]
+        int_arr: int[]
+        float_arr: float[]
+        string_arr: string[]
+      then:
+        - logger.log:
+            # yamllint disable rule:line-length
+            format: "Bool: %s (%u), Int: %ld (%u), Float: %f (%u), String: %s (%u)"
+            # yamllint enable rule:line-length
+            args:
+              - YESNO(bool_arr[0])
+              - bool_arr.size()
+              - (long) int_arr[0]
+              - int_arr.size()
+              - float_arr[0]
+              - float_arr.size()
+              - string_arr[0].c_str()
+              - string_arr.size()
diff --git a/tests/components/api/common.yaml b/tests/components/api/common.yaml
index 7ac11e4da6..6115838b6d 100644
--- a/tests/components/api/common.yaml
+++ b/tests/components/api/common.yaml
@@ -1,59 +1,5 @@
-esphome:
-  on_boot:
-    then:
-      - homeassistant.event:
-          event: esphome.button_pressed
-          data:
-            message: Button was pressed
-      - homeassistant.action:
-          action: notify.html5
-          data:
-            message: Button was pressed
-      - homeassistant.tag_scanned: pulse
+<<: !include common-base.yaml
 
 api:
-  port: 8000
-  password: pwd
-  reboot_timeout: 0min
   encryption:
     key: bOFFzzvfpg5DB94DuBGLXD/hMnhpDKgP9UQyBulwWVU=
-  actions:
-    - action: hello_world
-      variables:
-        name: string
-      then:
-        - logger.log:
-            format: Hello World %s!
-            args:
-              - name.c_str()
-    - action: empty_action
-      then:
-        - logger.log: Action Called
-    - action: all_types
-      variables:
-        bool_: bool
-        int_: int
-        float_: float
-        string_: string
-      then:
-        - logger.log: Something happened
-    - action: array_types
-      variables:
-        bool_arr: bool[]
-        int_arr: int[]
-        float_arr: float[]
-        string_arr: string[]
-      then:
-        - logger.log:
-            # yamllint disable rule:line-length
-            format: "Bool: %s (%u), Int: %ld (%u), Float: %f (%u), String: %s (%u)"
-            # yamllint enable rule:line-length
-            args:
-              - YESNO(bool_arr[0])
-              - bool_arr.size()
-              - (long) int_arr[0]
-              - int_arr.size()
-              - float_arr[0]
-              - float_arr.size()
-              - string_arr[0].c_str()
-              - string_arr.size()
diff --git a/tests/components/api/test-dynamic-encryption.esp32-idf.yaml b/tests/components/api/test-dynamic-encryption.esp32-idf.yaml
index d8f8c247f4..504871716b 100644
--- a/tests/components/api/test-dynamic-encryption.esp32-idf.yaml
+++ b/tests/components/api/test-dynamic-encryption.esp32-idf.yaml
@@ -1,10 +1,5 @@
-packages:
-  common: !include common.yaml
+<<: !include common-base.yaml
 
 wifi:
   ssid: MySSID
   password: password1
-
-api:
-  encryption:
-    key: !remove
diff --git a/tests/components/api/test.esp32-ard.yaml b/tests/components/api/test.esp32-ard.yaml
deleted file mode 100644
index 46c01d926f..0000000000
--- a/tests/components/api/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-<<: !include common.yaml
-
-wifi:
-  ssid: MySSID
-  password: password1
diff --git a/tests/components/api/test.esp32-c3-ard.yaml b/tests/components/api/test.esp32-c3-ard.yaml
deleted file mode 100644
index 46c01d926f..0000000000
--- a/tests/components/api/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-<<: !include common.yaml
-
-wifi:
-  ssid: MySSID
-  password: password1
diff --git a/tests/components/as3935_i2c/common.yaml b/tests/components/as3935_i2c/common.yaml
index d76cc37fc1..d659486e83 100644
--- a/tests/components/as3935_i2c/common.yaml
+++ b/tests/components/as3935_i2c/common.yaml
@@ -1,17 +1,16 @@
-i2c:
-  - id: i2c_as3935
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 as3935_i2c:
+  id: as3935_i2c_id
+  i2c_id: i2c_bus
   irq_pin: ${irq_pin}
 
 binary_sensor:
   - platform: as3935
+    as3935_id: as3935_i2c_id
     name: Storm Alert
 
 sensor:
   - platform: as3935
+    as3935_id: as3935_i2c_id
     lightning_energy:
       name: Lightning Energy
     distance:
diff --git a/tests/components/as3935_i2c/test.esp32-ard.yaml b/tests/components/as3935_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 52d5a045cb..0000000000
--- a/tests/components/as3935_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,11 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  irq_pin: GPIO15
-
-packages:
-  as3935: !include common.yaml
-
-# Trigger issue: https://github.com/esphome/issues/issues/6990
-# Compile with no binary sensor results in error
-binary_sensor: !remove
diff --git a/tests/components/as3935_i2c/test.esp32-c3-ard.yaml b/tests/components/as3935_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index 799acabd5a..0000000000
--- a/tests/components/as3935_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  irq_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/as3935_i2c/test.esp32-c3-idf.yaml b/tests/components/as3935_i2c/test.esp32-c3-idf.yaml
index 799acabd5a..59296a1e6e 100644
--- a/tests/components/as3935_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/as3935_i2c/test.esp32-c3-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/as3935_i2c/test.esp32-idf.yaml b/tests/components/as3935_i2c/test.esp32-idf.yaml
index 2c57d412f6..49629536e7 100644
--- a/tests/components/as3935_i2c/test.esp32-idf.yaml
+++ b/tests/components/as3935_i2c/test.esp32-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   irq_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/as3935_i2c/test.esp8266-ard.yaml b/tests/components/as3935_i2c/test.esp8266-ard.yaml
index c8e6a43f44..dc7609ab37 100644
--- a/tests/components/as3935_i2c/test.esp8266-ard.yaml
+++ b/tests/components/as3935_i2c/test.esp8266-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/as3935_i2c/test.rp2040-ard.yaml b/tests/components/as3935_i2c/test.rp2040-ard.yaml
index 799acabd5a..b80562ad22 100644
--- a/tests/components/as3935_i2c/test.rp2040-ard.yaml
+++ b/tests/components/as3935_i2c/test.rp2040-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/as3935_spi/common.yaml b/tests/components/as3935_spi/common.yaml
index c3fb93dff1..d2942dc01d 100644
--- a/tests/components/as3935_spi/common.yaml
+++ b/tests/components/as3935_spi/common.yaml
@@ -1,19 +1,16 @@
-spi:
-  - id: spi_as3935
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 as3935_spi:
+  id: as3935_spi_id
   cs_pin: ${cs_pin}
   irq_pin: ${irq_pin}
 
 binary_sensor:
   - platform: as3935
+    as3935_id: as3935_spi_id
     name: Storm Alert
 
 sensor:
   - platform: as3935
+    as3935_id: as3935_spi_id
     lightning_energy:
       name: Lightning Energy
     distance:
diff --git a/tests/components/as3935_spi/test.esp32-ard.yaml b/tests/components/as3935_spi/test.esp32-ard.yaml
deleted file mode 100644
index e00f522dd4..0000000000
--- a/tests/components/as3935_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  irq_pin: GPIO13
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/as3935_spi/test.esp32-c3-ard.yaml b/tests/components/as3935_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index fcf35f528e..0000000000
--- a/tests/components/as3935_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  irq_pin: GPIO9
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/as3935_spi/test.esp32-c3-idf.yaml b/tests/components/as3935_spi/test.esp32-c3-idf.yaml
index fcf35f528e..5e7e2dc82c 100644
--- a/tests/components/as3935_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/as3935_spi/test.esp32-c3-idf.yaml
@@ -1,8 +1,7 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   irq_pin: GPIO9
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/as3935_spi/test.esp32-idf.yaml b/tests/components/as3935_spi/test.esp32-idf.yaml
index e00f522dd4..19791e24b7 100644
--- a/tests/components/as3935_spi/test.esp32-idf.yaml
+++ b/tests/components/as3935_spi/test.esp32-idf.yaml
@@ -1,8 +1,8 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   irq_pin: GPIO13
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/as3935_spi/test.esp8266-ard.yaml b/tests/components/as3935_spi/test.esp8266-ard.yaml
index b90e661ec0..8475dc4c50 100644
--- a/tests/components/as3935_spi/test.esp8266-ard.yaml
+++ b/tests/components/as3935_spi/test.esp8266-ard.yaml
@@ -1,8 +1,11 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   irq_pin: GPIO5
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/as3935_spi/test.rp2040-ard.yaml b/tests/components/as3935_spi/test.rp2040-ard.yaml
index 8f5941e1b2..7c4a74a236 100644
--- a/tests/components/as3935_spi/test.rp2040-ard.yaml
+++ b/tests/components/as3935_spi/test.rp2040-ard.yaml
@@ -5,4 +5,7 @@ substitutions:
   irq_pin: GPIO5
   cs_pin: GPIO6
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/as5600/common.yaml b/tests/components/as5600/common.yaml
index 860f5bf803..d867c66a21 100644
--- a/tests/components/as5600/common.yaml
+++ b/tests/components/as5600/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_as5600
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 as5600:
+  i2c_id: i2c_bus
   dir_pin: ${dir_pin}
   direction: clockwise
   start_position: 90deg
diff --git a/tests/components/as5600/test.esp32-ard.yaml b/tests/components/as5600/test.esp32-ard.yaml
deleted file mode 100644
index fa08763501..0000000000
--- a/tests/components/as5600/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  dir_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/as5600/test.esp32-c3-ard.yaml b/tests/components/as5600/test.esp32-c3-ard.yaml
deleted file mode 100644
index a0623c91e5..0000000000
--- a/tests/components/as5600/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  dir_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/as5600/test.esp32-c3-idf.yaml b/tests/components/as5600/test.esp32-c3-idf.yaml
index a0623c91e5..03a87ed6c4 100644
--- a/tests/components/as5600/test.esp32-c3-idf.yaml
+++ b/tests/components/as5600/test.esp32-c3-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   dir_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/as5600/test.esp32-idf.yaml b/tests/components/as5600/test.esp32-idf.yaml
index fa08763501..9d25a7f09a 100644
--- a/tests/components/as5600/test.esp32-idf.yaml
+++ b/tests/components/as5600/test.esp32-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   dir_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/as5600/test.esp8266-ard.yaml b/tests/components/as5600/test.esp8266-ard.yaml
index 5e27f8c134..8d18740b95 100644
--- a/tests/components/as5600/test.esp8266-ard.yaml
+++ b/tests/components/as5600/test.esp8266-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   dir_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/as5600/test.rp2040-ard.yaml b/tests/components/as5600/test.rp2040-ard.yaml
index a0623c91e5..4bcbb99c81 100644
--- a/tests/components/as5600/test.rp2040-ard.yaml
+++ b/tests/components/as5600/test.rp2040-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   dir_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/as7341/common.yaml b/tests/components/as7341/common.yaml
index 0351b344c6..3f94656c74 100644
--- a/tests/components/as7341/common.yaml
+++ b/tests/components/as7341/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_as7341
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: as7341
+    i2c_id: i2c_bus
     update_interval: 15s
     gain: X8
     atime: 120
diff --git a/tests/components/as7341/test.esp32-ard.yaml b/tests/components/as7341/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/as7341/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/as7341/test.esp32-c3-ard.yaml b/tests/components/as7341/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/as7341/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/as7341/test.esp32-c3-idf.yaml b/tests/components/as7341/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/as7341/test.esp32-c3-idf.yaml
+++ b/tests/components/as7341/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/as7341/test.esp32-idf.yaml b/tests/components/as7341/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/as7341/test.esp32-idf.yaml
+++ b/tests/components/as7341/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/as7341/test.esp8266-ard.yaml b/tests/components/as7341/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/as7341/test.esp8266-ard.yaml
+++ b/tests/components/as7341/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/as7341/test.rp2040-ard.yaml b/tests/components/as7341/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/as7341/test.rp2040-ard.yaml
+++ b/tests/components/as7341/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/at581x/common.yaml b/tests/components/at581x/common.yaml
index 018a0fded1..425be47c42 100644
--- a/tests/components/at581x/common.yaml
+++ b/tests/components/at581x/common.yaml
@@ -16,13 +16,9 @@ esphome:
           id: waveradar
 
 at581x:
+  i2c_id: i2c_bus
   id: waveradar
 
-i2c:
-  - id: i2c_at581x
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 switch:
   - platform: at581x
     name: Enable Radar
diff --git a/tests/components/at581x/test.esp32-ard.yaml b/tests/components/at581x/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/at581x/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/at581x/test.esp32-c3-ard.yaml b/tests/components/at581x/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/at581x/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/at581x/test.esp32-c3-idf.yaml b/tests/components/at581x/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/at581x/test.esp32-c3-idf.yaml
+++ b/tests/components/at581x/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/at581x/test.esp32-idf.yaml b/tests/components/at581x/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/at581x/test.esp32-idf.yaml
+++ b/tests/components/at581x/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/at581x/test.esp8266-ard.yaml b/tests/components/at581x/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/at581x/test.esp8266-ard.yaml
+++ b/tests/components/at581x/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/at581x/test.rp2040-ard.yaml b/tests/components/at581x/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/at581x/test.rp2040-ard.yaml
+++ b/tests/components/at581x/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/atc_mithermometer/test.esp32-c3-idf.yaml b/tests/components/atc_mithermometer/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/atc_mithermometer/test.esp32-c3-idf.yaml
+++ b/tests/components/atc_mithermometer/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/atc_mithermometer/test.esp32-idf.yaml b/tests/components/atc_mithermometer/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/atc_mithermometer/test.esp32-idf.yaml
+++ b/tests/components/atc_mithermometer/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/atm90e26/common.yaml b/tests/components/atm90e26/common.yaml
index 49c3a73ec8..478be7b8b3 100644
--- a/tests/components/atm90e26/common.yaml
+++ b/tests/components/atm90e26/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_atm90e26
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: atm90e26
     cs_pin: ${cs_pin}
diff --git a/tests/components/atm90e26/test.esp32-ard.yaml b/tests/components/atm90e26/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/atm90e26/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/atm90e26/test.esp32-c3-ard.yaml b/tests/components/atm90e26/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/atm90e26/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/atm90e26/test.esp32-c3-idf.yaml b/tests/components/atm90e26/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/atm90e26/test.esp32-c3-idf.yaml
+++ b/tests/components/atm90e26/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/atm90e26/test.esp32-idf.yaml b/tests/components/atm90e26/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/atm90e26/test.esp32-idf.yaml
+++ b/tests/components/atm90e26/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/atm90e26/test.esp8266-ard.yaml b/tests/components/atm90e26/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/atm90e26/test.esp8266-ard.yaml
+++ b/tests/components/atm90e26/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/atm90e26/test.rp2040-ard.yaml b/tests/components/atm90e26/test.rp2040-ard.yaml
index c8bfab0023..5d0c35c2d2 100644
--- a/tests/components/atm90e26/test.rp2040-ard.yaml
+++ b/tests/components/atm90e26/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO6
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/atm90e32/common.yaml b/tests/components/atm90e32/common.yaml
index 3eeed8395f..b8b480ab62 100644
--- a/tests/components/atm90e32/common.yaml
+++ b/tests/components/atm90e32/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_atm90e32
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: atm90e32
     cs_pin: ${cs_pin}
diff --git a/tests/components/atm90e32/test.esp32-ard.yaml b/tests/components/atm90e32/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/atm90e32/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/atm90e32/test.esp32-c3-ard.yaml b/tests/components/atm90e32/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/atm90e32/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/atm90e32/test.esp32-c3-idf.yaml b/tests/components/atm90e32/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/atm90e32/test.esp32-c3-idf.yaml
+++ b/tests/components/atm90e32/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/atm90e32/test.esp32-idf.yaml b/tests/components/atm90e32/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/atm90e32/test.esp32-idf.yaml
+++ b/tests/components/atm90e32/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/atm90e32/test.esp8266-ard.yaml b/tests/components/atm90e32/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/atm90e32/test.esp8266-ard.yaml
+++ b/tests/components/atm90e32/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/atm90e32/test.rp2040-ard.yaml b/tests/components/atm90e32/test.rp2040-ard.yaml
index c8bfab0023..5d0c35c2d2 100644
--- a/tests/components/atm90e32/test.rp2040-ard.yaml
+++ b/tests/components/atm90e32/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO6
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/axs15231/common.yaml b/tests/components/axs15231/common.yaml
index 1c0c79975f..d4fd3becbb 100644
--- a/tests/components/axs15231/common.yaml
+++ b/tests/components/axs15231/common.yaml
@@ -1,20 +1,18 @@
-i2c:
-  - id: i2c_axs15231
-    scl: 3
-    sda: 21
-
 display:
   - platform: ssd1306_i2c
-    id: ssd1306_display
+    i2c_id: i2c_bus
+    id: ssd1306_i2c_display
     model: SSD1306_128X64
     reset_pin: 19
     pages:
-      - id: page1
+      - id: axs15231_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
 
 touchscreen:
   - platform: axs15231
-    display: ssd1306_display
+    i2c_id: i2c_bus
+    id: axs15231_touchscreen
+    display: ssd1306_i2c_display
     interrupt_pin: 20
     reset_pin: 18
diff --git a/tests/components/axs15231/test.esp32-c3-idf.yaml b/tests/components/axs15231/test.esp32-c3-idf.yaml
index dade44d145..9990d96d29 100644
--- a/tests/components/axs15231/test.esp32-c3-idf.yaml
+++ b/tests/components/axs15231/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/axs15231/test.esp32-idf.yaml b/tests/components/axs15231/test.esp32-idf.yaml
index dade44d145..b47e39c389 100644
--- a/tests/components/axs15231/test.esp32-idf.yaml
+++ b/tests/components/axs15231/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/axs15231/test.esp8266-ard.yaml b/tests/components/axs15231/test.esp8266-ard.yaml
index c09d139574..eb599da773 100644
--- a/tests/components/axs15231/test.esp8266-ard.yaml
+++ b/tests/components/axs15231/test.esp8266-ard.yaml
@@ -1,10 +1,9 @@
-i2c:
-  - id: i2c_axs15231
-    scl: 5
-    sda: 4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 display:
   - platform: ssd1306_i2c
+    i2c_id: i2c_bus
     id: ssd1306_display
     model: SSD1306_128X64
     reset_pin: 13
@@ -15,5 +14,6 @@ display:
 
 touchscreen:
   - platform: axs15231
+    i2c_id: i2c_bus
     display: ssd1306_display
     interrupt_pin: 12
diff --git a/tests/components/axs15231/test.rp2040-ard.yaml b/tests/components/axs15231/test.rp2040-ard.yaml
index dade44d145..319a7c71a6 100644
--- a/tests/components/axs15231/test.rp2040-ard.yaml
+++ b/tests/components/axs15231/test.rp2040-ard.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/b_parasite/test.esp32-c3-idf.yaml b/tests/components/b_parasite/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/b_parasite/test.esp32-c3-idf.yaml
+++ b/tests/components/b_parasite/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/b_parasite/test.esp32-idf.yaml b/tests/components/b_parasite/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/b_parasite/test.esp32-idf.yaml
+++ b/tests/components/b_parasite/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ballu/common.yaml b/tests/components/ballu/common.yaml
index 52f86aa26a..178c39b8ce 100644
--- a/tests/components/ballu/common.yaml
+++ b/tests/components/ballu/common.yaml
@@ -1,7 +1,3 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: heatpumpir
     protocol: ballu
@@ -10,3 +6,4 @@ climate:
     name: HeatpumpIR Climate
     min_temperature: 18
     max_temperature: 30
+    transmitter_id: xmitr
diff --git a/tests/components/ballu/test.esp32-ard.yaml b/tests/components/ballu/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/ballu/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/ballu/test.esp8266-ard.yaml b/tests/components/ballu/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/ballu/test.esp8266-ard.yaml
+++ b/tests/components/ballu/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/airthings_wave_plus/test.esp32-ard.yaml b/tests/components/bang_bang/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/airthings_wave_plus/test.esp32-ard.yaml
rename to tests/components/bang_bang/test.nrf52-adafruit.yaml
diff --git a/tests/components/airthings_wave_plus/test.esp32-c3-ard.yaml b/tests/components/bang_bang/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/airthings_wave_plus/test.esp32-c3-ard.yaml
rename to tests/components/bang_bang/test.nrf52-mcumgr.yaml
diff --git a/tests/components/bedjet/test.esp32-c3-idf.yaml b/tests/components/bedjet/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/bedjet/test.esp32-c3-idf.yaml
+++ b/tests/components/bedjet/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bedjet/test.esp32-idf.yaml b/tests/components/bedjet/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/bedjet/test.esp32-idf.yaml
+++ b/tests/components/bedjet/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bh1750/common.yaml b/tests/components/bh1750/common.yaml
index c0e0bc1c59..46ea99b7e3 100644
--- a/tests/components/bh1750/common.yaml
+++ b/tests/components/bh1750/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_bh1750
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: bh1750
+    i2c_id: i2c_bus
     name: Living Room Brightness
     address: 0x23
     update_interval: 30s
diff --git a/tests/components/bh1750/test.esp32-ard.yaml b/tests/components/bh1750/test.esp32-ard.yaml
deleted file mode 100644
index 3b761d3fc1..0000000000
--- a/tests/components/bh1750/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
-
-<<: !include common.yaml
diff --git a/tests/components/bh1750/test.esp32-c3-ard.yaml b/tests/components/bh1750/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/bh1750/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/bh1750/test.esp32-c3-idf.yaml b/tests/components/bh1750/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/bh1750/test.esp32-c3-idf.yaml
+++ b/tests/components/bh1750/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bh1750/test.esp32-idf.yaml b/tests/components/bh1750/test.esp32-idf.yaml
index 3b761d3fc1..b47e39c389 100644
--- a/tests/components/bh1750/test.esp32-idf.yaml
+++ b/tests/components/bh1750/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bh1750/test.esp8266-ard.yaml b/tests/components/bh1750/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/bh1750/test.esp8266-ard.yaml
+++ b/tests/components/bh1750/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bh1750/test.rp2040-ard.yaml b/tests/components/bh1750/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/bh1750/test.rp2040-ard.yaml
+++ b/tests/components/bh1750/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bh1900nux/common.yaml b/tests/components/bh1900nux/common.yaml
new file mode 100644
index 0000000000..3438418702
--- /dev/null
+++ b/tests/components/bh1900nux/common.yaml
@@ -0,0 +1,6 @@
+sensor:
+  - platform: bh1900nux
+    i2c_id: i2c_bus
+    name: Temperature Living Room
+    address: 0x48
+    update_interval: 30s
diff --git a/tests/components/bh1900nux/test.esp32-c3-idf.yaml b/tests/components/bh1900nux/test.esp32-c3-idf.yaml
new file mode 100644
index 0000000000..9990d96d29
--- /dev/null
+++ b/tests/components/bh1900nux/test.esp32-c3-idf.yaml
@@ -0,0 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/bh1900nux/test.esp32-idf.yaml b/tests/components/bh1900nux/test.esp32-idf.yaml
new file mode 100644
index 0000000000..b47e39c389
--- /dev/null
+++ b/tests/components/bh1900nux/test.esp32-idf.yaml
@@ -0,0 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/bh1900nux/test.esp8266-ard.yaml b/tests/components/bh1900nux/test.esp8266-ard.yaml
new file mode 100644
index 0000000000..4a98b9388a
--- /dev/null
+++ b/tests/components/bh1900nux/test.esp8266-ard.yaml
@@ -0,0 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/bh1900nux/test.rp2040-ard.yaml b/tests/components/bh1900nux/test.rp2040-ard.yaml
new file mode 100644
index 0000000000..319a7c71a6
--- /dev/null
+++ b/tests/components/bh1900nux/test.rp2040-ard.yaml
@@ -0,0 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/binary_sensor/common.yaml b/tests/components/binary_sensor/common.yaml
index 2b4a006352..ed6322768f 100644
--- a/tests/components/binary_sensor/common.yaml
+++ b/tests/components/binary_sensor/common.yaml
@@ -23,9 +23,8 @@ binary_sensor:
       - lambda: |-
           if (id(some_binary_sensor).state) {
             return x;
-          } else {
-            return {};
           }
+          return {};
       - settle: 100ms
       - timeout: 10s
 
diff --git a/tests/components/binary_sensor/test.esp32-ard.yaml b/tests/components/binary_sensor/test.esp32-ard.yaml
deleted file mode 100644
index 25cb37a0b4..0000000000
--- a/tests/components/binary_sensor/test.esp32-ard.yaml
+++ /dev/null
@@ -1,2 +0,0 @@
-packages:
-  common: !include common.yaml
diff --git a/tests/components/binary_sensor/test.esp32-c3-ard.yaml b/tests/components/binary_sensor/test.esp32-c3-ard.yaml
deleted file mode 100644
index 25cb37a0b4..0000000000
--- a/tests/components/binary_sensor/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,2 +0,0 @@
-packages:
-  common: !include common.yaml
diff --git a/tests/components/alarm_control_panel/test.esp32-ard.yaml b/tests/components/binary_sensor/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/alarm_control_panel/test.esp32-ard.yaml
rename to tests/components/binary_sensor/test.nrf52-adafruit.yaml
diff --git a/tests/components/alarm_control_panel/test.esp32-c3-ard.yaml b/tests/components/binary_sensor/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/alarm_control_panel/test.esp32-c3-ard.yaml
rename to tests/components/binary_sensor/test.nrf52-mcumgr.yaml
diff --git a/tests/components/binary_sensor_map/common.yaml b/tests/components/binary_sensor_map/common.yaml
index 2fed5ae515..c054022583 100644
--- a/tests/components/binary_sensor_map/common.yaml
+++ b/tests/components/binary_sensor_map/common.yaml
@@ -4,25 +4,22 @@ binary_sensor:
     lambda: |-
       if (millis() > 10000) {
         return true;
-      } else {
-        return false;
       }
+      return false;
   - platform: template
     id: bin2
     lambda: |-
       if (millis() > 20000) {
         return true;
-      } else {
-        return false;
       }
+      return false;
   - platform: template
     id: bin3
     lambda: |-
       if (millis() > 30000) {
         return true;
-      } else {
-        return false;
       }
+      return false;
 
 sensor:
   - platform: binary_sensor_map
diff --git a/tests/components/bl0906/common.yaml b/tests/components/bl0906/common.yaml
index 29b82a5958..006aa682f1 100644
--- a/tests/components/bl0906/common.yaml
+++ b/tests/components/bl0906/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_bl0906
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 19200
-
 sensor:
   - platform: bl0906
     id: bl
diff --git a/tests/components/bl0906/test.esp32-ard.yaml b/tests/components/bl0906/test.esp32-ard.yaml
deleted file mode 100644
index 811f6b72a6..0000000000
--- a/tests/components/bl0906/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/bl0906/test.esp32-c3-ard.yaml b/tests/components/bl0906/test.esp32-c3-ard.yaml
deleted file mode 100644
index c79d14c740..0000000000
--- a/tests/components/bl0906/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/bl0906/test.esp32-c3-idf.yaml b/tests/components/bl0906/test.esp32-c3-idf.yaml
index c79d14c740..147d967dd4 100644
--- a/tests/components/bl0906/test.esp32-c3-idf.yaml
+++ b/tests/components/bl0906/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
+packages:
+  uart_19200: !include ../../test_build_components/common/uart_19200/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bl0906/test.esp32-idf.yaml b/tests/components/bl0906/test.esp32-idf.yaml
index 811f6b72a6..76222997a8 100644
--- a/tests/components/bl0906/test.esp32-idf.yaml
+++ b/tests/components/bl0906/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
+packages:
+  uart_19200: !include ../../test_build_components/common/uart_19200/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bl0906/test.esp8266-ard.yaml b/tests/components/bl0906/test.esp8266-ard.yaml
index 3b44f9c9c3..ac781ea834 100644
--- a/tests/components/bl0906/test.esp8266-ard.yaml
+++ b/tests/components/bl0906/test.esp8266-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO1
   rx_pin: GPIO3
 
+packages:
+  uart_19200: !include ../../test_build_components/common/uart_19200/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bl0906/test.rp2040-ard.yaml b/tests/components/bl0906/test.rp2040-ard.yaml
index b516342f3b..f4dada6605 100644
--- a/tests/components/bl0906/test.rp2040-ard.yaml
+++ b/tests/components/bl0906/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart_19200: !include ../../test_build_components/common/uart_19200/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bl0939/common.yaml b/tests/components/bl0939/common.yaml
index 7a6b635b70..a47aa05606 100644
--- a/tests/components/bl0939/common.yaml
+++ b/tests/components/bl0939/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_bl0939
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: bl0939
     voltage:
diff --git a/tests/components/bl0939/test.esp32-ard.yaml b/tests/components/bl0939/test.esp32-ard.yaml
deleted file mode 100644
index 811f6b72a6..0000000000
--- a/tests/components/bl0939/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/bl0939/test.esp32-c3-ard.yaml b/tests/components/bl0939/test.esp32-c3-ard.yaml
deleted file mode 100644
index c79d14c740..0000000000
--- a/tests/components/bl0939/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/bl0939/test.esp32-c3-idf.yaml b/tests/components/bl0939/test.esp32-c3-idf.yaml
index c79d14c740..4b7c8351a7 100644
--- a/tests/components/bl0939/test.esp32-c3-idf.yaml
+++ b/tests/components/bl0939/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bl0939/test.esp32-idf.yaml b/tests/components/bl0939/test.esp32-idf.yaml
index 811f6b72a6..64baa4ec9d 100644
--- a/tests/components/bl0939/test.esp32-idf.yaml
+++ b/tests/components/bl0939/test.esp32-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO12
   rx_pin: GPIO14
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bl0939/test.esp8266-ard.yaml b/tests/components/bl0939/test.esp8266-ard.yaml
index 3b44f9c9c3..89ca3ab5ae 100644
--- a/tests/components/bl0939/test.esp8266-ard.yaml
+++ b/tests/components/bl0939/test.esp8266-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO1
   rx_pin: GPIO3
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bl0939/test.rp2040-ard.yaml b/tests/components/bl0939/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/bl0939/test.rp2040-ard.yaml
+++ b/tests/components/bl0939/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bl0940/common.yaml b/tests/components/bl0940/common.yaml
index 97a997d2b4..0b73fd6d55 100644
--- a/tests/components/bl0940/common.yaml
+++ b/tests/components/bl0940/common.yaml
@@ -1,11 +1,11 @@
-uart:
-  - id: uart_bl0939
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
+button:
+  - platform: bl0940
+    bl0940_id: test_id
+    name: Cal Reset
 
 sensor:
   - platform: bl0940
+    id: test_id
     voltage:
       name: BL0940 Voltage
     current:
@@ -18,3 +18,18 @@ sensor:
       name: BL0940 Internal temperature
     external_temperature:
       name: BL0940 External temperature
+
+number:
+  - platform: bl0940
+    id: bl0940_number_id
+    bl0940_id: test_id
+    current_calibration:
+      name: Cal Current
+      min_value: -5
+      max_value: 5
+    voltage_calibration:
+      name: Cal Voltage
+      step: 0.01
+    power_calibration:
+      name: Cal Power
+      disabled_by_default: true
diff --git a/tests/components/bl0940/test.esp32-ard.yaml b/tests/components/bl0940/test.esp32-ard.yaml
deleted file mode 100644
index 811f6b72a6..0000000000
--- a/tests/components/bl0940/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/bl0940/test.esp32-c3-ard.yaml b/tests/components/bl0940/test.esp32-c3-ard.yaml
deleted file mode 100644
index c79d14c740..0000000000
--- a/tests/components/bl0940/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/bl0940/test.esp32-c3-idf.yaml b/tests/components/bl0940/test.esp32-c3-idf.yaml
index c79d14c740..4b7c8351a7 100644
--- a/tests/components/bl0940/test.esp32-c3-idf.yaml
+++ b/tests/components/bl0940/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bl0940/test.esp32-idf.yaml b/tests/components/bl0940/test.esp32-idf.yaml
index 811f6b72a6..64baa4ec9d 100644
--- a/tests/components/bl0940/test.esp32-idf.yaml
+++ b/tests/components/bl0940/test.esp32-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO12
   rx_pin: GPIO14
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bl0940/test.esp8266-ard.yaml b/tests/components/bl0940/test.esp8266-ard.yaml
index 3b44f9c9c3..89ca3ab5ae 100644
--- a/tests/components/bl0940/test.esp8266-ard.yaml
+++ b/tests/components/bl0940/test.esp8266-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO1
   rx_pin: GPIO3
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bl0940/test.rp2040-ard.yaml b/tests/components/bl0940/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/bl0940/test.rp2040-ard.yaml
+++ b/tests/components/bl0940/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bl0942/common.yaml b/tests/components/bl0942/common.yaml
index 32da24885f..1aaab8bb86 100644
--- a/tests/components/bl0942/common.yaml
+++ b/tests/components/bl0942/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_bl0939
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: bl0942
     reset: true
diff --git a/tests/components/bl0942/test.bk72xx-ard.yaml b/tests/components/bl0942/test.bk72xx-ard.yaml
index 96e13c83a9..0caf71ba1f 100644
--- a/tests/components/bl0942/test.bk72xx-ard.yaml
+++ b/tests/components/bl0942/test.bk72xx-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: TX1
-  rx_pin: RX1
+packages:
+  uart: !include ../../test_build_components/common/uart/bk72xx-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bl0942/test.esp32-ard.yaml b/tests/components/bl0942/test.esp32-ard.yaml
deleted file mode 100644
index 811f6b72a6..0000000000
--- a/tests/components/bl0942/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/bl0942/test.esp32-c3-ard.yaml b/tests/components/bl0942/test.esp32-c3-ard.yaml
deleted file mode 100644
index c79d14c740..0000000000
--- a/tests/components/bl0942/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/bl0942/test.esp32-c3-idf.yaml b/tests/components/bl0942/test.esp32-c3-idf.yaml
index c79d14c740..4b7c8351a7 100644
--- a/tests/components/bl0942/test.esp32-c3-idf.yaml
+++ b/tests/components/bl0942/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bl0942/test.esp32-idf.yaml b/tests/components/bl0942/test.esp32-idf.yaml
index 811f6b72a6..64baa4ec9d 100644
--- a/tests/components/bl0942/test.esp32-idf.yaml
+++ b/tests/components/bl0942/test.esp32-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO12
   rx_pin: GPIO14
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bl0942/test.esp8266-ard.yaml b/tests/components/bl0942/test.esp8266-ard.yaml
index 3b44f9c9c3..89ca3ab5ae 100644
--- a/tests/components/bl0942/test.esp8266-ard.yaml
+++ b/tests/components/bl0942/test.esp8266-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO1
   rx_pin: GPIO3
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bl0942/test.rp2040-ard.yaml b/tests/components/bl0942/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/bl0942/test.rp2040-ard.yaml
+++ b/tests/components/bl0942/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ble_client/test.esp32-c3-idf.yaml b/tests/components/ble_client/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/ble_client/test.esp32-c3-idf.yaml
+++ b/tests/components/ble_client/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ble_client/test.esp32-idf.yaml b/tests/components/ble_client/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/ble_client/test.esp32-idf.yaml
+++ b/tests/components/ble_client/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ble_nus/test.nrf52-adafruit.yaml b/tests/components/ble_nus/test.nrf52-adafruit.yaml
new file mode 100644
index 0000000000..20eec16956
--- /dev/null
+++ b/tests/components/ble_nus/test.nrf52-adafruit.yaml
@@ -0,0 +1,2 @@
+ble_nus:
+  type: logs
diff --git a/tests/components/ble_nus/test.nrf52-mcumgr.yaml b/tests/components/ble_nus/test.nrf52-mcumgr.yaml
new file mode 100644
index 0000000000..20eec16956
--- /dev/null
+++ b/tests/components/ble_nus/test.nrf52-mcumgr.yaml
@@ -0,0 +1,2 @@
+ble_nus:
+  type: logs
diff --git a/tests/components/ble_presence/test.esp32-c3-idf.yaml b/tests/components/ble_presence/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/ble_presence/test.esp32-c3-idf.yaml
+++ b/tests/components/ble_presence/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ble_presence/test.esp32-idf.yaml b/tests/components/ble_presence/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/ble_presence/test.esp32-idf.yaml
+++ b/tests/components/ble_presence/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ble_rssi/test.esp32-c3-idf.yaml b/tests/components/ble_rssi/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/ble_rssi/test.esp32-c3-idf.yaml
+++ b/tests/components/ble_rssi/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ble_rssi/test.esp32-idf.yaml b/tests/components/ble_rssi/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/ble_rssi/test.esp32-idf.yaml
+++ b/tests/components/ble_rssi/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ble_scanner/test.esp32-c3-idf.yaml b/tests/components/ble_scanner/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/ble_scanner/test.esp32-c3-idf.yaml
+++ b/tests/components/ble_scanner/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ble_scanner/test.esp32-idf.yaml b/tests/components/ble_scanner/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/ble_scanner/test.esp32-idf.yaml
+++ b/tests/components/ble_scanner/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bluetooth_proxy/test.esp32-p4-idf.yaml b/tests/components/bluetooth_proxy/test.esp32-p4-idf.yaml
new file mode 100644
index 0000000000..edb18c1ada
--- /dev/null
+++ b/tests/components/bluetooth_proxy/test.esp32-p4-idf.yaml
@@ -0,0 +1,19 @@
+<<: !include common.yaml
+
+esp32_ble_tracker:
+  max_connections: 9
+
+bluetooth_proxy:
+  active: true
+  connection_slots: 9
+
+esp32_hosted:
+  active_high: true
+  variant: ESP32C6
+  reset_pin: GPIO54
+  cmd_pin: GPIO19
+  clk_pin: GPIO18
+  d0_pin: GPIO14
+  d1_pin: GPIO15
+  d2_pin: GPIO16
+  d3_pin: GPIO17
diff --git a/tests/components/bluetooth_proxy/test.esp32-s3-ard.yaml b/tests/components/bluetooth_proxy/test.esp32-s3-ard.yaml
deleted file mode 100644
index bf01b65b6f..0000000000
--- a/tests/components/bluetooth_proxy/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-<<: !include common.yaml
-
-esp32_ble_tracker:
-  max_connections: 3
-
-bluetooth_proxy:
-  active: true
-  connection_slots: 2
diff --git a/tests/components/bme280_i2c/common.yaml b/tests/components/bme280_i2c/common.yaml
index e74ce9bf6d..a31a6f9a6c 100644
--- a/tests/components/bme280_i2c/common.yaml
+++ b/tests/components/bme280_i2c/common.yaml
@@ -1,19 +1,14 @@
-i2c:
-  - id: i2c_bme280
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: bme280_i2c
-    i2c_id: i2c_bme280
+    i2c_id: i2c_bus
     address: 0x76
     temperature:
-      id: bme280_temperature
+      id: bme280_i2c_temperature
       name: BME280 Temperature
     humidity:
-      id: bme280_humidity
+      id: bme280_i2c_humidity
       name: BME280 Humidity
     pressure:
-      id: bme280_pressure
+      id: bme280_i2c_pressure
       name: BME280 Pressure
     update_interval: 15s
diff --git a/tests/components/bme280_i2c/test.esp32-ard.yaml b/tests/components/bme280_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/bme280_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/bme280_i2c/test.esp32-c3-ard.yaml b/tests/components/bme280_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/bme280_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/bme280_i2c/test.esp32-c3-idf.yaml b/tests/components/bme280_i2c/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/bme280_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/bme280_i2c/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme280_i2c/test.esp32-idf.yaml b/tests/components/bme280_i2c/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/bme280_i2c/test.esp32-idf.yaml
+++ b/tests/components/bme280_i2c/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme280_i2c/test.esp8266-ard.yaml b/tests/components/bme280_i2c/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/bme280_i2c/test.esp8266-ard.yaml
+++ b/tests/components/bme280_i2c/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme280_i2c/test.rp2040-ard.yaml b/tests/components/bme280_i2c/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/bme280_i2c/test.rp2040-ard.yaml
+++ b/tests/components/bme280_i2c/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme280_spi/common.yaml b/tests/components/bme280_spi/common.yaml
index 303ecf9f73..d97b475f0e 100644
--- a/tests/components/bme280_spi/common.yaml
+++ b/tests/components/bme280_spi/common.yaml
@@ -1,20 +1,13 @@
-spi:
-  - id: spi_bme280
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: bme280_spi
-    spi_id: spi_bme280
     cs_pin: ${cs_pin}
     temperature:
-      id: bme280_temperature
+      id: bme280_spi_temperature
       name: BME280 Temperature
     humidity:
-      id: bme280_humidity
+      id: bme280_spi_humidity
       name: BME280 Humidity
     pressure:
-      id: bme280_pressure
+      id: bme280_spi_pressure
       name: BME280 Pressure
     update_interval: 15s
diff --git a/tests/components/bme280_spi/test.esp32-ard.yaml b/tests/components/bme280_spi/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/bme280_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/bme280_spi/test.esp32-c3-ard.yaml b/tests/components/bme280_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/bme280_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/bme280_spi/test.esp32-c3-idf.yaml b/tests/components/bme280_spi/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/bme280_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/bme280_spi/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme280_spi/test.esp32-idf.yaml b/tests/components/bme280_spi/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/bme280_spi/test.esp32-idf.yaml
+++ b/tests/components/bme280_spi/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bme280_spi/test.esp8266-ard.yaml b/tests/components/bme280_spi/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/bme280_spi/test.esp8266-ard.yaml
+++ b/tests/components/bme280_spi/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bme280_spi/test.rp2040-ard.yaml b/tests/components/bme280_spi/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/bme280_spi/test.rp2040-ard.yaml
+++ b/tests/components/bme280_spi/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bme680/common.yaml b/tests/components/bme680/common.yaml
index 13a42488f2..d5a7267060 100644
--- a/tests/components/bme680/common.yaml
+++ b/tests/components/bme680/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_bme680
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: bme680
+    i2c_id: i2c_bus
     temperature:
       name: BME680 Temperature
       oversampling: 16x
diff --git a/tests/components/bme680/test.esp32-ard.yaml b/tests/components/bme680/test.esp32-ard.yaml
deleted file mode 100644
index 3b761d3fc1..0000000000
--- a/tests/components/bme680/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
-
-<<: !include common.yaml
diff --git a/tests/components/bme680/test.esp32-c3-ard.yaml b/tests/components/bme680/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/bme680/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/bme680/test.esp32-c3-idf.yaml b/tests/components/bme680/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/bme680/test.esp32-c3-idf.yaml
+++ b/tests/components/bme680/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme680/test.esp32-idf.yaml b/tests/components/bme680/test.esp32-idf.yaml
index 3b761d3fc1..b47e39c389 100644
--- a/tests/components/bme680/test.esp32-idf.yaml
+++ b/tests/components/bme680/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme680/test.esp8266-ard.yaml b/tests/components/bme680/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/bme680/test.esp8266-ard.yaml
+++ b/tests/components/bme680/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme680/test.rp2040-ard.yaml b/tests/components/bme680/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/bme680/test.rp2040-ard.yaml
+++ b/tests/components/bme680/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme680_bsec/common.yaml b/tests/components/bme680_bsec/common.yaml
index 7d2e9e210b..1a78ab2ae0 100644
--- a/tests/components/bme680_bsec/common.yaml
+++ b/tests/components/bme680_bsec/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_bme680
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 bme680_bsec:
+  i2c_id: i2c_bus
   address: 0x77
 
 sensor:
diff --git a/tests/components/bme680_bsec/test.esp32-ard.yaml b/tests/components/bme680_bsec/test.esp32-ard.yaml
index 3b761d3fc1..7c503b0ccb 100644
--- a/tests/components/bme680_bsec/test.esp32-ard.yaml
+++ b/tests/components/bme680_bsec/test.esp32-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme680_bsec/test.esp8266-ard.yaml b/tests/components/bme680_bsec/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/bme680_bsec/test.esp8266-ard.yaml
+++ b/tests/components/bme680_bsec/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme68x_bsec2_i2c/common.yaml b/tests/components/bme68x_bsec2_i2c/common.yaml
index b8a16ee7bb..bee964f433 100644
--- a/tests/components/bme68x_bsec2_i2c/common.yaml
+++ b/tests/components/bme68x_bsec2_i2c/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_bme68x
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 bme68x_bsec2_i2c:
+  i2c_id: i2c_bus
   address: 0x76
   model: bme688
   algorithm_output: classification
diff --git a/tests/components/bme68x_bsec2_i2c/test.esp32-ard.yaml b/tests/components/bme68x_bsec2_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/bme68x_bsec2_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/bme68x_bsec2_i2c/test.esp32-c3-ard.yaml b/tests/components/bme68x_bsec2_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index 84a9dd4bb4..0000000000
--- a/tests/components/bme68x_bsec2_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO6
-  sda_pin: GPIO7
-
-<<: !include common.yaml
diff --git a/tests/components/bme68x_bsec2_i2c/test.esp32-c3-idf.yaml b/tests/components/bme68x_bsec2_i2c/test.esp32-c3-idf.yaml
index 84a9dd4bb4..9990d96d29 100644
--- a/tests/components/bme68x_bsec2_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/bme68x_bsec2_i2c/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO6
-  sda_pin: GPIO7
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme68x_bsec2_i2c/test.esp32-idf.yaml b/tests/components/bme68x_bsec2_i2c/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/bme68x_bsec2_i2c/test.esp32-idf.yaml
+++ b/tests/components/bme68x_bsec2_i2c/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme68x_bsec2_i2c/test.esp32-s2-ard.yaml b/tests/components/bme68x_bsec2_i2c/test.esp32-s2-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/bme68x_bsec2_i2c/test.esp32-s2-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/bme68x_bsec2_i2c/test.esp32-s2-idf.yaml b/tests/components/bme68x_bsec2_i2c/test.esp32-s2-idf.yaml
index 63c3bd6afd..54f59a59fc 100644
--- a/tests/components/bme68x_bsec2_i2c/test.esp32-s2-idf.yaml
+++ b/tests/components/bme68x_bsec2_i2c/test.esp32-s2-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-s2-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme68x_bsec2_i2c/test.esp32-s3-ard.yaml b/tests/components/bme68x_bsec2_i2c/test.esp32-s3-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/bme68x_bsec2_i2c/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/bme68x_bsec2_i2c/test.esp32-s3-idf.yaml b/tests/components/bme68x_bsec2_i2c/test.esp32-s3-idf.yaml
index 63c3bd6afd..0fd8684a2c 100644
--- a/tests/components/bme68x_bsec2_i2c/test.esp32-s3-idf.yaml
+++ b/tests/components/bme68x_bsec2_i2c/test.esp32-s3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-s3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme68x_bsec2_i2c/test.esp8266-ard.yaml b/tests/components/bme68x_bsec2_i2c/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/bme68x_bsec2_i2c/test.esp8266-ard.yaml
+++ b/tests/components/bme68x_bsec2_i2c/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bme68x_bsec2_i2c/test.rp2040-ard.yaml b/tests/components/bme68x_bsec2_i2c/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/bme68x_bsec2_i2c/test.rp2040-ard.yaml
+++ b/tests/components/bme68x_bsec2_i2c/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmi160/common.yaml b/tests/components/bmi160/common.yaml
index 6aa9aa6ed0..7375732db2 100644
--- a/tests/components/bmi160/common.yaml
+++ b/tests/components/bmi160/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_bmi160
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: bmi160
+    i2c_id: i2c_bus
     address: 0x68
     acceleration_x:
       name: BMI160 Accel X
diff --git a/tests/components/bmi160/test.esp32-ard.yaml b/tests/components/bmi160/test.esp32-ard.yaml
deleted file mode 100644
index 3b761d3fc1..0000000000
--- a/tests/components/bmi160/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
-
-<<: !include common.yaml
diff --git a/tests/components/bmi160/test.esp32-c3-ard.yaml b/tests/components/bmi160/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/bmi160/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/bmi160/test.esp32-c3-idf.yaml b/tests/components/bmi160/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/bmi160/test.esp32-c3-idf.yaml
+++ b/tests/components/bmi160/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmi160/test.esp32-idf.yaml b/tests/components/bmi160/test.esp32-idf.yaml
index 3b761d3fc1..b47e39c389 100644
--- a/tests/components/bmi160/test.esp32-idf.yaml
+++ b/tests/components/bmi160/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmi160/test.esp8266-ard.yaml b/tests/components/bmi160/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/bmi160/test.esp8266-ard.yaml
+++ b/tests/components/bmi160/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmi160/test.rp2040-ard.yaml b/tests/components/bmi160/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/bmi160/test.rp2040-ard.yaml
+++ b/tests/components/bmi160/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp085/common.yaml b/tests/components/bmp085/common.yaml
index 219bc51fbb..ad358f4409 100644
--- a/tests/components/bmp085/common.yaml
+++ b/tests/components/bmp085/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_bmp085
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: bmp085
+    i2c_id: i2c_bus
     temperature:
       name: Outside Temperature
     pressure:
diff --git a/tests/components/bmp085/test.esp32-ard.yaml b/tests/components/bmp085/test.esp32-ard.yaml
deleted file mode 100644
index 3b761d3fc1..0000000000
--- a/tests/components/bmp085/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
-
-<<: !include common.yaml
diff --git a/tests/components/bmp085/test.esp32-c3-ard.yaml b/tests/components/bmp085/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/bmp085/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/bmp085/test.esp32-c3-idf.yaml b/tests/components/bmp085/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/bmp085/test.esp32-c3-idf.yaml
+++ b/tests/components/bmp085/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp085/test.esp32-idf.yaml b/tests/components/bmp085/test.esp32-idf.yaml
index 3b761d3fc1..b47e39c389 100644
--- a/tests/components/bmp085/test.esp32-idf.yaml
+++ b/tests/components/bmp085/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp085/test.esp8266-ard.yaml b/tests/components/bmp085/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/bmp085/test.esp8266-ard.yaml
+++ b/tests/components/bmp085/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp085/test.rp2040-ard.yaml b/tests/components/bmp085/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/bmp085/test.rp2040-ard.yaml
+++ b/tests/components/bmp085/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp280_i2c/common.yaml b/tests/components/bmp280_i2c/common.yaml
index edf52b2cd4..77a9db7fc5 100644
--- a/tests/components/bmp280_i2c/common.yaml
+++ b/tests/components/bmp280_i2c/common.yaml
@@ -1,17 +1,12 @@
-i2c:
-  - id: i2c_bmp280
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: bmp280_i2c
-    i2c_id: i2c_bmp280
+    i2c_id: i2c_bus
     address: 0x77
     temperature:
-      id: bmp280_temperature
+      id: bmp280_i2c_temperature
       name: Outside Temperature
     pressure:
       name: Outside Pressure
-      id: bmp280_pressure
+      id: bmp280_i2c_pressure
     iir_filter: 16x
     update_interval: 15s
diff --git a/tests/components/bmp280_i2c/test.esp32-ard.yaml b/tests/components/bmp280_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/bmp280_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/bmp280_i2c/test.esp32-c3-ard.yaml b/tests/components/bmp280_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/bmp280_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/bmp280_i2c/test.esp32-c3-idf.yaml b/tests/components/bmp280_i2c/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/bmp280_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/bmp280_i2c/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp280_i2c/test.esp32-idf.yaml b/tests/components/bmp280_i2c/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/bmp280_i2c/test.esp32-idf.yaml
+++ b/tests/components/bmp280_i2c/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp280_i2c/test.esp8266-ard.yaml b/tests/components/bmp280_i2c/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/bmp280_i2c/test.esp8266-ard.yaml
+++ b/tests/components/bmp280_i2c/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp280_i2c/test.rp2040-ard.yaml b/tests/components/bmp280_i2c/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/bmp280_i2c/test.rp2040-ard.yaml
+++ b/tests/components/bmp280_i2c/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp280_spi/common.yaml b/tests/components/bmp280_spi/common.yaml
index 798804de5b..1be54f6b74 100644
--- a/tests/components/bmp280_spi/common.yaml
+++ b/tests/components/bmp280_spi/common.yaml
@@ -1,18 +1,11 @@
-spi:
-  - id: spi_bmp280
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: bmp280_spi
-    spi_id: spi_bmp280
     cs_pin: ${cs_pin}
     temperature:
-      id: bmp280_temperature
+      id: bmp280_spi_temperature
       name: Outside Temperature
     pressure:
       name: Outside Pressure
-      id: bmp280_pressure
+      id: bmp280_spi_pressure
     iir_filter: 16x
     update_interval: 15s
diff --git a/tests/components/bmp280_spi/test.esp32-ard.yaml b/tests/components/bmp280_spi/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/bmp280_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/bmp280_spi/test.esp32-c3-ard.yaml b/tests/components/bmp280_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/bmp280_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/bmp280_spi/test.esp32-c3-idf.yaml b/tests/components/bmp280_spi/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/bmp280_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/bmp280_spi/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp280_spi/test.esp32-idf.yaml b/tests/components/bmp280_spi/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/bmp280_spi/test.esp32-idf.yaml
+++ b/tests/components/bmp280_spi/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bmp280_spi/test.esp8266-ard.yaml b/tests/components/bmp280_spi/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/bmp280_spi/test.esp8266-ard.yaml
+++ b/tests/components/bmp280_spi/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bmp280_spi/test.rp2040-ard.yaml b/tests/components/bmp280_spi/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/bmp280_spi/test.rp2040-ard.yaml
+++ b/tests/components/bmp280_spi/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bmp3xx_i2c/common.yaml b/tests/components/bmp3xx_i2c/common.yaml
index 6641b7a1b8..e651072f25 100644
--- a/tests/components/bmp3xx_i2c/common.yaml
+++ b/tests/components/bmp3xx_i2c/common.yaml
@@ -1,15 +1,12 @@
-i2c:
-  - id: i2c_bmp3xx
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: bmp3xx_i2c
-    i2c_id: i2c_bmp3xx
+    i2c_id: i2c_bus
     address: 0x77
     temperature:
+      id: bmp3xx_i2c_temperature
       name: BMP Temperature
       oversampling: 16x
     pressure:
+      id: bmp3xx_i2c_pressure
       name: BMP Pressure
     iir_filter: 2X
diff --git a/tests/components/bmp3xx_i2c/test.esp32-ard.yaml b/tests/components/bmp3xx_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/bmp3xx_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/bmp3xx_i2c/test.esp32-c3-ard.yaml b/tests/components/bmp3xx_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/bmp3xx_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/bmp3xx_i2c/test.esp32-c3-idf.yaml b/tests/components/bmp3xx_i2c/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/bmp3xx_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/bmp3xx_i2c/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp3xx_i2c/test.esp32-idf.yaml b/tests/components/bmp3xx_i2c/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/bmp3xx_i2c/test.esp32-idf.yaml
+++ b/tests/components/bmp3xx_i2c/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp3xx_i2c/test.esp8266-ard.yaml b/tests/components/bmp3xx_i2c/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/bmp3xx_i2c/test.esp8266-ard.yaml
+++ b/tests/components/bmp3xx_i2c/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp3xx_i2c/test.rp2040-ard.yaml b/tests/components/bmp3xx_i2c/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/bmp3xx_i2c/test.rp2040-ard.yaml
+++ b/tests/components/bmp3xx_i2c/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp3xx_spi/common.yaml b/tests/components/bmp3xx_spi/common.yaml
index 8d5f897661..b59d46c967 100644
--- a/tests/components/bmp3xx_spi/common.yaml
+++ b/tests/components/bmp3xx_spi/common.yaml
@@ -1,16 +1,11 @@
-spi:
-  - id: spi_bmp3xx
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: bmp3xx_spi
-    spi_id: spi_bmp3xx
     cs_pin: ${cs_pin}
     temperature:
+      id: bmp3xx_spi_temperature
       name: BMP Temperature
       oversampling: 16x
     pressure:
+      id: bmp3xx_spi_pressure
       name: BMP Pressure
     iir_filter: 2X
diff --git a/tests/components/bmp3xx_spi/test.esp32-ard.yaml b/tests/components/bmp3xx_spi/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/bmp3xx_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/bmp3xx_spi/test.esp32-c3-ard.yaml b/tests/components/bmp3xx_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/bmp3xx_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/bmp3xx_spi/test.esp32-c3-idf.yaml b/tests/components/bmp3xx_spi/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/bmp3xx_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/bmp3xx_spi/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp3xx_spi/test.esp32-idf.yaml b/tests/components/bmp3xx_spi/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/bmp3xx_spi/test.esp32-idf.yaml
+++ b/tests/components/bmp3xx_spi/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bmp3xx_spi/test.esp8266-ard.yaml b/tests/components/bmp3xx_spi/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/bmp3xx_spi/test.esp8266-ard.yaml
+++ b/tests/components/bmp3xx_spi/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bmp3xx_spi/test.rp2040-ard.yaml b/tests/components/bmp3xx_spi/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/bmp3xx_spi/test.rp2040-ard.yaml
+++ b/tests/components/bmp3xx_spi/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/bmp581/common.yaml b/tests/components/bmp581/common.yaml
index 71ad4bfb1a..250b1f5857 100644
--- a/tests/components/bmp581/common.yaml
+++ b/tests/components/bmp581/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_bmp581
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: bmp581
+    i2c_id: i2c_bus
     temperature:
       name: BMP581 Temperature
       iir_filter: 2x
diff --git a/tests/components/bmp581/test.esp32-ard.yaml b/tests/components/bmp581/test.esp32-ard.yaml
deleted file mode 100644
index 3b761d3fc1..0000000000
--- a/tests/components/bmp581/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
-
-<<: !include common.yaml
diff --git a/tests/components/bmp581/test.esp32-c3-ard.yaml b/tests/components/bmp581/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/bmp581/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/bmp581/test.esp32-c3-idf.yaml b/tests/components/bmp581/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/bmp581/test.esp32-c3-idf.yaml
+++ b/tests/components/bmp581/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp581/test.esp32-idf.yaml b/tests/components/bmp581/test.esp32-idf.yaml
index 3b761d3fc1..b47e39c389 100644
--- a/tests/components/bmp581/test.esp32-idf.yaml
+++ b/tests/components/bmp581/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp581/test.esp8266-ard.yaml b/tests/components/bmp581/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/bmp581/test.esp8266-ard.yaml
+++ b/tests/components/bmp581/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bmp581/test.rp2040-ard.yaml b/tests/components/bmp581/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/bmp581/test.rp2040-ard.yaml
+++ b/tests/components/bmp581/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/bp1658cj/test.esp32-ard.yaml b/tests/components/bp1658cj/test.esp32-ard.yaml
deleted file mode 100644
index d295973e3f..0000000000
--- a/tests/components/bp1658cj/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/bp1658cj/test.esp32-c3-ard.yaml b/tests/components/bp1658cj/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7808481215..0000000000
--- a/tests/components/bp1658cj/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/bp1658cj/test.esp32-idf.yaml b/tests/components/bp1658cj/test.esp32-idf.yaml
index d295973e3f..a4ecdb6c49 100644
--- a/tests/components/bp1658cj/test.esp32-idf.yaml
+++ b/tests/components/bp1658cj/test.esp32-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
+  clock_pin: GPIO4
+  data_pin: GPIO5
 
 <<: !include common.yaml
diff --git a/tests/components/bp1658cj/test.esp8266-ard.yaml b/tests/components/bp1658cj/test.esp8266-ard.yaml
index 7808481215..7c7f1e1a11 100644
--- a/tests/components/bp1658cj/test.esp8266-ard.yaml
+++ b/tests/components/bp1658cj/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
+  clock_pin: GPIO0
+  data_pin: GPIO2
 
 <<: !include common.yaml
diff --git a/tests/components/bp5758d/test.esp32-ard.yaml b/tests/components/bp5758d/test.esp32-ard.yaml
deleted file mode 100644
index d295973e3f..0000000000
--- a/tests/components/bp5758d/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/bp5758d/test.esp32-c3-ard.yaml b/tests/components/bp5758d/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7808481215..0000000000
--- a/tests/components/bp5758d/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/bp5758d/test.esp32-idf.yaml b/tests/components/bp5758d/test.esp32-idf.yaml
index d295973e3f..a4ecdb6c49 100644
--- a/tests/components/bp5758d/test.esp32-idf.yaml
+++ b/tests/components/bp5758d/test.esp32-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
+  clock_pin: GPIO4
+  data_pin: GPIO5
 
 <<: !include common.yaml
diff --git a/tests/components/bp5758d/test.esp8266-ard.yaml b/tests/components/bp5758d/test.esp8266-ard.yaml
index 7808481215..7c7f1e1a11 100644
--- a/tests/components/bp5758d/test.esp8266-ard.yaml
+++ b/tests/components/bp5758d/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
+  clock_pin: GPIO0
+  data_pin: GPIO2
 
 <<: !include common.yaml
diff --git a/tests/components/alpha3/test.esp32-ard.yaml b/tests/components/button/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/alpha3/test.esp32-ard.yaml
rename to tests/components/button/test.nrf52-adafruit.yaml
diff --git a/tests/components/alpha3/test.esp32-c3-ard.yaml b/tests/components/button/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/alpha3/test.esp32-c3-ard.yaml
rename to tests/components/button/test.nrf52-mcumgr.yaml
diff --git a/tests/components/bytebuffer/test.esp32-ard.yaml b/tests/components/bytebuffer/test.esp32-ard.yaml
deleted file mode 100644
index 380ca87628..0000000000
--- a/tests/components/bytebuffer/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-!include common.yaml
diff --git a/tests/components/bytebuffer/test.esp32-c3-ard.yaml b/tests/components/bytebuffer/test.esp32-c3-ard.yaml
deleted file mode 100644
index 380ca87628..0000000000
--- a/tests/components/bytebuffer/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-!include common.yaml
diff --git a/tests/components/camera/common.yaml b/tests/components/camera/common.yaml
index 3daf1e8565..76cca4cf94 100644
--- a/tests/components/camera/common.yaml
+++ b/tests/components/camera/common.yaml
@@ -1,18 +1,2 @@
-esphome:
-  includes:
-    - ../../../esphome/components/camera/
-
-script:
-  - id: interface_compile_check
-    then:
-      - lambda: |-
-            using namespace esphome::camera;
-            class MockCamera : public Camera {
-              public:
-                void add_image_callback(std::function)> &&callback) override {}
-                CameraImageReader *create_image_reader() override { return 0; }
-                void request_image(CameraRequester requester) override {}
-                void start_stream(CameraRequester requester) override {}
-                void stop_stream(CameraRequester requester) override {}
-            };
-            MockCamera* camera = new MockCamera();
+# Camera is a base component auto-loaded by esp32_camera
+# The hardware configuration comes from the camera package
diff --git a/tests/components/camera/test.esp32-idf.yaml b/tests/components/camera/test.esp32-idf.yaml
index dade44d145..ef8f74d4eb 100644
--- a/tests/components/camera/test.esp32-idf.yaml
+++ b/tests/components/camera/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c_camera: !include ../../test_build_components/common/i2c_camera/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/camera_encoder/common.yaml b/tests/components/camera_encoder/common.yaml
new file mode 100644
index 0000000000..8fd7a8ce47
--- /dev/null
+++ b/tests/components/camera_encoder/common.yaml
@@ -0,0 +1,5 @@
+camera_encoder:
+  id: jpeg_encoder
+  quality: 80
+  buffer_size: 4096
+  buffer_expand_size: 1024
diff --git a/tests/components/camera_encoder/test.esp32-idf.yaml b/tests/components/camera_encoder/test.esp32-idf.yaml
new file mode 100644
index 0000000000..ef8f74d4eb
--- /dev/null
+++ b/tests/components/camera_encoder/test.esp32-idf.yaml
@@ -0,0 +1,4 @@
+packages:
+  i2c_camera: !include ../../test_build_components/common/i2c_camera/esp32-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/canbus/test.esp32-idf.yaml b/tests/components/canbus/test.esp32-idf.yaml
index dade44d145..2d29656c94 100644
--- a/tests/components/canbus/test.esp32-idf.yaml
+++ b/tests/components/canbus/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/cap1188/common.yaml b/tests/components/cap1188/common.yaml
index e83bf5d5d2..3e4ed972ff 100644
--- a/tests/components/cap1188/common.yaml
+++ b/tests/components/cap1188/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_cap1188
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 cap1188:
   id: cap1188_component
+  i2c_id: i2c_bus
   address: 0x29
   reset_pin: ${reset_pin}
   touch_threshold: 0x20
diff --git a/tests/components/cap1188/test.esp32-ard.yaml b/tests/components/cap1188/test.esp32-ard.yaml
deleted file mode 100644
index 1ca773e06c..0000000000
--- a/tests/components/cap1188/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  reset_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/cap1188/test.esp32-c3-ard.yaml b/tests/components/cap1188/test.esp32-c3-ard.yaml
deleted file mode 100644
index 1e6670c196..0000000000
--- a/tests/components/cap1188/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  reset_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/cap1188/test.esp32-c3-idf.yaml b/tests/components/cap1188/test.esp32-c3-idf.yaml
index 1e6670c196..c97f30d52c 100644
--- a/tests/components/cap1188/test.esp32-c3-idf.yaml
+++ b/tests/components/cap1188/test.esp32-c3-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/cap1188/test.esp32-idf.yaml b/tests/components/cap1188/test.esp32-idf.yaml
index 1ca773e06c..4ff2241ec9 100644
--- a/tests/components/cap1188/test.esp32-idf.yaml
+++ b/tests/components/cap1188/test.esp32-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   reset_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/cap1188/test.esp8266-ard.yaml b/tests/components/cap1188/test.esp8266-ard.yaml
index dfdc12a3d1..b8bb94edde 100644
--- a/tests/components/cap1188/test.esp8266-ard.yaml
+++ b/tests/components/cap1188/test.esp8266-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/cap1188/test.rp2040-ard.yaml b/tests/components/cap1188/test.rp2040-ard.yaml
index 1e6670c196..1bf10642c5 100644
--- a/tests/components/cap1188/test.rp2040-ard.yaml
+++ b/tests/components/cap1188/test.rp2040-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ccs811/common.yaml b/tests/components/ccs811/common.yaml
index a781996c66..0d912fd3ac 100644
--- a/tests/components/ccs811/common.yaml
+++ b/tests/components/ccs811/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_ccs811
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ccs811
+    i2c_id: i2c_bus
     eco2:
       name: CCS811 eCO2
     tvoc:
diff --git a/tests/components/ccs811/test.esp32-ard.yaml b/tests/components/ccs811/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ccs811/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ccs811/test.esp32-c3-ard.yaml b/tests/components/ccs811/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ccs811/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ccs811/test.esp32-c3-idf.yaml b/tests/components/ccs811/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ccs811/test.esp32-c3-idf.yaml
+++ b/tests/components/ccs811/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ccs811/test.esp32-idf.yaml b/tests/components/ccs811/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ccs811/test.esp32-idf.yaml
+++ b/tests/components/ccs811/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ccs811/test.esp8266-ard.yaml b/tests/components/ccs811/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ccs811/test.esp8266-ard.yaml
+++ b/tests/components/ccs811/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ccs811/test.rp2040-ard.yaml b/tests/components/ccs811/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ccs811/test.rp2040-ard.yaml
+++ b/tests/components/ccs811/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/cd74hc4067/test.esp32-ard.yaml b/tests/components/cd74hc4067/test.esp32-ard.yaml
deleted file mode 100644
index c4dd280943..0000000000
--- a/tests/components/cd74hc4067/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  pin_s0: GPIO12
-  pin_s1: GPIO13
-  pin_s2: GPIO14
-  pin_s3: GPIO15
-  pin: GPIO39
-
-<<: !include common.yaml
diff --git a/tests/components/cd74hc4067/test.esp32-c3-ard.yaml b/tests/components/cd74hc4067/test.esp32-c3-ard.yaml
deleted file mode 100644
index 5e8784c1fc..0000000000
--- a/tests/components/cd74hc4067/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  pin_s0: GPIO2
-  pin_s1: GPIO3
-  pin_s2: GPIO4
-  pin_s3: GPIO5
-  pin: GPIO0
-
-<<: !include common.yaml
diff --git a/tests/components/ch422g/common.yaml b/tests/components/ch422g/common.yaml
index d65956ecac..ad3707eee0 100644
--- a/tests/components/ch422g/common.yaml
+++ b/tests/components/ch422g/common.yaml
@@ -1,5 +1,6 @@
 ch422g:
   - id: ch422g_hub
+    i2c_id: i2c_bus
 
 binary_sensor:
   - platform: gpio
diff --git a/tests/components/ch422g/test.esp32-ard.yaml b/tests/components/ch422g/test.esp32-ard.yaml
deleted file mode 100644
index cd3f1bbeef..0000000000
--- a/tests/components/ch422g/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-i2c:
-  - id: i2c_ch422g
-    scl: 16
-    sda: 17
-
-<<: !include common.yaml
diff --git a/tests/components/ch422g/test.esp32-c3-ard.yaml b/tests/components/ch422g/test.esp32-c3-ard.yaml
deleted file mode 100644
index cd822cb308..0000000000
--- a/tests/components/ch422g/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-i2c:
-  - id: i2c_ch422g
-    scl: 5
-    sda: 4
-
-<<: !include common.yaml
diff --git a/tests/components/ch422g/test.esp32-c3-idf.yaml b/tests/components/ch422g/test.esp32-c3-idf.yaml
index cd822cb308..9990d96d29 100644
--- a/tests/components/ch422g/test.esp32-c3-idf.yaml
+++ b/tests/components/ch422g/test.esp32-c3-idf.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_ch422g
-    scl: 5
-    sda: 4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ch422g/test.esp32-idf.yaml b/tests/components/ch422g/test.esp32-idf.yaml
index cd3f1bbeef..b47e39c389 100644
--- a/tests/components/ch422g/test.esp32-idf.yaml
+++ b/tests/components/ch422g/test.esp32-idf.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_ch422g
-    scl: 16
-    sda: 17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ch422g/test.esp8266-ard.yaml b/tests/components/ch422g/test.esp8266-ard.yaml
index cd822cb308..4a98b9388a 100644
--- a/tests/components/ch422g/test.esp8266-ard.yaml
+++ b/tests/components/ch422g/test.esp8266-ard.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_ch422g
-    scl: 5
-    sda: 4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ch422g/test.rp2040-ard.yaml b/tests/components/ch422g/test.rp2040-ard.yaml
index cd822cb308..319a7c71a6 100644
--- a/tests/components/ch422g/test.rp2040-ard.yaml
+++ b/tests/components/ch422g/test.rp2040-ard.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_ch422g
-    scl: 5
-    sda: 4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/chsc6x/test.esp32-ard.yaml b/tests/components/chsc6x/test.esp32-ard.yaml
deleted file mode 100644
index 9bc58b66f6..0000000000
--- a/tests/components/chsc6x/test.esp32-ard.yaml
+++ /dev/null
@@ -1,25 +0,0 @@
-i2c:
-  - id: i2c_chsc6x
-    scl: 3
-    sda: 21
-
-spi:
-  clk_pin: 16
-  mosi_pin: 17
-
-display:
-  - platform: ili9xxx
-    id: ili9xxx_display
-    model: GC9A01A
-    invert_colors: True
-    cs_pin: 18
-    dc_pin: 19
-    pages:
-      - id: page1
-        lambda: |-
-          it.rectangle(0, 0, it.get_width(), it.get_height());
-
-touchscreen:
-  - platform: chsc6x
-    display: ili9xxx_display
-    interrupt_pin: 20
diff --git a/tests/components/chsc6x/test.esp32-c3-ard.yaml b/tests/components/chsc6x/test.esp32-c3-ard.yaml
deleted file mode 100644
index b0f55eb2e6..0000000000
--- a/tests/components/chsc6x/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,25 +0,0 @@
-i2c:
-  - id: i2c_chsc6x
-    scl: 3
-    sda: 9
-
-spi:
-  clk_pin: 5
-  mosi_pin: 4
-
-display:
-  - platform: ili9xxx
-    id: ili9xxx_display
-    model: GC9A01A
-    invert_colors: True
-    cs_pin: 18
-    dc_pin: 19
-    pages:
-      - id: page1
-        lambda: |-
-          it.rectangle(0, 0, it.get_width(), it.get_height());
-
-touchscreen:
-  - platform: chsc6x
-    display: ili9xxx_display
-    interrupt_pin: 20
diff --git a/tests/components/chsc6x/test.esp32-c3-idf.yaml b/tests/components/chsc6x/test.esp32-c3-idf.yaml
index b0f55eb2e6..f0de4107d7 100644
--- a/tests/components/chsc6x/test.esp32-c3-idf.yaml
+++ b/tests/components/chsc6x/test.esp32-c3-idf.yaml
@@ -1,19 +1,14 @@
-i2c:
-  - id: i2c_chsc6x
-    scl: 3
-    sda: 9
-
-spi:
-  clk_pin: 5
-  mosi_pin: 4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 display:
   - platform: ili9xxx
     id: ili9xxx_display
     model: GC9A01A
     invert_colors: True
-    cs_pin: 18
-    dc_pin: 19
+    cs_pin: 11
+    dc_pin: 7
     pages:
       - id: page1
         lambda: |-
diff --git a/tests/components/chsc6x/test.esp32-idf.yaml b/tests/components/chsc6x/test.esp32-idf.yaml
index 9bc58b66f6..fa7c72150e 100644
--- a/tests/components/chsc6x/test.esp32-idf.yaml
+++ b/tests/components/chsc6x/test.esp32-idf.yaml
@@ -1,19 +1,15 @@
-i2c:
-  - id: i2c_chsc6x
-    scl: 3
-    sda: 21
-
-spi:
-  clk_pin: 16
-  mosi_pin: 17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
 display:
   - platform: ili9xxx
+    spi_id: spi_bus
     id: ili9xxx_display
     model: GC9A01A
     invert_colors: True
-    cs_pin: 18
-    dc_pin: 19
+    cs_pin: 22
+    dc_pin: 21
     pages:
       - id: page1
         lambda: |-
@@ -21,5 +17,6 @@ display:
 
 touchscreen:
   - platform: chsc6x
+    i2c_id: i2c_bus
     display: ili9xxx_display
     interrupt_pin: 20
diff --git a/tests/components/chsc6x/test.rp2040-ard.yaml b/tests/components/chsc6x/test.rp2040-ard.yaml
index dbd0d59fc4..89cc1b7477 100644
--- a/tests/components/chsc6x/test.rp2040-ard.yaml
+++ b/tests/components/chsc6x/test.rp2040-ard.yaml
@@ -1,19 +1,14 @@
-i2c:
-  - id: i2c_chsc6x
-    scl: 1
-    sda: 0
-
-spi:
-  clk_pin: 2
-  mosi_pin: 3
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
 
 display:
   - platform: ili9xxx
     id: ili9xxx_display
     model: GC9A01A
     invert_colors: True
-    cs_pin: 18
-    dc_pin: 19
+    cs_pin: 20
+    dc_pin: 21
     pages:
       - id: page1
         lambda: |-
@@ -22,4 +17,4 @@ display:
 touchscreen:
   - platform: chsc6x
     display: ili9xxx_display
-    interrupt_pin: 20
+    interrupt_pin: 22
diff --git a/tests/components/climate_ir_lg/common.yaml b/tests/components/climate_ir_lg/common.yaml
index c8f84411c0..da0d656b21 100644
--- a/tests/components/climate_ir_lg/common.yaml
+++ b/tests/components/climate_ir_lg/common.yaml
@@ -1,7 +1,4 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: climate_ir_lg
     name: LG Climate
+    transmitter_id: xmitr
diff --git a/tests/components/climate_ir_lg/test.esp32-ard.yaml b/tests/components/climate_ir_lg/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/climate_ir_lg/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/climate_ir_lg/test.esp32-c3-ard.yaml b/tests/components/climate_ir_lg/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/climate_ir_lg/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/climate_ir_lg/test.esp32-c3-idf.yaml b/tests/components/climate_ir_lg/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/climate_ir_lg/test.esp32-c3-idf.yaml
+++ b/tests/components/climate_ir_lg/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/climate_ir_lg/test.esp32-idf.yaml b/tests/components/climate_ir_lg/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/climate_ir_lg/test.esp32-idf.yaml
+++ b/tests/components/climate_ir_lg/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/climate_ir_lg/test.esp8266-ard.yaml b/tests/components/climate_ir_lg/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/climate_ir_lg/test.esp8266-ard.yaml
+++ b/tests/components/climate_ir_lg/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/cm1106/common.yaml b/tests/components/cm1106/common.yaml
index a01e78024e..ed2fd67007 100644
--- a/tests/components/cm1106/common.yaml
+++ b/tests/components/cm1106/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_cm1106
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: cm1106
     co2:
diff --git a/tests/components/cm1106/test.esp32-ard.yaml b/tests/components/cm1106/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/cm1106/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/cm1106/test.esp32-c3-ard.yaml b/tests/components/cm1106/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/cm1106/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/cm1106/test.esp32-c3-idf.yaml b/tests/components/cm1106/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/cm1106/test.esp32-c3-idf.yaml
+++ b/tests/components/cm1106/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/cm1106/test.esp32-idf.yaml b/tests/components/cm1106/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/cm1106/test.esp32-idf.yaml
+++ b/tests/components/cm1106/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/cm1106/test.esp8266-ard.yaml b/tests/components/cm1106/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/cm1106/test.esp8266-ard.yaml
+++ b/tests/components/cm1106/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/cm1106/test.rp2040-ard.yaml b/tests/components/cm1106/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/cm1106/test.rp2040-ard.yaml
+++ b/tests/components/cm1106/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/color_temperature/test.esp32-ard.yaml b/tests/components/color_temperature/test.esp32-ard.yaml
deleted file mode 100644
index 1831adda6e..0000000000
--- a/tests/components/color_temperature/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin_o1: GPIO16
-  pin_o2: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/color_temperature/test.esp32-c3-ard.yaml b/tests/components/color_temperature/test.esp32-c3-ard.yaml
deleted file mode 100644
index 016f315d9f..0000000000
--- a/tests/components/color_temperature/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin_o1: GPIO6
-  pin_o2: GPIO7
-
-<<: !include common.yaml
diff --git a/tests/components/combination/common.yaml b/tests/components/combination/common.yaml
index 62246190af..0e5d512d08 100644
--- a/tests/components/combination/common.yaml
+++ b/tests/components/combination/common.yaml
@@ -4,17 +4,15 @@ sensor:
     lambda: |-
       if (millis() > 10000) {
         return 0.6;
-      } else {
-        return 0.0;
       }
+      return 0.0;
   - platform: template
     id: template_temperature2
     lambda: |-
       if (millis() > 20000) {
         return 0.8;
-      } else {
-        return 0.0;
       }
+      return 0.0;
   - platform: combination
     type: kalman
     name: Kalman-filtered temperature
diff --git a/tests/components/const/common.yaml b/tests/components/const/common.yaml
index f4b15f2b90..109db65b63 100644
--- a/tests/components/const/common.yaml
+++ b/tests/components/const/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  id: quad_spi
-  clk_pin: 15
-  type: quad
-  data_pins: [14, 10, 16, 12]
-
 display:
   - platform: qspi_dbi
     model: RM690B0
diff --git a/tests/components/const/test.esp32-s3-idf.yaml b/tests/components/const/test.esp32-s3-idf.yaml
index dade44d145..c335dee1f3 100644
--- a/tests/components/const/test.esp32-s3-idf.yaml
+++ b/tests/components/const/test.esp32-s3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  qspi: !include ../../test_build_components/common/qspi/esp32-s3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/coolix/common.yaml b/tests/components/coolix/common.yaml
index abe609c3ea..a1f68b8be0 100644
--- a/tests/components/coolix/common.yaml
+++ b/tests/components/coolix/common.yaml
@@ -1,7 +1,4 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: coolix
     name: Coolix Climate
+    transmitter_id: xmitr
diff --git a/tests/components/coolix/test.esp32-ard.yaml b/tests/components/coolix/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/coolix/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/coolix/test.esp32-c3-ard.yaml b/tests/components/coolix/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/coolix/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/coolix/test.esp32-c3-idf.yaml b/tests/components/coolix/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/coolix/test.esp32-c3-idf.yaml
+++ b/tests/components/coolix/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/coolix/test.esp32-idf.yaml b/tests/components/coolix/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/coolix/test.esp32-idf.yaml
+++ b/tests/components/coolix/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/coolix/test.esp8266-ard.yaml b/tests/components/coolix/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/coolix/test.esp8266-ard.yaml
+++ b/tests/components/coolix/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/copy/test.esp32-ard.yaml b/tests/components/copy/test.esp32-ard.yaml
deleted file mode 100644
index e5337726dc..0000000000
--- a/tests/components/copy/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  pwm_platform: ledc
-  pin: GPIO12
-
-<<: !include common.yaml
diff --git a/tests/components/copy/test.esp32-c3-ard.yaml b/tests/components/copy/test.esp32-c3-ard.yaml
deleted file mode 100644
index 76272beb77..0000000000
--- a/tests/components/copy/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  pwm_platform: ledc
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/cs5460a/common.yaml b/tests/components/cs5460a/common.yaml
index d97b01716b..9ecd934eda 100644
--- a/tests/components/cs5460a/common.yaml
+++ b/tests/components/cs5460a/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_cs5460a
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: cs5460a
     id: cs5460a1
diff --git a/tests/components/cs5460a/test.esp32-ard.yaml b/tests/components/cs5460a/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/cs5460a/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/cs5460a/test.esp32-c3-ard.yaml b/tests/components/cs5460a/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/cs5460a/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/cs5460a/test.esp32-c3-idf.yaml b/tests/components/cs5460a/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/cs5460a/test.esp32-c3-idf.yaml
+++ b/tests/components/cs5460a/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/cs5460a/test.esp32-idf.yaml b/tests/components/cs5460a/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/cs5460a/test.esp32-idf.yaml
+++ b/tests/components/cs5460a/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/cs5460a/test.esp8266-ard.yaml b/tests/components/cs5460a/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/cs5460a/test.esp8266-ard.yaml
+++ b/tests/components/cs5460a/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/cs5460a/test.rp2040-ard.yaml b/tests/components/cs5460a/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/cs5460a/test.rp2040-ard.yaml
+++ b/tests/components/cs5460a/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/cse7761/common.yaml b/tests/components/cse7761/common.yaml
index 60cce3864a..77b19957b4 100644
--- a/tests/components/cse7761/common.yaml
+++ b/tests/components/cse7761/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_cse7761
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 38400
-
 sensor:
   - platform: cse7761
     voltage:
diff --git a/tests/components/cse7761/test.esp32-ard.yaml b/tests/components/cse7761/test.esp32-ard.yaml
deleted file mode 100644
index 811f6b72a6..0000000000
--- a/tests/components/cse7761/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/cse7761/test.esp32-c3-ard.yaml b/tests/components/cse7761/test.esp32-c3-ard.yaml
deleted file mode 100644
index c79d14c740..0000000000
--- a/tests/components/cse7761/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/cse7761/test.esp32-c3-idf.yaml b/tests/components/cse7761/test.esp32-c3-idf.yaml
index c79d14c740..4e11c6e7cb 100644
--- a/tests/components/cse7761/test.esp32-c3-idf.yaml
+++ b/tests/components/cse7761/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
+packages:
+  uart_38400: !include ../../test_build_components/common/uart_38400/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/cse7761/test.esp32-idf.yaml b/tests/components/cse7761/test.esp32-idf.yaml
index 811f6b72a6..a6a8fee7e9 100644
--- a/tests/components/cse7761/test.esp32-idf.yaml
+++ b/tests/components/cse7761/test.esp32-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO12
   rx_pin: GPIO14
 
+packages:
+  uart_38400: !include ../../test_build_components/common/uart_38400/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/cse7761/test.esp8266-ard.yaml b/tests/components/cse7761/test.esp8266-ard.yaml
index 3b44f9c9c3..134274ffb8 100644
--- a/tests/components/cse7761/test.esp8266-ard.yaml
+++ b/tests/components/cse7761/test.esp8266-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO1
   rx_pin: GPIO3
 
+packages:
+  uart_38400: !include ../../test_build_components/common/uart_38400/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/cse7761/test.rp2040-ard.yaml b/tests/components/cse7761/test.rp2040-ard.yaml
index b516342f3b..b813e0f7f1 100644
--- a/tests/components/cse7761/test.rp2040-ard.yaml
+++ b/tests/components/cse7761/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart_38400: !include ../../test_build_components/common/uart_38400/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/cse7766/common.yaml b/tests/components/cse7766/common.yaml
index f12b135a77..6db19691f1 100644
--- a/tests/components/cse7766/common.yaml
+++ b/tests/components/cse7766/common.yaml
@@ -1,10 +1,3 @@
-uart:
-  - id: uart_cse7766
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 4800
-    parity: EVEN
-
 sensor:
   - platform: cse7766
     voltage:
diff --git a/tests/components/cse7766/test.esp32-ard.yaml b/tests/components/cse7766/test.esp32-ard.yaml
deleted file mode 100644
index 811f6b72a6..0000000000
--- a/tests/components/cse7766/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/cse7766/test.esp32-c3-ard.yaml b/tests/components/cse7766/test.esp32-c3-ard.yaml
deleted file mode 100644
index c79d14c740..0000000000
--- a/tests/components/cse7766/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/cse7766/test.esp32-c3-idf.yaml b/tests/components/cse7766/test.esp32-c3-idf.yaml
index c79d14c740..dc95c985c7 100644
--- a/tests/components/cse7766/test.esp32-c3-idf.yaml
+++ b/tests/components/cse7766/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
+packages:
+  uart_4800_even: !include ../../test_build_components/common/uart_4800_even/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/cse7766/test.esp32-idf.yaml b/tests/components/cse7766/test.esp32-idf.yaml
index 811f6b72a6..911b867708 100644
--- a/tests/components/cse7766/test.esp32-idf.yaml
+++ b/tests/components/cse7766/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
+packages:
+  uart_4800_even: !include ../../test_build_components/common/uart_4800_even/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/cse7766/test.esp8266-ard.yaml b/tests/components/cse7766/test.esp8266-ard.yaml
index 3b44f9c9c3..77e529ca48 100644
--- a/tests/components/cse7766/test.esp8266-ard.yaml
+++ b/tests/components/cse7766/test.esp8266-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO1
   rx_pin: GPIO3
 
+packages:
+  uart_4800_even: !include ../../test_build_components/common/uart_4800_even/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/cse7766/test.rp2040-ard.yaml b/tests/components/cse7766/test.rp2040-ard.yaml
index b516342f3b..b7056670ef 100644
--- a/tests/components/cse7766/test.rp2040-ard.yaml
+++ b/tests/components/cse7766/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart_4800_even: !include ../../test_build_components/common/uart_4800_even/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/cst226/common.yaml b/tests/components/cst226/common.yaml
index d0b8ea3a86..79d7e7fd53 100644
--- a/tests/components/cst226/common.yaml
+++ b/tests/components/cst226/common.yaml
@@ -1,15 +1,5 @@
-i2c:
-  - id: i2c_cst226
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
-spi:
-  - id: spi_ili9xxx
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
-  - id: my_display
+  - id: cst226_display
     platform: ili9xxx
     model: ili9342
     cs_pin: ${cs_pin}
@@ -19,7 +9,9 @@ display:
 
 touchscreen:
   - id: ts_cst226
+    i2c_id: i2c_bus
     platform: cst226
+    display: cst226_display
     interrupt_pin: ${interrupt_pin}
     reset_pin: ${reset_pin}
 
diff --git a/tests/components/cst226/test.esp32-ard.yaml b/tests/components/cst226/test.esp32-ard.yaml
deleted file mode 100644
index 11e2c4fd43..0000000000
--- a/tests/components/cst226/test.esp32-ard.yaml
+++ /dev/null
@@ -1,12 +0,0 @@
-substitutions:
-  clk_pin: GPIO0
-  mosi_pin: GPIO2
-  cs_pin: GPIO4
-  dc_pin: GPIO5
-  disp_reset_pin: GPIO12
-  scl_pin: GPIO13
-  sda_pin: GPIO14
-  interrupt_pin: GPIO15
-  reset_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/cst226/test.esp32-c3-ard.yaml b/tests/components/cst226/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2f9bd72882..0000000000
--- a/tests/components/cst226/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,12 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  cs_pin: GPIO8
-  dc_pin: GPIO9
-  disp_reset_pin: GPIO10
-  scl_pin: GPIO0
-  sda_pin: GPIO1
-  interrupt_pin: GPIO2
-  reset_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/cst226/test.esp32-c3-idf.yaml b/tests/components/cst226/test.esp32-c3-idf.yaml
index 2f9bd72882..ffc12867d0 100644
--- a/tests/components/cst226/test.esp32-c3-idf.yaml
+++ b/tests/components/cst226/test.esp32-c3-idf.yaml
@@ -1,12 +1,11 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  cs_pin: GPIO8
+  cs_pin: GPIO7
   dc_pin: GPIO9
-  disp_reset_pin: GPIO10
-  scl_pin: GPIO0
-  sda_pin: GPIO1
+  disp_reset_pin: GPIO18
   interrupt_pin: GPIO2
   reset_pin: GPIO3
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/cst226/test.esp32-idf.yaml b/tests/components/cst226/test.esp32-idf.yaml
index 11e2c4fd43..984f08db47 100644
--- a/tests/components/cst226/test.esp32-idf.yaml
+++ b/tests/components/cst226/test.esp32-idf.yaml
@@ -1,12 +1,12 @@
 substitutions:
-  clk_pin: GPIO0
-  mosi_pin: GPIO2
   cs_pin: GPIO4
   dc_pin: GPIO5
   disp_reset_pin: GPIO12
-  scl_pin: GPIO13
-  sda_pin: GPIO14
   interrupt_pin: GPIO15
-  reset_pin: GPIO16
+  reset_pin: GPIO25
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/cst816/common.yaml b/tests/components/cst816/common.yaml
index 9400e4eef0..889a477dd2 100644
--- a/tests/components/cst816/common.yaml
+++ b/tests/components/cst816/common.yaml
@@ -1,15 +1,5 @@
-i2c:
-  - id: i2c_cst816
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
-spi:
-  - id: spi_ili9xxx
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
-  - id: my_display
+  - id: cst816_display
     platform: ili9xxx
     dimensions: 480x320
     model: ST7796
@@ -25,7 +15,9 @@ display:
 
 touchscreen:
   - id: ts_cst816
+    i2c_id: i2c_bus
     platform: cst816
+    display: cst816_display
     interrupt_pin: ${interrupt_pin}
     reset_pin: ${reset_pin}
     skip_probe: false
@@ -36,6 +28,7 @@ touchscreen:
 
 binary_sensor:
   - platform: touchscreen
+    touchscreen_id: ts_cst816
     name: Home Button
     use_raw: true
     x_min: 0
diff --git a/tests/components/cst816/test.esp32-ard.yaml b/tests/components/cst816/test.esp32-ard.yaml
deleted file mode 100644
index 11e2c4fd43..0000000000
--- a/tests/components/cst816/test.esp32-ard.yaml
+++ /dev/null
@@ -1,12 +0,0 @@
-substitutions:
-  clk_pin: GPIO0
-  mosi_pin: GPIO2
-  cs_pin: GPIO4
-  dc_pin: GPIO5
-  disp_reset_pin: GPIO12
-  scl_pin: GPIO13
-  sda_pin: GPIO14
-  interrupt_pin: GPIO15
-  reset_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/cst816/test.esp32-c3-ard.yaml b/tests/components/cst816/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2f9bd72882..0000000000
--- a/tests/components/cst816/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,12 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  cs_pin: GPIO8
-  dc_pin: GPIO9
-  disp_reset_pin: GPIO10
-  scl_pin: GPIO0
-  sda_pin: GPIO1
-  interrupt_pin: GPIO2
-  reset_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/cst816/test.esp32-c3-idf.yaml b/tests/components/cst816/test.esp32-c3-idf.yaml
index 2f9bd72882..ffc12867d0 100644
--- a/tests/components/cst816/test.esp32-c3-idf.yaml
+++ b/tests/components/cst816/test.esp32-c3-idf.yaml
@@ -1,12 +1,11 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  cs_pin: GPIO8
+  cs_pin: GPIO7
   dc_pin: GPIO9
-  disp_reset_pin: GPIO10
-  scl_pin: GPIO0
-  sda_pin: GPIO1
+  disp_reset_pin: GPIO18
   interrupt_pin: GPIO2
   reset_pin: GPIO3
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/cst816/test.esp32-idf.yaml b/tests/components/cst816/test.esp32-idf.yaml
index 11e2c4fd43..984f08db47 100644
--- a/tests/components/cst816/test.esp32-idf.yaml
+++ b/tests/components/cst816/test.esp32-idf.yaml
@@ -1,12 +1,12 @@
 substitutions:
-  clk_pin: GPIO0
-  mosi_pin: GPIO2
   cs_pin: GPIO4
   dc_pin: GPIO5
   disp_reset_pin: GPIO12
-  scl_pin: GPIO13
-  sda_pin: GPIO14
   interrupt_pin: GPIO15
-  reset_pin: GPIO16
+  reset_pin: GPIO25
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ct_clamp/test.esp32-ard.yaml b/tests/components/ct_clamp/test.esp32-ard.yaml
deleted file mode 100644
index 0a70e3f733..0000000000
--- a/tests/components/ct_clamp/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO39
-
-<<: !include common.yaml
diff --git a/tests/components/ct_clamp/test.esp32-c3-ard.yaml b/tests/components/ct_clamp/test.esp32-c3-ard.yaml
deleted file mode 100644
index a8f29c98ae..0000000000
--- a/tests/components/ct_clamp/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO0
-
-<<: !include common.yaml
diff --git a/tests/components/current_based/common.yaml b/tests/components/current_based/common.yaml
index 25dc9671b7..503c4596e9 100644
--- a/tests/components/current_based/common.yaml
+++ b/tests/components/current_based/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_ade7953
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ade7953_i2c
+    i2c_id: i2c_bus
     irq_pin: ${irq_pin}
     voltage:
       name: ADE7953 Voltage
diff --git a/tests/components/current_based/test.esp32-ard.yaml b/tests/components/current_based/test.esp32-ard.yaml
deleted file mode 100644
index 2c57d412f6..0000000000
--- a/tests/components/current_based/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  irq_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/current_based/test.esp32-c3-ard.yaml b/tests/components/current_based/test.esp32-c3-ard.yaml
deleted file mode 100644
index 799acabd5a..0000000000
--- a/tests/components/current_based/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  irq_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/current_based/test.esp32-c3-idf.yaml b/tests/components/current_based/test.esp32-c3-idf.yaml
index 799acabd5a..59296a1e6e 100644
--- a/tests/components/current_based/test.esp32-c3-idf.yaml
+++ b/tests/components/current_based/test.esp32-c3-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/current_based/test.esp32-idf.yaml b/tests/components/current_based/test.esp32-idf.yaml
index 2c57d412f6..49629536e7 100644
--- a/tests/components/current_based/test.esp32-idf.yaml
+++ b/tests/components/current_based/test.esp32-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   irq_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/current_based/test.esp8266-ard.yaml b/tests/components/current_based/test.esp8266-ard.yaml
index c8e6a43f44..dc7609ab37 100644
--- a/tests/components/current_based/test.esp8266-ard.yaml
+++ b/tests/components/current_based/test.esp8266-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/current_based/test.rp2040-ard.yaml b/tests/components/current_based/test.rp2040-ard.yaml
index 799acabd5a..b80562ad22 100644
--- a/tests/components/current_based/test.rp2040-ard.yaml
+++ b/tests/components/current_based/test.rp2040-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/cwww/common.yaml b/tests/components/cwww/common.yaml
index 0ad5beeaae..7fa5ab668c 100644
--- a/tests/components/cwww/common.yaml
+++ b/tests/components/cwww/common.yaml
@@ -1,11 +1,3 @@
-output:
-  - platform: ${light_platform}
-    id: light_output_1
-    pin: ${pin_o1}
-  - platform: ${light_platform}
-    id: light_output_2
-    pin: ${pin_o2}
-
 light:
   - platform: cwww
     name: CWWW Light
diff --git a/tests/components/cwww/test.esp32-ard.yaml b/tests/components/cwww/test.esp32-ard.yaml
deleted file mode 100644
index 1831adda6e..0000000000
--- a/tests/components/cwww/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin_o1: GPIO16
-  pin_o2: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/cwww/test.esp32-c3-ard.yaml b/tests/components/cwww/test.esp32-c3-ard.yaml
deleted file mode 100644
index 016f315d9f..0000000000
--- a/tests/components/cwww/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin_o1: GPIO6
-  pin_o2: GPIO7
-
-<<: !include common.yaml
diff --git a/tests/components/cwww/test.esp32-c3-idf.yaml b/tests/components/cwww/test.esp32-c3-idf.yaml
index 982394ded6..51571b34cf 100644
--- a/tests/components/cwww/test.esp32-c3-idf.yaml
+++ b/tests/components/cwww/test.esp32-c3-idf.yaml
@@ -3,12 +3,15 @@ substitutions:
   pin_o1: GPIO6
   pin_o2: GPIO7
 
-packages:
-  device_base: !include common.yaml
-
 output:
-  - id: !extend light_output_1
+  - platform: ${light_platform}
+    id: light_output_1
+    pin: ${pin_o1}
     channel: 0
-  - id: !extend light_output_2
+  - platform: ${light_platform}
+    id: light_output_2
+    pin: ${pin_o2}
     channel: 1
     phase_angle: 180°
+
+<<: !include common.yaml
diff --git a/tests/components/cwww/test.esp32-idf.yaml b/tests/components/cwww/test.esp32-idf.yaml
index d2a998e6b3..01edf0b0b5 100644
--- a/tests/components/cwww/test.esp32-idf.yaml
+++ b/tests/components/cwww/test.esp32-idf.yaml
@@ -3,12 +3,15 @@ substitutions:
   pin_o1: GPIO16
   pin_o2: GPIO17
 
-packages:
-  device_base: !include common.yaml
-
 output:
-  - id: !extend light_output_1
+  - platform: ${light_platform}
+    id: light_output_1
+    pin: ${pin_o1}
     channel: 0
-  - id: !extend light_output_2
+  - platform: ${light_platform}
+    id: light_output_2
+    pin: ${pin_o2}
     channel: 1
     phase_angle: 180°
+
+<<: !include common.yaml
diff --git a/tests/components/cwww/test.esp8266-ard.yaml b/tests/components/cwww/test.esp8266-ard.yaml
index 75a5b9d64d..49d73b7d3d 100644
--- a/tests/components/cwww/test.esp8266-ard.yaml
+++ b/tests/components/cwww/test.esp8266-ard.yaml
@@ -3,4 +3,12 @@ substitutions:
   pin_o1: GPIO12
   pin_o2: GPIO13
 
+output:
+  - platform: ${light_platform}
+    id: light_output_1
+    pin: ${pin_o1}
+  - platform: ${light_platform}
+    id: light_output_2
+    pin: ${pin_o2}
+
 <<: !include common.yaml
diff --git a/tests/components/cwww/test.rp2040-ard.yaml b/tests/components/cwww/test.rp2040-ard.yaml
index 537177aca1..ba8e0ad071 100644
--- a/tests/components/cwww/test.rp2040-ard.yaml
+++ b/tests/components/cwww/test.rp2040-ard.yaml
@@ -3,4 +3,12 @@ substitutions:
   pin_o1: GPIO12
   pin_o2: GPIO13
 
+output:
+  - platform: ${light_platform}
+    id: light_output_1
+    pin: ${pin_o1}
+  - platform: ${light_platform}
+    id: light_output_2
+    pin: ${pin_o2}
+
 <<: !include common.yaml
diff --git a/tests/components/dac7678/common.yaml b/tests/components/dac7678/common.yaml
index efad81a5ff..6c9c032686 100644
--- a/tests/components/dac7678/common.yaml
+++ b/tests/components/dac7678/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_dac7678
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 dac7678:
+  i2c_id: i2c_bus
   address: 0x4A
   id: dac7678_hub
   internal_reference: true
diff --git a/tests/components/dac7678/test.esp32-ard.yaml b/tests/components/dac7678/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/dac7678/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/dac7678/test.esp32-c3-ard.yaml b/tests/components/dac7678/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/dac7678/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/dac7678/test.esp32-c3-idf.yaml b/tests/components/dac7678/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/dac7678/test.esp32-c3-idf.yaml
+++ b/tests/components/dac7678/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dac7678/test.esp32-idf.yaml b/tests/components/dac7678/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/dac7678/test.esp32-idf.yaml
+++ b/tests/components/dac7678/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dac7678/test.esp8266-ard.yaml b/tests/components/dac7678/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/dac7678/test.esp8266-ard.yaml
+++ b/tests/components/dac7678/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dac7678/test.rp2040-ard.yaml b/tests/components/dac7678/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/dac7678/test.rp2040-ard.yaml
+++ b/tests/components/dac7678/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/daikin/common.yaml b/tests/components/daikin/common.yaml
index 27f381b422..fd73841686 100644
--- a/tests/components/daikin/common.yaml
+++ b/tests/components/daikin/common.yaml
@@ -1,7 +1,3 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: heatpumpir
     protocol: daikin
@@ -10,3 +6,4 @@ climate:
     name: HeatpumpIR Climate
     min_temperature: 18
     max_temperature: 30
+    transmitter_id: xmitr
diff --git a/tests/components/daikin/test.esp32-ard.yaml b/tests/components/daikin/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/daikin/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/daikin/test.esp8266-ard.yaml b/tests/components/daikin/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/daikin/test.esp8266-ard.yaml
+++ b/tests/components/daikin/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/daikin_arc/common.yaml b/tests/components/daikin_arc/common.yaml
index 5c0510f6df..53df3cf911 100644
--- a/tests/components/daikin_arc/common.yaml
+++ b/tests/components/daikin_arc/common.yaml
@@ -1,18 +1,3 @@
-remote_transmitter:
-  pin: ${tx_pin}
-  carrier_duty_percent: 50%
-  id: tsvr
-
-remote_receiver:
-  id: rcvr
-  pin:
-    number: ${rx_pin}
-    inverted: true
-    mode:
-      input: true
-      pullup: true
-  tolerance: 40%
-
 climate:
   - platform: daikin_arc
     name: Daikin AC
diff --git a/tests/components/daikin_arc/test.esp32-ard.yaml b/tests/components/daikin_arc/test.esp32-ard.yaml
deleted file mode 100644
index cd59eb0832..0000000000
--- a/tests/components/daikin_arc/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO2
-  rx_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/daikin_arc/test.esp8266-ard.yaml b/tests/components/daikin_arc/test.esp8266-ard.yaml
index 8e08490d0c..aa8651e556 100644
--- a/tests/components/daikin_arc/test.esp8266-ard.yaml
+++ b/tests/components/daikin_arc/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
-substitutions:
-  tx_pin: GPIO5
-  rx_pin: GPIO4
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
+  remote_receiver: !include ../../test_build_components/common/remote_receiver/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/daikin_brc/common.yaml b/tests/components/daikin_brc/common.yaml
index c9d7baa989..89786954ba 100644
--- a/tests/components/daikin_brc/common.yaml
+++ b/tests/components/daikin_brc/common.yaml
@@ -1,7 +1,4 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: daikin_brc
     name: Daikin_brc Climate
+    transmitter_id: xmitr
diff --git a/tests/components/daikin_brc/test.esp32-ard.yaml b/tests/components/daikin_brc/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/daikin_brc/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/daikin_brc/test.esp32-c3-ard.yaml b/tests/components/daikin_brc/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/daikin_brc/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/daikin_brc/test.esp32-c3-idf.yaml b/tests/components/daikin_brc/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/daikin_brc/test.esp32-c3-idf.yaml
+++ b/tests/components/daikin_brc/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/daikin_brc/test.esp32-idf.yaml b/tests/components/daikin_brc/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/daikin_brc/test.esp32-idf.yaml
+++ b/tests/components/daikin_brc/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/daikin_brc/test.esp8266-ard.yaml b/tests/components/daikin_brc/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/daikin_brc/test.esp8266-ard.yaml
+++ b/tests/components/daikin_brc/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dallas_temp/common.yaml b/tests/components/dallas_temp/common.yaml
index fb51f4818e..6d865d8e93 100644
--- a/tests/components/dallas_temp/common.yaml
+++ b/tests/components/dallas_temp/common.yaml
@@ -1,6 +1,6 @@
 one_wire:
   - platform: gpio
-    pin: 4
+    pin: ${one_wire_pin}
 
 sensor:
   - platform: dallas_temp
diff --git a/tests/components/dallas_temp/test.esp32-c3-idf.yaml b/tests/components/dallas_temp/test.esp32-c3-idf.yaml
index dade44d145..49bf988eb4 100644
--- a/tests/components/dallas_temp/test.esp32-c3-idf.yaml
+++ b/tests/components/dallas_temp/test.esp32-c3-idf.yaml
@@ -1 +1,7 @@
+substitutions:
+  one_wire_pin: "4"
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/dallas_temp/test.esp32-idf.yaml b/tests/components/dallas_temp/test.esp32-idf.yaml
index dade44d145..7f9a7d4c2c 100644
--- a/tests/components/dallas_temp/test.esp32-idf.yaml
+++ b/tests/components/dallas_temp/test.esp32-idf.yaml
@@ -1 +1,7 @@
+substitutions:
+  one_wire_pin: "4"
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/dallas_temp/test.esp8266-ard.yaml b/tests/components/dallas_temp/test.esp8266-ard.yaml
index dade44d145..f58b3e0ff3 100644
--- a/tests/components/dallas_temp/test.esp8266-ard.yaml
+++ b/tests/components/dallas_temp/test.esp8266-ard.yaml
@@ -1 +1,7 @@
+substitutions:
+  one_wire_pin: "13"
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/dallas_temp/test.rp2040-ard.yaml b/tests/components/dallas_temp/test.rp2040-ard.yaml
index dade44d145..d86645aa64 100644
--- a/tests/components/dallas_temp/test.rp2040-ard.yaml
+++ b/tests/components/dallas_temp/test.rp2040-ard.yaml
@@ -1 +1,7 @@
+substitutions:
+  one_wire_pin: "10"
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/daly_bms/common.yaml b/tests/components/daly_bms/common.yaml
index a4cb849f9f..222999e25e 100644
--- a/tests/components/daly_bms/common.yaml
+++ b/tests/components/daly_bms/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_daly_bms
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 4800
-
 daly_bms:
   update_interval: 20s
 
diff --git a/tests/components/daly_bms/test.esp32-ard.yaml b/tests/components/daly_bms/test.esp32-ard.yaml
deleted file mode 100644
index 811f6b72a6..0000000000
--- a/tests/components/daly_bms/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/daly_bms/test.esp32-c3-ard.yaml b/tests/components/daly_bms/test.esp32-c3-ard.yaml
deleted file mode 100644
index c79d14c740..0000000000
--- a/tests/components/daly_bms/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/daly_bms/test.esp32-c3-idf.yaml b/tests/components/daly_bms/test.esp32-c3-idf.yaml
index c79d14c740..4b7c8351a7 100644
--- a/tests/components/daly_bms/test.esp32-c3-idf.yaml
+++ b/tests/components/daly_bms/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/daly_bms/test.esp32-idf.yaml b/tests/components/daly_bms/test.esp32-idf.yaml
index 811f6b72a6..64baa4ec9d 100644
--- a/tests/components/daly_bms/test.esp32-idf.yaml
+++ b/tests/components/daly_bms/test.esp32-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO12
   rx_pin: GPIO14
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/daly_bms/test.esp8266-ard.yaml b/tests/components/daly_bms/test.esp8266-ard.yaml
index 3b44f9c9c3..89ca3ab5ae 100644
--- a/tests/components/daly_bms/test.esp8266-ard.yaml
+++ b/tests/components/daly_bms/test.esp8266-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO1
   rx_pin: GPIO3
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/daly_bms/test.rp2040-ard.yaml b/tests/components/daly_bms/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/daly_bms/test.rp2040-ard.yaml
+++ b/tests/components/daly_bms/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/am43/test.esp32-ard.yaml b/tests/components/datetime/test.bk72xx-ard.yaml
similarity index 100%
rename from tests/components/am43/test.esp32-ard.yaml
rename to tests/components/datetime/test.bk72xx-ard.yaml
diff --git a/tests/components/inkplate6/test.esp32-idf.yaml b/tests/components/datetime/test.esp32-idf.yaml
similarity index 100%
rename from tests/components/inkplate6/test.esp32-idf.yaml
rename to tests/components/datetime/test.esp32-idf.yaml
diff --git a/tests/components/am43/test.esp32-c3-ard.yaml b/tests/components/datetime/test.esp8266-ard.yaml
similarity index 100%
rename from tests/components/am43/test.esp32-c3-ard.yaml
rename to tests/components/datetime/test.esp8266-ard.yaml
diff --git a/tests/components/analog_threshold/test.esp32-ard.yaml b/tests/components/datetime/test.ln882x-ard.yaml
similarity index 100%
rename from tests/components/analog_threshold/test.esp32-ard.yaml
rename to tests/components/datetime/test.ln882x-ard.yaml
diff --git a/tests/components/analog_threshold/test.esp32-c3-ard.yaml b/tests/components/datetime/test.rp2040-ard.yaml
similarity index 100%
rename from tests/components/analog_threshold/test.esp32-c3-ard.yaml
rename to tests/components/datetime/test.rp2040-ard.yaml
diff --git a/tests/components/debug/test.esp32-c3-ard.yaml b/tests/components/debug/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7d43491862..0000000000
--- a/tests/components/debug/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-<<: !include common.yaml
-
-esp32:
-  cpu_frequency: 80MHz
diff --git a/tests/components/anova/test.esp32-ard.yaml b/tests/components/debug/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/anova/test.esp32-ard.yaml
rename to tests/components/debug/test.nrf52-adafruit.yaml
diff --git a/tests/components/anova/test.esp32-c3-ard.yaml b/tests/components/debug/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/anova/test.esp32-c3-ard.yaml
rename to tests/components/debug/test.nrf52-mcumgr.yaml
diff --git a/tests/components/deep_sleep/common-esp32-all.yaml b/tests/components/deep_sleep/common-esp32-all.yaml
new file mode 100644
index 0000000000..b97eec76b9
--- /dev/null
+++ b/tests/components/deep_sleep/common-esp32-all.yaml
@@ -0,0 +1,14 @@
+deep_sleep:
+  run_duration:
+    default: 10s
+    gpio_wakeup_reason: 30s
+    touch_wakeup_reason: 15s
+  sleep_duration: 50s
+  wakeup_pin: ${wakeup_pin}
+  wakeup_pin_mode: INVERT_WAKEUP
+  esp32_ext1_wakeup:
+    pins:
+      - number: GPIO2
+      - number: GPIO13
+    mode: ANY_HIGH
+  touch_wakeup: true
diff --git a/tests/components/deep_sleep/common-esp32-ext1.yaml b/tests/components/deep_sleep/common-esp32-ext1.yaml
new file mode 100644
index 0000000000..9ed4279a33
--- /dev/null
+++ b/tests/components/deep_sleep/common-esp32-ext1.yaml
@@ -0,0 +1,12 @@
+deep_sleep:
+  run_duration:
+    default: 10s
+    gpio_wakeup_reason: 30s
+  sleep_duration: 50s
+  wakeup_pin: ${wakeup_pin}
+  wakeup_pin_mode: INVERT_WAKEUP
+  esp32_ext1_wakeup:
+    pins:
+      - number: GPIO2
+      - number: GPIO5
+    mode: ANY_HIGH
diff --git a/tests/components/deep_sleep/test.esp32-c6-idf.yaml b/tests/components/deep_sleep/test.esp32-c6-idf.yaml
new file mode 100644
index 0000000000..11abe70711
--- /dev/null
+++ b/tests/components/deep_sleep/test.esp32-c6-idf.yaml
@@ -0,0 +1,5 @@
+substitutions:
+  wakeup_pin: GPIO4
+
+<<: !include common.yaml
+<<: !include common-esp32-ext1.yaml
diff --git a/tests/components/deep_sleep/test.esp32-idf.yaml b/tests/components/deep_sleep/test.esp32-idf.yaml
index 10c17af0f5..e45eb08349 100644
--- a/tests/components/deep_sleep/test.esp32-idf.yaml
+++ b/tests/components/deep_sleep/test.esp32-idf.yaml
@@ -2,4 +2,4 @@ substitutions:
   wakeup_pin: GPIO4
 
 <<: !include common.yaml
-<<: !include common-esp32.yaml
+<<: !include common-esp32-all.yaml
diff --git a/tests/components/deep_sleep/test.esp32-ard.yaml b/tests/components/deep_sleep/test.esp32-s2-idf.yaml
similarity index 63%
rename from tests/components/deep_sleep/test.esp32-ard.yaml
rename to tests/components/deep_sleep/test.esp32-s2-idf.yaml
index 10c17af0f5..e45eb08349 100644
--- a/tests/components/deep_sleep/test.esp32-ard.yaml
+++ b/tests/components/deep_sleep/test.esp32-s2-idf.yaml
@@ -2,4 +2,4 @@ substitutions:
   wakeup_pin: GPIO4
 
 <<: !include common.yaml
-<<: !include common-esp32.yaml
+<<: !include common-esp32-all.yaml
diff --git a/tests/components/deep_sleep/test.esp32-c3-ard.yaml b/tests/components/deep_sleep/test.esp32-s3-idf.yaml
similarity index 63%
rename from tests/components/deep_sleep/test.esp32-c3-ard.yaml
rename to tests/components/deep_sleep/test.esp32-s3-idf.yaml
index 10c17af0f5..e45eb08349 100644
--- a/tests/components/deep_sleep/test.esp32-c3-ard.yaml
+++ b/tests/components/deep_sleep/test.esp32-s3-idf.yaml
@@ -2,4 +2,4 @@ substitutions:
   wakeup_pin: GPIO4
 
 <<: !include common.yaml
-<<: !include common-esp32.yaml
+<<: !include common-esp32-all.yaml
diff --git a/tests/components/delonghi/common.yaml b/tests/components/delonghi/common.yaml
index 8e9a1293d7..c3935adcbd 100644
--- a/tests/components/delonghi/common.yaml
+++ b/tests/components/delonghi/common.yaml
@@ -1,7 +1,4 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: delonghi
     name: Delonghi Climate
+    transmitter_id: xmitr
diff --git a/tests/components/delonghi/test.esp32-ard.yaml b/tests/components/delonghi/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/delonghi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/delonghi/test.esp32-c3-ard.yaml b/tests/components/delonghi/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/delonghi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/delonghi/test.esp32-c3-idf.yaml b/tests/components/delonghi/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/delonghi/test.esp32-c3-idf.yaml
+++ b/tests/components/delonghi/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/delonghi/test.esp32-idf.yaml b/tests/components/delonghi/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/delonghi/test.esp32-idf.yaml
+++ b/tests/components/delonghi/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/delonghi/test.esp8266-ard.yaml b/tests/components/delonghi/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/delonghi/test.esp8266-ard.yaml
+++ b/tests/components/delonghi/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dfplayer/common.yaml b/tests/components/dfplayer/common.yaml
index d7446141c3..5d2540c275 100644
--- a/tests/components/dfplayer/common.yaml
+++ b/tests/components/dfplayer/common.yaml
@@ -26,12 +26,6 @@ esphome:
       - dfplayer.volume_down
       - dfplayer.sleep
 
-uart:
-  - id: uart_dfplayer
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 dfplayer:
   on_finished_playback:
     then:
diff --git a/tests/components/dfplayer/test.esp32-ard.yaml b/tests/components/dfplayer/test.esp32-ard.yaml
deleted file mode 100644
index 811f6b72a6..0000000000
--- a/tests/components/dfplayer/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/dfplayer/test.esp32-c3-ard.yaml b/tests/components/dfplayer/test.esp32-c3-ard.yaml
deleted file mode 100644
index c79d14c740..0000000000
--- a/tests/components/dfplayer/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/dfplayer/test.esp32-c3-idf.yaml b/tests/components/dfplayer/test.esp32-c3-idf.yaml
index c79d14c740..4b7c8351a7 100644
--- a/tests/components/dfplayer/test.esp32-c3-idf.yaml
+++ b/tests/components/dfplayer/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dfplayer/test.esp32-idf.yaml b/tests/components/dfplayer/test.esp32-idf.yaml
index 811f6b72a6..64baa4ec9d 100644
--- a/tests/components/dfplayer/test.esp32-idf.yaml
+++ b/tests/components/dfplayer/test.esp32-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO12
   rx_pin: GPIO14
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/dfplayer/test.esp8266-ard.yaml b/tests/components/dfplayer/test.esp8266-ard.yaml
index 3b44f9c9c3..89ca3ab5ae 100644
--- a/tests/components/dfplayer/test.esp8266-ard.yaml
+++ b/tests/components/dfplayer/test.esp8266-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO1
   rx_pin: GPIO3
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/dfplayer/test.rp2040-ard.yaml b/tests/components/dfplayer/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/dfplayer/test.rp2040-ard.yaml
+++ b/tests/components/dfplayer/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/dfrobot_sen0395/common.yaml b/tests/components/dfrobot_sen0395/common.yaml
index 8c349911d3..7f980574ed 100644
--- a/tests/components/dfrobot_sen0395/common.yaml
+++ b/tests/components/dfrobot_sen0395/common.yaml
@@ -14,12 +14,6 @@ esphome:
           sensitivity: 6
       - dfrobot_sen0395.reset
 
-uart:
-  - id: uart_dfrobot_sen0395
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 dfrobot_sen0395:
   - id: mmwave
 
diff --git a/tests/components/dfrobot_sen0395/test.esp32-ard.yaml b/tests/components/dfrobot_sen0395/test.esp32-ard.yaml
deleted file mode 100644
index 811f6b72a6..0000000000
--- a/tests/components/dfrobot_sen0395/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/dfrobot_sen0395/test.esp32-c3-ard.yaml b/tests/components/dfrobot_sen0395/test.esp32-c3-ard.yaml
deleted file mode 100644
index c79d14c740..0000000000
--- a/tests/components/dfrobot_sen0395/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/dfrobot_sen0395/test.esp32-c3-idf.yaml b/tests/components/dfrobot_sen0395/test.esp32-c3-idf.yaml
index c79d14c740..4b7c8351a7 100644
--- a/tests/components/dfrobot_sen0395/test.esp32-c3-idf.yaml
+++ b/tests/components/dfrobot_sen0395/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dfrobot_sen0395/test.esp32-idf.yaml b/tests/components/dfrobot_sen0395/test.esp32-idf.yaml
index 811f6b72a6..64baa4ec9d 100644
--- a/tests/components/dfrobot_sen0395/test.esp32-idf.yaml
+++ b/tests/components/dfrobot_sen0395/test.esp32-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO12
   rx_pin: GPIO14
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/dfrobot_sen0395/test.esp8266-ard.yaml b/tests/components/dfrobot_sen0395/test.esp8266-ard.yaml
index 3b44f9c9c3..89ca3ab5ae 100644
--- a/tests/components/dfrobot_sen0395/test.esp8266-ard.yaml
+++ b/tests/components/dfrobot_sen0395/test.esp8266-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO1
   rx_pin: GPIO3
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/dfrobot_sen0395/test.rp2040-ard.yaml b/tests/components/dfrobot_sen0395/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/dfrobot_sen0395/test.rp2040-ard.yaml
+++ b/tests/components/dfrobot_sen0395/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/dht12/common.yaml b/tests/components/dht12/common.yaml
index 91346e0e27..2602ae13cf 100644
--- a/tests/components/dht12/common.yaml
+++ b/tests/components/dht12/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_dht12
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: dht12
+    i2c_id: i2c_bus
     temperature:
       name: DHT12 Temperature
     humidity:
diff --git a/tests/components/dht12/test.esp32-ard.yaml b/tests/components/dht12/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/dht12/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/dht12/test.esp32-c3-ard.yaml b/tests/components/dht12/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/dht12/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/dht12/test.esp32-c3-idf.yaml b/tests/components/dht12/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/dht12/test.esp32-c3-idf.yaml
+++ b/tests/components/dht12/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dht12/test.esp32-idf.yaml b/tests/components/dht12/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/dht12/test.esp32-idf.yaml
+++ b/tests/components/dht12/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dht12/test.esp8266-ard.yaml b/tests/components/dht12/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/dht12/test.esp8266-ard.yaml
+++ b/tests/components/dht12/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dht12/test.rp2040-ard.yaml b/tests/components/dht12/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/dht12/test.rp2040-ard.yaml
+++ b/tests/components/dht12/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/display/common.yaml b/tests/components/display/common.yaml
index 4fc4fafa25..27abb23e03 100644
--- a/tests/components/display/common.yaml
+++ b/tests/components/display/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 16
-    mosi_pin: 17
-    miso_pin: 15
-
 display:
   - platform: ili9xxx
     id: main_lcd
diff --git a/tests/components/dps310/common.yaml b/tests/components/dps310/common.yaml
index e6c0c9fab8..847a79f2e6 100644
--- a/tests/components/dps310/common.yaml
+++ b/tests/components/dps310/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_dps310
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: dps310
+    i2c_id: i2c_bus
     temperature:
       name: DPS310 Temperature
     pressure:
diff --git a/tests/components/dps310/test.esp32-ard.yaml b/tests/components/dps310/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/dps310/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/dps310/test.esp32-c3-ard.yaml b/tests/components/dps310/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/dps310/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/dps310/test.esp32-c3-idf.yaml b/tests/components/dps310/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/dps310/test.esp32-c3-idf.yaml
+++ b/tests/components/dps310/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dps310/test.esp32-idf.yaml b/tests/components/dps310/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/dps310/test.esp32-idf.yaml
+++ b/tests/components/dps310/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dps310/test.esp8266-ard.yaml b/tests/components/dps310/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/dps310/test.esp8266-ard.yaml
+++ b/tests/components/dps310/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dps310/test.rp2040-ard.yaml b/tests/components/dps310/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/dps310/test.rp2040-ard.yaml
+++ b/tests/components/dps310/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ds1307/common.yaml b/tests/components/ds1307/common.yaml
index 3466a9eec8..cdd2b9cf56 100644
--- a/tests/components/ds1307/common.yaml
+++ b/tests/components/ds1307/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_ds1307
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 time:
   - platform: ds1307
+    i2c_id: i2c_bus
     id: ds1307_time
     update_interval: never
diff --git a/tests/components/ds1307/test.esp32-ard.yaml b/tests/components/ds1307/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ds1307/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ds1307/test.esp32-c3-ard.yaml b/tests/components/ds1307/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ds1307/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ds1307/test.esp32-c3-idf.yaml b/tests/components/ds1307/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ds1307/test.esp32-c3-idf.yaml
+++ b/tests/components/ds1307/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ds1307/test.esp32-idf.yaml b/tests/components/ds1307/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ds1307/test.esp32-idf.yaml
+++ b/tests/components/ds1307/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ds1307/test.esp8266-ard.yaml b/tests/components/ds1307/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ds1307/test.esp8266-ard.yaml
+++ b/tests/components/ds1307/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ds1307/test.rp2040-ard.yaml b/tests/components/ds1307/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ds1307/test.rp2040-ard.yaml
+++ b/tests/components/ds1307/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ds2484/common.yaml b/tests/components/ds2484/common.yaml
index 9d2882a3c0..1e5dcd7dba 100644
--- a/tests/components/ds2484/common.yaml
+++ b/tests/components/ds2484/common.yaml
@@ -1,11 +1,6 @@
-i2c:
-  - id: i2c_ds2484
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 one_wire:
   platform: ds2484
-  i2c_id: i2c_ds2484
+  i2c_id: i2c_bus
   address: 0x18
   active_pullup: true
   strong_pullup: false
diff --git a/tests/components/ds2484/test.esp32-ard.yaml b/tests/components/ds2484/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ds2484/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ds2484/test.esp32-c3-ard.yaml b/tests/components/ds2484/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ds2484/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ds2484/test.esp32-c3-idf.yaml b/tests/components/ds2484/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ds2484/test.esp32-c3-idf.yaml
+++ b/tests/components/ds2484/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ds2484/test.esp32-idf.yaml b/tests/components/ds2484/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ds2484/test.esp32-idf.yaml
+++ b/tests/components/ds2484/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ds2484/test.esp8266-ard.yaml b/tests/components/ds2484/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ds2484/test.esp8266-ard.yaml
+++ b/tests/components/ds2484/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ds2484/test.rp2040-ard.yaml b/tests/components/ds2484/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ds2484/test.rp2040-ard.yaml
+++ b/tests/components/ds2484/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/dsmr/common.yaml b/tests/components/dsmr/common.yaml
index 2901b811fe..038bf2806b 100644
--- a/tests/components/dsmr/common.yaml
+++ b/tests/components/dsmr/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_dsmr
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 dsmr:
   decryption_key: 00112233445566778899aabbccddeeff
   max_telegram_length: 1000
diff --git a/tests/components/dsmr/test.esp32-ard.yaml b/tests/components/dsmr/test.esp32-ard.yaml
index 7a65a48ec0..f218b297aa 100644
--- a/tests/components/dsmr/test.esp32-ard.yaml
+++ b/tests/components/dsmr/test.esp32-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
   request_pin: GPIO15
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/dsmr/test.esp32-c3-ard.yaml b/tests/components/dsmr/test.esp32-c3-ard.yaml
deleted file mode 100644
index 72998506ac..0000000000
--- a/tests/components/dsmr/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  request_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/dsmr/test.esp8266-ard.yaml b/tests/components/dsmr/test.esp8266-ard.yaml
index a47bd58806..08bcf16fc9 100644
--- a/tests/components/dsmr/test.esp8266-ard.yaml
+++ b/tests/components/dsmr/test.esp8266-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
   request_pin: GPIO15
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/dsmr/test.rp2040-ard.yaml b/tests/components/dsmr/test.rp2040-ard.yaml
index 72998506ac..8684cb76aa 100644
--- a/tests/components/dsmr/test.rp2040-ard.yaml
+++ b/tests/components/dsmr/test.rp2040-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
   request_pin: GPIO6
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/atc_mithermometer/test.esp32-ard.yaml b/tests/components/duty_cycle/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/atc_mithermometer/test.esp32-ard.yaml
rename to tests/components/duty_cycle/test.nrf52-adafruit.yaml
diff --git a/tests/components/atc_mithermometer/test.esp32-c3-ard.yaml b/tests/components/duty_cycle/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/atc_mithermometer/test.esp32-c3-ard.yaml
rename to tests/components/duty_cycle/test.nrf52-mcumgr.yaml
diff --git a/tests/components/duty_time/common.yaml b/tests/components/duty_time/common.yaml
index 28fa4afd1c..761d10f16a 100644
--- a/tests/components/duty_time/common.yaml
+++ b/tests/components/duty_time/common.yaml
@@ -4,9 +4,8 @@ binary_sensor:
     lambda: |-
       if (millis() > 10000) {
         return true;
-      } else {
-        return false;
       }
+      return false;
 
 sensor:
   - platform: duty_time
diff --git a/tests/components/duty_time/test.esp32-c3-ard.yaml b/tests/components/duty_time/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/duty_time/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/axs15231/test.esp32-ard.yaml b/tests/components/duty_time/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/axs15231/test.esp32-ard.yaml
rename to tests/components/duty_time/test.nrf52-adafruit.yaml
diff --git a/tests/components/axs15231/test.esp32-c3-ard.yaml b/tests/components/duty_time/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/axs15231/test.esp32-c3-ard.yaml
rename to tests/components/duty_time/test.nrf52-mcumgr.yaml
diff --git a/tests/components/e131/test.esp32-ard.yaml b/tests/components/e131/test.esp32-ard.yaml
deleted file mode 100644
index 32cf2a64ba..0000000000
--- a/tests/components/e131/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  light_platform: esp32_rmt_led_strip
-  pin: GPIO2
-
-<<: !include common-ard.yaml
diff --git a/tests/components/e131/test.esp32-c3-ard.yaml b/tests/components/e131/test.esp32-c3-ard.yaml
deleted file mode 100644
index 32cf2a64ba..0000000000
--- a/tests/components/e131/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  light_platform: esp32_rmt_led_strip
-  pin: GPIO2
-
-<<: !include common-ard.yaml
diff --git a/tests/components/ee895/common.yaml b/tests/components/ee895/common.yaml
index 63d77abcaf..f1b17ca9d9 100644
--- a/tests/components/ee895/common.yaml
+++ b/tests/components/ee895/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_ee895
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ee895
+    i2c_id: i2c_bus
     address: 0x5F
     co2:
       name: EE895 CO2
diff --git a/tests/components/ee895/test.esp32-ard.yaml b/tests/components/ee895/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ee895/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ee895/test.esp32-c3-ard.yaml b/tests/components/ee895/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ee895/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ee895/test.esp32-c3-idf.yaml b/tests/components/ee895/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ee895/test.esp32-c3-idf.yaml
+++ b/tests/components/ee895/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ee895/test.esp32-idf.yaml b/tests/components/ee895/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ee895/test.esp32-idf.yaml
+++ b/tests/components/ee895/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ee895/test.esp8266-ard.yaml b/tests/components/ee895/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ee895/test.esp8266-ard.yaml
+++ b/tests/components/ee895/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ee895/test.rp2040-ard.yaml b/tests/components/ee895/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ee895/test.rp2040-ard.yaml
+++ b/tests/components/ee895/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ektf2232/common.yaml b/tests/components/ektf2232/common.yaml
index 3271839fd4..1c4d768b08 100644
--- a/tests/components/ektf2232/common.yaml
+++ b/tests/components/ektf2232/common.yaml
@@ -1,23 +1,21 @@
-i2c:
-  - id: i2c_ektf2232
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 display:
   - platform: ssd1306_i2c
-    id: ssd1306_display
+    i2c_id: i2c_bus
+    id: ssd1306_i2c_display
     model: SSD1306_128X64
-    reset_pin: ${reset_pin}
+    reset_pin: ${display_reset_pin}
     pages:
-      - id: page1
+      - id: ektf2232_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
 
 touchscreen:
   - platform: ektf2232
+    i2c_id: i2c_bus
+    id: ektf2232_touchscreen
     interrupt_pin: ${interrupt_pin}
-    rts_pin: ${rts_pin}
-    display: ssd1306_display
+    reset_pin: ${touch_reset_pin}
+    display: ssd1306_i2c_display
     on_touch:
       - logger.log:
           format: Touch at (%d, %d)
diff --git a/tests/components/ektf2232/test.esp32-ard.yaml b/tests/components/ektf2232/test.esp32-ard.yaml
deleted file mode 100644
index b8f491c0c3..0000000000
--- a/tests/components/ektf2232/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  reset_pin: GPIO13
-  interrupt_pin: GPIO14
-  rts_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/ektf2232/test.esp32-c3-ard.yaml b/tests/components/ektf2232/test.esp32-c3-ard.yaml
deleted file mode 100644
index 9f2149b9d7..0000000000
--- a/tests/components/ektf2232/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  reset_pin: GPIO3
-  interrupt_pin: GPIO6
-  rts_pin: GPIO7
-
-<<: !include common.yaml
diff --git a/tests/components/ektf2232/test.esp32-c3-idf.yaml b/tests/components/ektf2232/test.esp32-c3-idf.yaml
index 9f2149b9d7..708d352a59 100644
--- a/tests/components/ektf2232/test.esp32-c3-idf.yaml
+++ b/tests/components/ektf2232/test.esp32-c3-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  reset_pin: GPIO3
+  display_reset_pin: GPIO3
   interrupt_pin: GPIO6
-  rts_pin: GPIO7
+  touch_reset_pin: GPIO7
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ektf2232/test.esp32-idf.yaml b/tests/components/ektf2232/test.esp32-idf.yaml
index b8f491c0c3..3fab081bed 100644
--- a/tests/components/ektf2232/test.esp32-idf.yaml
+++ b/tests/components/ektf2232/test.esp32-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  reset_pin: GPIO13
+  display_reset_pin: GPIO13
   interrupt_pin: GPIO14
-  rts_pin: GPIO15
+  touch_reset_pin: GPIO15
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ektf2232/test.esp8266-ard.yaml b/tests/components/ektf2232/test.esp8266-ard.yaml
index 6d91a6533f..38a3894deb 100644
--- a/tests/components/ektf2232/test.esp8266-ard.yaml
+++ b/tests/components/ektf2232/test.esp8266-ard.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  reset_pin: GPIO3
-  interrupt_pin: GPIO12
-  rts_pin: GPIO13
+  display_reset_pin: GPIO3
+  interrupt_pin: GPIO15
+  touch_reset_pin: GPIO16
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ektf2232/test.rp2040-ard.yaml b/tests/components/ektf2232/test.rp2040-ard.yaml
index 9f2149b9d7..cda1b67715 100644
--- a/tests/components/ektf2232/test.rp2040-ard.yaml
+++ b/tests/components/ektf2232/test.rp2040-ard.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  reset_pin: GPIO3
+  display_reset_pin: GPIO3
   interrupt_pin: GPIO6
-  rts_pin: GPIO7
+  touch_reset_pin: GPIO7
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/emc2101/common.yaml b/tests/components/emc2101/common.yaml
index 795b02c6d3..d9e6fe2b96 100644
--- a/tests/components/emc2101/common.yaml
+++ b/tests/components/emc2101/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_emc2101
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 emc2101:
+  i2c_id: i2c_bus
   pwm:
     resolution: 8
 
diff --git a/tests/components/emc2101/test.esp32-ard.yaml b/tests/components/emc2101/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/emc2101/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/emc2101/test.esp32-c3-ard.yaml b/tests/components/emc2101/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/emc2101/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/emc2101/test.esp32-c3-idf.yaml b/tests/components/emc2101/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/emc2101/test.esp32-c3-idf.yaml
+++ b/tests/components/emc2101/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/emc2101/test.esp32-idf.yaml b/tests/components/emc2101/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/emc2101/test.esp32-idf.yaml
+++ b/tests/components/emc2101/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/emc2101/test.esp8266-ard.yaml b/tests/components/emc2101/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/emc2101/test.esp8266-ard.yaml
+++ b/tests/components/emc2101/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/emc2101/test.rp2040-ard.yaml b/tests/components/emc2101/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/emc2101/test.rp2040-ard.yaml
+++ b/tests/components/emc2101/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/emmeti/common.yaml b/tests/components/emmeti/common.yaml
index ac4201e19b..77f381ecf9 100644
--- a/tests/components/emmeti/common.yaml
+++ b/tests/components/emmeti/common.yaml
@@ -1,14 +1,5 @@
-remote_transmitter:
-  id: tx
-  pin: ${remote_transmitter_pin}
-  carrier_duty_percent: 100%
-
-remote_receiver:
-  id: rcvr
-  pin: ${remote_receiver_pin}
-
 climate:
   - platform: emmeti
     name: Emmeti
     receiver_id: rcvr
-    transmitter_id: tx
+    transmitter_id: xmitr
diff --git a/tests/components/emmeti/test.esp32-ard.yaml b/tests/components/emmeti/test.esp32-ard.yaml
deleted file mode 100644
index 2689ff279e..0000000000
--- a/tests/components/emmeti/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  remote_transmitter_pin: GPIO33
-  remote_receiver_pin: GPIO32
-
-<<: !include common.yaml
diff --git a/tests/components/emmeti/test.esp32-idf.yaml b/tests/components/emmeti/test.esp32-idf.yaml
index 2689ff279e..b241dbd159 100644
--- a/tests/components/emmeti/test.esp32-idf.yaml
+++ b/tests/components/emmeti/test.esp32-idf.yaml
@@ -1,5 +1,5 @@
-substitutions:
-  remote_transmitter_pin: GPIO33
-  remote_receiver_pin: GPIO32
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
+  remote_receiver: !include ../../test_build_components/common/remote_receiver/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/emmeti/test.esp8266-ard.yaml b/tests/components/emmeti/test.esp8266-ard.yaml
index 2fb00aea61..aa8651e556 100644
--- a/tests/components/emmeti/test.esp8266-ard.yaml
+++ b/tests/components/emmeti/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
-substitutions:
-  remote_transmitter_pin: GPIO4
-  remote_receiver_pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
+  remote_receiver: !include ../../test_build_components/common/remote_receiver/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/endstop/common.yaml b/tests/components/endstop/common.yaml
index 341fbf7260..b92b1e13b9 100644
--- a/tests/components/endstop/common.yaml
+++ b/tests/components/endstop/common.yaml
@@ -4,9 +4,8 @@ binary_sensor:
     lambda: |-
       if (millis() > 10000) {
         return true;
-      } else {
-        return false;
       }
+      return false;
 
 switch:
   - platform: template
diff --git a/tests/components/endstop/test.esp32-ard.yaml b/tests/components/endstop/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/endstop/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/endstop/test.esp32-c3-ard.yaml b/tests/components/endstop/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/endstop/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/ens160_i2c/common.yaml b/tests/components/ens160_i2c/common.yaml
index 39a5b35067..1da2bacec2 100644
--- a/tests/components/ens160_i2c/common.yaml
+++ b/tests/components/ens160_i2c/common.yaml
@@ -1,15 +1,13 @@
-i2c:
-  - id: i2c_ens160
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ens160_i2c
-    i2c_id: i2c_ens160
+    i2c_id: i2c_bus
     address: 0x53
     eco2:
+      id: ens160_i2c_eco2
       name: "ENS160 eCO2"
     tvoc:
+      id: ens160_i2c_tvoc
       name: "ENS160 Total Volatile Organic Compounds"
     aqi:
+      id: ens160_i2c_aqi
       name: "ENS160 Air Quality Index"
diff --git a/tests/components/ens160_i2c/test.esp32-ard.yaml b/tests/components/ens160_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ens160_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ens160_i2c/test.esp32-c3-ard.yaml b/tests/components/ens160_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ens160_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ens160_i2c/test.esp32-c3-idf.yaml b/tests/components/ens160_i2c/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ens160_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/ens160_i2c/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ens160_i2c/test.esp32-idf.yaml b/tests/components/ens160_i2c/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ens160_i2c/test.esp32-idf.yaml
+++ b/tests/components/ens160_i2c/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ens160_i2c/test.esp8266-ard.yaml b/tests/components/ens160_i2c/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ens160_i2c/test.esp8266-ard.yaml
+++ b/tests/components/ens160_i2c/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ens160_i2c/test.rp2040-ard.yaml b/tests/components/ens160_i2c/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ens160_i2c/test.rp2040-ard.yaml
+++ b/tests/components/ens160_i2c/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ens160_spi/common.yaml b/tests/components/ens160_spi/common.yaml
index c8b663272f..46a3f40b40 100644
--- a/tests/components/ens160_spi/common.yaml
+++ b/tests/components/ens160_spi/common.yaml
@@ -1,16 +1,12 @@
-spi:
-  - id: spi_ens160
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: ens160_spi
-    spi_id: spi_ens160
     cs_pin: ${cs_pin}
     eco2:
+      id: ens160_spi_eco2
       name: "ENS160 eCO2"
     tvoc:
+      id: ens160_spi_tvoc
       name: "ENS160 Total Volatile Organic Compounds"
     aqi:
+      id: ens160_spi_aqi
       name: "ENS160 Air Quality Index"
diff --git a/tests/components/ens160_spi/test.esp32-ard.yaml b/tests/components/ens160_spi/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/ens160_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/ens160_spi/test.esp32-c3-ard.yaml b/tests/components/ens160_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/ens160_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/ens160_spi/test.esp32-c3-idf.yaml b/tests/components/ens160_spi/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/ens160_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/ens160_spi/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ens160_spi/test.esp32-idf.yaml b/tests/components/ens160_spi/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/ens160_spi/test.esp32-idf.yaml
+++ b/tests/components/ens160_spi/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ens160_spi/test.esp8266-ard.yaml b/tests/components/ens160_spi/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/ens160_spi/test.esp8266-ard.yaml
+++ b/tests/components/ens160_spi/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ens160_spi/test.rp2040-ard.yaml b/tests/components/ens160_spi/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/ens160_spi/test.rp2040-ard.yaml
+++ b/tests/components/ens160_spi/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ens210/common.yaml b/tests/components/ens210/common.yaml
index b276154449..6bc8f824cc 100644
--- a/tests/components/ens210/common.yaml
+++ b/tests/components/ens210/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_ens210
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ens210
+    i2c_id: i2c_bus
     temperature:
       name: ENS210 Temperature
     humidity:
diff --git a/tests/components/ens210/test.esp32-ard.yaml b/tests/components/ens210/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ens210/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ens210/test.esp32-c3-ard.yaml b/tests/components/ens210/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ens210/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ens210/test.esp32-c3-idf.yaml b/tests/components/ens210/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ens210/test.esp32-c3-idf.yaml
+++ b/tests/components/ens210/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ens210/test.esp32-idf.yaml b/tests/components/ens210/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ens210/test.esp32-idf.yaml
+++ b/tests/components/ens210/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ens210/test.esp8266-ard.yaml b/tests/components/ens210/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ens210/test.esp8266-ard.yaml
+++ b/tests/components/ens210/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ens210/test.rp2040-ard.yaml b/tests/components/ens210/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ens210/test.rp2040-ard.yaml
+++ b/tests/components/ens210/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/epaper_spi/test.esp32-s3-idf.yaml b/tests/components/epaper_spi/test.esp32-s3-idf.yaml
new file mode 100644
index 0000000000..34aefb82b4
--- /dev/null
+++ b/tests/components/epaper_spi/test.esp32-s3-idf.yaml
@@ -0,0 +1,15 @@
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-s3-idf.yaml
+
+display:
+  - platform: epaper_spi
+    spi_id: spi_bus
+    model: 7.3in-spectra-e6
+    cs_pin: GPIO5
+    dc_pin: GPIO17
+    reset_pin: GPIO16
+    busy_pin: GPIO4
+    rotation: 0
+    update_interval: 60s
+    lambda: |-
+      it.circle(64, 64, 50, Color::BLACK);
diff --git a/tests/components/es7210/common.yaml b/tests/components/es7210/common.yaml
index 3fab177cb3..b9f0b0c3e9 100644
--- a/tests/components/es7210/common.yaml
+++ b/tests/components/es7210/common.yaml
@@ -4,13 +4,9 @@ esphome:
       - audio_adc.set_mic_gain: 0db
       - audio_adc.set_mic_gain: !lambda 'return 4;'
 
-i2c:
-  - id: i2c_aic3204
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 audio_adc:
   - platform: es7210
+    i2c_id: i2c_bus
     id: es7210_adc
     bits_per_sample: 16bit
     sample_rate: 16000
diff --git a/tests/components/es7210/test.esp32-ard.yaml b/tests/components/es7210/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/es7210/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/es7210/test.esp32-c3-ard.yaml b/tests/components/es7210/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/es7210/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/es7210/test.esp32-c3-idf.yaml b/tests/components/es7210/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/es7210/test.esp32-c3-idf.yaml
+++ b/tests/components/es7210/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/es7210/test.esp32-idf.yaml b/tests/components/es7210/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/es7210/test.esp32-idf.yaml
+++ b/tests/components/es7210/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/es7243e/common.yaml b/tests/components/es7243e/common.yaml
index 3de76909e9..0fcfb97273 100644
--- a/tests/components/es7243e/common.yaml
+++ b/tests/components/es7243e/common.yaml
@@ -4,11 +4,7 @@ esphome:
       - audio_adc.set_mic_gain: 0db
       - audio_adc.set_mic_gain: !lambda 'return 4;'
 
-i2c:
-  - id: i2c_es7243e
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 audio_adc:
   - platform: es7243e
+    i2c_id: i2c_bus
     id: es7243e_adc
diff --git a/tests/components/es7243e/test.esp32-ard.yaml b/tests/components/es7243e/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/es7243e/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/es7243e/test.esp32-c3-ard.yaml b/tests/components/es7243e/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/es7243e/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/es7243e/test.esp32-c3-idf.yaml b/tests/components/es7243e/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/es7243e/test.esp32-c3-idf.yaml
+++ b/tests/components/es7243e/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/es7243e/test.esp32-idf.yaml b/tests/components/es7243e/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/es7243e/test.esp32-idf.yaml
+++ b/tests/components/es7243e/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/es8156/common.yaml b/tests/components/es8156/common.yaml
index addaa0b70a..c41a332bd7 100644
--- a/tests/components/es8156/common.yaml
+++ b/tests/components/es8156/common.yaml
@@ -6,10 +6,6 @@ esphome:
       - audio_dac.set_volume:
           volume: 50%
 
-i2c:
-  - id: i2c_es8156
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 audio_dac:
   - platform: es8156
+    i2c_id: i2c_bus
diff --git a/tests/components/es8156/test.esp32-ard.yaml b/tests/components/es8156/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/es8156/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/es8156/test.esp32-c3-ard.yaml b/tests/components/es8156/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/es8156/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/es8156/test.esp32-c3-idf.yaml b/tests/components/es8156/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/es8156/test.esp32-c3-idf.yaml
+++ b/tests/components/es8156/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/es8156/test.esp32-idf.yaml b/tests/components/es8156/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/es8156/test.esp32-idf.yaml
+++ b/tests/components/es8156/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/es8156/test.esp8266-ard.yaml b/tests/components/es8156/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/es8156/test.esp8266-ard.yaml
+++ b/tests/components/es8156/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/es8311/common.yaml b/tests/components/es8311/common.yaml
index d833d1c043..e855761ef4 100644
--- a/tests/components/es8311/common.yaml
+++ b/tests/components/es8311/common.yaml
@@ -6,10 +6,6 @@ esphome:
       - audio_dac.set_volume:
           volume: 50%
 
-i2c:
-  - id: i2c_aic3204
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 audio_dac:
   - platform: es8311
+    i2c_id: i2c_bus
diff --git a/tests/components/es8311/test.esp32-ard.yaml b/tests/components/es8311/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/es8311/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/es8311/test.esp32-c3-ard.yaml b/tests/components/es8311/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/es8311/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/es8311/test.esp32-c3-idf.yaml b/tests/components/es8311/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/es8311/test.esp32-c3-idf.yaml
+++ b/tests/components/es8311/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/es8311/test.esp32-idf.yaml b/tests/components/es8311/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/es8311/test.esp32-idf.yaml
+++ b/tests/components/es8311/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/es8311/test.esp8266-ard.yaml b/tests/components/es8311/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/es8311/test.esp8266-ard.yaml
+++ b/tests/components/es8311/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/es8388/common.yaml b/tests/components/es8388/common.yaml
index 6a63de5aa1..cc7465ab93 100644
--- a/tests/components/es8388/common.yaml
+++ b/tests/components/es8388/common.yaml
@@ -7,13 +7,9 @@ esphome:
       - audio_dac.set_volume:
           volume: 50%
 
-i2c:
-  - id: i2c_es8388
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 audio_dac:
   - platform: es8388
+    i2c_id: i2c_bus
     id: es8388_parent
 
 select:
diff --git a/tests/components/es8388/test.esp32-ard.yaml b/tests/components/es8388/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/es8388/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/es8388/test.esp32-c3-ard.yaml b/tests/components/es8388/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/es8388/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/es8388/test.esp32-c3-idf.yaml b/tests/components/es8388/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/es8388/test.esp32-c3-idf.yaml
+++ b/tests/components/es8388/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/es8388/test.esp32-idf.yaml b/tests/components/es8388/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/es8388/test.esp32-idf.yaml
+++ b/tests/components/es8388/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/es8388/test.esp8266-ard.yaml b/tests/components/es8388/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/es8388/test.esp8266-ard.yaml
+++ b/tests/components/es8388/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/esp32/test.esp32-idf.yaml b/tests/components/esp32/test.esp32-idf.yaml
index ccf0b7cbd5..6338fe98dd 100644
--- a/tests/components/esp32/test.esp32-idf.yaml
+++ b/tests/components/esp32/test.esp32-idf.yaml
@@ -5,6 +5,7 @@ esp32:
     advanced:
       enable_lwip_mdns_queries: true
       enable_lwip_bridge_interface: true
+      disable_libc_locks_in_iram: false  # Test explicit opt-out of RAM optimization
 
 wifi:
   ssid: MySSID
diff --git a/tests/components/esp32/test.esp32-s3-idf.yaml b/tests/components/esp32/test.esp32-s3-idf.yaml
new file mode 100644
index 0000000000..4ae5e6b999
--- /dev/null
+++ b/tests/components/esp32/test.esp32-s3-idf.yaml
@@ -0,0 +1,13 @@
+esp32:
+  variant: esp32s3
+  framework:
+    type: esp-idf
+    advanced:
+      execute_from_psram: true
+      disable_libc_locks_in_iram: true  # Test default RAM optimization enabled
+
+psram:
+  mode: octal
+  speed: 80MHz
+
+logger:
diff --git a/tests/components/esp32_ble/test.esp32-c3-ard.yaml b/tests/components/esp32_ble/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esp32_ble/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/esp32_ble_beacon/test.esp32-ard.yaml b/tests/components/esp32_ble_beacon/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esp32_ble_beacon/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/esp32_ble_beacon/test.esp32-c3-ard.yaml b/tests/components/esp32_ble_beacon/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esp32_ble_beacon/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/esp32_ble_client/test.esp32-ard.yaml b/tests/components/esp32_ble_client/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esp32_ble_client/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/esp32_ble_client/test.esp32-c3-ard.yaml b/tests/components/esp32_ble_client/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esp32_ble_client/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/esp32_ble_client/test.esp32-c3-idf.yaml b/tests/components/esp32_ble_client/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/esp32_ble_client/test.esp32-c3-idf.yaml
+++ b/tests/components/esp32_ble_client/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/esp32_ble_client/test.esp32-idf.yaml b/tests/components/esp32_ble_client/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/esp32_ble_client/test.esp32-idf.yaml
+++ b/tests/components/esp32_ble_client/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/esp32_ble_server/test.esp32-ard.yaml b/tests/components/esp32_ble_server/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esp32_ble_server/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/esp32_ble_server/test.esp32-c3-ard.yaml b/tests/components/esp32_ble_server/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esp32_ble_server/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/esp32_ble_tracker/test-on-scan-end.esp32-idf.yaml b/tests/components/esp32_ble_tracker/test-on-scan-end.esp32-idf.yaml
new file mode 100644
index 0000000000..4e9849a540
--- /dev/null
+++ b/tests/components/esp32_ble_tracker/test-on-scan-end.esp32-idf.yaml
@@ -0,0 +1,3 @@
+esp32_ble_tracker:
+  on_scan_end:
+    - logger.log: "Scan ended!"
diff --git a/tests/components/esp32_ble_tracker/test.esp32-ard.yaml b/tests/components/esp32_ble_tracker/test.esp32-ard.yaml
deleted file mode 100644
index 3bfdb8773f..0000000000
--- a/tests/components/esp32_ble_tracker/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-<<: !include common.yaml
-
-esp32_ble_tracker:
-  software_coexistence: true
-  max_connections: 3
diff --git a/tests/components/esp32_ble_tracker/test.esp32-c3-ard.yaml b/tests/components/esp32_ble_tracker/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2e3c48117a..0000000000
--- a/tests/components/esp32_ble_tracker/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-<<: !include common.yaml
-
-esp32_ble_tracker:
-  max_connections: 3
-  software_coexistence: false
diff --git a/tests/components/esp32_ble_tracker/test.esp32-c3-idf.yaml b/tests/components/esp32_ble_tracker/test.esp32-c3-idf.yaml
index b71896bad5..ea6f0d4022 100644
--- a/tests/components/esp32_ble_tracker/test.esp32-c3-idf.yaml
+++ b/tests/components/esp32_ble_tracker/test.esp32-c3-idf.yaml
@@ -1,3 +1,6 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
 
 esp32_ble_tracker:
diff --git a/tests/components/esp32_ble_tracker/test.esp32-idf.yaml b/tests/components/esp32_ble_tracker/test.esp32-idf.yaml
index 1ffcfb9988..8f6e3c5731 100644
--- a/tests/components/esp32_ble_tracker/test.esp32-idf.yaml
+++ b/tests/components/esp32_ble_tracker/test.esp32-idf.yaml
@@ -1,3 +1,6 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
 
 esp32_ble_tracker:
diff --git a/tests/components/esp32_camera/common.yaml b/tests/components/esp32_camera/common.yaml
index 2f5f792f1c..eac5109bc8 100644
--- a/tests/components/esp32_camera/common.yaml
+++ b/tests/components/esp32_camera/common.yaml
@@ -1,28 +1 @@
-esp32_camera:
-  name: ESP32 Camera
-  data_pins:
-    - number: 17
-    - number: 35
-    - number: 34
-    - number: 5
-    - number: 39
-    - number: 18
-    - number: 36
-    - number: 19
-  vsync_pin: 22
-  href_pin: 26
-  pixel_clock_pin: 21
-  external_clock:
-    pin: 27
-    frequency: 20MHz
-  i2c_pins:
-    sda: 25
-    scl: 23
-  reset_pin: 15
-  power_down_pin: 1
-  resolution: 640x480
-  jpeg_quality: 10
-  on_image:
-    then:
-      - lambda: |-
-          ESP_LOGD("main", "image len=%d, data=%c", image.length, image.data[0]);
+# ESP32 camera hardware configuration comes from the camera package
diff --git a/tests/components/esp32_camera/test.esp32-ard.yaml b/tests/components/esp32_camera/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esp32_camera/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/esp32_camera/test.esp32-idf.yaml b/tests/components/esp32_camera/test.esp32-idf.yaml
index dade44d145..ef8f74d4eb 100644
--- a/tests/components/esp32_camera/test.esp32-idf.yaml
+++ b/tests/components/esp32_camera/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c_camera: !include ../../test_build_components/common/i2c_camera/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/esp32_camera_web_server/common.yaml b/tests/components/esp32_camera_web_server/common.yaml
index fe2a6a2739..2a1858f681 100644
--- a/tests/components/esp32_camera_web_server/common.yaml
+++ b/tests/components/esp32_camera_web_server/common.yaml
@@ -1,32 +1,3 @@
-esp32_camera:
-  name: ESP32 Camera
-  data_pins:
-    - number: 17
-    - number: 35
-    - number: 34
-    - number: 5
-    - number: 39
-    - number: 18
-    - number: 36
-    - number: 19
-  vsync_pin: 22
-  href_pin: 26
-  pixel_clock_pin: 21
-  external_clock:
-    pin: 27
-    frequency: 20MHz
-  i2c_pins:
-    sda: 25
-    scl: 23
-  reset_pin: 15
-  power_down_pin: 1
-  resolution: 640x480
-  jpeg_quality: 10
-  on_image:
-    then:
-      - lambda: |-
-          ESP_LOGD("main", "image len=%d, data=%c", image.length, image.data[0]);
-
 esp32_camera_web_server:
   - port: 8080
     mode: stream
diff --git a/tests/components/esp32_camera_web_server/test.esp32-ard.yaml b/tests/components/esp32_camera_web_server/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esp32_camera_web_server/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/esp32_camera_web_server/test.esp32-idf.yaml b/tests/components/esp32_camera_web_server/test.esp32-idf.yaml
index dade44d145..ef8f74d4eb 100644
--- a/tests/components/esp32_camera_web_server/test.esp32-idf.yaml
+++ b/tests/components/esp32_camera_web_server/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c_camera: !include ../../test_build_components/common/i2c_camera/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/esp32_can/test.esp32-ard.yaml b/tests/components/esp32_can/test.esp32-ard.yaml
deleted file mode 100644
index 811f6b72a6..0000000000
--- a/tests/components/esp32_can/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/esp32_can/test.esp32-c3-ard.yaml b/tests/components/esp32_can/test.esp32-c3-ard.yaml
deleted file mode 100644
index c79d14c740..0000000000
--- a/tests/components/esp32_can/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO7
-  rx_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/esp32_can/test.esp32-c3-idf.yaml b/tests/components/esp32_can/test.esp32-c3-idf.yaml
index c79d14c740..22effb1bd0 100644
--- a/tests/components/esp32_can/test.esp32-c3-idf.yaml
+++ b/tests/components/esp32_can/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
   tx_pin: GPIO7
-  rx_pin: GPIO8
+  rx_pin: GPIO9
 
 <<: !include common.yaml
diff --git a/tests/components/esp32_can/test.esp32-c6-idf.yaml b/tests/components/esp32_can/test.esp32-c6-idf.yaml
new file mode 100644
index 0000000000..6ef730c378
--- /dev/null
+++ b/tests/components/esp32_can/test.esp32-c6-idf.yaml
@@ -0,0 +1,89 @@
+esphome:
+  on_boot:
+    then:
+      - canbus.send:
+          # Extended ID explicit
+          canbus_id: esp32_internal_can
+          use_extended_id: true
+          can_id: 0x100
+          data: [0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08]
+      - canbus.send:
+          # Standard ID by default
+          canbus_id: esp32_internal_can
+          can_id: 0x100
+          data: [0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08]
+      - canbus.send:
+          # Extended ID explicit
+          canbus_id: esp32_internal_can_2
+          use_extended_id: true
+          can_id: 0x100
+          data: [0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08]
+      - canbus.send:
+          # Standard ID by default
+          canbus_id: esp32_internal_can_2
+          can_id: 0x100
+          data: [0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08]
+
+canbus:
+  - platform: esp32_can
+    id: esp32_internal_can
+    rx_pin: GPIO8
+    tx_pin: GPIO7
+    can_id: 4
+    bit_rate: 50kbps
+    on_frame:
+      - can_id: 500
+        then:
+          - lambda: |-
+              std::string b(x.begin(), x.end());
+              ESP_LOGD("canbus1", "canid 500 %s", b.c_str() );
+      - can_id: 0b00000000000000000000001000000
+        can_id_mask: 0b11111000000000011111111000000
+        use_extended_id: true
+        then:
+          - lambda: |-
+              auto pdo_id = can_id >> 14;
+              switch (pdo_id)
+              {
+                case 117:
+                  ESP_LOGD("canbus1", "exhaust_fan_duty");
+                  break;
+                case 118:
+                  ESP_LOGD("canbus1", "supply_fan_duty");
+                  break;
+                case 119:
+                  ESP_LOGD("canbus1", "supply_fan_flow");
+                  break;
+                // to be continued...
+              }
+  - platform: esp32_can
+    id: esp32_internal_can_2
+    rx_pin: GPIO10
+    tx_pin: GPIO9
+    can_id: 4
+    bit_rate: 50kbps
+    on_frame:
+      - can_id: 500
+        then:
+          - lambda: |-
+              std::string b(x.begin(), x.end());
+              ESP_LOGD("canbus2", "canid 500 %s", b.c_str() );
+      - can_id: 0b00000000000000000000001000000
+        can_id_mask: 0b11111000000000011111111000000
+        use_extended_id: true
+        then:
+          - lambda: |-
+              auto pdo_id = can_id >> 14;
+              switch (pdo_id)
+              {
+                case 117:
+                  ESP_LOGD("canbus2", "exhaust_fan_duty");
+                  break;
+                case 118:
+                  ESP_LOGD("canbus2", "supply_fan_duty");
+                  break;
+                case 119:
+                  ESP_LOGD("canbus2", "supply_fan_flow");
+                  break;
+                // to be continued...
+              }
diff --git a/tests/components/esp32_dac/test.esp32-ard.yaml b/tests/components/esp32_dac/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esp32_dac/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/esp32_improv/test.esp32-ard.yaml b/tests/components/esp32_improv/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esp32_improv/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/esp32_improv/test.esp32-c3-ard.yaml b/tests/components/esp32_improv/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esp32_improv/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/esp32_rmt_led_strip/test.esp32-ard.yaml b/tests/components/esp32_rmt_led_strip/test.esp32-ard.yaml
deleted file mode 100644
index 0949b676d5..0000000000
--- a/tests/components/esp32_rmt_led_strip/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  pin1: GPIO13
-  pin2: GPIO14
-
-packages:
-  common: !include common.yaml
diff --git a/tests/components/esp32_rmt_led_strip/test.esp32-c3-ard.yaml b/tests/components/esp32_rmt_led_strip/test.esp32-c3-ard.yaml
deleted file mode 100644
index 6cc0667e77..0000000000
--- a/tests/components/esp32_rmt_led_strip/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  pin1: GPIO3
-  pin2: GPIO4
-
-packages:
-  common: !include common.yaml
diff --git a/tests/components/esp32_rmt_led_strip/test.esp32-s3-idf.yaml b/tests/components/esp32_rmt_led_strip/test.esp32-s3-idf.yaml
index ad273903b2..6bf0639a52 100644
--- a/tests/components/esp32_rmt_led_strip/test.esp32-s3-idf.yaml
+++ b/tests/components/esp32_rmt_led_strip/test.esp32-s3-idf.yaml
@@ -2,11 +2,22 @@ substitutions:
   pin1: GPIO3
   pin2: GPIO4
 
-packages:
-  common: !include common.yaml
-
+# WARNING: Using !extend or !remove prevents automatic component grouping in CI, making builds slower.
 light:
-  - id: !extend led_strip1
+  - platform: esp32_rmt_led_strip
+    id: led_strip1
+    pin: ${pin1}
+    num_leds: 60
+    rgb_order: GRB
+    chipset: ws2812
     use_dma: "true"
-  - id: !extend led_strip2
+  - platform: esp32_rmt_led_strip
+    id: led_strip2
+    pin: ${pin2}
+    num_leds: 60
+    rgb_order: RGB
+    bit0_high: 100us
+    bit0_low: 100us
+    bit1_high: 100us
+    bit1_low: 100us
     use_dma: "false"
diff --git a/tests/components/esp32_touch/common-get-value.yaml b/tests/components/esp32_touch/common-get-value.yaml
new file mode 100644
index 0000000000..4066303797
--- /dev/null
+++ b/tests/components/esp32_touch/common-get-value.yaml
@@ -0,0 +1,18 @@
+binary_sensor:
+  - platform: esp32_touch
+    name: ESP32 Touch Pad Get Value
+    pin: ${pin}
+    threshold: 1000
+    id: esp32_touch_pad_get_value
+    on_press:
+      then:
+        - lambda: |-
+            // Test that get_value() compiles and works
+            uint32_t value = id(esp32_touch_pad_get_value).get_value();
+            ESP_LOGD("test", "Touch value on press: %u", value);
+    on_release:
+      then:
+        - lambda: |-
+            // Test get_value() on release
+            uint32_t value = id(esp32_touch_pad_get_value).get_value();
+            ESP_LOGD("test", "Touch value on release: %u", value);
diff --git a/tests/components/esp32_touch/test.esp32-ard.yaml b/tests/components/esp32_touch/test.esp32-ard.yaml
deleted file mode 100644
index 25316b8646..0000000000
--- a/tests/components/esp32_touch/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO27
-
-<<: !include common.yaml
diff --git a/tests/components/esp32_touch/test.esp32-idf.yaml b/tests/components/esp32_touch/test.esp32-idf.yaml
index 25316b8646..5158613eb1 100644
--- a/tests/components/esp32_touch/test.esp32-idf.yaml
+++ b/tests/components/esp32_touch/test.esp32-idf.yaml
@@ -2,3 +2,4 @@ substitutions:
   pin: GPIO27
 
 <<: !include common.yaml
+<<: !include common-get-value.yaml
diff --git a/tests/components/esp32_touch/test.esp32-s2-ard.yaml b/tests/components/esp32_touch/test.esp32-s2-ard.yaml
deleted file mode 100644
index 575d758fae..0000000000
--- a/tests/components/esp32_touch/test.esp32-s2-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO12
-
-<<: !include common-variants.yaml
diff --git a/tests/components/esp32_touch/test.esp32-s2-idf.yaml b/tests/components/esp32_touch/test.esp32-s2-idf.yaml
index 575d758fae..b9f5671969 100644
--- a/tests/components/esp32_touch/test.esp32-s2-idf.yaml
+++ b/tests/components/esp32_touch/test.esp32-s2-idf.yaml
@@ -2,3 +2,4 @@ substitutions:
   pin: GPIO12
 
 <<: !include common-variants.yaml
+<<: !include common-get-value.yaml
diff --git a/tests/components/esp32_touch/test.esp32-s3-ard.yaml b/tests/components/esp32_touch/test.esp32-s3-ard.yaml
deleted file mode 100644
index 575d758fae..0000000000
--- a/tests/components/esp32_touch/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO12
-
-<<: !include common-variants.yaml
diff --git a/tests/components/esp32_touch/test.esp32-s3-idf.yaml b/tests/components/esp32_touch/test.esp32-s3-idf.yaml
index 575d758fae..b9f5671969 100644
--- a/tests/components/esp32_touch/test.esp32-s3-idf.yaml
+++ b/tests/components/esp32_touch/test.esp32-s3-idf.yaml
@@ -2,3 +2,4 @@ substitutions:
   pin: GPIO12
 
 <<: !include common-variants.yaml
+<<: !include common-get-value.yaml
diff --git a/tests/components/esphome/test.esp32-ard.yaml b/tests/components/esphome/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esphome/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/esphome/test.esp32-c3-ard.yaml b/tests/components/esphome/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/esphome/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/b_parasite/test.esp32-ard.yaml b/tests/components/esphome/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/b_parasite/test.esp32-ard.yaml
rename to tests/components/esphome/test.nrf52-adafruit.yaml
diff --git a/tests/components/b_parasite/test.esp32-c3-ard.yaml b/tests/components/esphome/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/b_parasite/test.esp32-c3-ard.yaml
rename to tests/components/esphome/test.nrf52-mcumgr.yaml
diff --git a/tests/components/espnow/common.yaml b/tests/components/espnow/common.yaml
new file mode 100644
index 0000000000..abb31c12b8
--- /dev/null
+++ b/tests/components/espnow/common.yaml
@@ -0,0 +1,52 @@
+espnow:
+  auto_add_peer: false
+  channel: 1
+  peers:
+    - 11:22:33:44:55:66
+  on_receive:
+    - logger.log:
+        format: "Received from: %s = '%s'  RSSI: %d"
+        args:
+          - format_mac_address_pretty(info.src_addr).c_str()
+          - format_hex_pretty(data, size).c_str()
+          - info.rx_ctrl->rssi
+    - espnow.send:
+        address: 11:22:33:44:55:66
+        data: "Hello from ESPHome"
+        on_sent:
+          - logger.log: "ESPNow message sent successfully"
+        on_error:
+          - logger.log: "ESPNow message failed to send"
+        wait_for_sent: true
+        continue_on_error: true
+
+    - espnow.send:
+        address: 11:22:33:44:55:66
+        data: [0x01, 0x02, 0x03, 0x04, 0x05]
+    - espnow.send:
+        address: 11:22:33:44:55:66
+        data: !lambda 'return {0x01, 0x02, 0x03, 0x04, 0x05};'
+    - espnow.broadcast:
+        data: "Hello, World!"
+    - espnow.broadcast:
+        data: [0x01, 0x02, 0x03, 0x04, 0x05]
+    - espnow.broadcast:
+        data: !lambda 'return {0x01, 0x02, 0x03, 0x04, 0x05};'
+    - espnow.peer.add:
+        address: 11:22:33:44:55:66
+    - espnow.peer.delete:
+        address: 11:22:33:44:55:66
+  on_broadcast:
+    - logger.log:
+        format: "Broadcast from: %s = '%s'  RSSI: %d"
+        args:
+          - format_mac_address_pretty(info.src_addr).c_str()
+          - format_hex_pretty(data, size).c_str()
+          - info.rx_ctrl->rssi
+  on_unknown_peer:
+    - logger.log:
+        format: "Unknown peer: %s = '%s'  RSSI: %d"
+        args:
+          - format_mac_address_pretty(info.src_addr).c_str()
+          - format_hex_pretty(data, size).c_str()
+          - info.rx_ctrl->rssi
diff --git a/tests/components/espnow/test.esp32-idf.yaml b/tests/components/espnow/test.esp32-idf.yaml
new file mode 100644
index 0000000000..b47e39c389
--- /dev/null
+++ b/tests/components/espnow/test.esp32-idf.yaml
@@ -0,0 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/ethernet/common-dm9051.yaml b/tests/components/ethernet/common-dm9051.yaml
index c878ca6e59..4526e7732d 100644
--- a/tests/components/ethernet/common-dm9051.yaml
+++ b/tests/components/ethernet/common-dm9051.yaml
@@ -12,3 +12,4 @@ ethernet:
     gateway: 192.168.178.1
     subnet: 255.255.255.0
   domain: .local
+  mac_address: "02:AA:BB:CC:DD:01"
diff --git a/tests/components/ethernet/common-dp83848.yaml b/tests/components/ethernet/common-dp83848.yaml
index 140c7d0d1b..7cedfeaf08 100644
--- a/tests/components/ethernet/common-dp83848.yaml
+++ b/tests/components/ethernet/common-dp83848.yaml
@@ -12,3 +12,4 @@ ethernet:
     gateway: 192.168.178.1
     subnet: 255.255.255.0
   domain: .local
+  mac_address: "02:AA:BB:CC:DD:01"
diff --git a/tests/components/ethernet/common-ip101.yaml b/tests/components/ethernet/common-ip101.yaml
index b5589220de..2dece15171 100644
--- a/tests/components/ethernet/common-ip101.yaml
+++ b/tests/components/ethernet/common-ip101.yaml
@@ -12,3 +12,4 @@ ethernet:
     gateway: 192.168.178.1
     subnet: 255.255.255.0
   domain: .local
+  mac_address: "02:AA:BB:CC:DD:01"
diff --git a/tests/components/ethernet/common-jl1101.yaml b/tests/components/ethernet/common-jl1101.yaml
index 2ada9495a0..b6ea884102 100644
--- a/tests/components/ethernet/common-jl1101.yaml
+++ b/tests/components/ethernet/common-jl1101.yaml
@@ -12,3 +12,4 @@ ethernet:
     gateway: 192.168.178.1
     subnet: 255.255.255.0
   domain: .local
+  mac_address: "02:AA:BB:CC:DD:01"
diff --git a/tests/components/ethernet/common-ksz8081.yaml b/tests/components/ethernet/common-ksz8081.yaml
index 7da8adb09a..f70d42319e 100644
--- a/tests/components/ethernet/common-ksz8081.yaml
+++ b/tests/components/ethernet/common-ksz8081.yaml
@@ -12,3 +12,4 @@ ethernet:
     gateway: 192.168.178.1
     subnet: 255.255.255.0
   domain: .local
+  mac_address: "02:AA:BB:CC:DD:01"
diff --git a/tests/components/ethernet/common-ksz8081rna.yaml b/tests/components/ethernet/common-ksz8081rna.yaml
index df04f06132..18efdae0e1 100644
--- a/tests/components/ethernet/common-ksz8081rna.yaml
+++ b/tests/components/ethernet/common-ksz8081rna.yaml
@@ -12,3 +12,4 @@ ethernet:
     gateway: 192.168.178.1
     subnet: 255.255.255.0
   domain: .local
+  mac_address: "02:AA:BB:CC:DD:01"
diff --git a/tests/components/ethernet/common-lan8670.yaml b/tests/components/ethernet/common-lan8670.yaml
new file mode 100644
index 0000000000..ec2f24273d
--- /dev/null
+++ b/tests/components/ethernet/common-lan8670.yaml
@@ -0,0 +1,14 @@
+ethernet:
+  type: LAN8670
+  mdc_pin: 23
+  mdio_pin: 25
+  clk:
+    pin: 0
+    mode: CLK_EXT_IN
+  phy_addr: 0
+  power_pin: 26
+  manual_ip:
+    static_ip: 192.168.178.56
+    gateway: 192.168.178.1
+    subnet: 255.255.255.0
+  domain: .local
diff --git a/tests/components/ethernet/common-lan8720.yaml b/tests/components/ethernet/common-lan8720.yaml
index f227752f42..204c1d9210 100644
--- a/tests/components/ethernet/common-lan8720.yaml
+++ b/tests/components/ethernet/common-lan8720.yaml
@@ -12,3 +12,4 @@ ethernet:
     gateway: 192.168.178.1
     subnet: 255.255.255.0
   domain: .local
+  mac_address: "02:AA:BB:CC:DD:01"
diff --git a/tests/components/ethernet/common-rtl8201.yaml b/tests/components/ethernet/common-rtl8201.yaml
index 7c9c9d913c..8b9f2b86f2 100644
--- a/tests/components/ethernet/common-rtl8201.yaml
+++ b/tests/components/ethernet/common-rtl8201.yaml
@@ -12,3 +12,4 @@ ethernet:
     gateway: 192.168.178.1
     subnet: 255.255.255.0
   domain: .local
+  mac_address: "02:AA:BB:CC:DD:01"
diff --git a/tests/components/ethernet/common-w5500.yaml b/tests/components/ethernet/common-w5500.yaml
index 76661a75c3..b3e96f000d 100644
--- a/tests/components/ethernet/common-w5500.yaml
+++ b/tests/components/ethernet/common-w5500.yaml
@@ -12,3 +12,4 @@ ethernet:
     gateway: 192.168.178.1
     subnet: 255.255.255.0
   domain: .local
+  mac_address: "02:AA:BB:CC:DD:01"
diff --git a/tests/components/ethernet/test-dm9051.esp32-ard.yaml b/tests/components/ethernet/test-dm9051.esp32-ard.yaml
deleted file mode 100644
index 23e3b97740..0000000000
--- a/tests/components/ethernet/test-dm9051.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-dm9051.yaml
diff --git a/tests/components/ethernet/test-dp83848.esp32-ard.yaml b/tests/components/ethernet/test-dp83848.esp32-ard.yaml
deleted file mode 100644
index 906bfba17c..0000000000
--- a/tests/components/ethernet/test-dp83848.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-dp83848.yaml
diff --git a/tests/components/ethernet/test-ip101.esp32-ard.yaml b/tests/components/ethernet/test-ip101.esp32-ard.yaml
deleted file mode 100644
index e52329d7ea..0000000000
--- a/tests/components/ethernet/test-ip101.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-ip101.yaml
diff --git a/tests/components/ethernet/test-ksz8081.esp32-ard.yaml b/tests/components/ethernet/test-ksz8081.esp32-ard.yaml
deleted file mode 100644
index 8f3c750c77..0000000000
--- a/tests/components/ethernet/test-ksz8081.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-ksz8081.yaml
diff --git a/tests/components/ethernet/test-ksz8081rna.esp32-ard.yaml b/tests/components/ethernet/test-ksz8081rna.esp32-ard.yaml
deleted file mode 100644
index a48e591996..0000000000
--- a/tests/components/ethernet/test-ksz8081rna.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-ksz8081rna.yaml
diff --git a/tests/components/ethernet/test-lan8670.esp32-idf.yaml b/tests/components/ethernet/test-lan8670.esp32-idf.yaml
new file mode 100644
index 0000000000..914a06ae88
--- /dev/null
+++ b/tests/components/ethernet/test-lan8670.esp32-idf.yaml
@@ -0,0 +1 @@
+<<: !include common-lan8670.yaml
diff --git a/tests/components/ethernet/test-lan8720.esp32-ard.yaml b/tests/components/ethernet/test-lan8720.esp32-ard.yaml
deleted file mode 100644
index 3df9ac874a..0000000000
--- a/tests/components/ethernet/test-lan8720.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-lan8720.yaml
diff --git a/tests/components/ethernet/test-rtl8201.esp32-ard.yaml b/tests/components/ethernet/test-rtl8201.esp32-ard.yaml
deleted file mode 100644
index e69f88dc94..0000000000
--- a/tests/components/ethernet/test-rtl8201.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-rtl8201.yaml
diff --git a/tests/components/ethernet/test-w5500.esp32-ard.yaml b/tests/components/ethernet/test-w5500.esp32-ard.yaml
deleted file mode 100644
index 36f1b5365f..0000000000
--- a/tests/components/ethernet/test-w5500.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-w5500.yaml
diff --git a/tests/components/ethernet_info/test.esp32-ard.yaml b/tests/components/ethernet_info/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/ethernet_info/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/ethernet_info/test.esp32-idf.yaml b/tests/components/ethernet_info/test.esp32-idf.yaml
index dade44d145..b47e39c389 100644
--- a/tests/components/ethernet_info/test.esp32-idf.yaml
+++ b/tests/components/ethernet_info/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/event/test.esp32-ard.yaml b/tests/components/event/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/event/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/event/test.esp32-c3-ard.yaml b/tests/components/event/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/event/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/bang_bang/test.esp32-ard.yaml b/tests/components/event/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/bang_bang/test.esp32-ard.yaml
rename to tests/components/event/test.nrf52-adafruit.yaml
diff --git a/tests/components/bang_bang/test.esp32-c3-ard.yaml b/tests/components/event/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/bang_bang/test.esp32-c3-ard.yaml
rename to tests/components/event/test.nrf52-mcumgr.yaml
diff --git a/tests/components/exposure_notifications/test.esp32-ard.yaml b/tests/components/exposure_notifications/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/exposure_notifications/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/exposure_notifications/test.esp32-c3-ard.yaml b/tests/components/exposure_notifications/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/exposure_notifications/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/exposure_notifications/test.esp32-c3-idf.yaml b/tests/components/exposure_notifications/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/exposure_notifications/test.esp32-c3-idf.yaml
+++ b/tests/components/exposure_notifications/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/exposure_notifications/test.esp32-idf.yaml b/tests/components/exposure_notifications/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/exposure_notifications/test.esp32-idf.yaml
+++ b/tests/components/exposure_notifications/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/external_components/test.esp32-ard.yaml b/tests/components/external_components/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/external_components/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/external_components/test.esp32-c3-ard.yaml b/tests/components/external_components/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/external_components/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/ezo/common.yaml b/tests/components/ezo/common.yaml
index edeebd1a12..79afbb8aa6 100644
--- a/tests/components/ezo/common.yaml
+++ b/tests/components/ezo/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_ezo
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ezo
+    i2c_id: i2c_bus
     id: ph_ezo
     address: 99
     unit_of_measurement: pH
diff --git a/tests/components/ezo/test.esp32-ard.yaml b/tests/components/ezo/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ezo/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ezo/test.esp32-c3-ard.yaml b/tests/components/ezo/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ezo/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ezo/test.esp32-c3-idf.yaml b/tests/components/ezo/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ezo/test.esp32-c3-idf.yaml
+++ b/tests/components/ezo/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ezo/test.esp32-idf.yaml b/tests/components/ezo/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ezo/test.esp32-idf.yaml
+++ b/tests/components/ezo/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ezo/test.esp8266-ard.yaml b/tests/components/ezo/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ezo/test.esp8266-ard.yaml
+++ b/tests/components/ezo/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ezo/test.rp2040-ard.yaml b/tests/components/ezo/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ezo/test.rp2040-ard.yaml
+++ b/tests/components/ezo/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ezo_pmp/common.yaml b/tests/components/ezo_pmp/common.yaml
index 00919797be..bd2fd9193c 100644
--- a/tests/components/ezo_pmp/common.yaml
+++ b/tests/components/ezo_pmp/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_ezo_pmp
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 ezo_pmp:
+  i2c_id: i2c_bus
   id: hcl_pump
   update_interval: 1s
 
diff --git a/tests/components/ezo_pmp/test.esp32-ard.yaml b/tests/components/ezo_pmp/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ezo_pmp/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ezo_pmp/test.esp32-c3-ard.yaml b/tests/components/ezo_pmp/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ezo_pmp/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ezo_pmp/test.esp32-c3-idf.yaml b/tests/components/ezo_pmp/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ezo_pmp/test.esp32-c3-idf.yaml
+++ b/tests/components/ezo_pmp/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ezo_pmp/test.esp32-idf.yaml b/tests/components/ezo_pmp/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ezo_pmp/test.esp32-idf.yaml
+++ b/tests/components/ezo_pmp/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ezo_pmp/test.esp8266-ard.yaml b/tests/components/ezo_pmp/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ezo_pmp/test.esp8266-ard.yaml
+++ b/tests/components/ezo_pmp/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ezo_pmp/test.rp2040-ard.yaml b/tests/components/ezo_pmp/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ezo_pmp/test.rp2040-ard.yaml
+++ b/tests/components/ezo_pmp/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/factory_reset/common.yaml b/tests/components/factory_reset/common.yaml
index ad3abd603e..7617dc4b81 100644
--- a/tests/components/factory_reset/common.yaml
+++ b/tests/components/factory_reset/common.yaml
@@ -1,3 +1,7 @@
 button:
   - platform: factory_reset
     name: Reset to Factory Default Settings
+
+factory_reset:
+  resets_required: 5
+  max_delay: 10s
diff --git a/tests/components/bedjet/test.esp32-ard.yaml b/tests/components/factory_reset/test.bk72xx-ard.yaml
similarity index 100%
rename from tests/components/bedjet/test.esp32-ard.yaml
rename to tests/components/factory_reset/test.bk72xx-ard.yaml
diff --git a/tests/components/factory_reset/test.esp32-ard.yaml b/tests/components/factory_reset/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/factory_reset/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/factory_reset/test.esp32-c3-ard.yaml b/tests/components/factory_reset/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/factory_reset/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/factory_reset/test.esp8266-ard.yaml b/tests/components/factory_reset/test.esp8266-ard.yaml
index dade44d145..591a319b81 100644
--- a/tests/components/factory_reset/test.esp8266-ard.yaml
+++ b/tests/components/factory_reset/test.esp8266-ard.yaml
@@ -1 +1,4 @@
+esp8266:
+  restore_from_flash: true
+
 <<: !include common.yaml
diff --git a/tests/components/factory_reset/test.rp2040-ard.yaml b/tests/components/factory_reset/test.rp2040-ard.yaml
index dade44d145..ad3abd603e 100644
--- a/tests/components/factory_reset/test.rp2040-ard.yaml
+++ b/tests/components/factory_reset/test.rp2040-ard.yaml
@@ -1 +1,3 @@
-<<: !include common.yaml
+button:
+  - platform: factory_reset
+    name: Reset to Factory Default Settings
diff --git a/tests/components/feedback/test.esp32-ard.yaml b/tests/components/feedback/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/feedback/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/feedback/test.esp32-c3-ard.yaml b/tests/components/feedback/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/feedback/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/fingerprint_grow/common.yaml b/tests/components/fingerprint_grow/common.yaml
index e759ea5a02..2d2fc61437 100644
--- a/tests/components/fingerprint_grow/common.yaml
+++ b/tests/components/fingerprint_grow/common.yaml
@@ -9,12 +9,6 @@ esphome:
           finger_id: 2
       - fingerprint_grow.delete_all:
 
-uart:
-  - id: uart_fingerprint_grow
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 57600
-
 fingerprint_grow:
   sensing_pin: ${sensing_pin}
   password: 0x12FE37DC
diff --git a/tests/components/fingerprint_grow/test.esp32-ard.yaml b/tests/components/fingerprint_grow/test.esp32-ard.yaml
deleted file mode 100644
index 4aef3d8be6..0000000000
--- a/tests/components/fingerprint_grow/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-  sensing_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/fingerprint_grow/test.esp32-c3-ard.yaml b/tests/components/fingerprint_grow/test.esp32-c3-ard.yaml
deleted file mode 100644
index faab50e152..0000000000
--- a/tests/components/fingerprint_grow/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  sensing_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/fingerprint_grow/test.esp32-c3-idf.yaml b/tests/components/fingerprint_grow/test.esp32-c3-idf.yaml
index faab50e152..dff4f7fd92 100644
--- a/tests/components/fingerprint_grow/test.esp32-c3-idf.yaml
+++ b/tests/components/fingerprint_grow/test.esp32-c3-idf.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
   sensing_pin: GPIO6
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/fingerprint_grow/test.esp32-idf.yaml b/tests/components/fingerprint_grow/test.esp32-idf.yaml
index 4aef3d8be6..7149f07d16 100644
--- a/tests/components/fingerprint_grow/test.esp32-idf.yaml
+++ b/tests/components/fingerprint_grow/test.esp32-idf.yaml
@@ -3,4 +3,7 @@ substitutions:
   rx_pin: GPIO14
   sensing_pin: GPIO15
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/fingerprint_grow/test.esp8266-ard.yaml b/tests/components/fingerprint_grow/test.esp8266-ard.yaml
index f2a864596a..204fdf0302 100644
--- a/tests/components/fingerprint_grow/test.esp8266-ard.yaml
+++ b/tests/components/fingerprint_grow/test.esp8266-ard.yaml
@@ -1,6 +1,9 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
   sensing_pin: GPIO15
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/fingerprint_grow/test.rp2040-ard.yaml b/tests/components/fingerprint_grow/test.rp2040-ard.yaml
index faab50e152..80ea1d22ca 100644
--- a/tests/components/fingerprint_grow/test.rp2040-ard.yaml
+++ b/tests/components/fingerprint_grow/test.rp2040-ard.yaml
@@ -3,4 +3,7 @@ substitutions:
   rx_pin: GPIO5
   sensing_pin: GPIO6
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/font/common.yaml b/tests/components/font/common.yaml
index fb50fc3336..6ba52e3d97 100644
--- a/tests/components/font/common.yaml
+++ b/tests/components/font/common.yaml
@@ -47,12 +47,9 @@ font:
     id: bdf_font
     size: 7
 
-i2c:
-  scl: ${i2c_scl}
-  sda: ${i2c_sda}
-
 display:
   - platform: ssd1306_i2c
+    i2c_id: i2c_bus
     id: ssd1306_display
     model: SSD1306_128X64
     reset_pin: ${display_reset_pin}
diff --git a/tests/components/font/test.esp32-ard.yaml b/tests/components/font/test.esp32-ard.yaml
deleted file mode 100644
index d98600a51b..0000000000
--- a/tests/components/font/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  i2c_scl: GPIO16
-  i2c_sda: GPIO17
-  display_reset_pin: GPIO13
-
-packages:
-  common: !include common.yaml
diff --git a/tests/components/font/test.esp32-c3-ard.yaml b/tests/components/font/test.esp32-c3-ard.yaml
deleted file mode 100644
index ad14a2e9a6..0000000000
--- a/tests/components/font/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  i2c_scl: GPIO5
-  i2c_sda: GPIO4
-  display_reset_pin: GPIO3
-
-packages:
-  common: !include common.yaml
diff --git a/tests/components/font/test.esp32-c3-idf.yaml b/tests/components/font/test.esp32-c3-idf.yaml
index ad14a2e9a6..2090db7e6d 100644
--- a/tests/components/font/test.esp32-c3-idf.yaml
+++ b/tests/components/font/test.esp32-c3-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  i2c_scl: GPIO5
-  i2c_sda: GPIO4
   display_reset_pin: GPIO3
 
 packages:
-  common: !include common.yaml
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/font/test.esp32-idf.yaml b/tests/components/font/test.esp32-idf.yaml
index d98600a51b..a6a94a39da 100644
--- a/tests/components/font/test.esp32-idf.yaml
+++ b/tests/components/font/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  i2c_scl: GPIO16
-  i2c_sda: GPIO17
   display_reset_pin: GPIO13
 
 packages:
-  common: !include common.yaml
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/font/test.esp8266-ard.yaml b/tests/components/font/test.esp8266-ard.yaml
index ad14a2e9a6..173f7dfaa5 100644
--- a/tests/components/font/test.esp8266-ard.yaml
+++ b/tests/components/font/test.esp8266-ard.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  i2c_scl: GPIO5
-  i2c_sda: GPIO4
   display_reset_pin: GPIO3
 
 packages:
-  common: !include common.yaml
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/font/test.rp2040-ard.yaml b/tests/components/font/test.rp2040-ard.yaml
index ad14a2e9a6..aeecf352b4 100644
--- a/tests/components/font/test.rp2040-ard.yaml
+++ b/tests/components/font/test.rp2040-ard.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  i2c_scl: GPIO5
-  i2c_sda: GPIO4
   display_reset_pin: GPIO3
 
 packages:
-  common: !include common.yaml
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/fs3000/common.yaml b/tests/components/fs3000/common.yaml
index e87ac28aa9..e8ec4d0773 100644
--- a/tests/components/fs3000/common.yaml
+++ b/tests/components/fs3000/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_fs3000
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: fs3000
+    i2c_id: i2c_bus
     name: Air Velocity
     model: 1005
     update_interval: 60s
diff --git a/tests/components/fs3000/test.esp32-ard.yaml b/tests/components/fs3000/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/fs3000/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/fs3000/test.esp32-c3-ard.yaml b/tests/components/fs3000/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/fs3000/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/fs3000/test.esp32-c3-idf.yaml b/tests/components/fs3000/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/fs3000/test.esp32-c3-idf.yaml
+++ b/tests/components/fs3000/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/fs3000/test.esp32-idf.yaml b/tests/components/fs3000/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/fs3000/test.esp32-idf.yaml
+++ b/tests/components/fs3000/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/fs3000/test.esp8266-ard.yaml b/tests/components/fs3000/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/fs3000/test.esp8266-ard.yaml
+++ b/tests/components/fs3000/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/fs3000/test.rp2040-ard.yaml b/tests/components/fs3000/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/fs3000/test.rp2040-ard.yaml
+++ b/tests/components/fs3000/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ft5x06/common.yaml b/tests/components/ft5x06/common.yaml
index 322ee88b6d..b1f7ca4cc9 100644
--- a/tests/components/ft5x06/common.yaml
+++ b/tests/components/ft5x06/common.yaml
@@ -1,20 +1,19 @@
-i2c:
-  - id: i2c_ft5x06
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 display:
   - platform: ssd1306_i2c
-    id: ssd1306_display
+    i2c_id: i2c_bus
+    id: ft5x06_display
     model: SSD1306_128X64
     reset_pin: ${reset_pin}
     pages:
-      - id: page1
+      - id: ft5x06_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
 
 touchscreen:
   - platform: ft5x06
+    i2c_id: i2c_bus
+    display: ft5x06_display
+    id: ft5x06_touchscreen
     on_touch:
       - logger.log:
           format: Touch at (%d, %d)
diff --git a/tests/components/ft5x06/test.esp32-ard.yaml b/tests/components/ft5x06/test.esp32-ard.yaml
deleted file mode 100644
index 1ca773e06c..0000000000
--- a/tests/components/ft5x06/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  reset_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/ft5x06/test.esp32-c3-ard.yaml b/tests/components/ft5x06/test.esp32-c3-ard.yaml
deleted file mode 100644
index 1e6670c196..0000000000
--- a/tests/components/ft5x06/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  reset_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/ft5x06/test.esp32-c3-idf.yaml b/tests/components/ft5x06/test.esp32-c3-idf.yaml
index 1e6670c196..c97f30d52c 100644
--- a/tests/components/ft5x06/test.esp32-c3-idf.yaml
+++ b/tests/components/ft5x06/test.esp32-c3-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ft5x06/test.esp32-idf.yaml b/tests/components/ft5x06/test.esp32-idf.yaml
index 1ca773e06c..4ff2241ec9 100644
--- a/tests/components/ft5x06/test.esp32-idf.yaml
+++ b/tests/components/ft5x06/test.esp32-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   reset_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ft5x06/test.esp8266-ard.yaml b/tests/components/ft5x06/test.esp8266-ard.yaml
index dfdc12a3d1..b8bb94edde 100644
--- a/tests/components/ft5x06/test.esp8266-ard.yaml
+++ b/tests/components/ft5x06/test.esp8266-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ft5x06/test.rp2040-ard.yaml b/tests/components/ft5x06/test.rp2040-ard.yaml
index 1e6670c196..1bf10642c5 100644
--- a/tests/components/ft5x06/test.rp2040-ard.yaml
+++ b/tests/components/ft5x06/test.rp2040-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ft63x6/common.yaml b/tests/components/ft63x6/common.yaml
index 1fae6da5f4..ea5a60a2f6 100644
--- a/tests/components/ft63x6/common.yaml
+++ b/tests/components/ft63x6/common.yaml
@@ -1,25 +1,19 @@
-spi:
-  - id: spi_ft63x6
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
-i2c:
-  - id: i2c_ft63x6
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 display:
   - platform: ssd1306_i2c
-    id: ssd1306_display
+    i2c_id: i2c_bus
+    id: ft63x6_display
     model: SSD1306_128X64
     reset_pin: ${reset_pin}
     pages:
-      - id: page1
+      - id: ft63x6_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
 
 touchscreen:
   - platform: ft63x6
+    i2c_id: i2c_bus
+    display: ft63x6_display
+    id: ft63x6_touchscreen
     interrupt_pin: ${interrupt_pin}
     transform:
       swap_xy: true
@@ -42,6 +36,7 @@ touchscreen:
 
 binary_sensor:
   - platform: touchscreen
+    touchscreen_id: ft63x6_touchscreen
     name: Bottom Left Touch
     use_raw: true
     x_min: 0
diff --git a/tests/components/ft63x6/test.esp32-ard.yaml b/tests/components/ft63x6/test.esp32-ard.yaml
deleted file mode 100644
index 47b5796e8b..0000000000
--- a/tests/components/ft63x6/test.esp32-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO0
-  mosi_pin: GPIO2
-  scl_pin: GPIO13
-  sda_pin: GPIO14
-  interrupt_pin: GPIO15
-  reset_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/ft63x6/test.esp32-c3-ard.yaml b/tests/components/ft63x6/test.esp32-c3-ard.yaml
deleted file mode 100644
index 397ac1e464..0000000000
--- a/tests/components/ft63x6/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  scl_pin: GPIO0
-  sda_pin: GPIO1
-  interrupt_pin: GPIO2
-  reset_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/ft63x6/test.esp32-c3-idf.yaml b/tests/components/ft63x6/test.esp32-c3-idf.yaml
index 397ac1e464..febf38d8e1 100644
--- a/tests/components/ft63x6/test.esp32-c3-idf.yaml
+++ b/tests/components/ft63x6/test.esp32-c3-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  scl_pin: GPIO0
-  sda_pin: GPIO1
   interrupt_pin: GPIO2
   reset_pin: GPIO3
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ft63x6/test.esp32-idf.yaml b/tests/components/ft63x6/test.esp32-idf.yaml
index 47b5796e8b..590f6a919c 100644
--- a/tests/components/ft63x6/test.esp32-idf.yaml
+++ b/tests/components/ft63x6/test.esp32-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO0
-  mosi_pin: GPIO2
-  scl_pin: GPIO13
-  sda_pin: GPIO14
   interrupt_pin: GPIO15
-  reset_pin: GPIO16
+  reset_pin: GPIO4
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ft63x6/test.esp8266-ard.yaml b/tests/components/ft63x6/test.esp8266-ard.yaml
index a4223733af..3ac5c645e3 100644
--- a/tests/components/ft63x6/test.esp8266-ard.yaml
+++ b/tests/components/ft63x6/test.esp8266-ard.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  interrupt_pin: GPIO12
+  interrupt_pin: GPIO0
   reset_pin: GPIO16
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ft63x6/test.rp2040-ard.yaml b/tests/components/ft63x6/test.rp2040-ard.yaml
index 397ac1e464..2dd70ff33a 100644
--- a/tests/components/ft63x6/test.rp2040-ard.yaml
+++ b/tests/components/ft63x6/test.rp2040-ard.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  scl_pin: GPIO0
-  sda_pin: GPIO1
   interrupt_pin: GPIO2
   reset_pin: GPIO3
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/fujitsu_general/common.yaml b/tests/components/fujitsu_general/common.yaml
index 3359b89f2a..51bd1441c0 100644
--- a/tests/components/fujitsu_general/common.yaml
+++ b/tests/components/fujitsu_general/common.yaml
@@ -1,7 +1,4 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: fujitsu_general
     name: Fujitsu General Climate
+    transmitter_id: xmitr
diff --git a/tests/components/fujitsu_general/test.esp32-ard.yaml b/tests/components/fujitsu_general/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/fujitsu_general/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/fujitsu_general/test.esp32-c3-ard.yaml b/tests/components/fujitsu_general/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/fujitsu_general/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/fujitsu_general/test.esp32-c3-idf.yaml b/tests/components/fujitsu_general/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/fujitsu_general/test.esp32-c3-idf.yaml
+++ b/tests/components/fujitsu_general/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/fujitsu_general/test.esp32-idf.yaml b/tests/components/fujitsu_general/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/fujitsu_general/test.esp32-idf.yaml
+++ b/tests/components/fujitsu_general/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/fujitsu_general/test.esp8266-ard.yaml b/tests/components/fujitsu_general/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/fujitsu_general/test.esp8266-ard.yaml
+++ b/tests/components/fujitsu_general/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gcja5/common.yaml b/tests/components/gcja5/common.yaml
index 8f6250045d..64374c0e50 100644
--- a/tests/components/gcja5/common.yaml
+++ b/tests/components/gcja5/common.yaml
@@ -1,12 +1,6 @@
-uart:
-  - id: uart_gcja5
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-    parity: EVEN
-
 sensor:
   - platform: gcja5
+    uart_id: uart_bus
     pm_1_0:
       name: "Particulate Matter <1.0µm Concentration"
     pm_2_5:
diff --git a/tests/components/gcja5/test.esp32-ard.yaml b/tests/components/gcja5/test.esp32-ard.yaml
deleted file mode 100644
index 811f6b72a6..0000000000
--- a/tests/components/gcja5/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/gcja5/test.esp32-c3-ard.yaml b/tests/components/gcja5/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/gcja5/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/gcja5/test.esp32-c3-idf.yaml b/tests/components/gcja5/test.esp32-c3-idf.yaml
index b516342f3b..236529042f 100644
--- a/tests/components/gcja5/test.esp32-c3-idf.yaml
+++ b/tests/components/gcja5/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart_9600_even: !include ../../test_build_components/common/uart_9600_even/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gcja5/test.esp32-idf.yaml b/tests/components/gcja5/test.esp32-idf.yaml
index 811f6b72a6..5ce1861902 100644
--- a/tests/components/gcja5/test.esp32-idf.yaml
+++ b/tests/components/gcja5/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
+packages:
+  uart_9600_even: !include ../../test_build_components/common/uart_9600_even/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gcja5/test.esp8266-ard.yaml b/tests/components/gcja5/test.esp8266-ard.yaml
index b516342f3b..a3f8cf43d4 100644
--- a/tests/components/gcja5/test.esp8266-ard.yaml
+++ b/tests/components/gcja5/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart_9600_even: !include ../../test_build_components/common/uart_9600_even/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gcja5/test.rp2040-ard.yaml b/tests/components/gcja5/test.rp2040-ard.yaml
index b516342f3b..7c1f4f41e2 100644
--- a/tests/components/gcja5/test.rp2040-ard.yaml
+++ b/tests/components/gcja5/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart_9600_even: !include ../../test_build_components/common/uart_9600_even/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gdk101/common.yaml b/tests/components/gdk101/common.yaml
index 7805ad43f0..4eb5586ade 100644
--- a/tests/components/gdk101/common.yaml
+++ b/tests/components/gdk101/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_gdk101
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 gdk101:
+  i2c_id: i2c_bus
   id: my_gdk101
 
 sensor:
diff --git a/tests/components/gdk101/test.esp32-ard.yaml b/tests/components/gdk101/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/gdk101/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/gdk101/test.esp32-idf.yaml b/tests/components/gdk101/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/gdk101/test.esp32-idf.yaml
+++ b/tests/components/gdk101/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gdk101/test.esp8266-ard.yaml b/tests/components/gdk101/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/gdk101/test.esp8266-ard.yaml
+++ b/tests/components/gdk101/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gdk101/test.rp2040-ard.yaml b/tests/components/gdk101/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/gdk101/test.rp2040-ard.yaml
+++ b/tests/components/gdk101/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gl_r01_i2c/common.yaml b/tests/components/gl_r01_i2c/common.yaml
index fe0705bdc6..272a94d349 100644
--- a/tests/components/gl_r01_i2c/common.yaml
+++ b/tests/components/gl_r01_i2c/common.yaml
@@ -1,12 +1,7 @@
-i2c:
-  - id: i2c_gl_r01_i2c
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: gl_r01_i2c
+    i2c_id: i2c_bus
     id: tof
     name: "ToF sensor"
-    i2c_id: i2c_gl_r01_i2c
     address: 0x74
     update_interval: 15s
diff --git a/tests/components/gl_r01_i2c/test.esp32-ard.yaml b/tests/components/gl_r01_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/gl_r01_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/gl_r01_i2c/test.esp32-c3-ard.yaml b/tests/components/gl_r01_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/gl_r01_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/gl_r01_i2c/test.esp32-c3-idf.yaml b/tests/components/gl_r01_i2c/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/gl_r01_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/gl_r01_i2c/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gl_r01_i2c/test.esp32-idf.yaml b/tests/components/gl_r01_i2c/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/gl_r01_i2c/test.esp32-idf.yaml
+++ b/tests/components/gl_r01_i2c/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gl_r01_i2c/test.esp8266-ard.yaml b/tests/components/gl_r01_i2c/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/gl_r01_i2c/test.esp8266-ard.yaml
+++ b/tests/components/gl_r01_i2c/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gl_r01_i2c/test.rp2040-ard.yaml b/tests/components/gl_r01_i2c/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/gl_r01_i2c/test.rp2040-ard.yaml
+++ b/tests/components/gl_r01_i2c/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/globals/test.esp32-ard.yaml b/tests/components/globals/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/globals/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/globals/test.esp32-c3-ard.yaml b/tests/components/globals/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/globals/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/gp2y1010au0f/test.esp32-ard.yaml b/tests/components/gp2y1010au0f/test.esp32-ard.yaml
deleted file mode 100644
index d9494a95b7..0000000000
--- a/tests/components/gp2y1010au0f/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  adc_pin: A0
-  output_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/gp2y1010au0f/test.esp32-c3-ard.yaml b/tests/components/gp2y1010au0f/test.esp32-c3-ard.yaml
deleted file mode 100644
index 0e331c893c..0000000000
--- a/tests/components/gp2y1010au0f/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  adc_pin: GPIO0
-  output_pin: GPIO1
-
-<<: !include common.yaml
diff --git a/tests/components/gp2y1010au0f/test.esp8266-ard.yaml b/tests/components/gp2y1010au0f/test.esp8266-ard.yaml
index a61053426a..c6a7c7bf13 100644
--- a/tests/components/gp2y1010au0f/test.esp8266-ard.yaml
+++ b/tests/components/gp2y1010au0f/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
   adc_pin: A0
-  output_pin: GPIO5
+  output_pin: GPIO0
 
 <<: !include common.yaml
diff --git a/tests/components/gp8403/common.yaml b/tests/components/gp8403/common.yaml
index 7074185273..b04fbc4fbc 100644
--- a/tests/components/gp8403/common.yaml
+++ b/tests/components/gp8403/common.yaml
@@ -1,12 +1,9 @@
-i2c:
-  - id: i2c_gp8403
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 gp8403:
   - id: gp8403_5v
+    i2c_id: i2c_bus
     voltage: 5V
   - id: gp8403_10v
+    i2c_id: i2c_bus
     voltage: 10V
 
 output:
diff --git a/tests/components/gp8403/test.esp32-ard.yaml b/tests/components/gp8403/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/gp8403/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/gp8403/test.esp32-c3-ard.yaml b/tests/components/gp8403/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/gp8403/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/gp8403/test.esp32-c3-idf.yaml b/tests/components/gp8403/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/gp8403/test.esp32-c3-idf.yaml
+++ b/tests/components/gp8403/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gp8403/test.esp32-idf.yaml b/tests/components/gp8403/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/gp8403/test.esp32-idf.yaml
+++ b/tests/components/gp8403/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gp8403/test.esp8266-ard.yaml b/tests/components/gp8403/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/gp8403/test.esp8266-ard.yaml
+++ b/tests/components/gp8403/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gp8403/test.rp2040-ard.yaml b/tests/components/gp8403/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/gp8403/test.rp2040-ard.yaml
+++ b/tests/components/gp8403/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gpio/test.esp32-ard.yaml b/tests/components/gpio/test.esp32-ard.yaml
deleted file mode 100644
index 09f41abb79..0000000000
--- a/tests/components/gpio/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  binary_sensor_pin: GPIO12
-  output_pin: GPIO13
-  switch_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/gpio/test.esp32-c3-ard.yaml b/tests/components/gpio/test.esp32-c3-ard.yaml
deleted file mode 100644
index fc7c9942d0..0000000000
--- a/tests/components/gpio/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  binary_sensor_pin: GPIO2
-  output_pin: GPIO3
-  switch_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/gpio/test.esp8266-ard.yaml b/tests/components/gpio/test.esp8266-ard.yaml
index 09f41abb79..e1660ec47c 100644
--- a/tests/components/gpio/test.esp8266-ard.yaml
+++ b/tests/components/gpio/test.esp8266-ard.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  binary_sensor_pin: GPIO12
-  output_pin: GPIO13
-  switch_pin: GPIO14
+  binary_sensor_pin: GPIO0
+  output_pin: GPIO2
+  switch_pin: GPIO15
 
 <<: !include common.yaml
diff --git a/tests/components/gpio/test.nrf52-adafruit.yaml b/tests/components/gpio/test.nrf52-adafruit.yaml
new file mode 100644
index 0000000000..912b9537c4
--- /dev/null
+++ b/tests/components/gpio/test.nrf52-adafruit.yaml
@@ -0,0 +1,14 @@
+binary_sensor:
+  - platform: gpio
+    pin: 2
+    id: gpio_binary_sensor
+
+output:
+  - platform: gpio
+    pin: P0.3
+    id: gpio_output
+
+switch:
+  - platform: gpio
+    pin: P1.2
+    id: gpio_switch
diff --git a/tests/components/gpio/test.nrf52-mcumgr.yaml b/tests/components/gpio/test.nrf52-mcumgr.yaml
new file mode 100644
index 0000000000..912b9537c4
--- /dev/null
+++ b/tests/components/gpio/test.nrf52-mcumgr.yaml
@@ -0,0 +1,14 @@
+binary_sensor:
+  - platform: gpio
+    pin: 2
+    id: gpio_binary_sensor
+
+output:
+  - platform: gpio
+    pin: P0.3
+    id: gpio_output
+
+switch:
+  - platform: gpio
+    pin: P1.2
+    id: gpio_switch
diff --git a/tests/components/gps/common.yaml b/tests/components/gps/common.yaml
index 53dc67e457..a99e3ef7e0 100644
--- a/tests/components/gps/common.yaml
+++ b/tests/components/gps/common.yaml
@@ -1,10 +1,3 @@
-uart:
-  - id: uart_gps
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-    parity: EVEN
-
 gps:
   latitude:
     name: "Latitude"
diff --git a/tests/components/gps/test.esp32-ard.yaml b/tests/components/gps/test.esp32-ard.yaml
deleted file mode 100644
index 811f6b72a6..0000000000
--- a/tests/components/gps/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/gps/test.esp32-c3-ard.yaml b/tests/components/gps/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/gps/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/gps/test.esp32-c3-idf.yaml b/tests/components/gps/test.esp32-c3-idf.yaml
new file mode 100644
index 0000000000..4b7c8351a7
--- /dev/null
+++ b/tests/components/gps/test.esp32-c3-idf.yaml
@@ -0,0 +1,5 @@
+substitutions:
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/gps/test.esp32-idf.yaml b/tests/components/gps/test.esp32-idf.yaml
new file mode 100644
index 0000000000..64baa4ec9d
--- /dev/null
+++ b/tests/components/gps/test.esp32-idf.yaml
@@ -0,0 +1,8 @@
+substitutions:
+  tx_pin: GPIO12
+  rx_pin: GPIO14
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/gps/test.esp8266-ard.yaml b/tests/components/gps/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/gps/test.esp8266-ard.yaml
+++ b/tests/components/gps/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gps/test.rp2040-ard.yaml b/tests/components/gps/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/gps/test.rp2040-ard.yaml
+++ b/tests/components/gps/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/graph/common.yaml b/tests/components/graph/common.yaml
index d2a6a9422d..11e2a16ca1 100644
--- a/tests/components/graph/common.yaml
+++ b/tests/components/graph/common.yaml
@@ -1,8 +1,3 @@
-i2c:
-  - id: i2c_graph
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: template
     id: some_sensor
@@ -16,10 +11,11 @@ graph:
 
 display:
   - platform: ssd1306_i2c
+    i2c_id: i2c_bus
     id: ssd1306_display
     model: SSD1306_128X64
     reset_pin: ${reset_pin}
     pages:
-      - id: page1
+      - id: graph_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
diff --git a/tests/components/graph/test.esp32-ard.yaml b/tests/components/graph/test.esp32-ard.yaml
deleted file mode 100644
index 1ca773e06c..0000000000
--- a/tests/components/graph/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  reset_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/graph/test.esp32-c3-ard.yaml b/tests/components/graph/test.esp32-c3-ard.yaml
deleted file mode 100644
index 1e6670c196..0000000000
--- a/tests/components/graph/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  reset_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/graph/test.esp32-c3-idf.yaml b/tests/components/graph/test.esp32-c3-idf.yaml
index 1e6670c196..c97f30d52c 100644
--- a/tests/components/graph/test.esp32-c3-idf.yaml
+++ b/tests/components/graph/test.esp32-c3-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/graph/test.esp32-idf.yaml b/tests/components/graph/test.esp32-idf.yaml
index 1ca773e06c..4ff2241ec9 100644
--- a/tests/components/graph/test.esp32-idf.yaml
+++ b/tests/components/graph/test.esp32-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   reset_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/graph/test.esp8266-ard.yaml b/tests/components/graph/test.esp8266-ard.yaml
index dfdc12a3d1..b8bb94edde 100644
--- a/tests/components/graph/test.esp8266-ard.yaml
+++ b/tests/components/graph/test.esp8266-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/graph/test.rp2040-ard.yaml b/tests/components/graph/test.rp2040-ard.yaml
index 1e6670c196..1bf10642c5 100644
--- a/tests/components/graph/test.rp2040-ard.yaml
+++ b/tests/components/graph/test.rp2040-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/graphical_display_menu/common.yaml b/tests/components/graphical_display_menu/common.yaml
index d979a6a30e..6cee2af232 100644
--- a/tests/components/graphical_display_menu/common.yaml
+++ b/tests/components/graphical_display_menu/common.yaml
@@ -1,15 +1,10 @@
-i2c:
-  - id: i2c_graphical_display_menu
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 display:
   - platform: ssd1306_i2c
-    id: ssd1306_display
+    id: ssd1306_i2c_display
     model: SSD1306_128X64
     reset_pin: ${reset_pin}
     pages:
-      - id: page1
+      - id: graphical_display_menu_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
 
@@ -41,7 +36,7 @@ switch:
 
 graphical_display_menu:
   id: test_graphical_display_menu
-  display: ssd1306_display
+  display: ssd1306_i2c_display
   font: roboto
   active: false
   mode: rotary
diff --git a/tests/components/graphical_display_menu/test.esp32-ard.yaml b/tests/components/graphical_display_menu/test.esp32-ard.yaml
deleted file mode 100644
index 1ca773e06c..0000000000
--- a/tests/components/graphical_display_menu/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  reset_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/graphical_display_menu/test.esp32-c3-ard.yaml b/tests/components/graphical_display_menu/test.esp32-c3-ard.yaml
deleted file mode 100644
index 1e6670c196..0000000000
--- a/tests/components/graphical_display_menu/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  reset_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/graphical_display_menu/test.esp32-c3-idf.yaml b/tests/components/graphical_display_menu/test.esp32-c3-idf.yaml
index 1e6670c196..c97f30d52c 100644
--- a/tests/components/graphical_display_menu/test.esp32-c3-idf.yaml
+++ b/tests/components/graphical_display_menu/test.esp32-c3-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/graphical_display_menu/test.esp32-idf.yaml b/tests/components/graphical_display_menu/test.esp32-idf.yaml
index 1ca773e06c..4ff2241ec9 100644
--- a/tests/components/graphical_display_menu/test.esp32-idf.yaml
+++ b/tests/components/graphical_display_menu/test.esp32-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   reset_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/graphical_display_menu/test.esp8266-ard.yaml b/tests/components/graphical_display_menu/test.esp8266-ard.yaml
index dfdc12a3d1..b8bb94edde 100644
--- a/tests/components/graphical_display_menu/test.esp8266-ard.yaml
+++ b/tests/components/graphical_display_menu/test.esp8266-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/graphical_display_menu/test.rp2040-ard.yaml b/tests/components/graphical_display_menu/test.rp2040-ard.yaml
index 1e6670c196..1bf10642c5 100644
--- a/tests/components/graphical_display_menu/test.rp2040-ard.yaml
+++ b/tests/components/graphical_display_menu/test.rp2040-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO6
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/gree/common.yaml b/tests/components/gree/common.yaml
index c221184bbf..e706076034 100644
--- a/tests/components/gree/common.yaml
+++ b/tests/components/gree/common.yaml
@@ -1,8 +1,5 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: gree
     name: GREE
     model: generic
+    transmitter_id: xmitr
diff --git a/tests/components/gree/test.esp32-ard.yaml b/tests/components/gree/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/gree/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/gree/test.esp32-c3-ard.yaml b/tests/components/gree/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/gree/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/gree/test.esp32-c3-idf.yaml b/tests/components/gree/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/gree/test.esp32-c3-idf.yaml
+++ b/tests/components/gree/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gree/test.esp32-idf.yaml b/tests/components/gree/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/gree/test.esp32-idf.yaml
+++ b/tests/components/gree/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/gree/test.esp8266-ard.yaml b/tests/components/gree/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/gree/test.esp8266-ard.yaml
+++ b/tests/components/gree/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/grove_gas_mc_v2/common.yaml b/tests/components/grove_gas_mc_v2/common.yaml
index dbdb950033..0729e6b9c7 100644
--- a/tests/components/grove_gas_mc_v2/common.yaml
+++ b/tests/components/grove_gas_mc_v2/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_grove_gas_mc_v2
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: grove_gas_mc_v2
+    i2c_id: i2c_bus
     nitrogen_dioxide:
       name: "Nitrogen Dioxide"
     ethanol:
diff --git a/tests/components/grove_gas_mc_v2/test.esp32-ard.yaml b/tests/components/grove_gas_mc_v2/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/grove_gas_mc_v2/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/grove_gas_mc_v2/test.esp32-c3-ard.yaml b/tests/components/grove_gas_mc_v2/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/grove_gas_mc_v2/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/grove_gas_mc_v2/test.esp32-c3-idf.yaml b/tests/components/grove_gas_mc_v2/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/grove_gas_mc_v2/test.esp32-c3-idf.yaml
+++ b/tests/components/grove_gas_mc_v2/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/grove_gas_mc_v2/test.esp32-idf.yaml b/tests/components/grove_gas_mc_v2/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/grove_gas_mc_v2/test.esp32-idf.yaml
+++ b/tests/components/grove_gas_mc_v2/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/grove_gas_mc_v2/test.esp8266-ard.yaml b/tests/components/grove_gas_mc_v2/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/grove_gas_mc_v2/test.esp8266-ard.yaml
+++ b/tests/components/grove_gas_mc_v2/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/grove_gas_mc_v2/test.rp2040-ard.yaml b/tests/components/grove_gas_mc_v2/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/grove_gas_mc_v2/test.rp2040-ard.yaml
+++ b/tests/components/grove_gas_mc_v2/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/grove_tb6612fng/common.yaml b/tests/components/grove_tb6612fng/common.yaml
index 0db6c00bf7..52d5ead96e 100644
--- a/tests/components/grove_tb6612fng/common.yaml
+++ b/tests/components/grove_tb6612fng/common.yaml
@@ -13,11 +13,7 @@ esphome:
           channel: 1
           id: test_motor
 
-i2c:
-  - id: i2c_grove_tb6612fng
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 grove_tb6612fng:
   id: test_motor
+  i2c_id: i2c_bus
   address: 0x14
diff --git a/tests/components/grove_tb6612fng/test.esp32-ard.yaml b/tests/components/grove_tb6612fng/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/grove_tb6612fng/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/grove_tb6612fng/test.esp32-c3-ard.yaml b/tests/components/grove_tb6612fng/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/grove_tb6612fng/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/grove_tb6612fng/test.esp32-c3-idf.yaml b/tests/components/grove_tb6612fng/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/grove_tb6612fng/test.esp32-c3-idf.yaml
+++ b/tests/components/grove_tb6612fng/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/grove_tb6612fng/test.esp32-idf.yaml b/tests/components/grove_tb6612fng/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/grove_tb6612fng/test.esp32-idf.yaml
+++ b/tests/components/grove_tb6612fng/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/grove_tb6612fng/test.esp8266-ard.yaml b/tests/components/grove_tb6612fng/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/grove_tb6612fng/test.esp8266-ard.yaml
+++ b/tests/components/grove_tb6612fng/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/grove_tb6612fng/test.rp2040-ard.yaml b/tests/components/grove_tb6612fng/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/grove_tb6612fng/test.rp2040-ard.yaml
+++ b/tests/components/grove_tb6612fng/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/growatt_solar/common.yaml b/tests/components/growatt_solar/common.yaml
index 7cc6b2a139..f304e84fcf 100644
--- a/tests/components/growatt_solar/common.yaml
+++ b/tests/components/growatt_solar/common.yaml
@@ -1,14 +1,6 @@
-uart:
-  - id: uart_growatt_solar
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
-modbus:
-  flow_control_pin: ${flow_control_pin}
-
 sensor:
   - platform: growatt_solar
+    modbus_id: modbus_bus
     update_interval: 10s
     protocol_version: RTU
     inverter_status:
diff --git a/tests/components/growatt_solar/test.esp32-ard.yaml b/tests/components/growatt_solar/test.esp32-ard.yaml
deleted file mode 100644
index bd767a8ece..0000000000
--- a/tests/components/growatt_solar/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-  flow_control_pin: GPIO13
-
-<<: !include common.yaml
diff --git a/tests/components/growatt_solar/test.esp32-c3-ard.yaml b/tests/components/growatt_solar/test.esp32-c3-ard.yaml
deleted file mode 100644
index 452031a5aa..0000000000
--- a/tests/components/growatt_solar/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  flow_control_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/growatt_solar/test.esp32-c3-idf.yaml b/tests/components/growatt_solar/test.esp32-c3-idf.yaml
index 452031a5aa..17940aafcf 100644
--- a/tests/components/growatt_solar/test.esp32-c3-idf.yaml
+++ b/tests/components/growatt_solar/test.esp32-c3-idf.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
   flow_control_pin: GPIO3
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/growatt_solar/test.esp32-idf.yaml b/tests/components/growatt_solar/test.esp32-idf.yaml
index bd767a8ece..a755bfa66a 100644
--- a/tests/components/growatt_solar/test.esp32-idf.yaml
+++ b/tests/components/growatt_solar/test.esp32-idf.yaml
@@ -3,4 +3,7 @@ substitutions:
   rx_pin: GPIO14
   flow_control_pin: GPIO13
 
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/growatt_solar/test.esp8266-ard.yaml b/tests/components/growatt_solar/test.esp8266-ard.yaml
index 29c98d0957..6daa08c22b 100644
--- a/tests/components/growatt_solar/test.esp8266-ard.yaml
+++ b/tests/components/growatt_solar/test.esp8266-ard.yaml
@@ -1,6 +1,9 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  flow_control_pin: GPIO13
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+  flow_control_pin: GPIO15
+
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/growatt_solar/test.rp2040-ard.yaml b/tests/components/growatt_solar/test.rp2040-ard.yaml
index 452031a5aa..a00283a9e0 100644
--- a/tests/components/growatt_solar/test.rp2040-ard.yaml
+++ b/tests/components/growatt_solar/test.rp2040-ard.yaml
@@ -3,4 +3,7 @@ substitutions:
   rx_pin: GPIO5
   flow_control_pin: GPIO3
 
+packages:
+  modbus: !include ../../test_build_components/common/modbus/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/gt911/common.yaml b/tests/components/gt911/common.yaml
index 7bb88108da..ff464cda24 100644
--- a/tests/components/gt911/common.yaml
+++ b/tests/components/gt911/common.yaml
@@ -1,23 +1,21 @@
-i2c:
-  - id: i2c_gt911
-    scl: 5
-    sda: 4
-
 display:
   - platform: ssd1306_i2c
-    id: ssd1306_display
+    i2c_id: i2c_bus
+    id: ssd1306_i2c_display
     model: SSD1306_128X64
-    reset_pin: 10
+    reset_pin: ${display_reset_pin}
     pages:
-      - id: page1
+      - id: gt911_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
 
 touchscreen:
   - platform: gt911
-    display: ssd1306_display
-    interrupt_pin: 20
-    reset_pin: 21
+    i2c_id: i2c_bus
+    id: gt911_touchscreen
+    display: ssd1306_i2c_display
+    interrupt_pin: ${interrupt_pin}
+    reset_pin: ${reset_pin}
 
 binary_sensor:
   - platform: gt911
diff --git a/tests/components/gt911/test.esp32-ard.yaml b/tests/components/gt911/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/gt911/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/gt911/test.esp32-c3-ard.yaml b/tests/components/gt911/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/gt911/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/gt911/test.esp32-c3-idf.yaml b/tests/components/gt911/test.esp32-c3-idf.yaml
index dade44d145..5e15963b7e 100644
--- a/tests/components/gt911/test.esp32-c3-idf.yaml
+++ b/tests/components/gt911/test.esp32-c3-idf.yaml
@@ -1 +1,9 @@
+substitutions:
+  display_reset_pin: "18"
+  interrupt_pin: "20"
+  reset_pin: "21"
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/gt911/test.esp32-idf.yaml b/tests/components/gt911/test.esp32-idf.yaml
index dade44d145..3bce86d9a3 100644
--- a/tests/components/gt911/test.esp32-idf.yaml
+++ b/tests/components/gt911/test.esp32-idf.yaml
@@ -1 +1,9 @@
+substitutions:
+  display_reset_pin: "10"
+  interrupt_pin: "20"
+  reset_pin: "21"
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/gt911/test.esp8266-ard.yaml b/tests/components/gt911/test.esp8266-ard.yaml
index 8b76eff29e..c3bc159b5b 100644
--- a/tests/components/gt911/test.esp8266-ard.yaml
+++ b/tests/components/gt911/test.esp8266-ard.yaml
@@ -1,24 +1,9 @@
-i2c:
-  - id: i2c_gt911
-    scl: 5
-    sda: 4
+substitutions:
+  display_reset_pin: "10"
+  interrupt_pin: "12"
+  reset_pin: "13"
 
-display:
-  - platform: ssd1306_i2c
-    id: ssd1306_display
-    model: SSD1306_128X64
-    reset_pin: 13
-    pages:
-      - id: page1
-        lambda: |-
-          it.rectangle(0, 0, it.get_width(), it.get_height());
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
-touchscreen:
-  - platform: gt911
-    display: ssd1306_display
-    interrupt_pin: 12
-
-binary_sensor:
-  - platform: gt911
-    id: touch_key_911
-    index: 0
+<<: !include common.yaml
diff --git a/tests/components/gt911/test.rp2040-ard.yaml b/tests/components/gt911/test.rp2040-ard.yaml
index dade44d145..0c7f0bc504 100644
--- a/tests/components/gt911/test.rp2040-ard.yaml
+++ b/tests/components/gt911/test.rp2040-ard.yaml
@@ -1 +1,9 @@
+substitutions:
+  display_reset_pin: "10"
+  interrupt_pin: "20"
+  reset_pin: "21"
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/haier/common.yaml b/tests/components/haier/common.yaml
index 368b88b69c..926a1d1b7a 100644
--- a/tests/components/haier/common.yaml
+++ b/tests/components/haier/common.yaml
@@ -2,16 +2,9 @@ wifi:
   ssid: MySSID
   password: password1
 
-uart:
-  - id: uart_haier
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 climate:
   - platform: haier
     id: haier_ac
-    uart_id: uart_haier
     protocol: hOn
     name: Haier AC
     wifi_signal: true
diff --git a/tests/components/haier/test.esp32-ard.yaml b/tests/components/haier/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/haier/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/haier/test.esp32-c3-ard.yaml b/tests/components/haier/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/haier/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/haier/test.esp32-c3-idf.yaml b/tests/components/haier/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/haier/test.esp32-c3-idf.yaml
+++ b/tests/components/haier/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/haier/test.esp32-idf.yaml b/tests/components/haier/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/haier/test.esp32-idf.yaml
+++ b/tests/components/haier/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/haier/test.esp8266-ard.yaml b/tests/components/haier/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/haier/test.esp8266-ard.yaml
+++ b/tests/components/haier/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/haier/test.rp2040-ard.yaml b/tests/components/haier/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/haier/test.rp2040-ard.yaml
+++ b/tests/components/haier/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/havells_solar/common.yaml b/tests/components/havells_solar/common.yaml
index 370b0d357d..2f00f61f14 100644
--- a/tests/components/havells_solar/common.yaml
+++ b/tests/components/havells_solar/common.yaml
@@ -1,14 +1,6 @@
-uart:
-  - id: uart_havells_solar
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
-modbus:
-  flow_control_pin: ${flow_control_pin}
-
 sensor:
   - platform: havells_solar
+    modbus_id: modbus_bus
     update_interval: 60s
     phase_a:
       voltage:
diff --git a/tests/components/havells_solar/test.esp32-ard.yaml b/tests/components/havells_solar/test.esp32-ard.yaml
deleted file mode 100644
index bd767a8ece..0000000000
--- a/tests/components/havells_solar/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-  flow_control_pin: GPIO13
-
-<<: !include common.yaml
diff --git a/tests/components/havells_solar/test.esp32-c3-ard.yaml b/tests/components/havells_solar/test.esp32-c3-ard.yaml
deleted file mode 100644
index 452031a5aa..0000000000
--- a/tests/components/havells_solar/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  flow_control_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/havells_solar/test.esp32-c3-idf.yaml b/tests/components/havells_solar/test.esp32-c3-idf.yaml
index 452031a5aa..17940aafcf 100644
--- a/tests/components/havells_solar/test.esp32-c3-idf.yaml
+++ b/tests/components/havells_solar/test.esp32-c3-idf.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
   flow_control_pin: GPIO3
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/havells_solar/test.esp32-idf.yaml b/tests/components/havells_solar/test.esp32-idf.yaml
index bd767a8ece..a755bfa66a 100644
--- a/tests/components/havells_solar/test.esp32-idf.yaml
+++ b/tests/components/havells_solar/test.esp32-idf.yaml
@@ -3,4 +3,7 @@ substitutions:
   rx_pin: GPIO14
   flow_control_pin: GPIO13
 
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/havells_solar/test.esp8266-ard.yaml b/tests/components/havells_solar/test.esp8266-ard.yaml
index 29c98d0957..6daa08c22b 100644
--- a/tests/components/havells_solar/test.esp8266-ard.yaml
+++ b/tests/components/havells_solar/test.esp8266-ard.yaml
@@ -1,6 +1,9 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  flow_control_pin: GPIO13
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+  flow_control_pin: GPIO15
+
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/havells_solar/test.rp2040-ard.yaml b/tests/components/havells_solar/test.rp2040-ard.yaml
index 452031a5aa..a00283a9e0 100644
--- a/tests/components/havells_solar/test.rp2040-ard.yaml
+++ b/tests/components/havells_solar/test.rp2040-ard.yaml
@@ -3,4 +3,7 @@ substitutions:
   rx_pin: GPIO5
   flow_control_pin: GPIO3
 
+packages:
+  modbus: !include ../../test_build_components/common/modbus/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/hbridge/common.yaml b/tests/components/hbridge/common.yaml
index 0504cdea03..ca619d6851 100644
--- a/tests/components/hbridge/common.yaml
+++ b/tests/components/hbridge/common.yaml
@@ -31,9 +31,3 @@ fan:
     on_preset_set:
       then:
         - logger.log: Preset mode was changed!
-
-switch:
-  - platform: hbridge
-    id: switch_hbridge
-    on_pin: ${hbridge_on_pin}
-    off_pin: ${hbridge_off_pin}
diff --git a/tests/components/hbridge/test.esp32-ard.yaml b/tests/components/hbridge/test.esp32-ard.yaml
deleted file mode 100644
index e50d537749..0000000000
--- a/tests/components/hbridge/test.esp32-ard.yaml
+++ /dev/null
@@ -1,17 +0,0 @@
-substitutions:
-  pwm_platform: ledc
-  output1_pin: "14"
-  output2_pin: "15"
-  output3_pin: "12"
-  output4_pin: "13"
-  hbridge_on_pin: "4"
-  hbridge_off_pin: "5"
-
-packages:
-  common: !include common.yaml
-
-switch:
-  - id: !extend switch_hbridge
-    pulse_length: 60ms
-    wait_time: 10ms
-    optimistic: false
diff --git a/tests/components/hbridge/test.esp32-c3-ard.yaml b/tests/components/hbridge/test.esp32-c3-ard.yaml
deleted file mode 100644
index b9e8738442..0000000000
--- a/tests/components/hbridge/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,16 +0,0 @@
-substitutions:
-  pwm_platform: "ledc"
-  output1_pin: "4"
-  output2_pin: "5"
-  output3_pin: "6"
-  output4_pin: "7"
-  hbridge_on_pin: "2"
-  hbridge_off_pin: "3"
-
-packages:
-  common: !include common.yaml
-
-switch:
-  - id: !extend switch_hbridge
-    wait_time: 10ms
-    optimistic: true
diff --git a/tests/components/hbridge/test.esp32-c3-idf.yaml b/tests/components/hbridge/test.esp32-c3-idf.yaml
index c73f08b6de..93a6cb5818 100644
--- a/tests/components/hbridge/test.esp32-c3-idf.yaml
+++ b/tests/components/hbridge/test.esp32-c3-idf.yaml
@@ -7,9 +7,11 @@ substitutions:
   hbridge_on_pin: "2"
   hbridge_off_pin: "3"
 
-packages:
-  common: !include common.yaml
+<<: !include common.yaml
 
 switch:
-  - id: !extend switch_hbridge
+  - platform: hbridge
+    id: switch_hbridge
+    on_pin: ${hbridge_on_pin}
+    off_pin: ${hbridge_off_pin}
     pulse_length: 60ms
diff --git a/tests/components/hbridge/test.esp32-idf.yaml b/tests/components/hbridge/test.esp32-idf.yaml
index dbbfa738c7..7f3d9b928a 100644
--- a/tests/components/hbridge/test.esp32-idf.yaml
+++ b/tests/components/hbridge/test.esp32-idf.yaml
@@ -7,10 +7,12 @@ substitutions:
   hbridge_on_pin: "4"
   hbridge_off_pin: "5"
 
-packages:
-  common: !include common.yaml
+<<: !include common.yaml
 
 switch:
-  - id: !extend switch_hbridge
+  - platform: hbridge
+    id: switch_hbridge
+    on_pin: ${hbridge_on_pin}
+    off_pin: ${hbridge_off_pin}
     pulse_length: 60ms
     wait_time: 10ms
diff --git a/tests/components/hbridge/test.esp8266-ard.yaml b/tests/components/hbridge/test.esp8266-ard.yaml
index f560da5d38..0346c8801d 100644
--- a/tests/components/hbridge/test.esp8266-ard.yaml
+++ b/tests/components/hbridge/test.esp8266-ard.yaml
@@ -7,10 +7,12 @@ substitutions:
   hbridge_on_pin: "14"
   hbridge_off_pin: "15"
 
-packages:
-  common: !include common.yaml
+<<: !include common.yaml
 
 switch:
-  - id: !extend switch_hbridge
+  - platform: hbridge
+    id: switch_hbridge
+    on_pin: ${hbridge_on_pin}
+    off_pin: ${hbridge_off_pin}
     pulse_length: 60ms
     wait_time: 10ms
diff --git a/tests/components/hbridge/test.rp2040-ard.yaml b/tests/components/hbridge/test.rp2040-ard.yaml
index aa6e290cab..f9cb2b5618 100644
--- a/tests/components/hbridge/test.rp2040-ard.yaml
+++ b/tests/components/hbridge/test.rp2040-ard.yaml
@@ -7,10 +7,12 @@ substitutions:
   hbridge_on_pin: "2"
   hbridge_off_pin: "3"
 
-packages:
-  common: !include common.yaml
+<<: !include common.yaml
 
 switch:
-  - id: !extend switch_hbridge
+  - platform: hbridge
+    id: switch_hbridge
+    on_pin: ${hbridge_on_pin}
+    off_pin: ${hbridge_off_pin}
     wait_time: 10ms
     optimistic: true
diff --git a/tests/components/hdc1080/common.yaml b/tests/components/hdc1080/common.yaml
index d260d4737c..a559392cb1 100644
--- a/tests/components/hdc1080/common.yaml
+++ b/tests/components/hdc1080/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_hdc1080
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: hdc1080
+    i2c_id: i2c_bus
     temperature:
       name: Temperature
     humidity:
diff --git a/tests/components/hdc1080/test.esp32-ard.yaml b/tests/components/hdc1080/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/hdc1080/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/hdc1080/test.esp32-c3-ard.yaml b/tests/components/hdc1080/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/hdc1080/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/hdc1080/test.esp32-c3-idf.yaml b/tests/components/hdc1080/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/hdc1080/test.esp32-c3-idf.yaml
+++ b/tests/components/hdc1080/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hdc1080/test.esp32-idf.yaml b/tests/components/hdc1080/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/hdc1080/test.esp32-idf.yaml
+++ b/tests/components/hdc1080/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hdc1080/test.esp8266-ard.yaml b/tests/components/hdc1080/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/hdc1080/test.esp8266-ard.yaml
+++ b/tests/components/hdc1080/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hdc1080/test.rp2040-ard.yaml b/tests/components/hdc1080/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/hdc1080/test.rp2040-ard.yaml
+++ b/tests/components/hdc1080/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/he60r/common.yaml b/tests/components/he60r/common.yaml
index e10e745f57..ab5ec67f0e 100644
--- a/tests/components/he60r/common.yaml
+++ b/tests/components/he60r/common.yaml
@@ -1,10 +1,3 @@
-uart:
-  - id: uart_he60r
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 1200
-    parity: EVEN
-
 cover:
   - platform: he60r
     id: garage_door
diff --git a/tests/components/he60r/test.esp32-ard.yaml b/tests/components/he60r/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/he60r/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/he60r/test.esp32-c3-ard.yaml b/tests/components/he60r/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/he60r/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/he60r/test.esp32-c3-idf.yaml b/tests/components/he60r/test.esp32-c3-idf.yaml
index b516342f3b..b0c8c5de3d 100644
--- a/tests/components/he60r/test.esp32-c3-idf.yaml
+++ b/tests/components/he60r/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart_1200_even: !include ../../test_build_components/common/uart_1200_even/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/he60r/test.esp32-idf.yaml b/tests/components/he60r/test.esp32-idf.yaml
index f486544afa..f52c2a75f8 100644
--- a/tests/components/he60r/test.esp32-idf.yaml
+++ b/tests/components/he60r/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+packages:
+  uart_1200_even: !include ../../test_build_components/common/uart_1200_even/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/he60r/test.esp8266-ard.yaml b/tests/components/he60r/test.esp8266-ard.yaml
index b516342f3b..e28024fa4d 100644
--- a/tests/components/he60r/test.esp8266-ard.yaml
+++ b/tests/components/he60r/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart_1200_even: !include ../../test_build_components/common/uart_1200_even/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/he60r/test.rp2040-ard.yaml b/tests/components/he60r/test.rp2040-ard.yaml
index b516342f3b..a576b03d63 100644
--- a/tests/components/he60r/test.rp2040-ard.yaml
+++ b/tests/components/he60r/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart_1200_even: !include ../../test_build_components/common/uart_1200_even/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/heatpumpir/common.yaml b/tests/components/heatpumpir/common.yaml
index d740f31518..a2779f9803 100644
--- a/tests/components/heatpumpir/common.yaml
+++ b/tests/components/heatpumpir/common.yaml
@@ -1,7 +1,3 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: heatpumpir
     protocol: mitsubishi_heavy_zm
@@ -10,6 +6,7 @@ climate:
     name: HeatpumpIR Climate Mitsubishi
     min_temperature: 18
     max_temperature: 30
+    transmitter_id: xmitr
   - platform: heatpumpir
     protocol: daikin
     horizontal_default: mleft
@@ -17,6 +14,7 @@ climate:
     name: HeatpumpIR Climate Daikin
     min_temperature: 18
     max_temperature: 30
+    transmitter_id: xmitr
   - platform: heatpumpir
     protocol: panasonic_altdke
     horizontal_default: mright
@@ -24,3 +22,4 @@ climate:
     name: HeatpumpIR Climate Panasonic
     min_temperature: 18
     max_temperature: 30
+    transmitter_id: xmitr
diff --git a/tests/components/heatpumpir/test.bk72xx-ard.yaml b/tests/components/heatpumpir/test.bk72xx-ard.yaml
index 06e1aea364..6cce191825 100644
--- a/tests/components/heatpumpir/test.bk72xx-ard.yaml
+++ b/tests/components/heatpumpir/test.bk72xx-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO6
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/bk72xx-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/heatpumpir/test.esp32-ard.yaml b/tests/components/heatpumpir/test.esp32-ard.yaml
index 7b012aa64c..01009de071 100644
--- a/tests/components/heatpumpir/test.esp32-ard.yaml
+++ b/tests/components/heatpumpir/test.esp32-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/heatpumpir/test.esp32-c3-ard.yaml b/tests/components/heatpumpir/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/heatpumpir/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/heatpumpir/test.esp8266-ard.yaml b/tests/components/heatpumpir/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/heatpumpir/test.esp8266-ard.yaml
+++ b/tests/components/heatpumpir/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hitachi_ac344/common.yaml b/tests/components/hitachi_ac344/common.yaml
index 960f032035..797e7875e5 100644
--- a/tests/components/hitachi_ac344/common.yaml
+++ b/tests/components/hitachi_ac344/common.yaml
@@ -1,7 +1,4 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: hitachi_ac344
     name: Hitachi Climate
+    transmitter_id: xmitr
diff --git a/tests/components/hitachi_ac344/test.bk72xx-ard.yaml b/tests/components/hitachi_ac344/test.bk72xx-ard.yaml
index 06e1aea364..6cce191825 100644
--- a/tests/components/hitachi_ac344/test.bk72xx-ard.yaml
+++ b/tests/components/hitachi_ac344/test.bk72xx-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO6
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/bk72xx-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hitachi_ac344/test.esp32-ard.yaml b/tests/components/hitachi_ac344/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/hitachi_ac344/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/hitachi_ac344/test.esp32-c3-ard.yaml b/tests/components/hitachi_ac344/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/hitachi_ac344/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/hitachi_ac344/test.esp32-c3-idf.yaml b/tests/components/hitachi_ac344/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/hitachi_ac344/test.esp32-c3-idf.yaml
+++ b/tests/components/hitachi_ac344/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hitachi_ac344/test.esp32-idf.yaml b/tests/components/hitachi_ac344/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/hitachi_ac344/test.esp32-idf.yaml
+++ b/tests/components/hitachi_ac344/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hitachi_ac344/test.esp8266-ard.yaml b/tests/components/hitachi_ac344/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/hitachi_ac344/test.esp8266-ard.yaml
+++ b/tests/components/hitachi_ac344/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hitachi_ac424/common.yaml b/tests/components/hitachi_ac424/common.yaml
index ad904c73a3..615bda4544 100644
--- a/tests/components/hitachi_ac424/common.yaml
+++ b/tests/components/hitachi_ac424/common.yaml
@@ -1,7 +1,4 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: hitachi_ac424
     name: Hitachi Climate
+    transmitter_id: xmitr
diff --git a/tests/components/hitachi_ac424/test.bk72xx-ard.yaml b/tests/components/hitachi_ac424/test.bk72xx-ard.yaml
index 06e1aea364..6cce191825 100644
--- a/tests/components/hitachi_ac424/test.bk72xx-ard.yaml
+++ b/tests/components/hitachi_ac424/test.bk72xx-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO6
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/bk72xx-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hitachi_ac424/test.esp32-ard.yaml b/tests/components/hitachi_ac424/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/hitachi_ac424/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/hitachi_ac424/test.esp32-c3-ard.yaml b/tests/components/hitachi_ac424/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/hitachi_ac424/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/hitachi_ac424/test.esp32-c3-idf.yaml b/tests/components/hitachi_ac424/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/hitachi_ac424/test.esp32-c3-idf.yaml
+++ b/tests/components/hitachi_ac424/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hitachi_ac424/test.esp32-idf.yaml b/tests/components/hitachi_ac424/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/hitachi_ac424/test.esp32-idf.yaml
+++ b/tests/components/hitachi_ac424/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hitachi_ac424/test.esp8266-ard.yaml b/tests/components/hitachi_ac424/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/hitachi_ac424/test.esp8266-ard.yaml
+++ b/tests/components/hitachi_ac424/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hlw8012/test.esp32-ard.yaml b/tests/components/hlw8012/test.esp32-ard.yaml
deleted file mode 100644
index 8b42b21b54..0000000000
--- a/tests/components/hlw8012/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  sel_pin: GPIO12
-  cf_pin: GPIO13
-  cf1_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/hlw8012/test.esp32-c3-ard.yaml b/tests/components/hlw8012/test.esp32-c3-ard.yaml
deleted file mode 100644
index 8b0d069ce2..0000000000
--- a/tests/components/hlw8012/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  sel_pin: GPIO2
-  cf_pin: GPIO3
-  cf1_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/hlw8012/test.esp8266-ard.yaml b/tests/components/hlw8012/test.esp8266-ard.yaml
index 8b42b21b54..ec9c0e43dc 100644
--- a/tests/components/hlw8012/test.esp8266-ard.yaml
+++ b/tests/components/hlw8012/test.esp8266-ard.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  sel_pin: GPIO12
-  cf_pin: GPIO13
-  cf1_pin: GPIO14
+  sel_pin: GPIO0
+  cf_pin: GPIO2
+  cf1_pin: GPIO15
 
 <<: !include common.yaml
diff --git a/tests/components/hm3301/common.yaml b/tests/components/hm3301/common.yaml
index b533130569..a56ac7bc65 100644
--- a/tests/components/hm3301/common.yaml
+++ b/tests/components/hm3301/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_hm3301
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: hm3301
+    i2c_id: i2c_bus
     pm_1_0:
       name: PM1.0
     pm_2_5:
diff --git a/tests/components/hm3301/test.esp32-ard.yaml b/tests/components/hm3301/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/hm3301/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/hm3301/test.esp32-c3-ard.yaml b/tests/components/hm3301/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/hm3301/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/hm3301/test.esp32-c3-idf.yaml b/tests/components/hm3301/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/hm3301/test.esp32-c3-idf.yaml
+++ b/tests/components/hm3301/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hm3301/test.esp32-idf.yaml b/tests/components/hm3301/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/hm3301/test.esp32-idf.yaml
+++ b/tests/components/hm3301/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hm3301/test.esp8266-ard.yaml b/tests/components/hm3301/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/hm3301/test.esp8266-ard.yaml
+++ b/tests/components/hm3301/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hm3301/test.rp2040-ard.yaml b/tests/components/hm3301/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/hm3301/test.rp2040-ard.yaml
+++ b/tests/components/hm3301/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hmc5883l/common.yaml b/tests/components/hmc5883l/common.yaml
index 1c90f5f1c6..19e4ba4c45 100644
--- a/tests/components/hmc5883l/common.yaml
+++ b/tests/components/hmc5883l/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_hmc5883l
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: hmc5883l
+    i2c_id: i2c_bus
     address: 0x68
     field_strength_x:
       name: HMC5883L Field Strength X
diff --git a/tests/components/hmc5883l/test.esp32-ard.yaml b/tests/components/hmc5883l/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/hmc5883l/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/hmc5883l/test.esp32-c3-ard.yaml b/tests/components/hmc5883l/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/hmc5883l/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/hmc5883l/test.esp32-c3-idf.yaml b/tests/components/hmc5883l/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/hmc5883l/test.esp32-c3-idf.yaml
+++ b/tests/components/hmc5883l/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hmc5883l/test.esp32-idf.yaml b/tests/components/hmc5883l/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/hmc5883l/test.esp32-idf.yaml
+++ b/tests/components/hmc5883l/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hmc5883l/test.esp8266-ard.yaml b/tests/components/hmc5883l/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/hmc5883l/test.esp8266-ard.yaml
+++ b/tests/components/hmc5883l/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hmc5883l/test.rp2040-ard.yaml b/tests/components/hmc5883l/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/hmc5883l/test.rp2040-ard.yaml
+++ b/tests/components/hmc5883l/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/homeassistant/test-tag-scanned.esp32-idf.yaml b/tests/components/homeassistant/test-tag-scanned.esp32-idf.yaml
new file mode 100644
index 0000000000..ef148174d7
--- /dev/null
+++ b/tests/components/homeassistant/test-tag-scanned.esp32-idf.yaml
@@ -0,0 +1,14 @@
+wifi:
+  ssid: MySSID
+  password: password1
+
+api:
+
+esphome:
+  on_boot:
+    then:
+      - homeassistant.tag_scanned: 'test_tag_123'
+      - homeassistant.tag_scanned:
+          tag: 'another_tag'
+      - homeassistant.tag_scanned:
+          tag: !lambda 'return "dynamic_tag";'
diff --git a/tests/components/homeassistant/test.esp32-ard.yaml b/tests/components/homeassistant/test.esp32-ard.yaml
deleted file mode 100644
index 25cb37a0b4..0000000000
--- a/tests/components/homeassistant/test.esp32-ard.yaml
+++ /dev/null
@@ -1,2 +0,0 @@
-packages:
-  common: !include common.yaml
diff --git a/tests/components/homeassistant/test.esp32-c3-ard.yaml b/tests/components/homeassistant/test.esp32-c3-ard.yaml
deleted file mode 100644
index 25cb37a0b4..0000000000
--- a/tests/components/homeassistant/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,2 +0,0 @@
-packages:
-  common: !include common.yaml
diff --git a/tests/components/honeywell_hih_i2c/common.yaml b/tests/components/honeywell_hih_i2c/common.yaml
index a5f3eef187..de588a30bd 100644
--- a/tests/components/honeywell_hih_i2c/common.yaml
+++ b/tests/components/honeywell_hih_i2c/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_honeywell_hih
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: honeywell_hih_i2c
+    i2c_id: i2c_bus
     temperature:
       name: Temperature
     humidity:
diff --git a/tests/components/honeywell_hih_i2c/test.esp32-ard.yaml b/tests/components/honeywell_hih_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/honeywell_hih_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/honeywell_hih_i2c/test.esp32-c3-ard.yaml b/tests/components/honeywell_hih_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/honeywell_hih_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/honeywell_hih_i2c/test.esp32-c3-idf.yaml b/tests/components/honeywell_hih_i2c/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/honeywell_hih_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/honeywell_hih_i2c/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/honeywell_hih_i2c/test.esp32-idf.yaml b/tests/components/honeywell_hih_i2c/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/honeywell_hih_i2c/test.esp32-idf.yaml
+++ b/tests/components/honeywell_hih_i2c/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/honeywell_hih_i2c/test.esp8266-ard.yaml b/tests/components/honeywell_hih_i2c/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/honeywell_hih_i2c/test.esp8266-ard.yaml
+++ b/tests/components/honeywell_hih_i2c/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/honeywell_hih_i2c/test.rp2040-ard.yaml b/tests/components/honeywell_hih_i2c/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/honeywell_hih_i2c/test.rp2040-ard.yaml
+++ b/tests/components/honeywell_hih_i2c/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/honeywellabp/common.yaml b/tests/components/honeywellabp/common.yaml
index 21a3ef6ee3..f82d289ace 100644
--- a/tests/components/honeywellabp/common.yaml
+++ b/tests/components/honeywellabp/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_bme280
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: honeywellabp
     cs_pin: ${cs_pin}
diff --git a/tests/components/honeywellabp/test.esp32-ard.yaml b/tests/components/honeywellabp/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/honeywellabp/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/honeywellabp/test.esp32-c3-ard.yaml b/tests/components/honeywellabp/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/honeywellabp/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/honeywellabp/test.esp32-c3-idf.yaml b/tests/components/honeywellabp/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/honeywellabp/test.esp32-c3-idf.yaml
+++ b/tests/components/honeywellabp/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/honeywellabp/test.esp32-idf.yaml b/tests/components/honeywellabp/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/honeywellabp/test.esp32-idf.yaml
+++ b/tests/components/honeywellabp/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/honeywellabp/test.esp8266-ard.yaml b/tests/components/honeywellabp/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/honeywellabp/test.esp8266-ard.yaml
+++ b/tests/components/honeywellabp/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/honeywellabp/test.rp2040-ard.yaml b/tests/components/honeywellabp/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/honeywellabp/test.rp2040-ard.yaml
+++ b/tests/components/honeywellabp/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/honeywellabp2_i2c/common.yaml b/tests/components/honeywellabp2_i2c/common.yaml
index 6752a69866..e1b060edcf 100644
--- a/tests/components/honeywellabp2_i2c/common.yaml
+++ b/tests/components/honeywellabp2_i2c/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_honeywellabp2
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: honeywellabp2_i2c
+    i2c_id: i2c_bus
     address: 0x28
     pressure:
       name: Honeywell2 pressure
diff --git a/tests/components/honeywellabp2_i2c/test.esp32-ard.yaml b/tests/components/honeywellabp2_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/honeywellabp2_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/honeywellabp2_i2c/test.esp32-c3-ard.yaml b/tests/components/honeywellabp2_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/honeywellabp2_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/honeywellabp2_i2c/test.esp32-c3-idf.yaml b/tests/components/honeywellabp2_i2c/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/honeywellabp2_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/honeywellabp2_i2c/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/honeywellabp2_i2c/test.esp32-idf.yaml b/tests/components/honeywellabp2_i2c/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/honeywellabp2_i2c/test.esp32-idf.yaml
+++ b/tests/components/honeywellabp2_i2c/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/honeywellabp2_i2c/test.esp8266-ard.yaml b/tests/components/honeywellabp2_i2c/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/honeywellabp2_i2c/test.esp8266-ard.yaml
+++ b/tests/components/honeywellabp2_i2c/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/honeywellabp2_i2c/test.rp2040-ard.yaml b/tests/components/honeywellabp2_i2c/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/honeywellabp2_i2c/test.rp2040-ard.yaml
+++ b/tests/components/honeywellabp2_i2c/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/host/common.yaml b/tests/components/host/common.yaml
index fca0c5d597..5c329c8245 100644
--- a/tests/components/host/common.yaml
+++ b/tests/components/host/common.yaml
@@ -8,3 +8,10 @@ logger:
 
 host:
   mac_address: "62:23:45:AF:B3:DD"
+
+esphome:
+  on_boot:
+    - lambda: |-
+        static const uint8_t my_addr[6] = {0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF};
+        if (!mac_address_is_valid(my_addr))
+          ESP_LOGD("test", "Invalid mac address %X", my_addr[0]); // etc.
diff --git a/tests/components/hrxl_maxsonar_wr/common.yaml b/tests/components/hrxl_maxsonar_wr/common.yaml
index d74ada716d..84376aa5ba 100644
--- a/tests/components/hrxl_maxsonar_wr/common.yaml
+++ b/tests/components/hrxl_maxsonar_wr/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_hrxl_maxsonar_wr
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 115200
-
 sensor:
   - platform: hrxl_maxsonar_wr
     id: hrxl_maxsonar_wr_sensor
diff --git a/tests/components/hrxl_maxsonar_wr/test.esp32-ard.yaml b/tests/components/hrxl_maxsonar_wr/test.esp32-ard.yaml
deleted file mode 100644
index 811f6b72a6..0000000000
--- a/tests/components/hrxl_maxsonar_wr/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/hrxl_maxsonar_wr/test.esp32-c3-ard.yaml b/tests/components/hrxl_maxsonar_wr/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/hrxl_maxsonar_wr/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/hrxl_maxsonar_wr/test.esp32-c3-idf.yaml b/tests/components/hrxl_maxsonar_wr/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/hrxl_maxsonar_wr/test.esp32-c3-idf.yaml
+++ b/tests/components/hrxl_maxsonar_wr/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hrxl_maxsonar_wr/test.esp32-idf.yaml b/tests/components/hrxl_maxsonar_wr/test.esp32-idf.yaml
index 811f6b72a6..64baa4ec9d 100644
--- a/tests/components/hrxl_maxsonar_wr/test.esp32-idf.yaml
+++ b/tests/components/hrxl_maxsonar_wr/test.esp32-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO12
   rx_pin: GPIO14
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/hrxl_maxsonar_wr/test.esp8266-ard.yaml b/tests/components/hrxl_maxsonar_wr/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/hrxl_maxsonar_wr/test.esp8266-ard.yaml
+++ b/tests/components/hrxl_maxsonar_wr/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hrxl_maxsonar_wr/test.rp2040-ard.yaml b/tests/components/hrxl_maxsonar_wr/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/hrxl_maxsonar_wr/test.rp2040-ard.yaml
+++ b/tests/components/hrxl_maxsonar_wr/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/hte501/common.yaml b/tests/components/hte501/common.yaml
index 7c57b5bc6a..e0641de193 100644
--- a/tests/components/hte501/common.yaml
+++ b/tests/components/hte501/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_hte501
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: hte501
+    i2c_id: i2c_bus
     address: 0x40
     temperature:
       name: Temperature
diff --git a/tests/components/hte501/test.esp32-ard.yaml b/tests/components/hte501/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/hte501/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/hte501/test.esp32-c3-ard.yaml b/tests/components/hte501/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/hte501/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/hte501/test.esp32-c3-idf.yaml b/tests/components/hte501/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/hte501/test.esp32-c3-idf.yaml
+++ b/tests/components/hte501/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hte501/test.esp32-idf.yaml b/tests/components/hte501/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/hte501/test.esp32-idf.yaml
+++ b/tests/components/hte501/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hte501/test.esp8266-ard.yaml b/tests/components/hte501/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/hte501/test.esp8266-ard.yaml
+++ b/tests/components/hte501/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hte501/test.rp2040-ard.yaml b/tests/components/hte501/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/hte501/test.rp2040-ard.yaml
+++ b/tests/components/hte501/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/http_request/common.yaml b/tests/components/http_request/common.yaml
index 97961007e2..9ff9f9fb67 100644
--- a/tests/components/http_request/common.yaml
+++ b/tests/components/http_request/common.yaml
@@ -51,6 +51,7 @@ script:
 
 ota:
   - platform: http_request
+    id: http_request_ota
     on_begin:
       then:
         - logger.log: "OTA start"
@@ -77,10 +78,12 @@ button:
     on_press:
       then:
         - ota.http_request.flash:
+            id: http_request_ota
             md5_url: http://my.ha.net:8123/local/esphome/firmware.md5
             url: http://my.ha.net:8123/local/esphome/firmware.bin
 
         - ota.http_request.flash:
+            id: http_request_ota
             md5: 0123456789abcdef0123456789abcdef
             url: http://my.ha.net:8123/local/esphome/firmware.bin
 
@@ -90,6 +93,7 @@ update:
   - platform: http_request
     name: OTA Update
     id: ota_update
+    ota_id: http_request_ota
     source: http://my.ha.net:8123/local/esphome/manifest.json
     on_update_available:
       - logger.log: "A new update is available"
diff --git a/tests/components/http_request/test.esp32-c3-ard.yaml b/tests/components/http_request/test.esp32-c3-ard.yaml
deleted file mode 100644
index c1937b5a10..0000000000
--- a/tests/components/http_request/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  verify_ssl: "false"
-
-<<: !include common.yaml
diff --git a/tests/components/htu21d/common.yaml b/tests/components/htu21d/common.yaml
index f12c1ca46e..ad4b23d460 100644
--- a/tests/components/htu21d/common.yaml
+++ b/tests/components/htu21d/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_htu21d
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: htu21d
+    i2c_id: i2c_bus
     model: htu21d
     temperature:
       name: Temperature
diff --git a/tests/components/htu21d/test.esp32-ard.yaml b/tests/components/htu21d/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/htu21d/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/htu21d/test.esp32-c3-ard.yaml b/tests/components/htu21d/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/htu21d/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/htu21d/test.esp32-c3-idf.yaml b/tests/components/htu21d/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/htu21d/test.esp32-c3-idf.yaml
+++ b/tests/components/htu21d/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/htu21d/test.esp32-idf.yaml b/tests/components/htu21d/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/htu21d/test.esp32-idf.yaml
+++ b/tests/components/htu21d/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/htu21d/test.esp8266-ard.yaml b/tests/components/htu21d/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/htu21d/test.esp8266-ard.yaml
+++ b/tests/components/htu21d/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/htu21d/test.rp2040-ard.yaml b/tests/components/htu21d/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/htu21d/test.rp2040-ard.yaml
+++ b/tests/components/htu21d/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/htu31d/common.yaml b/tests/components/htu31d/common.yaml
index 735cdc7cbf..41d718ea69 100644
--- a/tests/components/htu31d/common.yaml
+++ b/tests/components/htu31d/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_htu31d
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: htu31d
+    i2c_id: i2c_bus
     temperature:
       name: Living Room Temperature
     humidity:
diff --git a/tests/components/htu31d/test.esp32-ard.yaml b/tests/components/htu31d/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/htu31d/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/htu31d/test.esp32-c3-ard.yaml b/tests/components/htu31d/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/htu31d/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/htu31d/test.esp32-c3-idf.yaml b/tests/components/htu31d/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/htu31d/test.esp32-c3-idf.yaml
+++ b/tests/components/htu31d/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/htu31d/test.esp32-idf.yaml b/tests/components/htu31d/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/htu31d/test.esp32-idf.yaml
+++ b/tests/components/htu31d/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/htu31d/test.esp8266-ard.yaml b/tests/components/htu31d/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/htu31d/test.esp8266-ard.yaml
+++ b/tests/components/htu31d/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/htu31d/test.rp2040-ard.yaml b/tests/components/htu31d/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/htu31d/test.rp2040-ard.yaml
+++ b/tests/components/htu31d/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hx711/test.esp32-ard.yaml b/tests/components/hx711/test.esp32-ard.yaml
deleted file mode 100644
index 6423867395..0000000000
--- a/tests/components/hx711/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  dout_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/hx711/test.esp32-c3-ard.yaml b/tests/components/hx711/test.esp32-c3-ard.yaml
deleted file mode 100644
index 08a6e705c0..0000000000
--- a/tests/components/hx711/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clk_pin: GPIO5
-  dout_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/hx711/test.esp32-c3-idf.yaml b/tests/components/hx711/test.esp32-c3-idf.yaml
index 08a6e705c0..defef165e3 100644
--- a/tests/components/hx711/test.esp32-c3-idf.yaml
+++ b/tests/components/hx711/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clk_pin: GPIO5
-  dout_pin: GPIO4
+  clk_pin: GPIO4
+  dout_pin: GPIO5
 
 <<: !include common.yaml
diff --git a/tests/components/hx711/test.esp32-idf.yaml b/tests/components/hx711/test.esp32-idf.yaml
index 6423867395..defef165e3 100644
--- a/tests/components/hx711/test.esp32-idf.yaml
+++ b/tests/components/hx711/test.esp32-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clk_pin: GPIO16
-  dout_pin: GPIO17
+  clk_pin: GPIO4
+  dout_pin: GPIO5
 
 <<: !include common.yaml
diff --git a/tests/components/hx711/test.esp8266-ard.yaml b/tests/components/hx711/test.esp8266-ard.yaml
index 08a6e705c0..e7c017ed99 100644
--- a/tests/components/hx711/test.esp8266-ard.yaml
+++ b/tests/components/hx711/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clk_pin: GPIO5
-  dout_pin: GPIO4
+  clk_pin: GPIO0
+  dout_pin: GPIO2
 
 <<: !include common.yaml
diff --git a/tests/components/hx711/test.rp2040-ard.yaml b/tests/components/hx711/test.rp2040-ard.yaml
index 08a6e705c0..defef165e3 100644
--- a/tests/components/hx711/test.rp2040-ard.yaml
+++ b/tests/components/hx711/test.rp2040-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clk_pin: GPIO5
-  dout_pin: GPIO4
+  clk_pin: GPIO4
+  dout_pin: GPIO5
 
 <<: !include common.yaml
diff --git a/tests/components/hydreon_rgxx/common.yaml b/tests/components/hydreon_rgxx/common.yaml
index e11c6c91c9..e96879fdec 100644
--- a/tests/components/hydreon_rgxx/common.yaml
+++ b/tests/components/hydreon_rgxx/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_hydreon_rgxx
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 115200
-
 binary_sensor:
   - platform: hydreon_rgxx
     hydreon_rgxx_id: hydreon_rg9
diff --git a/tests/components/hydreon_rgxx/test.esp32-ard.yaml b/tests/components/hydreon_rgxx/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/hydreon_rgxx/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/hydreon_rgxx/test.esp32-c3-ard.yaml b/tests/components/hydreon_rgxx/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/hydreon_rgxx/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/hydreon_rgxx/test.esp32-c3-idf.yaml b/tests/components/hydreon_rgxx/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/hydreon_rgxx/test.esp32-c3-idf.yaml
+++ b/tests/components/hydreon_rgxx/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hydreon_rgxx/test.esp32-idf.yaml b/tests/components/hydreon_rgxx/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/hydreon_rgxx/test.esp32-idf.yaml
+++ b/tests/components/hydreon_rgxx/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hydreon_rgxx/test.esp8266-ard.yaml b/tests/components/hydreon_rgxx/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/hydreon_rgxx/test.esp8266-ard.yaml
+++ b/tests/components/hydreon_rgxx/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hydreon_rgxx/test.rp2040-ard.yaml b/tests/components/hydreon_rgxx/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/hydreon_rgxx/test.rp2040-ard.yaml
+++ b/tests/components/hydreon_rgxx/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/hyt271/common.yaml b/tests/components/hyt271/common.yaml
index 7a4371173f..5771d882da 100644
--- a/tests/components/hyt271/common.yaml
+++ b/tests/components/hyt271/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_hyt271
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: hyt271
+    i2c_id: i2c_bus
     temperature:
       name: Temperature
     humidity:
diff --git a/tests/components/hyt271/test.esp32-ard.yaml b/tests/components/hyt271/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/hyt271/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/hyt271/test.esp32-c3-ard.yaml b/tests/components/hyt271/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/hyt271/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/hyt271/test.esp32-c3-idf.yaml b/tests/components/hyt271/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/hyt271/test.esp32-c3-idf.yaml
+++ b/tests/components/hyt271/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hyt271/test.esp32-idf.yaml b/tests/components/hyt271/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/hyt271/test.esp32-idf.yaml
+++ b/tests/components/hyt271/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hyt271/test.esp8266-ard.yaml b/tests/components/hyt271/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/hyt271/test.esp8266-ard.yaml
+++ b/tests/components/hyt271/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/hyt271/test.rp2040-ard.yaml b/tests/components/hyt271/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/hyt271/test.rp2040-ard.yaml
+++ b/tests/components/hyt271/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/i2c/common.yaml b/tests/components/i2c/common.yaml
index d70cd595ad..e69de29bb2 100644
--- a/tests/components/i2c/common.yaml
+++ b/tests/components/i2c/common.yaml
@@ -1,4 +0,0 @@
-i2c:
-  - id: i2c_i2c
-    scl: ${scl_pin}
-    sda: ${sda_pin}
diff --git a/tests/components/i2c/test.esp32-ard.yaml b/tests/components/i2c/test.esp32-ard.yaml
index 63c3bd6afd..7c503b0ccb 100644
--- a/tests/components/i2c/test.esp32-ard.yaml
+++ b/tests/components/i2c/test.esp32-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/i2c/test.esp32-c3-ard.yaml b/tests/components/i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/i2c/test.esp32-c3-idf.yaml b/tests/components/i2c/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/i2c/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/i2c/test.esp32-idf.yaml b/tests/components/i2c/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/i2c/test.esp32-idf.yaml
+++ b/tests/components/i2c/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/i2c/test.esp8266-ard.yaml b/tests/components/i2c/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/i2c/test.esp8266-ard.yaml
+++ b/tests/components/i2c/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/i2c/test.rp2040-ard.yaml b/tests/components/i2c/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/i2c/test.rp2040-ard.yaml
+++ b/tests/components/i2c/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/i2c_device/common.yaml b/tests/components/i2c_device/common.yaml
index 35a6f8f7f0..fed399eb8c 100644
--- a/tests/components/i2c_device/common.yaml
+++ b/tests/components/i2c_device/common.yaml
@@ -1,8 +1,4 @@
-i2c:
-  - id: i2c_i2c_device
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 i2c_device:
   id: i2cdev
+  i2c_id: i2c_bus
   address: 0x2C
diff --git a/tests/components/i2c_device/test.esp32-ard.yaml b/tests/components/i2c_device/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/i2c_device/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/i2c_device/test.esp32-c3-ard.yaml b/tests/components/i2c_device/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/i2c_device/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/i2c_device/test.esp32-c3-idf.yaml b/tests/components/i2c_device/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/i2c_device/test.esp32-c3-idf.yaml
+++ b/tests/components/i2c_device/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/i2c_device/test.esp32-idf.yaml b/tests/components/i2c_device/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/i2c_device/test.esp32-idf.yaml
+++ b/tests/components/i2c_device/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/i2c_device/test.esp8266-ard.yaml b/tests/components/i2c_device/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/i2c_device/test.esp8266-ard.yaml
+++ b/tests/components/i2c_device/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/i2c_device/test.rp2040-ard.yaml b/tests/components/i2c_device/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/i2c_device/test.rp2040-ard.yaml
+++ b/tests/components/i2c_device/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/i2s_audio/test.esp32-ard.yaml b/tests/components/i2s_audio/test.esp32-ard.yaml
deleted file mode 100644
index ce751d7d4a..0000000000
--- a/tests/components/i2s_audio/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  i2s_bclk_pin: GPIO15
-  i2s_lrclk_pin: GPIO16
-  i2s_mclk_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/i2s_audio/test.esp32-c3-ard.yaml b/tests/components/i2s_audio/test.esp32-c3-ard.yaml
deleted file mode 100644
index 5490846ae9..0000000000
--- a/tests/components/i2s_audio/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  i2s_bclk_pin: GPIO5
-  i2s_lrclk_pin: GPIO6
-  i2s_mclk_pin: GPIO7
-
-<<: !include common.yaml
diff --git a/tests/components/i2s_audio/test.esp32-idf.yaml b/tests/components/i2s_audio/test.esp32-idf.yaml
index ce751d7d4a..ead1eaa1e9 100644
--- a/tests/components/i2s_audio/test.esp32-idf.yaml
+++ b/tests/components/i2s_audio/test.esp32-idf.yaml
@@ -1,6 +1,9 @@
 substitutions:
   i2s_bclk_pin: GPIO15
-  i2s_lrclk_pin: GPIO16
-  i2s_mclk_pin: GPIO17
+  i2s_lrclk_pin: GPIO4
+  i2s_mclk_pin: GPIO5
+
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/iaqcore/common.yaml b/tests/components/iaqcore/common.yaml
index 927b836c52..e0f9d09a00 100644
--- a/tests/components/iaqcore/common.yaml
+++ b/tests/components/iaqcore/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_iaqcore
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: iaqcore
+    i2c_id: i2c_bus
     co2:
       name: iAQ Core CO2 Sensor
     tvoc:
diff --git a/tests/components/iaqcore/test.esp32-ard.yaml b/tests/components/iaqcore/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/iaqcore/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/iaqcore/test.esp32-c3-ard.yaml b/tests/components/iaqcore/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/iaqcore/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/iaqcore/test.esp32-c3-idf.yaml b/tests/components/iaqcore/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/iaqcore/test.esp32-c3-idf.yaml
+++ b/tests/components/iaqcore/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/iaqcore/test.esp32-idf.yaml b/tests/components/iaqcore/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/iaqcore/test.esp32-idf.yaml
+++ b/tests/components/iaqcore/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/iaqcore/test.esp8266-ard.yaml b/tests/components/iaqcore/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/iaqcore/test.esp8266-ard.yaml
+++ b/tests/components/iaqcore/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/iaqcore/test.rp2040-ard.yaml b/tests/components/iaqcore/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/iaqcore/test.rp2040-ard.yaml
+++ b/tests/components/iaqcore/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ili9xxx/common.yaml b/tests/components/ili9xxx/common.yaml
index b182d110cd..47384b1398 100644
--- a/tests/components/ili9xxx/common.yaml
+++ b/tests/components/ili9xxx/common.yaml
@@ -1,8 +1,3 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   - platform: ili9xxx
     invert_colors: true
diff --git a/tests/components/ili9xxx/test.esp32-ard.yaml b/tests/components/ili9xxx/test.esp32-ard.yaml
deleted file mode 100644
index 2e006d2521..0000000000
--- a/tests/components/ili9xxx/test.esp32-ard.yaml
+++ /dev/null
@@ -1,11 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  cs_pin1: GPIO12
-  dc_pin1: GPIO13
-  reset_pin1: GPIO14
-  cs_pin2: GPIO25
-  dc_pin2: GPIO26
-  reset_pin2: GPIO27
-
-<<: !include common.yaml
diff --git a/tests/components/ili9xxx/test.esp32-c3-ard.yaml b/tests/components/ili9xxx/test.esp32-c3-ard.yaml
deleted file mode 100644
index 3037785e81..0000000000
--- a/tests/components/ili9xxx/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,12 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin1: GPIO8
-  dc_pin1: GPIO9
-  reset_pin1: GPIO10
-  cs_pin2: GPIO2
-  dc_pin2: GPIO3
-  reset_pin2: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ili9xxx/test.esp32-c3-idf.yaml b/tests/components/ili9xxx/test.esp32-c3-idf.yaml
index 3037785e81..1eea1e85f7 100644
--- a/tests/components/ili9xxx/test.esp32-c3-idf.yaml
+++ b/tests/components/ili9xxx/test.esp32-c3-idf.yaml
@@ -1,12 +1,11 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin1: GPIO8
   dc_pin1: GPIO9
   reset_pin1: GPIO10
   cs_pin2: GPIO2
   dc_pin2: GPIO3
-  reset_pin2: GPIO4
+  reset_pin2: GPIO7
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ili9xxx/test.esp32-idf.yaml b/tests/components/ili9xxx/test.esp32-idf.yaml
index 2e006d2521..866e57573b 100644
--- a/tests/components/ili9xxx/test.esp32-idf.yaml
+++ b/tests/components/ili9xxx/test.esp32-idf.yaml
@@ -1,6 +1,4 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
   cs_pin1: GPIO12
   dc_pin1: GPIO13
   reset_pin1: GPIO14
@@ -8,4 +6,7 @@ substitutions:
   dc_pin2: GPIO26
   reset_pin2: GPIO27
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ili9xxx/test.esp8266-ard.yaml b/tests/components/ili9xxx/test.esp8266-ard.yaml
index 5dee055fd4..feb7773794 100644
--- a/tests/components/ili9xxx/test.esp8266-ard.yaml
+++ b/tests/components/ili9xxx/test.esp8266-ard.yaml
@@ -9,4 +9,7 @@ substitutions:
   dc_pin2: GPIO4
   reset_pin2: GPIO0
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ili9xxx/test.rp2040-ard.yaml b/tests/components/ili9xxx/test.rp2040-ard.yaml
index 74c9b906e9..2acde3e629 100644
--- a/tests/components/ili9xxx/test.rp2040-ard.yaml
+++ b/tests/components/ili9xxx/test.rp2040-ard.yaml
@@ -9,4 +9,7 @@ substitutions:
   dc_pin2: GPIO21
   reset_pin2: GPIO22
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/image/test.esp32-idf.yaml b/tests/components/image/test.esp32-idf.yaml
index 814f16c36c..aea2b4bbb0 100644
--- a/tests/components/image/test.esp32-idf.yaml
+++ b/tests/components/image/test.esp32-idf.yaml
@@ -1,14 +1,12 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 16
-    mosi_pin: 17
-    miso_pin: 18
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
 display:
   - platform: ili9xxx
     id: main_lcd
+    spi_id: spi_bus
     model: ili9342
-    cs_pin: 19
+    cs_pin: 15
     dc_pin: 13
     reset_pin: 21
     invert_colors: true
diff --git a/tests/components/image/test.esp8266-ard.yaml b/tests/components/image/test.esp8266-ard.yaml
index 626076d44e..2e7bfc5ae5 100644
--- a/tests/components/image/test.esp8266-ard.yaml
+++ b/tests/components/image/test.esp8266-ard.yaml
@@ -1,12 +1,10 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 14
-    mosi_pin: 13
-    miso_pin: 12
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
 
 display:
   - platform: ili9xxx
     id: main_lcd
+    spi_id: spi_bus
     model: ili9342
     cs_pin: 5
     dc_pin: 15
diff --git a/tests/components/image/test.rp2040-ard.yaml b/tests/components/image/test.rp2040-ard.yaml
index 5167c99a7d..40f17d57fe 100644
--- a/tests/components/image/test.rp2040-ard.yaml
+++ b/tests/components/image/test.rp2040-ard.yaml
@@ -1,12 +1,10 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 2
-    mosi_pin: 3
-    miso_pin: 4
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
 
 display:
   - platform: ili9xxx
     id: main_lcd
+    spi_id: spi_bus
     model: ili9342
     cs_pin: 20
     dc_pin: 21
diff --git a/tests/components/improv_serial/test-uart0.esp32-ard.yaml b/tests/components/improv_serial/test-uart0.esp32-ard.yaml
deleted file mode 100644
index ef8c799241..0000000000
--- a/tests/components/improv_serial/test-uart0.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-uart0.yaml
diff --git a/tests/components/improv_serial/test-uart0.esp32-c3-ard.yaml b/tests/components/improv_serial/test-uart0.esp32-c3-ard.yaml
deleted file mode 100644
index ef8c799241..0000000000
--- a/tests/components/improv_serial/test-uart0.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-uart0.yaml
diff --git a/tests/components/improv_serial/test-uart0.esp32-s2-ard.yaml b/tests/components/improv_serial/test-uart0.esp32-s2-ard.yaml
deleted file mode 100644
index ef8c799241..0000000000
--- a/tests/components/improv_serial/test-uart0.esp32-s2-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-uart0.yaml
diff --git a/tests/components/improv_serial/test-uart0.esp32-s3-ard.yaml b/tests/components/improv_serial/test-uart0.esp32-s3-ard.yaml
deleted file mode 100644
index ef8c799241..0000000000
--- a/tests/components/improv_serial/test-uart0.esp32-s3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-uart0.yaml
diff --git a/tests/components/improv_serial/test-usb_cdc.esp32-s3-ard.yaml b/tests/components/improv_serial/test-usb_cdc.esp32-s3-ard.yaml
deleted file mode 100644
index cfdaec9771..0000000000
--- a/tests/components/improv_serial/test-usb_cdc.esp32-s3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-usb_cdc.yaml
diff --git a/tests/components/ina219/common.yaml b/tests/components/ina219/common.yaml
index 4ca4c9ed8f..71291a082d 100644
--- a/tests/components/ina219/common.yaml
+++ b/tests/components/ina219/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_ina219
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ina219
+    i2c_id: i2c_bus
     address: 0x40
     shunt_resistance: 0.1 ohm
     current:
diff --git a/tests/components/ina219/test.esp32-ard.yaml b/tests/components/ina219/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ina219/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ina219/test.esp32-c3-ard.yaml b/tests/components/ina219/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ina219/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ina219/test.esp32-c3-idf.yaml b/tests/components/ina219/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ina219/test.esp32-c3-idf.yaml
+++ b/tests/components/ina219/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina219/test.esp32-idf.yaml b/tests/components/ina219/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ina219/test.esp32-idf.yaml
+++ b/tests/components/ina219/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina219/test.esp8266-ard.yaml b/tests/components/ina219/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ina219/test.esp8266-ard.yaml
+++ b/tests/components/ina219/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina219/test.rp2040-ard.yaml b/tests/components/ina219/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ina219/test.rp2040-ard.yaml
+++ b/tests/components/ina219/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina226/common.yaml b/tests/components/ina226/common.yaml
index 8bcd038e50..7cbaf8cb2f 100644
--- a/tests/components/ina226/common.yaml
+++ b/tests/components/ina226/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_ina226
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ina226
+    i2c_id: i2c_bus
     address: 0x40
     shunt_resistance: 0.1 ohm
     current:
diff --git a/tests/components/ina226/test.esp32-ard.yaml b/tests/components/ina226/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ina226/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ina226/test.esp32-c3-ard.yaml b/tests/components/ina226/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ina226/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ina226/test.esp32-c3-idf.yaml b/tests/components/ina226/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ina226/test.esp32-c3-idf.yaml
+++ b/tests/components/ina226/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina226/test.esp32-idf.yaml b/tests/components/ina226/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ina226/test.esp32-idf.yaml
+++ b/tests/components/ina226/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina226/test.esp8266-ard.yaml b/tests/components/ina226/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ina226/test.esp8266-ard.yaml
+++ b/tests/components/ina226/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina226/test.rp2040-ard.yaml b/tests/components/ina226/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ina226/test.rp2040-ard.yaml
+++ b/tests/components/ina226/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina260/common.yaml b/tests/components/ina260/common.yaml
index ab94b2e509..f630d0bb47 100644
--- a/tests/components/ina260/common.yaml
+++ b/tests/components/ina260/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_ina260
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ina260
+    i2c_id: i2c_bus
     address: 0x40
     current:
       name: INA260 Current
diff --git a/tests/components/ina260/test.esp32-ard.yaml b/tests/components/ina260/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ina260/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ina260/test.esp32-c3-ard.yaml b/tests/components/ina260/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ina260/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ina260/test.esp32-c3-idf.yaml b/tests/components/ina260/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ina260/test.esp32-c3-idf.yaml
+++ b/tests/components/ina260/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina260/test.esp32-idf.yaml b/tests/components/ina260/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ina260/test.esp32-idf.yaml
+++ b/tests/components/ina260/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina260/test.esp8266-ard.yaml b/tests/components/ina260/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ina260/test.esp8266-ard.yaml
+++ b/tests/components/ina260/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina260/test.rp2040-ard.yaml b/tests/components/ina260/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ina260/test.rp2040-ard.yaml
+++ b/tests/components/ina260/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina2xx_i2c/common.yaml b/tests/components/ina2xx_i2c/common.yaml
index 320b680b6b..748ab94c98 100644
--- a/tests/components/ina2xx_i2c/common.yaml
+++ b/tests/components/ina2xx_i2c/common.yaml
@@ -1,20 +1,28 @@
-i2c:
-  - id: i2c_ina2xx
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ina2xx_i2c
-    i2c_id: i2c_ina2xx
+    i2c_id: i2c_bus
     address: 0x40
     model: INA228
     shunt_resistance: 0.001130 ohm
     max_current: 40 A
     adc_range: 1
     temperature_coefficient: 50
-    shunt_voltage: "INA2xx Shunt Voltage"
-    bus_voltage: "INA2xx Bus Voltage"
-    current: "INA2xx Current"
-    power: "INA2xx Power"
-    energy: "INA2xx Energy"
-    charge: "INA2xx Charge"
+    reset_on_boot: true
+    shunt_voltage:
+      id: ina2xx_i2c_shunt_voltage
+      name: "INA2xx Shunt Voltage"
+    bus_voltage:
+      id: ina2xx_i2c_bus_voltage
+      name: "INA2xx Bus Voltage"
+    current:
+      id: ina2xx_i2c_current
+      name: "INA2xx Current"
+    power:
+      id: ina2xx_i2c_power
+      name: "INA2xx Power"
+    energy:
+      id: ina2xx_i2c_energy
+      name: "INA2xx Energy"
+    charge:
+      id: ina2xx_i2c_charge
+      name: "INA2xx Charge"
diff --git a/tests/components/ina2xx_i2c/test.esp32-ard.yaml b/tests/components/ina2xx_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ina2xx_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ina2xx_i2c/test.esp32-c3-ard.yaml b/tests/components/ina2xx_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ina2xx_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ina2xx_i2c/test.esp32-c3-idf.yaml b/tests/components/ina2xx_i2c/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ina2xx_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/ina2xx_i2c/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina2xx_i2c/test.esp32-idf.yaml b/tests/components/ina2xx_i2c/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ina2xx_i2c/test.esp32-idf.yaml
+++ b/tests/components/ina2xx_i2c/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina2xx_i2c/test.esp8266-ard.yaml b/tests/components/ina2xx_i2c/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ina2xx_i2c/test.esp8266-ard.yaml
+++ b/tests/components/ina2xx_i2c/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina2xx_i2c/test.rp2040-ard.yaml b/tests/components/ina2xx_i2c/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ina2xx_i2c/test.rp2040-ard.yaml
+++ b/tests/components/ina2xx_i2c/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina2xx_spi/common.yaml b/tests/components/ina2xx_spi/common.yaml
index 3eab7e6f0a..8de77eba26 100644
--- a/tests/components/ina2xx_spi/common.yaml
+++ b/tests/components/ina2xx_spi/common.yaml
@@ -1,21 +1,26 @@
-spi:
-  - id: spi_ina2xx
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: ina2xx_spi
-    spi_id: spi_ina2xx
     cs_pin: ${cs_pin}
     model: INA229
     shunt_resistance: 0.001130 ohm
     max_current: 40 A
     adc_range: 1
     temperature_coefficient: 50
-    shunt_voltage: "INA2xx Shunt Voltage"
-    bus_voltage: "INA2xx Bus Voltage"
-    current: "INA2xx Current"
-    power: "INA2xx Power"
-    energy: "INA2xx Energy"
-    charge: "INA2xx Charge"
+    shunt_voltage:
+      id: ina2xx_spi_shunt_voltage
+      name: "INA2xx Shunt Voltage"
+    bus_voltage:
+      id: ina2xx_spi_bus_voltage
+      name: "INA2xx Bus Voltage"
+    current:
+      id: ina2xx_spi_current
+      name: "INA2xx Current"
+    power:
+      id: ina2xx_spi_power
+      name: "INA2xx Power"
+    energy:
+      id: ina2xx_spi_energy
+      name: "INA2xx Energy"
+    charge:
+      id: ina2xx_spi_charge
+      name: "INA2xx Charge"
diff --git a/tests/components/ina2xx_spi/test.esp32-ard.yaml b/tests/components/ina2xx_spi/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/ina2xx_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/ina2xx_spi/test.esp32-c3-ard.yaml b/tests/components/ina2xx_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/ina2xx_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/ina2xx_spi/test.esp32-c3-idf.yaml b/tests/components/ina2xx_spi/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/ina2xx_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/ina2xx_spi/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina2xx_spi/test.esp32-idf.yaml b/tests/components/ina2xx_spi/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/ina2xx_spi/test.esp32-idf.yaml
+++ b/tests/components/ina2xx_spi/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ina2xx_spi/test.esp8266-ard.yaml b/tests/components/ina2xx_spi/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/ina2xx_spi/test.esp8266-ard.yaml
+++ b/tests/components/ina2xx_spi/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ina2xx_spi/test.rp2040-ard.yaml b/tests/components/ina2xx_spi/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/ina2xx_spi/test.rp2040-ard.yaml
+++ b/tests/components/ina2xx_spi/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ina3221/common.yaml b/tests/components/ina3221/common.yaml
index ba1abdfe3a..570d1b0a12 100644
--- a/tests/components/ina3221/common.yaml
+++ b/tests/components/ina3221/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_ina3221
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ina3221
+    i2c_id: i2c_bus
     address: 0x40
     channel_1:
       shunt_resistance: 0.1 ohm
diff --git a/tests/components/ina3221/test.esp32-ard.yaml b/tests/components/ina3221/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ina3221/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ina3221/test.esp32-c3-ard.yaml b/tests/components/ina3221/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ina3221/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ina3221/test.esp32-c3-idf.yaml b/tests/components/ina3221/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ina3221/test.esp32-c3-idf.yaml
+++ b/tests/components/ina3221/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina3221/test.esp32-idf.yaml b/tests/components/ina3221/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ina3221/test.esp32-idf.yaml
+++ b/tests/components/ina3221/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina3221/test.esp8266-ard.yaml b/tests/components/ina3221/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ina3221/test.esp8266-ard.yaml
+++ b/tests/components/ina3221/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ina3221/test.rp2040-ard.yaml b/tests/components/ina3221/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ina3221/test.rp2040-ard.yaml
+++ b/tests/components/ina3221/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/inkbird_ibsth1_mini/test.esp32-ard.yaml b/tests/components/inkbird_ibsth1_mini/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/inkbird_ibsth1_mini/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/inkbird_ibsth1_mini/test.esp32-c3-ard.yaml b/tests/components/inkbird_ibsth1_mini/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/inkbird_ibsth1_mini/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/inkbird_ibsth1_mini/test.esp32-c3-idf.yaml b/tests/components/inkbird_ibsth1_mini/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/inkbird_ibsth1_mini/test.esp32-c3-idf.yaml
+++ b/tests/components/inkbird_ibsth1_mini/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/inkbird_ibsth1_mini/test.esp32-idf.yaml b/tests/components/inkbird_ibsth1_mini/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/inkbird_ibsth1_mini/test.esp32-idf.yaml
+++ b/tests/components/inkbird_ibsth1_mini/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/inkplate6/common.yaml b/tests/components/inkplate/common.yaml
similarity index 94%
rename from tests/components/inkplate6/common.yaml
rename to tests/components/inkplate/common.yaml
index 6cb5d055b6..bb18ff4f7e 100644
--- a/tests/components/inkplate6/common.yaml
+++ b/tests/components/inkplate/common.yaml
@@ -1,13 +1,9 @@
-i2c:
-  - id: i2c_inkplate6
-    scl: 16
-    sda: 17
-
 esp32:
   cpu_frequency: 240MHz
 
 display:
-  - platform: inkplate6
+  - platform: inkplate
+    i2c_id: i2c_bus
     id: inkplate_display
     greyscale: false
     partial_updating: false
diff --git a/tests/components/inkplate/test.esp32-idf.yaml b/tests/components/inkplate/test.esp32-idf.yaml
new file mode 100644
index 0000000000..b47e39c389
--- /dev/null
+++ b/tests/components/inkplate/test.esp32-idf.yaml
@@ -0,0 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/inkplate6/test.esp32-ard.yaml b/tests/components/inkplate6/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/inkplate6/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/integration/test.esp32-ard.yaml b/tests/components/integration/test.esp32-ard.yaml
deleted file mode 100644
index f84495e521..0000000000
--- a/tests/components/integration/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: A0
-
-<<: !include common-esp32.yaml
diff --git a/tests/components/integration/test.esp32-c3-ard.yaml b/tests/components/integration/test.esp32-c3-ard.yaml
deleted file mode 100644
index 5105e645f3..0000000000
--- a/tests/components/integration/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO1
-
-<<: !include common-esp32.yaml
diff --git a/tests/components/internal_temperature/test.esp32-ard.yaml b/tests/components/internal_temperature/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/internal_temperature/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/internal_temperature/test.esp32-c3-ard.yaml b/tests/components/internal_temperature/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/internal_temperature/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/internal_temperature/test.esp32-s2-ard.yaml b/tests/components/internal_temperature/test.esp32-s2-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/internal_temperature/test.esp32-s2-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/internal_temperature/test.esp32-s3-ard.yaml b/tests/components/internal_temperature/test.esp32-s3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/internal_temperature/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/interval/test.esp32-ard.yaml b/tests/components/interval/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/interval/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/interval/test.esp32-c3-ard.yaml b/tests/components/interval/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/interval/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/bedjet/test.esp32-c3-ard.yaml b/tests/components/interval/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/bedjet/test.esp32-c3-ard.yaml
rename to tests/components/interval/test.nrf52-adafruit.yaml
diff --git a/tests/components/binary_sensor_map/test.esp32-ard.yaml b/tests/components/interval/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/binary_sensor_map/test.esp32-ard.yaml
rename to tests/components/interval/test.nrf52-mcumgr.yaml
diff --git a/tests/components/jsn_sr04t/common.yaml b/tests/components/jsn_sr04t/common.yaml
index d6871d5e91..bc2c301c2e 100644
--- a/tests/components/jsn_sr04t/common.yaml
+++ b/tests/components/jsn_sr04t/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_jsn_sr04t
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: jsn_sr04t
     id: jsn_sr04t_sensor
diff --git a/tests/components/jsn_sr04t/test.esp32-ard.yaml b/tests/components/jsn_sr04t/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/jsn_sr04t/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/jsn_sr04t/test.esp32-c3-ard.yaml b/tests/components/jsn_sr04t/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/jsn_sr04t/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/jsn_sr04t/test.esp32-c3-idf.yaml b/tests/components/jsn_sr04t/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/jsn_sr04t/test.esp32-c3-idf.yaml
+++ b/tests/components/jsn_sr04t/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/jsn_sr04t/test.esp32-idf.yaml b/tests/components/jsn_sr04t/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/jsn_sr04t/test.esp32-idf.yaml
+++ b/tests/components/jsn_sr04t/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/jsn_sr04t/test.esp8266-ard.yaml b/tests/components/jsn_sr04t/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/jsn_sr04t/test.esp8266-ard.yaml
+++ b/tests/components/jsn_sr04t/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/jsn_sr04t/test.rp2040-ard.yaml b/tests/components/jsn_sr04t/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/jsn_sr04t/test.rp2040-ard.yaml
+++ b/tests/components/jsn_sr04t/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/kamstrup_kmp/common.yaml b/tests/components/kamstrup_kmp/common.yaml
index b348d03c72..b3ebea523c 100644
--- a/tests/components/kamstrup_kmp/common.yaml
+++ b/tests/components/kamstrup_kmp/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  tx_pin: ${uart_tx_pin}
-  rx_pin: ${uart_rx_pin}
-  baud_rate: 1200
-  stop_bits: 2
-
 sensor:
   - platform: kamstrup_kmp
     heat_energy:
diff --git a/tests/components/kamstrup_kmp/test.esp32-ard.yaml b/tests/components/kamstrup_kmp/test.esp32-ard.yaml
deleted file mode 100644
index adc2c4d24a..0000000000
--- a/tests/components/kamstrup_kmp/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  uart_tx_pin: GPIO1
-  uart_rx_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/kamstrup_kmp/test.esp32-idf.yaml b/tests/components/kamstrup_kmp/test.esp32-idf.yaml
index adc2c4d24a..1016905720 100644
--- a/tests/components/kamstrup_kmp/test.esp32-idf.yaml
+++ b/tests/components/kamstrup_kmp/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  uart_tx_pin: GPIO1
-  uart_rx_pin: GPIO3
+packages:
+  uart_1200: !include ../../test_build_components/common/uart_1200/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/kamstrup_kmp/test.esp8266-ard.yaml b/tests/components/kamstrup_kmp/test.esp8266-ard.yaml
index adc2c4d24a..f55c18eb76 100644
--- a/tests/components/kamstrup_kmp/test.esp8266-ard.yaml
+++ b/tests/components/kamstrup_kmp/test.esp8266-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   uart_tx_pin: GPIO1
   uart_rx_pin: GPIO3
 
+packages:
+  uart_1200: !include ../../test_build_components/common/uart_1200/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/key_collector/test.esp32-ard.yaml b/tests/components/key_collector/test.esp32-ard.yaml
deleted file mode 100644
index de144aa46b..0000000000
--- a/tests/components/key_collector/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  pin_r0: GPIO12
-  pin_r1: GPIO13
-  pin_c0: GPIO14
-  pin_c1: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/key_collector/test.esp32-c3-ard.yaml b/tests/components/key_collector/test.esp32-c3-ard.yaml
deleted file mode 100644
index b580ab7843..0000000000
--- a/tests/components/key_collector/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  pin_r0: GPIO2
-  pin_r1: GPIO3
-  pin_c0: GPIO4
-  pin_c1: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/kmeteriso/common.yaml b/tests/components/kmeteriso/common.yaml
index 6b68175904..8542bb6a06 100644
--- a/tests/components/kmeteriso/common.yaml
+++ b/tests/components/kmeteriso/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_kmeteriso
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: kmeteriso
+    i2c_id: i2c_bus
     temperature:
       name: Outside Temperature
     internal_temperature:
diff --git a/tests/components/kmeteriso/test.esp32-ard.yaml b/tests/components/kmeteriso/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/kmeteriso/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/kmeteriso/test.esp32-c3-ard.yaml b/tests/components/kmeteriso/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/kmeteriso/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/kmeteriso/test.esp32-c3-idf.yaml b/tests/components/kmeteriso/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/kmeteriso/test.esp32-c3-idf.yaml
+++ b/tests/components/kmeteriso/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/kmeteriso/test.esp32-idf.yaml b/tests/components/kmeteriso/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/kmeteriso/test.esp32-idf.yaml
+++ b/tests/components/kmeteriso/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/kmeteriso/test.esp8266-ard.yaml b/tests/components/kmeteriso/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/kmeteriso/test.esp8266-ard.yaml
+++ b/tests/components/kmeteriso/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/kmeteriso/test.rp2040-ard.yaml b/tests/components/kmeteriso/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/kmeteriso/test.rp2040-ard.yaml
+++ b/tests/components/kmeteriso/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/kuntze/common.yaml b/tests/components/kuntze/common.yaml
index 4daecea242..32c6fa3809 100644
--- a/tests/components/kuntze/common.yaml
+++ b/tests/components/kuntze/common.yaml
@@ -1,14 +1,6 @@
-uart:
-  - id: uart_kuntze
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
-modbus:
-  flow_control_pin: ${flow_control_pin}
-
 sensor:
   - platform: kuntze
+    modbus_id: modbus_bus
     ph:
       name: Kuntze pH
     temperature:
diff --git a/tests/components/kuntze/test.esp32-ard.yaml b/tests/components/kuntze/test.esp32-ard.yaml
deleted file mode 100644
index bd767a8ece..0000000000
--- a/tests/components/kuntze/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-  flow_control_pin: GPIO13
-
-<<: !include common.yaml
diff --git a/tests/components/kuntze/test.esp32-c3-ard.yaml b/tests/components/kuntze/test.esp32-c3-ard.yaml
deleted file mode 100644
index 452031a5aa..0000000000
--- a/tests/components/kuntze/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  flow_control_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/kuntze/test.esp32-c3-idf.yaml b/tests/components/kuntze/test.esp32-c3-idf.yaml
index 452031a5aa..17940aafcf 100644
--- a/tests/components/kuntze/test.esp32-c3-idf.yaml
+++ b/tests/components/kuntze/test.esp32-c3-idf.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
   flow_control_pin: GPIO3
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/kuntze/test.esp32-idf.yaml b/tests/components/kuntze/test.esp32-idf.yaml
index bd767a8ece..a755bfa66a 100644
--- a/tests/components/kuntze/test.esp32-idf.yaml
+++ b/tests/components/kuntze/test.esp32-idf.yaml
@@ -3,4 +3,7 @@ substitutions:
   rx_pin: GPIO14
   flow_control_pin: GPIO13
 
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/kuntze/test.esp8266-ard.yaml b/tests/components/kuntze/test.esp8266-ard.yaml
index 29c98d0957..6daa08c22b 100644
--- a/tests/components/kuntze/test.esp8266-ard.yaml
+++ b/tests/components/kuntze/test.esp8266-ard.yaml
@@ -1,6 +1,9 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  flow_control_pin: GPIO13
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+  flow_control_pin: GPIO15
+
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/kuntze/test.rp2040-ard.yaml b/tests/components/kuntze/test.rp2040-ard.yaml
index 452031a5aa..a00283a9e0 100644
--- a/tests/components/kuntze/test.rp2040-ard.yaml
+++ b/tests/components/kuntze/test.rp2040-ard.yaml
@@ -3,4 +3,7 @@ substitutions:
   rx_pin: GPIO5
   flow_control_pin: GPIO3
 
+packages:
+  modbus: !include ../../test_build_components/common/modbus/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/lc709203f/common.yaml b/tests/components/lc709203f/common.yaml
index 53177c0d4a..3711e5372c 100644
--- a/tests/components/lc709203f/common.yaml
+++ b/tests/components/lc709203f/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_lc709203f
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: lc709203f
+    i2c_id: i2c_bus
     size: 2000
     voltage: 3.7
     battery_voltage:
diff --git a/tests/components/lc709203f/test.esp32-ard.yaml b/tests/components/lc709203f/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/lc709203f/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/lc709203f/test.esp32-c3-ard.yaml b/tests/components/lc709203f/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/lc709203f/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/lc709203f/test.esp32-c3-idf.yaml b/tests/components/lc709203f/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/lc709203f/test.esp32-c3-idf.yaml
+++ b/tests/components/lc709203f/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/lc709203f/test.esp32-idf.yaml b/tests/components/lc709203f/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/lc709203f/test.esp32-idf.yaml
+++ b/tests/components/lc709203f/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/lc709203f/test.esp8266-ard.yaml b/tests/components/lc709203f/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/lc709203f/test.esp8266-ard.yaml
+++ b/tests/components/lc709203f/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/lc709203f/test.rp2040-ard.yaml b/tests/components/lc709203f/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/lc709203f/test.rp2040-ard.yaml
+++ b/tests/components/lc709203f/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/lcd_gpio/test.esp32-ard.yaml b/tests/components/lcd_gpio/test.esp32-ard.yaml
deleted file mode 100644
index 9c2af456b5..0000000000
--- a/tests/components/lcd_gpio/test.esp32-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  d0_pin: GPIO12
-  d1_pin: GPIO13
-  d2_pin: GPIO14
-  d3_pin: GPIO15
-  enable_pin: GPIO16
-  rs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/lcd_gpio/test.esp32-c3-ard.yaml b/tests/components/lcd_gpio/test.esp32-c3-ard.yaml
deleted file mode 100644
index b6b05f3ab4..0000000000
--- a/tests/components/lcd_gpio/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  d0_pin: GPIO1
-  d1_pin: GPIO2
-  d2_pin: GPIO3
-  d3_pin: GPIO4
-  enable_pin: GPIO5
-  rs_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/lcd_gpio/test.esp32-idf.yaml b/tests/components/lcd_gpio/test.esp32-idf.yaml
index 9c2af456b5..93abad5e38 100644
--- a/tests/components/lcd_gpio/test.esp32-idf.yaml
+++ b/tests/components/lcd_gpio/test.esp32-idf.yaml
@@ -3,7 +3,7 @@ substitutions:
   d1_pin: GPIO13
   d2_pin: GPIO14
   d3_pin: GPIO15
-  enable_pin: GPIO16
+  enable_pin: GPIO4
   rs_pin: GPIO5
 
 <<: !include common.yaml
diff --git a/tests/components/lcd_gpio/test.esp8266-ard.yaml b/tests/components/lcd_gpio/test.esp8266-ard.yaml
index 9c2af456b5..50471a16f4 100644
--- a/tests/components/lcd_gpio/test.esp8266-ard.yaml
+++ b/tests/components/lcd_gpio/test.esp8266-ard.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  d0_pin: GPIO12
-  d1_pin: GPIO13
+  d0_pin: GPIO0
+  d1_pin: GPIO2
   d2_pin: GPIO14
   d3_pin: GPIO15
   enable_pin: GPIO16
diff --git a/tests/components/lcd_menu/test.esp32-ard.yaml b/tests/components/lcd_menu/test.esp32-ard.yaml
deleted file mode 100644
index 9c2af456b5..0000000000
--- a/tests/components/lcd_menu/test.esp32-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  d0_pin: GPIO12
-  d1_pin: GPIO13
-  d2_pin: GPIO14
-  d3_pin: GPIO15
-  enable_pin: GPIO16
-  rs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/lcd_menu/test.esp32-c3-ard.yaml b/tests/components/lcd_menu/test.esp32-c3-ard.yaml
deleted file mode 100644
index b6b05f3ab4..0000000000
--- a/tests/components/lcd_menu/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  d0_pin: GPIO1
-  d1_pin: GPIO2
-  d2_pin: GPIO3
-  d3_pin: GPIO4
-  enable_pin: GPIO5
-  rs_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/lcd_menu/test.esp32-idf.yaml b/tests/components/lcd_menu/test.esp32-idf.yaml
index 9c2af456b5..93abad5e38 100644
--- a/tests/components/lcd_menu/test.esp32-idf.yaml
+++ b/tests/components/lcd_menu/test.esp32-idf.yaml
@@ -3,7 +3,7 @@ substitutions:
   d1_pin: GPIO13
   d2_pin: GPIO14
   d3_pin: GPIO15
-  enable_pin: GPIO16
+  enable_pin: GPIO4
   rs_pin: GPIO5
 
 <<: !include common.yaml
diff --git a/tests/components/lcd_menu/test.esp8266-ard.yaml b/tests/components/lcd_menu/test.esp8266-ard.yaml
index 9c2af456b5..50471a16f4 100644
--- a/tests/components/lcd_menu/test.esp8266-ard.yaml
+++ b/tests/components/lcd_menu/test.esp8266-ard.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  d0_pin: GPIO12
-  d1_pin: GPIO13
+  d0_pin: GPIO0
+  d1_pin: GPIO2
   d2_pin: GPIO14
   d3_pin: GPIO15
   enable_pin: GPIO16
diff --git a/tests/components/lcd_pcf8574/common.yaml b/tests/components/lcd_pcf8574/common.yaml
index 8b03cf5497..1ec4400332 100644
--- a/tests/components/lcd_pcf8574/common.yaml
+++ b/tests/components/lcd_pcf8574/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_lcd_pcf8574
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 display:
   - platform: lcd_pcf8574
+    i2c_id: i2c_bus
     dimensions: 18x4
     address: 0x3F
     user_characters:
diff --git a/tests/components/lcd_pcf8574/test.esp32-ard.yaml b/tests/components/lcd_pcf8574/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/lcd_pcf8574/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/lcd_pcf8574/test.esp32-c3-ard.yaml b/tests/components/lcd_pcf8574/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/lcd_pcf8574/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/lcd_pcf8574/test.esp32-c3-idf.yaml b/tests/components/lcd_pcf8574/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/lcd_pcf8574/test.esp32-c3-idf.yaml
+++ b/tests/components/lcd_pcf8574/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/lcd_pcf8574/test.esp32-idf.yaml b/tests/components/lcd_pcf8574/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/lcd_pcf8574/test.esp32-idf.yaml
+++ b/tests/components/lcd_pcf8574/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/lcd_pcf8574/test.esp8266-ard.yaml b/tests/components/lcd_pcf8574/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/lcd_pcf8574/test.esp8266-ard.yaml
+++ b/tests/components/lcd_pcf8574/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/lcd_pcf8574/test.rp2040-ard.yaml b/tests/components/lcd_pcf8574/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/lcd_pcf8574/test.rp2040-ard.yaml
+++ b/tests/components/lcd_pcf8574/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ld2410/common.yaml b/tests/components/ld2410/common.yaml
index e975fcf757..7d168bf7ec 100644
--- a/tests/components/ld2410/common.yaml
+++ b/tests/components/ld2410/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_ld2410
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 ld2410:
   id: my_ld2410
 
diff --git a/tests/components/ld2410/test.esp32-ard.yaml b/tests/components/ld2410/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/ld2410/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/ld2410/test.esp32-c3-ard.yaml b/tests/components/ld2410/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/ld2410/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/ld2410/test.esp32-c3-idf.yaml b/tests/components/ld2410/test.esp32-c3-idf.yaml
index b516342f3b..7a8f790ed8 100644
--- a/tests/components/ld2410/test.esp32-c3-idf.yaml
+++ b/tests/components/ld2410/test.esp32-c3-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ld2410/test.esp32-idf.yaml b/tests/components/ld2410/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/ld2410/test.esp32-idf.yaml
+++ b/tests/components/ld2410/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ld2410/test.esp8266-ard.yaml b/tests/components/ld2410/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/ld2410/test.esp8266-ard.yaml
+++ b/tests/components/ld2410/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ld2410/test.rp2040-ard.yaml b/tests/components/ld2410/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/ld2410/test.rp2040-ard.yaml
+++ b/tests/components/ld2410/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ld2412/common.yaml b/tests/components/ld2412/common.yaml
new file mode 100644
index 0000000000..18c4612ffe
--- /dev/null
+++ b/tests/components/ld2412/common.yaml
@@ -0,0 +1,227 @@
+ld2412:
+  id: my_ld2412
+
+binary_sensor:
+  - platform: ld2412
+    dynamic_background_correction_status:
+      name: Dynamic Background Correction Status
+    has_target:
+      name: Presence
+    has_moving_target:
+      name: Moving Target
+    has_still_target:
+      name: Still Target
+
+button:
+  - platform: ld2412
+    factory_reset:
+      name: Factory reset
+    restart:
+      name: Restart
+    query_params:
+      name: Query params
+    start_dynamic_background_correction:
+      name: Start Dynamic Background Correction
+
+number:
+  - platform: ld2412
+    light_threshold:
+      name: Light Threshold
+    timeout:
+      name: Presence timeout
+    min_distance_gate:
+      name: Minimum distance gate
+    max_distance_gate:
+      name: Maximum distance gate
+    gate_0:
+      move_threshold:
+        name: Gate 0 Move Threshold
+      still_threshold:
+        name: Gate 0 Still Threshold
+    gate_1:
+      move_threshold:
+        name: Gate 1 Move Threshold
+      still_threshold:
+        name: Gate 1 Still Threshold
+    gate_2:
+      move_threshold:
+        name: Gate 2 Move Threshold
+      still_threshold:
+        name: Gate 2 Still Threshold
+    gate_3:
+      move_threshold:
+        name: Gate 3 Move Threshold
+      still_threshold:
+        name: Gate 3 Still Threshold
+    gate_4:
+      move_threshold:
+        name: Gate 4 Move Threshold
+      still_threshold:
+        name: Gate 4 Still Threshold
+    gate_5:
+      move_threshold:
+        name: Gate 5 Move Threshold
+      still_threshold:
+        name: Gate 5 Still Threshold
+    gate_6:
+      move_threshold:
+        name: Gate 6 Move Threshold
+      still_threshold:
+        name: Gate 6 Still Threshold
+    gate_7:
+      move_threshold:
+        name: Gate 7 Move Threshold
+      still_threshold:
+        name: Gate 7 Still Threshold
+    gate_8:
+      move_threshold:
+        name: Gate 8 Move Threshold
+      still_threshold:
+        name: Gate 8 Still Threshold
+    gate_9:
+      move_threshold:
+        name: Gate 9 Move Threshold
+      still_threshold:
+        name: Gate 9 Still Threshold
+    gate_10:
+      move_threshold:
+        name: Gate 10 Move Threshold
+      still_threshold:
+        name: Gate 10 Still Threshold
+    gate_11:
+      move_threshold:
+        name: Gate 11 Move Threshold
+      still_threshold:
+        name: Gate 11 Still Threshold
+    gate_12:
+      move_threshold:
+        name: Gate 12 Move Threshold
+      still_threshold:
+        name: Gate 12 Still Threshold
+    gate_13:
+      move_threshold:
+        name: Gate 13 Move Threshold
+      still_threshold:
+        name: Gate 13 Still Threshold
+
+select:
+  - platform: ld2412
+    light_function:
+      name: Light Function
+    out_pin_level:
+      name: Hardware output pin level
+    distance_resolution:
+      name: Distance resolution
+    baud_rate:
+      name: Baud rate
+      on_value:
+        - delay: 3s
+        - lambda: |-
+            id(uart_bus).flush();
+            uint32_t new_baud_rate = stoi(x);
+            ESP_LOGD("change_baud_rate", "Changing baud rate from %i to %i",id(uart_bus).get_baud_rate(), new_baud_rate);
+            if (id(uart_bus).get_baud_rate() != new_baud_rate) {
+              id(uart_bus).set_baud_rate(new_baud_rate);
+            #if defined(USE_ESP8266) || defined(USE_ESP32)
+              id(uart_bus).load_settings();
+            #endif
+            }
+
+sensor:
+  - platform: ld2412
+    light:
+      name: Light
+    moving_distance:
+      name: Moving Distance
+    still_distance:
+      name: Still Distance
+    moving_energy:
+      name: Move Energy
+    still_energy:
+      name: Still Energy
+    detection_distance:
+      name: Detection Distance
+    gate_0:
+      move_energy:
+        name: Gate 0 Move Energy
+      still_energy:
+        name: Gate 0 Still Energy
+    gate_1:
+      move_energy:
+        name: Gate 1 Move Energy
+      still_energy:
+        name: Gate 1 Still Energy
+    gate_2:
+      move_energy:
+        name: Gate 2 Move Energy
+      still_energy:
+        name: Gate 2 Still Energy
+    gate_3:
+      move_energy:
+        name: Gate 3 Move Energy
+      still_energy:
+        name: Gate 3 Still Energy
+    gate_4:
+      move_energy:
+        name: Gate 4 Move Energy
+      still_energy:
+        name: Gate 4 Still Energy
+    gate_5:
+      move_energy:
+        name: Gate 5 Move Energy
+      still_energy:
+        name: Gate 5 Still Energy
+    gate_6:
+      move_energy:
+        name: Gate 6 Move Energy
+      still_energy:
+        name: Gate 6 Still Energy
+    gate_7:
+      move_energy:
+        name: Gate 7 Move Energy
+      still_energy:
+        name: Gate 7 Still Energy
+    gate_8:
+      move_energy:
+        name: Gate 8 Move Energy
+      still_energy:
+        name: Gate 8 Still Energy
+    gate_9:
+      move_energy:
+        name: Gate 9 Move Energy
+      still_energy:
+        name: Gate 9 Still Energy
+    gate_10:
+      move_energy:
+        name: Gate 10 Move Energy
+      still_energy:
+        name: Gate 10 Still Energy
+    gate_11:
+      move_energy:
+        name: Gate 11 Move Energy
+      still_energy:
+        name: Gate 11 Still Energy
+    gate_12:
+      move_energy:
+        name: Gate 12 Move Energy
+      still_energy:
+        name: Gate 12 Still Energy
+    gate_13:
+      move_energy:
+        name: Gate 13 Move Energy
+      still_energy:
+        name: Gate 13 Still Energy
+
+switch:
+  - platform: ld2412
+    bluetooth:
+      name: Bluetooth
+    engineering_mode:
+      name: Engineering Mode
+
+text_sensor:
+  - platform: ld2412
+    version:
+      name: Firmware version
+    mac_address:
+      name: MAC address
diff --git a/tests/components/ld2412/test.esp32-c3-idf.yaml b/tests/components/ld2412/test.esp32-c3-idf.yaml
new file mode 100644
index 0000000000..7a8f790ed8
--- /dev/null
+++ b/tests/components/ld2412/test.esp32-c3-idf.yaml
@@ -0,0 +1,8 @@
+substitutions:
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/ld2412/test.esp32-idf.yaml b/tests/components/ld2412/test.esp32-idf.yaml
new file mode 100644
index 0000000000..2d29656c94
--- /dev/null
+++ b/tests/components/ld2412/test.esp32-idf.yaml
@@ -0,0 +1,4 @@
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/ld2412/test.esp8266-ard.yaml b/tests/components/ld2412/test.esp8266-ard.yaml
new file mode 100644
index 0000000000..96ab4ef6ac
--- /dev/null
+++ b/tests/components/ld2412/test.esp8266-ard.yaml
@@ -0,0 +1,8 @@
+substitutions:
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/ld2412/test.rp2040-ard.yaml b/tests/components/ld2412/test.rp2040-ard.yaml
new file mode 100644
index 0000000000..b28f2b5e05
--- /dev/null
+++ b/tests/components/ld2412/test.rp2040-ard.yaml
@@ -0,0 +1,8 @@
+substitutions:
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/ld2420/common.yaml b/tests/components/ld2420/common.yaml
index 76748aa8f0..1539df4d1b 100644
--- a/tests/components/ld2420/common.yaml
+++ b/tests/components/ld2420/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_ld2420
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 ld2420:
   id: my_ld2420
 
diff --git a/tests/components/ld2420/test.esp32-ard.yaml b/tests/components/ld2420/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/ld2420/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/ld2420/test.esp32-c3-ard.yaml b/tests/components/ld2420/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/ld2420/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/ld2420/test.esp32-c3-idf.yaml b/tests/components/ld2420/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/ld2420/test.esp32-c3-idf.yaml
+++ b/tests/components/ld2420/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ld2420/test.esp32-idf.yaml b/tests/components/ld2420/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/ld2420/test.esp32-idf.yaml
+++ b/tests/components/ld2420/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ld2420/test.esp8266-ard.yaml b/tests/components/ld2420/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/ld2420/test.esp8266-ard.yaml
+++ b/tests/components/ld2420/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ld2420/test.rp2040-ard.yaml b/tests/components/ld2420/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/ld2420/test.rp2040-ard.yaml
+++ b/tests/components/ld2420/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ld2450/common.yaml b/tests/components/ld2450/common.yaml
index 2e62efb0f5..9dcefffd09 100644
--- a/tests/components/ld2450/common.yaml
+++ b/tests/components/ld2450/common.yaml
@@ -1,15 +1,5 @@
-uart:
-  - id: ld2450_uart
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 256000
-    parity: NONE
-    stop_bits: 1
-
 ld2450:
   - id: ld2450_radar
-    uart_id: ld2450_uart
-    throttle: 1000ms
 
 button:
   - platform: ld2450
diff --git a/tests/components/ld2450/test.esp32-ard.yaml b/tests/components/ld2450/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/ld2450/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/ld2450/test.esp32-c3-ard.yaml b/tests/components/ld2450/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/ld2450/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/ld2450/test.esp32-c3-idf.yaml b/tests/components/ld2450/test.esp32-c3-idf.yaml
index b516342f3b..7a8f790ed8 100644
--- a/tests/components/ld2450/test.esp32-c3-idf.yaml
+++ b/tests/components/ld2450/test.esp32-c3-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ld2450/test.esp32-idf.yaml b/tests/components/ld2450/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/ld2450/test.esp32-idf.yaml
+++ b/tests/components/ld2450/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ld2450/test.esp8266-ard.yaml b/tests/components/ld2450/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/ld2450/test.esp8266-ard.yaml
+++ b/tests/components/ld2450/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ld2450/test.rp2040-ard.yaml b/tests/components/ld2450/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/ld2450/test.rp2040-ard.yaml
+++ b/tests/components/ld2450/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ledc/test.esp32-ard.yaml b/tests/components/ledc/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/ledc/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/ledc/test.esp32-c3-ard.yaml b/tests/components/ledc/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/ledc/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/light/test.esp32-ard.yaml b/tests/components/light/test.esp32-ard.yaml
deleted file mode 100644
index 925197182c..0000000000
--- a/tests/components/light/test.esp32-ard.yaml
+++ /dev/null
@@ -1,21 +0,0 @@
-output:
-  - platform: gpio
-    id: test_binary
-    pin: 12
-  - platform: ledc
-    id: test_ledc_1
-    pin: 13
-  - platform: ledc
-    id: test_ledc_2
-    pin: 14
-  - platform: ledc
-    id: test_ledc_3
-    pin: 15
-  - platform: ledc
-    id: test_ledc_4
-    pin: 16
-  - platform: ledc
-    id: test_ledc_5
-    pin: 17
-
-<<: !include common.yaml
diff --git a/tests/components/light/test.esp32-c3-ard.yaml b/tests/components/light/test.esp32-c3-ard.yaml
deleted file mode 100644
index 317d5748a3..0000000000
--- a/tests/components/light/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,21 +0,0 @@
-output:
-  - platform: gpio
-    id: test_binary
-    pin: 0
-  - platform: ledc
-    id: test_ledc_1
-    pin: 1
-  - platform: ledc
-    id: test_ledc_2
-    pin: 2
-  - platform: ledc
-    id: test_ledc_3
-    pin: 3
-  - platform: ledc
-    id: test_ledc_4
-    pin: 4
-  - platform: ledc
-    id: test_ledc_5
-    pin: 5
-
-<<: !include common.yaml
diff --git a/tests/components/light/test.nrf52-adafruit.yaml b/tests/components/light/test.nrf52-adafruit.yaml
new file mode 100644
index 0000000000..cb421ed4bb
--- /dev/null
+++ b/tests/components/light/test.nrf52-adafruit.yaml
@@ -0,0 +1,19 @@
+esphome:
+  on_boot:
+    then:
+      - light.toggle: test_binary_light
+
+output:
+  - platform: gpio
+    id: test_binary
+    pin: 0
+
+light:
+  - platform: binary
+    id: test_binary_light
+    name: Binary Light
+    output: test_binary
+    effects:
+      - strobe:
+    on_state:
+      - logger.log: Binary light state changed
diff --git a/tests/components/light/test.nrf52-mcumgr.yaml b/tests/components/light/test.nrf52-mcumgr.yaml
new file mode 100644
index 0000000000..cb421ed4bb
--- /dev/null
+++ b/tests/components/light/test.nrf52-mcumgr.yaml
@@ -0,0 +1,19 @@
+esphome:
+  on_boot:
+    then:
+      - light.toggle: test_binary_light
+
+output:
+  - platform: gpio
+    id: test_binary
+    pin: 0
+
+light:
+  - platform: binary
+    id: test_binary_light
+    name: Binary Light
+    output: test_binary
+    effects:
+      - strobe:
+    on_state:
+      - logger.log: Binary light state changed
diff --git a/tests/components/lilygo_t5_47/common.yaml b/tests/components/lilygo_t5_47/common.yaml
index f539c58d74..18f1ba10ae 100644
--- a/tests/components/lilygo_t5_47/common.yaml
+++ b/tests/components/lilygo_t5_47/common.yaml
@@ -1,23 +1,20 @@
-i2c:
-  - id: i2c_lilygo_t5_47
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 display:
   - platform: ssd1306_i2c
-    id: ssd1306_display
+    i2c_id: i2c_bus
+    id: ssd1306_i2c_display
     model: SSD1306_128X64
     reset_pin: ${reset_pin}
     pages:
-      - id: page1
+      - id: lilygo_t5_47_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
 
 touchscreen:
   - platform: lilygo_t5_47
+    i2c_id: i2c_bus
     id: lilygo_touchscreen
     interrupt_pin: ${interrupt_pin}
-    display: ssd1306_display
+    display: ssd1306_i2c_display
     on_touch:
       - logger.log:
           format: Touch at (%d, %d)
diff --git a/tests/components/lilygo_t5_47/test.esp32-ard.yaml b/tests/components/lilygo_t5_47/test.esp32-ard.yaml
deleted file mode 100644
index 342f0b6d8b..0000000000
--- a/tests/components/lilygo_t5_47/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  scl_pin: GPIO13
-  sda_pin: GPIO14
-  interrupt_pin: GPIO15
-  reset_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/lilygo_t5_47/test.esp32-c3-ard.yaml b/tests/components/lilygo_t5_47/test.esp32-c3-ard.yaml
deleted file mode 100644
index 061a98ce24..0000000000
--- a/tests/components/lilygo_t5_47/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  scl_pin: GPIO0
-  sda_pin: GPIO1
-  interrupt_pin: GPIO2
-  reset_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/lilygo_t5_47/test.esp32-c3-idf.yaml b/tests/components/lilygo_t5_47/test.esp32-c3-idf.yaml
index 061a98ce24..febf38d8e1 100644
--- a/tests/components/lilygo_t5_47/test.esp32-c3-idf.yaml
+++ b/tests/components/lilygo_t5_47/test.esp32-c3-idf.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  scl_pin: GPIO0
-  sda_pin: GPIO1
   interrupt_pin: GPIO2
   reset_pin: GPIO3
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/lilygo_t5_47/test.esp32-idf.yaml b/tests/components/lilygo_t5_47/test.esp32-idf.yaml
index 342f0b6d8b..590f6a919c 100644
--- a/tests/components/lilygo_t5_47/test.esp32-idf.yaml
+++ b/tests/components/lilygo_t5_47/test.esp32-idf.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  scl_pin: GPIO13
-  sda_pin: GPIO14
   interrupt_pin: GPIO15
-  reset_pin: GPIO16
+  reset_pin: GPIO4
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/lilygo_t5_47/test.esp8266-ard.yaml b/tests/components/lilygo_t5_47/test.esp8266-ard.yaml
index b446a75f13..3684d5e77b 100644
--- a/tests/components/lilygo_t5_47/test.esp8266-ard.yaml
+++ b/tests/components/lilygo_t5_47/test.esp8266-ard.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  interrupt_pin: GPIO12
+  interrupt_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/lilygo_t5_47/test.rp2040-ard.yaml b/tests/components/lilygo_t5_47/test.rp2040-ard.yaml
index 061a98ce24..2dd70ff33a 100644
--- a/tests/components/lilygo_t5_47/test.rp2040-ard.yaml
+++ b/tests/components/lilygo_t5_47/test.rp2040-ard.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  scl_pin: GPIO0
-  sda_pin: GPIO1
   interrupt_pin: GPIO2
   reset_pin: GPIO3
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/lm75b/common.yaml b/tests/components/lm75b/common.yaml
new file mode 100644
index 0000000000..39c39ed8dc
--- /dev/null
+++ b/tests/components/lm75b/common.yaml
@@ -0,0 +1,5 @@
+sensor:
+  - platform: lm75b
+    i2c_id: i2c_bus
+    name: LM75B Temperature
+    update_interval: 30s
diff --git a/tests/components/lm75b/test.esp32-c3-idf.yaml b/tests/components/lm75b/test.esp32-c3-idf.yaml
new file mode 100644
index 0000000000..9990d96d29
--- /dev/null
+++ b/tests/components/lm75b/test.esp32-c3-idf.yaml
@@ -0,0 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/lm75b/test.esp32-idf.yaml b/tests/components/lm75b/test.esp32-idf.yaml
new file mode 100644
index 0000000000..b47e39c389
--- /dev/null
+++ b/tests/components/lm75b/test.esp32-idf.yaml
@@ -0,0 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/lm75b/test.esp8266-ard.yaml b/tests/components/lm75b/test.esp8266-ard.yaml
new file mode 100644
index 0000000000..4a98b9388a
--- /dev/null
+++ b/tests/components/lm75b/test.esp8266-ard.yaml
@@ -0,0 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/lm75b/test.rp2040-ard.yaml b/tests/components/lm75b/test.rp2040-ard.yaml
new file mode 100644
index 0000000000..319a7c71a6
--- /dev/null
+++ b/tests/components/lm75b/test.rp2040-ard.yaml
@@ -0,0 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/lock/common.yaml b/tests/components/lock/common.yaml
index 67da46653c..9ba7f34857 100644
--- a/tests/components/lock/common.yaml
+++ b/tests/components/lock/common.yaml
@@ -17,9 +17,8 @@ lock:
     lambda: |-
       if (millis() > 10000) {
         return LOCK_STATE_LOCKED;
-      } else {
-        return LOCK_STATE_UNLOCKED;
       }
+      return LOCK_STATE_UNLOCKED;
     optimistic: true
     assumed_state: false
     on_unlock:
@@ -27,7 +26,9 @@ lock:
           id: test_lock1
           state: !lambda "return LOCK_STATE_UNLOCKED;"
     on_lock:
-      - lock.template.publish: LOCKED
+      - lock.template.publish:
+          id: test_lock1
+          state: LOCKED
   - platform: output
     name: Generic Output Lock
     id: test_lock2
diff --git a/tests/components/lock/test.esp32-ard.yaml b/tests/components/lock/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/lock/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/lock/test.esp32-c3-ard.yaml b/tests/components/lock/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/lock/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/binary_sensor_map/test.esp32-c3-ard.yaml b/tests/components/lock/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/binary_sensor_map/test.esp32-c3-ard.yaml
rename to tests/components/lock/test.nrf52-adafruit.yaml
diff --git a/tests/components/ble_client/test.esp32-ard.yaml b/tests/components/lock/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/ble_client/test.esp32-ard.yaml
rename to tests/components/lock/test.nrf52-mcumgr.yaml
diff --git a/tests/components/logger/common-default_uart.yaml b/tests/components/logger/common-default_uart.yaml
index e8b56043eb..7939a5f9c5 100644
--- a/tests/components/logger/common-default_uart.yaml
+++ b/tests/components/logger/common-default_uart.yaml
@@ -6,11 +6,16 @@ esphome:
           format: "Warning: Logger level is %d"
           args: [id(logger_id).get_log_level()]
       - logger.set_level: WARN
+      - logger.set_level:
+          level: ERROR
+          tag: mqtt.client
 
 logger:
   id: logger_id
   level: DEBUG
   initial_level: INFO
+  logs:
+    mqtt.component: WARN
 
 select:
   - platform: logger
diff --git a/tests/components/logger/test-on_message.host.yaml b/tests/components/logger/test-on_message.host.yaml
new file mode 100644
index 0000000000..12211a257b
--- /dev/null
+++ b/tests/components/logger/test-on_message.host.yaml
@@ -0,0 +1,18 @@
+logger:
+  id: logger_id
+  level: DEBUG
+  on_message:
+    - level: DEBUG
+      then:
+        - lambda: |-
+            ESP_LOGD("test", "Got message level %d: %s - %s", level, tag, message);
+    - level: WARN
+      then:
+        - lambda: |-
+            ESP_LOGW("test", "Warning level %d from %s", level, tag);
+    - level: ERROR
+      then:
+        - lambda: |-
+            // Test that level is uint8_t by using it in calculations
+            uint8_t adjusted_level = level + 1;
+            ESP_LOGE("test", "Error with adjusted level %d", adjusted_level);
diff --git a/tests/components/logger/test-usb_cdc.esp32-c3-ard.yaml b/tests/components/logger/test-usb_cdc.esp32-c3-ard.yaml
deleted file mode 100644
index cfdaec9771..0000000000
--- a/tests/components/logger/test-usb_cdc.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-usb_cdc.yaml
diff --git a/tests/components/improv_serial/test-usb_cdc.esp32-c3-ard.yaml b/tests/components/logger/test-usb_cdc.esp32-c3-idf.yaml
similarity index 100%
rename from tests/components/improv_serial/test-usb_cdc.esp32-c3-ard.yaml
rename to tests/components/logger/test-usb_cdc.esp32-c3-idf.yaml
diff --git a/tests/components/logger/test-usb_cdc.esp32-s2-ard.yaml b/tests/components/logger/test-usb_cdc.esp32-s2-ard.yaml
deleted file mode 100644
index cfdaec9771..0000000000
--- a/tests/components/logger/test-usb_cdc.esp32-s2-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-usb_cdc.yaml
diff --git a/tests/components/logger/test-usb_cdc.esp32-s3-ard.yaml b/tests/components/logger/test-usb_cdc.esp32-s3-ard.yaml
deleted file mode 100644
index cfdaec9771..0000000000
--- a/tests/components/logger/test-usb_cdc.esp32-s3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-usb_cdc.yaml
diff --git a/tests/components/improv_serial/test-usb_cdc.esp32-s2-ard.yaml b/tests/components/logger/test-usb_cdc.esp32-s3-idf.yaml
similarity index 100%
rename from tests/components/improv_serial/test-usb_cdc.esp32-s2-ard.yaml
rename to tests/components/logger/test-usb_cdc.esp32-s3-idf.yaml
diff --git a/tests/components/logger/test.esp32-ard.yaml b/tests/components/logger/test.esp32-ard.yaml
deleted file mode 100644
index 3fe04e18a3..0000000000
--- a/tests/components/logger/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-default_uart.yaml
diff --git a/tests/components/logger/test.esp32-c3-ard.yaml b/tests/components/logger/test.esp32-c3-ard.yaml
deleted file mode 100644
index 3fe04e18a3..0000000000
--- a/tests/components/logger/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-default_uart.yaml
diff --git a/tests/components/logger/test.nrf52-adafruit.yaml b/tests/components/logger/test.nrf52-adafruit.yaml
new file mode 100644
index 0000000000..70b485daac
--- /dev/null
+++ b/tests/components/logger/test.nrf52-adafruit.yaml
@@ -0,0 +1,7 @@
+esphome:
+  on_boot:
+    then:
+      - logger.log: Hello world
+
+logger:
+  level: DEBUG
diff --git a/tests/components/logger/test.nrf52-mcumgr.yaml b/tests/components/logger/test.nrf52-mcumgr.yaml
new file mode 100644
index 0000000000..70b485daac
--- /dev/null
+++ b/tests/components/logger/test.nrf52-mcumgr.yaml
@@ -0,0 +1,7 @@
+esphome:
+  on_boot:
+    then:
+      - logger.log: Hello world
+
+logger:
+  level: DEBUG
diff --git a/tests/components/lps22/common.yaml b/tests/components/lps22/common.yaml
index e6de4752ba..026b3620cd 100644
--- a/tests/components/lps22/common.yaml
+++ b/tests/components/lps22/common.yaml
@@ -1,5 +1,6 @@
 sensor:
   - platform: lps22
+    i2c_id: i2c_bus
     address: 0x5d
     update_interval: 10s
     temperature:
diff --git a/tests/components/lps22/test.esp32-ard.yaml b/tests/components/lps22/test.esp32-ard.yaml
deleted file mode 100644
index 0da6a9577e..0000000000
--- a/tests/components/lps22/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-i2c:
-  - id: i2c_lps22
-    scl: 16
-    sda: 17
-
-<<: !include common.yaml
diff --git a/tests/components/lps22/test.esp32-c3-ard.yaml b/tests/components/lps22/test.esp32-c3-ard.yaml
deleted file mode 100644
index 6091393d31..0000000000
--- a/tests/components/lps22/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-i2c:
-  - id: i2c_lps22
-    scl: 5
-    sda: 4
-
-<<: !include common.yaml
diff --git a/tests/components/lps22/test.esp32-c3-idf.yaml b/tests/components/lps22/test.esp32-c3-idf.yaml
index 6091393d31..9990d96d29 100644
--- a/tests/components/lps22/test.esp32-c3-idf.yaml
+++ b/tests/components/lps22/test.esp32-c3-idf.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_lps22
-    scl: 5
-    sda: 4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/lps22/test.esp32-idf.yaml b/tests/components/lps22/test.esp32-idf.yaml
index 0da6a9577e..b47e39c389 100644
--- a/tests/components/lps22/test.esp32-idf.yaml
+++ b/tests/components/lps22/test.esp32-idf.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_lps22
-    scl: 16
-    sda: 17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/lps22/test.esp8266-ard.yaml b/tests/components/lps22/test.esp8266-ard.yaml
index 6091393d31..4a98b9388a 100644
--- a/tests/components/lps22/test.esp8266-ard.yaml
+++ b/tests/components/lps22/test.esp8266-ard.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_lps22
-    scl: 5
-    sda: 4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/lps22/test.rp2040-ard.yaml b/tests/components/lps22/test.rp2040-ard.yaml
index 6091393d31..319a7c71a6 100644
--- a/tests/components/lps22/test.rp2040-ard.yaml
+++ b/tests/components/lps22/test.rp2040-ard.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_lps22
-    scl: 5
-    sda: 4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ltr390/common.yaml b/tests/components/ltr390/common.yaml
index e5e331e7ba..c168da557b 100644
--- a/tests/components/ltr390/common.yaml
+++ b/tests/components/ltr390/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_ltr390
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: ltr390
+    i2c_id: i2c_bus
     uv:
       name: LTR390 UV 1
     uv_index:
@@ -19,6 +15,7 @@ sensor:
     address: 0x53
     update_interval: 60s
   - platform: ltr390
+    i2c_id: i2c_bus
     uv:
       name: LTR390 UV 2
     uv_index:
diff --git a/tests/components/ltr390/test.esp32-ard.yaml b/tests/components/ltr390/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ltr390/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ltr390/test.esp32-c3-ard.yaml b/tests/components/ltr390/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ltr390/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ltr390/test.esp32-c3-idf.yaml b/tests/components/ltr390/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ltr390/test.esp32-c3-idf.yaml
+++ b/tests/components/ltr390/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ltr390/test.esp32-idf.yaml b/tests/components/ltr390/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ltr390/test.esp32-idf.yaml
+++ b/tests/components/ltr390/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ltr390/test.esp8266-ard.yaml b/tests/components/ltr390/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ltr390/test.esp8266-ard.yaml
+++ b/tests/components/ltr390/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ltr390/test.rp2040-ard.yaml b/tests/components/ltr390/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ltr390/test.rp2040-ard.yaml
+++ b/tests/components/ltr390/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ltr501/common.yaml b/tests/components/ltr501/common.yaml
index b7074f52f2..77c6f13739 100644
--- a/tests/components/ltr501/common.yaml
+++ b/tests/components/ltr501/common.yaml
@@ -1,7 +1,6 @@
 sensor:
   - platform: ltr501
     address: 0x23
-    i2c_id: i2c_ltr501
     type: ALS_PS
     gain: 1X
     integration_time: 100ms
diff --git a/tests/components/ltr501/test.esp32-ard.yaml b/tests/components/ltr501/test.esp32-ard.yaml
deleted file mode 100644
index 4c710c74fe..0000000000
--- a/tests/components/ltr501/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-i2c:
-  - id: i2c_ltr501
-    scl: 16
-    sda: 17
-
-<<: !include common.yaml
diff --git a/tests/components/ltr501/test.esp32-c3-ard.yaml b/tests/components/ltr501/test.esp32-c3-ard.yaml
deleted file mode 100644
index 9e7de2768d..0000000000
--- a/tests/components/ltr501/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-i2c:
-  - id: i2c_ltr501
-    scl: 5
-    sda: 4
-
-<<: !include common.yaml
diff --git a/tests/components/ltr501/test.esp32-c3-idf.yaml b/tests/components/ltr501/test.esp32-c3-idf.yaml
index 9e7de2768d..72703301a1 100644
--- a/tests/components/ltr501/test.esp32-c3-idf.yaml
+++ b/tests/components/ltr501/test.esp32-c3-idf.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_ltr501
-    scl: 5
-    sda: 4
+packages:
+  i2c_low_freq: !include ../../test_build_components/common/i2c_low_freq/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ltr501/test.esp32-idf.yaml b/tests/components/ltr501/test.esp32-idf.yaml
index 4c710c74fe..7a5d01898a 100644
--- a/tests/components/ltr501/test.esp32-idf.yaml
+++ b/tests/components/ltr501/test.esp32-idf.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_ltr501
-    scl: 16
-    sda: 17
+packages:
+  i2c_low_freq: !include ../../test_build_components/common/i2c_low_freq/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ltr501/test.esp8266-ard.yaml b/tests/components/ltr501/test.esp8266-ard.yaml
index 9e7de2768d..9e23bb3778 100644
--- a/tests/components/ltr501/test.esp8266-ard.yaml
+++ b/tests/components/ltr501/test.esp8266-ard.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_ltr501
-    scl: 5
-    sda: 4
+packages:
+  i2c_low_freq: !include ../../test_build_components/common/i2c_low_freq/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ltr501/test.rp2040-ard.yaml b/tests/components/ltr501/test.rp2040-ard.yaml
index 9e7de2768d..a7eb30036f 100644
--- a/tests/components/ltr501/test.rp2040-ard.yaml
+++ b/tests/components/ltr501/test.rp2040-ard.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_ltr501
-    scl: 5
-    sda: 4
+packages:
+  i2c_low_freq: !include ../../test_build_components/common/i2c_low_freq/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ltr_als_ps/common.yaml b/tests/components/ltr_als_ps/common.yaml
index aa5c8abed7..edad4b2e4f 100644
--- a/tests/components/ltr_als_ps/common.yaml
+++ b/tests/components/ltr_als_ps/common.yaml
@@ -1,7 +1,6 @@
 sensor:
   - platform: ltr_als_ps
     address: 0x23
-    i2c_id: i2c_als_ps
     gain: 1x
     integration_time: 100ms
     ps_cooldown: 5 s
diff --git a/tests/components/ltr_als_ps/test.esp32-ard.yaml b/tests/components/ltr_als_ps/test.esp32-ard.yaml
deleted file mode 100644
index 2349292a64..0000000000
--- a/tests/components/ltr_als_ps/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-i2c:
-  - id: i2c_als_ps
-    scl: 16
-    sda: 17
-
-<<: !include common.yaml
diff --git a/tests/components/ltr_als_ps/test.esp32-c3-ard.yaml b/tests/components/ltr_als_ps/test.esp32-c3-ard.yaml
deleted file mode 100644
index d64d70f018..0000000000
--- a/tests/components/ltr_als_ps/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-i2c:
-  - id: i2c_als_ps
-    scl: 5
-    sda: 4
-
-<<: !include common.yaml
diff --git a/tests/components/ltr_als_ps/test.esp32-c3-idf.yaml b/tests/components/ltr_als_ps/test.esp32-c3-idf.yaml
index d64d70f018..72703301a1 100644
--- a/tests/components/ltr_als_ps/test.esp32-c3-idf.yaml
+++ b/tests/components/ltr_als_ps/test.esp32-c3-idf.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_als_ps
-    scl: 5
-    sda: 4
+packages:
+  i2c_low_freq: !include ../../test_build_components/common/i2c_low_freq/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ltr_als_ps/test.esp32-idf.yaml b/tests/components/ltr_als_ps/test.esp32-idf.yaml
index 2349292a64..7a5d01898a 100644
--- a/tests/components/ltr_als_ps/test.esp32-idf.yaml
+++ b/tests/components/ltr_als_ps/test.esp32-idf.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_als_ps
-    scl: 16
-    sda: 17
+packages:
+  i2c_low_freq: !include ../../test_build_components/common/i2c_low_freq/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ltr_als_ps/test.esp8266-ard.yaml b/tests/components/ltr_als_ps/test.esp8266-ard.yaml
index d64d70f018..9e23bb3778 100644
--- a/tests/components/ltr_als_ps/test.esp8266-ard.yaml
+++ b/tests/components/ltr_als_ps/test.esp8266-ard.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_als_ps
-    scl: 5
-    sda: 4
+packages:
+  i2c_low_freq: !include ../../test_build_components/common/i2c_low_freq/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ltr_als_ps/test.rp2040-ard.yaml b/tests/components/ltr_als_ps/test.rp2040-ard.yaml
index d64d70f018..a7eb30036f 100644
--- a/tests/components/ltr_als_ps/test.rp2040-ard.yaml
+++ b/tests/components/ltr_als_ps/test.rp2040-ard.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_als_ps
-    scl: 5
-    sda: 4
+packages:
+  i2c_low_freq: !include ../../test_build_components/common/i2c_low_freq/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/lvgl/common.yaml b/tests/components/lvgl/common.yaml
index a035900386..d9b7013a1e 100644
--- a/tests/components/lvgl/common.yaml
+++ b/tests/components/lvgl/common.yaml
@@ -1,5 +1,6 @@
 touchscreen:
   - platform: ft63x6
+    i2c_id: i2c_bus
     id: tft_touch
     display: tft_display
     update_interval: 50ms
diff --git a/tests/components/lvgl/lvgl-package.yaml b/tests/components/lvgl/lvgl-package.yaml
index fbcd2a3fba..582531e943 100644
--- a/tests/components/lvgl/lvgl-package.yaml
+++ b/tests/components/lvgl/lvgl-package.yaml
@@ -78,7 +78,7 @@ lvgl:
     - id: date_style
       text_font: roboto10
       align: center
-      text_color: color_id2
+      text_color: !lambda return color_id2;
       bg_opa: cover
       radius: 4
       pad_all: 2
@@ -684,7 +684,7 @@ lvgl:
             width: 120
             range_from: -10
             range_to: 1000
-            step: 5.0
+            selected_digit: 2
             rollover: false
             digits: 6
             decimal_places: 2
@@ -723,6 +723,20 @@ lvgl:
                 arc_color: 0xFFFF00
               focused:
                 arc_color: 0x808080
+        - arc:
+            align: center
+            id: lv_arc_1
+            value: !lambda return 75;
+            min_value: !lambda return 50;
+            max_value: !lambda return 60;
+            arc_color: 0xFF0000
+            indicator:
+              arc_width: !lambda return 20;
+              arc_color: 0xF000FF
+              pressed:
+                arc_color: 0xFFFF00
+              focused:
+                arc_color: 0x808080
         - bar:
             id: bar_id
             align: top_mid
@@ -738,7 +752,7 @@ lvgl:
                     id: bar_id
                     value: !lambda return (int)((float)rand() / RAND_MAX * 100);
                     start_value: !lambda return (int)((float)rand() / RAND_MAX * 100);
-                    mode: symmetrical
+                    mode: range
                 - logger.log:
                     format: "bar value %f"
                     args: [x]
@@ -928,6 +942,12 @@ lvgl:
                           angle_range: 360
                           rotation: !lambda return 2700;
                           indicators:
+                            - tick_style:
+                                start_value: 0
+                                end_value: 60
+                                color_start: 0x0000bd
+                                color_end: 0xbd0000
+                                width: !lambda return 1;
                             - line:
                                 opa: 50%
                                 id: minute_hand
diff --git a/tests/components/lvgl/test.esp32-ard.yaml b/tests/components/lvgl/test.esp32-ard.yaml
deleted file mode 100644
index f85bedbde6..0000000000
--- a/tests/components/lvgl/test.esp32-ard.yaml
+++ /dev/null
@@ -1,58 +0,0 @@
-spi:
-  clk_pin: 14
-  mosi_pin: 13
-
-i2c:
-  sda: GPIO18
-  scl: GPIO19
-
-display:
-  - platform: ili9xxx
-    model: st7789v
-    id: tft_display
-    dimensions:
-      width: 240
-      height: 320
-    transform:
-      swap_xy: false
-      mirror_x: true
-      mirror_y: true
-    data_rate: 80MHz
-    cs_pin: GPIO22
-    dc_pin: GPIO21
-    auto_clear_enabled: false
-    invert_colors: false
-    update_interval: never
-
-binary_sensor:
-  - platform: gpio
-    internal: true
-    id: up_button
-    pin:
-      number: GPIO38
-      inverted: true
-  - platform: gpio
-    internal: true
-    id: down_button
-    pin:
-      number: GPIO37
-      inverted: true
-  - platform: gpio
-    internal: true
-    id: select_button
-    pin:
-      number: GPIO39
-      inverted: true
-lvgl:
-  draw_rounding: 8
-  encoders:
-    group: switches
-    initial_focus: button_button
-    enter_button: select_button
-    sensor:
-      left_button: up_button
-      right_button: down_button
-
-packages:
-  lvgl: !include lvgl-package.yaml
-  xvgl: !include common.yaml
diff --git a/tests/components/lvgl/test.esp32-idf.yaml b/tests/components/lvgl/test.esp32-idf.yaml
index eacace1d4b..6170b0f4fb 100644
--- a/tests/components/lvgl/test.esp32-idf.yaml
+++ b/tests/components/lvgl/test.esp32-idf.yaml
@@ -1,10 +1,7 @@
-spi:
-  clk_pin: 14
-  mosi_pin: 13
-
-i2c:
-  sda: GPIO18
-  scl: GPIO19
+packages:
+  lvgl: !include lvgl-package.yaml
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 sensor:
   - platform: rotary_encoder
@@ -25,6 +22,7 @@ binary_sensor:
 
 display:
   - platform: ili9xxx
+    spi_id: spi_bus
     model: st7789v
     id: second_display
     dimensions:
@@ -44,6 +42,7 @@ display:
     update_interval: never
 
   - platform: ili9xxx
+    spi_id: spi_bus
     model: st7789v
     id: tft_display
     dimensions:
@@ -60,9 +59,6 @@ display:
     invert_colors: false
     update_interval: never
 
-packages:
-  lvgl: !include lvgl-package.yaml
-
 lvgl:
   displays:
     - tft_display
diff --git a/tests/components/m5stack_8angle/common.yaml b/tests/components/m5stack_8angle/common.yaml
index d7f988ed3a..d0af24116c 100644
--- a/tests/components/m5stack_8angle/common.yaml
+++ b/tests/components/m5stack_8angle/common.yaml
@@ -1,10 +1,5 @@
-i2c:
-  sda: 0
-  scl: 1
-  id: bus_external
-
 m5stack_8angle:
-  i2c_id: bus_external
+  i2c_id: i2c_bus
   id: m5stack_8angle_base
 
 light:
diff --git a/tests/components/m5stack_8angle/test.esp32-ard.yaml b/tests/components/m5stack_8angle/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/m5stack_8angle/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/m5stack_8angle/test.esp32-c3-ard.yaml b/tests/components/m5stack_8angle/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/m5stack_8angle/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/m5stack_8angle/test.esp32-c3-idf.yaml b/tests/components/m5stack_8angle/test.esp32-c3-idf.yaml
index dade44d145..9990d96d29 100644
--- a/tests/components/m5stack_8angle/test.esp32-c3-idf.yaml
+++ b/tests/components/m5stack_8angle/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/m5stack_8angle/test.esp32-idf.yaml b/tests/components/m5stack_8angle/test.esp32-idf.yaml
index dade44d145..b47e39c389 100644
--- a/tests/components/m5stack_8angle/test.esp32-idf.yaml
+++ b/tests/components/m5stack_8angle/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/m5stack_8angle/test.esp8266-ard.yaml b/tests/components/m5stack_8angle/test.esp8266-ard.yaml
index dade44d145..4a98b9388a 100644
--- a/tests/components/m5stack_8angle/test.esp8266-ard.yaml
+++ b/tests/components/m5stack_8angle/test.esp8266-ard.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/m5stack_8angle/test.rp2040-ard.yaml b/tests/components/m5stack_8angle/test.rp2040-ard.yaml
index dade44d145..319a7c71a6 100644
--- a/tests/components/m5stack_8angle/test.rp2040-ard.yaml
+++ b/tests/components/m5stack_8angle/test.rp2040-ard.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mapping/.gitattributes b/tests/components/mapping/.gitattributes
new file mode 100644
index 0000000000..9d74867fcf
--- /dev/null
+++ b/tests/components/mapping/.gitattributes
@@ -0,0 +1 @@
+*.ttf        -text
diff --git a/tests/components/mapping/common.yaml b/tests/components/mapping/common.yaml
index 07ca458146..7ffcfa4f67 100644
--- a/tests/components/mapping/common.yaml
+++ b/tests/components/mapping/common.yaml
@@ -50,6 +50,14 @@ mapping:
       red: red_id
       blue: blue_id
       green: green_id
+  - id: string_map_2
+    from: string
+    to: string
+    entries:
+      one: "one"
+      two: "two"
+      three: "three"
+      seventy-seven: "seventy-seven"
 
 color:
   - id: red_id
@@ -65,7 +73,14 @@ color:
     green: 0.0
     blue: 1.0
 
+font:
+  - file: "$component_dir/helvetica.ttf"
+    id: font_id
+    size: 20
+
 display:
   lambda: |-
-      it.image(0, 0, id(weather_map)[0]);
-      it.image(0, 100, id(weather_map)[1]);
+      std::string value = id(int_map)[2];
+      it.print(0, 0, id(font_id), TextAlign::TOP_LEFT, value.c_str());
+      it.image(0, 0, id(weather_map)["clear-night"]);
+      it.image(0, 100, id(weather_map)["sunny"]);
diff --git a/tests/components/mapping/helvetica.ttf b/tests/components/mapping/helvetica.ttf
new file mode 100644
index 0000000000..7aec6f3f3c
Binary files /dev/null and b/tests/components/mapping/helvetica.ttf differ
diff --git a/tests/components/mapping/test.esp32-ard.yaml b/tests/components/mapping/test.esp32-ard.yaml
deleted file mode 100644
index 951a6061f6..0000000000
--- a/tests/components/mapping/test.esp32-ard.yaml
+++ /dev/null
@@ -1,17 +0,0 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 16
-    mosi_pin: 17
-    miso_pin: 15
-
-display:
-  - platform: ili9xxx
-    id: main_lcd
-    model: ili9342
-    cs_pin: 12
-    dc_pin: 13
-    reset_pin: 21
-    invert_colors: false
-
-packages:
-  map: !include common.yaml
diff --git a/tests/components/mapping/test.esp32-c3-ard.yaml b/tests/components/mapping/test.esp32-c3-ard.yaml
deleted file mode 100644
index 55e5719e50..0000000000
--- a/tests/components/mapping/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,17 +0,0 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 6
-    mosi_pin: 7
-    miso_pin: 5
-
-display:
-  - platform: ili9xxx
-    id: main_lcd
-    model: ili9342
-    cs_pin: 8
-    dc_pin: 9
-    reset_pin: 10
-    invert_colors: false
-
-packages:
-  map: !include common.yaml
diff --git a/tests/components/mapping/test.esp32-c3-idf.yaml b/tests/components/mapping/test.esp32-c3-idf.yaml
index 55e5719e50..7911eb7edc 100644
--- a/tests/components/mapping/test.esp32-c3-idf.yaml
+++ b/tests/components/mapping/test.esp32-c3-idf.yaml
@@ -1,17 +1,13 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 6
-    mosi_pin: 7
-    miso_pin: 5
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
+  map: !include common.yaml
 
 display:
-  - platform: ili9xxx
-    id: main_lcd
-    model: ili9342
-    cs_pin: 8
-    dc_pin: 9
-    reset_pin: 10
-    invert_colors: false
-
-packages:
-  map: !include common.yaml
+  spi_id: spi_bus
+  platform: ili9xxx
+  id: main_lcd
+  model: ili9342
+  cs_pin: 8
+  dc_pin: 9
+  reset_pin: 10
+  invert_colors: false
diff --git a/tests/components/mapping/test.esp32-idf.yaml b/tests/components/mapping/test.esp32-idf.yaml
index 951a6061f6..a35b6940c7 100644
--- a/tests/components/mapping/test.esp32-idf.yaml
+++ b/tests/components/mapping/test.esp32-idf.yaml
@@ -1,17 +1,13 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 16
-    mosi_pin: 17
-    miso_pin: 15
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+  map: !include common.yaml
 
 display:
-  - platform: ili9xxx
-    id: main_lcd
-    model: ili9342
-    cs_pin: 12
-    dc_pin: 13
-    reset_pin: 21
-    invert_colors: false
-
-packages:
-  map: !include common.yaml
+  spi_id: spi_bus
+  platform: ili9xxx
+  id: main_lcd
+  model: ili9342
+  cs_pin: 12
+  dc_pin: 13
+  reset_pin: 21
+  invert_colors: false
diff --git a/tests/components/mapping/test.esp8266-ard.yaml b/tests/components/mapping/test.esp8266-ard.yaml
index dd4642b8fe..c59821a211 100644
--- a/tests/components/mapping/test.esp8266-ard.yaml
+++ b/tests/components/mapping/test.esp8266-ard.yaml
@@ -1,17 +1,13 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 14
-    mosi_pin: 13
-    miso_pin: 12
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+  map: !include common.yaml
 
 display:
-  - platform: ili9xxx
-    id: main_lcd
-    model: ili9342
-    cs_pin: 5
-    dc_pin: 15
-    reset_pin: 16
-    invert_colors: false
-
-packages:
-  map: !include common.yaml
+  spi_id: spi_bus
+  platform: ili9xxx
+  id: main_lcd
+  model: ili9342
+  cs_pin: 5
+  dc_pin: 15
+  reset_pin: 16
+  invert_colors: false
diff --git a/tests/components/mapping/test.host.yaml b/tests/components/mapping/test.host.yaml
index 98406767a4..19937e1f21 100644
--- a/tests/components/mapping/test.host.yaml
+++ b/tests/components/mapping/test.host.yaml
@@ -1,12 +1,12 @@
 display:
-  - platform: sdl
-    id: sdl_display
-    update_interval: 1s
-    auto_clear_enabled: false
-    show_test_card: true
-    dimensions:
-      width: 450
-      height: 600
+  platform: sdl
+  id: sdl_display
+  update_interval: 1s
+  auto_clear_enabled: false
+  show_test_card: true
+  dimensions:
+    width: 450
+    height: 600
 
 packages:
   map: !include common.yaml
diff --git a/tests/components/mapping/test.rp2040-ard.yaml b/tests/components/mapping/test.rp2040-ard.yaml
index 1b7e796246..acc305a701 100644
--- a/tests/components/mapping/test.rp2040-ard.yaml
+++ b/tests/components/mapping/test.rp2040-ard.yaml
@@ -1,17 +1,13 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: 2
-    mosi_pin: 3
-    miso_pin: 4
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+  map: !include common.yaml
 
 display:
-  - platform: ili9xxx
-    id: main_lcd
-    model: ili9342
-    cs_pin: 20
-    dc_pin: 21
-    reset_pin: 22
-    invert_colors: false
-
-packages:
-  map: !include common.yaml
+  spi_id: spi_bus
+  platform: ili9xxx
+  id: main_lcd
+  model: ili9342
+  cs_pin: 20
+  dc_pin: 21
+  reset_pin: 22
+  invert_colors: false
diff --git a/tests/components/matrix_keypad/test.esp32-ard.yaml b/tests/components/matrix_keypad/test.esp32-ard.yaml
deleted file mode 100644
index 70bb70638d..0000000000
--- a/tests/components/matrix_keypad/test.esp32-ard.yaml
+++ /dev/null
@@ -1,15 +0,0 @@
-packages:
-  common: !include common.yaml
-
-matrix_keypad:
-  id: keypad
-  rows:
-    - pin: 12
-    - pin: 13
-  columns:
-    - pin: 14
-    - pin: 15
-  keys: "1234"
-  has_pulldowns: true
-  on_key:
-    - lambda: ESP_LOGI("KEY", "key %d pressed", x);
diff --git a/tests/components/matrix_keypad/test.esp32-c3-ard.yaml b/tests/components/matrix_keypad/test.esp32-c3-ard.yaml
deleted file mode 100644
index 75d9c0b263..0000000000
--- a/tests/components/matrix_keypad/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,15 +0,0 @@
-packages:
-  common: !include common.yaml
-
-matrix_keypad:
-  id: keypad
-  rows:
-    - pin: 1
-    - pin: 2
-  columns:
-    - pin: 3
-    - pin: 4
-  keys: "1234"
-  has_pulldowns: true
-  on_key:
-    - lambda: ESP_LOGI("KEY", "key %d pressed", x);
diff --git a/tests/components/max17043/common.yaml b/tests/components/max17043/common.yaml
index c2f324212e..f58006c460 100644
--- a/tests/components/max17043/common.yaml
+++ b/tests/components/max17043/common.yaml
@@ -3,15 +3,10 @@ esphome:
     then:
       - max17043.sleep_mode: max17043_id
 
-i2c:
-  - id: i2c_id
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: max17043
+    i2c_id: i2c_bus
     id: max17043_id
-    i2c_id: i2c_id
     battery_voltage:
       name: "Battery Voltage"
     battery_level:
diff --git a/tests/components/max17043/test.esp32-ard.yaml b/tests/components/max17043/test.esp32-ard.yaml
deleted file mode 100644
index c6615f51cd..0000000000
--- a/tests/components/max17043/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  sda_pin: GPIO21
-  scl_pin: GPIO22
-
-<<: !include common.yaml
diff --git a/tests/components/max17043/test.esp32-c3-ard.yaml b/tests/components/max17043/test.esp32-c3-ard.yaml
deleted file mode 100644
index 9a1477d4b9..0000000000
--- a/tests/components/max17043/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  sda_pin: GPIO8
-  scl_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/max17043/test.esp32-c3-idf.yaml b/tests/components/max17043/test.esp32-c3-idf.yaml
index 9a1477d4b9..9990d96d29 100644
--- a/tests/components/max17043/test.esp32-c3-idf.yaml
+++ b/tests/components/max17043/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  sda_pin: GPIO8
-  scl_pin: GPIO10
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max17043/test.esp32-idf.yaml b/tests/components/max17043/test.esp32-idf.yaml
index c6615f51cd..b47e39c389 100644
--- a/tests/components/max17043/test.esp32-idf.yaml
+++ b/tests/components/max17043/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  sda_pin: GPIO21
-  scl_pin: GPIO22
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max17043/test.esp8266-ard.yaml b/tests/components/max17043/test.esp8266-ard.yaml
index a87353b78b..4a98b9388a 100644
--- a/tests/components/max17043/test.esp8266-ard.yaml
+++ b/tests/components/max17043/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  sda_pin: GPIO4
-  scl_pin: GPIO5
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max17043/test.rp2040-ard.yaml b/tests/components/max17043/test.rp2040-ard.yaml
index c6615f51cd..319a7c71a6 100644
--- a/tests/components/max17043/test.rp2040-ard.yaml
+++ b/tests/components/max17043/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  sda_pin: GPIO21
-  scl_pin: GPIO22
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max31855/common.yaml b/tests/components/max31855/common.yaml
index 7136c597d5..905b111d71 100644
--- a/tests/components/max31855/common.yaml
+++ b/tests/components/max31855/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_max31855
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: max31855
     name: MAX31855 Temperature
diff --git a/tests/components/max31855/test.esp32-ard.yaml b/tests/components/max31855/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/max31855/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/max31855/test.esp32-c3-ard.yaml b/tests/components/max31855/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/max31855/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/max31855/test.esp32-c3-idf.yaml b/tests/components/max31855/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/max31855/test.esp32-c3-idf.yaml
+++ b/tests/components/max31855/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max31855/test.esp32-idf.yaml b/tests/components/max31855/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/max31855/test.esp32-idf.yaml
+++ b/tests/components/max31855/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max31855/test.esp8266-ard.yaml b/tests/components/max31855/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/max31855/test.esp8266-ard.yaml
+++ b/tests/components/max31855/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max31855/test.rp2040-ard.yaml b/tests/components/max31855/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/max31855/test.rp2040-ard.yaml
+++ b/tests/components/max31855/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max31856/common.yaml b/tests/components/max31856/common.yaml
index 4f7c3ad408..9d420662d7 100644
--- a/tests/components/max31856/common.yaml
+++ b/tests/components/max31856/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_max31856
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: max31856
     name: MAX31856 Temperature
diff --git a/tests/components/max31856/test.esp32-ard.yaml b/tests/components/max31856/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/max31856/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/max31856/test.esp32-c3-ard.yaml b/tests/components/max31856/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/max31856/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/max31856/test.esp32-c3-idf.yaml b/tests/components/max31856/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/max31856/test.esp32-c3-idf.yaml
+++ b/tests/components/max31856/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max31856/test.esp32-idf.yaml b/tests/components/max31856/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/max31856/test.esp32-idf.yaml
+++ b/tests/components/max31856/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max31856/test.esp8266-ard.yaml b/tests/components/max31856/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/max31856/test.esp8266-ard.yaml
+++ b/tests/components/max31856/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max31856/test.rp2040-ard.yaml b/tests/components/max31856/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/max31856/test.rp2040-ard.yaml
+++ b/tests/components/max31856/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max31865/common.yaml b/tests/components/max31865/common.yaml
index 5bb7bda5aa..6e71f17efc 100644
--- a/tests/components/max31865/common.yaml
+++ b/tests/components/max31865/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_max31865
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: max31865
     name: MAX31865 Temperature
diff --git a/tests/components/max31865/test.esp32-ard.yaml b/tests/components/max31865/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/max31865/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/max31865/test.esp32-c3-ard.yaml b/tests/components/max31865/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/max31865/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/max31865/test.esp32-c3-idf.yaml b/tests/components/max31865/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/max31865/test.esp32-c3-idf.yaml
+++ b/tests/components/max31865/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max31865/test.esp32-idf.yaml b/tests/components/max31865/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/max31865/test.esp32-idf.yaml
+++ b/tests/components/max31865/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max31865/test.esp8266-ard.yaml b/tests/components/max31865/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/max31865/test.esp8266-ard.yaml
+++ b/tests/components/max31865/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max31865/test.rp2040-ard.yaml b/tests/components/max31865/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/max31865/test.rp2040-ard.yaml
+++ b/tests/components/max31865/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max44009/common.yaml b/tests/components/max44009/common.yaml
index ef51740895..523387e1cc 100644
--- a/tests/components/max44009/common.yaml
+++ b/tests/components/max44009/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_max44009
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: max44009
+    i2c_id: i2c_bus
     name: MAX44009 Brightness
     internal: true
     mode: low_power
diff --git a/tests/components/max44009/test.esp32-ard.yaml b/tests/components/max44009/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/max44009/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/max44009/test.esp32-c3-ard.yaml b/tests/components/max44009/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/max44009/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/max44009/test.esp32-c3-idf.yaml b/tests/components/max44009/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/max44009/test.esp32-c3-idf.yaml
+++ b/tests/components/max44009/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max44009/test.esp32-idf.yaml b/tests/components/max44009/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/max44009/test.esp32-idf.yaml
+++ b/tests/components/max44009/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max44009/test.esp8266-ard.yaml b/tests/components/max44009/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/max44009/test.esp8266-ard.yaml
+++ b/tests/components/max44009/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max44009/test.rp2040-ard.yaml b/tests/components/max44009/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/max44009/test.rp2040-ard.yaml
+++ b/tests/components/max44009/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max6675/common.yaml b/tests/components/max6675/common.yaml
index 5b4e04b317..31a4e7035a 100644
--- a/tests/components/max6675/common.yaml
+++ b/tests/components/max6675/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_max6675
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sensor:
   - platform: max6675
     name: Temperature
diff --git a/tests/components/max6675/test.esp32-ard.yaml b/tests/components/max6675/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/max6675/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/max6675/test.esp32-c3-ard.yaml b/tests/components/max6675/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/max6675/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/max6675/test.esp32-c3-idf.yaml b/tests/components/max6675/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/max6675/test.esp32-c3-idf.yaml
+++ b/tests/components/max6675/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max6675/test.esp32-idf.yaml b/tests/components/max6675/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/max6675/test.esp32-idf.yaml
+++ b/tests/components/max6675/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max6675/test.esp8266-ard.yaml b/tests/components/max6675/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/max6675/test.esp8266-ard.yaml
+++ b/tests/components/max6675/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max6675/test.rp2040-ard.yaml b/tests/components/max6675/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/max6675/test.rp2040-ard.yaml
+++ b/tests/components/max6675/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max6956/common.yaml b/tests/components/max6956/common.yaml
index e44e3464f8..665a606027 100644
--- a/tests/components/max6956/common.yaml
+++ b/tests/components/max6956/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_max6956
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 max6956:
   - id: max6956_1
+    i2c_id: i2c_bus
     address: 0x40
 
 binary_sensor:
diff --git a/tests/components/max6956/test.esp32-ard.yaml b/tests/components/max6956/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/max6956/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/max6956/test.esp32-c3-ard.yaml b/tests/components/max6956/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/max6956/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/max6956/test.esp32-c3-idf.yaml b/tests/components/max6956/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/max6956/test.esp32-c3-idf.yaml
+++ b/tests/components/max6956/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max6956/test.esp32-idf.yaml b/tests/components/max6956/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/max6956/test.esp32-idf.yaml
+++ b/tests/components/max6956/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max6956/test.esp8266-ard.yaml b/tests/components/max6956/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/max6956/test.esp8266-ard.yaml
+++ b/tests/components/max6956/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max6956/test.rp2040-ard.yaml b/tests/components/max6956/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/max6956/test.rp2040-ard.yaml
+++ b/tests/components/max6956/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max7219/common.yaml b/tests/components/max7219/common.yaml
index 0060db191e..5d7f54af17 100644
--- a/tests/components/max7219/common.yaml
+++ b/tests/components/max7219/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_max6675
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 display:
   - platform: max7219
     cs_pin: ${cs_pin}
diff --git a/tests/components/max7219/test.esp32-ard.yaml b/tests/components/max7219/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/max7219/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/max7219/test.esp32-c3-ard.yaml b/tests/components/max7219/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/max7219/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/max7219/test.esp32-c3-idf.yaml b/tests/components/max7219/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/max7219/test.esp32-c3-idf.yaml
+++ b/tests/components/max7219/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max7219/test.esp32-idf.yaml b/tests/components/max7219/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/max7219/test.esp32-idf.yaml
+++ b/tests/components/max7219/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max7219/test.esp8266-ard.yaml b/tests/components/max7219/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/max7219/test.esp8266-ard.yaml
+++ b/tests/components/max7219/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max7219/test.rp2040-ard.yaml b/tests/components/max7219/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/max7219/test.rp2040-ard.yaml
+++ b/tests/components/max7219/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max7219digit/common.yaml b/tests/components/max7219digit/common.yaml
index 84edc7eb3d..525b7b8d3e 100644
--- a/tests/components/max7219digit/common.yaml
+++ b/tests/components/max7219digit/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_max7219digit
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 display:
   - platform: max7219digit
     cs_pin: ${cs_pin}
diff --git a/tests/components/max7219digit/test.esp32-ard.yaml b/tests/components/max7219digit/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/max7219digit/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/max7219digit/test.esp32-c3-ard.yaml b/tests/components/max7219digit/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/max7219digit/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/max7219digit/test.esp32-c3-idf.yaml b/tests/components/max7219digit/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/max7219digit/test.esp32-c3-idf.yaml
+++ b/tests/components/max7219digit/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max7219digit/test.esp32-idf.yaml b/tests/components/max7219digit/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/max7219digit/test.esp32-idf.yaml
+++ b/tests/components/max7219digit/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max7219digit/test.esp8266-ard.yaml b/tests/components/max7219digit/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/max7219digit/test.esp8266-ard.yaml
+++ b/tests/components/max7219digit/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max7219digit/test.rp2040-ard.yaml b/tests/components/max7219digit/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/max7219digit/test.rp2040-ard.yaml
+++ b/tests/components/max7219digit/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/max9611/common.yaml b/tests/components/max9611/common.yaml
index c3c00fdf85..ca9ee59038 100644
--- a/tests/components/max9611/common.yaml
+++ b/tests/components/max9611/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_max9611
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: max9611
+    i2c_id: i2c_bus
     shunt_resistance: 0.2 ohm
     gain: 1X
     voltage:
diff --git a/tests/components/max9611/test.esp32-ard.yaml b/tests/components/max9611/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/max9611/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/max9611/test.esp32-c3-ard.yaml b/tests/components/max9611/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/max9611/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/max9611/test.esp32-c3-idf.yaml b/tests/components/max9611/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/max9611/test.esp32-c3-idf.yaml
+++ b/tests/components/max9611/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max9611/test.esp32-idf.yaml b/tests/components/max9611/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/max9611/test.esp32-idf.yaml
+++ b/tests/components/max9611/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max9611/test.esp8266-ard.yaml b/tests/components/max9611/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/max9611/test.esp8266-ard.yaml
+++ b/tests/components/max9611/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/max9611/test.rp2040-ard.yaml b/tests/components/max9611/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/max9611/test.rp2040-ard.yaml
+++ b/tests/components/max9611/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp23008/common.yaml b/tests/components/mcp23008/common.yaml
index 1954766d25..4a407adfd8 100644
--- a/tests/components/mcp23008/common.yaml
+++ b/tests/components/mcp23008/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_mcp23008
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 mcp23008:
+  i2c_id: i2c_bus
   id: mcp23008_hub
 
 binary_sensor:
diff --git a/tests/components/mcp23008/test.esp32-ard.yaml b/tests/components/mcp23008/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mcp23008/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mcp23008/test.esp32-c3-ard.yaml b/tests/components/mcp23008/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mcp23008/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mcp23008/test.esp32-c3-idf.yaml b/tests/components/mcp23008/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mcp23008/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp23008/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp23008/test.esp32-idf.yaml b/tests/components/mcp23008/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mcp23008/test.esp32-idf.yaml
+++ b/tests/components/mcp23008/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp23008/test.esp8266-ard.yaml b/tests/components/mcp23008/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mcp23008/test.esp8266-ard.yaml
+++ b/tests/components/mcp23008/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp23008/test.rp2040-ard.yaml b/tests/components/mcp23008/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mcp23008/test.rp2040-ard.yaml
+++ b/tests/components/mcp23008/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp23016/common.yaml b/tests/components/mcp23016/common.yaml
index 109cb34b21..e8e3ad9d08 100644
--- a/tests/components/mcp23016/common.yaml
+++ b/tests/components/mcp23016/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_mcp23016
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 mcp23016:
+  i2c_id: i2c_bus
   id: mcp23016_hub
 
 binary_sensor:
diff --git a/tests/components/mcp23016/test.esp32-ard.yaml b/tests/components/mcp23016/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mcp23016/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mcp23016/test.esp32-c3-ard.yaml b/tests/components/mcp23016/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mcp23016/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mcp23016/test.esp32-c3-idf.yaml b/tests/components/mcp23016/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mcp23016/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp23016/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp23016/test.esp32-idf.yaml b/tests/components/mcp23016/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mcp23016/test.esp32-idf.yaml
+++ b/tests/components/mcp23016/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp23016/test.esp8266-ard.yaml b/tests/components/mcp23016/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mcp23016/test.esp8266-ard.yaml
+++ b/tests/components/mcp23016/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp23016/test.rp2040-ard.yaml b/tests/components/mcp23016/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mcp23016/test.rp2040-ard.yaml
+++ b/tests/components/mcp23016/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp23017/common.yaml b/tests/components/mcp23017/common.yaml
index 74949bba76..54a97e911f 100644
--- a/tests/components/mcp23017/common.yaml
+++ b/tests/components/mcp23017/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_mcp23017
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 mcp23017:
+  i2c_id: i2c_bus
   id: mcp23017_hub
 
 binary_sensor:
diff --git a/tests/components/mcp23017/test.esp32-ard.yaml b/tests/components/mcp23017/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mcp23017/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mcp23017/test.esp32-c3-ard.yaml b/tests/components/mcp23017/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mcp23017/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mcp23017/test.esp32-c3-idf.yaml b/tests/components/mcp23017/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mcp23017/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp23017/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp23017/test.esp32-idf.yaml b/tests/components/mcp23017/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mcp23017/test.esp32-idf.yaml
+++ b/tests/components/mcp23017/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp23017/test.esp8266-ard.yaml b/tests/components/mcp23017/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mcp23017/test.esp8266-ard.yaml
+++ b/tests/components/mcp23017/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp23017/test.rp2040-ard.yaml b/tests/components/mcp23017/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mcp23017/test.rp2040-ard.yaml
+++ b/tests/components/mcp23017/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp23s08/common.yaml b/tests/components/mcp23s08/common.yaml
index b89088fe15..2170ae0459 100644
--- a/tests/components/mcp23s08/common.yaml
+++ b/tests/components/mcp23s08/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_mcp23s08
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 mcp23s08:
   - id: mcp23s08_hub
     cs_pin: ${cs_pin}
diff --git a/tests/components/mcp23s08/test.esp32-ard.yaml b/tests/components/mcp23s08/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/mcp23s08/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/mcp23s08/test.esp32-c3-ard.yaml b/tests/components/mcp23s08/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/mcp23s08/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/mcp23s08/test.esp32-c3-idf.yaml b/tests/components/mcp23s08/test.esp32-c3-idf.yaml
index 2415ba5dc6..b11ec9cdc6 100644
--- a/tests/components/mcp23s08/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp23s08/test.esp32-c3-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp23s08/test.esp32-idf.yaml b/tests/components/mcp23s08/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/mcp23s08/test.esp32-idf.yaml
+++ b/tests/components/mcp23s08/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp23s08/test.esp8266-ard.yaml b/tests/components/mcp23s08/test.esp8266-ard.yaml
index dbd158d030..595f31046a 100644
--- a/tests/components/mcp23s08/test.esp8266-ard.yaml
+++ b/tests/components/mcp23s08/test.esp8266-ard.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp23s08/test.rp2040-ard.yaml b/tests/components/mcp23s08/test.rp2040-ard.yaml
index f6c3f1eeca..79ea6ce90b 100644
--- a/tests/components/mcp23s08/test.rp2040-ard.yaml
+++ b/tests/components/mcp23s08/test.rp2040-ard.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO2
-  mosi_pin: GPIO3
-  miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp23s17/common.yaml b/tests/components/mcp23s17/common.yaml
index 3fb27ef625..a89beeb16b 100644
--- a/tests/components/mcp23s17/common.yaml
+++ b/tests/components/mcp23s17/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_mcp23s17
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 mcp23s17:
   - id: mcp23s17_hub
     cs_pin: ${cs_pin}
diff --git a/tests/components/mcp23s17/test.esp32-ard.yaml b/tests/components/mcp23s17/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/mcp23s17/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/mcp23s17/test.esp32-c3-ard.yaml b/tests/components/mcp23s17/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/mcp23s17/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/mcp23s17/test.esp32-c3-idf.yaml b/tests/components/mcp23s17/test.esp32-c3-idf.yaml
index 2415ba5dc6..b11ec9cdc6 100644
--- a/tests/components/mcp23s17/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp23s17/test.esp32-c3-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp23s17/test.esp32-idf.yaml b/tests/components/mcp23s17/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/mcp23s17/test.esp32-idf.yaml
+++ b/tests/components/mcp23s17/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp23s17/test.esp8266-ard.yaml b/tests/components/mcp23s17/test.esp8266-ard.yaml
index dbd158d030..595f31046a 100644
--- a/tests/components/mcp23s17/test.esp8266-ard.yaml
+++ b/tests/components/mcp23s17/test.esp8266-ard.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp23s17/test.rp2040-ard.yaml b/tests/components/mcp23s17/test.rp2040-ard.yaml
index f6c3f1eeca..79ea6ce90b 100644
--- a/tests/components/mcp23s17/test.rp2040-ard.yaml
+++ b/tests/components/mcp23s17/test.rp2040-ard.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO2
-  mosi_pin: GPIO3
-  miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp2515/common.yaml b/tests/components/mcp2515/common.yaml
index 96a72a3ec3..15639df5fe 100644
--- a/tests/components/mcp2515/common.yaml
+++ b/tests/components/mcp2515/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_mcp2515
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 canbus:
   - platform: mcp2515
     id: mcp2515_can
diff --git a/tests/components/mcp2515/test.esp32-ard.yaml b/tests/components/mcp2515/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/mcp2515/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/mcp2515/test.esp32-c3-ard.yaml b/tests/components/mcp2515/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/mcp2515/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/mcp2515/test.esp32-c3-idf.yaml b/tests/components/mcp2515/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/mcp2515/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp2515/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp2515/test.esp32-idf.yaml b/tests/components/mcp2515/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/mcp2515/test.esp32-idf.yaml
+++ b/tests/components/mcp2515/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp2515/test.esp8266-ard.yaml b/tests/components/mcp2515/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/mcp2515/test.esp8266-ard.yaml
+++ b/tests/components/mcp2515/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp2515/test.rp2040-ard.yaml b/tests/components/mcp2515/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/mcp2515/test.rp2040-ard.yaml
+++ b/tests/components/mcp2515/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp3008/common.yaml b/tests/components/mcp3008/common.yaml
index 646d3a20e9..57d0155f73 100644
--- a/tests/components/mcp3008/common.yaml
+++ b/tests/components/mcp3008/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_mcp3008
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 mcp3008:
   - id: mcp3008_hub
     cs_pin: ${cs_pin}
diff --git a/tests/components/mcp3008/test.esp32-ard.yaml b/tests/components/mcp3008/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/mcp3008/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/mcp3008/test.esp32-c3-ard.yaml b/tests/components/mcp3008/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/mcp3008/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/mcp3008/test.esp32-c3-idf.yaml b/tests/components/mcp3008/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/mcp3008/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp3008/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp3008/test.esp32-idf.yaml b/tests/components/mcp3008/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/mcp3008/test.esp32-idf.yaml
+++ b/tests/components/mcp3008/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp3008/test.esp8266-ard.yaml b/tests/components/mcp3008/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/mcp3008/test.esp8266-ard.yaml
+++ b/tests/components/mcp3008/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp3008/test.rp2040-ard.yaml b/tests/components/mcp3008/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/mcp3008/test.rp2040-ard.yaml
+++ b/tests/components/mcp3008/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp3204/common.yaml b/tests/components/mcp3204/common.yaml
index f102500c81..eca6ec44f4 100644
--- a/tests/components/mcp3204/common.yaml
+++ b/tests/components/mcp3204/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_mcp3204
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 mcp3204:
   - id: mcp3204_hub
     cs_pin: ${cs_pin}
diff --git a/tests/components/mcp3204/test.esp32-ard.yaml b/tests/components/mcp3204/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/mcp3204/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/mcp3204/test.esp32-c3-ard.yaml b/tests/components/mcp3204/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/mcp3204/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/mcp3204/test.esp32-c3-idf.yaml b/tests/components/mcp3204/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/mcp3204/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp3204/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp3204/test.esp32-idf.yaml b/tests/components/mcp3204/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/mcp3204/test.esp32-idf.yaml
+++ b/tests/components/mcp3204/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp3204/test.esp8266-ard.yaml b/tests/components/mcp3204/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/mcp3204/test.esp8266-ard.yaml
+++ b/tests/components/mcp3204/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp3204/test.rp2040-ard.yaml b/tests/components/mcp3204/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/mcp3204/test.rp2040-ard.yaml
+++ b/tests/components/mcp3204/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mcp4461/common.yaml b/tests/components/mcp4461/common.yaml
index ce1866fdb8..92fd789dcb 100644
--- a/tests/components/mcp4461/common.yaml
+++ b/tests/components/mcp4461/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_mcp4461
-    sda: ${sda_pin}
-    scl: ${scl_pin}
-
 mcp4461:
   - id: mcp4461_digipot_01
+    i2c_id: i2c_bus
 
 output:
   - platform: mcp4461
diff --git a/tests/components/mcp4461/test.esp32-ard.yaml b/tests/components/mcp4461/test.esp32-ard.yaml
deleted file mode 100644
index c5deb7ca0a..0000000000
--- a/tests/components/mcp4461/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  sda_pin: GPIO16
-  scl_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mcp4461/test.esp32-c3-ard.yaml b/tests/components/mcp4461/test.esp32-c3-ard.yaml
deleted file mode 100644
index a87353b78b..0000000000
--- a/tests/components/mcp4461/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  sda_pin: GPIO4
-  scl_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/mcp4461/test.esp32-c3-idf.yaml b/tests/components/mcp4461/test.esp32-c3-idf.yaml
index a87353b78b..9990d96d29 100644
--- a/tests/components/mcp4461/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp4461/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  sda_pin: GPIO4
-  scl_pin: GPIO5
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp4461/test.esp32-idf.yaml b/tests/components/mcp4461/test.esp32-idf.yaml
index c5deb7ca0a..b47e39c389 100644
--- a/tests/components/mcp4461/test.esp32-idf.yaml
+++ b/tests/components/mcp4461/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  sda_pin: GPIO16
-  scl_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp4461/test.esp8266-ard.yaml b/tests/components/mcp4461/test.esp8266-ard.yaml
index a87353b78b..4a98b9388a 100644
--- a/tests/components/mcp4461/test.esp8266-ard.yaml
+++ b/tests/components/mcp4461/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  sda_pin: GPIO4
-  scl_pin: GPIO5
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp4725/common.yaml b/tests/components/mcp4725/common.yaml
index 0ccc649f2e..9352ebfd19 100644
--- a/tests/components/mcp4725/common.yaml
+++ b/tests/components/mcp4725/common.yaml
@@ -1,8 +1,4 @@
-i2c:
-  - id: i2c_mcp4725
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 output:
   - platform: mcp4725
     id: mcp4725_dac_output
+    i2c_id: i2c_bus
diff --git a/tests/components/mcp4725/test.esp32-ard.yaml b/tests/components/mcp4725/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mcp4725/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mcp4725/test.esp32-c3-ard.yaml b/tests/components/mcp4725/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mcp4725/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mcp4725/test.esp32-c3-idf.yaml b/tests/components/mcp4725/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mcp4725/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp4725/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp4725/test.esp32-idf.yaml b/tests/components/mcp4725/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mcp4725/test.esp32-idf.yaml
+++ b/tests/components/mcp4725/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp4725/test.esp8266-ard.yaml b/tests/components/mcp4725/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mcp4725/test.esp8266-ard.yaml
+++ b/tests/components/mcp4725/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp4725/test.rp2040-ard.yaml b/tests/components/mcp4725/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mcp4725/test.rp2040-ard.yaml
+++ b/tests/components/mcp4725/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp4728/common.yaml b/tests/components/mcp4728/common.yaml
index b42818e4e6..e60f4795e1 100644
--- a/tests/components/mcp4728/common.yaml
+++ b/tests/components/mcp4728/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_mcp4728
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 mcp4728:
   - id: mcp4728_dac
+    i2c_id: i2c_bus
 
 output:
   - platform: mcp4728
diff --git a/tests/components/mcp4728/test.esp32-ard.yaml b/tests/components/mcp4728/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mcp4728/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mcp4728/test.esp32-c3-ard.yaml b/tests/components/mcp4728/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mcp4728/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mcp4728/test.esp32-c3-idf.yaml b/tests/components/mcp4728/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mcp4728/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp4728/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp4728/test.esp32-idf.yaml b/tests/components/mcp4728/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mcp4728/test.esp32-idf.yaml
+++ b/tests/components/mcp4728/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp4728/test.esp8266-ard.yaml b/tests/components/mcp4728/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mcp4728/test.esp8266-ard.yaml
+++ b/tests/components/mcp4728/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp4728/test.rp2040-ard.yaml b/tests/components/mcp4728/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mcp4728/test.rp2040-ard.yaml
+++ b/tests/components/mcp4728/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp47a1/common.yaml b/tests/components/mcp47a1/common.yaml
index 59e28d37b3..5a5786ff0f 100644
--- a/tests/components/mcp47a1/common.yaml
+++ b/tests/components/mcp47a1/common.yaml
@@ -1,8 +1,4 @@
-i2c:
-  - id: i2c_mcp47a1
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 output:
   - platform: mcp47a1
     id: output_mcp47a1
+    i2c_id: i2c_bus
diff --git a/tests/components/mcp47a1/test.esp32-ard.yaml b/tests/components/mcp47a1/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mcp47a1/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mcp47a1/test.esp32-c3-ard.yaml b/tests/components/mcp47a1/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mcp47a1/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mcp47a1/test.esp32-c3-idf.yaml b/tests/components/mcp47a1/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mcp47a1/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp47a1/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp47a1/test.esp32-idf.yaml b/tests/components/mcp47a1/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mcp47a1/test.esp32-idf.yaml
+++ b/tests/components/mcp47a1/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp47a1/test.esp8266-ard.yaml b/tests/components/mcp47a1/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mcp47a1/test.esp8266-ard.yaml
+++ b/tests/components/mcp47a1/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp47a1/test.rp2040-ard.yaml b/tests/components/mcp47a1/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mcp47a1/test.rp2040-ard.yaml
+++ b/tests/components/mcp47a1/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp9600/common.yaml b/tests/components/mcp9600/common.yaml
index e3c9df79e4..33e33d183e 100644
--- a/tests/components/mcp9600/common.yaml
+++ b/tests/components/mcp9600/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_mcp9600
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: mcp9600
+    i2c_id: i2c_bus
     thermocouple_type: K
     hot_junction:
       name: Thermocouple Temperature
diff --git a/tests/components/mcp9600/test.esp32-ard.yaml b/tests/components/mcp9600/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mcp9600/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mcp9600/test.esp32-c3-ard.yaml b/tests/components/mcp9600/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mcp9600/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mcp9600/test.esp32-c3-idf.yaml b/tests/components/mcp9600/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mcp9600/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp9600/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp9600/test.esp32-idf.yaml b/tests/components/mcp9600/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mcp9600/test.esp32-idf.yaml
+++ b/tests/components/mcp9600/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp9600/test.esp8266-ard.yaml b/tests/components/mcp9600/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mcp9600/test.esp8266-ard.yaml
+++ b/tests/components/mcp9600/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp9600/test.rp2040-ard.yaml b/tests/components/mcp9600/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mcp9600/test.rp2040-ard.yaml
+++ b/tests/components/mcp9600/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp9808/common.yaml b/tests/components/mcp9808/common.yaml
index ccfd5d13ce..86d0661552 100644
--- a/tests/components/mcp9808/common.yaml
+++ b/tests/components/mcp9808/common.yaml
@@ -1,8 +1,4 @@
-i2c:
-  - id: i2c_mcp9808
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: mcp9808
+    i2c_id: i2c_bus
     name: MCP9808 Temperature
diff --git a/tests/components/mcp9808/test.esp32-ard.yaml b/tests/components/mcp9808/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mcp9808/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mcp9808/test.esp32-c3-ard.yaml b/tests/components/mcp9808/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mcp9808/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mcp9808/test.esp32-c3-idf.yaml b/tests/components/mcp9808/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mcp9808/test.esp32-c3-idf.yaml
+++ b/tests/components/mcp9808/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp9808/test.esp32-idf.yaml b/tests/components/mcp9808/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mcp9808/test.esp32-idf.yaml
+++ b/tests/components/mcp9808/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp9808/test.esp8266-ard.yaml b/tests/components/mcp9808/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mcp9808/test.esp8266-ard.yaml
+++ b/tests/components/mcp9808/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mcp9808/test.rp2040-ard.yaml b/tests/components/mcp9808/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mcp9808/test.rp2040-ard.yaml
+++ b/tests/components/mcp9808/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mdns/test-comprehensive.esp8266-ard.yaml b/tests/components/mdns/test-comprehensive.esp8266-ard.yaml
new file mode 100644
index 0000000000..3129ca3143
--- /dev/null
+++ b/tests/components/mdns/test-comprehensive.esp8266-ard.yaml
@@ -0,0 +1,45 @@
+# Comprehensive ESP8266 test for mdns with multiple network components
+# Tests the complete priority chain:
+# wifi (60) -> mdns (55) -> ota (54) -> web_server_ota (52)
+
+esphome:
+  name: mdns-comprehensive-test
+
+esp8266:
+  board: esp01_1m
+
+logger:
+  level: DEBUG
+
+wifi:
+  ssid: MySSID
+  password: password1
+
+# web_server_base should run at priority 65 (before wifi)
+web_server:
+  port: 80
+
+# mdns should run at priority 55 (after wifi at 60)
+mdns:
+  services:
+    - service: _http
+      protocol: _tcp
+      port: 80
+      txt:
+        version: "1.0"
+        path: "/"
+
+# OTA should run at priority 54 (after mdns)
+ota:
+  - platform: esphome
+    password: "otapassword"
+
+# Test status LED at priority 80
+status_led:
+  pin:
+    number: GPIO2
+    inverted: true
+
+# Include API at priority 40
+api:
+  password: "apipassword"
diff --git a/tests/components/mdns/test-enabled.esp32-ard.yaml b/tests/components/mdns/test-enabled.esp32-ard.yaml
deleted file mode 100644
index 97fd63d70e..0000000000
--- a/tests/components/mdns/test-enabled.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-enabled.yaml
diff --git a/tests/components/mdns/test-enabled.esp32-c3-ard.yaml b/tests/components/mdns/test-enabled.esp32-c3-ard.yaml
deleted file mode 100644
index 97fd63d70e..0000000000
--- a/tests/components/mdns/test-enabled.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-enabled.yaml
diff --git a/tests/components/media_player/test.esp32-ard.yaml b/tests/components/media_player/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/media_player/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/mhz19/common.yaml b/tests/components/mhz19/common.yaml
index 8b7e732068..94989fecbe 100644
--- a/tests/components/mhz19/common.yaml
+++ b/tests/components/mhz19/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_mhz19
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: mhz19
     co2:
diff --git a/tests/components/mhz19/test.esp32-ard.yaml b/tests/components/mhz19/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/mhz19/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/mhz19/test.esp32-c3-ard.yaml b/tests/components/mhz19/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/mhz19/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/mhz19/test.esp32-c3-idf.yaml b/tests/components/mhz19/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/mhz19/test.esp32-c3-idf.yaml
+++ b/tests/components/mhz19/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mhz19/test.esp32-idf.yaml b/tests/components/mhz19/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/mhz19/test.esp32-idf.yaml
+++ b/tests/components/mhz19/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mhz19/test.esp8266-ard.yaml b/tests/components/mhz19/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/mhz19/test.esp8266-ard.yaml
+++ b/tests/components/mhz19/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mhz19/test.rp2040-ard.yaml b/tests/components/mhz19/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/mhz19/test.rp2040-ard.yaml
+++ b/tests/components/mhz19/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/micronova/common.yaml b/tests/components/micronova/common.yaml
index 661c9330c6..3cf8e36fb6 100644
--- a/tests/components/micronova/common.yaml
+++ b/tests/components/micronova/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_micronova
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 micronova:
   enable_rx_pin: ${enable_rx_pin}
 
diff --git a/tests/components/micronova/test.esp32-ard.yaml b/tests/components/micronova/test.esp32-ard.yaml
deleted file mode 100644
index 35d041e047..0000000000
--- a/tests/components/micronova/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-  enable_rx_pin: GPIO13
-
-<<: !include common.yaml
diff --git a/tests/components/micronova/test.esp32-c3-ard.yaml b/tests/components/micronova/test.esp32-c3-ard.yaml
deleted file mode 100644
index 993071999f..0000000000
--- a/tests/components/micronova/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  enable_rx_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/micronova/test.esp32-c3-idf.yaml b/tests/components/micronova/test.esp32-c3-idf.yaml
index 993071999f..eed876ae74 100644
--- a/tests/components/micronova/test.esp32-c3-idf.yaml
+++ b/tests/components/micronova/test.esp32-c3-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
   enable_rx_pin: GPIO3
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/micronova/test.esp32-idf.yaml b/tests/components/micronova/test.esp32-idf.yaml
index 35d041e047..5cc3a234ca 100644
--- a/tests/components/micronova/test.esp32-idf.yaml
+++ b/tests/components/micronova/test.esp32-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
   enable_rx_pin: GPIO13
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/micronova/test.esp8266-ard.yaml b/tests/components/micronova/test.esp8266-ard.yaml
index 048fb82d72..ffe1e0a063 100644
--- a/tests/components/micronova/test.esp8266-ard.yaml
+++ b/tests/components/micronova/test.esp8266-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  enable_rx_pin: GPIO13
+  enable_rx_pin: GPIO15
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/micronova/test.rp2040-ard.yaml b/tests/components/micronova/test.rp2040-ard.yaml
index 993071999f..6dc030e6b6 100644
--- a/tests/components/micronova/test.rp2040-ard.yaml
+++ b/tests/components/micronova/test.rp2040-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
   enable_rx_pin: GPIO3
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/microphone/test.esp32-ard.yaml b/tests/components/microphone/test.esp32-ard.yaml
deleted file mode 100644
index 392df582cc..0000000000
--- a/tests/components/microphone/test.esp32-ard.yaml
+++ /dev/null
@@ -1,13 +0,0 @@
-substitutions:
-  i2s_bclk_pin: GPIO15
-  i2s_lrclk_pin: GPIO16
-  i2s_mclk_pin: GPIO17
-  i2s_din_pin: GPIO33
-
-<<: !include common.yaml
-
-microphone:
-  - platform: i2s_audio
-    id: mic_id_adc
-    adc_pin: 32
-    adc_type: internal
diff --git a/tests/components/microphone/test.esp32-c3-ard.yaml b/tests/components/microphone/test.esp32-c3-ard.yaml
deleted file mode 100644
index c28dc553f5..0000000000
--- a/tests/components/microphone/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  i2s_bclk_pin: GPIO6
-  i2s_lrclk_pin: GPIO7
-  i2s_mclk_pin: GPIO8
-  i2s_din_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/microphone/test.esp32-idf.yaml b/tests/components/microphone/test.esp32-idf.yaml
index fe9feb9888..830f0156d7 100644
--- a/tests/components/microphone/test.esp32-idf.yaml
+++ b/tests/components/microphone/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
   i2s_bclk_pin: GPIO15
-  i2s_lrclk_pin: GPIO16
-  i2s_mclk_pin: GPIO17
+  i2s_lrclk_pin: GPIO4
+  i2s_mclk_pin: GPIO5
   i2s_din_pin: GPIO33
 
 i2s_audio:
diff --git a/tests/components/mics_4514/common.yaml b/tests/components/mics_4514/common.yaml
index 0bc3f3e654..3c1d264680 100644
--- a/tests/components/mics_4514/common.yaml
+++ b/tests/components/mics_4514/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_mics_4514
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: mics_4514
+    i2c_id: i2c_bus
     update_interval: 60s
     nitrogen_dioxide:
       name: MICS-4514 NO2
diff --git a/tests/components/mics_4514/test.esp32-ard.yaml b/tests/components/mics_4514/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mics_4514/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mics_4514/test.esp32-c3-ard.yaml b/tests/components/mics_4514/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mics_4514/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mics_4514/test.esp32-c3-idf.yaml b/tests/components/mics_4514/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mics_4514/test.esp32-c3-idf.yaml
+++ b/tests/components/mics_4514/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mics_4514/test.esp32-idf.yaml b/tests/components/mics_4514/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mics_4514/test.esp32-idf.yaml
+++ b/tests/components/mics_4514/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mics_4514/test.esp8266-ard.yaml b/tests/components/mics_4514/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mics_4514/test.esp8266-ard.yaml
+++ b/tests/components/mics_4514/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mics_4514/test.rp2040-ard.yaml b/tests/components/mics_4514/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mics_4514/test.rp2040-ard.yaml
+++ b/tests/components/mics_4514/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/midea/common.yaml b/tests/components/midea/common.yaml
index 07385e29a1..fec85aee96 100644
--- a/tests/components/midea/common.yaml
+++ b/tests/components/midea/common.yaml
@@ -2,16 +2,6 @@ wifi:
   ssid: MySSID
   password: password1
 
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
-uart:
-  - id: uart_midea
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 climate:
   - platform: midea
     id: midea_unit
@@ -22,7 +12,7 @@ climate:
           x.set_mode(CLIMATE_MODE_FAN_ONLY);
     on_state:
       - logger.log: State changed!
-    transmitter_id:
+    transmitter_id: xmitr
     period: 1s
     num_attempts: 5
     timeout: 2s
diff --git a/tests/components/midea/test.esp32-ard.yaml b/tests/components/midea/test.esp32-ard.yaml
index 7f55d6a52d..1e3fe0ff51 100644
--- a/tests/components/midea/test.esp32-ard.yaml
+++ b/tests/components/midea/test.esp32-ard.yaml
@@ -1,6 +1,5 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-ard.yaml
+  uart: !include ../../test_build_components/common/uart/esp32-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/midea/test.esp32-c3-ard.yaml b/tests/components/midea/test.esp32-c3-ard.yaml
deleted file mode 100644
index a879df3ca4..0000000000
--- a/tests/components/midea/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/midea/test.esp8266-ard.yaml b/tests/components/midea/test.esp8266-ard.yaml
index 4f50bd7e5e..9825ff85a1 100644
--- a/tests/components/midea/test.esp8266-ard.yaml
+++ b/tests/components/midea/test.esp8266-ard.yaml
@@ -1,6 +1,5 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  pin: GPIO15
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/midea_ir/common.yaml b/tests/components/midea_ir/common.yaml
index e8d89cecc2..e4cc4bb19c 100644
--- a/tests/components/midea_ir/common.yaml
+++ b/tests/components/midea_ir/common.yaml
@@ -1,8 +1,5 @@
-remote_transmitter:
-  pin: 4
-  carrier_duty_percent: 50%
-
 climate:
   - platform: midea_ir
     name: Midea IR
     use_fahrenheit: true
+    transmitter_id: xmitr
diff --git a/tests/components/midea_ir/test.esp32-ard.yaml b/tests/components/midea_ir/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/midea_ir/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/midea_ir/test.esp32-c3-ard.yaml b/tests/components/midea_ir/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/midea_ir/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/midea_ir/test.esp32-c3-idf.yaml b/tests/components/midea_ir/test.esp32-c3-idf.yaml
index dade44d145..43d5343715 100644
--- a/tests/components/midea_ir/test.esp32-c3-idf.yaml
+++ b/tests/components/midea_ir/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/midea_ir/test.esp32-idf.yaml b/tests/components/midea_ir/test.esp32-idf.yaml
index dade44d145..e891f9dc85 100644
--- a/tests/components/midea_ir/test.esp32-idf.yaml
+++ b/tests/components/midea_ir/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/midea_ir/test.esp8266-ard.yaml b/tests/components/midea_ir/test.esp8266-ard.yaml
index dade44d145..4bed2f03e5 100644
--- a/tests/components/midea_ir/test.esp8266-ard.yaml
+++ b/tests/components/midea_ir/test.esp8266-ard.yaml
@@ -1 +1,4 @@
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mipi_dsi/test.esp32-p4-idf.yaml b/tests/components/mipi_dsi/test.esp32-p4-idf.yaml
new file mode 100644
index 0000000000..770b11d089
--- /dev/null
+++ b/tests/components/mipi_dsi/test.esp32-p4-idf.yaml
@@ -0,0 +1,18 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-p4-idf.yaml
+
+esp_ldo:
+  - id: ldo_id
+    channel: 3
+    voltage: 2.5V
+
+display:
+  - platform: mipi_dsi
+    model: JC1060P470
+    enable_pin: GPIO22
+
+#light:
+  #- platform: mipi_dsi
+    #id: backlight_id
+
+psram:
diff --git a/tests/components/mipi_rgb/test.esp32-s3-idf.yaml b/tests/components/mipi_rgb/test.esp32-s3-idf.yaml
new file mode 100644
index 0000000000..642292f7c4
--- /dev/null
+++ b/tests/components/mipi_rgb/test.esp32-s3-idf.yaml
@@ -0,0 +1,61 @@
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-s3-idf.yaml
+
+psram:
+  mode: octal
+
+display:
+  - platform: mipi_rgb
+    spi_id: spi_bus
+    model: ZX2D10GE01R-V4848
+    update_interval: 1s
+    color_order: BGR
+    draw_rounding: 2
+    pixel_mode: 18bit
+    invert_colors: false
+    use_axis_flips: true
+    pclk_frequency: 15000000.0
+    pclk_inverted: true
+    byte_order: big_endian
+    hsync_pulse_width: 10
+    hsync_back_porch: 10
+    hsync_front_porch: 10
+    vsync_pulse_width: 2
+    vsync_back_porch: 12
+    vsync_front_porch: 14
+    data_pins:
+      red:
+        - number: 10
+        - number: 16
+        - number: 9
+        - number: 15
+        - number: 46
+          ignore_strapping_warning: true
+      green:
+        - number: 8
+        - number: 13
+        - number: 18
+        - number: 12
+        - number: 11
+        - number: 17
+      blue:
+        - number: 47
+        - number: 1
+        - number: 0
+          ignore_strapping_warning: true
+        - number: 42
+        - number: 14
+    de_pin:
+      number: 39
+    pclk_pin:
+      number: 45
+      ignore_strapping_warning: true
+    hsync_pin:
+      number: 38
+    vsync_pin:
+      number: 48
+    data_rate: 1000000.0
+    spi_mode: MODE0
+    cs_pin:
+      number: 21
+    show_test_card: true
diff --git a/tests/components/mipi_spi/common.yaml b/tests/components/mipi_spi/common.yaml
index 2c84489ec7..692a9f436e 100644
--- a/tests/components/mipi_spi/common.yaml
+++ b/tests/components/mipi_spi/common.yaml
@@ -1,11 +1,3 @@
-spi:
-  - id: spi_single
-    clk_pin:
-      number: ${clk_pin}
-      allow_other_uses: true
-    mosi_pin:
-      number: ${mosi_pin}
-
 display:
   - platform: mipi_spi
     spi_16: true
@@ -18,7 +10,7 @@ display:
     invert_colors: true
     show_test_card: true
     spi_mode: mode0
-    draw_rounding: 8
+    draw_rounding: 4
     use_axis_flips: true
     init_sequence:
       - [0xd0, 1, 2, 3]
@@ -30,8 +22,5 @@ display:
     dimensions:
       width: 100
       height: 200
-    enable_pin:
-      - number: ${clk_pin}
-        allow_other_uses: true
-      - number: ${enable_pin}
+    enable_pin: ${enable_pin}
     bus_mode: single
diff --git a/tests/components/mipi_spi/test-esp32-2432s028.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-esp32-2432s028.esp32-s3-idf.yaml
deleted file mode 100644
index a28776798c..0000000000
--- a/tests/components/mipi_spi/test-esp32-2432s028.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,41 +0,0 @@
-spi:
-  - id: quad_spi
-    type: quad
-    interface: spi3
-    clk_pin:
-      number: 47
-    data_pins:
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: octal_spi
-    type: octal
-    interface: hardware
-    clk_pin:
-      number: 0
-    data_pins:
-      - 36
-      - 37
-      - 38
-      - 39
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: spi_id_3
-    interface: any
-    clk_pin: 8
-    mosi_pin: 9
-
-display:
-  - platform: mipi_spi
-    model: ESP32-2432S028
diff --git a/tests/components/mipi_spi/test-jc3248w535.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-jc3248w535.esp32-s3-idf.yaml
deleted file mode 100644
index 02b8f78d58..0000000000
--- a/tests/components/mipi_spi/test-jc3248w535.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,41 +0,0 @@
-spi:
-  - id: quad_spi
-    type: quad
-    interface: spi3
-    clk_pin:
-      number: 47
-    data_pins:
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: octal_spi
-    type: octal
-    interface: hardware
-    clk_pin:
-      number: 0
-    data_pins:
-      - 36
-      - 37
-      - 38
-      - 39
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: spi_id_3
-    interface: any
-    clk_pin: 8
-    mosi_pin: 9
-
-display:
-  - platform: mipi_spi
-    model: JC3248W535
diff --git a/tests/components/mipi_spi/test-jc3636w518.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-jc3636w518.esp32-s3-idf.yaml
deleted file mode 100644
index 147d4833ac..0000000000
--- a/tests/components/mipi_spi/test-jc3636w518.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,19 +0,0 @@
-spi:
-  - id: quad_spi
-    type: quad
-    interface: spi3
-    clk_pin:
-      number: 36
-    data_pins:
-      - number: 40
-      - number: 41
-      - number: 42
-      - number: 43
-  - id: spi_id_3
-    interface: any
-    clk_pin: 8
-    mosi_pin: 9
-
-display:
-  - platform: mipi_spi
-    model: JC3636W518
diff --git a/tests/components/mipi_spi/test-lvgl.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-lvgl.esp32-s3-idf.yaml
new file mode 100644
index 0000000000..48f34f3449
--- /dev/null
+++ b/tests/components/mipi_spi/test-lvgl.esp32-s3-idf.yaml
@@ -0,0 +1,11 @@
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-s3-idf.yaml
+
+display:
+  - platform: mipi_spi
+    spi_id: spi_bus
+    model: t-display-s3-pro
+
+lvgl:
+
+psram:
diff --git a/tests/components/mipi_spi/test-pico-restouch-lcd-35.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-pico-restouch-lcd-35.esp32-s3-idf.yaml
deleted file mode 100644
index 8d96f31fd5..0000000000
--- a/tests/components/mipi_spi/test-pico-restouch-lcd-35.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-spi:
-  - id: spi_id_3
-    interface: any
-    clk_pin: 8
-    mosi_pin: 9
-
-display:
-  - platform: mipi_spi
-    model: Pico-ResTouch-LCD-3.5
diff --git a/tests/components/mipi_spi/test-s3box.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-s3box.esp32-s3-idf.yaml
deleted file mode 100644
index 98f6955bf3..0000000000
--- a/tests/components/mipi_spi/test-s3box.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,41 +0,0 @@
-spi:
-  - id: quad_spi
-    type: quad
-    interface: spi3
-    clk_pin:
-      number: 47
-    data_pins:
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: octal_spi
-    type: octal
-    interface: hardware
-    clk_pin:
-      number: 0
-    data_pins:
-      - 36
-      - 37
-      - 38
-      - 39
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: spi_id_3
-    interface: any
-    clk_pin: 8
-    mosi_pin: 9
-
-display:
-  - platform: mipi_spi
-    model: S3BOX
diff --git a/tests/components/mipi_spi/test-s3boxlite.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-s3boxlite.esp32-s3-idf.yaml
deleted file mode 100644
index 11ad869d54..0000000000
--- a/tests/components/mipi_spi/test-s3boxlite.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,41 +0,0 @@
-spi:
-  - id: quad_spi
-    type: quad
-    interface: spi3
-    clk_pin:
-      number: 47
-    data_pins:
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: octal_spi
-    type: octal
-    interface: hardware
-    clk_pin:
-      number: 0
-    data_pins:
-      - 36
-      - 37
-      - 38
-      - 39
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: spi_id_3
-    interface: any
-    clk_pin: 8
-    mosi_pin: 9
-
-display:
-  - platform: mipi_spi
-    model: S3BOXLITE
diff --git a/tests/components/mipi_spi/test-t-display-s3-amoled-plus.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-t-display-s3-amoled-plus.esp32-s3-idf.yaml
deleted file mode 100644
index dc328f950c..0000000000
--- a/tests/components/mipi_spi/test-t-display-s3-amoled-plus.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-spi:
-  - id: spi_id_3
-    interface: any
-    clk_pin: 8
-    mosi_pin: 9
-
-display:
-  - platform: mipi_spi
-    model: T-DISPLAY-S3-AMOLED-PLUS
diff --git a/tests/components/mipi_spi/test-t-display-s3-amoled.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-t-display-s3-amoled.esp32-s3-idf.yaml
deleted file mode 100644
index f0432270dc..0000000000
--- a/tests/components/mipi_spi/test-t-display-s3-amoled.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,15 +0,0 @@
-spi:
-  - id: quad_spi
-    type: quad
-    interface: spi3
-    clk_pin:
-      number: 47
-    data_pins:
-      - number: 40
-      - number: 41
-      - number: 42
-      - number: 43
-
-display:
-  - platform: mipi_spi
-    model: T-DISPLAY-S3-AMOLED
diff --git a/tests/components/mipi_spi/test-t-display-s3-pro.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-t-display-s3-pro.esp32-s3-idf.yaml
deleted file mode 100644
index 5cda38e096..0000000000
--- a/tests/components/mipi_spi/test-t-display-s3-pro.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-spi:
-  - id: spi_id_3
-    interface: any
-    clk_pin: 8
-    mosi_pin: 40
-
-display:
-  - platform: mipi_spi
-    model: T-DISPLAY-S3-PRO
diff --git a/tests/components/mipi_spi/test-t-display-s3.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-t-display-s3.esp32-s3-idf.yaml
deleted file mode 100644
index 144bde8366..0000000000
--- a/tests/components/mipi_spi/test-t-display-s3.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,37 +0,0 @@
-spi:
-  - id: quad_spi
-    type: quad
-    interface: spi3
-    clk_pin:
-      number: 47
-    data_pins:
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: octal_spi
-    type: octal
-    interface: hardware
-    clk_pin:
-      number: 0
-    data_pins:
-      - 36
-      - 37
-      - 38
-      - 39
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-
-display:
-  - platform: mipi_spi
-    model: T-DISPLAY-S3
diff --git a/tests/components/mipi_spi/test-t-display.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-t-display.esp32-s3-idf.yaml
deleted file mode 100644
index 39339b5ae2..0000000000
--- a/tests/components/mipi_spi/test-t-display.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,41 +0,0 @@
-spi:
-  - id: quad_spi
-    type: quad
-    interface: spi3
-    clk_pin:
-      number: 47
-    data_pins:
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: octal_spi
-    type: octal
-    interface: hardware
-    clk_pin:
-      number: 0
-    data_pins:
-      - 36
-      - 37
-      - 38
-      - 39
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: spi_id_3
-    interface: any
-    clk_pin: 8
-    mosi_pin: 9
-
-display:
-  - platform: mipi_spi
-    model: T-DISPLAY
diff --git a/tests/components/mipi_spi/test-t-embed.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-t-embed.esp32-s3-idf.yaml
deleted file mode 100644
index 6c9edb25b3..0000000000
--- a/tests/components/mipi_spi/test-t-embed.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-spi:
-  - id: spi_id_3
-    interface: any
-    clk_pin: 8
-    mosi_pin: 40
-
-display:
-  - platform: mipi_spi
-    model: T-EMBED
diff --git a/tests/components/mipi_spi/test-t4-s3.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-t4-s3.esp32-s3-idf.yaml
deleted file mode 100644
index 46eaedb7cb..0000000000
--- a/tests/components/mipi_spi/test-t4-s3.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,41 +0,0 @@
-spi:
-  - id: quad_spi
-    type: quad
-    interface: spi3
-    clk_pin:
-      number: 47
-    data_pins:
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: octal_spi
-    type: octal
-    interface: hardware
-    clk_pin:
-      number: 0
-    data_pins:
-      - 36
-      - 37
-      - 38
-      - 39
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: spi_id_3
-    interface: any
-    clk_pin: 8
-    mosi_pin: 9
-
-display:
-  - platform: mipi_spi
-    model: T4-S3
diff --git a/tests/components/mipi_spi/test-wt32-sc01-plus.esp32-s3-idf.yaml b/tests/components/mipi_spi/test-wt32-sc01-plus.esp32-s3-idf.yaml
deleted file mode 100644
index 3efb05ec89..0000000000
--- a/tests/components/mipi_spi/test-wt32-sc01-plus.esp32-s3-idf.yaml
+++ /dev/null
@@ -1,37 +0,0 @@
-spi:
-  - id: quad_spi
-    type: quad
-    interface: spi3
-    clk_pin:
-      number: 47
-    data_pins:
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-  - id: octal_spi
-    type: octal
-    interface: hardware
-    clk_pin:
-      number: 9
-    data_pins:
-      - 36
-      - 37
-      - 38
-      - 39
-      - allow_other_uses: true
-        number: 40
-      - allow_other_uses: true
-        number: 41
-      - allow_other_uses: true
-        number: 42
-      - allow_other_uses: true
-        number: 43
-
-display:
-  - platform: mipi_spi
-    model: WT32-SC01-PLUS
diff --git a/tests/components/mipi_spi/test.esp32-ard.yaml b/tests/components/mipi_spi/test.esp32-ard.yaml
deleted file mode 100644
index a5ef77dabc..0000000000
--- a/tests/components/mipi_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,15 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  dc_pin: GPIO14
-  cs_pin: GPIO13
-  enable_pin: GPIO19
-  reset_pin: GPIO20
-
-display:
-  - platform: mipi_spi
-    model: LANBON-L8
-
-packages:
-  display: !include common.yaml
diff --git a/tests/components/mipi_spi/test.esp32-c3-ard.yaml b/tests/components/mipi_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index c17748c569..0000000000
--- a/tests/components/mipi_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,10 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  dc_pin: GPIO21
-  cs_pin: GPIO18
-  enable_pin: GPIO19
-  reset_pin: GPIO20
-
-<<: !include common.yaml
diff --git a/tests/components/mipi_spi/test.esp32-c3-idf.yaml b/tests/components/mipi_spi/test.esp32-c3-idf.yaml
index c17748c569..55e25d8318 100644
--- a/tests/components/mipi_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/mipi_spi/test.esp32-c3-idf.yaml
@@ -1,10 +1,10 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  dc_pin: GPIO21
-  cs_pin: GPIO18
-  enable_pin: GPIO19
+  dc_pin: GPIO7
+  cs_pin: GPIO8
+  enable_pin: GPIO9
   reset_pin: GPIO20
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mipi_spi/test.esp32-idf.yaml b/tests/components/mipi_spi/test.esp32-idf.yaml
index 653ccb4910..b173b8de87 100644
--- a/tests/components/mipi_spi/test.esp32-idf.yaml
+++ b/tests/components/mipi_spi/test.esp32-idf.yaml
@@ -1,15 +1,10 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  dc_pin: GPIO21
-  cs_pin: GPIO18
-  enable_pin: GPIO19
+  dc_pin: GPIO14
+  cs_pin: GPIO13
+  enable_pin: GPIO4
   reset_pin: GPIO20
 
 packages:
-  display: !include common.yaml
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
-display:
-  - platform: mipi_spi
-    model: m5core
+<<: !include common.yaml
diff --git a/tests/components/mipi_spi/test.rp2040-ard.yaml b/tests/components/mipi_spi/test.rp2040-ard.yaml
index 5d7333853b..6336652999 100644
--- a/tests/components/mipi_spi/test.rp2040-ard.yaml
+++ b/tests/components/mipi_spi/test.rp2040-ard.yaml
@@ -1,10 +1,10 @@
 substitutions:
-  clk_pin: GPIO2
-  mosi_pin: GPIO3
-  miso_pin: GPIO4
   dc_pin: GPIO14
   cs_pin: GPIO13
-  enable_pin: GPIO19
+  enable_pin: GPIO17
   reset_pin: GPIO20
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mitsubishi/common.yaml b/tests/components/mitsubishi/common.yaml
index c0fc959c5b..4a2deda163 100644
--- a/tests/components/mitsubishi/common.yaml
+++ b/tests/components/mitsubishi/common.yaml
@@ -1,7 +1,4 @@
-remote_transmitter:
-  pin: 4
-  carrier_duty_percent: 50%
-
 climate:
   - platform: mitsubishi
     name: Mitsubishi
+    transmitter_id: xmitr
diff --git a/tests/components/mitsubishi/test.esp32-ard.yaml b/tests/components/mitsubishi/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/mitsubishi/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/mitsubishi/test.esp32-c3-ard.yaml b/tests/components/mitsubishi/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/mitsubishi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/mitsubishi/test.esp32-c3-idf.yaml b/tests/components/mitsubishi/test.esp32-c3-idf.yaml
index dade44d145..43d5343715 100644
--- a/tests/components/mitsubishi/test.esp32-c3-idf.yaml
+++ b/tests/components/mitsubishi/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mitsubishi/test.esp32-idf.yaml b/tests/components/mitsubishi/test.esp32-idf.yaml
index dade44d145..e891f9dc85 100644
--- a/tests/components/mitsubishi/test.esp32-idf.yaml
+++ b/tests/components/mitsubishi/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mitsubishi/test.esp8266-ard.yaml b/tests/components/mitsubishi/test.esp8266-ard.yaml
index dade44d145..4bed2f03e5 100644
--- a/tests/components/mitsubishi/test.esp8266-ard.yaml
+++ b/tests/components/mitsubishi/test.esp8266-ard.yaml
@@ -1 +1,4 @@
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mixer/test.esp32-ard.yaml b/tests/components/mixer/test.esp32-ard.yaml
deleted file mode 100644
index 96d2d37458..0000000000
--- a/tests/components/mixer/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  lrclk_pin: GPIO16
-  bclk_pin: GPIO17
-  mclk_pin: GPIO15
-  dout_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/mixer/test.esp32-c3-ard.yaml b/tests/components/mixer/test.esp32-c3-ard.yaml
deleted file mode 100644
index f1721f0862..0000000000
--- a/tests/components/mixer/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  lrclk_pin: GPIO4
-  bclk_pin: GPIO5
-  mclk_pin: GPIO6
-  dout_pin: GPIO7
-
-<<: !include common.yaml
diff --git a/tests/components/mixer/test.esp32-idf.yaml b/tests/components/mixer/test.esp32-idf.yaml
index 96d2d37458..6712f1e468 100644
--- a/tests/components/mixer/test.esp32-idf.yaml
+++ b/tests/components/mixer/test.esp32-idf.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  lrclk_pin: GPIO16
-  bclk_pin: GPIO17
+  lrclk_pin: GPIO4
+  bclk_pin: GPIO5
   mclk_pin: GPIO15
   dout_pin: GPIO14
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mixer/test.esp32-s3-ard.yaml b/tests/components/mixer/test.esp32-s3-ard.yaml
deleted file mode 100644
index f1721f0862..0000000000
--- a/tests/components/mixer/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  lrclk_pin: GPIO4
-  bclk_pin: GPIO5
-  mclk_pin: GPIO6
-  dout_pin: GPIO7
-
-<<: !include common.yaml
diff --git a/tests/components/mlx90393/common.yaml b/tests/components/mlx90393/common.yaml
index 58f3b6ecf5..9e85e06c89 100644
--- a/tests/components/mlx90393/common.yaml
+++ b/tests/components/mlx90393/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_mlx90393
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: mlx90393
+    i2c_id: i2c_bus
     oversampling: 3
     gain: 1X
     temperature_compensation: true
diff --git a/tests/components/mlx90393/test.esp32-ard.yaml b/tests/components/mlx90393/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mlx90393/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mlx90393/test.esp32-c3-ard.yaml b/tests/components/mlx90393/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mlx90393/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mlx90393/test.esp32-c3-idf.yaml b/tests/components/mlx90393/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mlx90393/test.esp32-c3-idf.yaml
+++ b/tests/components/mlx90393/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mlx90393/test.esp32-idf.yaml b/tests/components/mlx90393/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mlx90393/test.esp32-idf.yaml
+++ b/tests/components/mlx90393/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mlx90393/test.esp32-s3-ard.yaml b/tests/components/mlx90393/test.esp32-s3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mlx90393/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mlx90393/test.esp32-s3-idf.yaml b/tests/components/mlx90393/test.esp32-s3-idf.yaml
index ee2c29ca4e..0fd8684a2c 100644
--- a/tests/components/mlx90393/test.esp32-s3-idf.yaml
+++ b/tests/components/mlx90393/test.esp32-s3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-s3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mlx90393/test.esp8266-ard.yaml b/tests/components/mlx90393/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mlx90393/test.esp8266-ard.yaml
+++ b/tests/components/mlx90393/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mlx90393/test.rp2040-ard.yaml b/tests/components/mlx90393/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mlx90393/test.rp2040-ard.yaml
+++ b/tests/components/mlx90393/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mlx90614/common.yaml b/tests/components/mlx90614/common.yaml
index 03279e6e14..8d408fb016 100644
--- a/tests/components/mlx90614/common.yaml
+++ b/tests/components/mlx90614/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_mlx90614
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: mlx90614
+    i2c_id: i2c_bus
     ambient:
       name: Ambient
     object:
diff --git a/tests/components/mlx90614/test.esp32-ard.yaml b/tests/components/mlx90614/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mlx90614/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mlx90614/test.esp32-c3-ard.yaml b/tests/components/mlx90614/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mlx90614/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mlx90614/test.esp32-c3-idf.yaml b/tests/components/mlx90614/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mlx90614/test.esp32-c3-idf.yaml
+++ b/tests/components/mlx90614/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mlx90614/test.esp32-idf.yaml b/tests/components/mlx90614/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mlx90614/test.esp32-idf.yaml
+++ b/tests/components/mlx90614/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mlx90614/test.esp8266-ard.yaml b/tests/components/mlx90614/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mlx90614/test.esp8266-ard.yaml
+++ b/tests/components/mlx90614/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mlx90614/test.rp2040-ard.yaml b/tests/components/mlx90614/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mlx90614/test.rp2040-ard.yaml
+++ b/tests/components/mlx90614/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mmc5603/common.yaml b/tests/components/mmc5603/common.yaml
index 83235f596e..6f6e35e9af 100644
--- a/tests/components/mmc5603/common.yaml
+++ b/tests/components/mmc5603/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_mmc5603
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: mmc5603
+    i2c_id: i2c_bus
     address: 0x30
     field_strength_x:
       name: HMC5883L Field Strength X
diff --git a/tests/components/mmc5603/test.esp32-ard.yaml b/tests/components/mmc5603/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mmc5603/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mmc5603/test.esp32-c3-ard.yaml b/tests/components/mmc5603/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mmc5603/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mmc5603/test.esp32-c3-idf.yaml b/tests/components/mmc5603/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mmc5603/test.esp32-c3-idf.yaml
+++ b/tests/components/mmc5603/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mmc5603/test.esp32-idf.yaml b/tests/components/mmc5603/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mmc5603/test.esp32-idf.yaml
+++ b/tests/components/mmc5603/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mmc5603/test.esp8266-ard.yaml b/tests/components/mmc5603/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mmc5603/test.esp8266-ard.yaml
+++ b/tests/components/mmc5603/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mmc5603/test.rp2040-ard.yaml b/tests/components/mmc5603/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mmc5603/test.rp2040-ard.yaml
+++ b/tests/components/mmc5603/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mmc5983/common.yaml b/tests/components/mmc5983/common.yaml
index 949ff527e7..963a2527c1 100644
--- a/tests/components/mmc5983/common.yaml
+++ b/tests/components/mmc5983/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_mmc5983
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: mmc5983
+    i2c_id: i2c_bus
     field_strength_x:
       name: "Magnet X"
       id: magnet_x
diff --git a/tests/components/mmc5983/test.esp32-ard.yaml b/tests/components/mmc5983/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mmc5983/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mmc5983/test.esp32-c3-ard.yaml b/tests/components/mmc5983/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mmc5983/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mmc5983/test.esp32-c3-idf.yaml b/tests/components/mmc5983/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mmc5983/test.esp32-c3-idf.yaml
+++ b/tests/components/mmc5983/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mmc5983/test.esp32-idf.yaml b/tests/components/mmc5983/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mmc5983/test.esp32-idf.yaml
+++ b/tests/components/mmc5983/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mmc5983/test.esp8266-ard.yaml b/tests/components/mmc5983/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mmc5983/test.esp8266-ard.yaml
+++ b/tests/components/mmc5983/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mmc5983/test.rp2040-ard.yaml b/tests/components/mmc5983/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mmc5983/test.rp2040-ard.yaml
+++ b/tests/components/mmc5983/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/modbus/common.yaml b/tests/components/modbus/common.yaml
index 3ec9518585..d636143ec9 100644
--- a/tests/components/modbus/common.yaml
+++ b/tests/components/modbus/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_modbus
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 modbus:
   id: mod_bus1
   flow_control_pin: ${flow_control_pin}
diff --git a/tests/components/modbus/test.esp32-ard.yaml b/tests/components/modbus/test.esp32-ard.yaml
deleted file mode 100644
index bd767a8ece..0000000000
--- a/tests/components/modbus/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO12
-  rx_pin: GPIO14
-  flow_control_pin: GPIO13
-
-<<: !include common.yaml
diff --git a/tests/components/modbus/test.esp32-c3-ard.yaml b/tests/components/modbus/test.esp32-c3-ard.yaml
deleted file mode 100644
index 452031a5aa..0000000000
--- a/tests/components/modbus/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  flow_control_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/modbus/test.esp32-c3-idf.yaml b/tests/components/modbus/test.esp32-c3-idf.yaml
index 452031a5aa..430c6818cb 100644
--- a/tests/components/modbus/test.esp32-c3-idf.yaml
+++ b/tests/components/modbus/test.esp32-c3-idf.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
   flow_control_pin: GPIO3
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/modbus/test.esp32-idf.yaml b/tests/components/modbus/test.esp32-idf.yaml
index bd767a8ece..8a08f8a821 100644
--- a/tests/components/modbus/test.esp32-idf.yaml
+++ b/tests/components/modbus/test.esp32-idf.yaml
@@ -3,4 +3,7 @@ substitutions:
   rx_pin: GPIO14
   flow_control_pin: GPIO13
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/modbus/test.esp8266-ard.yaml b/tests/components/modbus/test.esp8266-ard.yaml
index 29c98d0957..dfea36d957 100644
--- a/tests/components/modbus/test.esp8266-ard.yaml
+++ b/tests/components/modbus/test.esp8266-ard.yaml
@@ -1,6 +1,9 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  flow_control_pin: GPIO13
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+  flow_control_pin: GPIO15
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/modbus/test.rp2040-ard.yaml b/tests/components/modbus/test.rp2040-ard.yaml
index 452031a5aa..43f12ea28d 100644
--- a/tests/components/modbus/test.rp2040-ard.yaml
+++ b/tests/components/modbus/test.rp2040-ard.yaml
@@ -3,4 +3,7 @@ substitutions:
   rx_pin: GPIO5
   flow_control_pin: GPIO3
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/modbus_controller/common.yaml b/tests/components/modbus_controller/common.yaml
index 7d342ee353..ae5520e57d 100644
--- a/tests/components/modbus_controller/common.yaml
+++ b/tests/components/modbus_controller/common.yaml
@@ -1,25 +1,12 @@
-uart:
-  - id: uart_modbus_client
-    tx_pin: ${client_tx_pin}
-    rx_pin: ${client_rx_pin}
-    baud_rate: 9600
-  - id: uart_modbus_server
-    tx_pin: ${server_tx_pin}
-    rx_pin: ${server_rx_pin}
-    baud_rate: 9600
-
 modbus:
-  - id: mod_bus1
-    uart_id: uart_modbus_client
-    flow_control_pin: ${flow_control_pin}
   - id: mod_bus2
-    uart_id: uart_modbus_server
+    uart_id: uart_bus
     role: server
 
 modbus_controller:
   - id: modbus_controller1
     address: 0x2
-    modbus_id: mod_bus1
+    modbus_id: modbus_bus
     allow_duplicate_commands: false
     on_online:
       then:
@@ -45,6 +32,22 @@ modbus_controller:
           printf("address=%d, value=%d", x);
           return true;
     max_cmd_retries: 0
+  - id: modbus_controller4
+    modbus_id: mod_bus2
+    address: 0x4
+    server_courtesy_response:
+      enabled: true
+      register_last_address: 100
+      register_value: 0
+    server_registers:
+      - address: 0x0001
+        value_type: U_WORD
+        read_lambda: |-
+          return 0x8;
+      - address: 0x0005
+        value_type: U_WORD
+        read_lambda: |-
+          return (random_uint32() % 100);
 binary_sensor:
   - platform: modbus_controller
     modbus_controller_id: modbus_controller1
diff --git a/tests/components/modbus_controller/test.esp32-ard.yaml b/tests/components/modbus_controller/test.esp32-ard.yaml
deleted file mode 100644
index 548b8c0666..0000000000
--- a/tests/components/modbus_controller/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  client_tx_pin: GPIO12
-  client_rx_pin: GPIO14
-  server_tx_pin: GPIO16
-  server_rx_pin: GPIO17
-  flow_control_pin: GPIO13
-
-<<: !include common.yaml
diff --git a/tests/components/modbus_controller/test.esp32-c3-ard.yaml b/tests/components/modbus_controller/test.esp32-c3-ard.yaml
deleted file mode 100644
index f5b770ff58..0000000000
--- a/tests/components/modbus_controller/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  client_tx_pin: GPIO4
-  client_rx_pin: GPIO5
-  server_tx_pin: GPIO6
-  server_rx_pin: GPIO7
-  flow_control_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/modbus_controller/test.esp32-c3-idf.yaml b/tests/components/modbus_controller/test.esp32-c3-idf.yaml
index f5b770ff58..db826676ee 100644
--- a/tests/components/modbus_controller/test.esp32-c3-idf.yaml
+++ b/tests/components/modbus_controller/test.esp32-c3-idf.yaml
@@ -1,8 +1,4 @@
-substitutions:
-  client_tx_pin: GPIO4
-  client_rx_pin: GPIO5
-  server_tx_pin: GPIO6
-  server_rx_pin: GPIO7
-  flow_control_pin: GPIO3
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/modbus_controller/test.esp32-idf.yaml b/tests/components/modbus_controller/test.esp32-idf.yaml
index 548b8c0666..ace2d95a0b 100644
--- a/tests/components/modbus_controller/test.esp32-idf.yaml
+++ b/tests/components/modbus_controller/test.esp32-idf.yaml
@@ -1,8 +1,4 @@
-substitutions:
-  client_tx_pin: GPIO12
-  client_rx_pin: GPIO14
-  server_tx_pin: GPIO16
-  server_rx_pin: GPIO17
-  flow_control_pin: GPIO13
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/modbus_controller/test.esp8266-ard.yaml b/tests/components/modbus_controller/test.esp8266-ard.yaml
index c68a57cbde..560629b0cd 100644
--- a/tests/components/modbus_controller/test.esp8266-ard.yaml
+++ b/tests/components/modbus_controller/test.esp8266-ard.yaml
@@ -1,8 +1,4 @@
-substitutions:
-  client_tx_pin: GPIO1
-  client_rx_pin: GPIO3
-  server_tx_pin: GPIO4
-  server_rx_pin: GPIO5
-  flow_control_pin: GPIO13
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/modbus_controller/test.rp2040-ard.yaml b/tests/components/modbus_controller/test.rp2040-ard.yaml
index 80528bc12b..eeebbd2a8a 100644
--- a/tests/components/modbus_controller/test.rp2040-ard.yaml
+++ b/tests/components/modbus_controller/test.rp2040-ard.yaml
@@ -1,8 +1,4 @@
-substitutions:
-  client_tx_pin: GPIO2
-  client_rx_pin: GPIO3
-  server_tx_pin: GPIO4
-  server_rx_pin: GPIO5
-  flow_control_pin: GPIO6
+packages:
+  modbus: !include ../../test_build_components/common/modbus/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/monochromatic/test.esp32-ard.yaml b/tests/components/monochromatic/test.esp32-ard.yaml
deleted file mode 100644
index feabf013fd..0000000000
--- a/tests/components/monochromatic/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/monochromatic/test.esp32-c3-ard.yaml b/tests/components/monochromatic/test.esp32-c3-ard.yaml
deleted file mode 100644
index feabf013fd..0000000000
--- a/tests/components/monochromatic/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/mopeka_ble/test.esp32-ard.yaml b/tests/components/mopeka_ble/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/mopeka_ble/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/mopeka_ble/test.esp32-c3-ard.yaml b/tests/components/mopeka_ble/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/mopeka_ble/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/mopeka_ble/test.esp32-c3-idf.yaml b/tests/components/mopeka_ble/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/mopeka_ble/test.esp32-c3-idf.yaml
+++ b/tests/components/mopeka_ble/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mopeka_ble/test.esp32-idf.yaml b/tests/components/mopeka_ble/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/mopeka_ble/test.esp32-idf.yaml
+++ b/tests/components/mopeka_ble/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mopeka_pro_check/test.esp32-ard.yaml b/tests/components/mopeka_pro_check/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/mopeka_pro_check/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/mopeka_pro_check/test.esp32-c3-ard.yaml b/tests/components/mopeka_pro_check/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/mopeka_pro_check/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/mopeka_pro_check/test.esp32-c3-idf.yaml b/tests/components/mopeka_pro_check/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/mopeka_pro_check/test.esp32-c3-idf.yaml
+++ b/tests/components/mopeka_pro_check/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mopeka_pro_check/test.esp32-idf.yaml b/tests/components/mopeka_pro_check/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/mopeka_pro_check/test.esp32-idf.yaml
+++ b/tests/components/mopeka_pro_check/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mopeka_std_check/test.esp32-ard.yaml b/tests/components/mopeka_std_check/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/mopeka_std_check/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/mopeka_std_check/test.esp32-c3-ard.yaml b/tests/components/mopeka_std_check/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/mopeka_std_check/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/mopeka_std_check/test.esp32-c3-idf.yaml b/tests/components/mopeka_std_check/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/mopeka_std_check/test.esp32-c3-idf.yaml
+++ b/tests/components/mopeka_std_check/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mopeka_std_check/test.esp32-idf.yaml b/tests/components/mopeka_std_check/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/mopeka_std_check/test.esp32-idf.yaml
+++ b/tests/components/mopeka_std_check/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mpl3115a2/common.yaml b/tests/components/mpl3115a2/common.yaml
index f2f65885b3..aa6c2b77fc 100644
--- a/tests/components/mpl3115a2/common.yaml
+++ b/tests/components/mpl3115a2/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_mpl3115a2
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: mpl3115a2
+    i2c_id: i2c_bus
     temperature:
       name: MPL3115A2 Temperature
     pressure:
diff --git a/tests/components/mpl3115a2/test.esp32-ard.yaml b/tests/components/mpl3115a2/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mpl3115a2/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mpl3115a2/test.esp32-c3-ard.yaml b/tests/components/mpl3115a2/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mpl3115a2/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mpl3115a2/test.esp32-c3-idf.yaml b/tests/components/mpl3115a2/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mpl3115a2/test.esp32-c3-idf.yaml
+++ b/tests/components/mpl3115a2/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mpl3115a2/test.esp32-idf.yaml b/tests/components/mpl3115a2/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mpl3115a2/test.esp32-idf.yaml
+++ b/tests/components/mpl3115a2/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mpl3115a2/test.esp8266-ard.yaml b/tests/components/mpl3115a2/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mpl3115a2/test.esp8266-ard.yaml
+++ b/tests/components/mpl3115a2/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mpl3115a2/test.rp2040-ard.yaml b/tests/components/mpl3115a2/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mpl3115a2/test.rp2040-ard.yaml
+++ b/tests/components/mpl3115a2/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mpr121/common.yaml b/tests/components/mpr121/common.yaml
index fcf61b57f3..67a06cf9c1 100644
--- a/tests/components/mpr121/common.yaml
+++ b/tests/components/mpr121/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_mpr121
-    scl: ${i2c_scl}
-    sda: ${i2c_sda}
-
 mpr121:
+  i2c_id: i2c_bus
   id: mpr121_first
   address: 0x5A
 
diff --git a/tests/components/mpr121/test.esp32-ard.yaml b/tests/components/mpr121/test.esp32-ard.yaml
deleted file mode 100644
index 1037d5d35b..0000000000
--- a/tests/components/mpr121/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  i2c_scl: GPIO16
-  i2c_sda: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mpr121/test.esp32-c3-ard.yaml b/tests/components/mpr121/test.esp32-c3-ard.yaml
deleted file mode 100644
index d7ae0d5161..0000000000
--- a/tests/components/mpr121/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  i2c_scl: GPIO5
-  i2c_sda: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mpr121/test.esp32-c3-idf.yaml b/tests/components/mpr121/test.esp32-c3-idf.yaml
index d7ae0d5161..d1abb03369 100644
--- a/tests/components/mpr121/test.esp32-c3-idf.yaml
+++ b/tests/components/mpr121/test.esp32-c3-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   i2c_scl: GPIO5
   i2c_sda: GPIO4
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mpr121/test.esp32-idf.yaml b/tests/components/mpr121/test.esp32-idf.yaml
index 1037d5d35b..4598505c3a 100644
--- a/tests/components/mpr121/test.esp32-idf.yaml
+++ b/tests/components/mpr121/test.esp32-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   i2c_scl: GPIO16
   i2c_sda: GPIO17
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mpr121/test.esp8266-ard.yaml b/tests/components/mpr121/test.esp8266-ard.yaml
index d7ae0d5161..5565bb8c35 100644
--- a/tests/components/mpr121/test.esp8266-ard.yaml
+++ b/tests/components/mpr121/test.esp8266-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   i2c_scl: GPIO5
   i2c_sda: GPIO4
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mpr121/test.rp2040-ard.yaml b/tests/components/mpr121/test.rp2040-ard.yaml
index d7ae0d5161..888762a742 100644
--- a/tests/components/mpr121/test.rp2040-ard.yaml
+++ b/tests/components/mpr121/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   i2c_scl: GPIO5
   i2c_sda: GPIO4
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/mpu6050/common.yaml b/tests/components/mpu6050/common.yaml
index e9d4995534..7ac123f8c7 100644
--- a/tests/components/mpu6050/common.yaml
+++ b/tests/components/mpu6050/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_mpu6050
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: mpu6050
+    i2c_id: i2c_bus
     address: 0x68
     accel_x:
       name: MPU6050 Accel X
diff --git a/tests/components/mpu6050/test.esp32-ard.yaml b/tests/components/mpu6050/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mpu6050/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mpu6050/test.esp32-c3-ard.yaml b/tests/components/mpu6050/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mpu6050/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mpu6050/test.esp32-c3-idf.yaml b/tests/components/mpu6050/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mpu6050/test.esp32-c3-idf.yaml
+++ b/tests/components/mpu6050/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mpu6050/test.esp32-idf.yaml b/tests/components/mpu6050/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mpu6050/test.esp32-idf.yaml
+++ b/tests/components/mpu6050/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mpu6050/test.esp8266-ard.yaml b/tests/components/mpu6050/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mpu6050/test.esp8266-ard.yaml
+++ b/tests/components/mpu6050/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mpu6050/test.rp2040-ard.yaml b/tests/components/mpu6050/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mpu6050/test.rp2040-ard.yaml
+++ b/tests/components/mpu6050/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mpu6886/common.yaml b/tests/components/mpu6886/common.yaml
index 9c3e283cc3..9f7324f384 100644
--- a/tests/components/mpu6886/common.yaml
+++ b/tests/components/mpu6886/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_mpu6886
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: mpu6886
+    i2c_id: i2c_bus
     address: 0x68
     accel_x:
       name: MPU6886 Accel X
diff --git a/tests/components/mpu6886/test.esp32-ard.yaml b/tests/components/mpu6886/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/mpu6886/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/mpu6886/test.esp32-c3-ard.yaml b/tests/components/mpu6886/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/mpu6886/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/mpu6886/test.esp32-c3-idf.yaml b/tests/components/mpu6886/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/mpu6886/test.esp32-c3-idf.yaml
+++ b/tests/components/mpu6886/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mpu6886/test.esp32-idf.yaml b/tests/components/mpu6886/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/mpu6886/test.esp32-idf.yaml
+++ b/tests/components/mpu6886/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mpu6886/test.esp8266-ard.yaml b/tests/components/mpu6886/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/mpu6886/test.esp8266-ard.yaml
+++ b/tests/components/mpu6886/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mpu6886/test.rp2040-ard.yaml b/tests/components/mpu6886/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/mpu6886/test.rp2040-ard.yaml
+++ b/tests/components/mpu6886/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/mqtt/common-update.yaml b/tests/components/mqtt/common-update.yaml
index 25f57cfef2..2e21e9687a 100644
--- a/tests/components/mqtt/common-update.yaml
+++ b/tests/components/mqtt/common-update.yaml
@@ -6,8 +6,10 @@ http_request:
 
 ota:
   - platform: http_request
+    id: mqtt_http_request_ota
 
 update:
   - platform: http_request
     name: "OTA Update"
+    ota_id: mqtt_http_request_ota
     source: https://example.com/ota.json
diff --git a/tests/components/mqtt/common.yaml b/tests/components/mqtt/common.yaml
index 1ab8872fdb..3f1b83bb01 100644
--- a/tests/components/mqtt/common.yaml
+++ b/tests/components/mqtt/common.yaml
@@ -72,10 +72,9 @@ binary_sensor:
       if (id(template_sens).state > 30) {
         // Garage Door is open.
         return true;
-      } else {
-        // Garage Door is closed.
-        return false;
       }
+      // Garage Door is closed.
+      return false;
     on_state:
       - mqtt.publish:
           topic: some/topic/binary_sensor
@@ -217,9 +216,8 @@ cover:
     lambda: |-
       if (id(some_binary_sensor).state) {
         return COVER_OPEN;
-      } else {
-        return COVER_CLOSED;
       }
+      return COVER_CLOSED;
     open_action:
       - logger.log: open_action
     close_action:
@@ -321,9 +319,8 @@ lock:
     lambda: |-
       if (id(some_binary_sensor).state) {
         return LOCK_STATE_LOCKED;
-      } else {
-        return LOCK_STATE_UNLOCKED;
       }
+      return LOCK_STATE_UNLOCKED;
     lock_action:
       - logger.log: lock_action
     unlock_action:
@@ -360,9 +357,8 @@ sensor:
     lambda: |-
       if (id(some_binary_sensor).state) {
         return 42.0;
-      } else {
-        return 0.0;
       }
+      return 0.0;
     update_interval: 60s
     on_value:
       - mqtt.publish:
@@ -390,9 +386,8 @@ switch:
     lambda: |-
       if (id(some_binary_sensor).state) {
         return true;
-      } else {
-        return false;
       }
+      return false;
     turn_on_action:
       - logger.log: turn_on_action
     turn_off_action:
@@ -436,9 +431,8 @@ valve:
     lambda: |-
       if (id(some_binary_sensor).state) {
         return VALVE_OPEN;
-      } else {
-        return VALVE_CLOSED;
       }
+      return VALVE_CLOSED;
 
 alarm_control_panel:
   - platform: template
diff --git a/tests/components/mqtt/test.esp32-ard.yaml b/tests/components/mqtt/test.esp32-ard.yaml
deleted file mode 100644
index 4c70fb37d9..0000000000
--- a/tests/components/mqtt/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  verify_ssl: "false"
-
-packages:
-  common: !include common.yaml
-  update: !include common-update.yaml
diff --git a/tests/components/mqtt/test.esp32-c3-ard.yaml b/tests/components/mqtt/test.esp32-c3-ard.yaml
deleted file mode 100644
index 4c70fb37d9..0000000000
--- a/tests/components/mqtt/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  verify_ssl: "false"
-
-packages:
-  common: !include common.yaml
-  update: !include common-update.yaml
diff --git a/tests/components/mqtt_subscribe/test.esp32-ard.yaml b/tests/components/mqtt_subscribe/test.esp32-ard.yaml
deleted file mode 100644
index 28589ee06c..0000000000
--- a/tests/components/mqtt_subscribe/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-ard.yaml
diff --git a/tests/components/mqtt_subscribe/test.esp32-c3-ard.yaml b/tests/components/mqtt_subscribe/test.esp32-c3-ard.yaml
deleted file mode 100644
index 28589ee06c..0000000000
--- a/tests/components/mqtt_subscribe/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-ard.yaml
diff --git a/tests/components/ms5611/common.yaml b/tests/components/ms5611/common.yaml
index d0417faa86..12644fa330 100644
--- a/tests/components/ms5611/common.yaml
+++ b/tests/components/ms5611/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_ms5611
-    scl: ${i2c_scl}
-    sda: ${i2c_sda}
-
 sensor:
   - platform: ms5611
+    i2c_id: i2c_bus
     temperature:
       name: Outside Temperature
     pressure:
diff --git a/tests/components/ms5611/test.esp32-ard.yaml b/tests/components/ms5611/test.esp32-ard.yaml
deleted file mode 100644
index 1037d5d35b..0000000000
--- a/tests/components/ms5611/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  i2c_scl: GPIO16
-  i2c_sda: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ms5611/test.esp32-c3-ard.yaml b/tests/components/ms5611/test.esp32-c3-ard.yaml
deleted file mode 100644
index d7ae0d5161..0000000000
--- a/tests/components/ms5611/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  i2c_scl: GPIO5
-  i2c_sda: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ms5611/test.esp32-c3-idf.yaml b/tests/components/ms5611/test.esp32-c3-idf.yaml
index d7ae0d5161..d1abb03369 100644
--- a/tests/components/ms5611/test.esp32-c3-idf.yaml
+++ b/tests/components/ms5611/test.esp32-c3-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   i2c_scl: GPIO5
   i2c_sda: GPIO4
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ms5611/test.esp32-idf.yaml b/tests/components/ms5611/test.esp32-idf.yaml
index 1037d5d35b..4598505c3a 100644
--- a/tests/components/ms5611/test.esp32-idf.yaml
+++ b/tests/components/ms5611/test.esp32-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   i2c_scl: GPIO16
   i2c_sda: GPIO17
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ms5611/test.esp8266-ard.yaml b/tests/components/ms5611/test.esp8266-ard.yaml
index d7ae0d5161..5565bb8c35 100644
--- a/tests/components/ms5611/test.esp8266-ard.yaml
+++ b/tests/components/ms5611/test.esp8266-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   i2c_scl: GPIO5
   i2c_sda: GPIO4
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ms5611/test.rp2040-ard.yaml b/tests/components/ms5611/test.rp2040-ard.yaml
index d7ae0d5161..888762a742 100644
--- a/tests/components/ms5611/test.rp2040-ard.yaml
+++ b/tests/components/ms5611/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   i2c_scl: GPIO5
   i2c_sda: GPIO4
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/msa3xx/common.yaml b/tests/components/msa3xx/common.yaml
index 8de6a8a89a..297ed8a741 100644
--- a/tests/components/msa3xx/common.yaml
+++ b/tests/components/msa3xx/common.yaml
@@ -1,5 +1,5 @@
 msa3xx:
-  i2c_id: i2c_msa3xx
+  i2c_id: i2c_bus
   type: msa301
   range: 4G
   resolution: 14
diff --git a/tests/components/msa3xx/test.esp32-ard.yaml b/tests/components/msa3xx/test.esp32-ard.yaml
deleted file mode 100644
index 7202e7b9bf..0000000000
--- a/tests/components/msa3xx/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-i2c:
-  - id: i2c_msa3xx
-    scl: GPIO16
-    sda: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/msa3xx/test.esp32-c3-ard.yaml b/tests/components/msa3xx/test.esp32-c3-ard.yaml
deleted file mode 100644
index b972ce8cdb..0000000000
--- a/tests/components/msa3xx/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-i2c:
-  - id: i2c_msa3xx
-    scl: GPIO5
-    sda: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/msa3xx/test.esp32-c3-idf.yaml b/tests/components/msa3xx/test.esp32-c3-idf.yaml
index b972ce8cdb..9990d96d29 100644
--- a/tests/components/msa3xx/test.esp32-c3-idf.yaml
+++ b/tests/components/msa3xx/test.esp32-c3-idf.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_msa3xx
-    scl: GPIO5
-    sda: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/msa3xx/test.esp32-idf.yaml b/tests/components/msa3xx/test.esp32-idf.yaml
index 7202e7b9bf..b47e39c389 100644
--- a/tests/components/msa3xx/test.esp32-idf.yaml
+++ b/tests/components/msa3xx/test.esp32-idf.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_msa3xx
-    scl: GPIO16
-    sda: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/msa3xx/test.esp8266-ard.yaml b/tests/components/msa3xx/test.esp8266-ard.yaml
index b972ce8cdb..4a98b9388a 100644
--- a/tests/components/msa3xx/test.esp8266-ard.yaml
+++ b/tests/components/msa3xx/test.esp8266-ard.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_msa3xx
-    scl: GPIO5
-    sda: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/msa3xx/test.rp2040-ard.yaml b/tests/components/msa3xx/test.rp2040-ard.yaml
index b972ce8cdb..319a7c71a6 100644
--- a/tests/components/msa3xx/test.rp2040-ard.yaml
+++ b/tests/components/msa3xx/test.rp2040-ard.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_msa3xx
-    scl: GPIO5
-    sda: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/my9231/test.esp32-ard.yaml b/tests/components/my9231/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/my9231/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/my9231/test.esp32-c3-ard.yaml b/tests/components/my9231/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/my9231/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/nau7802/common.yaml b/tests/components/nau7802/common.yaml
index 2501911a3f..5c52c33dad 100644
--- a/tests/components/nau7802/common.yaml
+++ b/tests/components/nau7802/common.yaml
@@ -1,8 +1,8 @@
 sensor:
   - platform: nau7802
+    i2c_id: i2c_bus
     id: test_id
     name: weight
-    i2c_id: i2c_nau7802
     gain: 32
     ldo_voltage: "3.0v"
     samples_per_second: 10
diff --git a/tests/components/nau7802/test.esp32-ard.yaml b/tests/components/nau7802/test.esp32-ard.yaml
deleted file mode 100644
index 73a4aa4251..0000000000
--- a/tests/components/nau7802/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-i2c:
-  - id: i2c_nau7802
-    scl: 16
-    sda: 17
-
-<<: !include common.yaml
diff --git a/tests/components/nau7802/test.esp32-c3-ard.yaml b/tests/components/nau7802/test.esp32-c3-ard.yaml
deleted file mode 100644
index 769468f9ec..0000000000
--- a/tests/components/nau7802/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-i2c:
-  - id: i2c_nau7802
-    scl: 5
-    sda: 4
-
-<<: !include common.yaml
diff --git a/tests/components/nau7802/test.esp32-c3-idf.yaml b/tests/components/nau7802/test.esp32-c3-idf.yaml
index 769468f9ec..9990d96d29 100644
--- a/tests/components/nau7802/test.esp32-c3-idf.yaml
+++ b/tests/components/nau7802/test.esp32-c3-idf.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_nau7802
-    scl: 5
-    sda: 4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/nau7802/test.esp32-idf.yaml b/tests/components/nau7802/test.esp32-idf.yaml
index 73a4aa4251..b47e39c389 100644
--- a/tests/components/nau7802/test.esp32-idf.yaml
+++ b/tests/components/nau7802/test.esp32-idf.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_nau7802
-    scl: 16
-    sda: 17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/nau7802/test.esp8266-ard.yaml b/tests/components/nau7802/test.esp8266-ard.yaml
index 769468f9ec..4a98b9388a 100644
--- a/tests/components/nau7802/test.esp8266-ard.yaml
+++ b/tests/components/nau7802/test.esp8266-ard.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_nau7802
-    scl: 5
-    sda: 4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/nau7802/test.rp2040-ard.yaml b/tests/components/nau7802/test.rp2040-ard.yaml
index 769468f9ec..319a7c71a6 100644
--- a/tests/components/nau7802/test.rp2040-ard.yaml
+++ b/tests/components/nau7802/test.rp2040-ard.yaml
@@ -1,6 +1,4 @@
-i2c:
-  - id: i2c_nau7802
-    scl: 5
-    sda: 4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/neopixelbus/test.esp32-c3-ard.yaml b/tests/components/neopixelbus/test.esp32-c3-ard.yaml
deleted file mode 100644
index f2b53ab1e9..0000000000
--- a/tests/components/neopixelbus/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,19 +0,0 @@
-light:
-  - platform: neopixelbus
-    id: addr3
-    name: Neopixelbus Light
-    gamma_correct: 2.8
-    color_correct: [0.0, 0.0, 0.0, 0.0]
-    default_transition_length: 10s
-    effects:
-      - addressable_flicker:
-          name: Flicker Effect With Custom Values
-          update_interval: 16ms
-          intensity: 5%
-    type: GRBW
-    variant: SK6812
-    method:
-      type: esp32_rmt
-      channel: 0
-    num_leds: 5
-    pin: 4
diff --git a/tests/components/network/test-ipv6.bk72xx-ard.yaml b/tests/components/network/test-ipv6.bk72xx-ard.yaml
index d0c4bbfcb9..da1324b17e 100644
--- a/tests/components/network/test-ipv6.bk72xx-ard.yaml
+++ b/tests/components/network/test-ipv6.bk72xx-ard.yaml
@@ -1,8 +1,4 @@
 substitutions:
   network_enable_ipv6: "true"
 
-bk72xx:
-  framework:
-    version: 1.7.0
-
 <<: !include common.yaml
diff --git a/tests/components/network/test-ipv6.esp32-ard.yaml b/tests/components/network/test-ipv6.esp32-ard.yaml
deleted file mode 100644
index da1324b17e..0000000000
--- a/tests/components/network/test-ipv6.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  network_enable_ipv6: "true"
-
-<<: !include common.yaml
diff --git a/tests/components/network/test-ipv6.esp32-c3-ard.yaml b/tests/components/network/test-ipv6.esp32-c3-ard.yaml
deleted file mode 100644
index da1324b17e..0000000000
--- a/tests/components/network/test-ipv6.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  network_enable_ipv6: "true"
-
-<<: !include common.yaml
diff --git a/tests/components/ble_client/test.esp32-c3-ard.yaml b/tests/components/network/test.bk72xx-ard.yaml
similarity index 100%
rename from tests/components/ble_client/test.esp32-c3-ard.yaml
rename to tests/components/network/test.bk72xx-ard.yaml
diff --git a/tests/components/network/test.esp32-ard.yaml b/tests/components/network/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/network/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/network/test.esp32-c3-ard.yaml b/tests/components/network/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/network/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/nextion/common.yaml b/tests/components/nextion/common.yaml
index 767c868d0b..f5ee12a51c 100644
--- a/tests/components/nextion/common.yaml
+++ b/tests/components/nextion/common.yaml
@@ -236,12 +236,6 @@ wifi:
   ssid: MySSID
   password: password1
 
-uart:
-  - id: uart_nextion
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 115200
-
 binary_sensor:
   - platform: nextion
     page_id: 0
diff --git a/tests/components/nextion/test.esp32-ard.yaml b/tests/components/nextion/test.esp32-ard.yaml
index d5e02b8b85..7e94a9b4a5 100644
--- a/tests/components/nextion/test.esp32-ard.yaml
+++ b/tests/components/nextion/test.esp32-ard.yaml
@@ -1,8 +1,5 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
 packages:
+  uart: !include ../../test_build_components/common/uart/esp32-ard.yaml
   base: !include common.yaml
 
 display:
diff --git a/tests/components/nextion/test.esp32-c3-ard.yaml b/tests/components/nextion/test.esp32-c3-ard.yaml
deleted file mode 100644
index 5135c7e4f4..0000000000
--- a/tests/components/nextion/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,10 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-packages:
-  base: !include common.yaml
-
-display:
-  - id: !extend main_lcd
-    tft_url: http://esphome.io/default35.tft
diff --git a/tests/components/nextion/test.esp32-c3-idf.yaml b/tests/components/nextion/test.esp32-c3-idf.yaml
index 5135c7e4f4..888693f909 100644
--- a/tests/components/nextion/test.esp32-c3-idf.yaml
+++ b/tests/components/nextion/test.esp32-c3-idf.yaml
@@ -1,8 +1,5 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
 packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
   base: !include common.yaml
 
 display:
diff --git a/tests/components/nextion/test.esp32-idf.yaml b/tests/components/nextion/test.esp32-idf.yaml
index d5e02b8b85..99820f0f8d 100644
--- a/tests/components/nextion/test.esp32-idf.yaml
+++ b/tests/components/nextion/test.esp32-idf.yaml
@@ -1,8 +1,5 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
 packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
   base: !include common.yaml
 
 display:
diff --git a/tests/components/nextion/test.esp8266-ard.yaml b/tests/components/nextion/test.esp8266-ard.yaml
index 5135c7e4f4..49f79b2f4c 100644
--- a/tests/components/nextion/test.esp8266-ard.yaml
+++ b/tests/components/nextion/test.esp8266-ard.yaml
@@ -1,8 +1,5 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
 packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
   base: !include common.yaml
 
 display:
diff --git a/tests/components/nextion/test.rp2040-ard.yaml b/tests/components/nextion/test.rp2040-ard.yaml
index 44534b97a8..23cc71ee7c 100644
--- a/tests/components/nextion/test.rp2040-ard.yaml
+++ b/tests/components/nextion/test.rp2040-ard.yaml
@@ -1,6 +1,3 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
 packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
   base: !include common.yaml
diff --git a/tests/components/noblex/common.yaml b/tests/components/noblex/common.yaml
index f5e471a9a7..8053d84d4b 100644
--- a/tests/components/noblex/common.yaml
+++ b/tests/components/noblex/common.yaml
@@ -1,12 +1,3 @@
-remote_receiver:
-  id: rcvr
-  pin: 4
-  dump: all
-
-remote_transmitter:
-  pin: 2
-  carrier_duty_percent: 50%
-
 sensor:
   - platform: template
     id: noblex_ac_sensor
diff --git a/tests/components/noblex/test.esp32-ard.yaml b/tests/components/noblex/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/noblex/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/noblex/test.esp32-c3-ard.yaml b/tests/components/noblex/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/noblex/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/noblex/test.esp32-c3-idf.yaml b/tests/components/noblex/test.esp32-c3-idf.yaml
index dade44d145..fe77c44eed 100644
--- a/tests/components/noblex/test.esp32-c3-idf.yaml
+++ b/tests/components/noblex/test.esp32-c3-idf.yaml
@@ -1 +1,5 @@
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
+  remote_receiver: !include ../../test_build_components/common/remote_receiver/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/noblex/test.esp32-idf.yaml b/tests/components/noblex/test.esp32-idf.yaml
index dade44d145..b241dbd159 100644
--- a/tests/components/noblex/test.esp32-idf.yaml
+++ b/tests/components/noblex/test.esp32-idf.yaml
@@ -1 +1,5 @@
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
+  remote_receiver: !include ../../test_build_components/common/remote_receiver/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/noblex/test.esp8266-ard.yaml b/tests/components/noblex/test.esp8266-ard.yaml
index dade44d145..aa8651e556 100644
--- a/tests/components/noblex/test.esp8266-ard.yaml
+++ b/tests/components/noblex/test.esp8266-ard.yaml
@@ -1 +1,5 @@
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
+  remote_receiver: !include ../../test_build_components/common/remote_receiver/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/npi19/common.yaml b/tests/components/npi19/common.yaml
index 012e05695e..03550c269f 100644
--- a/tests/components/npi19/common.yaml
+++ b/tests/components/npi19/common.yaml
@@ -1,13 +1,7 @@
-i2c:
-  id: i2c_bus
-  scl: ${scl_pin}
-  sda: ${sda_pin}
-  frequency: 200kHz
-
 sensor:
   - platform: npi19
-    update_interval: 1s
     i2c_id: i2c_bus
+    update_interval: 1s
 
     temperature:
       name: water temperature
diff --git a/tests/components/npi19/test.esp32-ard.yaml b/tests/components/npi19/test.esp32-ard.yaml
deleted file mode 100644
index 3b761d3fc1..0000000000
--- a/tests/components/npi19/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
-
-<<: !include common.yaml
diff --git a/tests/components/npi19/test.esp32-idf.yaml b/tests/components/npi19/test.esp32-idf.yaml
index 3b761d3fc1..b47e39c389 100644
--- a/tests/components/npi19/test.esp32-idf.yaml
+++ b/tests/components/npi19/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/npi19/test.esp32-s3-ard.yaml b/tests/components/npi19/test.esp32-s3-ard.yaml
deleted file mode 100644
index 4942e3c2b3..0000000000
--- a/tests/components/npi19/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO40
-  sda_pin: GPIO41
-
-<<: !include common.yaml
diff --git a/tests/components/npi19/test.esp32-s3-idf.yaml b/tests/components/npi19/test.esp32-s3-idf.yaml
index 4942e3c2b3..0fd8684a2c 100644
--- a/tests/components/npi19/test.esp32-s3-idf.yaml
+++ b/tests/components/npi19/test.esp32-s3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO40
-  sda_pin: GPIO41
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-s3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/npi19/test.esp8266-ard.yaml b/tests/components/npi19/test.esp8266-ard.yaml
index 3be5e53dcb..4a98b9388a 100644
--- a/tests/components/npi19/test.esp8266-ard.yaml
+++ b/tests/components/npi19/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO05
-  sda_pin: GPIO04
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/nrf52/test.nrf52-adafruit.yaml b/tests/components/nrf52/test.nrf52-adafruit.yaml
new file mode 100644
index 0000000000..3fe80209b6
--- /dev/null
+++ b/tests/components/nrf52/test.nrf52-adafruit.yaml
@@ -0,0 +1,7 @@
+nrf52:
+  dfu:
+    reset_pin:
+      number: 14
+      inverted: true
+      mode:
+        output: true
diff --git a/tests/components/nrf52/test.nrf52-xiao-ble.yaml b/tests/components/nrf52/test.nrf52-xiao-ble.yaml
new file mode 100644
index 0000000000..3fe80209b6
--- /dev/null
+++ b/tests/components/nrf52/test.nrf52-xiao-ble.yaml
@@ -0,0 +1,7 @@
+nrf52:
+  dfu:
+    reset_pin:
+      number: 14
+      inverted: true
+      mode:
+        output: true
diff --git a/tests/components/ntc/test.esp32-ard.yaml b/tests/components/ntc/test.esp32-ard.yaml
deleted file mode 100644
index 06864605a6..0000000000
--- a/tests/components/ntc/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO32
-
-<<: !include common.yaml
diff --git a/tests/components/ntc/test.esp32-c3-ard.yaml b/tests/components/ntc/test.esp32-c3-ard.yaml
deleted file mode 100644
index 37fb325f4a..0000000000
--- a/tests/components/ntc/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ntc/test.esp32-s2-ard.yaml b/tests/components/ntc/test.esp32-s2-ard.yaml
deleted file mode 100644
index 37fb325f4a..0000000000
--- a/tests/components/ntc/test.esp32-s2-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ntc/test.esp32-s3-ard.yaml b/tests/components/ntc/test.esp32-s3-ard.yaml
deleted file mode 100644
index 37fb325f4a..0000000000
--- a/tests/components/ntc/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/online_image/common-esp32.yaml b/tests/components/online_image/common-esp32.yaml
index 787a1ad368..32c909d351 100644
--- a/tests/components/online_image/common-esp32.yaml
+++ b/tests/components/online_image/common-esp32.yaml
@@ -1,13 +1,11 @@
-<<: !include common.yaml
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
-spi:
-  - id: spi_main_lcd
-    clk_pin: 16
-    mosi_pin: 17
-    miso_pin: 18
+<<: !include common.yaml
 
 display:
   - platform: ili9xxx
+    spi_id: spi_bus
     id: main_lcd
     model: ili9342
     cs_pin: 20
diff --git a/tests/components/online_image/common-esp8266.yaml b/tests/components/online_image/common-esp8266.yaml
index ba15b5025c..d7722d171a 100644
--- a/tests/components/online_image/common-esp8266.yaml
+++ b/tests/components/online_image/common-esp8266.yaml
@@ -1,13 +1,11 @@
-<<: !include common.yaml
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
 
-spi:
-  - id: spi_main_lcd
-    clk_pin: 14
-    mosi_pin: 13
-    miso_pin: 12
+<<: !include common.yaml
 
 display:
   - platform: ili9xxx
+    spi_id: spi_bus
     id: main_lcd
     model: ili9342
     cs_pin: 15
diff --git a/tests/components/online_image/common-rp2040.yaml b/tests/components/online_image/common-rp2040.yaml
index 16bb2b2c44..25891b94bc 100644
--- a/tests/components/online_image/common-rp2040.yaml
+++ b/tests/components/online_image/common-rp2040.yaml
@@ -1,13 +1,11 @@
-<<: !include common.yaml
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
 
-spi:
-  - id: spi_main_lcd
-    clk_pin: 18
-    mosi_pin: 19
-    miso_pin: 16
+<<: !include common.yaml
 
 display:
   - platform: ili9xxx
+    spi_id: spi_bus
     id: main_lcd
     model: ili9342
     cs_pin: 20
diff --git a/tests/components/online_image/test.esp32-ard.yaml b/tests/components/online_image/test.esp32-ard.yaml
deleted file mode 100644
index 4111cbd0ad..0000000000
--- a/tests/components/online_image/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-<<: !include common-esp32.yaml
-
-http_request:
-  verify_ssl: false
diff --git a/tests/components/opentherm/test.esp32-ard.yaml b/tests/components/opentherm/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/opentherm/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/opentherm/test.esp32-c3-ard.yaml b/tests/components/opentherm/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/opentherm/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/opt3001/common.yaml b/tests/components/opt3001/common.yaml
index dab4f824f8..7b2cb339af 100644
--- a/tests/components/opt3001/common.yaml
+++ b/tests/components/opt3001/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_opt3001
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: opt3001
+    i2c_id: i2c_bus
     name: Living Room Brightness
     address: 0x44
     update_interval: 30s
diff --git a/tests/components/opt3001/test.esp32-ard.yaml b/tests/components/opt3001/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/opt3001/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/opt3001/test.esp32-c3-ard.yaml b/tests/components/opt3001/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/opt3001/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/opt3001/test.esp32-c3-idf.yaml b/tests/components/opt3001/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/opt3001/test.esp32-c3-idf.yaml
+++ b/tests/components/opt3001/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/opt3001/test.esp32-idf.yaml b/tests/components/opt3001/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/opt3001/test.esp32-idf.yaml
+++ b/tests/components/opt3001/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/opt3001/test.esp8266-ard.yaml b/tests/components/opt3001/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/opt3001/test.esp8266-ard.yaml
+++ b/tests/components/opt3001/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/opt3001/test.rp2040-ard.yaml b/tests/components/opt3001/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/opt3001/test.rp2040-ard.yaml
+++ b/tests/components/opt3001/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ota/test.esp32-c3-ard.yaml b/tests/components/ota/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/ota/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/output/common.yaml b/tests/components/output/common.yaml
index 5f31ae50a8..81d802e9bf 100644
--- a/tests/components/output/common.yaml
+++ b/tests/components/output/common.yaml
@@ -6,6 +6,12 @@ esphome:
       - output.set_level:
           id: light_output_1
           level: 50%
+      - output.set_min_power:
+          id: light_output_1
+          min_power: 20%
+      - output.set_max_power:
+          id: light_output_1
+          max_power: 80%
 
 output:
   - platform: ${output_platform}
diff --git a/tests/components/output/test.esp32-ard.yaml b/tests/components/output/test.esp32-ard.yaml
deleted file mode 100644
index 7687f2a7c8..0000000000
--- a/tests/components/output/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  output_platform: ledc
-  pin: GPIO12
-
-<<: !include common.yaml
diff --git a/tests/components/output/test.esp32-c3-ard.yaml b/tests/components/output/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2227643703..0000000000
--- a/tests/components/output/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  output_platform: ledc
-  pin: GPIO1
-
-<<: !include common.yaml
diff --git a/tests/components/packages/garage-door.yaml b/tests/components/packages/garage-door.yaml
new file mode 100644
index 0000000000..e16265d1e1
--- /dev/null
+++ b/tests/components/packages/garage-door.yaml
@@ -0,0 +1,5 @@
+switch:
+  - name: ${door_name} Garage Door Switch
+    platform: gpio
+    pin: ${door_pin}
+    id: ${door_id}
diff --git a/tests/components/packages/test-vars.esp32-idf.yaml b/tests/components/packages/test-vars.esp32-idf.yaml
new file mode 100644
index 0000000000..f12467d9f9
--- /dev/null
+++ b/tests/components/packages/test-vars.esp32-idf.yaml
@@ -0,0 +1,19 @@
+packages:
+  left_garage_door: !include
+    file: garage-door.yaml
+    vars:
+      door_name: Left
+      door_pin: 1
+      door_id: left_garage_door
+  middle_garage_door: !include
+    file: garage-door.yaml
+    vars:
+      door_name: Middle
+      door_pin: 2
+      door_id: middle_garage_door
+  right_garage_door: !include
+    file: garage-door.yaml
+    vars:
+      door_name: Right
+      door_pin: 3
+      door_id: right_garage_door
diff --git a/tests/components/packages/test.esp32-ard.yaml b/tests/components/packages/test.esp32-ard.yaml
deleted file mode 100644
index 9e4ceb09d6..0000000000
--- a/tests/components/packages/test.esp32-ard.yaml
+++ /dev/null
@@ -1,11 +0,0 @@
-packages:
-  - sensor:
-      - platform: template
-        id: inline_sensor
-  - !include package.yaml
-  - github://esphome/esphome/tests/components/template/common.yaml@dev
-  - url: https://github.com/esphome/esphome
-    path: tests/components/absolute_humidity
-    file: common.yaml
-    ref: dev
-    refresh: 1d
diff --git a/tests/components/packet_transport/test.esp32-ard.yaml b/tests/components/packet_transport/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/packet_transport/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/packet_transport/test.esp32-c3-ard.yaml b/tests/components/packet_transport/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/packet_transport/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/packet_transport/test.esp32-c3-idf.yaml b/tests/components/packet_transport/test.esp32-c3-idf.yaml
index dade44d145..9990d96d29 100644
--- a/tests/components/packet_transport/test.esp32-c3-idf.yaml
+++ b/tests/components/packet_transport/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/packet_transport/test.esp32-idf.yaml b/tests/components/packet_transport/test.esp32-idf.yaml
index dade44d145..b47e39c389 100644
--- a/tests/components/packet_transport/test.esp32-idf.yaml
+++ b/tests/components/packet_transport/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/packet_transport/test.esp8266-ard.yaml b/tests/components/packet_transport/test.esp8266-ard.yaml
index dade44d145..4a98b9388a 100644
--- a/tests/components/packet_transport/test.esp8266-ard.yaml
+++ b/tests/components/packet_transport/test.esp8266-ard.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/packet_transport/test.host.yaml b/tests/components/packet_transport/test.host.yaml
index e735c37e4d..49fdbbc9b2 100644
--- a/tests/components/packet_transport/test.host.yaml
+++ b/tests/components/packet_transport/test.host.yaml
@@ -1,4 +1,40 @@
-packages:
-  common: !include common.yaml
+udp:
+  listen_address: 239.0.60.53
+  addresses: ["239.0.60.53"]
 
-wifi: !remove
+packet_transport:
+  platform: udp
+  update_interval: 5s
+  encryption: "our key goes here"
+  rolling_code_enable: true
+  ping_pong_enable: true
+  binary_sensors:
+    - binary_sensor_id1
+    - id: binary_sensor_id1
+      broadcast_id: other_id
+  sensors:
+    - sensor_id1
+    - id: sensor_id1
+      broadcast_id: other_id
+  providers:
+    - name: some-device-name
+      encryption: "their key goes here"
+
+sensor:
+  - platform: template
+    id: sensor_id1
+  - platform: packet_transport
+    provider: some-device-name
+    id: our_id
+    remote_id: some_sensor_id
+
+binary_sensor:
+  - platform: packet_transport
+    provider: unencrypted-device
+    id: other_binary_sensor_id
+  - platform: packet_transport
+    provider: some-device-name
+    type: status
+    name: Some-Device Status
+  - platform: template
+    id: binary_sensor_id1
diff --git a/tests/components/packet_transport/test.rp2040-ard.yaml b/tests/components/packet_transport/test.rp2040-ard.yaml
index dade44d145..319a7c71a6 100644
--- a/tests/components/packet_transport/test.rp2040-ard.yaml
+++ b/tests/components/packet_transport/test.rp2040-ard.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/partition/test.esp32-ard.yaml b/tests/components/partition/test.esp32-ard.yaml
deleted file mode 100644
index 32cf2a64ba..0000000000
--- a/tests/components/partition/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  light_platform: esp32_rmt_led_strip
-  pin: GPIO2
-
-<<: !include common-ard.yaml
diff --git a/tests/components/partition/test.esp32-c3-ard.yaml b/tests/components/partition/test.esp32-c3-ard.yaml
deleted file mode 100644
index 32cf2a64ba..0000000000
--- a/tests/components/partition/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  light_platform: esp32_rmt_led_strip
-  pin: GPIO2
-
-<<: !include common-ard.yaml
diff --git a/tests/components/pca6416a/common.yaml b/tests/components/pca6416a/common.yaml
index ea538387e4..9ad6e2fb15 100644
--- a/tests/components/pca6416a/common.yaml
+++ b/tests/components/pca6416a/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_pca6416a
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 pca6416a:
   - id: pca6416a_hub
+    i2c_id: i2c_bus
     address: 0x21
 
 binary_sensor:
diff --git a/tests/components/pca6416a/test.esp32-ard.yaml b/tests/components/pca6416a/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/pca6416a/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/pca6416a/test.esp32-c3-ard.yaml b/tests/components/pca6416a/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/pca6416a/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/pca6416a/test.esp32-c3-idf.yaml b/tests/components/pca6416a/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/pca6416a/test.esp32-c3-idf.yaml
+++ b/tests/components/pca6416a/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pca6416a/test.esp32-idf.yaml b/tests/components/pca6416a/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/pca6416a/test.esp32-idf.yaml
+++ b/tests/components/pca6416a/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pca6416a/test.esp8266-ard.yaml b/tests/components/pca6416a/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/pca6416a/test.esp8266-ard.yaml
+++ b/tests/components/pca6416a/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pca6416a/test.rp2040-ard.yaml b/tests/components/pca6416a/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/pca6416a/test.rp2040-ard.yaml
+++ b/tests/components/pca6416a/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pca9554/common.yaml b/tests/components/pca9554/common.yaml
index db2ec2b826..9e5e7f3342 100644
--- a/tests/components/pca9554/common.yaml
+++ b/tests/components/pca9554/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_pca9554
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 pca9554:
   - id: pca9554_hub
+    i2c_id: i2c_bus
     pin_count: 8
     address: 0x3F
 
diff --git a/tests/components/pca9554/test.esp32-ard.yaml b/tests/components/pca9554/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/pca9554/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/pca9554/test.esp32-c3-ard.yaml b/tests/components/pca9554/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/pca9554/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/pca9554/test.esp32-c3-idf.yaml b/tests/components/pca9554/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/pca9554/test.esp32-c3-idf.yaml
+++ b/tests/components/pca9554/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pca9554/test.esp32-idf.yaml b/tests/components/pca9554/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/pca9554/test.esp32-idf.yaml
+++ b/tests/components/pca9554/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pca9554/test.esp8266-ard.yaml b/tests/components/pca9554/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/pca9554/test.esp8266-ard.yaml
+++ b/tests/components/pca9554/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pca9554/test.rp2040-ard.yaml b/tests/components/pca9554/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/pca9554/test.rp2040-ard.yaml
+++ b/tests/components/pca9554/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pca9685/common.yaml b/tests/components/pca9685/common.yaml
index 57cdc5de9b..2e238b481c 100644
--- a/tests/components/pca9685/common.yaml
+++ b/tests/components/pca9685/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_pca9685
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 pca9685:
+  i2c_id: i2c_bus
   frequency: 500
   address: 0x0
 
diff --git a/tests/components/pca9685/test.esp32-ard.yaml b/tests/components/pca9685/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/pca9685/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/pca9685/test.esp32-c3-ard.yaml b/tests/components/pca9685/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/pca9685/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/pca9685/test.esp32-c3-idf.yaml b/tests/components/pca9685/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/pca9685/test.esp32-c3-idf.yaml
+++ b/tests/components/pca9685/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pca9685/test.esp32-idf.yaml b/tests/components/pca9685/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/pca9685/test.esp32-idf.yaml
+++ b/tests/components/pca9685/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pca9685/test.esp8266-ard.yaml b/tests/components/pca9685/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/pca9685/test.esp8266-ard.yaml
+++ b/tests/components/pca9685/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pca9685/test.rp2040-ard.yaml b/tests/components/pca9685/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/pca9685/test.rp2040-ard.yaml
+++ b/tests/components/pca9685/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pcd8544/common.yaml b/tests/components/pcd8544/common.yaml
index 0fb969eeb8..418cd97253 100644
--- a/tests/components/pcd8544/common.yaml
+++ b/tests/components/pcd8544/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_pcd8544
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 display:
   - platform: pcd8544
     cs_pin: ${cs_pin}
diff --git a/tests/components/pcd8544/test.esp32-ard.yaml b/tests/components/pcd8544/test.esp32-ard.yaml
deleted file mode 100644
index 09e9db5a38..0000000000
--- a/tests/components/pcd8544/test.esp32-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO18
-  cs_pin: GPIO12
-  dc_pin: GPIO13
-  reset_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/pcd8544/test.esp32-c3-ard.yaml b/tests/components/pcd8544/test.esp32-c3-ard.yaml
deleted file mode 100644
index c5c932c92c..0000000000
--- a/tests/components/pcd8544/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-  dc_pin: GPIO9
-  reset_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/pcd8544/test.esp32-c3-idf.yaml b/tests/components/pcd8544/test.esp32-c3-idf.yaml
index c5c932c92c..b112cf4c31 100644
--- a/tests/components/pcd8544/test.esp32-c3-idf.yaml
+++ b/tests/components/pcd8544/test.esp32-c3-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
   dc_pin: GPIO9
   reset_pin: GPIO10
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pcd8544/test.esp32-idf.yaml b/tests/components/pcd8544/test.esp32-idf.yaml
index 09e9db5a38..ff174a4656 100644
--- a/tests/components/pcd8544/test.esp32-idf.yaml
+++ b/tests/components/pcd8544/test.esp32-idf.yaml
@@ -1,9 +1,9 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO18
   cs_pin: GPIO12
   dc_pin: GPIO13
   reset_pin: GPIO14
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pcd8544/test.esp8266-ard.yaml b/tests/components/pcd8544/test.esp8266-ard.yaml
index 3f023a60eb..56cb29f29e 100644
--- a/tests/components/pcd8544/test.esp8266-ard.yaml
+++ b/tests/components/pcd8544/test.esp8266-ard.yaml
@@ -1,9 +1,12 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
   miso_pin: GPIO12
   cs_pin: GPIO5
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pcd8544/test.rp2040-ard.yaml b/tests/components/pcd8544/test.rp2040-ard.yaml
index d7fd6ee294..66caa956f7 100644
--- a/tests/components/pcd8544/test.rp2040-ard.yaml
+++ b/tests/components/pcd8544/test.rp2040-ard.yaml
@@ -6,4 +6,7 @@ substitutions:
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pcf85063/common.yaml b/tests/components/pcf85063/common.yaml
index f3b68412c5..170029ad85 100644
--- a/tests/components/pcf85063/common.yaml
+++ b/tests/components/pcf85063/common.yaml
@@ -3,10 +3,6 @@ esphome:
     - pcf85063.read_time
     - pcf85063.write_time
 
-i2c:
-  - id: i2c_pcf85063
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 time:
   - platform: pcf85063
+    i2c_id: i2c_bus
diff --git a/tests/components/pcf85063/test.esp32-ard.yaml b/tests/components/pcf85063/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/pcf85063/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/pcf85063/test.esp32-c3-ard.yaml b/tests/components/pcf85063/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/pcf85063/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/pcf85063/test.esp32-c3-idf.yaml b/tests/components/pcf85063/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/pcf85063/test.esp32-c3-idf.yaml
+++ b/tests/components/pcf85063/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pcf85063/test.esp32-idf.yaml b/tests/components/pcf85063/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/pcf85063/test.esp32-idf.yaml
+++ b/tests/components/pcf85063/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pcf85063/test.esp8266-ard.yaml b/tests/components/pcf85063/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/pcf85063/test.esp8266-ard.yaml
+++ b/tests/components/pcf85063/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pcf85063/test.rp2040-ard.yaml b/tests/components/pcf85063/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/pcf85063/test.rp2040-ard.yaml
+++ b/tests/components/pcf85063/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pcf8563/common.yaml b/tests/components/pcf8563/common.yaml
index 30be6b893c..ac5f3afed1 100644
--- a/tests/components/pcf8563/common.yaml
+++ b/tests/components/pcf8563/common.yaml
@@ -3,10 +3,6 @@ esphome:
     - pcf8563.read_time
     - pcf8563.write_time
 
-i2c:
-  - id: i2c_pcf8563
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 time:
   - platform: pcf8563
+    i2c_id: i2c_bus
diff --git a/tests/components/pcf8563/test.esp32-ard.yaml b/tests/components/pcf8563/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/pcf8563/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/pcf8563/test.esp32-c3-ard.yaml b/tests/components/pcf8563/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/pcf8563/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/pcf8563/test.esp32-c3-idf.yaml b/tests/components/pcf8563/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/pcf8563/test.esp32-c3-idf.yaml
+++ b/tests/components/pcf8563/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pcf8563/test.esp32-idf.yaml b/tests/components/pcf8563/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/pcf8563/test.esp32-idf.yaml
+++ b/tests/components/pcf8563/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pcf8563/test.esp8266-ard.yaml b/tests/components/pcf8563/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/pcf8563/test.esp8266-ard.yaml
+++ b/tests/components/pcf8563/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pcf8563/test.rp2040-ard.yaml b/tests/components/pcf8563/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/pcf8563/test.rp2040-ard.yaml
+++ b/tests/components/pcf8563/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pcf8574/common.yaml b/tests/components/pcf8574/common.yaml
index f5fcfad64a..09fa33164e 100644
--- a/tests/components/pcf8574/common.yaml
+++ b/tests/components/pcf8574/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_pcf8574
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 pcf8574:
   - id: pcf8574_hub
+    i2c_id: i2c_bus
     address: 0x21
     pcf8575: false
 
diff --git a/tests/components/pcf8574/test.esp32-ard.yaml b/tests/components/pcf8574/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/pcf8574/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/pcf8574/test.esp32-c3-ard.yaml b/tests/components/pcf8574/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/pcf8574/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/pcf8574/test.esp32-c3-idf.yaml b/tests/components/pcf8574/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/pcf8574/test.esp32-c3-idf.yaml
+++ b/tests/components/pcf8574/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pcf8574/test.esp32-idf.yaml b/tests/components/pcf8574/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/pcf8574/test.esp32-idf.yaml
+++ b/tests/components/pcf8574/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pcf8574/test.esp8266-ard.yaml b/tests/components/pcf8574/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/pcf8574/test.esp8266-ard.yaml
+++ b/tests/components/pcf8574/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pcf8574/test.rp2040-ard.yaml b/tests/components/pcf8574/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/pcf8574/test.rp2040-ard.yaml
+++ b/tests/components/pcf8574/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pi4ioe5v6408/common.yaml b/tests/components/pi4ioe5v6408/common.yaml
index 4130dc2652..2344622081 100644
--- a/tests/components/pi4ioe5v6408/common.yaml
+++ b/tests/components/pi4ioe5v6408/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  id: i2c_pi4ioe5v6408
-  sda: ${i2c_sda}
-  scl: ${i2c_scl}
-
 pi4ioe5v6408:
+  i2c_id: i2c_bus
   id: pi4ioe1
   address: 0x44
 
diff --git a/tests/components/pi4ioe5v6408/test.esp32-ard.yaml b/tests/components/pi4ioe5v6408/test.esp32-ard.yaml
deleted file mode 100644
index 55e6edfbf3..0000000000
--- a/tests/components/pi4ioe5v6408/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  i2c_sda: GPIO21
-  i2c_scl: GPIO22
-
-<<: !include common.yaml
diff --git a/tests/components/pi4ioe5v6408/test.esp32-idf.yaml b/tests/components/pi4ioe5v6408/test.esp32-idf.yaml
index 55e6edfbf3..9a4779d822 100644
--- a/tests/components/pi4ioe5v6408/test.esp32-idf.yaml
+++ b/tests/components/pi4ioe5v6408/test.esp32-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   i2c_sda: GPIO21
   i2c_scl: GPIO22
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pi4ioe5v6408/test.rp2040-ard.yaml b/tests/components/pi4ioe5v6408/test.rp2040-ard.yaml
index b7b6b13bfe..3429a2952a 100644
--- a/tests/components/pi4ioe5v6408/test.rp2040-ard.yaml
+++ b/tests/components/pi4ioe5v6408/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   i2c_sda: GPIO4
   i2c_scl: GPIO5
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pid/common.yaml b/tests/components/pid/common.yaml
index 5f7762872f..262e75591e 100644
--- a/tests/components/pid/common.yaml
+++ b/tests/components/pid/common.yaml
@@ -27,9 +27,8 @@ sensor:
     lambda: |-
       if (millis() > 10000) {
         return 42.0;
-      } else {
-        return 0.0;
       }
+      return 0.0;
     update_interval: 60s
 
 climate:
diff --git a/tests/components/pid/test.esp32-ard.yaml b/tests/components/pid/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/pid/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/pid/test.esp32-c3-ard.yaml b/tests/components/pid/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/pid/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/pipsolar/common.yaml b/tests/components/pipsolar/common.yaml
index 64e2c0476d..819764a7ea 100644
--- a/tests/components/pipsolar/common.yaml
+++ b/tests/components/pipsolar/common.yaml
@@ -5,12 +5,6 @@ esphome:
           id: inverter0_battery_recharge_voltage_out
           value: 48.0
 
-uart:
-  - id: uart_pipsolar
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 115200
-
 pipsolar:
   id: inverter0
 
diff --git a/tests/components/pipsolar/test.esp32-ard.yaml b/tests/components/pipsolar/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/pipsolar/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/pipsolar/test.esp32-c3-ard.yaml b/tests/components/pipsolar/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/pipsolar/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/pipsolar/test.esp32-c3-idf.yaml b/tests/components/pipsolar/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/pipsolar/test.esp32-c3-idf.yaml
+++ b/tests/components/pipsolar/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pipsolar/test.esp32-idf.yaml b/tests/components/pipsolar/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/pipsolar/test.esp32-idf.yaml
+++ b/tests/components/pipsolar/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pipsolar/test.esp32-s2-idf.yaml b/tests/components/pipsolar/test.esp32-s2-idf.yaml
index b516342f3b..2d29656c94 100644
--- a/tests/components/pipsolar/test.esp32-s2-idf.yaml
+++ b/tests/components/pipsolar/test.esp32-s2-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pipsolar/test.esp8266-ard.yaml b/tests/components/pipsolar/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/pipsolar/test.esp8266-ard.yaml
+++ b/tests/components/pipsolar/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pipsolar/test.rp2040-ard.yaml b/tests/components/pipsolar/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/pipsolar/test.rp2040-ard.yaml
+++ b/tests/components/pipsolar/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pm1006/common.yaml b/tests/components/pm1006/common.yaml
index 4e3e880f4e..43955bb099 100644
--- a/tests/components/pm1006/common.yaml
+++ b/tests/components/pm1006/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_pm1006
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: pm1006
     pm_2_5:
diff --git a/tests/components/pm1006/test.esp32-ard.yaml b/tests/components/pm1006/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/pm1006/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/pm1006/test.esp32-c3-ard.yaml b/tests/components/pm1006/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/pm1006/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/pm1006/test.esp32-c3-idf.yaml b/tests/components/pm1006/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/pm1006/test.esp32-c3-idf.yaml
+++ b/tests/components/pm1006/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pm1006/test.esp32-idf.yaml b/tests/components/pm1006/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/pm1006/test.esp32-idf.yaml
+++ b/tests/components/pm1006/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pm1006/test.esp8266-ard.yaml b/tests/components/pm1006/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/pm1006/test.esp8266-ard.yaml
+++ b/tests/components/pm1006/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pm1006/test.rp2040-ard.yaml b/tests/components/pm1006/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/pm1006/test.rp2040-ard.yaml
+++ b/tests/components/pm1006/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pm2005/common.yaml b/tests/components/pm2005/common.yaml
index b8f6683b22..034752d0b9 100644
--- a/tests/components/pm2005/common.yaml
+++ b/tests/components/pm2005/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_pm2005
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: pm2005
+    i2c_id: i2c_bus
     pm_1_0:
       name: PM1.0
     pm_2_5:
diff --git a/tests/components/pm2005/test.esp32-ard.yaml b/tests/components/pm2005/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/pm2005/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/pm2005/test.esp32-c3-ard.yaml b/tests/components/pm2005/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/pm2005/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/pm2005/test.esp32-c3-idf.yaml b/tests/components/pm2005/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/pm2005/test.esp32-c3-idf.yaml
+++ b/tests/components/pm2005/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pm2005/test.esp32-idf.yaml b/tests/components/pm2005/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/pm2005/test.esp32-idf.yaml
+++ b/tests/components/pm2005/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pm2005/test.esp8266-ard.yaml b/tests/components/pm2005/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/pm2005/test.esp8266-ard.yaml
+++ b/tests/components/pm2005/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pm2005/test.rp2040-ard.yaml b/tests/components/pm2005/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/pm2005/test.rp2040-ard.yaml
+++ b/tests/components/pm2005/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pmsa003i/common.yaml b/tests/components/pmsa003i/common.yaml
index 95e62da694..7267bd58f3 100644
--- a/tests/components/pmsa003i/common.yaml
+++ b/tests/components/pmsa003i/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_pmsa003i
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: pmsa003i
+    i2c_id: i2c_bus
     pm_1_0:
       name: PMSA003i PM1.0
     pm_2_5:
diff --git a/tests/components/pmsa003i/test.esp32-ard.yaml b/tests/components/pmsa003i/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/pmsa003i/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/pmsa003i/test.esp32-c3-ard.yaml b/tests/components/pmsa003i/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/pmsa003i/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/pmsa003i/test.esp32-c3-idf.yaml b/tests/components/pmsa003i/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/pmsa003i/test.esp32-c3-idf.yaml
+++ b/tests/components/pmsa003i/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pmsa003i/test.esp32-idf.yaml b/tests/components/pmsa003i/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/pmsa003i/test.esp32-idf.yaml
+++ b/tests/components/pmsa003i/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pmsa003i/test.esp8266-ard.yaml b/tests/components/pmsa003i/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/pmsa003i/test.esp8266-ard.yaml
+++ b/tests/components/pmsa003i/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pmsa003i/test.rp2040-ard.yaml b/tests/components/pmsa003i/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/pmsa003i/test.rp2040-ard.yaml
+++ b/tests/components/pmsa003i/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pmsx003/common.yaml b/tests/components/pmsx003/common.yaml
index 7b9ca5b091..3c60995804 100644
--- a/tests/components/pmsx003/common.yaml
+++ b/tests/components/pmsx003/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_pmsx003
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: pmsx003
     type: PMSX003
diff --git a/tests/components/pmsx003/test.esp32-ard.yaml b/tests/components/pmsx003/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/pmsx003/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/pmsx003/test.esp32-c3-ard.yaml b/tests/components/pmsx003/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/pmsx003/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/pmsx003/test.esp32-c3-idf.yaml b/tests/components/pmsx003/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/pmsx003/test.esp32-c3-idf.yaml
+++ b/tests/components/pmsx003/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pmsx003/test.esp32-idf.yaml b/tests/components/pmsx003/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/pmsx003/test.esp32-idf.yaml
+++ b/tests/components/pmsx003/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pmsx003/test.esp8266-ard.yaml b/tests/components/pmsx003/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/pmsx003/test.esp8266-ard.yaml
+++ b/tests/components/pmsx003/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pmsx003/test.rp2040-ard.yaml b/tests/components/pmsx003/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/pmsx003/test.rp2040-ard.yaml
+++ b/tests/components/pmsx003/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pmwcs3/common.yaml b/tests/components/pmwcs3/common.yaml
index dcf59d0b6e..e06400d4d4 100644
--- a/tests/components/pmwcs3/common.yaml
+++ b/tests/components/pmwcs3/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_pmwcs3
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: pmwcs3
+    i2c_id: i2c_bus
     e25:
       name: pmwcs3_e25
     ec:
diff --git a/tests/components/pmwcs3/test.esp32-ard.yaml b/tests/components/pmwcs3/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/pmwcs3/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/pmwcs3/test.esp32-c3-ard.yaml b/tests/components/pmwcs3/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/pmwcs3/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/pmwcs3/test.esp32-c3-idf.yaml b/tests/components/pmwcs3/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/pmwcs3/test.esp32-c3-idf.yaml
+++ b/tests/components/pmwcs3/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pmwcs3/test.esp32-idf.yaml b/tests/components/pmwcs3/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/pmwcs3/test.esp32-idf.yaml
+++ b/tests/components/pmwcs3/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pmwcs3/test.esp8266-ard.yaml b/tests/components/pmwcs3/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/pmwcs3/test.esp8266-ard.yaml
+++ b/tests/components/pmwcs3/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pmwcs3/test.rp2040-ard.yaml b/tests/components/pmwcs3/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/pmwcs3/test.rp2040-ard.yaml
+++ b/tests/components/pmwcs3/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pn532_i2c/common.yaml b/tests/components/pn532_i2c/common.yaml
index db9abd4b13..947e1151aa 100644
--- a/tests/components/pn532_i2c/common.yaml
+++ b/tests/components/pn532_i2c/common.yaml
@@ -1,13 +1,9 @@
-i2c:
-  - id: i2c_pn532
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 pn532_i2c:
-  id: pn532_nfcc
+  i2c_id: i2c_bus
+  id: pn532_nfcc_i2c
 
 binary_sensor:
   - platform: pn532
-    pn532_id: pn532_nfcc
+    pn532_id: pn532_nfcc_i2c
     name: PN532 NFC Tag
     uid: 74-10-37-94
diff --git a/tests/components/pn532_i2c/test.esp32-ard.yaml b/tests/components/pn532_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/pn532_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/pn532_i2c/test.esp32-c3-ard.yaml b/tests/components/pn532_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/pn532_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/pn532_i2c/test.esp32-c3-idf.yaml b/tests/components/pn532_i2c/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/pn532_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/pn532_i2c/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pn532_i2c/test.esp32-idf.yaml b/tests/components/pn532_i2c/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/pn532_i2c/test.esp32-idf.yaml
+++ b/tests/components/pn532_i2c/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pn532_i2c/test.esp8266-ard.yaml b/tests/components/pn532_i2c/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/pn532_i2c/test.esp8266-ard.yaml
+++ b/tests/components/pn532_i2c/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pn532_i2c/test.rp2040-ard.yaml b/tests/components/pn532_i2c/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/pn532_i2c/test.rp2040-ard.yaml
+++ b/tests/components/pn532_i2c/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pn532_spi/common.yaml b/tests/components/pn532_spi/common.yaml
index d5b8bc405e..f9149af35f 100644
--- a/tests/components/pn532_spi/common.yaml
+++ b/tests/components/pn532_spi/common.yaml
@@ -1,15 +1,9 @@
-spi:
-  - id: spi_pn532
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 pn532_spi:
-  id: pn532_nfcc
+  id: pn532_nfcc_spi
   cs_pin: ${cs_pin}
 
 binary_sensor:
   - platform: pn532
-    pn532_id: pn532_nfcc
+    pn532_id: pn532_nfcc_spi
     name: PN532 NFC Tag
     uid: 74-10-37-94
diff --git a/tests/components/pn532_spi/test.esp32-ard.yaml b/tests/components/pn532_spi/test.esp32-ard.yaml
deleted file mode 100644
index bce56f398a..0000000000
--- a/tests/components/pn532_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO18
-  cs_pin: GPIO12
-
-<<: !include common.yaml
diff --git a/tests/components/pn532_spi/test.esp32-c3-ard.yaml b/tests/components/pn532_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/pn532_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/pn532_spi/test.esp32-c3-idf.yaml b/tests/components/pn532_spi/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/pn532_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/pn532_spi/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pn532_spi/test.esp32-idf.yaml b/tests/components/pn532_spi/test.esp32-idf.yaml
index bce56f398a..9bb524aa65 100644
--- a/tests/components/pn532_spi/test.esp32-idf.yaml
+++ b/tests/components/pn532_spi/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO18
   cs_pin: GPIO12
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pn532_spi/test.esp8266-ard.yaml b/tests/components/pn532_spi/test.esp8266-ard.yaml
index bd5c203e35..1aac800592 100644
--- a/tests/components/pn532_spi/test.esp8266-ard.yaml
+++ b/tests/components/pn532_spi/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
-  cs_pin: GPIO5
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO15
+  cs_pin: GPIO16
+
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pn532_spi/test.rp2040-ard.yaml b/tests/components/pn532_spi/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/pn532_spi/test.rp2040-ard.yaml
+++ b/tests/components/pn532_spi/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pn7150_i2c/common.yaml b/tests/components/pn7150_i2c/common.yaml
index ef852b7a78..a317b72b9c 100644
--- a/tests/components/pn7150_i2c/common.yaml
+++ b/tests/components/pn7150_i2c/common.yaml
@@ -16,13 +16,9 @@ esphome:
       - tag.polling_off: nfcc_pn7150
       - tag.polling_on: nfcc_pn7150
 
-i2c:
-  - id: i2c_pn7150
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 pn7150_i2c:
   id: nfcc_pn7150
+  i2c_id: i2c_bus
   irq_pin: ${irq_pin}
   ven_pin: ${ven_pin}
   emulation_message: https://www.home-assistant.io/tag/pulse_ce
diff --git a/tests/components/pn7150_i2c/test.esp32-ard.yaml b/tests/components/pn7150_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 1643bec317..0000000000
--- a/tests/components/pn7150_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  irq_pin: GPIO14
-  ven_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/pn7150_i2c/test.esp32-c3-ard.yaml b/tests/components/pn7150_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2067143411..0000000000
--- a/tests/components/pn7150_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  irq_pin: GPIO6
-  ven_pin: GPIO7
-
-<<: !include common.yaml
diff --git a/tests/components/pn7150_i2c/test.esp32-c3-idf.yaml b/tests/components/pn7150_i2c/test.esp32-c3-idf.yaml
index 2067143411..cdf8445263 100644
--- a/tests/components/pn7150_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/pn7150_i2c/test.esp32-c3-idf.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq_pin: GPIO6
   ven_pin: GPIO7
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pn7150_i2c/test.esp32-idf.yaml b/tests/components/pn7150_i2c/test.esp32-idf.yaml
index 1643bec317..9a4cc6669e 100644
--- a/tests/components/pn7150_i2c/test.esp32-idf.yaml
+++ b/tests/components/pn7150_i2c/test.esp32-idf.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   irq_pin: GPIO14
   ven_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pn7150_i2c/test.esp8266-ard.yaml b/tests/components/pn7150_i2c/test.esp8266-ard.yaml
index 7111fc9e00..b907e56f80 100644
--- a/tests/components/pn7150_i2c/test.esp8266-ard.yaml
+++ b/tests/components/pn7150_i2c/test.esp8266-ard.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  irq_pin: GPIO12
-  ven_pin: GPIO13
+  irq_pin: GPIO15
+  ven_pin: GPIO16
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pn7150_i2c/test.rp2040-ard.yaml b/tests/components/pn7150_i2c/test.rp2040-ard.yaml
index 2067143411..b320b947e3 100644
--- a/tests/components/pn7150_i2c/test.rp2040-ard.yaml
+++ b/tests/components/pn7150_i2c/test.rp2040-ard.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq_pin: GPIO6
   ven_pin: GPIO7
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pn7160_i2c/common.yaml b/tests/components/pn7160_i2c/common.yaml
index 0a7c9bd6bb..fa9a876d1c 100644
--- a/tests/components/pn7160_i2c/common.yaml
+++ b/tests/components/pn7160_i2c/common.yaml
@@ -1,28 +1,24 @@
 esphome:
   on_boot:
     then:
-      - tag.set_clean_mode: nfcc_pn7160
-      - tag.set_format_mode: nfcc_pn7160
-      - tag.set_read_mode: nfcc_pn7160
+      - tag.set_clean_mode: nfcc_pn7160_i2c
+      - tag.set_format_mode: nfcc_pn7160_i2c
+      - tag.set_read_mode: nfcc_pn7160_i2c
       - tag.set_write_message:
           message: https://www.home-assistant.io/tag/pulse
           include_android_app_record: false
-      - tag.set_write_mode: nfcc_pn7160
+      - tag.set_write_mode: nfcc_pn7160_i2c
       - tag.set_emulation_message:
           message: https://www.home-assistant.io/tag/pulse
           include_android_app_record: false
-      - tag.emulation_off: nfcc_pn7160
-      - tag.emulation_on: nfcc_pn7160
-      - tag.polling_off: nfcc_pn7160
-      - tag.polling_on: nfcc_pn7160
-
-i2c:
-  - id: i2c_pn7160
-    scl: ${scl_pin}
-    sda: ${sda_pin}
+      - tag.emulation_off: nfcc_pn7160_i2c
+      - tag.emulation_on: nfcc_pn7160_i2c
+      - tag.polling_off: nfcc_pn7160_i2c
+      - tag.polling_on: nfcc_pn7160_i2c
 
 pn7150_i2c:
-  id: nfcc_pn7160
+  id: nfcc_pn7160_i2c
+  i2c_id: i2c_bus
   irq_pin: ${irq_pin}
   ven_pin: ${ven_pin}
   emulation_message: https://www.home-assistant.io/tag/pulse_ce
diff --git a/tests/components/pn7160_i2c/test.esp32-ard.yaml b/tests/components/pn7160_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 1643bec317..0000000000
--- a/tests/components/pn7160_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  irq_pin: GPIO14
-  ven_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/pn7160_i2c/test.esp32-c3-ard.yaml b/tests/components/pn7160_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2067143411..0000000000
--- a/tests/components/pn7160_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  irq_pin: GPIO6
-  ven_pin: GPIO7
-
-<<: !include common.yaml
diff --git a/tests/components/pn7160_i2c/test.esp32-c3-idf.yaml b/tests/components/pn7160_i2c/test.esp32-c3-idf.yaml
index 2067143411..cdf8445263 100644
--- a/tests/components/pn7160_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/pn7160_i2c/test.esp32-c3-idf.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq_pin: GPIO6
   ven_pin: GPIO7
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pn7160_i2c/test.esp32-idf.yaml b/tests/components/pn7160_i2c/test.esp32-idf.yaml
index 1643bec317..9a4cc6669e 100644
--- a/tests/components/pn7160_i2c/test.esp32-idf.yaml
+++ b/tests/components/pn7160_i2c/test.esp32-idf.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   irq_pin: GPIO14
   ven_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pn7160_i2c/test.esp8266-ard.yaml b/tests/components/pn7160_i2c/test.esp8266-ard.yaml
index 7111fc9e00..b907e56f80 100644
--- a/tests/components/pn7160_i2c/test.esp8266-ard.yaml
+++ b/tests/components/pn7160_i2c/test.esp8266-ard.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  irq_pin: GPIO12
-  ven_pin: GPIO13
+  irq_pin: GPIO15
+  ven_pin: GPIO16
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pn7160_i2c/test.rp2040-ard.yaml b/tests/components/pn7160_i2c/test.rp2040-ard.yaml
index 2067143411..b320b947e3 100644
--- a/tests/components/pn7160_i2c/test.rp2040-ard.yaml
+++ b/tests/components/pn7160_i2c/test.rp2040-ard.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   irq_pin: GPIO6
   ven_pin: GPIO7
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pn7160_spi/common.yaml b/tests/components/pn7160_spi/common.yaml
index 9e8d22f835..53b37b38f4 100644
--- a/tests/components/pn7160_spi/common.yaml
+++ b/tests/components/pn7160_spi/common.yaml
@@ -1,29 +1,23 @@
 esphome:
   on_boot:
     then:
-      - tag.set_clean_mode: nfcc_pn7160
-      - tag.set_format_mode: nfcc_pn7160
-      - tag.set_read_mode: nfcc_pn7160
+      - tag.set_clean_mode: nfcc_pn7160_spi
+      - tag.set_format_mode: nfcc_pn7160_spi
+      - tag.set_read_mode: nfcc_pn7160_spi
       - tag.set_write_message:
           message: https://www.home-assistant.io/tag/pulse
           include_android_app_record: false
-      - tag.set_write_mode: nfcc_pn7160
+      - tag.set_write_mode: nfcc_pn7160_spi
       - tag.set_emulation_message:
           message: https://www.home-assistant.io/tag/pulse
           include_android_app_record: false
-      - tag.emulation_off: nfcc_pn7160
-      - tag.emulation_on: nfcc_pn7160
-      - tag.polling_off: nfcc_pn7160
-      - tag.polling_on: nfcc_pn7160
-
-spi:
-  - id: spi_pn7160
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
+      - tag.emulation_off: nfcc_pn7160_spi
+      - tag.emulation_on: nfcc_pn7160_spi
+      - tag.polling_off: nfcc_pn7160_spi
+      - tag.polling_on: nfcc_pn7160_spi
 
 pn7160_spi:
-  id: nfcc_pn7160
+  id: nfcc_pn7160_spi
   cs_pin: ${cs_pin}
   irq_pin: ${irq_pin}
   ven_pin: ${ven_pin}
diff --git a/tests/components/pn7160_spi/test.esp32-ard.yaml b/tests/components/pn7160_spi/test.esp32-ard.yaml
deleted file mode 100644
index f6073d0416..0000000000
--- a/tests/components/pn7160_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO18
-  cs_pin: GPIO12
-  irq_pin: GPIO13
-  ven_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/pn7160_spi/test.esp32-c3-ard.yaml b/tests/components/pn7160_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index f8a07fad2f..0000000000
--- a/tests/components/pn7160_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-  irq_pin: GPIO9
-  ven_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/pn7160_spi/test.esp32-c3-idf.yaml b/tests/components/pn7160_spi/test.esp32-c3-idf.yaml
index f8a07fad2f..ac18bfff5c 100644
--- a/tests/components/pn7160_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/pn7160_spi/test.esp32-c3-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
   irq_pin: GPIO9
   ven_pin: GPIO10
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pn7160_spi/test.esp32-idf.yaml b/tests/components/pn7160_spi/test.esp32-idf.yaml
index f6073d0416..f903e4b7be 100644
--- a/tests/components/pn7160_spi/test.esp32-idf.yaml
+++ b/tests/components/pn7160_spi/test.esp32-idf.yaml
@@ -1,9 +1,9 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO18
   cs_pin: GPIO12
   irq_pin: GPIO13
   ven_pin: GPIO14
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pn7160_spi/test.esp8266-ard.yaml b/tests/components/pn7160_spi/test.esp8266-ard.yaml
index cbe27533a7..7ec89dc012 100644
--- a/tests/components/pn7160_spi/test.esp8266-ard.yaml
+++ b/tests/components/pn7160_spi/test.esp8266-ard.yaml
@@ -1,9 +1,12 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
   miso_pin: GPIO12
   cs_pin: GPIO5
   irq_pin: GPIO15
   ven_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pn7160_spi/test.rp2040-ard.yaml b/tests/components/pn7160_spi/test.rp2040-ard.yaml
index 70cd2425fa..b4a4b436cd 100644
--- a/tests/components/pn7160_spi/test.rp2040-ard.yaml
+++ b/tests/components/pn7160_spi/test.rp2040-ard.yaml
@@ -6,4 +6,7 @@ substitutions:
   irq_pin: GPIO15
   ven_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/power_supply/test.esp32-ard.yaml b/tests/components/power_supply/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/power_supply/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/power_supply/test.esp32-c3-ard.yaml b/tests/components/power_supply/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/power_supply/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/ble_presence/test.esp32-ard.yaml b/tests/components/power_supply/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/ble_presence/test.esp32-ard.yaml
rename to tests/components/power_supply/test.nrf52-adafruit.yaml
diff --git a/tests/components/ble_presence/test.esp32-c3-ard.yaml b/tests/components/power_supply/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/ble_presence/test.esp32-c3-ard.yaml
rename to tests/components/power_supply/test.nrf52-mcumgr.yaml
diff --git a/tests/components/prometheus/common.yaml b/tests/components/prometheus/common.yaml
index 131d135f8b..cf46e882a7 100644
--- a/tests/components/prometheus/common.yaml
+++ b/tests/components/prometheus/common.yaml
@@ -21,10 +21,12 @@ http_request:
 
 ota:
   - platform: http_request
+    id: prometheus_http_request_ota
 
 update:
   - platform: http_request
     name: Firmware Update
+    ota_id: prometheus_http_request_ota
     source: http://example.com/manifest.json
 
 sensor:
@@ -33,9 +35,8 @@ sensor:
     lambda: |-
       if (millis() > 10000) {
         return 42.0;
-      } else {
-        return 0.0;
       }
+      return 0.0;
     update_interval: 60s
 
 text_sensor:
@@ -47,9 +48,8 @@ text_sensor:
     lambda: |-
       if (millis() > 10000) {
         return {"Hello World"};
-      } else {
-        return {"Goodbye (cruel) World"};
       }
+      return {"Goodbye (cruel) World"};
     update_interval: 60s
 
 binary_sensor:
@@ -58,9 +58,8 @@ binary_sensor:
     lambda: |-
       if (millis() > 10000) {
         return true;
-      } else {
-        return false;
       }
+      return false;
 
 switch:
   - platform: template
@@ -68,9 +67,8 @@ switch:
     lambda: |-
       if (millis() > 10000) {
         return true;
-      } else {
-        return false;
       }
+      return false;
     optimistic: true
 
 fan:
@@ -83,9 +81,8 @@ cover:
     lambda: |-
       if (millis() > 10000) {
         return COVER_OPEN;
-      } else {
-        return COVER_CLOSED;
       }
+      return COVER_CLOSED;
 
 lock:
   - platform: template
@@ -93,9 +90,8 @@ lock:
     lambda: |-
       if (millis() > 10000) {
         return LOCK_STATE_LOCKED;
-      } else {
-        return LOCK_STATE_UNLOCKED;
       }
+      return LOCK_STATE_UNLOCKED;
     optimistic: true
 
 select:
@@ -126,13 +122,10 @@ valve:
     optimistic: true
     has_position: true
 
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: climate_ir_lg
     name: LG Climate
+    transmitter_id: xmitr
 
 prometheus:
   include_internal: true
diff --git a/tests/components/prometheus/test.esp32-ard.yaml b/tests/components/prometheus/test.esp32-ard.yaml
deleted file mode 100644
index 9eedaabd82..0000000000
--- a/tests/components/prometheus/test.esp32-ard.yaml
+++ /dev/null
@@ -1,38 +0,0 @@
-substitutions:
-  verify_ssl: "false"
-  pin: GPIO5
-
-<<: !include common.yaml
-
-i2s_audio:
-  i2s_lrclk_pin: 1
-  i2s_bclk_pin: 2
-  i2s_mclk_pin: 3
-
-media_player:
-  - platform: i2s_audio
-    name: "Media Player"
-    dac_type: external
-    i2s_dout_pin: 18
-    mute_pin: 19
-    on_state:
-      - media_player.play:
-      - media_player.play_media: http://localhost/media.mp3
-      - media_player.play_media: !lambda 'return "http://localhost/media.mp3";'
-    on_idle:
-      - media_player.pause:
-    on_play:
-      - media_player.stop:
-    on_pause:
-      - media_player.toggle:
-      - wait_until:
-          media_player.is_idle:
-      - wait_until:
-          media_player.is_playing:
-      - wait_until:
-          media_player.is_announcing:
-      - wait_until:
-          media_player.is_paused:
-      - media_player.volume_up:
-      - media_player.volume_down:
-      - media_player.volume_set: 50%
diff --git a/tests/components/prometheus/test.esp32-c3-ard.yaml b/tests/components/prometheus/test.esp32-c3-ard.yaml
deleted file mode 100644
index f00bca5947..0000000000
--- a/tests/components/prometheus/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  verify_ssl: "false"
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/prometheus/test.esp32-c3-idf.yaml b/tests/components/prometheus/test.esp32-c3-idf.yaml
index f00bca5947..fedeaf822a 100644
--- a/tests/components/prometheus/test.esp32-c3-idf.yaml
+++ b/tests/components/prometheus/test.esp32-c3-idf.yaml
@@ -1,5 +1,7 @@
 substitutions:
   verify_ssl: "false"
-  pin: GPIO2
+
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/prometheus/test.esp32-idf.yaml b/tests/components/prometheus/test.esp32-idf.yaml
index f00bca5947..e590417623 100644
--- a/tests/components/prometheus/test.esp32-idf.yaml
+++ b/tests/components/prometheus/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
   verify_ssl: "false"
-  pin: GPIO2
+
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/prometheus/test.esp8266-ard.yaml b/tests/components/prometheus/test.esp8266-ard.yaml
index 6ee1831769..bae76751e8 100644
--- a/tests/components/prometheus/test.esp8266-ard.yaml
+++ b/tests/components/prometheus/test.esp8266-ard.yaml
@@ -1,5 +1,7 @@
 substitutions:
   verify_ssl: "false"
-  pin: GPIO5
+
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/psram/test.esp32-ard.yaml b/tests/components/psram/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/psram/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/psram/test.esp32-s2-ard.yaml b/tests/components/psram/test.esp32-s2-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/psram/test.esp32-s2-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/psram/test.esp32-s3-ard.yaml b/tests/components/psram/test.esp32-s3-ard.yaml
deleted file mode 100644
index cfd39f77fe..0000000000
--- a/tests/components/psram/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,3 +0,0 @@
-psram:
-  mode: octal
-  speed: 80MHz
diff --git a/tests/components/pulse_counter/test.esp32-ard.yaml b/tests/components/pulse_counter/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/pulse_counter/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/pulse_counter/test.esp32-c3-ard.yaml b/tests/components/pulse_counter/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/pulse_counter/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/ble_rssi/test.esp32-ard.yaml b/tests/components/pulse_counter/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/ble_rssi/test.esp32-ard.yaml
rename to tests/components/pulse_counter/test.nrf52-adafruit.yaml
diff --git a/tests/components/ble_rssi/test.esp32-c3-ard.yaml b/tests/components/pulse_counter/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/ble_rssi/test.esp32-c3-ard.yaml
rename to tests/components/pulse_counter/test.nrf52-mcumgr.yaml
diff --git a/tests/components/pulse_meter/test.esp32-ard.yaml b/tests/components/pulse_meter/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/pulse_meter/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/pulse_meter/test.esp32-c3-ard.yaml b/tests/components/pulse_meter/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/pulse_meter/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/ble_scanner/test.esp32-ard.yaml b/tests/components/pulse_meter/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/ble_scanner/test.esp32-ard.yaml
rename to tests/components/pulse_meter/test.nrf52-adafruit.yaml
diff --git a/tests/components/ble_scanner/test.esp32-c3-ard.yaml b/tests/components/pulse_meter/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/ble_scanner/test.esp32-c3-ard.yaml
rename to tests/components/pulse_meter/test.nrf52-mcumgr.yaml
diff --git a/tests/components/pulse_width/test.esp32-ard.yaml b/tests/components/pulse_width/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/pulse_width/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/pulse_width/test.esp32-c3-ard.yaml b/tests/components/pulse_width/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/pulse_width/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/button/test.esp32-ard.yaml b/tests/components/pulse_width/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/button/test.esp32-ard.yaml
rename to tests/components/pulse_width/test.nrf52-adafruit.yaml
diff --git a/tests/components/button/test.esp32-c3-ard.yaml b/tests/components/pulse_width/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/button/test.esp32-c3-ard.yaml
rename to tests/components/pulse_width/test.nrf52-mcumgr.yaml
diff --git a/tests/components/pvvx_mithermometer/test.esp32-ard.yaml b/tests/components/pvvx_mithermometer/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/pvvx_mithermometer/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/pvvx_mithermometer/test.esp32-c3-ard.yaml b/tests/components/pvvx_mithermometer/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/pvvx_mithermometer/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/pvvx_mithermometer/test.esp32-c3-idf.yaml b/tests/components/pvvx_mithermometer/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/pvvx_mithermometer/test.esp32-c3-idf.yaml
+++ b/tests/components/pvvx_mithermometer/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pvvx_mithermometer/test.esp32-idf.yaml b/tests/components/pvvx_mithermometer/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/pvvx_mithermometer/test.esp32-idf.yaml
+++ b/tests/components/pvvx_mithermometer/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pylontech/common.yaml b/tests/components/pylontech/common.yaml
index 6852685be7..537450a3b6 100644
--- a/tests/components/pylontech/common.yaml
+++ b/tests/components/pylontech/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_pylontech0
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 115200
-
 pylontech:
   - id: pylontech0
   - id: pylontech1
diff --git a/tests/components/pylontech/test.esp32-ard.yaml b/tests/components/pylontech/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/pylontech/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/pylontech/test.esp32-c3-ard.yaml b/tests/components/pylontech/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/pylontech/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/pylontech/test.esp32-c3-idf.yaml b/tests/components/pylontech/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/pylontech/test.esp32-c3-idf.yaml
+++ b/tests/components/pylontech/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pylontech/test.esp32-idf.yaml b/tests/components/pylontech/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/pylontech/test.esp32-idf.yaml
+++ b/tests/components/pylontech/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pylontech/test.esp8266-ard.yaml b/tests/components/pylontech/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/pylontech/test.esp8266-ard.yaml
+++ b/tests/components/pylontech/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pylontech/test.rp2040-ard.yaml b/tests/components/pylontech/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/pylontech/test.rp2040-ard.yaml
+++ b/tests/components/pylontech/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pzem004t/common.yaml b/tests/components/pzem004t/common.yaml
index 75f7f30fc9..4584f9c273 100644
--- a/tests/components/pzem004t/common.yaml
+++ b/tests/components/pzem004t/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_pzem004t
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 115200
-
 sensor:
   - platform: pzem004t
     voltage:
diff --git a/tests/components/pzem004t/test.esp32-ard.yaml b/tests/components/pzem004t/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/pzem004t/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/pzem004t/test.esp32-c3-ard.yaml b/tests/components/pzem004t/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/pzem004t/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/pzem004t/test.esp32-c3-idf.yaml b/tests/components/pzem004t/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/pzem004t/test.esp32-c3-idf.yaml
+++ b/tests/components/pzem004t/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pzem004t/test.esp32-idf.yaml b/tests/components/pzem004t/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/pzem004t/test.esp32-idf.yaml
+++ b/tests/components/pzem004t/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pzem004t/test.esp8266-ard.yaml b/tests/components/pzem004t/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/pzem004t/test.esp8266-ard.yaml
+++ b/tests/components/pzem004t/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pzem004t/test.rp2040-ard.yaml b/tests/components/pzem004t/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/pzem004t/test.rp2040-ard.yaml
+++ b/tests/components/pzem004t/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pzemac/common.yaml b/tests/components/pzemac/common.yaml
index e50f4ad2f2..2566051baa 100644
--- a/tests/components/pzemac/common.yaml
+++ b/tests/components/pzemac/common.yaml
@@ -3,16 +3,9 @@ esphome:
     then:
       - pzemac.reset_energy: pzemac1
 
-uart:
-  - id: uart_pzemac
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
-modbus:
-
 sensor:
   - platform: pzemac
+    modbus_id: modbus_bus
     id: pzemac1
     voltage:
       name: PZEMAC Voltage
diff --git a/tests/components/pzemac/test.esp32-ard.yaml b/tests/components/pzemac/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/pzemac/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/pzemac/test.esp32-c3-ard.yaml b/tests/components/pzemac/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/pzemac/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/pzemac/test.esp32-c3-idf.yaml b/tests/components/pzemac/test.esp32-c3-idf.yaml
index b516342f3b..6c6e95488f 100644
--- a/tests/components/pzemac/test.esp32-c3-idf.yaml
+++ b/tests/components/pzemac/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pzemac/test.esp32-idf.yaml b/tests/components/pzemac/test.esp32-idf.yaml
index f486544afa..b631e16677 100644
--- a/tests/components/pzemac/test.esp32-idf.yaml
+++ b/tests/components/pzemac/test.esp32-idf.yaml
@@ -1,5 +1,9 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+  flow_control_pin: GPIO13
+
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pzemac/test.esp8266-ard.yaml b/tests/components/pzemac/test.esp8266-ard.yaml
index b516342f3b..421389ae97 100644
--- a/tests/components/pzemac/test.esp8266-ard.yaml
+++ b/tests/components/pzemac/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pzemac/test.rp2040-ard.yaml b/tests/components/pzemac/test.rp2040-ard.yaml
index b516342f3b..d78d84c983 100644
--- a/tests/components/pzemac/test.rp2040-ard.yaml
+++ b/tests/components/pzemac/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  modbus: !include ../../test_build_components/common/modbus/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/pzemdc/common.yaml b/tests/components/pzemdc/common.yaml
index db1868d682..78a0ab0d07 100644
--- a/tests/components/pzemdc/common.yaml
+++ b/tests/components/pzemdc/common.yaml
@@ -3,15 +3,9 @@ esphome:
     then:
       - pzemdc.reset_energy: pzemdc1
 
-uart:
-  - id: uart_pzemdc
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-    stop_bits: 2
-
 sensor:
   - platform: pzemdc
+    modbus_id: modbus_bus
     id: pzemdc1
     voltage:
       name: PZEMDC Voltage
diff --git a/tests/components/pzemdc/test.esp32-ard.yaml b/tests/components/pzemdc/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/pzemdc/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/pzemdc/test.esp32-c3-ard.yaml b/tests/components/pzemdc/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/pzemdc/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/pzemdc/test.esp32-c3-idf.yaml b/tests/components/pzemdc/test.esp32-c3-idf.yaml
index b516342f3b..6c6e95488f 100644
--- a/tests/components/pzemdc/test.esp32-c3-idf.yaml
+++ b/tests/components/pzemdc/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pzemdc/test.esp32-idf.yaml b/tests/components/pzemdc/test.esp32-idf.yaml
index f486544afa..b631e16677 100644
--- a/tests/components/pzemdc/test.esp32-idf.yaml
+++ b/tests/components/pzemdc/test.esp32-idf.yaml
@@ -1,5 +1,9 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+  flow_control_pin: GPIO13
+
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pzemdc/test.esp8266-ard.yaml b/tests/components/pzemdc/test.esp8266-ard.yaml
index b516342f3b..421389ae97 100644
--- a/tests/components/pzemdc/test.esp8266-ard.yaml
+++ b/tests/components/pzemdc/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/pzemdc/test.rp2040-ard.yaml b/tests/components/pzemdc/test.rp2040-ard.yaml
index b516342f3b..d78d84c983 100644
--- a/tests/components/pzemdc/test.rp2040-ard.yaml
+++ b/tests/components/pzemdc/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  modbus: !include ../../test_build_components/common/modbus/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/qmc5883l/common.yaml b/tests/components/qmc5883l/common.yaml
index 5d8ac73b4f..98d0350a60 100644
--- a/tests/components/qmc5883l/common.yaml
+++ b/tests/components/qmc5883l/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_qmc5883l
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: qmc5883l
+    i2c_id: i2c_bus
     address: 0x0D
     field_strength_x:
       name: QMC5883L Field Strength X
@@ -17,5 +13,7 @@ sensor:
     temperature:
       name: QMC5883L Temperature
     range: 800uT
+    data_rate: 200Hz
     oversampling: 256x
     update_interval: 15s
+    drdy_pin: ${drdy_pin}
diff --git a/tests/components/qmc5883l/test.esp32-ard.yaml b/tests/components/qmc5883l/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/qmc5883l/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/qmc5883l/test.esp32-c3-ard.yaml b/tests/components/qmc5883l/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/qmc5883l/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/qmc5883l/test.esp32-c3-idf.yaml b/tests/components/qmc5883l/test.esp32-c3-idf.yaml
index ee2c29ca4e..854ddc25e7 100644
--- a/tests/components/qmc5883l/test.esp32-c3-idf.yaml
+++ b/tests/components/qmc5883l/test.esp32-c3-idf.yaml
@@ -1,5 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+  drdy_pin: GPIO6
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/qmc5883l/test.esp32-idf.yaml b/tests/components/qmc5883l/test.esp32-idf.yaml
index 63c3bd6afd..07bf2b14e2 100644
--- a/tests/components/qmc5883l/test.esp32-idf.yaml
+++ b/tests/components/qmc5883l/test.esp32-idf.yaml
@@ -1,5 +1,7 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+  drdy_pin: GPIO12
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/qmc5883l/test.esp8266-ard.yaml b/tests/components/qmc5883l/test.esp8266-ard.yaml
index ee2c29ca4e..0c02b3b96a 100644
--- a/tests/components/qmc5883l/test.esp8266-ard.yaml
+++ b/tests/components/qmc5883l/test.esp8266-ard.yaml
@@ -1,5 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+  drdy_pin: GPIO2
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/qmc5883l/test.rp2040-ard.yaml b/tests/components/qmc5883l/test.rp2040-ard.yaml
index ee2c29ca4e..3b10540b1f 100644
--- a/tests/components/qmc5883l/test.rp2040-ard.yaml
+++ b/tests/components/qmc5883l/test.rp2040-ard.yaml
@@ -1,5 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+  drdy_pin: GPIO2
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/qmp6988/common.yaml b/tests/components/qmp6988/common.yaml
index cb4b221df0..2aea228a0d 100644
--- a/tests/components/qmp6988/common.yaml
+++ b/tests/components/qmp6988/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_qmp6988
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: qmp6988
+    i2c_id: i2c_bus
     temperature:
       name: QMP6988 Temperature
       oversampling: 32x
diff --git a/tests/components/qmp6988/test.esp32-ard.yaml b/tests/components/qmp6988/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/qmp6988/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/qmp6988/test.esp32-c3-ard.yaml b/tests/components/qmp6988/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/qmp6988/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/qmp6988/test.esp32-c3-idf.yaml b/tests/components/qmp6988/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/qmp6988/test.esp32-c3-idf.yaml
+++ b/tests/components/qmp6988/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/qmp6988/test.esp32-idf.yaml b/tests/components/qmp6988/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/qmp6988/test.esp32-idf.yaml
+++ b/tests/components/qmp6988/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/qmp6988/test.esp8266-ard.yaml b/tests/components/qmp6988/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/qmp6988/test.esp8266-ard.yaml
+++ b/tests/components/qmp6988/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/qmp6988/test.rp2040-ard.yaml b/tests/components/qmp6988/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/qmp6988/test.rp2040-ard.yaml
+++ b/tests/components/qmp6988/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/qr_code/common.yaml b/tests/components/qr_code/common.yaml
index 85c2ee076b..5fec26c1cc 100644
--- a/tests/components/qr_code/common.yaml
+++ b/tests/components/qr_code/common.yaml
@@ -1,11 +1,6 @@
-spi:
-  - id: spi_main_lcd
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   - platform: ili9xxx
-    id: main_lcd
+    id: qr_code_main_lcd
     model: ili9342
     cs_pin: ${cs_pin}
     dc_pin: ${dc_pin}
@@ -14,11 +9,11 @@ display:
     lambda: |-
       // Draw a QR code in the center of the screen
       auto scale = 2;
-      auto size = id(homepage_qr).get_size() * scale;
+      auto size = id(qr_code_homepage_qr).get_size() * scale;
       auto x = (it.get_width() / 2) - (size / 2);
       auto y = (it.get_height() / 2) - (size / 2);
-      it.qr_code(x, y, id(homepage_qr), Color(255,255,255), scale);
+      it.qr_code(x, y, id(qr_code_homepage_qr), Color(255,255,255), scale);
 
 qr_code:
-  - id: homepage_qr
+  - id: qr_code_homepage_qr
     value: https://esphome.io/index.html
diff --git a/tests/components/qr_code/test.esp32-ard.yaml b/tests/components/qr_code/test.esp32-ard.yaml
deleted file mode 100644
index bad5241f79..0000000000
--- a/tests/components/qr_code/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  cs_pin: GPIO12
-  dc_pin: GPIO13
-  reset_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/qr_code/test.esp32-c3-ard.yaml b/tests/components/qr_code/test.esp32-c3-ard.yaml
deleted file mode 100644
index c5c932c92c..0000000000
--- a/tests/components/qr_code/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-  dc_pin: GPIO9
-  reset_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/qr_code/test.esp32-c3-idf.yaml b/tests/components/qr_code/test.esp32-c3-idf.yaml
index c5c932c92c..b112cf4c31 100644
--- a/tests/components/qr_code/test.esp32-c3-idf.yaml
+++ b/tests/components/qr_code/test.esp32-c3-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
   dc_pin: GPIO9
   reset_pin: GPIO10
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/qr_code/test.esp32-idf.yaml b/tests/components/qr_code/test.esp32-idf.yaml
index bad5241f79..ff174a4656 100644
--- a/tests/components/qr_code/test.esp32-idf.yaml
+++ b/tests/components/qr_code/test.esp32-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
   cs_pin: GPIO12
   dc_pin: GPIO13
   reset_pin: GPIO14
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/qr_code/test.esp8266-ard.yaml b/tests/components/qr_code/test.esp8266-ard.yaml
index 3f023a60eb..56cb29f29e 100644
--- a/tests/components/qr_code/test.esp8266-ard.yaml
+++ b/tests/components/qr_code/test.esp8266-ard.yaml
@@ -1,9 +1,12 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
   miso_pin: GPIO12
   cs_pin: GPIO5
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/qr_code/test.rp2040-ard.yaml b/tests/components/qr_code/test.rp2040-ard.yaml
index d7fd6ee294..66caa956f7 100644
--- a/tests/components/qr_code/test.rp2040-ard.yaml
+++ b/tests/components/qr_code/test.rp2040-ard.yaml
@@ -6,4 +6,7 @@ substitutions:
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/qspi_dbi/common.yaml b/tests/components/qspi_dbi/common.yaml
index f4b15f2b90..109db65b63 100644
--- a/tests/components/qspi_dbi/common.yaml
+++ b/tests/components/qspi_dbi/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  id: quad_spi
-  clk_pin: 15
-  type: quad
-  data_pins: [14, 10, 16, 12]
-
 display:
   - platform: qspi_dbi
     model: RM690B0
diff --git a/tests/components/qspi_dbi/test.esp32-s3-idf.yaml b/tests/components/qspi_dbi/test.esp32-s3-idf.yaml
index dade44d145..c335dee1f3 100644
--- a/tests/components/qspi_dbi/test.esp32-s3-idf.yaml
+++ b/tests/components/qspi_dbi/test.esp32-s3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  qspi: !include ../../test_build_components/common/qspi/esp32-s3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/qwiic_pir/common.yaml b/tests/components/qwiic_pir/common.yaml
index d4b733405d..30418ff6b9 100644
--- a/tests/components/qwiic_pir/common.yaml
+++ b/tests/components/qwiic_pir/common.yaml
@@ -1,8 +1,4 @@
-i2c:
-  - id: i2c_qwiic_pir
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 binary_sensor:
   - platform: qwiic_pir
+    i2c_id: i2c_bus
     name: Qwiic PIR Motion Sensor
diff --git a/tests/components/qwiic_pir/test.esp32-ard.yaml b/tests/components/qwiic_pir/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/qwiic_pir/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/qwiic_pir/test.esp32-c3-ard.yaml b/tests/components/qwiic_pir/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/qwiic_pir/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/qwiic_pir/test.esp32-c3-idf.yaml b/tests/components/qwiic_pir/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/qwiic_pir/test.esp32-c3-idf.yaml
+++ b/tests/components/qwiic_pir/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/qwiic_pir/test.esp32-idf.yaml b/tests/components/qwiic_pir/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/qwiic_pir/test.esp32-idf.yaml
+++ b/tests/components/qwiic_pir/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/qwiic_pir/test.esp8266-ard.yaml b/tests/components/qwiic_pir/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/qwiic_pir/test.esp8266-ard.yaml
+++ b/tests/components/qwiic_pir/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/qwiic_pir/test.rp2040-ard.yaml b/tests/components/qwiic_pir/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/qwiic_pir/test.rp2040-ard.yaml
+++ b/tests/components/qwiic_pir/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/radon_eye_ble/test.esp32-ard.yaml b/tests/components/radon_eye_ble/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/radon_eye_ble/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/radon_eye_ble/test.esp32-c3-ard.yaml b/tests/components/radon_eye_ble/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/radon_eye_ble/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/radon_eye_ble/test.esp32-c3-idf.yaml b/tests/components/radon_eye_ble/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/radon_eye_ble/test.esp32-c3-idf.yaml
+++ b/tests/components/radon_eye_ble/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/radon_eye_ble/test.esp32-idf.yaml b/tests/components/radon_eye_ble/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/radon_eye_ble/test.esp32-idf.yaml
+++ b/tests/components/radon_eye_ble/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/radon_eye_rd200/test.esp32-ard.yaml b/tests/components/radon_eye_rd200/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/radon_eye_rd200/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/radon_eye_rd200/test.esp32-c3-ard.yaml b/tests/components/radon_eye_rd200/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/radon_eye_rd200/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/radon_eye_rd200/test.esp32-c3-idf.yaml b/tests/components/radon_eye_rd200/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/radon_eye_rd200/test.esp32-c3-idf.yaml
+++ b/tests/components/radon_eye_rd200/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/radon_eye_rd200/test.esp32-idf.yaml b/tests/components/radon_eye_rd200/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/radon_eye_rd200/test.esp32-idf.yaml
+++ b/tests/components/radon_eye_rd200/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/rc522_i2c/common.yaml b/tests/components/rc522_i2c/common.yaml
index b8b7a41bc7..0e624351d4 100644
--- a/tests/components/rc522_i2c/common.yaml
+++ b/tests/components/rc522_i2c/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_rc522
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 rc522_i2c:
-  - id: rc522_nfcc
+  - id: rc522_nfcc_i2c
+    i2c_id: i2c_bus
     update_interval: 1s
     on_tag:
       - lambda: |-
@@ -12,6 +8,6 @@ rc522_i2c:
 
 binary_sensor:
   - platform: rc522
-    rc522_id: rc522_nfcc
+    rc522_id: rc522_nfcc_i2c
     name: RC522 NFC Tag
     uid: 74-10-37-94
diff --git a/tests/components/rc522_i2c/test.esp32-ard.yaml b/tests/components/rc522_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/rc522_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/rc522_i2c/test.esp32-c3-ard.yaml b/tests/components/rc522_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/rc522_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/rc522_i2c/test.esp32-c3-idf.yaml b/tests/components/rc522_i2c/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/rc522_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/rc522_i2c/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/rc522_i2c/test.esp32-idf.yaml b/tests/components/rc522_i2c/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/rc522_i2c/test.esp32-idf.yaml
+++ b/tests/components/rc522_i2c/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/rc522_i2c/test.esp8266-ard.yaml b/tests/components/rc522_i2c/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/rc522_i2c/test.esp8266-ard.yaml
+++ b/tests/components/rc522_i2c/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/rc522_i2c/test.rp2040-ard.yaml b/tests/components/rc522_i2c/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/rc522_i2c/test.rp2040-ard.yaml
+++ b/tests/components/rc522_i2c/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/rc522_spi/common.yaml b/tests/components/rc522_spi/common.yaml
index 5c42858993..c4830850de 100644
--- a/tests/components/rc522_spi/common.yaml
+++ b/tests/components/rc522_spi/common.yaml
@@ -1,15 +1,9 @@
-spi:
-  - id: spi_rc522
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 rc522_spi:
-  id: rc522_nfcc
+  id: rc522_nfcc_spi
   cs_pin: ${cs_pin}
 
 binary_sensor:
   - platform: rc522
-    rc522_id: rc522_nfcc
-    name: PN532 NFC Tag
+    rc522_id: rc522_nfcc_spi
+    name: RC522 NFC Tag
     uid: 74-10-37-94
diff --git a/tests/components/rc522_spi/test.esp32-ard.yaml b/tests/components/rc522_spi/test.esp32-ard.yaml
deleted file mode 100644
index 54e027a614..0000000000
--- a/tests/components/rc522_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/rc522_spi/test.esp32-c3-ard.yaml b/tests/components/rc522_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/rc522_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/rc522_spi/test.esp32-c3-idf.yaml b/tests/components/rc522_spi/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/rc522_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/rc522_spi/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/rc522_spi/test.esp32-idf.yaml b/tests/components/rc522_spi/test.esp32-idf.yaml
index 54e027a614..a3352cf880 100644
--- a/tests/components/rc522_spi/test.esp32-idf.yaml
+++ b/tests/components/rc522_spi/test.esp32-idf.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/rc522_spi/test.esp8266-ard.yaml b/tests/components/rc522_spi/test.esp8266-ard.yaml
index dbd158d030..b4673ba8b7 100644
--- a/tests/components/rc522_spi/test.esp8266-ard.yaml
+++ b/tests/components/rc522_spi/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO16
   cs_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/rc522_spi/test.rp2040-ard.yaml b/tests/components/rc522_spi/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/rc522_spi/test.rp2040-ard.yaml
+++ b/tests/components/rc522_spi/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/rdm6300/common.yaml b/tests/components/rdm6300/common.yaml
index 118a295471..f1a5305013 100644
--- a/tests/components/rdm6300/common.yaml
+++ b/tests/components/rdm6300/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_rdm6300
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 115200
-
 rdm6300:
 
 binary_sensor:
diff --git a/tests/components/rdm6300/test.esp32-ard.yaml b/tests/components/rdm6300/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/rdm6300/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/rdm6300/test.esp32-c3-ard.yaml b/tests/components/rdm6300/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/rdm6300/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/rdm6300/test.esp32-c3-idf.yaml b/tests/components/rdm6300/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/rdm6300/test.esp32-c3-idf.yaml
+++ b/tests/components/rdm6300/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/rdm6300/test.esp32-idf.yaml b/tests/components/rdm6300/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/rdm6300/test.esp32-idf.yaml
+++ b/tests/components/rdm6300/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/rdm6300/test.esp8266-ard.yaml b/tests/components/rdm6300/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/rdm6300/test.esp8266-ard.yaml
+++ b/tests/components/rdm6300/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/rdm6300/test.rp2040-ard.yaml b/tests/components/rdm6300/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/rdm6300/test.rp2040-ard.yaml
+++ b/tests/components/rdm6300/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/remote_receiver/common-actions.yaml b/tests/components/remote_receiver/common-actions.yaml
index ca7713f58a..c2dc2f0c29 100644
--- a/tests/components/remote_receiver/common-actions.yaml
+++ b/tests/components/remote_receiver/common-actions.yaml
@@ -143,6 +143,11 @@ on_sony:
     - logger.log:
         format: "on_sony: %lu %u"
         args: ["long(x.data)", "x.nbits"]
+on_symphony:
+  then:
+    - logger.log:
+        format: "on_symphony: 0x%lX %u"
+        args: ["long(x.data)", "x.nbits"]
 on_toshiba_ac:
   then:
     - logger.log:
diff --git a/tests/components/remote_receiver/test.esp32-ard.yaml b/tests/components/remote_receiver/test.esp32-ard.yaml
deleted file mode 100644
index 10dd767598..0000000000
--- a/tests/components/remote_receiver/test.esp32-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  pin: GPIO2
-  clock_resolution: "2000000"
-  filter_symbols: "2"
-  receive_symbols: "4"
-  rmt_symbols: "64"
-
-packages:
-  common: !include esp32-common.yaml
diff --git a/tests/components/remote_receiver/test.esp32-c3-ard.yaml b/tests/components/remote_receiver/test.esp32-c3-ard.yaml
deleted file mode 100644
index 10dd767598..0000000000
--- a/tests/components/remote_receiver/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  pin: GPIO2
-  clock_resolution: "2000000"
-  filter_symbols: "2"
-  receive_symbols: "4"
-  rmt_symbols: "64"
-
-packages:
-  common: !include esp32-common.yaml
diff --git a/tests/components/remote_receiver/test.esp32-idf.yaml b/tests/components/remote_receiver/test.esp32-idf.yaml
index 10dd767598..cdeeab2c4a 100644
--- a/tests/components/remote_receiver/test.esp32-idf.yaml
+++ b/tests/components/remote_receiver/test.esp32-idf.yaml
@@ -1,6 +1,8 @@
 substitutions:
   pin: GPIO2
   clock_resolution: "2000000"
+  carrier_duty_percent: "25"
+  carrier_frequency: "30000"
   filter_symbols: "2"
   receive_symbols: "4"
   rmt_symbols: "64"
diff --git a/tests/components/remote_receiver/test.esp32-s3-idf.yaml b/tests/components/remote_receiver/test.esp32-s3-idf.yaml
index cdae8b1e4e..0d7a20e0c0 100644
--- a/tests/components/remote_receiver/test.esp32-s3-idf.yaml
+++ b/tests/components/remote_receiver/test.esp32-s3-idf.yaml
@@ -5,9 +5,22 @@ substitutions:
   receive_symbols: "4"
   rmt_symbols: "64"
 
-packages:
-  common: !include esp32-common.yaml
-
+# WARNING: Using !extend or !remove prevents automatic component grouping in CI, making builds slower.
 remote_receiver:
-  - id: !extend rcvr
+  - id: rcvr
+    pin: ${pin}
+    dump: all
+    tolerance: 25%
+    clock_resolution: ${clock_resolution}
+    filter_symbols: ${filter_symbols}
+    receive_symbols: ${receive_symbols}
+    rmt_symbols: ${rmt_symbols}
     use_dma: "true"
+    <<: !include common-actions.yaml
+
+binary_sensor:
+  - platform: remote_receiver
+    name: Panasonic Remote Input
+    panasonic:
+      address: 0x4004
+      command: 0x100BCBD
diff --git a/tests/components/remote_transmitter/common-buttons.yaml b/tests/components/remote_transmitter/common-buttons.yaml
index 29f48d995d..58127d1ab4 100644
--- a/tests/components/remote_transmitter/common-buttons.yaml
+++ b/tests/components/remote_transmitter/common-buttons.yaml
@@ -53,6 +53,12 @@ button:
       remote_transmitter.transmit_sony:
         data: 0xABCDEF
         nbits: 12
+  - platform: template
+    name: Symphony
+    on_press:
+      remote_transmitter.transmit_symphony:
+        data: 0xE88
+        nbits: 12
   - platform: template
     name: Panasonic
     on_press:
@@ -204,3 +210,9 @@ button:
           command: 0xEC
           rc_code_1: 0x0D
           rc_code_2: 0x0D
+  - platform: template
+    name: Digital Write
+    on_press:
+      - remote_transmitter.digital_write: true
+      - remote_transmitter.digital_write:
+          value: false
diff --git a/tests/components/remote_transmitter/test.esp32-ard.yaml b/tests/components/remote_transmitter/test.esp32-ard.yaml
deleted file mode 100644
index 0522f4d181..0000000000
--- a/tests/components/remote_transmitter/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  pin: GPIO2
-  clock_resolution: "2000000"
-  rmt_symbols: "64"
-
-packages:
-  common: !include esp32-common.yaml
diff --git a/tests/components/remote_transmitter/test.esp32-c3-ard.yaml b/tests/components/remote_transmitter/test.esp32-c3-ard.yaml
deleted file mode 100644
index 0522f4d181..0000000000
--- a/tests/components/remote_transmitter/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  pin: GPIO2
-  clock_resolution: "2000000"
-  rmt_symbols: "64"
-
-packages:
-  common: !include esp32-common.yaml
diff --git a/tests/components/remote_transmitter/test.esp32-s3-idf.yaml b/tests/components/remote_transmitter/test.esp32-s3-idf.yaml
index fe4c46d9e7..8a038beb4a 100644
--- a/tests/components/remote_transmitter/test.esp32-s3-idf.yaml
+++ b/tests/components/remote_transmitter/test.esp32-s3-idf.yaml
@@ -3,9 +3,14 @@ substitutions:
   clock_resolution: "2000000"
   rmt_symbols: "64"
 
-packages:
-  common: !include esp32-common.yaml
-
+# WARNING: Using !extend or !remove prevents automatic component grouping in CI, making builds slower.
 remote_transmitter:
-  - id: !extend xmitr
+  - id: xmitr
+    pin: ${pin}
+    carrier_duty_percent: 50%
+    clock_resolution: ${clock_resolution}
+    rmt_symbols: ${rmt_symbols}
     use_dma: "true"
+
+packages:
+  buttons: !include common-buttons.yaml
diff --git a/tests/components/resampler/test.esp32-ard.yaml b/tests/components/resampler/test.esp32-ard.yaml
deleted file mode 100644
index 96d2d37458..0000000000
--- a/tests/components/resampler/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  lrclk_pin: GPIO16
-  bclk_pin: GPIO17
-  mclk_pin: GPIO15
-  dout_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/resampler/test.esp32-c3-ard.yaml b/tests/components/resampler/test.esp32-c3-ard.yaml
deleted file mode 100644
index f1721f0862..0000000000
--- a/tests/components/resampler/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  lrclk_pin: GPIO4
-  bclk_pin: GPIO5
-  mclk_pin: GPIO6
-  dout_pin: GPIO7
-
-<<: !include common.yaml
diff --git a/tests/components/resampler/test.esp32-idf.yaml b/tests/components/resampler/test.esp32-idf.yaml
index 96d2d37458..6712f1e468 100644
--- a/tests/components/resampler/test.esp32-idf.yaml
+++ b/tests/components/resampler/test.esp32-idf.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  lrclk_pin: GPIO16
-  bclk_pin: GPIO17
+  lrclk_pin: GPIO4
+  bclk_pin: GPIO5
   mclk_pin: GPIO15
   dout_pin: GPIO14
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/resampler/test.esp32-s3-ard.yaml b/tests/components/resampler/test.esp32-s3-ard.yaml
deleted file mode 100644
index f1721f0862..0000000000
--- a/tests/components/resampler/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  lrclk_pin: GPIO4
-  bclk_pin: GPIO5
-  mclk_pin: GPIO6
-  dout_pin: GPIO7
-
-<<: !include common.yaml
diff --git a/tests/components/resistance/test.esp32-ard.yaml b/tests/components/resistance/test.esp32-ard.yaml
deleted file mode 100644
index 06864605a6..0000000000
--- a/tests/components/resistance/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO32
-
-<<: !include common.yaml
diff --git a/tests/components/resistance/test.esp32-c3-ard.yaml b/tests/components/resistance/test.esp32-c3-ard.yaml
deleted file mode 100644
index 37fb325f4a..0000000000
--- a/tests/components/resistance/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/resistance/test.esp32-s2-ard.yaml b/tests/components/resistance/test.esp32-s2-ard.yaml
deleted file mode 100644
index 1910f325ae..0000000000
--- a/tests/components/resistance/test.esp32-s2-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO1
-
-<<: !include common.yaml
diff --git a/tests/components/resistance/test.esp32-s3-ard.yaml b/tests/components/resistance/test.esp32-s3-ard.yaml
deleted file mode 100644
index 1910f325ae..0000000000
--- a/tests/components/resistance/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO1
-
-<<: !include common.yaml
diff --git a/tests/components/restart/test.esp32-ard.yaml b/tests/components/restart/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/restart/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/restart/test.esp32-c3-ard.yaml b/tests/components/restart/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/restart/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/camera/test.esp32-ard.yaml b/tests/components/restart/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/camera/test.esp32-ard.yaml
rename to tests/components/restart/test.nrf52-adafruit.yaml
diff --git a/tests/components/canbus/test.esp32-ard.yaml b/tests/components/restart/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/canbus/test.esp32-ard.yaml
rename to tests/components/restart/test.nrf52-mcumgr.yaml
diff --git a/tests/components/rf_bridge/common.yaml b/tests/components/rf_bridge/common.yaml
index eaadc4bb9c..427c3d783d 100644
--- a/tests/components/rf_bridge/common.yaml
+++ b/tests/components/rf_bridge/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_rf_bridge
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 115200
-
 rf_bridge:
   on_code_received:
     - lambda: |-
diff --git a/tests/components/rf_bridge/test.esp32-ard.yaml b/tests/components/rf_bridge/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/rf_bridge/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/rf_bridge/test.esp32-c3-ard.yaml b/tests/components/rf_bridge/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/rf_bridge/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/rf_bridge/test.esp32-c3-idf.yaml b/tests/components/rf_bridge/test.esp32-c3-idf.yaml
index b516342f3b..a19013bf54 100644
--- a/tests/components/rf_bridge/test.esp32-c3-idf.yaml
+++ b/tests/components/rf_bridge/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/rf_bridge/test.esp32-idf.yaml b/tests/components/rf_bridge/test.esp32-idf.yaml
index f486544afa..2d29656c94 100644
--- a/tests/components/rf_bridge/test.esp32-idf.yaml
+++ b/tests/components/rf_bridge/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/rf_bridge/test.esp8266-ard.yaml b/tests/components/rf_bridge/test.esp8266-ard.yaml
index b516342f3b..5a05efa259 100644
--- a/tests/components/rf_bridge/test.esp8266-ard.yaml
+++ b/tests/components/rf_bridge/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/rf_bridge/test.rp2040-ard.yaml b/tests/components/rf_bridge/test.rp2040-ard.yaml
index b516342f3b..f1df2daf83 100644
--- a/tests/components/rf_bridge/test.rp2040-ard.yaml
+++ b/tests/components/rf_bridge/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/rgb/test.esp32-ard.yaml b/tests/components/rgb/test.esp32-ard.yaml
deleted file mode 100644
index d78ccec952..0000000000
--- a/tests/components/rgb/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin1: GPIO12
-  pin2: GPIO13
-  pin3: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/rgb/test.esp32-c3-ard.yaml b/tests/components/rgb/test.esp32-c3-ard.yaml
deleted file mode 100644
index 1fe4a4bb90..0000000000
--- a/tests/components/rgb/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin1: GPIO2
-  pin2: GPIO3
-  pin3: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/rgbct/test.esp32-ard.yaml b/tests/components/rgbct/test.esp32-ard.yaml
deleted file mode 100644
index 1ecc626e9c..0000000000
--- a/tests/components/rgbct/test.esp32-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin1: GPIO12
-  pin2: GPIO13
-  pin3: GPIO14
-  pin4: GPIO15
-  pin5: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/rgbct/test.esp32-c3-ard.yaml b/tests/components/rgbct/test.esp32-c3-ard.yaml
deleted file mode 100644
index 27a1fbca4d..0000000000
--- a/tests/components/rgbct/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin1: GPIO2
-  pin2: GPIO3
-  pin3: GPIO4
-  pin4: GPIO5
-  pin5: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/rgbw/test.esp32-ard.yaml b/tests/components/rgbw/test.esp32-ard.yaml
deleted file mode 100644
index ea8efd1a34..0000000000
--- a/tests/components/rgbw/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin1: GPIO12
-  pin2: GPIO13
-  pin3: GPIO14
-  pin4: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/rgbw/test.esp32-c3-ard.yaml b/tests/components/rgbw/test.esp32-c3-ard.yaml
deleted file mode 100644
index b44734344e..0000000000
--- a/tests/components/rgbw/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin1: GPIO2
-  pin2: GPIO3
-  pin3: GPIO4
-  pin4: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/rgbww/test.esp32-ard.yaml b/tests/components/rgbww/test.esp32-ard.yaml
deleted file mode 100644
index 1ecc626e9c..0000000000
--- a/tests/components/rgbww/test.esp32-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin1: GPIO12
-  pin2: GPIO13
-  pin3: GPIO14
-  pin4: GPIO15
-  pin5: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/rgbww/test.esp32-c3-ard.yaml b/tests/components/rgbww/test.esp32-c3-ard.yaml
deleted file mode 100644
index 27a1fbca4d..0000000000
--- a/tests/components/rgbww/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  light_platform: ledc
-  pin1: GPIO2
-  pin2: GPIO3
-  pin3: GPIO4
-  pin4: GPIO5
-  pin5: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/rotary_encoder/test.esp32-ard.yaml b/tests/components/rotary_encoder/test.esp32-ard.yaml
deleted file mode 100644
index 48a624aa37..0000000000
--- a/tests/components/rotary_encoder/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  pin_a: GPIO12
-  pin_b: GPIO13
-  pin_reset: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/rotary_encoder/test.esp32-c3-ard.yaml b/tests/components/rotary_encoder/test.esp32-c3-ard.yaml
deleted file mode 100644
index b71a454bdd..0000000000
--- a/tests/components/rotary_encoder/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  pin_a: GPIO2
-  pin_b: GPIO3
-  pin_reset: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/rtttl/test.esp32-ard.yaml b/tests/components/rtttl/test.esp32-ard.yaml
deleted file mode 100644
index 26da1ce1d6..0000000000
--- a/tests/components/rtttl/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  output_platform: ledc
-  pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/rtttl/test.esp32-c3-ard.yaml b/tests/components/rtttl/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7476963591..0000000000
--- a/tests/components/rtttl/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  output_platform: ledc
-  pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/runtime_stats/common.yaml b/tests/components/runtime_stats/common.yaml
new file mode 100644
index 0000000000..b434d1b5a7
--- /dev/null
+++ b/tests/components/runtime_stats/common.yaml
@@ -0,0 +1,2 @@
+# Test runtime_stats component with default configuration
+runtime_stats:
diff --git a/tests/components/ruuvi_ble/test.esp32-ard.yaml b/tests/components/ruuvi_ble/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/ruuvi_ble/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/ruuvi_ble/test.esp32-c3-ard.yaml b/tests/components/ruuvi_ble/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/ruuvi_ble/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/ruuvi_ble/test.esp32-c3-idf.yaml b/tests/components/ruuvi_ble/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/ruuvi_ble/test.esp32-c3-idf.yaml
+++ b/tests/components/ruuvi_ble/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ruuvi_ble/test.esp32-idf.yaml b/tests/components/ruuvi_ble/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/ruuvi_ble/test.esp32-idf.yaml
+++ b/tests/components/ruuvi_ble/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ruuvitag/test.esp32-ard.yaml b/tests/components/ruuvitag/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/ruuvitag/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/ruuvitag/test.esp32-c3-ard.yaml b/tests/components/ruuvitag/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/ruuvitag/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/ruuvitag/test.esp32-c3-idf.yaml b/tests/components/ruuvitag/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/ruuvitag/test.esp32-c3-idf.yaml
+++ b/tests/components/ruuvitag/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ruuvitag/test.esp32-idf.yaml b/tests/components/ruuvitag/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/ruuvitag/test.esp32-idf.yaml
+++ b/tests/components/ruuvitag/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/safe_mode/test-enabled.esp32-ard.yaml b/tests/components/safe_mode/test-enabled.esp32-ard.yaml
deleted file mode 100644
index 97fd63d70e..0000000000
--- a/tests/components/safe_mode/test-enabled.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-enabled.yaml
diff --git a/tests/components/safe_mode/test-enabled.esp32-c3-ard.yaml b/tests/components/safe_mode/test-enabled.esp32-c3-ard.yaml
deleted file mode 100644
index 97fd63d70e..0000000000
--- a/tests/components/safe_mode/test-enabled.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-enabled.yaml
diff --git a/tests/components/scd30/common.yaml b/tests/components/scd30/common.yaml
index 1c45c67af0..f21d8944dc 100644
--- a/tests/components/scd30/common.yaml
+++ b/tests/components/scd30/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_scd30
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: scd30
+    i2c_id: i2c_bus
     co2:
       name: SCD30 CO2
     temperature:
diff --git a/tests/components/scd30/test.esp32-ard.yaml b/tests/components/scd30/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/scd30/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/scd30/test.esp32-c3-ard.yaml b/tests/components/scd30/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/scd30/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/scd30/test.esp32-c3-idf.yaml b/tests/components/scd30/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/scd30/test.esp32-c3-idf.yaml
+++ b/tests/components/scd30/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/scd30/test.esp32-idf.yaml b/tests/components/scd30/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/scd30/test.esp32-idf.yaml
+++ b/tests/components/scd30/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/scd30/test.esp8266-ard.yaml b/tests/components/scd30/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/scd30/test.esp8266-ard.yaml
+++ b/tests/components/scd30/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/scd30/test.rp2040-ard.yaml b/tests/components/scd30/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/scd30/test.rp2040-ard.yaml
+++ b/tests/components/scd30/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/scd4x/common.yaml b/tests/components/scd4x/common.yaml
index dfd35e57de..cedb88977e 100644
--- a/tests/components/scd4x/common.yaml
+++ b/tests/components/scd4x/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_scd4x
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: scd4x
+    i2c_id: i2c_bus
     id: scd40
     co2:
       name: SCD4X CO2
diff --git a/tests/components/scd4x/test.esp32-ard.yaml b/tests/components/scd4x/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/scd4x/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/scd4x/test.esp32-c3-ard.yaml b/tests/components/scd4x/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/scd4x/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/scd4x/test.esp32-c3-idf.yaml b/tests/components/scd4x/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/scd4x/test.esp32-c3-idf.yaml
+++ b/tests/components/scd4x/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/scd4x/test.esp32-idf.yaml b/tests/components/scd4x/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/scd4x/test.esp32-idf.yaml
+++ b/tests/components/scd4x/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/scd4x/test.esp8266-ard.yaml b/tests/components/scd4x/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/scd4x/test.esp8266-ard.yaml
+++ b/tests/components/scd4x/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/scd4x/test.rp2040-ard.yaml b/tests/components/scd4x/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/scd4x/test.rp2040-ard.yaml
+++ b/tests/components/scd4x/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/script/test.esp32-ard.yaml b/tests/components/script/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/script/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/script/test.esp32-c3-ard.yaml b/tests/components/script/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/script/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/canbus/test.esp32-c3-ard.yaml b/tests/components/script/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/canbus/test.esp32-c3-ard.yaml
rename to tests/components/script/test.nrf52-adafruit.yaml
diff --git a/tests/components/captive_portal/test.esp32-c3-ard.yaml b/tests/components/script/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/captive_portal/test.esp32-c3-ard.yaml
rename to tests/components/script/test.nrf52-mcumgr.yaml
diff --git a/tests/components/sdm_meter/common.yaml b/tests/components/sdm_meter/common.yaml
index 60c71a796b..760f134451 100644
--- a/tests/components/sdm_meter/common.yaml
+++ b/tests/components/sdm_meter/common.yaml
@@ -1,11 +1,6 @@
-uart:
-  - id: uart_sdm_meter
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: sdm_meter
+    modbus_id: modbus_bus
     phase_a:
       current:
         name: Phase A Current
diff --git a/tests/components/sdm_meter/test.esp32-ard.yaml b/tests/components/sdm_meter/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/sdm_meter/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/sdm_meter/test.esp32-c3-ard.yaml b/tests/components/sdm_meter/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/sdm_meter/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/sdm_meter/test.esp32-c3-idf.yaml b/tests/components/sdm_meter/test.esp32-c3-idf.yaml
index b516342f3b..6c6e95488f 100644
--- a/tests/components/sdm_meter/test.esp32-c3-idf.yaml
+++ b/tests/components/sdm_meter/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sdm_meter/test.esp32-idf.yaml b/tests/components/sdm_meter/test.esp32-idf.yaml
index f486544afa..b631e16677 100644
--- a/tests/components/sdm_meter/test.esp32-idf.yaml
+++ b/tests/components/sdm_meter/test.esp32-idf.yaml
@@ -1,5 +1,9 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+  flow_control_pin: GPIO13
+
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sdm_meter/test.esp8266-ard.yaml b/tests/components/sdm_meter/test.esp8266-ard.yaml
index b516342f3b..421389ae97 100644
--- a/tests/components/sdm_meter/test.esp8266-ard.yaml
+++ b/tests/components/sdm_meter/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sdm_meter/test.rp2040-ard.yaml b/tests/components/sdm_meter/test.rp2040-ard.yaml
index b516342f3b..d78d84c983 100644
--- a/tests/components/sdm_meter/test.rp2040-ard.yaml
+++ b/tests/components/sdm_meter/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  modbus: !include ../../test_build_components/common/modbus/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/sdp3x/common.yaml b/tests/components/sdp3x/common.yaml
index d3c5491ca5..5d06f8eddb 100644
--- a/tests/components/sdp3x/common.yaml
+++ b/tests/components/sdp3x/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_sdp3x
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: sdp3x
+    i2c_id: i2c_bus
     id: filter_pressure
     name: HVAC Filter Pressure drop
     accuracy_decimals: 3
diff --git a/tests/components/sdp3x/test.esp32-ard.yaml b/tests/components/sdp3x/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/sdp3x/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sdp3x/test.esp32-c3-ard.yaml b/tests/components/sdp3x/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/sdp3x/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sdp3x/test.esp32-c3-idf.yaml b/tests/components/sdp3x/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/sdp3x/test.esp32-c3-idf.yaml
+++ b/tests/components/sdp3x/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sdp3x/test.esp32-idf.yaml b/tests/components/sdp3x/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/sdp3x/test.esp32-idf.yaml
+++ b/tests/components/sdp3x/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sdp3x/test.esp8266-ard.yaml b/tests/components/sdp3x/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/sdp3x/test.esp8266-ard.yaml
+++ b/tests/components/sdp3x/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sdp3x/test.rp2040-ard.yaml b/tests/components/sdp3x/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/sdp3x/test.rp2040-ard.yaml
+++ b/tests/components/sdp3x/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sds011/common.yaml b/tests/components/sds011/common.yaml
index c7574e1d7d..abae0f9bd8 100644
--- a/tests/components/sds011/common.yaml
+++ b/tests/components/sds011/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_sdm_sds011
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 115200
-
 sensor:
   - platform: sds011
     pm_2_5:
diff --git a/tests/components/sds011/test.esp32-ard.yaml b/tests/components/sds011/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/sds011/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/sds011/test.esp32-c3-ard.yaml b/tests/components/sds011/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/sds011/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/sds011/test.esp32-c3-idf.yaml b/tests/components/sds011/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/sds011/test.esp32-c3-idf.yaml
+++ b/tests/components/sds011/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sds011/test.esp32-idf.yaml b/tests/components/sds011/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/sds011/test.esp32-idf.yaml
+++ b/tests/components/sds011/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sds011/test.esp8266-ard.yaml b/tests/components/sds011/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/sds011/test.esp8266-ard.yaml
+++ b/tests/components/sds011/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sds011/test.rp2040-ard.yaml b/tests/components/sds011/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/sds011/test.rp2040-ard.yaml
+++ b/tests/components/sds011/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/seeed_mr24hpc1/common.yaml b/tests/components/seeed_mr24hpc1/common.yaml
index 38692b3e5e..3a52888c5c 100644
--- a/tests/components/seeed_mr24hpc1/common.yaml
+++ b/tests/components/seeed_mr24hpc1/common.yaml
@@ -1,14 +1,5 @@
-uart:
-  - id: seeed_mr24hpc1_uart
-    tx_pin: ${uart_tx_pin}
-    rx_pin: ${uart_rx_pin}
-    baud_rate: 115200
-    parity: NONE
-    stop_bits: 1
-
 seeed_mr24hpc1:
   id: my_seeed_mr24hpc1
-  uart_id: seeed_mr24hpc1_uart
 
 sensor:
   - platform: seeed_mr24hpc1
diff --git a/tests/components/seeed_mr24hpc1/test.esp32-c3-ard.yaml b/tests/components/seeed_mr24hpc1/test.esp32-c3-ard.yaml
deleted file mode 100644
index 4fb884abf4..0000000000
--- a/tests/components/seeed_mr24hpc1/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  uart_tx_pin: GPIO5
-  uart_rx_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/seeed_mr24hpc1/test.esp32-c3-idf.yaml b/tests/components/seeed_mr24hpc1/test.esp32-c3-idf.yaml
index 4fb884abf4..a19013bf54 100644
--- a/tests/components/seeed_mr24hpc1/test.esp32-c3-idf.yaml
+++ b/tests/components/seeed_mr24hpc1/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  uart_tx_pin: GPIO5
-  uart_rx_pin: GPIO4
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/seeed_mr60bha2/common.yaml b/tests/components/seeed_mr60bha2/common.yaml
index 9eb0c8d527..7d1a02b3ea 100644
--- a/tests/components/seeed_mr60bha2/common.yaml
+++ b/tests/components/seeed_mr60bha2/common.yaml
@@ -1,11 +1,3 @@
-uart:
-  - id: seeed_mr60fda2_uart
-    tx_pin: ${uart_tx_pin}
-    rx_pin: ${uart_rx_pin}
-    baud_rate: 115200
-    parity: NONE
-    stop_bits: 1
-
 seeed_mr60bha2:
   id: my_seeed_mr60bha2
 
diff --git a/tests/components/seeed_mr60bha2/test.esp32-c3-ard.yaml b/tests/components/seeed_mr60bha2/test.esp32-c3-ard.yaml
deleted file mode 100644
index 4fb884abf4..0000000000
--- a/tests/components/seeed_mr60bha2/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  uart_tx_pin: GPIO5
-  uart_rx_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/seeed_mr60bha2/test.esp32-c3-idf.yaml b/tests/components/seeed_mr60bha2/test.esp32-c3-idf.yaml
index 4fb884abf4..fea89f5768 100644
--- a/tests/components/seeed_mr60bha2/test.esp32-c3-idf.yaml
+++ b/tests/components/seeed_mr60bha2/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  uart_tx_pin: GPIO5
-  uart_rx_pin: GPIO4
+packages:
+  uart_115200: !include ../../test_build_components/common/uart_115200/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/seeed_mr60fda2/common.yaml b/tests/components/seeed_mr60fda2/common.yaml
index 55a7cc1ab3..e0e4614481 100644
--- a/tests/components/seeed_mr60fda2/common.yaml
+++ b/tests/components/seeed_mr60fda2/common.yaml
@@ -1,14 +1,5 @@
-uart:
-  - id: seeed_mr60fda2_uart
-    tx_pin: ${uart_tx_pin}
-    rx_pin: ${uart_rx_pin}
-    baud_rate: 115200
-    parity: NONE
-    stop_bits: 1
-
 seeed_mr60fda2:
   id: my_seeed_mr60fda2
-  uart_id: seeed_mr60fda2_uart
 
 binary_sensor:
   - platform: seeed_mr60fda2
diff --git a/tests/components/seeed_mr60fda2/test.esp32-c3-ard.yaml b/tests/components/seeed_mr60fda2/test.esp32-c3-ard.yaml
deleted file mode 100644
index 4fb884abf4..0000000000
--- a/tests/components/seeed_mr60fda2/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  uart_tx_pin: GPIO5
-  uart_rx_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/seeed_mr60fda2/test.esp32-c3-idf.yaml b/tests/components/seeed_mr60fda2/test.esp32-c3-idf.yaml
index 4fb884abf4..fea89f5768 100644
--- a/tests/components/seeed_mr60fda2/test.esp32-c3-idf.yaml
+++ b/tests/components/seeed_mr60fda2/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  uart_tx_pin: GPIO5
-  uart_rx_pin: GPIO4
+packages:
+  uart_115200: !include ../../test_build_components/common/uart_115200/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/selec_meter/common.yaml b/tests/components/selec_meter/common.yaml
index f2714ce828..2febbee540 100644
--- a/tests/components/selec_meter/common.yaml
+++ b/tests/components/selec_meter/common.yaml
@@ -1,11 +1,6 @@
-uart:
-  - id: uart_selec_meter
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: selec_meter
+    modbus_id: modbus_bus
     total_active_energy:
       name: SelecEM2M Total Active Energy
     import_active_energy:
diff --git a/tests/components/selec_meter/test.esp32-ard.yaml b/tests/components/selec_meter/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/selec_meter/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/selec_meter/test.esp32-c3-ard.yaml b/tests/components/selec_meter/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/selec_meter/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/selec_meter/test.esp32-c3-idf.yaml b/tests/components/selec_meter/test.esp32-c3-idf.yaml
index b516342f3b..beb90e1471 100644
--- a/tests/components/selec_meter/test.esp32-c3-idf.yaml
+++ b/tests/components/selec_meter/test.esp32-c3-idf.yaml
@@ -1,5 +1,7 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  flow_control_pin: GPIO10
+
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/selec_meter/test.esp32-idf.yaml b/tests/components/selec_meter/test.esp32-idf.yaml
index f486544afa..4df9f5863e 100644
--- a/tests/components/selec_meter/test.esp32-idf.yaml
+++ b/tests/components/selec_meter/test.esp32-idf.yaml
@@ -1,5 +1,9 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+  flow_control_pin: GPIO26
+
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/selec_meter/test.esp8266-ard.yaml b/tests/components/selec_meter/test.esp8266-ard.yaml
index b516342f3b..6daa08c22b 100644
--- a/tests/components/selec_meter/test.esp8266-ard.yaml
+++ b/tests/components/selec_meter/test.esp8266-ard.yaml
@@ -1,5 +1,9 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+  flow_control_pin: GPIO15
+
+packages:
+  modbus: !include ../../test_build_components/common/modbus/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/selec_meter/test.rp2040-ard.yaml b/tests/components/selec_meter/test.rp2040-ard.yaml
index b516342f3b..a65500ccc3 100644
--- a/tests/components/selec_meter/test.rp2040-ard.yaml
+++ b/tests/components/selec_meter/test.rp2040-ard.yaml
@@ -1,5 +1,9 @@
 substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
+  flow_control_pin: GPIO6
+
+packages:
+  modbus: !include ../../test_build_components/common/modbus/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sen0321/common.yaml b/tests/components/sen0321/common.yaml
index 8b9fdff4a1..9228c7b820 100644
--- a/tests/components/sen0321/common.yaml
+++ b/tests/components/sen0321/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_sen0321
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: sen0321
+    i2c_id: i2c_bus
     name: Workshop Ozone Sensor
     id: sen0321_ozone
     update_interval: 10s
diff --git a/tests/components/sen0321/test.esp32-ard.yaml b/tests/components/sen0321/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/sen0321/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sen0321/test.esp32-c3-ard.yaml b/tests/components/sen0321/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/sen0321/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sen0321/test.esp32-c3-idf.yaml b/tests/components/sen0321/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/sen0321/test.esp32-c3-idf.yaml
+++ b/tests/components/sen0321/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sen0321/test.esp32-idf.yaml b/tests/components/sen0321/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/sen0321/test.esp32-idf.yaml
+++ b/tests/components/sen0321/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sen0321/test.esp8266-ard.yaml b/tests/components/sen0321/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/sen0321/test.esp8266-ard.yaml
+++ b/tests/components/sen0321/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sen0321/test.rp2040-ard.yaml b/tests/components/sen0321/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/sen0321/test.rp2040-ard.yaml
+++ b/tests/components/sen0321/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sen21231/common.yaml b/tests/components/sen21231/common.yaml
index 6fa1d04aa2..bfe0f9d7f8 100644
--- a/tests/components/sen21231/common.yaml
+++ b/tests/components/sen21231/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_sen21231
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: sen21231
+    i2c_id: i2c_bus
     id: sen21231_sensor1
     name: Person Sensor
diff --git a/tests/components/sen21231/test.esp32-ard.yaml b/tests/components/sen21231/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/sen21231/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sen21231/test.esp32-c3-ard.yaml b/tests/components/sen21231/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/sen21231/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sen21231/test.esp32-c3-idf.yaml b/tests/components/sen21231/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/sen21231/test.esp32-c3-idf.yaml
+++ b/tests/components/sen21231/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sen21231/test.esp32-idf.yaml b/tests/components/sen21231/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/sen21231/test.esp32-idf.yaml
+++ b/tests/components/sen21231/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sen21231/test.esp8266-ard.yaml b/tests/components/sen21231/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/sen21231/test.esp8266-ard.yaml
+++ b/tests/components/sen21231/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sen21231/test.rp2040-ard.yaml b/tests/components/sen21231/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/sen21231/test.rp2040-ard.yaml
+++ b/tests/components/sen21231/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sen5x/common.yaml b/tests/components/sen5x/common.yaml
index 9adf268048..a4462a16ea 100644
--- a/tests/components/sen5x/common.yaml
+++ b/tests/components/sen5x/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_sen5x
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: sen5x
+    i2c_id: i2c_bus
     id: sen54
     temperature:
       name: Temperature
diff --git a/tests/components/sen5x/test.esp32-ard.yaml b/tests/components/sen5x/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/sen5x/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sen5x/test.esp32-c3-ard.yaml b/tests/components/sen5x/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/sen5x/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sen5x/test.esp32-c3-idf.yaml b/tests/components/sen5x/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/sen5x/test.esp32-c3-idf.yaml
+++ b/tests/components/sen5x/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sen5x/test.esp32-idf.yaml b/tests/components/sen5x/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/sen5x/test.esp32-idf.yaml
+++ b/tests/components/sen5x/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sen5x/test.esp8266-ard.yaml b/tests/components/sen5x/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/sen5x/test.esp8266-ard.yaml
+++ b/tests/components/sen5x/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sen5x/test.rp2040-ard.yaml b/tests/components/sen5x/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/sen5x/test.rp2040-ard.yaml
+++ b/tests/components/sen5x/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/senseair/common.yaml b/tests/components/senseair/common.yaml
index 23a933affe..c8066896d0 100644
--- a/tests/components/senseair/common.yaml
+++ b/tests/components/senseair/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_senseair
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: senseair
     id: senseair0
diff --git a/tests/components/senseair/test.esp32-ard.yaml b/tests/components/senseair/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/senseair/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/senseair/test.esp32-c3-ard.yaml b/tests/components/senseair/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/senseair/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/senseair/test.esp32-c3-idf.yaml b/tests/components/senseair/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/senseair/test.esp32-c3-idf.yaml
+++ b/tests/components/senseair/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/senseair/test.esp32-idf.yaml b/tests/components/senseair/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/senseair/test.esp32-idf.yaml
+++ b/tests/components/senseair/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/senseair/test.esp8266-ard.yaml b/tests/components/senseair/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/senseair/test.esp8266-ard.yaml
+++ b/tests/components/senseair/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/senseair/test.rp2040-ard.yaml b/tests/components/senseair/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/senseair/test.rp2040-ard.yaml
+++ b/tests/components/senseair/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/servo/test.esp32-ard.yaml b/tests/components/servo/test.esp32-ard.yaml
deleted file mode 100644
index 26da1ce1d6..0000000000
--- a/tests/components/servo/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  output_platform: ledc
-  pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/servo/test.esp32-c3-ard.yaml b/tests/components/servo/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7476963591..0000000000
--- a/tests/components/servo/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  output_platform: ledc
-  pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sfa30/common.yaml b/tests/components/sfa30/common.yaml
index e3b38aa7fb..3cd2483abc 100644
--- a/tests/components/sfa30/common.yaml
+++ b/tests/components/sfa30/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_sfa30
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: sfa30
+    i2c_id: i2c_bus
     formaldehyde:
       name: SFA30 formaldehyde
     temperature:
diff --git a/tests/components/sfa30/test.esp32-ard.yaml b/tests/components/sfa30/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/sfa30/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sfa30/test.esp32-c3-ard.yaml b/tests/components/sfa30/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/sfa30/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sfa30/test.esp32-c3-idf.yaml b/tests/components/sfa30/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/sfa30/test.esp32-c3-idf.yaml
+++ b/tests/components/sfa30/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sfa30/test.esp32-idf.yaml b/tests/components/sfa30/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/sfa30/test.esp32-idf.yaml
+++ b/tests/components/sfa30/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sfa30/test.esp8266-ard.yaml b/tests/components/sfa30/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/sfa30/test.esp8266-ard.yaml
+++ b/tests/components/sfa30/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sfa30/test.rp2040-ard.yaml b/tests/components/sfa30/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/sfa30/test.rp2040-ard.yaml
+++ b/tests/components/sfa30/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sgp30/common.yaml b/tests/components/sgp30/common.yaml
index 1db5bc67d1..9d4ed46615 100644
--- a/tests/components/sgp30/common.yaml
+++ b/tests/components/sgp30/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_sgp30
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: sgp30
+    i2c_id: i2c_bus
     eco2:
       name: Workshop eCO2
       accuracy_decimals: 1
diff --git a/tests/components/sgp30/test.esp32-ard.yaml b/tests/components/sgp30/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/sgp30/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sgp30/test.esp32-c3-ard.yaml b/tests/components/sgp30/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/sgp30/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sgp30/test.esp32-c3-idf.yaml b/tests/components/sgp30/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/sgp30/test.esp32-c3-idf.yaml
+++ b/tests/components/sgp30/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sgp30/test.esp32-idf.yaml b/tests/components/sgp30/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/sgp30/test.esp32-idf.yaml
+++ b/tests/components/sgp30/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sgp30/test.esp8266-ard.yaml b/tests/components/sgp30/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/sgp30/test.esp8266-ard.yaml
+++ b/tests/components/sgp30/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sgp30/test.rp2040-ard.yaml b/tests/components/sgp30/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/sgp30/test.rp2040-ard.yaml
+++ b/tests/components/sgp30/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sgp4x/common.yaml b/tests/components/sgp4x/common.yaml
index adb678d542..4edda8fd1b 100644
--- a/tests/components/sgp4x/common.yaml
+++ b/tests/components/sgp4x/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_sgp4x
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: sgp4x
+    i2c_id: i2c_bus
     voc:
       name: VOC Index
       id: sgp40_voc_index
diff --git a/tests/components/sgp4x/test.esp32-ard.yaml b/tests/components/sgp4x/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/sgp4x/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sgp4x/test.esp32-c3-ard.yaml b/tests/components/sgp4x/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/sgp4x/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sgp4x/test.esp32-c3-idf.yaml b/tests/components/sgp4x/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/sgp4x/test.esp32-c3-idf.yaml
+++ b/tests/components/sgp4x/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sgp4x/test.esp32-idf.yaml b/tests/components/sgp4x/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/sgp4x/test.esp32-idf.yaml
+++ b/tests/components/sgp4x/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sgp4x/test.esp8266-ard.yaml b/tests/components/sgp4x/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/sgp4x/test.esp8266-ard.yaml
+++ b/tests/components/sgp4x/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sgp4x/test.rp2040-ard.yaml b/tests/components/sgp4x/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/sgp4x/test.rp2040-ard.yaml
+++ b/tests/components/sgp4x/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sha256/common.yaml b/tests/components/sha256/common.yaml
new file mode 100644
index 0000000000..fa884c1958
--- /dev/null
+++ b/tests/components/sha256/common.yaml
@@ -0,0 +1,32 @@
+esphome:
+  on_boot:
+    - lambda: |-
+        // Test SHA256 functionality
+        #ifdef USE_SHA256
+        using esphome::sha256::SHA256;
+        SHA256 hasher;
+        hasher.init();
+
+        // Test with "Hello World" - known SHA256
+        const char* test_string = "Hello World";
+        hasher.add(test_string, strlen(test_string));
+        hasher.calculate();
+
+        char hex_output[65];
+        hasher.get_hex(hex_output);
+        hex_output[64] = '\0';
+
+        ESP_LOGD("SHA256", "SHA256('Hello World') = %s", hex_output);
+
+        // Expected: a591a6d40bf420404a011733cfb7b190d62c65bf0bcda32b57b277d9ad9f146e
+        const char* expected = "a591a6d40bf420404a011733cfb7b190d62c65bf0bcda32b57b277d9ad9f146e";
+        if (strcmp(hex_output, expected) == 0) {
+          ESP_LOGI("SHA256", "Test PASSED");
+        } else {
+          ESP_LOGE("SHA256", "Test FAILED. Expected %s", expected);
+        }
+        #else
+        ESP_LOGW("SHA256", "SHA256 not available on this platform");
+        #endif
+
+sha256:
diff --git a/tests/components/color/test.esp32-ard.yaml b/tests/components/sha256/test.bk72xx-ard.yaml
similarity index 100%
rename from tests/components/color/test.esp32-ard.yaml
rename to tests/components/sha256/test.bk72xx-ard.yaml
diff --git a/tests/components/color/test.esp32-c3-ard.yaml b/tests/components/sha256/test.esp32-idf.yaml
similarity index 100%
rename from tests/components/color/test.esp32-c3-ard.yaml
rename to tests/components/sha256/test.esp32-idf.yaml
diff --git a/tests/components/combination/test.esp32-ard.yaml b/tests/components/sha256/test.esp8266-ard.yaml
similarity index 100%
rename from tests/components/combination/test.esp32-ard.yaml
rename to tests/components/sha256/test.esp8266-ard.yaml
diff --git a/tests/components/combination/test.esp32-c3-ard.yaml b/tests/components/sha256/test.host.yaml
similarity index 100%
rename from tests/components/combination/test.esp32-c3-ard.yaml
rename to tests/components/sha256/test.host.yaml
diff --git a/tests/components/dallas_temp/test.esp32-ard.yaml b/tests/components/sha256/test.rp2040-ard.yaml
similarity index 100%
rename from tests/components/dallas_temp/test.esp32-ard.yaml
rename to tests/components/sha256/test.rp2040-ard.yaml
diff --git a/tests/components/shelly_dimmer/common.yaml b/tests/components/shelly_dimmer/common.yaml
index 3acd0260d5..ba36fa995d 100644
--- a/tests/components/shelly_dimmer/common.yaml
+++ b/tests/components/shelly_dimmer/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_shelly_dimmer
-    tx_pin: 4
-    rx_pin: 5
-    baud_rate: 9600
-
 light:
   - platform: shelly_dimmer
     name: Shelly Dimmer Light
diff --git a/tests/components/shelly_dimmer/test.esp8266-ard.yaml b/tests/components/shelly_dimmer/test.esp8266-ard.yaml
index dade44d145..80a2cb2fc0 100644
--- a/tests/components/shelly_dimmer/test.esp8266-ard.yaml
+++ b/tests/components/shelly_dimmer/test.esp8266-ard.yaml
@@ -1 +1,4 @@
+packages:
+  uart_115200: !include ../../test_build_components/common/uart_115200/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/sht3xd/common.yaml b/tests/components/sht3xd/common.yaml
index 2426ebfbb9..02ce521747 100644
--- a/tests/components/sht3xd/common.yaml
+++ b/tests/components/sht3xd/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_sht3xd
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: sht3xd
+    i2c_id: i2c_bus
     temperature:
       name: SHT3XD Temperature
     humidity:
diff --git a/tests/components/sht3xd/test.esp32-ard.yaml b/tests/components/sht3xd/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/sht3xd/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sht3xd/test.esp32-c3-ard.yaml b/tests/components/sht3xd/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/sht3xd/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sht3xd/test.esp32-c3-idf.yaml b/tests/components/sht3xd/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/sht3xd/test.esp32-c3-idf.yaml
+++ b/tests/components/sht3xd/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sht3xd/test.esp32-idf.yaml b/tests/components/sht3xd/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/sht3xd/test.esp32-idf.yaml
+++ b/tests/components/sht3xd/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sht3xd/test.esp8266-ard.yaml b/tests/components/sht3xd/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/sht3xd/test.esp8266-ard.yaml
+++ b/tests/components/sht3xd/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sht3xd/test.rp2040-ard.yaml b/tests/components/sht3xd/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/sht3xd/test.rp2040-ard.yaml
+++ b/tests/components/sht3xd/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sht4x/common.yaml b/tests/components/sht4x/common.yaml
index 703a8fa32b..50d5ad8ca4 100644
--- a/tests/components/sht4x/common.yaml
+++ b/tests/components/sht4x/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_sht4x
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: sht4x
+    i2c_id: i2c_bus
     temperature:
       name: SHT4X Temperature
     humidity:
diff --git a/tests/components/sht4x/test.esp32-ard.yaml b/tests/components/sht4x/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/sht4x/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sht4x/test.esp32-c3-ard.yaml b/tests/components/sht4x/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/sht4x/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sht4x/test.esp32-c3-idf.yaml b/tests/components/sht4x/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/sht4x/test.esp32-c3-idf.yaml
+++ b/tests/components/sht4x/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sht4x/test.esp32-idf.yaml b/tests/components/sht4x/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/sht4x/test.esp32-idf.yaml
+++ b/tests/components/sht4x/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sht4x/test.esp8266-ard.yaml b/tests/components/sht4x/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/sht4x/test.esp8266-ard.yaml
+++ b/tests/components/sht4x/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sht4x/test.rp2040-ard.yaml b/tests/components/sht4x/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/sht4x/test.rp2040-ard.yaml
+++ b/tests/components/sht4x/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/shtcx/common.yaml b/tests/components/shtcx/common.yaml
index 0211319124..a326a1b4d6 100644
--- a/tests/components/shtcx/common.yaml
+++ b/tests/components/shtcx/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_shtcx
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: shtcx
+    i2c_id: i2c_bus
     temperature:
       name: SHTCX Temperature
     humidity:
diff --git a/tests/components/shtcx/test.esp32-ard.yaml b/tests/components/shtcx/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/shtcx/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/shtcx/test.esp32-c3-ard.yaml b/tests/components/shtcx/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/shtcx/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/shtcx/test.esp32-c3-idf.yaml b/tests/components/shtcx/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/shtcx/test.esp32-c3-idf.yaml
+++ b/tests/components/shtcx/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/shtcx/test.esp32-idf.yaml b/tests/components/shtcx/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/shtcx/test.esp32-idf.yaml
+++ b/tests/components/shtcx/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/shtcx/test.esp8266-ard.yaml b/tests/components/shtcx/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/shtcx/test.esp8266-ard.yaml
+++ b/tests/components/shtcx/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/shtcx/test.rp2040-ard.yaml b/tests/components/shtcx/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/shtcx/test.rp2040-ard.yaml
+++ b/tests/components/shtcx/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/shutdown/test.esp32-ard.yaml b/tests/components/shutdown/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/shutdown/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/shutdown/test.esp32-c3-ard.yaml b/tests/components/shutdown/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/shutdown/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/sigma_delta_output/test.esp32-ard.yaml b/tests/components/sigma_delta_output/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/sigma_delta_output/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/sigma_delta_output/test.esp32-c3-ard.yaml b/tests/components/sigma_delta_output/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/sigma_delta_output/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/sim800l/common.yaml b/tests/components/sim800l/common.yaml
index 1b4e2e1af6..503637b35f 100644
--- a/tests/components/sim800l/common.yaml
+++ b/tests/components/sim800l/common.yaml
@@ -11,12 +11,6 @@ esphome:
       - sim800l.send_ussd:
           ussd: test_ussd
 
-uart:
-  - id: uart_sim800l
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sim800l:
   on_sms_received:
     - lambda: |-
diff --git a/tests/components/sim800l/test.esp32-ard.yaml b/tests/components/sim800l/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/sim800l/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/sim800l/test.esp32-c3-ard.yaml b/tests/components/sim800l/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/sim800l/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/sim800l/test.esp32-c3-idf.yaml b/tests/components/sim800l/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/sim800l/test.esp32-c3-idf.yaml
+++ b/tests/components/sim800l/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sim800l/test.esp32-idf.yaml b/tests/components/sim800l/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/sim800l/test.esp32-idf.yaml
+++ b/tests/components/sim800l/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sim800l/test.esp8266-ard.yaml b/tests/components/sim800l/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/sim800l/test.esp8266-ard.yaml
+++ b/tests/components/sim800l/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sim800l/test.rp2040-ard.yaml b/tests/components/sim800l/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/sim800l/test.rp2040-ard.yaml
+++ b/tests/components/sim800l/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/slow_pwm/test.esp32-ard.yaml b/tests/components/slow_pwm/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/slow_pwm/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/slow_pwm/test.esp32-c3-ard.yaml b/tests/components/slow_pwm/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/slow_pwm/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/sm16716/test.esp32-ard.yaml b/tests/components/sm16716/test.esp32-ard.yaml
deleted file mode 100644
index d295973e3f..0000000000
--- a/tests/components/sm16716/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sm16716/test.esp32-c3-ard.yaml b/tests/components/sm16716/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7808481215..0000000000
--- a/tests/components/sm16716/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sm16716/test.esp32-idf.yaml b/tests/components/sm16716/test.esp32-idf.yaml
index d295973e3f..a4ecdb6c49 100644
--- a/tests/components/sm16716/test.esp32-idf.yaml
+++ b/tests/components/sm16716/test.esp32-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
+  clock_pin: GPIO4
+  data_pin: GPIO5
 
 <<: !include common.yaml
diff --git a/tests/components/sm16716/test.esp8266-ard.yaml b/tests/components/sm16716/test.esp8266-ard.yaml
index 7808481215..7c7f1e1a11 100644
--- a/tests/components/sm16716/test.esp8266-ard.yaml
+++ b/tests/components/sm16716/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
+  clock_pin: GPIO0
+  data_pin: GPIO2
 
 <<: !include common.yaml
diff --git a/tests/components/sm2135/test.esp32-ard.yaml b/tests/components/sm2135/test.esp32-ard.yaml
deleted file mode 100644
index d295973e3f..0000000000
--- a/tests/components/sm2135/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sm2135/test.esp32-c3-ard.yaml b/tests/components/sm2135/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7808481215..0000000000
--- a/tests/components/sm2135/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sm2135/test.esp32-idf.yaml b/tests/components/sm2135/test.esp32-idf.yaml
index d295973e3f..a4ecdb6c49 100644
--- a/tests/components/sm2135/test.esp32-idf.yaml
+++ b/tests/components/sm2135/test.esp32-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
+  clock_pin: GPIO4
+  data_pin: GPIO5
 
 <<: !include common.yaml
diff --git a/tests/components/sm2135/test.esp8266-ard.yaml b/tests/components/sm2135/test.esp8266-ard.yaml
index 7808481215..7c7f1e1a11 100644
--- a/tests/components/sm2135/test.esp8266-ard.yaml
+++ b/tests/components/sm2135/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
+  clock_pin: GPIO0
+  data_pin: GPIO2
 
 <<: !include common.yaml
diff --git a/tests/components/sm2235/test.esp32-ard.yaml b/tests/components/sm2235/test.esp32-ard.yaml
deleted file mode 100644
index d295973e3f..0000000000
--- a/tests/components/sm2235/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sm2235/test.esp32-c3-ard.yaml b/tests/components/sm2235/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7808481215..0000000000
--- a/tests/components/sm2235/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sm2235/test.esp32-idf.yaml b/tests/components/sm2235/test.esp32-idf.yaml
index d295973e3f..a4ecdb6c49 100644
--- a/tests/components/sm2235/test.esp32-idf.yaml
+++ b/tests/components/sm2235/test.esp32-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
+  clock_pin: GPIO4
+  data_pin: GPIO5
 
 <<: !include common.yaml
diff --git a/tests/components/sm2235/test.esp8266-ard.yaml b/tests/components/sm2235/test.esp8266-ard.yaml
index 7808481215..7c7f1e1a11 100644
--- a/tests/components/sm2235/test.esp8266-ard.yaml
+++ b/tests/components/sm2235/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
+  clock_pin: GPIO0
+  data_pin: GPIO2
 
 <<: !include common.yaml
diff --git a/tests/components/sm2335/test.esp32-ard.yaml b/tests/components/sm2335/test.esp32-ard.yaml
deleted file mode 100644
index d295973e3f..0000000000
--- a/tests/components/sm2335/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sm2335/test.esp32-c3-ard.yaml b/tests/components/sm2335/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7808481215..0000000000
--- a/tests/components/sm2335/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sm2335/test.esp32-idf.yaml b/tests/components/sm2335/test.esp32-idf.yaml
index d295973e3f..a4ecdb6c49 100644
--- a/tests/components/sm2335/test.esp32-idf.yaml
+++ b/tests/components/sm2335/test.esp32-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
+  clock_pin: GPIO4
+  data_pin: GPIO5
 
 <<: !include common.yaml
diff --git a/tests/components/sm2335/test.esp8266-ard.yaml b/tests/components/sm2335/test.esp8266-ard.yaml
index 7808481215..7c7f1e1a11 100644
--- a/tests/components/sm2335/test.esp8266-ard.yaml
+++ b/tests/components/sm2335/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
+  clock_pin: GPIO0
+  data_pin: GPIO2
 
 <<: !include common.yaml
diff --git a/tests/components/sm300d2/common.yaml b/tests/components/sm300d2/common.yaml
index a231b63816..729f243a65 100644
--- a/tests/components/sm300d2/common.yaml
+++ b/tests/components/sm300d2/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_sm300d2
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: sm300d2
     co2:
diff --git a/tests/components/sm300d2/test.esp32-ard.yaml b/tests/components/sm300d2/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/sm300d2/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/sm300d2/test.esp32-c3-ard.yaml b/tests/components/sm300d2/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/sm300d2/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/sm300d2/test.esp32-c3-idf.yaml b/tests/components/sm300d2/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/sm300d2/test.esp32-c3-idf.yaml
+++ b/tests/components/sm300d2/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sm300d2/test.esp32-idf.yaml b/tests/components/sm300d2/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/sm300d2/test.esp32-idf.yaml
+++ b/tests/components/sm300d2/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sm300d2/test.esp8266-ard.yaml b/tests/components/sm300d2/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/sm300d2/test.esp8266-ard.yaml
+++ b/tests/components/sm300d2/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sm300d2/test.rp2040-ard.yaml b/tests/components/sm300d2/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/sm300d2/test.rp2040-ard.yaml
+++ b/tests/components/sm300d2/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/sml/common.yaml b/tests/components/sml/common.yaml
index a50d25eeee..b1bd5949b6 100644
--- a/tests/components/sml/common.yaml
+++ b/tests/components/sml/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_sml
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sml:
   id: mysml
   on_data:
diff --git a/tests/components/sml/test.esp32-ard.yaml b/tests/components/sml/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/sml/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/sml/test.esp32-c3-ard.yaml b/tests/components/sml/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/sml/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/sml/test.esp32-c3-idf.yaml b/tests/components/sml/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/sml/test.esp32-c3-idf.yaml
+++ b/tests/components/sml/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sml/test.esp32-idf.yaml b/tests/components/sml/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/sml/test.esp32-idf.yaml
+++ b/tests/components/sml/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sml/test.esp8266-ard.yaml b/tests/components/sml/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/sml/test.esp8266-ard.yaml
+++ b/tests/components/sml/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sml/test.rp2040-ard.yaml b/tests/components/sml/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/sml/test.rp2040-ard.yaml
+++ b/tests/components/sml/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/smt100/common.yaml b/tests/components/smt100/common.yaml
index b12d7198fd..86c5636b94 100644
--- a/tests/components/smt100/common.yaml
+++ b/tests/components/smt100/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_smt100
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sensor:
   - platform: smt100
     counts:
diff --git a/tests/components/smt100/test.esp32-ard.yaml b/tests/components/smt100/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/smt100/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/smt100/test.esp32-c3-ard.yaml b/tests/components/smt100/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/smt100/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/smt100/test.esp32-c3-idf.yaml b/tests/components/smt100/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/smt100/test.esp32-c3-idf.yaml
+++ b/tests/components/smt100/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/smt100/test.esp32-idf.yaml b/tests/components/smt100/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/smt100/test.esp32-idf.yaml
+++ b/tests/components/smt100/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/smt100/test.esp8266-ard.yaml b/tests/components/smt100/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/smt100/test.esp8266-ard.yaml
+++ b/tests/components/smt100/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/smt100/test.rp2040-ard.yaml b/tests/components/smt100/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/smt100/test.rp2040-ard.yaml
+++ b/tests/components/smt100/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/sn74hc165/test.esp32-ard.yaml b/tests/components/sn74hc165/test.esp32-ard.yaml
deleted file mode 100644
index 27f963312f..0000000000
--- a/tests/components/sn74hc165/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clock_pin: GPIO13
-  data_pin: GPIO14
-  load_pin: GPIO15
-  clock_inhibit_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/sn74hc165/test.esp32-c3-ard.yaml b/tests/components/sn74hc165/test.esp32-c3-ard.yaml
deleted file mode 100644
index 0a3db917b7..0000000000
--- a/tests/components/sn74hc165/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clock_pin: GPIO3
-  data_pin: GPIO4
-  load_pin: GPIO5
-  clock_inhibit_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/sn74hc165/test.esp32-idf.yaml b/tests/components/sn74hc165/test.esp32-idf.yaml
index 27f963312f..8a997e16e1 100644
--- a/tests/components/sn74hc165/test.esp32-idf.yaml
+++ b/tests/components/sn74hc165/test.esp32-idf.yaml
@@ -2,6 +2,6 @@ substitutions:
   clock_pin: GPIO13
   data_pin: GPIO14
   load_pin: GPIO15
-  clock_inhibit_pin: GPIO16
+  clock_inhibit_pin: GPIO4
 
 <<: !include common.yaml
diff --git a/tests/components/sn74hc165/test.esp8266-ard.yaml b/tests/components/sn74hc165/test.esp8266-ard.yaml
index 27f963312f..00ffc10a9e 100644
--- a/tests/components/sn74hc165/test.esp8266-ard.yaml
+++ b/tests/components/sn74hc165/test.esp8266-ard.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  clock_pin: GPIO13
-  data_pin: GPIO14
+  clock_pin: GPIO0
+  data_pin: GPIO2
   load_pin: GPIO15
   clock_inhibit_pin: GPIO16
 
diff --git a/tests/components/sn74hc595/common.yaml b/tests/components/sn74hc595/common.yaml
index fc297909f5..3892c0564b 100644
--- a/tests/components/sn74hc595/common.yaml
+++ b/tests/components/sn74hc595/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_sn74hc595
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 sn74hc595:
   - id: sn74hc595_hub
     clock_pin: ${clock_pin}
diff --git a/tests/components/sn74hc595/test.esp32-ard.yaml b/tests/components/sn74hc595/test.esp32-ard.yaml
deleted file mode 100644
index a4bab64862..0000000000
--- a/tests/components/sn74hc595/test.esp32-ard.yaml
+++ /dev/null
@@ -1,12 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO18
-  clock_pin: GPIO15
-  data_pin: GPIO14
-  latch_pin1: GPIO21
-  oe_pin1: GPIO22
-  latch_pin2: GPIO23
-  oe_pin2: GPIO25
-
-<<: !include common.yaml
diff --git a/tests/components/sn74hc595/test.esp32-c3-ard.yaml b/tests/components/sn74hc595/test.esp32-c3-ard.yaml
deleted file mode 100644
index 14c928be88..0000000000
--- a/tests/components/sn74hc595/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,12 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO8
-  clock_pin: GPIO5
-  data_pin: GPIO4
-  latch_pin1: GPIO1
-  oe_pin1: GPIO2
-  latch_pin2: GPIO3
-  oe_pin2: GPIO9
-
-<<: !include common.yaml
diff --git a/tests/components/sn74hc595/test.esp32-c3-idf.yaml b/tests/components/sn74hc595/test.esp32-c3-idf.yaml
index 14c928be88..74b5e855fa 100644
--- a/tests/components/sn74hc595/test.esp32-c3-idf.yaml
+++ b/tests/components/sn74hc595/test.esp32-c3-idf.yaml
@@ -1,9 +1,9 @@
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
+
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO8
-  clock_pin: GPIO5
-  data_pin: GPIO4
+  clock_pin: GPIO7
+  data_pin: GPIO10
   latch_pin1: GPIO1
   oe_pin1: GPIO2
   latch_pin2: GPIO3
diff --git a/tests/components/sn74hc595/test.esp32-idf.yaml b/tests/components/sn74hc595/test.esp32-idf.yaml
index a4bab64862..4b47d2aeaa 100644
--- a/tests/components/sn74hc595/test.esp32-idf.yaml
+++ b/tests/components/sn74hc595/test.esp32-idf.yaml
@@ -1,12 +1,12 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO18
   clock_pin: GPIO15
   data_pin: GPIO14
   latch_pin1: GPIO21
   oe_pin1: GPIO22
-  latch_pin2: GPIO23
-  oe_pin2: GPIO25
+  latch_pin2: GPIO25
+  oe_pin2: GPIO26
+
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sn74hc595/test.esp8266-ard.yaml b/tests/components/sn74hc595/test.esp8266-ard.yaml
index cad11feca8..e0de8bb0a3 100644
--- a/tests/components/sn74hc595/test.esp8266-ard.yaml
+++ b/tests/components/sn74hc595/test.esp8266-ard.yaml
@@ -1,9 +1,9 @@
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
-  clock_pin: GPIO5
-  data_pin: GPIO4
+  clock_pin: GPIO15
+  data_pin: GPIO16
   latch_pin1: GPIO2
   oe_pin1: GPIO0
   latch_pin2: GPIO3
diff --git a/tests/components/sn74hc595/test.rp2040-ard.yaml b/tests/components/sn74hc595/test.rp2040-ard.yaml
index 14c928be88..93cf5efb0c 100644
--- a/tests/components/sn74hc595/test.rp2040-ard.yaml
+++ b/tests/components/sn74hc595/test.rp2040-ard.yaml
@@ -1,12 +1,12 @@
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO8
   clock_pin: GPIO5
-  data_pin: GPIO4
+  data_pin: GPIO0
   latch_pin1: GPIO1
-  oe_pin1: GPIO2
-  latch_pin2: GPIO3
+  oe_pin1: GPIO6
+  latch_pin2: GPIO7
   oe_pin2: GPIO9
 
 <<: !include common.yaml
diff --git a/tests/components/sntp/test.esp32-ard.yaml b/tests/components/sntp/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/sntp/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/sntp/test.esp32-c3-ard.yaml b/tests/components/sntp/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/sntp/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/sonoff_d1/common.yaml b/tests/components/sonoff_d1/common.yaml
index d2d4043b95..dc08380683 100644
--- a/tests/components/sonoff_d1/common.yaml
+++ b/tests/components/sonoff_d1/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_sonoff_d1
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 light:
   - platform: sonoff_d1
     id: d1_light
diff --git a/tests/components/sonoff_d1/test.esp32-ard.yaml b/tests/components/sonoff_d1/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/sonoff_d1/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/sonoff_d1/test.esp32-c3-ard.yaml b/tests/components/sonoff_d1/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/sonoff_d1/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/sonoff_d1/test.esp32-c3-idf.yaml b/tests/components/sonoff_d1/test.esp32-c3-idf.yaml
index b516342f3b..a19013bf54 100644
--- a/tests/components/sonoff_d1/test.esp32-c3-idf.yaml
+++ b/tests/components/sonoff_d1/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sonoff_d1/test.esp32-idf.yaml b/tests/components/sonoff_d1/test.esp32-idf.yaml
index f486544afa..2d29656c94 100644
--- a/tests/components/sonoff_d1/test.esp32-idf.yaml
+++ b/tests/components/sonoff_d1/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sonoff_d1/test.esp8266-ard.yaml b/tests/components/sonoff_d1/test.esp8266-ard.yaml
index b516342f3b..5a05efa259 100644
--- a/tests/components/sonoff_d1/test.esp8266-ard.yaml
+++ b/tests/components/sonoff_d1/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sonoff_d1/test.rp2040-ard.yaml b/tests/components/sonoff_d1/test.rp2040-ard.yaml
index b516342f3b..f1df2daf83 100644
--- a/tests/components/sonoff_d1/test.rp2040-ard.yaml
+++ b/tests/components/sonoff_d1/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sound_level/test.esp32-ard.yaml b/tests/components/sound_level/test.esp32-ard.yaml
deleted file mode 100644
index c6d1bfa330..0000000000
--- a/tests/components/sound_level/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  i2s_bclk_pin: GPIO25
-  i2s_lrclk_pin: GPIO26
-  i2s_dout_pin: GPIO27
-
-<<: !include common.yaml
diff --git a/tests/components/sound_level/test.esp32-c3-ard.yaml b/tests/components/sound_level/test.esp32-c3-ard.yaml
deleted file mode 100644
index aeb7d9f0af..0000000000
--- a/tests/components/sound_level/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  i2s_bclk_pin: GPIO6
-  i2s_lrclk_pin: GPIO7
-  i2s_dout_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/sound_level/test.esp32-idf.yaml b/tests/components/sound_level/test.esp32-idf.yaml
index c6d1bfa330..20e38e8df8 100644
--- a/tests/components/sound_level/test.esp32-idf.yaml
+++ b/tests/components/sound_level/test.esp32-idf.yaml
@@ -3,4 +3,7 @@ substitutions:
   i2s_lrclk_pin: GPIO26
   i2s_dout_pin: GPIO27
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/sound_level/test.esp32-s3-ard.yaml b/tests/components/sound_level/test.esp32-s3-ard.yaml
deleted file mode 100644
index 9c1f32d5bd..0000000000
--- a/tests/components/sound_level/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  i2s_bclk_pin: GPIO4
-  i2s_lrclk_pin: GPIO5
-  i2s_dout_pin: GPIO6
-
-<<: !include common.yaml
diff --git a/tests/components/speaker/audio_dac.esp32-ard.yaml b/tests/components/speaker/audio_dac.esp32-ard.yaml
index 75d9ddf92b..3f5d1bba7c 100644
--- a/tests/components/speaker/audio_dac.esp32-ard.yaml
+++ b/tests/components/speaker/audio_dac.esp32-ard.yaml
@@ -1,9 +1,10 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   i2s_bclk_pin: GPIO27
   i2s_lrclk_pin: GPIO26
   i2s_mclk_pin: GPIO25
   i2s_dout_pin: GPIO23
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-ard.yaml
+
 <<: !include common-audio_dac.yaml
diff --git a/tests/components/speaker/audio_dac.esp32-c3-ard.yaml b/tests/components/speaker/audio_dac.esp32-c3-ard.yaml
deleted file mode 100644
index 1004d2143e..0000000000
--- a/tests/components/speaker/audio_dac.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  i2s_bclk_pin: GPIO7
-  i2s_lrclk_pin: GPIO6
-  i2s_mclk_pin: GPIO9
-  i2s_dout_pin: GPIO8
-
-<<: !include common-audio_dac.yaml
diff --git a/tests/components/speaker/audio_dac.esp32-c3-idf.yaml b/tests/components/speaker/audio_dac.esp32-c3-idf.yaml
index 1004d2143e..30900f1920 100644
--- a/tests/components/speaker/audio_dac.esp32-c3-idf.yaml
+++ b/tests/components/speaker/audio_dac.esp32-c3-idf.yaml
@@ -1,9 +1,10 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   i2s_bclk_pin: GPIO7
   i2s_lrclk_pin: GPIO6
   i2s_mclk_pin: GPIO9
   i2s_dout_pin: GPIO8
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common-audio_dac.yaml
diff --git a/tests/components/speaker/audio_dac.esp32-idf.yaml b/tests/components/speaker/audio_dac.esp32-idf.yaml
index 75d9ddf92b..71c8b06e24 100644
--- a/tests/components/speaker/audio_dac.esp32-idf.yaml
+++ b/tests/components/speaker/audio_dac.esp32-idf.yaml
@@ -1,9 +1,10 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   i2s_bclk_pin: GPIO27
   i2s_lrclk_pin: GPIO26
   i2s_mclk_pin: GPIO25
   i2s_dout_pin: GPIO23
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common-audio_dac.yaml
diff --git a/tests/components/speaker/common-audio_dac.yaml b/tests/components/speaker/common-audio_dac.yaml
index 41b994d4d4..67bd6c28ef 100644
--- a/tests/components/speaker/common-audio_dac.yaml
+++ b/tests/components/speaker/common-audio_dac.yaml
@@ -14,11 +14,6 @@ esphome:
             - speaker.finish:
       - speaker.stop:
 
-i2c:
-  - id: i2c_audio_dac
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 i2s_audio:
   i2s_lrclk_pin: ${i2s_bclk_pin}
   i2s_bclk_pin: ${i2s_lrclk_pin}
@@ -26,6 +21,7 @@ i2s_audio:
 
 audio_dac:
   - platform: aic3204
+    i2c_id: i2c_bus
     id: internal_dac
 
 speaker:
diff --git a/tests/components/speaker/test.esp32-ard.yaml b/tests/components/speaker/test.esp32-ard.yaml
index e2439ebdf2..13350cd097 100644
--- a/tests/components/speaker/test.esp32-ard.yaml
+++ b/tests/components/speaker/test.esp32-ard.yaml
@@ -1,9 +1,10 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   i2s_bclk_pin: GPIO27
   i2s_lrclk_pin: GPIO26
   i2s_mclk_pin: GPIO25
-  i2s_dout_pin: GPIO23
+  i2s_dout_pin: GPIO4
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/speaker/test.esp32-c3-ard.yaml b/tests/components/speaker/test.esp32-c3-ard.yaml
deleted file mode 100644
index ddcf051fab..0000000000
--- a/tests/components/speaker/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  i2s_bclk_pin: GPIO7
-  i2s_lrclk_pin: GPIO6
-  i2s_mclk_pin: GPIO9
-  i2s_dout_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/speaker/test.esp32-c3-idf.yaml b/tests/components/speaker/test.esp32-c3-idf.yaml
index ddcf051fab..9d1a1cca3b 100644
--- a/tests/components/speaker/test.esp32-c3-idf.yaml
+++ b/tests/components/speaker/test.esp32-c3-idf.yaml
@@ -6,4 +6,7 @@ substitutions:
   i2s_mclk_pin: GPIO9
   i2s_dout_pin: GPIO8
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/speaker/test.esp32-idf.yaml b/tests/components/speaker/test.esp32-idf.yaml
index e2439ebdf2..27b8604656 100644
--- a/tests/components/speaker/test.esp32-idf.yaml
+++ b/tests/components/speaker/test.esp32-idf.yaml
@@ -1,9 +1,10 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   i2s_bclk_pin: GPIO27
   i2s_lrclk_pin: GPIO26
   i2s_mclk_pin: GPIO25
-  i2s_dout_pin: GPIO23
+  i2s_dout_pin: GPIO12
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/speed/test.esp32-ard.yaml b/tests/components/speed/test.esp32-ard.yaml
deleted file mode 100644
index 26da1ce1d6..0000000000
--- a/tests/components/speed/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  output_platform: ledc
-  pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/speed/test.esp32-c3-ard.yaml b/tests/components/speed/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7476963591..0000000000
--- a/tests/components/speed/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  output_platform: ledc
-  pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/spi/common.yaml b/tests/components/spi/common.yaml
index 04b4779957..e69de29bb2 100644
--- a/tests/components/spi/common.yaml
+++ b/tests/components/spi/common.yaml
@@ -1,5 +0,0 @@
-spi:
-  - id: spi_spi
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
diff --git a/tests/components/spi/test.esp32-c3-ard.yaml b/tests/components/spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index bfa12b1755..0000000000
--- a/tests/components/spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/spi/test.esp32-c3-idf.yaml b/tests/components/spi/test.esp32-c3-idf.yaml
index bfa12b1755..54463271a9 100644
--- a/tests/components/spi/test.esp32-c3-idf.yaml
+++ b/tests/components/spi/test.esp32-c3-idf.yaml
@@ -1,6 +1,5 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/spi/test.esp32-idf.yaml b/tests/components/spi/test.esp32-idf.yaml
index 448e54fea6..a8e18ca503 100644
--- a/tests/components/spi/test.esp32-idf.yaml
+++ b/tests/components/spi/test.esp32-idf.yaml
@@ -1,6 +1,4 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/spi/test.esp32-s3-ard.yaml b/tests/components/spi/test.esp32-s3-ard.yaml
new file mode 100644
index 0000000000..e4d4f20586
--- /dev/null
+++ b/tests/components/spi/test.esp32-s3-ard.yaml
@@ -0,0 +1,13 @@
+spi:
+  - id: three_spi
+    interface: spi3
+    clk_pin:
+      number: 47
+    mosi_pin:
+      number: 40
+  - id: hw_spi
+    interface: hardware
+    clk_pin:
+      number: 0
+    miso_pin:
+      number: 41
diff --git a/tests/components/spi/test.esp8266-ard.yaml b/tests/components/spi/test.esp8266-ard.yaml
index b9545d4f6a..4f9c816740 100644
--- a/tests/components/spi/test.esp8266-ard.yaml
+++ b/tests/components/spi/test.esp8266-ard.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO15
 
 <<: !include common.yaml
diff --git a/tests/components/spi_device/common.yaml b/tests/components/spi_device/common.yaml
index 0f6a5038fb..8511603bc2 100644
--- a/tests/components/spi_device/common.yaml
+++ b/tests/components/spi_device/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_device1
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 spi_device:
   - id: spi_device_test
     data_rate: 2MHz
diff --git a/tests/components/spi_device/test.esp32-ard.yaml b/tests/components/spi_device/test.esp32-ard.yaml
deleted file mode 100644
index 448e54fea6..0000000000
--- a/tests/components/spi_device/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/spi_device/test.esp32-c3-ard.yaml b/tests/components/spi_device/test.esp32-c3-ard.yaml
deleted file mode 100644
index bfa12b1755..0000000000
--- a/tests/components/spi_device/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/spi_device/test.esp32-c3-idf.yaml b/tests/components/spi_device/test.esp32-c3-idf.yaml
index bfa12b1755..6d64c2b23b 100644
--- a/tests/components/spi_device/test.esp32-c3-idf.yaml
+++ b/tests/components/spi_device/test.esp32-c3-idf.yaml
@@ -1,6 +1,4 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/spi_device/test.esp32-idf.yaml b/tests/components/spi_device/test.esp32-idf.yaml
index c4989cccbf..aace9192b1 100644
--- a/tests/components/spi_device/test.esp32-idf.yaml
+++ b/tests/components/spi_device/test.esp32-idf.yaml
@@ -1,7 +1,5 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  miso_pin: GPIO15
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
 <<: !include common.yaml
 spi_device:
diff --git a/tests/components/spi_device/test.esp8266-ard.yaml b/tests/components/spi_device/test.esp8266-ard.yaml
index b9545d4f6a..433a931284 100644
--- a/tests/components/spi_device/test.esp8266-ard.yaml
+++ b/tests/components/spi_device/test.esp8266-ard.yaml
@@ -1,6 +1,4 @@
-substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/spi_device/test.rp2040-ard.yaml b/tests/components/spi_device/test.rp2040-ard.yaml
index 81a8acafd8..149d530753 100644
--- a/tests/components/spi_device/test.rp2040-ard.yaml
+++ b/tests/components/spi_device/test.rp2040-ard.yaml
@@ -1,6 +1,4 @@
-substitutions:
-  clk_pin: GPIO2
-  mosi_pin: GPIO3
-  miso_pin: GPIO4
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/spi_led_strip/common.yaml b/tests/components/spi_led_strip/common.yaml
index 80b98a63a4..30e2ef22f7 100644
--- a/tests/components/spi_led_strip/common.yaml
+++ b/tests/components/spi_led_strip/common.yaml
@@ -1,8 +1,3 @@
-spi:
-  - id: spi_spi_led_strip
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 light:
   - platform: spi_led_strip
     num_leds: 4
diff --git a/tests/components/spi_led_strip/test.esp32-ard.yaml b/tests/components/spi_led_strip/test.esp32-ard.yaml
deleted file mode 100644
index 8906602ef4..0000000000
--- a/tests/components/spi_led_strip/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/spi_led_strip/test.esp32-c3-ard.yaml b/tests/components/spi_led_strip/test.esp32-c3-ard.yaml
deleted file mode 100644
index a85b587070..0000000000
--- a/tests/components/spi_led_strip/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-
-<<: !include common.yaml
diff --git a/tests/components/spi_led_strip/test.esp32-c3-idf.yaml b/tests/components/spi_led_strip/test.esp32-c3-idf.yaml
index a85b587070..6d64c2b23b 100644
--- a/tests/components/spi_led_strip/test.esp32-c3-idf.yaml
+++ b/tests/components/spi_led_strip/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/spi_led_strip/test.esp32-idf.yaml b/tests/components/spi_led_strip/test.esp32-idf.yaml
index 8906602ef4..a8e18ca503 100644
--- a/tests/components/spi_led_strip/test.esp32-idf.yaml
+++ b/tests/components/spi_led_strip/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/spi_led_strip/test.esp8266-ard.yaml b/tests/components/spi_led_strip/test.esp8266-ard.yaml
index 7baaa62ed5..433a931284 100644
--- a/tests/components/spi_led_strip/test.esp8266-ard.yaml
+++ b/tests/components/spi_led_strip/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/spi_led_strip/test.rp2040-ard.yaml b/tests/components/spi_led_strip/test.rp2040-ard.yaml
index 411cfbe00e..149d530753 100644
--- a/tests/components/spi_led_strip/test.rp2040-ard.yaml
+++ b/tests/components/spi_led_strip/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  clk_pin: GPIO2
-  mosi_pin: GPIO3
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sprinkler/test.esp32-ard.yaml b/tests/components/sprinkler/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/sprinkler/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/sprinkler/test.esp32-c3-ard.yaml b/tests/components/sprinkler/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/sprinkler/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/dallas_temp/test.esp32-c3-ard.yaml b/tests/components/sprinkler/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/dallas_temp/test.esp32-c3-ard.yaml
rename to tests/components/sprinkler/test.nrf52-adafruit.yaml
diff --git a/tests/components/datetime/test.all.yaml b/tests/components/sprinkler/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/datetime/test.all.yaml
rename to tests/components/sprinkler/test.nrf52-mcumgr.yaml
diff --git a/tests/components/sps30/common.yaml b/tests/components/sps30/common.yaml
index 2fbe2c747a..d40cd16b6d 100644
--- a/tests/components/sps30/common.yaml
+++ b/tests/components/sps30/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_sps30
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: sps30
+    i2c_id: i2c_bus
     pm_1_0:
       name: Workshop PM <1µm Weight concentration
       id: workshop_PM_1_0
diff --git a/tests/components/sps30/test.esp32-ard.yaml b/tests/components/sps30/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/sps30/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sps30/test.esp32-c3-ard.yaml b/tests/components/sps30/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/sps30/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sps30/test.esp32-c3-idf.yaml b/tests/components/sps30/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/sps30/test.esp32-c3-idf.yaml
+++ b/tests/components/sps30/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sps30/test.esp32-idf.yaml b/tests/components/sps30/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/sps30/test.esp32-idf.yaml
+++ b/tests/components/sps30/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sps30/test.esp8266-ard.yaml b/tests/components/sps30/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/sps30/test.esp8266-ard.yaml
+++ b/tests/components/sps30/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sps30/test.rp2040-ard.yaml b/tests/components/sps30/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/sps30/test.rp2040-ard.yaml
+++ b/tests/components/sps30/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ssd1306_i2c/common.yaml b/tests/components/ssd1306_i2c/common.yaml
index d17f83f03a..09eb569a8e 100644
--- a/tests/components/ssd1306_i2c/common.yaml
+++ b/tests/components/ssd1306_i2c/common.yaml
@@ -1,25 +1,21 @@
-i2c:
-  - id: i2c_ssd1306_i2c
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 display:
   - platform: ssd1306_i2c
+    i2c_id: i2c_bus
     model: SSD1306_128X64
     reset_pin: ${reset_pin}
     address: 0x3C
-    id: display1
+    id: ssd1306_i2c_display
     contrast: 60%
     pages:
-      - id: page1
+      - id: ssd1306_i2c_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
-      - id: page2
+      - id: ssd1306_i2c_page2
         lambda: |-
           it.rectangle(0, 0, 10, 10);
     on_page_change:
-      from: page1
-      to: page2
+      from: ssd1306_i2c_page1
+      to: ssd1306_i2c_page2
       then:
         lambda: |-
           ESP_LOGD("display", "1 -> 2");
diff --git a/tests/components/ssd1306_i2c/test.esp32-ard.yaml b/tests/components/ssd1306_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 1ca773e06c..0000000000
--- a/tests/components/ssd1306_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  reset_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1306_i2c/test.esp32-c3-ard.yaml b/tests/components/ssd1306_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index 4eaff7fa4a..0000000000
--- a/tests/components/ssd1306_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  reset_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1306_i2c/test.esp32-c3-idf.yaml b/tests/components/ssd1306_i2c/test.esp32-c3-idf.yaml
index 4eaff7fa4a..f8bfab2319 100644
--- a/tests/components/ssd1306_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/ssd1306_i2c/test.esp32-c3-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO3
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1306_i2c/test.esp32-idf.yaml b/tests/components/ssd1306_i2c/test.esp32-idf.yaml
index 1ca773e06c..4ff2241ec9 100644
--- a/tests/components/ssd1306_i2c/test.esp32-idf.yaml
+++ b/tests/components/ssd1306_i2c/test.esp32-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   reset_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1306_i2c/test.esp8266-ard.yaml b/tests/components/ssd1306_i2c/test.esp8266-ard.yaml
index af91c21a0d..352cc8cda8 100644
--- a/tests/components/ssd1306_i2c/test.esp8266-ard.yaml
+++ b/tests/components/ssd1306_i2c/test.esp8266-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO2
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1306_i2c/test.rp2040-ard.yaml b/tests/components/ssd1306_i2c/test.rp2040-ard.yaml
index 4eaff7fa4a..2972fde8a5 100644
--- a/tests/components/ssd1306_i2c/test.rp2040-ard.yaml
+++ b/tests/components/ssd1306_i2c/test.rp2040-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO3
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1306_spi/common.yaml b/tests/components/ssd1306_spi/common.yaml
index 71705f32d2..0297abc192 100644
--- a/tests/components/ssd1306_spi/common.yaml
+++ b/tests/components/ssd1306_spi/common.yaml
@@ -1,24 +1,20 @@
-spi:
-  - id: spi_ssd1306_spi
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   - platform: ssd1306_spi
+    id: ssd1306_spi_display
     model: SSD1306 128x64
     cs_pin: ${cs_pin}
     dc_pin: ${dc_pin}
     reset_pin: ${reset_pin}
     pages:
-      - id: page1
+      - id: ssd1306_spi_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
-      - id: page2
+      - id: ssd1306_spi_page2
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
     on_page_change:
-      from: page1
-      to: page2
+      from: ssd1306_spi_page1
+      to: ssd1306_spi_page2
       then:
         lambda: |-
           ESP_LOGD("display", "1 -> 2");
diff --git a/tests/components/ssd1306_spi/test.esp32-ard.yaml b/tests/components/ssd1306_spi/test.esp32-ard.yaml
deleted file mode 100644
index bad5241f79..0000000000
--- a/tests/components/ssd1306_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  cs_pin: GPIO12
-  dc_pin: GPIO13
-  reset_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1306_spi/test.esp32-c3-ard.yaml b/tests/components/ssd1306_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index c5c932c92c..0000000000
--- a/tests/components/ssd1306_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-  dc_pin: GPIO9
-  reset_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1306_spi/test.esp32-c3-idf.yaml b/tests/components/ssd1306_spi/test.esp32-c3-idf.yaml
index c5c932c92c..b112cf4c31 100644
--- a/tests/components/ssd1306_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/ssd1306_spi/test.esp32-c3-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
   dc_pin: GPIO9
   reset_pin: GPIO10
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ssd1306_spi/test.esp32-idf.yaml b/tests/components/ssd1306_spi/test.esp32-idf.yaml
index bad5241f79..ff174a4656 100644
--- a/tests/components/ssd1306_spi/test.esp32-idf.yaml
+++ b/tests/components/ssd1306_spi/test.esp32-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
   cs_pin: GPIO12
   dc_pin: GPIO13
   reset_pin: GPIO14
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1306_spi/test.esp8266-ard.yaml b/tests/components/ssd1306_spi/test.esp8266-ard.yaml
index 3f023a60eb..56cb29f29e 100644
--- a/tests/components/ssd1306_spi/test.esp8266-ard.yaml
+++ b/tests/components/ssd1306_spi/test.esp8266-ard.yaml
@@ -1,9 +1,12 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
   miso_pin: GPIO12
   cs_pin: GPIO5
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1306_spi/test.rp2040-ard.yaml b/tests/components/ssd1306_spi/test.rp2040-ard.yaml
index d7fd6ee294..66caa956f7 100644
--- a/tests/components/ssd1306_spi/test.rp2040-ard.yaml
+++ b/tests/components/ssd1306_spi/test.rp2040-ard.yaml
@@ -6,4 +6,7 @@ substitutions:
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1322_spi/common.yaml b/tests/components/ssd1322_spi/common.yaml
index b67392794c..c0e16b6f09 100644
--- a/tests/components/ssd1322_spi/common.yaml
+++ b/tests/components/ssd1322_spi/common.yaml
@@ -1,8 +1,3 @@
-spi:
-  - id: spi_ssd1322_spi
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   - platform: ssd1322_spi
     model: SSD1322 256x64
@@ -10,15 +5,15 @@ display:
     dc_pin: ${dc_pin}
     reset_pin: ${reset_pin}
     pages:
-      - id: page1
+      - id: ssd1322_spi_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
-      - id: page2
+      - id: ssd1322_spi_page2
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
     on_page_change:
-      from: page1
-      to: page2
+      from: ssd1322_spi_page1
+      to: ssd1322_spi_page2
       then:
         lambda: |-
           ESP_LOGD("display", "1 -> 2");
diff --git a/tests/components/ssd1322_spi/test.esp32-ard.yaml b/tests/components/ssd1322_spi/test.esp32-ard.yaml
deleted file mode 100644
index bad5241f79..0000000000
--- a/tests/components/ssd1322_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  cs_pin: GPIO12
-  dc_pin: GPIO13
-  reset_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1322_spi/test.esp32-c3-ard.yaml b/tests/components/ssd1322_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index c5c932c92c..0000000000
--- a/tests/components/ssd1322_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-  dc_pin: GPIO9
-  reset_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1322_spi/test.esp32-c3-idf.yaml b/tests/components/ssd1322_spi/test.esp32-c3-idf.yaml
index c5c932c92c..b112cf4c31 100644
--- a/tests/components/ssd1322_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/ssd1322_spi/test.esp32-c3-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
   dc_pin: GPIO9
   reset_pin: GPIO10
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ssd1322_spi/test.esp32-idf.yaml b/tests/components/ssd1322_spi/test.esp32-idf.yaml
index bad5241f79..ff174a4656 100644
--- a/tests/components/ssd1322_spi/test.esp32-idf.yaml
+++ b/tests/components/ssd1322_spi/test.esp32-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
   cs_pin: GPIO12
   dc_pin: GPIO13
   reset_pin: GPIO14
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1322_spi/test.esp8266-ard.yaml b/tests/components/ssd1322_spi/test.esp8266-ard.yaml
index 3f023a60eb..56cb29f29e 100644
--- a/tests/components/ssd1322_spi/test.esp8266-ard.yaml
+++ b/tests/components/ssd1322_spi/test.esp8266-ard.yaml
@@ -1,9 +1,12 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
   miso_pin: GPIO12
   cs_pin: GPIO5
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1322_spi/test.rp2040-ard.yaml b/tests/components/ssd1322_spi/test.rp2040-ard.yaml
index d7fd6ee294..66caa956f7 100644
--- a/tests/components/ssd1322_spi/test.rp2040-ard.yaml
+++ b/tests/components/ssd1322_spi/test.rp2040-ard.yaml
@@ -6,4 +6,7 @@ substitutions:
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1325_spi/common.yaml b/tests/components/ssd1325_spi/common.yaml
index eaa8b619ca..dea8502538 100644
--- a/tests/components/ssd1325_spi/common.yaml
+++ b/tests/components/ssd1325_spi/common.yaml
@@ -1,8 +1,3 @@
-spi:
-  - id: spi_ssd1325_spi
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   - platform: ssd1325_spi
     model: SSD1325 128x64
@@ -10,15 +5,15 @@ display:
     dc_pin: ${dc_pin}
     reset_pin: ${reset_pin}
     pages:
-      - id: page1
+      - id: ssd1325_spi_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
-      - id: page2
+      - id: ssd1325_spi_page2
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
     on_page_change:
-      from: page1
-      to: page2
+      from: ssd1325_spi_page1
+      to: ssd1325_spi_page2
       then:
         lambda: |-
           ESP_LOGD("display", "1 -> 2");
diff --git a/tests/components/ssd1325_spi/test.esp32-ard.yaml b/tests/components/ssd1325_spi/test.esp32-ard.yaml
deleted file mode 100644
index bad5241f79..0000000000
--- a/tests/components/ssd1325_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  cs_pin: GPIO12
-  dc_pin: GPIO13
-  reset_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1325_spi/test.esp32-c3-ard.yaml b/tests/components/ssd1325_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index c5c932c92c..0000000000
--- a/tests/components/ssd1325_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-  dc_pin: GPIO9
-  reset_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1325_spi/test.esp32-c3-idf.yaml b/tests/components/ssd1325_spi/test.esp32-c3-idf.yaml
index c5c932c92c..b112cf4c31 100644
--- a/tests/components/ssd1325_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/ssd1325_spi/test.esp32-c3-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
   dc_pin: GPIO9
   reset_pin: GPIO10
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ssd1325_spi/test.esp32-idf.yaml b/tests/components/ssd1325_spi/test.esp32-idf.yaml
index bad5241f79..ff174a4656 100644
--- a/tests/components/ssd1325_spi/test.esp32-idf.yaml
+++ b/tests/components/ssd1325_spi/test.esp32-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
   cs_pin: GPIO12
   dc_pin: GPIO13
   reset_pin: GPIO14
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1325_spi/test.esp8266-ard.yaml b/tests/components/ssd1325_spi/test.esp8266-ard.yaml
index 3f023a60eb..56cb29f29e 100644
--- a/tests/components/ssd1325_spi/test.esp8266-ard.yaml
+++ b/tests/components/ssd1325_spi/test.esp8266-ard.yaml
@@ -1,9 +1,12 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
   miso_pin: GPIO12
   cs_pin: GPIO5
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1325_spi/test.rp2040-ard.yaml b/tests/components/ssd1325_spi/test.rp2040-ard.yaml
index d7fd6ee294..66caa956f7 100644
--- a/tests/components/ssd1325_spi/test.rp2040-ard.yaml
+++ b/tests/components/ssd1325_spi/test.rp2040-ard.yaml
@@ -6,4 +6,7 @@ substitutions:
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1327_i2c/common.yaml b/tests/components/ssd1327_i2c/common.yaml
index 72a122c3d7..c5f2123d9a 100644
--- a/tests/components/ssd1327_i2c/common.yaml
+++ b/tests/components/ssd1327_i2c/common.yaml
@@ -1,24 +1,20 @@
-i2c:
-  - id: i2c_ssd1327_i2c
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 display:
   - platform: ssd1327_i2c
+    i2c_id: i2c_bus
     model: SSD1327_128x128
     reset_pin: ${reset_pin}
     address: 0x3C
-    id: display1
+    id: ssd1327_i2c_display
     pages:
-      - id: page1
+      - id: ssd1327_i2c_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
-      - id: page2
+      - id: ssd1327_i2c_page2
         lambda: |-
           it.rectangle(0, 0, 10, 10);
     on_page_change:
-      from: page1
-      to: page2
+      from: ssd1327_i2c_page1
+      to: ssd1327_i2c_page2
       then:
         lambda: |-
           ESP_LOGD("display", "1 -> 2");
diff --git a/tests/components/ssd1327_i2c/test.esp32-ard.yaml b/tests/components/ssd1327_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 1ca773e06c..0000000000
--- a/tests/components/ssd1327_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  reset_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1327_i2c/test.esp32-c3-ard.yaml b/tests/components/ssd1327_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index 4eaff7fa4a..0000000000
--- a/tests/components/ssd1327_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  reset_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1327_i2c/test.esp32-c3-idf.yaml b/tests/components/ssd1327_i2c/test.esp32-c3-idf.yaml
index 4eaff7fa4a..f8bfab2319 100644
--- a/tests/components/ssd1327_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/ssd1327_i2c/test.esp32-c3-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO3
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1327_i2c/test.esp32-idf.yaml b/tests/components/ssd1327_i2c/test.esp32-idf.yaml
index 1ca773e06c..4ff2241ec9 100644
--- a/tests/components/ssd1327_i2c/test.esp32-idf.yaml
+++ b/tests/components/ssd1327_i2c/test.esp32-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   reset_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1327_i2c/test.esp8266-ard.yaml b/tests/components/ssd1327_i2c/test.esp8266-ard.yaml
index af91c21a0d..352cc8cda8 100644
--- a/tests/components/ssd1327_i2c/test.esp8266-ard.yaml
+++ b/tests/components/ssd1327_i2c/test.esp8266-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO2
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1327_i2c/test.rp2040-ard.yaml b/tests/components/ssd1327_i2c/test.rp2040-ard.yaml
index 4eaff7fa4a..2972fde8a5 100644
--- a/tests/components/ssd1327_i2c/test.rp2040-ard.yaml
+++ b/tests/components/ssd1327_i2c/test.rp2040-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO3
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1327_spi/common.yaml b/tests/components/ssd1327_spi/common.yaml
index 85f4d4736b..b46e61e080 100644
--- a/tests/components/ssd1327_spi/common.yaml
+++ b/tests/components/ssd1327_spi/common.yaml
@@ -1,24 +1,20 @@
-spi:
-  - id: spi_ssd1327_spi
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   - platform: ssd1327_spi
+    id: ssd1327_spi_display
     model: SSD1327 128x128
     cs_pin: ${cs_pin}
     dc_pin: ${dc_pin}
     reset_pin: ${reset_pin}
     pages:
-      - id: page1
+      - id: ssd1327_spi_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
-      - id: page2
+      - id: ssd1327_spi_page2
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
     on_page_change:
-      from: page1
-      to: page2
+      from: ssd1327_spi_page1
+      to: ssd1327_spi_page2
       then:
         lambda: |-
           ESP_LOGD("display", "1 -> 2");
diff --git a/tests/components/ssd1327_spi/test.esp32-ard.yaml b/tests/components/ssd1327_spi/test.esp32-ard.yaml
deleted file mode 100644
index bad5241f79..0000000000
--- a/tests/components/ssd1327_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  cs_pin: GPIO12
-  dc_pin: GPIO13
-  reset_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1327_spi/test.esp32-c3-ard.yaml b/tests/components/ssd1327_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index c5c932c92c..0000000000
--- a/tests/components/ssd1327_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-  dc_pin: GPIO9
-  reset_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1327_spi/test.esp32-c3-idf.yaml b/tests/components/ssd1327_spi/test.esp32-c3-idf.yaml
index c5c932c92c..b112cf4c31 100644
--- a/tests/components/ssd1327_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/ssd1327_spi/test.esp32-c3-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
   dc_pin: GPIO9
   reset_pin: GPIO10
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ssd1327_spi/test.esp32-idf.yaml b/tests/components/ssd1327_spi/test.esp32-idf.yaml
index bad5241f79..ff174a4656 100644
--- a/tests/components/ssd1327_spi/test.esp32-idf.yaml
+++ b/tests/components/ssd1327_spi/test.esp32-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
   cs_pin: GPIO12
   dc_pin: GPIO13
   reset_pin: GPIO14
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1327_spi/test.esp8266-ard.yaml b/tests/components/ssd1327_spi/test.esp8266-ard.yaml
index 3f023a60eb..56cb29f29e 100644
--- a/tests/components/ssd1327_spi/test.esp8266-ard.yaml
+++ b/tests/components/ssd1327_spi/test.esp8266-ard.yaml
@@ -1,9 +1,12 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
   miso_pin: GPIO12
   cs_pin: GPIO5
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1327_spi/test.rp2040-ard.yaml b/tests/components/ssd1327_spi/test.rp2040-ard.yaml
index d7fd6ee294..66caa956f7 100644
--- a/tests/components/ssd1327_spi/test.rp2040-ard.yaml
+++ b/tests/components/ssd1327_spi/test.rp2040-ard.yaml
@@ -6,4 +6,7 @@ substitutions:
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1331_spi/common.yaml b/tests/components/ssd1331_spi/common.yaml
index 40726101e8..2b708e9bd9 100644
--- a/tests/components/ssd1331_spi/common.yaml
+++ b/tests/components/ssd1331_spi/common.yaml
@@ -1,23 +1,18 @@
-spi:
-  - id: spi_ssd1331_spi
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   - platform: ssd1331_spi
     cs_pin: ${cs_pin}
     dc_pin: ${dc_pin}
     reset_pin: ${reset_pin}
     pages:
-      - id: page1
+      - id: ssd1331_spi_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
-      - id: page2
+      - id: ssd1331_spi_page2
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
     on_page_change:
-      from: page1
-      to: page2
+      from: ssd1331_spi_page1
+      to: ssd1331_spi_page2
       then:
         lambda: |-
           ESP_LOGD("display", "1 -> 2");
diff --git a/tests/components/ssd1331_spi/test.esp32-ard.yaml b/tests/components/ssd1331_spi/test.esp32-ard.yaml
deleted file mode 100644
index bad5241f79..0000000000
--- a/tests/components/ssd1331_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  cs_pin: GPIO12
-  dc_pin: GPIO13
-  reset_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1331_spi/test.esp32-c3-ard.yaml b/tests/components/ssd1331_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index c5c932c92c..0000000000
--- a/tests/components/ssd1331_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-  dc_pin: GPIO9
-  reset_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1331_spi/test.esp32-c3-idf.yaml b/tests/components/ssd1331_spi/test.esp32-c3-idf.yaml
index c5c932c92c..b112cf4c31 100644
--- a/tests/components/ssd1331_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/ssd1331_spi/test.esp32-c3-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
   dc_pin: GPIO9
   reset_pin: GPIO10
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ssd1331_spi/test.esp32-idf.yaml b/tests/components/ssd1331_spi/test.esp32-idf.yaml
index bad5241f79..ff174a4656 100644
--- a/tests/components/ssd1331_spi/test.esp32-idf.yaml
+++ b/tests/components/ssd1331_spi/test.esp32-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
   cs_pin: GPIO12
   dc_pin: GPIO13
   reset_pin: GPIO14
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1331_spi/test.esp8266-ard.yaml b/tests/components/ssd1331_spi/test.esp8266-ard.yaml
index 3f023a60eb..56cb29f29e 100644
--- a/tests/components/ssd1331_spi/test.esp8266-ard.yaml
+++ b/tests/components/ssd1331_spi/test.esp8266-ard.yaml
@@ -1,9 +1,12 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
   miso_pin: GPIO12
   cs_pin: GPIO5
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1331_spi/test.rp2040-ard.yaml b/tests/components/ssd1331_spi/test.rp2040-ard.yaml
index d7fd6ee294..66caa956f7 100644
--- a/tests/components/ssd1331_spi/test.rp2040-ard.yaml
+++ b/tests/components/ssd1331_spi/test.rp2040-ard.yaml
@@ -6,4 +6,7 @@ substitutions:
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1351_spi/common.yaml b/tests/components/ssd1351_spi/common.yaml
index fa495ff0c3..e4d3d55f3f 100644
--- a/tests/components/ssd1351_spi/common.yaml
+++ b/tests/components/ssd1351_spi/common.yaml
@@ -1,8 +1,3 @@
-spi:
-  - id: spi_ssd1351_spi
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   - platform: ssd1351_spi
     model: "SSD1351 128x128"
@@ -10,15 +5,15 @@ display:
     dc_pin: ${dc_pin}
     reset_pin: ${reset_pin}
     pages:
-      - id: page1
+      - id: ssd1351_spi_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
-      - id: page2
+      - id: ssd1351_spi_page2
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
     on_page_change:
-      from: page1
-      to: page2
+      from: ssd1351_spi_page1
+      to: ssd1351_spi_page2
       then:
         lambda: |-
           ESP_LOGD("display", "1 -> 2");
diff --git a/tests/components/ssd1351_spi/test.esp32-ard.yaml b/tests/components/ssd1351_spi/test.esp32-ard.yaml
deleted file mode 100644
index bad5241f79..0000000000
--- a/tests/components/ssd1351_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  cs_pin: GPIO12
-  dc_pin: GPIO13
-  reset_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1351_spi/test.esp32-c3-ard.yaml b/tests/components/ssd1351_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index c5c932c92c..0000000000
--- a/tests/components/ssd1351_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-  dc_pin: GPIO9
-  reset_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/ssd1351_spi/test.esp32-c3-idf.yaml b/tests/components/ssd1351_spi/test.esp32-c3-idf.yaml
index c5c932c92c..b112cf4c31 100644
--- a/tests/components/ssd1351_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/ssd1351_spi/test.esp32-c3-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
   dc_pin: GPIO9
   reset_pin: GPIO10
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ssd1351_spi/test.esp32-idf.yaml b/tests/components/ssd1351_spi/test.esp32-idf.yaml
index bad5241f79..ff174a4656 100644
--- a/tests/components/ssd1351_spi/test.esp32-idf.yaml
+++ b/tests/components/ssd1351_spi/test.esp32-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
   cs_pin: GPIO12
   dc_pin: GPIO13
   reset_pin: GPIO14
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1351_spi/test.esp8266-ard.yaml b/tests/components/ssd1351_spi/test.esp8266-ard.yaml
index 3f023a60eb..56cb29f29e 100644
--- a/tests/components/ssd1351_spi/test.esp8266-ard.yaml
+++ b/tests/components/ssd1351_spi/test.esp8266-ard.yaml
@@ -1,9 +1,12 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
   miso_pin: GPIO12
   cs_pin: GPIO5
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ssd1351_spi/test.rp2040-ard.yaml b/tests/components/ssd1351_spi/test.rp2040-ard.yaml
index d7fd6ee294..66caa956f7 100644
--- a/tests/components/ssd1351_spi/test.rp2040-ard.yaml
+++ b/tests/components/ssd1351_spi/test.rp2040-ard.yaml
@@ -6,4 +6,7 @@ substitutions:
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/st7567_i2c/common.yaml b/tests/components/st7567_i2c/common.yaml
index 41c65e5110..c81d6825e3 100644
--- a/tests/components/st7567_i2c/common.yaml
+++ b/tests/components/st7567_i2c/common.yaml
@@ -1,23 +1,19 @@
-i2c:
-  - id: i2c_st7567_i2c
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 display:
   - platform: st7567_i2c
+    i2c_id: i2c_bus
     reset_pin: ${reset_pin}
     address: 0x3C
-    id: display1
+    id: st7567_i2c_display
     pages:
-      - id: page1
+      - id: st7567_i2c_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
-      - id: page2
+      - id: st7567_i2c_page2
         lambda: |-
           it.rectangle(0, 0, 10, 10);
     on_page_change:
-      from: page1
-      to: page2
+      from: st7567_i2c_page1
+      to: st7567_i2c_page2
       then:
         lambda: |-
           ESP_LOGD("display", "1 -> 2");
diff --git a/tests/components/st7567_i2c/test.esp32-ard.yaml b/tests/components/st7567_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 1ca773e06c..0000000000
--- a/tests/components/st7567_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-  reset_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/st7567_i2c/test.esp32-c3-ard.yaml b/tests/components/st7567_i2c/test.esp32-c3-ard.yaml
deleted file mode 100644
index 4eaff7fa4a..0000000000
--- a/tests/components/st7567_i2c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-  reset_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/st7567_i2c/test.esp32-c3-idf.yaml b/tests/components/st7567_i2c/test.esp32-c3-idf.yaml
index 4eaff7fa4a..f8bfab2319 100644
--- a/tests/components/st7567_i2c/test.esp32-c3-idf.yaml
+++ b/tests/components/st7567_i2c/test.esp32-c3-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO3
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/st7567_i2c/test.esp32-idf.yaml b/tests/components/st7567_i2c/test.esp32-idf.yaml
index 1ca773e06c..4ff2241ec9 100644
--- a/tests/components/st7567_i2c/test.esp32-idf.yaml
+++ b/tests/components/st7567_i2c/test.esp32-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
   reset_pin: GPIO15
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/st7567_i2c/test.esp8266-ard.yaml b/tests/components/st7567_i2c/test.esp8266-ard.yaml
index af91c21a0d..352cc8cda8 100644
--- a/tests/components/st7567_i2c/test.esp8266-ard.yaml
+++ b/tests/components/st7567_i2c/test.esp8266-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO2
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/st7567_i2c/test.rp2040-ard.yaml b/tests/components/st7567_i2c/test.rp2040-ard.yaml
index 4eaff7fa4a..2972fde8a5 100644
--- a/tests/components/st7567_i2c/test.rp2040-ard.yaml
+++ b/tests/components/st7567_i2c/test.rp2040-ard.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
   reset_pin: GPIO3
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/st7567_spi/common.yaml b/tests/components/st7567_spi/common.yaml
index 037a700239..25a8932ee1 100644
--- a/tests/components/st7567_spi/common.yaml
+++ b/tests/components/st7567_spi/common.yaml
@@ -1,23 +1,19 @@
-spi:
-  - id: spi_st7567_spi
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   - platform: st7567_spi
+    id: st7567_spi_display
     cs_pin: ${cs_pin}
     dc_pin: ${dc_pin}
     reset_pin: ${reset_pin}
     pages:
-      - id: page1
+      - id: st7567_spi_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
-      - id: page2
+      - id: st7567_spi_page2
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
     on_page_change:
-      from: page1
-      to: page2
+      from: st7567_spi_page1
+      to: st7567_spi_page2
       then:
         lambda: |-
           ESP_LOGD("display", "1 -> 2");
diff --git a/tests/components/st7567_spi/test.esp32-ard.yaml b/tests/components/st7567_spi/test.esp32-ard.yaml
deleted file mode 100644
index bad5241f79..0000000000
--- a/tests/components/st7567_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  cs_pin: GPIO12
-  dc_pin: GPIO13
-  reset_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/st7567_spi/test.esp32-c3-ard.yaml b/tests/components/st7567_spi/test.esp32-c3-ard.yaml
deleted file mode 100644
index c5c932c92c..0000000000
--- a/tests/components/st7567_spi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-  dc_pin: GPIO9
-  reset_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/st7567_spi/test.esp32-c3-idf.yaml b/tests/components/st7567_spi/test.esp32-c3-idf.yaml
index c5c932c92c..b112cf4c31 100644
--- a/tests/components/st7567_spi/test.esp32-c3-idf.yaml
+++ b/tests/components/st7567_spi/test.esp32-c3-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
   dc_pin: GPIO9
   reset_pin: GPIO10
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/st7567_spi/test.esp32-idf.yaml b/tests/components/st7567_spi/test.esp32-idf.yaml
index bad5241f79..ff174a4656 100644
--- a/tests/components/st7567_spi/test.esp32-idf.yaml
+++ b/tests/components/st7567_spi/test.esp32-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
   cs_pin: GPIO12
   dc_pin: GPIO13
   reset_pin: GPIO14
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/st7567_spi/test.esp8266-ard.yaml b/tests/components/st7567_spi/test.esp8266-ard.yaml
index 3f023a60eb..56cb29f29e 100644
--- a/tests/components/st7567_spi/test.esp8266-ard.yaml
+++ b/tests/components/st7567_spi/test.esp8266-ard.yaml
@@ -1,9 +1,12 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
   miso_pin: GPIO12
   cs_pin: GPIO5
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/st7567_spi/test.rp2040-ard.yaml b/tests/components/st7567_spi/test.rp2040-ard.yaml
index d7fd6ee294..66caa956f7 100644
--- a/tests/components/st7567_spi/test.rp2040-ard.yaml
+++ b/tests/components/st7567_spi/test.rp2040-ard.yaml
@@ -6,4 +6,7 @@ substitutions:
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/st7701s/common.yaml b/tests/components/st7701s/common.yaml
index b94fadfe52..751862dea5 100644
--- a/tests/components/st7701s/common.yaml
+++ b/tests/components/st7701s/common.yaml
@@ -1,8 +1,3 @@
-spi:
-  - id: spi_st7701s
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   - platform: st7701s
     spi_mode: MODE3
diff --git a/tests/components/st7701s/test.esp32-s3-idf.yaml b/tests/components/st7701s/test.esp32-s3-idf.yaml
index cd09b31f6e..87e5248888 100644
--- a/tests/components/st7701s/test.esp32-s3-idf.yaml
+++ b/tests/components/st7701s/test.esp32-s3-idf.yaml
@@ -1,5 +1,7 @@
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-s3-idf.yaml
+
 substitutions:
-  clk_pin: GPIO41
   mosi_pin: GPIO48
   cs_pin: GPIO44
   de_pin: GPIO18
diff --git a/tests/components/st7735/common.yaml b/tests/components/st7735/common.yaml
index c140652eda..242c4bccd6 100644
--- a/tests/components/st7735/common.yaml
+++ b/tests/components/st7735/common.yaml
@@ -1,8 +1,3 @@
-spi:
-  - id: spi_st7735
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   - platform: st7735
     model: INITR_18BLACKTAB
@@ -14,15 +9,15 @@ display:
     col_start: 0
     row_start: 0
     pages:
-      - id: page1
+      - id: st7735_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
-      - id: page2
+      - id: st7735_page2
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
     on_page_change:
-      from: page1
-      to: page2
+      from: st7735_page1
+      to: st7735_page2
       then:
         lambda: |-
           ESP_LOGD("display", "1 -> 2");
diff --git a/tests/components/st7735/test.esp32-ard.yaml b/tests/components/st7735/test.esp32-ard.yaml
deleted file mode 100644
index bad5241f79..0000000000
--- a/tests/components/st7735/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  cs_pin: GPIO12
-  dc_pin: GPIO13
-  reset_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/st7735/test.esp32-c3-ard.yaml b/tests/components/st7735/test.esp32-c3-ard.yaml
deleted file mode 100644
index c5c932c92c..0000000000
--- a/tests/components/st7735/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-  dc_pin: GPIO9
-  reset_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/st7735/test.esp32-c3-idf.yaml b/tests/components/st7735/test.esp32-c3-idf.yaml
index c5c932c92c..b112cf4c31 100644
--- a/tests/components/st7735/test.esp32-c3-idf.yaml
+++ b/tests/components/st7735/test.esp32-c3-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
   dc_pin: GPIO9
   reset_pin: GPIO10
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/st7735/test.esp32-idf.yaml b/tests/components/st7735/test.esp32-idf.yaml
index bad5241f79..ff174a4656 100644
--- a/tests/components/st7735/test.esp32-idf.yaml
+++ b/tests/components/st7735/test.esp32-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
   cs_pin: GPIO12
   dc_pin: GPIO13
   reset_pin: GPIO14
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/st7735/test.esp8266-ard.yaml b/tests/components/st7735/test.esp8266-ard.yaml
index 3f023a60eb..56cb29f29e 100644
--- a/tests/components/st7735/test.esp8266-ard.yaml
+++ b/tests/components/st7735/test.esp8266-ard.yaml
@@ -1,9 +1,12 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
   miso_pin: GPIO12
   cs_pin: GPIO5
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/st7735/test.rp2040-ard.yaml b/tests/components/st7735/test.rp2040-ard.yaml
index d7fd6ee294..66caa956f7 100644
--- a/tests/components/st7735/test.rp2040-ard.yaml
+++ b/tests/components/st7735/test.rp2040-ard.yaml
@@ -6,4 +6,7 @@ substitutions:
   dc_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/st7789v/common.yaml b/tests/components/st7789v/common.yaml
index d5f74809e0..19cef5656a 100644
--- a/tests/components/st7789v/common.yaml
+++ b/tests/components/st7789v/common.yaml
@@ -1,24 +1,20 @@
-spi:
-  - id: spi_st7789v
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   - platform: st7789v
     model: TTGO TDisplay 135x240
     cs_pin: ${cs_pin}
     dc_pin: ${dc_pin}
     reset_pin: ${reset_pin}
+    backlight_pin: ${backlight_pin}
     pages:
-      - id: page1
+      - id: st7789v_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
-      - id: page2
+      - id: st7789v_page2
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
     on_page_change:
-      from: page1
-      to: page2
+      from: st7789v_page1
+      to: st7789v_page2
       then:
         lambda: |-
           ESP_LOGD("display", "1 -> 2");
diff --git a/tests/components/st7789v/test.esp32-ard.yaml b/tests/components/st7789v/test.esp32-ard.yaml
deleted file mode 100644
index bad5241f79..0000000000
--- a/tests/components/st7789v/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  cs_pin: GPIO12
-  dc_pin: GPIO13
-  reset_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/st7789v/test.esp32-c3-ard.yaml b/tests/components/st7789v/test.esp32-c3-ard.yaml
deleted file mode 100644
index c5c932c92c..0000000000
--- a/tests/components/st7789v/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-  dc_pin: GPIO9
-  reset_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/st7789v/test.esp32-c3-idf.yaml b/tests/components/st7789v/test.esp32-c3-idf.yaml
index c5c932c92c..b4d70edb31 100644
--- a/tests/components/st7789v/test.esp32-c3-idf.yaml
+++ b/tests/components/st7789v/test.esp32-c3-idf.yaml
@@ -1,9 +1,10 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
   dc_pin: GPIO9
   reset_pin: GPIO10
+  backlight_pin: GPIO7
+
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/st7789v/test.esp32-idf.yaml b/tests/components/st7789v/test.esp32-idf.yaml
index bad5241f79..3b6d584e5c 100644
--- a/tests/components/st7789v/test.esp32-idf.yaml
+++ b/tests/components/st7789v/test.esp32-idf.yaml
@@ -1,8 +1,10 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
   cs_pin: GPIO12
   dc_pin: GPIO13
   reset_pin: GPIO14
+  backlight_pin: GPIO15
+
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/st7789v/test.esp8266-ard.yaml b/tests/components/st7789v/test.esp8266-ard.yaml
index 3f023a60eb..0ddca66ef4 100644
--- a/tests/components/st7789v/test.esp8266-ard.yaml
+++ b/tests/components/st7789v/test.esp8266-ard.yaml
@@ -1,9 +1,13 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
   miso_pin: GPIO12
   cs_pin: GPIO5
   dc_pin: GPIO15
   reset_pin: GPIO16
+  backlight_pin: GPIO4
+
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/st7789v/test.rp2040-ard.yaml b/tests/components/st7789v/test.rp2040-ard.yaml
index d7fd6ee294..464e720549 100644
--- a/tests/components/st7789v/test.rp2040-ard.yaml
+++ b/tests/components/st7789v/test.rp2040-ard.yaml
@@ -5,5 +5,9 @@ substitutions:
   cs_pin: GPIO5
   dc_pin: GPIO15
   reset_pin: GPIO16
+  backlight_pin: GPIO6
+
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/st7920/common.yaml b/tests/components/st7920/common.yaml
index 9ede271f01..477ac14bd4 100644
--- a/tests/components/st7920/common.yaml
+++ b/tests/components/st7920/common.yaml
@@ -1,23 +1,18 @@
-spi:
-  - id: spi_st7920
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   - platform: st7920
     cs_pin: ${cs_pin}
     height: 128
     width: 64
     pages:
-      - id: page1
+      - id: st7920_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
-      - id: page2
+      - id: st7920_page2
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
     on_page_change:
-      from: page1
-      to: page2
+      from: st7920_page1
+      to: st7920_page2
       then:
         lambda: |-
           ESP_LOGD("display", "1 -> 2");
diff --git a/tests/components/st7920/test.esp32-ard.yaml b/tests/components/st7920/test.esp32-ard.yaml
deleted file mode 100644
index 04d2633d2b..0000000000
--- a/tests/components/st7920/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
-  cs_pin: GPIO12
-
-<<: !include common.yaml
diff --git a/tests/components/st7920/test.esp32-c3-ard.yaml b/tests/components/st7920/test.esp32-c3-ard.yaml
deleted file mode 100644
index 2415ba5dc6..0000000000
--- a/tests/components/st7920/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
-  cs_pin: GPIO8
-
-<<: !include common.yaml
diff --git a/tests/components/st7920/test.esp32-c3-idf.yaml b/tests/components/st7920/test.esp32-c3-idf.yaml
index 2415ba5dc6..15d986e157 100644
--- a/tests/components/st7920/test.esp32-c3-idf.yaml
+++ b/tests/components/st7920/test.esp32-c3-idf.yaml
@@ -1,7 +1,6 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  miso_pin: GPIO5
   cs_pin: GPIO8
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/st7920/test.esp32-idf.yaml b/tests/components/st7920/test.esp32-idf.yaml
index 04d2633d2b..9bb524aa65 100644
--- a/tests/components/st7920/test.esp32-idf.yaml
+++ b/tests/components/st7920/test.esp32-idf.yaml
@@ -1,6 +1,7 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
   cs_pin: GPIO12
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/st7920/test.esp8266-ard.yaml b/tests/components/st7920/test.esp8266-ard.yaml
index bd5c203e35..1aac800592 100644
--- a/tests/components/st7920/test.esp8266-ard.yaml
+++ b/tests/components/st7920/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
-  cs_pin: GPIO5
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
+  miso_pin: GPIO15
+  cs_pin: GPIO16
+
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/st7920/test.rp2040-ard.yaml b/tests/components/st7920/test.rp2040-ard.yaml
index f6c3f1eeca..1ded24de1c 100644
--- a/tests/components/st7920/test.rp2040-ard.yaml
+++ b/tests/components/st7920/test.rp2040-ard.yaml
@@ -4,4 +4,7 @@ substitutions:
   miso_pin: GPIO4
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/statsD/test.esp32-ard.yaml b/tests/components/statsD/test.esp32-ard.yaml
deleted file mode 100644
index 25cb37a0b4..0000000000
--- a/tests/components/statsD/test.esp32-ard.yaml
+++ /dev/null
@@ -1,2 +0,0 @@
-packages:
-  common: !include common.yaml
diff --git a/tests/components/statsD/test.esp32-c3-ard.yaml b/tests/components/statsD/test.esp32-c3-ard.yaml
deleted file mode 100644
index 25cb37a0b4..0000000000
--- a/tests/components/statsD/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,2 +0,0 @@
-packages:
-  common: !include common.yaml
diff --git a/tests/components/status/test.esp32-ard.yaml b/tests/components/status/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/status/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/status/test.esp32-c3-ard.yaml b/tests/components/status/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/status/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/status_led/test.esp32-ard.yaml b/tests/components/status_led/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/status_led/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/status_led/test.esp32-c3-ard.yaml b/tests/components/status_led/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/status_led/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/stepper/test.esp32-ard.yaml b/tests/components/stepper/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/stepper/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/stepper/test.esp32-c3-ard.yaml b/tests/components/stepper/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/stepper/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/sts3x/common.yaml b/tests/components/sts3x/common.yaml
index 1feac4bc3f..1a61fa9212 100644
--- a/tests/components/sts3x/common.yaml
+++ b/tests/components/sts3x/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_sts3x
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: sts3x
+    i2c_id: i2c_bus
     id: sts3x_sensor
     name: STS3X Temperature
     address: 0x4A
diff --git a/tests/components/sts3x/test.esp32-ard.yaml b/tests/components/sts3x/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/sts3x/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sts3x/test.esp32-c3-ard.yaml b/tests/components/sts3x/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/sts3x/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sts3x/test.esp32-c3-idf.yaml b/tests/components/sts3x/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/sts3x/test.esp32-c3-idf.yaml
+++ b/tests/components/sts3x/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sts3x/test.esp32-idf.yaml b/tests/components/sts3x/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/sts3x/test.esp32-idf.yaml
+++ b/tests/components/sts3x/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sts3x/test.esp8266-ard.yaml b/tests/components/sts3x/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/sts3x/test.esp8266-ard.yaml
+++ b/tests/components/sts3x/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sts3x/test.rp2040-ard.yaml b/tests/components/sts3x/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/sts3x/test.rp2040-ard.yaml
+++ b/tests/components/sts3x/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sun/test.esp32-ard.yaml b/tests/components/sun/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/sun/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/sun/test.esp32-c3-ard.yaml b/tests/components/sun/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/sun/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/sun_gtil2/common.yaml b/tests/components/sun_gtil2/common.yaml
index 15cc892d90..37fd4b056f 100644
--- a/tests/components/sun_gtil2/common.yaml
+++ b/tests/components/sun_gtil2/common.yaml
@@ -1,8 +1,3 @@
-uart:
-  - id: uart_sun_gtil2
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 sun_gtil2:
 
 sensor:
diff --git a/tests/components/sun_gtil2/test.esp32-ard.yaml b/tests/components/sun_gtil2/test.esp32-ard.yaml
deleted file mode 100644
index ad420099ff..0000000000
--- a/tests/components/sun_gtil2/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/sun_gtil2/test.esp32-c3-ard.yaml b/tests/components/sun_gtil2/test.esp32-c3-ard.yaml
deleted file mode 100644
index b8a6b85616..0000000000
--- a/tests/components/sun_gtil2/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/sun_gtil2/test.esp32-c3-idf.yaml b/tests/components/sun_gtil2/test.esp32-c3-idf.yaml
index b8a6b85616..4b7c8351a7 100644
--- a/tests/components/sun_gtil2/test.esp32-c3-idf.yaml
+++ b/tests/components/sun_gtil2/test.esp32-c3-idf.yaml
@@ -1,4 +1,5 @@
 substitutions:
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sun_gtil2/test.esp32-idf.yaml b/tests/components/sun_gtil2/test.esp32-idf.yaml
index ad420099ff..29c7835b4e 100644
--- a/tests/components/sun_gtil2/test.esp32-idf.yaml
+++ b/tests/components/sun_gtil2/test.esp32-idf.yaml
@@ -1,4 +1,7 @@
 substitutions:
-  rx_pin: GPIO16
+  rx_pin: GPIO4
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sun_gtil2/test.esp8266-ard.yaml b/tests/components/sun_gtil2/test.esp8266-ard.yaml
index b8a6b85616..a591df42cb 100644
--- a/tests/components/sun_gtil2/test.esp8266-ard.yaml
+++ b/tests/components/sun_gtil2/test.esp8266-ard.yaml
@@ -1,4 +1,7 @@
 substitutions:
-  rx_pin: GPIO5
+  rx_pin: GPIO0
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sun_gtil2/test.rp2040-ard.yaml b/tests/components/sun_gtil2/test.rp2040-ard.yaml
index b8a6b85616..281ea9480e 100644
--- a/tests/components/sun_gtil2/test.rp2040-ard.yaml
+++ b/tests/components/sun_gtil2/test.rp2040-ard.yaml
@@ -1,4 +1,7 @@
 substitutions:
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/switch/common.yaml b/tests/components/switch/common.yaml
index 8d6972f91b..afdf26c150 100644
--- a/tests/components/switch/common.yaml
+++ b/tests/components/switch/common.yaml
@@ -9,3 +9,23 @@ switch:
     name: "Template Switch"
     id: the_switch
     optimistic: true
+    on_state:
+      - if:
+          condition:
+            - lambda: return x;
+          then:
+            - logger.log: "Switch turned ON"
+          else:
+            - logger.log: "Switch turned OFF"
+    on_turn_on:
+      - logger.log: "Switch is now ON"
+    on_turn_off:
+      - logger.log: "Switch is now OFF"
+
+esphome:
+  on_boot:
+    - switch.turn_on: the_switch
+    - switch.turn_off: the_switch
+    - switch.control:
+        id: the_switch
+        state: !lambda return (1 > 2);
diff --git a/tests/components/switch/test.esp32-ard.yaml b/tests/components/switch/test.esp32-ard.yaml
deleted file mode 100644
index 25cb37a0b4..0000000000
--- a/tests/components/switch/test.esp32-ard.yaml
+++ /dev/null
@@ -1,2 +0,0 @@
-packages:
-  common: !include common.yaml
diff --git a/tests/components/switch/test.esp32-c3-ard.yaml b/tests/components/switch/test.esp32-c3-ard.yaml
deleted file mode 100644
index 25cb37a0b4..0000000000
--- a/tests/components/switch/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,2 +0,0 @@
-packages:
-  common: !include common.yaml
diff --git a/tests/components/debug/test.esp32-s2-ard.yaml b/tests/components/switch/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/debug/test.esp32-s2-ard.yaml
rename to tests/components/switch/test.nrf52-adafruit.yaml
diff --git a/tests/components/debug/test.esp32-s3-ard.yaml b/tests/components/switch/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/debug/test.esp32-s3-ard.yaml
rename to tests/components/switch/test.nrf52-mcumgr.yaml
diff --git a/tests/components/sx126x/common.yaml b/tests/components/sx126x/common.yaml
index 3f888c3ce4..3f540a4bae 100644
--- a/tests/components/sx126x/common.yaml
+++ b/tests/components/sx126x/common.yaml
@@ -1,8 +1,3 @@
-spi:
-  clk_pin: ${clk_pin}
-  mosi_pin: ${mosi_pin}
-  miso_pin: ${miso_pin}
-
 sx126x:
   dio1_pin: ${dio1_pin}
   cs_pin: ${cs_pin}
@@ -11,6 +6,10 @@ sx126x:
   pa_power: 3
   bandwidth: 125_0kHz
   crc_enable: true
+  crc_initial: 0x1D0F
+  crc_polynomial: 0x1021
+  crc_size: 2
+  crc_inverted: true
   frequency: 433920000
   modulation: LORA
   rx_start: true
diff --git a/tests/components/sx126x/test.esp32-ard.yaml b/tests/components/sx126x/test.esp32-ard.yaml
deleted file mode 100644
index 9770f52229..0000000000
--- a/tests/components/sx126x/test.esp32-ard.yaml
+++ /dev/null
@@ -1,10 +0,0 @@
-substitutions:
-  clk_pin: GPIO5
-  mosi_pin: GPIO27
-  miso_pin: GPIO19
-  cs_pin: GPIO18
-  rst_pin: GPIO23
-  busy_pin: GPIO25
-  dio1_pin: GPIO26
-
-<<: !include common.yaml
diff --git a/tests/components/sx126x/test.esp32-c3-ard.yaml b/tests/components/sx126x/test.esp32-c3-ard.yaml
deleted file mode 100644
index 91450e24ce..0000000000
--- a/tests/components/sx126x/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,10 +0,0 @@
-substitutions:
-  clk_pin: GPIO5
-  mosi_pin: GPIO18
-  miso_pin: GPIO19
-  cs_pin: GPIO1
-  rst_pin: GPIO2
-  busy_pin: GPIO4
-  dio1_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/sx126x/test.esp32-c3-idf.yaml b/tests/components/sx126x/test.esp32-c3-idf.yaml
index 91450e24ce..e27f11032e 100644
--- a/tests/components/sx126x/test.esp32-c3-idf.yaml
+++ b/tests/components/sx126x/test.esp32-c3-idf.yaml
@@ -1,10 +1,10 @@
 substitutions:
-  clk_pin: GPIO5
-  mosi_pin: GPIO18
-  miso_pin: GPIO19
   cs_pin: GPIO1
   rst_pin: GPIO2
-  busy_pin: GPIO4
+  busy_pin: GPIO7
   dio1_pin: GPIO3
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/sx126x/test.esp32-idf.yaml b/tests/components/sx126x/test.esp32-idf.yaml
index 9770f52229..854638ea9c 100644
--- a/tests/components/sx126x/test.esp32-idf.yaml
+++ b/tests/components/sx126x/test.esp32-idf.yaml
@@ -1,10 +1,10 @@
 substitutions:
-  clk_pin: GPIO5
-  mosi_pin: GPIO27
-  miso_pin: GPIO19
-  cs_pin: GPIO18
-  rst_pin: GPIO23
+  cs_pin: GPIO12
+  rst_pin: GPIO13
   busy_pin: GPIO25
   dio1_pin: GPIO26
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/sx126x/test.esp8266-ard.yaml b/tests/components/sx126x/test.esp8266-ard.yaml
index d2c07c5bb7..98a3fbbd67 100644
--- a/tests/components/sx126x/test.esp8266-ard.yaml
+++ b/tests/components/sx126x/test.esp8266-ard.yaml
@@ -1,10 +1,13 @@
 substitutions:
-  clk_pin: GPIO5
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO15
+  miso_pin: GPIO16
   cs_pin: GPIO1
   rst_pin: GPIO2
   busy_pin: GPIO4
   dio1_pin: GPIO3
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/sx126x/test.rp2040-ard.yaml b/tests/components/sx126x/test.rp2040-ard.yaml
index 8881e96971..9bc12a370d 100644
--- a/tests/components/sx126x/test.rp2040-ard.yaml
+++ b/tests/components/sx126x/test.rp2040-ard.yaml
@@ -7,4 +7,7 @@ substitutions:
   busy_pin: GPIO8
   dio1_pin: GPIO7
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/sx127x/common.yaml b/tests/components/sx127x/common.yaml
index 63adc2e91c..540381fc08 100644
--- a/tests/components/sx127x/common.yaml
+++ b/tests/components/sx127x/common.yaml
@@ -1,8 +1,3 @@
-spi:
-  clk_pin: ${clk_pin}
-  mosi_pin: ${mosi_pin}
-  miso_pin: ${miso_pin}
-
 sx127x:
   cs_pin: ${cs_pin}
   rst_pin: ${rst_pin}
diff --git a/tests/components/sx127x/test.esp32-ard.yaml b/tests/components/sx127x/test.esp32-ard.yaml
deleted file mode 100644
index 71270462a2..0000000000
--- a/tests/components/sx127x/test.esp32-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO5
-  mosi_pin: GPIO27
-  miso_pin: GPIO19
-  cs_pin: GPIO18
-  rst_pin: GPIO23
-  dio0_pin: GPIO26
-
-<<: !include common.yaml
diff --git a/tests/components/sx127x/test.esp32-c3-ard.yaml b/tests/components/sx127x/test.esp32-c3-ard.yaml
deleted file mode 100644
index 36535a950d..0000000000
--- a/tests/components/sx127x/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO5
-  mosi_pin: GPIO18
-  miso_pin: GPIO19
-  cs_pin: GPIO1
-  rst_pin: GPIO2
-  dio0_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/sx127x/test.esp32-c3-idf.yaml b/tests/components/sx127x/test.esp32-c3-idf.yaml
index 36535a950d..dfee192545 100644
--- a/tests/components/sx127x/test.esp32-c3-idf.yaml
+++ b/tests/components/sx127x/test.esp32-c3-idf.yaml
@@ -1,9 +1,8 @@
 substitutions:
-  clk_pin: GPIO5
-  mosi_pin: GPIO18
-  miso_pin: GPIO19
   cs_pin: GPIO1
   rst_pin: GPIO2
   dio0_pin: GPIO3
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sx127x/test.esp32-idf.yaml b/tests/components/sx127x/test.esp32-idf.yaml
index 71270462a2..c9d58bb27e 100644
--- a/tests/components/sx127x/test.esp32-idf.yaml
+++ b/tests/components/sx127x/test.esp32-idf.yaml
@@ -1,9 +1,9 @@
 substitutions:
-  clk_pin: GPIO5
-  mosi_pin: GPIO27
-  miso_pin: GPIO19
-  cs_pin: GPIO18
-  rst_pin: GPIO23
+  cs_pin: GPIO12
+  rst_pin: GPIO13
   dio0_pin: GPIO26
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/sx127x/test.esp8266-ard.yaml b/tests/components/sx127x/test.esp8266-ard.yaml
index 64c01edd44..d58166137c 100644
--- a/tests/components/sx127x/test.esp8266-ard.yaml
+++ b/tests/components/sx127x/test.esp8266-ard.yaml
@@ -1,9 +1,12 @@
 substitutions:
-  clk_pin: GPIO5
-  mosi_pin: GPIO13
-  miso_pin: GPIO12
+  clk_pin: GPIO0
+  mosi_pin: GPIO15
+  miso_pin: GPIO16
   cs_pin: GPIO1
   rst_pin: GPIO2
   dio0_pin: GPIO3
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/sx127x/test.rp2040-ard.yaml b/tests/components/sx127x/test.rp2040-ard.yaml
index 0af7b29790..09a9b3203b 100644
--- a/tests/components/sx127x/test.rp2040-ard.yaml
+++ b/tests/components/sx127x/test.rp2040-ard.yaml
@@ -6,4 +6,7 @@ substitutions:
   rst_pin: GPIO6
   dio0_pin: GPIO7
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/sx1509/common.yaml b/tests/components/sx1509/common.yaml
index a83217e579..cf7e234f09 100644
--- a/tests/components/sx1509/common.yaml
+++ b/tests/components/sx1509/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_sx1509
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sx1509:
   - id: sx1509_hub
+    i2c_id: i2c_bus
     address: 0x3E
     keypad:
       key_rows: 2
diff --git a/tests/components/sx1509/test.esp32-ard.yaml b/tests/components/sx1509/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/sx1509/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/sx1509/test.esp32-c3-ard.yaml b/tests/components/sx1509/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/sx1509/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/sx1509/test.esp32-c3-idf.yaml b/tests/components/sx1509/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/sx1509/test.esp32-c3-idf.yaml
+++ b/tests/components/sx1509/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sx1509/test.esp32-idf.yaml b/tests/components/sx1509/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/sx1509/test.esp32-idf.yaml
+++ b/tests/components/sx1509/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sx1509/test.esp8266-ard.yaml b/tests/components/sx1509/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/sx1509/test.esp8266-ard.yaml
+++ b/tests/components/sx1509/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/sx1509/test.rp2040-ard.yaml b/tests/components/sx1509/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/sx1509/test.rp2040-ard.yaml
+++ b/tests/components/sx1509/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/syslog/common.yaml b/tests/components/syslog/common.yaml
index cd6e63c9ec..daa913f009 100644
--- a/tests/components/syslog/common.yaml
+++ b/tests/components/syslog/common.yaml
@@ -6,7 +6,8 @@ udp:
   addresses: ["239.0.60.53"]
 
 time:
-  platform: host
+  - platform: host
+    id: host_time
 
 syslog:
   port: 514
diff --git a/tests/components/syslog/test.esp32-ard.yaml b/tests/components/syslog/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/syslog/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/syslog/test.esp32-c3-ard.yaml b/tests/components/syslog/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/syslog/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/syslog/test.host.yaml b/tests/components/syslog/test.host.yaml
index e735c37e4d..31122437d5 100644
--- a/tests/components/syslog/test.host.yaml
+++ b/tests/components/syslog/test.host.yaml
@@ -1,4 +1,11 @@
-packages:
-  common: !include common.yaml
+udp:
+  addresses: ["239.0.60.53"]
 
-wifi: !remove
+time:
+  platform: host
+
+syslog:
+  port: 514
+  strip: true
+  level: info
+  facility: 16
diff --git a/tests/components/t6615/common.yaml b/tests/components/t6615/common.yaml
index 3ad715ae2b..4317130461 100644
--- a/tests/components/t6615/common.yaml
+++ b/tests/components/t6615/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_t6615
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 19200
-
 sensor:
   - platform: t6615
     co2:
diff --git a/tests/components/t6615/test.esp32-ard.yaml b/tests/components/t6615/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/t6615/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/t6615/test.esp32-c3-ard.yaml b/tests/components/t6615/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/t6615/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/t6615/test.esp32-c3-idf.yaml b/tests/components/t6615/test.esp32-c3-idf.yaml
index b516342f3b..147d967dd4 100644
--- a/tests/components/t6615/test.esp32-c3-idf.yaml
+++ b/tests/components/t6615/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart_19200: !include ../../test_build_components/common/uart_19200/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/t6615/test.esp32-idf.yaml b/tests/components/t6615/test.esp32-idf.yaml
index f486544afa..4bb9e7fff6 100644
--- a/tests/components/t6615/test.esp32-idf.yaml
+++ b/tests/components/t6615/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart_19200: !include ../../test_build_components/common/uart_19200/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/t6615/test.esp8266-ard.yaml b/tests/components/t6615/test.esp8266-ard.yaml
index b516342f3b..1dd8409590 100644
--- a/tests/components/t6615/test.esp8266-ard.yaml
+++ b/tests/components/t6615/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart_19200: !include ../../test_build_components/common/uart_19200/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/t6615/test.rp2040-ard.yaml b/tests/components/t6615/test.rp2040-ard.yaml
index b516342f3b..f4dada6605 100644
--- a/tests/components/t6615/test.rp2040-ard.yaml
+++ b/tests/components/t6615/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart_19200: !include ../../test_build_components/common/uart_19200/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/tc74/common.yaml b/tests/components/tc74/common.yaml
index 88f5c91e12..c2430ee2d4 100644
--- a/tests/components/tc74/common.yaml
+++ b/tests/components/tc74/common.yaml
@@ -1,8 +1,4 @@
-i2c:
-  - id: i2c_tc74
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: tc74
+    i2c_id: i2c_bus
     name: TC74 Temperature
diff --git a/tests/components/tc74/test.esp32-ard.yaml b/tests/components/tc74/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/tc74/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/tc74/test.esp32-c3-ard.yaml b/tests/components/tc74/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/tc74/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/tc74/test.esp32-c3-idf.yaml b/tests/components/tc74/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/tc74/test.esp32-c3-idf.yaml
+++ b/tests/components/tc74/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tc74/test.esp32-idf.yaml b/tests/components/tc74/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/tc74/test.esp32-idf.yaml
+++ b/tests/components/tc74/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tc74/test.esp8266-ard.yaml b/tests/components/tc74/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/tc74/test.esp8266-ard.yaml
+++ b/tests/components/tc74/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tc74/test.rp2040-ard.yaml b/tests/components/tc74/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/tc74/test.rp2040-ard.yaml
+++ b/tests/components/tc74/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tca9548a/common.yaml b/tests/components/tca9548a/common.yaml
index 67e812e08b..bfaf1e26fb 100644
--- a/tests/components/tca9548a/common.yaml
+++ b/tests/components/tca9548a/common.yaml
@@ -1,15 +1,10 @@
-i2c:
-  - id: i2c_tca9548a
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 tca9548a:
   - id: multiplex0
+    i2c_id: i2c_bus
     address: 0x70
     channels:
       - bus_id: multiplex0_chan0
         channel: 0
-    i2c_id: i2c_tca9548a
   - id: multiplex1
+    i2c_id: i2c_bus
     address: 0x71
-    i2c_id: multiplex0_chan0
diff --git a/tests/components/tca9548a/test.esp32-ard.yaml b/tests/components/tca9548a/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/tca9548a/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/tca9548a/test.esp32-c3-ard.yaml b/tests/components/tca9548a/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/tca9548a/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/tca9548a/test.esp32-c3-idf.yaml b/tests/components/tca9548a/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/tca9548a/test.esp32-c3-idf.yaml
+++ b/tests/components/tca9548a/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tca9548a/test.esp32-idf.yaml b/tests/components/tca9548a/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/tca9548a/test.esp32-idf.yaml
+++ b/tests/components/tca9548a/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tca9548a/test.esp8266-ard.yaml b/tests/components/tca9548a/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/tca9548a/test.esp8266-ard.yaml
+++ b/tests/components/tca9548a/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tca9548a/test.rp2040-ard.yaml b/tests/components/tca9548a/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/tca9548a/test.rp2040-ard.yaml
+++ b/tests/components/tca9548a/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tca9555/common.yaml b/tests/components/tca9555/common.yaml
index 0fc3086786..82b4c959d8 100644
--- a/tests/components/tca9555/common.yaml
+++ b/tests/components/tca9555/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_tca9555
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 tca9555:
   - id: tca9555_hub
+    i2c_id: i2c_bus
     address: 0x21
 
 binary_sensor:
diff --git a/tests/components/tca9555/test.esp32-ard.yaml b/tests/components/tca9555/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/tca9555/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/tca9555/test.esp32-c3-ard.yaml b/tests/components/tca9555/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/tca9555/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/tca9555/test.esp32-c3-idf.yaml b/tests/components/tca9555/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/tca9555/test.esp32-c3-idf.yaml
+++ b/tests/components/tca9555/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tca9555/test.esp32-idf.yaml b/tests/components/tca9555/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/tca9555/test.esp32-idf.yaml
+++ b/tests/components/tca9555/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tca9555/test.esp8266-ard.yaml b/tests/components/tca9555/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/tca9555/test.esp8266-ard.yaml
+++ b/tests/components/tca9555/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tca9555/test.rp2040-ard.yaml b/tests/components/tca9555/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/tca9555/test.rp2040-ard.yaml
+++ b/tests/components/tca9555/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tcl112/common.yaml b/tests/components/tcl112/common.yaml
index 0e43de4a4a..1074712f94 100644
--- a/tests/components/tcl112/common.yaml
+++ b/tests/components/tcl112/common.yaml
@@ -1,7 +1,3 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 sensor:
   - platform: template
     id: tcl112_sensor
@@ -13,3 +9,4 @@ climate:
     supports_heat: true
     supports_cool: true
     sensor: tcl112_sensor
+    transmitter_id: xmitr
diff --git a/tests/components/tcl112/test.esp32-ard.yaml b/tests/components/tcl112/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/tcl112/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/tcl112/test.esp32-c3-ard.yaml b/tests/components/tcl112/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/tcl112/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/tcl112/test.esp32-c3-idf.yaml b/tests/components/tcl112/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/tcl112/test.esp32-c3-idf.yaml
+++ b/tests/components/tcl112/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tcl112/test.esp32-idf.yaml b/tests/components/tcl112/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/tcl112/test.esp32-idf.yaml
+++ b/tests/components/tcl112/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tcl112/test.esp8266-ard.yaml b/tests/components/tcl112/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/tcl112/test.esp8266-ard.yaml
+++ b/tests/components/tcl112/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tcs34725/common.yaml b/tests/components/tcs34725/common.yaml
index 5296988fa5..e16862035e 100644
--- a/tests/components/tcs34725/common.yaml
+++ b/tests/components/tcs34725/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_tcs34725
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: tcs34725
+    i2c_id: i2c_bus
     red_channel:
       name: Red Channel
     green_channel:
diff --git a/tests/components/tcs34725/test.esp32-ard.yaml b/tests/components/tcs34725/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/tcs34725/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/tcs34725/test.esp32-c3-ard.yaml b/tests/components/tcs34725/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/tcs34725/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/tcs34725/test.esp32-c3-idf.yaml b/tests/components/tcs34725/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/tcs34725/test.esp32-c3-idf.yaml
+++ b/tests/components/tcs34725/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tcs34725/test.esp32-idf.yaml b/tests/components/tcs34725/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/tcs34725/test.esp32-idf.yaml
+++ b/tests/components/tcs34725/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tcs34725/test.esp8266-ard.yaml b/tests/components/tcs34725/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/tcs34725/test.esp8266-ard.yaml
+++ b/tests/components/tcs34725/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tcs34725/test.rp2040-ard.yaml b/tests/components/tcs34725/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/tcs34725/test.rp2040-ard.yaml
+++ b/tests/components/tcs34725/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tee501/common.yaml b/tests/components/tee501/common.yaml
index c01ab7e37a..82091faccf 100644
--- a/tests/components/tee501/common.yaml
+++ b/tests/components/tee501/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_tee501
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: tee501
+    i2c_id: i2c_bus
     name: TEE501 Temperature
     address: 0x48
diff --git a/tests/components/tee501/test.esp32-ard.yaml b/tests/components/tee501/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/tee501/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/tee501/test.esp32-c3-ard.yaml b/tests/components/tee501/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/tee501/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/tee501/test.esp32-c3-idf.yaml b/tests/components/tee501/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/tee501/test.esp32-c3-idf.yaml
+++ b/tests/components/tee501/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tee501/test.esp32-idf.yaml b/tests/components/tee501/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/tee501/test.esp32-idf.yaml
+++ b/tests/components/tee501/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tee501/test.esp8266-ard.yaml b/tests/components/tee501/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/tee501/test.esp8266-ard.yaml
+++ b/tests/components/tee501/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tee501/test.rp2040-ard.yaml b/tests/components/tee501/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/tee501/test.rp2040-ard.yaml
+++ b/tests/components/tee501/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/teleinfo/common.yaml b/tests/components/teleinfo/common.yaml
index 90b684e977..dcb2cb14bd 100644
--- a/tests/components/teleinfo/common.yaml
+++ b/tests/components/teleinfo/common.yaml
@@ -1,10 +1,3 @@
-uart:
-  - id: uart_teleinfo
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 1200
-    parity: EVEN
-
 button:
   - platform: template
     name: Poller component suspend test
diff --git a/tests/components/teleinfo/test.esp32-ard.yaml b/tests/components/teleinfo/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/teleinfo/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/teleinfo/test.esp32-c3-ard.yaml b/tests/components/teleinfo/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/teleinfo/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/teleinfo/test.esp32-c3-idf.yaml b/tests/components/teleinfo/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/teleinfo/test.esp32-c3-idf.yaml
+++ b/tests/components/teleinfo/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/teleinfo/test.esp32-idf.yaml b/tests/components/teleinfo/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/teleinfo/test.esp32-idf.yaml
+++ b/tests/components/teleinfo/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/teleinfo/test.esp8266-ard.yaml b/tests/components/teleinfo/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/teleinfo/test.esp8266-ard.yaml
+++ b/tests/components/teleinfo/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/teleinfo/test.rp2040-ard.yaml b/tests/components/teleinfo/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/teleinfo/test.rp2040-ard.yaml
+++ b/tests/components/teleinfo/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/tem3200/common.yaml b/tests/components/tem3200/common.yaml
index 392c853bf4..cef7317450 100644
--- a/tests/components/tem3200/common.yaml
+++ b/tests/components/tem3200/common.yaml
@@ -1,13 +1,7 @@
-i2c:
-  id: i2c_bus
-  scl: ${scl_pin}
-  sda: ${sda_pin}
-  frequency: 200kHz
-
 sensor:
   - platform: tem3200
-    update_interval: 1s
     i2c_id: i2c_bus
+    update_interval: 1s
 
     temperature:
       name: water temperature
diff --git a/tests/components/tem3200/test.esp32-ard.yaml b/tests/components/tem3200/test.esp32-ard.yaml
deleted file mode 100644
index 3b761d3fc1..0000000000
--- a/tests/components/tem3200/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
-
-<<: !include common.yaml
diff --git a/tests/components/tem3200/test.esp32-idf.yaml b/tests/components/tem3200/test.esp32-idf.yaml
index 3b761d3fc1..b47e39c389 100644
--- a/tests/components/tem3200/test.esp32-idf.yaml
+++ b/tests/components/tem3200/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tem3200/test.esp32-s3-ard.yaml b/tests/components/tem3200/test.esp32-s3-ard.yaml
deleted file mode 100644
index 4942e3c2b3..0000000000
--- a/tests/components/tem3200/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO40
-  sda_pin: GPIO41
-
-<<: !include common.yaml
diff --git a/tests/components/tem3200/test.esp32-s3-idf.yaml b/tests/components/tem3200/test.esp32-s3-idf.yaml
index 4942e3c2b3..e9d826aa7c 100644
--- a/tests/components/tem3200/test.esp32-s3-idf.yaml
+++ b/tests/components/tem3200/test.esp32-s3-idf.yaml
@@ -2,4 +2,7 @@ substitutions:
   scl_pin: GPIO40
   sda_pin: GPIO41
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-s3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/tem3200/test.esp8266-ard.yaml b/tests/components/tem3200/test.esp8266-ard.yaml
index 3be5e53dcb..4a98b9388a 100644
--- a/tests/components/tem3200/test.esp8266-ard.yaml
+++ b/tests/components/tem3200/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO05
-  sda_pin: GPIO04
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/template/common-base.yaml b/tests/components/template/common-base.yaml
new file mode 100644
index 0000000000..ea812532d4
--- /dev/null
+++ b/tests/components/template/common-base.yaml
@@ -0,0 +1,336 @@
+esphome:
+  on_boot:
+    - sensor.template.publish:
+        id: template_sens
+        state: 42.0
+
+    # Templated
+    - sensor.template.publish:
+        id: template_sens
+        state: !lambda "return 42.0;"
+
+    - datetime.date.set:
+        id: test_date
+        date:
+          year: 2021
+          month: 1
+          day: 1
+    - datetime.date.set:
+        id: test_date
+        date: !lambda "return {.day_of_month = 1, .month = 1, .year = 2021};"
+    - datetime.date.set:
+        id: test_date
+        date: "2021-01-01"
+
+binary_sensor:
+  - platform: template
+    id: some_binary_sensor
+    name: "Garage Door Open"
+    lambda: |-
+      if (id(template_sens).state > 30) {
+        // Garage Door is open.
+        return true;
+      } else {
+        // Garage Door is closed.
+        return false;
+      }
+  - platform: template
+    id: other_binary_sensor
+    name: "Garage Door Closed"
+    condition:
+      sensor.in_range:
+        id: template_sens
+        below: 30.0
+    filters:
+      - invert:
+      - delayed_on: 100ms
+      - delayed_off: 100ms
+      - delayed_on_off: !lambda "if (id(test_switch).state) return 1000; else return 0;"
+      - delayed_on_off:
+          time_on: 10s
+          time_off: !lambda "if (id(test_switch).state) return 1000; else return 0;"
+      - autorepeat:
+          - delay: 1s
+            time_off: 100ms
+            time_on: 900ms
+          - delay: 5s
+            time_off: 100ms
+            time_on: 400ms
+      - lambda: |-
+          if (id(other_binary_sensor).state) {
+            return x;
+          }
+          return {};
+      - settle: 500ms
+      - timeout: 5s
+
+sensor:
+  - platform: template
+    name: "Template Sensor"
+    id: template_sens
+    lambda: |-
+      if (id(some_binary_sensor).state) {
+        return 42.0;
+      }
+      return 0.0;
+    update_interval: 60s
+    filters:
+      - calibrate_linear:
+          - 0.0 -> 0.0
+          - 40.0 -> 45.0
+          - 100.0 -> 102.5
+      - calibrate_polynomial:
+          degree: 2
+          datapoints:
+            # Map 0.0 (from sensor) to 0.0 (true value)
+            - 0.0 -> 0.0
+            - 10.0 -> 12.1
+            - 13.0 -> 14.0
+      - clamp:
+          max_value: 10.0
+          min_value: -10.0
+      - debounce: 0.1s
+      - delta: 5.0
+      - exponential_moving_average:
+          alpha: 0.1
+          send_every: 15
+      - filter_out:
+          - 10
+          - 20
+          - !lambda return 10;
+      - filter_out: 10
+      - filter_out: !lambda return NAN;
+      - heartbeat: 5s
+      - lambda: return x * (9.0/5.0) + 32.0;
+      - max:
+          window_size: 10
+          send_every: 2
+          send_first_at: 1
+      - median:
+          window_size: 7
+          send_every: 4
+          send_first_at: 3
+      - min:
+          window_size: 10
+          send_every: 2
+          send_first_at: 1
+      - multiply: 1
+      - multiply: !lambda return 2;
+      - offset: 10
+      - offset: !lambda return 10;
+      - or:
+          - quantile:
+              window_size: 7
+              send_every: 4
+              send_first_at: 3
+              quantile: .9
+          - round: 1
+      - round_to_multiple_of: 0.25
+      - skip_initial: 3
+      - sliding_window_moving_average:
+          window_size: 15
+          send_every: 15
+      - throttle: 1s
+      - throttle_average: 2s
+      - throttle_with_priority: 5s
+      - throttle_with_priority:
+          timeout: 3s
+          value: 42.0
+      - throttle_with_priority:
+          timeout: 3s
+          value: !lambda return 1.0f / 2.0f;
+      - throttle_with_priority:
+          timeout: 3s
+          value:
+            - 42.0
+            - !lambda return 2.0f / 2.0f;
+            - nan
+      - timeout:
+          timeout: 10s
+          value: !lambda return 10;
+      - timeout:
+          timeout: 1h
+          value: 20.0
+      - timeout:
+          timeout: 1min
+          value: last
+      - timeout:
+          timeout: 1d
+      - to_ntc_resistance:
+          calibration:
+            - 10.0kOhm -> 25°C
+            - 27.219kOhm -> 0°C
+            - 14.674kOhm -> 15°C
+      - to_ntc_temperature:
+          calibration:
+            - 10.0kOhm -> 25°C
+            - 27.219kOhm -> 0°C
+            - 14.674kOhm -> 15°C
+
+output:
+  - platform: template
+    id: outputsplit
+    type: float
+    write_action:
+      - logger.log: "write_action"
+
+switch:
+  - platform: template
+    id: test_switch
+    name: "Template Switch"
+    lambda: |-
+      if (id(some_binary_sensor).state) {
+        return true;
+      }
+      return false;
+    turn_on_action:
+      - logger.log: "turn_on_action"
+    turn_off_action:
+      - logger.log: "turn_off_action"
+
+button:
+  - platform: template
+    name: "Template Button"
+    on_press:
+      - logger.log: Button Pressed
+
+cover:
+  - platform: template
+    name: "Template Cover"
+    lambda: |-
+      if (id(some_binary_sensor).state) {
+        return COVER_OPEN;
+      }
+      return COVER_CLOSED;
+    open_action:
+      - logger.log: open_action
+    close_action:
+      - logger.log: close_action
+    stop_action:
+      - logger.log: stop_action
+    optimistic: true
+
+number:
+  - platform: template
+    name: "Template number"
+    optimistic: true
+    min_value: 0
+    max_value: 100
+    step: 1
+
+select:
+  - platform: template
+    name: "Template select"
+    optimistic: true
+    options:
+      - one
+      - two
+      - three
+    initial_option: two
+
+lock:
+  - platform: template
+    name: "Template Lock"
+    lambda: |-
+      if (id(some_binary_sensor).state) {
+        return LOCK_STATE_LOCKED;
+      }
+      return LOCK_STATE_UNLOCKED;
+    lock_action:
+      - logger.log: lock_action
+    unlock_action:
+      - logger.log: unlock_action
+    open_action:
+      - logger.log: open_action
+
+valve:
+  - platform: template
+    id: template_valve
+    name: "Template Valve"
+    lambda: |-
+      if (id(some_binary_sensor).state) {
+        return VALVE_OPEN;
+      }
+      return VALVE_CLOSED;
+    open_action:
+      - logger.log: open_action
+    close_action:
+      - logger.log: close_action
+      - valve.template.publish:
+          id: template_valve
+          state: CLOSED
+    stop_action:
+      - logger.log: stop_action
+    optimistic: true
+
+text:
+  - platform: template
+    name: "Template text"
+    optimistic: true
+    min_length: 0
+    max_length: 100
+    mode: text
+  - platform: template
+    name: "Template text lambda"
+    mode: text
+    update_interval: 1s
+    lambda: |
+      return std::string{"Hello!"};
+    set_action:
+      then:
+        - logger.log:
+            format: Template Text set to %s
+            args: ["x.c_str()"]
+
+alarm_control_panel:
+  - platform: template
+    name: Alarm Panel
+    codes:
+      - "1234"
+
+datetime:
+  - platform: template
+    name: Date
+    id: test_date
+    type: date
+    initial_value: "2000-1-2"
+    set_action:
+      - logger.log: "set_value"
+    on_value:
+      - logger.log:
+          format: "Date: %04d-%02d-%02d"
+          args:
+            - x.year
+            - x.month
+            - x.day_of_month
+  - platform: template
+    name: Time
+    id: test_time
+    type: time
+    initial_value: "12:34:56am"
+    set_action:
+      - logger.log: "set_value"
+    on_value:
+      - logger.log:
+          format: "Time: %02d:%02d:%02d"
+          args:
+            - x.hour
+            - x.minute
+            - x.second
+  - platform: template
+    name: DateTime
+    id: test_datetime
+    type: datetime
+    initial_value: "2000-1-2 12:34:56"
+    set_action:
+      - logger.log: "set_value"
+    on_value:
+      - logger.log:
+          format: "DateTime: %04d-%02d-%02d %02d:%02d:%02d"
+          args:
+            - x.year
+            - x.month
+            - x.day_of_month
+            - x.hour
+            - x.minute
+            - x.second
diff --git a/tests/components/template/common.yaml b/tests/components/template/common.yaml
index fd9342b3e5..d06f3ce131 100644
--- a/tests/components/template/common.yaml
+++ b/tests/components/template/common.yaml
@@ -1,259 +1,8 @@
-sensor:
-  - platform: template
-    name: "Template Sensor"
-    id: template_sens
-    lambda: |-
-      if (id(some_binary_sensor).state) {
-        return 42.0;
-      } else {
-        return 0.0;
-      }
-    update_interval: 60s
-    filters:
-      - offset: 10
-      - multiply: 1
-      - offset: !lambda return 10;
-      - multiply: !lambda return 2;
-      - filter_out:
-          - 10
-          - 20
-          - !lambda return 10;
-      - filter_out: 10
-      - filter_out: !lambda return NAN;
-      - timeout:
-          timeout: 10s
-          value: !lambda return 10;
-      - timeout:
-          timeout: 1h
-          value: 20.0
-      - timeout:
-          timeout: 1d
-      - to_ntc_resistance:
-          calibration:
-            - 10.0kOhm -> 25°C
-            - 27.219kOhm -> 0°C
-            - 14.674kOhm -> 15°C
-      - to_ntc_temperature:
-          calibration:
-            - 10.0kOhm -> 25°C
-            - 27.219kOhm -> 0°C
-            - 14.674kOhm -> 15°C
-
-esphome:
-  on_boot:
-    - sensor.template.publish:
-        id: template_sens
-        state: 42.0
-
-    # Templated
-    - sensor.template.publish:
-        id: template_sens
-        state: !lambda "return 42.0;"
-
-    - datetime.date.set:
-        id: test_date
-        date:
-          year: 2021
-          month: 1
-          day: 1
-    - datetime.date.set:
-        id: test_date
-        date: !lambda "return {.day_of_month = 1, .month = 1, .year = 2021};"
-    - datetime.date.set:
-        id: test_date
-        date: "2021-01-01"
-
-binary_sensor:
-  - platform: template
-    id: some_binary_sensor
-    name: "Garage Door Open"
-    lambda: |-
-      if (id(template_sens).state > 30) {
-        // Garage Door is open.
-        return true;
-      } else {
-        // Garage Door is closed.
-        return false;
-      }
-  - platform: template
-    id: other_binary_sensor
-    name: "Garage Door Closed"
-    condition:
-      sensor.in_range:
-        id: template_sens
-        below: 30.0
-
-output:
-  - platform: template
-    id: outputsplit
-    type: float
-    write_action:
-      - logger.log: "write_action"
-
-switch:
-  - platform: template
-    name: "Template Switch"
-    lambda: |-
-      if (id(some_binary_sensor).state) {
-        return true;
-      } else {
-        return false;
-      }
-    turn_on_action:
-      - logger.log: "turn_on_action"
-    turn_off_action:
-      - logger.log: "turn_off_action"
-
-button:
-  - platform: template
-    name: "Template Button"
-    on_press:
-      - logger.log: Button Pressed
-
-cover:
-  - platform: template
-    name: "Template Cover"
-    lambda: |-
-      if (id(some_binary_sensor).state) {
-        return COVER_OPEN;
-      } else {
-        return COVER_CLOSED;
-      }
-    open_action:
-      - logger.log: open_action
-    close_action:
-      - logger.log: close_action
-    stop_action:
-      - logger.log: stop_action
-    optimistic: true
-
-number:
-  - platform: template
-    name: "Template number"
-    optimistic: true
-    min_value: 0
-    max_value: 100
-    step: 1
-
-select:
-  - platform: template
-    name: "Template select"
-    optimistic: true
-    options:
-      - one
-      - two
-      - three
-    initial_option: two
-
-lock:
-  - platform: template
-    name: "Template Lock"
-    lambda: |-
-      if (id(some_binary_sensor).state) {
-        return LOCK_STATE_LOCKED;
-      } else {
-        return LOCK_STATE_UNLOCKED;
-      }
-    lock_action:
-      - logger.log: lock_action
-    unlock_action:
-      - logger.log: unlock_action
-    open_action:
-      - logger.log: open_action
-
-valve:
-  - platform: template
-    name: "Template Valve"
-    lambda: |-
-      if (id(some_binary_sensor).state) {
-        return VALVE_OPEN;
-      } else {
-        return VALVE_CLOSED;
-      }
-    open_action:
-      - logger.log: open_action
-    close_action:
-      - logger.log: close_action
-      - valve.template.publish:
-          state: CLOSED
-    stop_action:
-      - logger.log: stop_action
-    optimistic: true
-
-text:
-  - platform: template
-    name: "Template text"
-    optimistic: true
-    min_length: 0
-    max_length: 100
-    mode: text
-  - platform: template
-    name: "Template text lambda"
-    mode: text
-    update_interval: 1s
-    lambda: |
-      return std::string{"Hello!"};
-    set_action:
-      then:
-        - logger.log:
-            format: Template Text set to %s
-            args: ["x.c_str()"]
-
-alarm_control_panel:
-  - platform: template
-    name: Alarm Panel
-    codes:
-      - "1234"
-
-datetime:
-  - platform: template
-    name: Date
-    id: test_date
-    type: date
-    initial_value: "2000-1-2"
-    set_action:
-      - logger.log: "set_value"
-    on_value:
-      - logger.log:
-          format: "Date: %04d-%02d-%02d"
-          args:
-            - x.year
-            - x.month
-            - x.day_of_month
-  - platform: template
-    name: Time
-    id: test_time
-    type: time
-    initial_value: "12:34:56am"
-    set_action:
-      - logger.log: "set_value"
-    on_value:
-      - logger.log:
-          format: "Time: %02d:%02d:%02d"
-          args:
-            - x.hour
-            - x.minute
-            - x.second
-  - platform: template
-    name: DateTime
-    id: test_datetime
-    type: datetime
-    initial_value: "2000-1-2 12:34:56"
-    set_action:
-      - logger.log: "set_value"
-    on_value:
-      - logger.log:
-          format: "DateTime: %04d-%02d-%02d %02d:%02d:%02d"
-          args:
-            - x.year
-            - x.month
-            - x.day_of_month
-            - x.hour
-            - x.minute
-            - x.second
+<<: !include common-base.yaml
 
 time:
   - platform: sntp  # Required for datetime
+    id: sntp_time
 
 wifi:  # Required for sntp time
   ap:
diff --git a/tests/components/template/test.esp32-ard.yaml b/tests/components/template/test.esp32-ard.yaml
deleted file mode 100644
index 25cb37a0b4..0000000000
--- a/tests/components/template/test.esp32-ard.yaml
+++ /dev/null
@@ -1,2 +0,0 @@
-packages:
-  common: !include common.yaml
diff --git a/tests/components/template/test.esp32-c3-ard.yaml b/tests/components/template/test.esp32-c3-ard.yaml
deleted file mode 100644
index 25cb37a0b4..0000000000
--- a/tests/components/template/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,2 +0,0 @@
-packages:
-  common: !include common.yaml
diff --git a/tests/components/template/test.nrf52-adafruit.yaml b/tests/components/template/test.nrf52-adafruit.yaml
new file mode 100644
index 0000000000..45751c6398
--- /dev/null
+++ b/tests/components/template/test.nrf52-adafruit.yaml
@@ -0,0 +1 @@
+<<: !include common-base.yaml
diff --git a/tests/components/template/test.nrf52-mcumgr.yaml b/tests/components/template/test.nrf52-mcumgr.yaml
new file mode 100644
index 0000000000..45751c6398
--- /dev/null
+++ b/tests/components/template/test.nrf52-mcumgr.yaml
@@ -0,0 +1 @@
+<<: !include common-base.yaml
diff --git a/tests/components/thermostat/common.yaml b/tests/components/thermostat/common.yaml
index d630a93efc..4aa87c0ac3 100644
--- a/tests/components/thermostat/common.yaml
+++ b/tests/components/thermostat/common.yaml
@@ -69,6 +69,11 @@ climate:
       - logger.log: swing_vertical_action
     swing_both_action:
       - logger.log: swing_both_action
+    humidity_control_humidify_action:
+      - logger.log: humidity_control_humidify_action
+    humidity_control_off_action:
+      - logger.log: humidity_control_off_action
+    humidity_hysteresis: 1.0
     startup_delay: true
     supplemental_cooling_delta: 2.0
     cool_deadband: 0.5
diff --git a/tests/components/thermostat/test.esp32-ard.yaml b/tests/components/thermostat/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/thermostat/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/thermostat/test.esp32-c3-ard.yaml b/tests/components/thermostat/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/thermostat/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/dht/test.esp32-ard.yaml b/tests/components/thermostat/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/dht/test.esp32-ard.yaml
rename to tests/components/thermostat/test.nrf52-adafruit.yaml
diff --git a/tests/components/dht/test.esp32-c3-ard.yaml b/tests/components/thermostat/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/dht/test.esp32-c3-ard.yaml
rename to tests/components/thermostat/test.nrf52-mcumgr.yaml
diff --git a/tests/components/time/test.esp32-ard.yaml b/tests/components/time/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/time/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/time/test.esp32-c3-ard.yaml b/tests/components/time/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/time/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/time/test.nrf52-adafruit.yaml b/tests/components/time/test.nrf52-adafruit.yaml
new file mode 100644
index 0000000000..a5502f8028
--- /dev/null
+++ b/tests/components/time/test.nrf52-adafruit.yaml
@@ -0,0 +1 @@
+time:
diff --git a/tests/components/time/test.nrf52-mcumgr.yaml b/tests/components/time/test.nrf52-mcumgr.yaml
new file mode 100644
index 0000000000..a5502f8028
--- /dev/null
+++ b/tests/components/time/test.nrf52-mcumgr.yaml
@@ -0,0 +1 @@
+time:
diff --git a/tests/components/time_based/test.esp32-ard.yaml b/tests/components/time_based/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/time_based/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/time_based/test.esp32-c3-ard.yaml b/tests/components/time_based/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/time_based/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/tlc59208f/common.yaml b/tests/components/tlc59208f/common.yaml
index 49460dcefc..1943063347 100644
--- a/tests/components/tlc59208f/common.yaml
+++ b/tests/components/tlc59208f/common.yaml
@@ -1,15 +1,13 @@
-i2c:
-  - id: i2c_tlc59208f
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 tlc59208f:
   - address: 0x20
     id: tlc59208f_1
+    i2c_id: i2c_bus
   - address: 0x22
     id: tlc59208f_2
+    i2c_id: i2c_bus
   - address: 0x24
     id: tlc59208f_3
+    i2c_id: i2c_bus
 
 output:
   - platform: tlc59208f
diff --git a/tests/components/tlc59208f/test.esp32-ard.yaml b/tests/components/tlc59208f/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/tlc59208f/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/tlc59208f/test.esp32-c3-ard.yaml b/tests/components/tlc59208f/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/tlc59208f/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/tlc59208f/test.esp32-c3-idf.yaml b/tests/components/tlc59208f/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/tlc59208f/test.esp32-c3-idf.yaml
+++ b/tests/components/tlc59208f/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tlc59208f/test.esp32-idf.yaml b/tests/components/tlc59208f/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/tlc59208f/test.esp32-idf.yaml
+++ b/tests/components/tlc59208f/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tlc59208f/test.esp8266-ard.yaml b/tests/components/tlc59208f/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/tlc59208f/test.esp8266-ard.yaml
+++ b/tests/components/tlc59208f/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tlc59208f/test.rp2040-ard.yaml b/tests/components/tlc59208f/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/tlc59208f/test.rp2040-ard.yaml
+++ b/tests/components/tlc59208f/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tlc5947/test.esp32-ard.yaml b/tests/components/tlc5947/test.esp32-ard.yaml
deleted file mode 100644
index 52411bc1e9..0000000000
--- a/tests/components/tlc5947/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clock_pin: GPIO15
-  data_pin: GPIO14
-  lat_pin: GPIO13
-
-packages:
-  common: !include common.yaml
diff --git a/tests/components/tlc5947/test.esp32-c3-ard.yaml b/tests/components/tlc5947/test.esp32-c3-ard.yaml
deleted file mode 100644
index 4694c43642..0000000000
--- a/tests/components/tlc5947/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
-  lat_pin: GPIO3
-
-packages:
-  common: !include common.yaml
diff --git a/tests/components/tlc5947/test.esp8266-ard.yaml b/tests/components/tlc5947/test.esp8266-ard.yaml
index 44da5a07b3..1eb9bcfae2 100644
--- a/tests/components/tlc5947/test.esp8266-ard.yaml
+++ b/tests/components/tlc5947/test.esp8266-ard.yaml
@@ -1,7 +1,7 @@
 substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
-  lat_pin: GPIO13
+  clock_pin: GPIO0
+  data_pin: GPIO2
+  lat_pin: GPIO15
 
 packages:
   common: !include common.yaml
diff --git a/tests/components/tlc5971/test.esp32-ard.yaml b/tests/components/tlc5971/test.esp32-ard.yaml
deleted file mode 100644
index 52411bc1e9..0000000000
--- a/tests/components/tlc5971/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clock_pin: GPIO15
-  data_pin: GPIO14
-  lat_pin: GPIO13
-
-packages:
-  common: !include common.yaml
diff --git a/tests/components/tlc5971/test.esp32-c3-ard.yaml b/tests/components/tlc5971/test.esp32-c3-ard.yaml
deleted file mode 100644
index d898a21d46..0000000000
--- a/tests/components/tlc5971/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
-
-packages:
-  common: !include common.yaml
diff --git a/tests/components/tlc5971/test.esp32-s2-ard.yaml b/tests/components/tlc5971/test.esp32-s2-ard.yaml
deleted file mode 100644
index 7bba0b0117..0000000000
--- a/tests/components/tlc5971/test.esp32-s2-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  clock_pin: GPIO36
-  data_pin: GPIO35
-
-packages:
-  common: !include common.yaml
diff --git a/tests/components/tlc5971/test.esp8266-ard.yaml b/tests/components/tlc5971/test.esp8266-ard.yaml
index 52411bc1e9..7923874f96 100644
--- a/tests/components/tlc5971/test.esp8266-ard.yaml
+++ b/tests/components/tlc5971/test.esp8266-ard.yaml
@@ -1,7 +1,7 @@
 substitutions:
   clock_pin: GPIO15
-  data_pin: GPIO14
-  lat_pin: GPIO13
+  data_pin: GPIO0
+  lat_pin: GPIO2
 
 packages:
   common: !include common.yaml
diff --git a/tests/components/tm1621/test.esp32-ard.yaml b/tests/components/tm1621/test.esp32-ard.yaml
deleted file mode 100644
index 0441e4bffe..0000000000
--- a/tests/components/tm1621/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  cs_pin: GPIO16
-  data_pin: GPIO17
-  read_pin: GPIO12
-  write_pin: GPIO13
-
-<<: !include common.yaml
diff --git a/tests/components/tm1621/test.esp32-c3-ard.yaml b/tests/components/tm1621/test.esp32-c3-ard.yaml
deleted file mode 100644
index 562ced7485..0000000000
--- a/tests/components/tm1621/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  cs_pin: GPIO6
-  data_pin: GPIO7
-  read_pin: GPIO2
-  write_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/tm1621/test.esp32-idf.yaml b/tests/components/tm1621/test.esp32-idf.yaml
index 0441e4bffe..21402f3216 100644
--- a/tests/components/tm1621/test.esp32-idf.yaml
+++ b/tests/components/tm1621/test.esp32-idf.yaml
@@ -1,7 +1,10 @@
 substitutions:
-  cs_pin: GPIO16
-  data_pin: GPIO17
+  cs_pin: GPIO4
+  data_pin: GPIO5
   read_pin: GPIO12
   write_pin: GPIO13
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/tm1621/test.esp8266-ard.yaml b/tests/components/tm1621/test.esp8266-ard.yaml
index ee7b62ce35..304ca71eb2 100644
--- a/tests/components/tm1621/test.esp8266-ard.yaml
+++ b/tests/components/tm1621/test.esp8266-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
   cs_pin: GPIO15
-  data_pin: GPIO14
-  read_pin: GPIO12
-  write_pin: GPIO13
+  data_pin: GPIO0
+  read_pin: GPIO2
+  write_pin: GPIO16
+
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tm1621/test.rp2040-ard.yaml b/tests/components/tm1621/test.rp2040-ard.yaml
index 562ced7485..9a880da876 100644
--- a/tests/components/tm1621/test.rp2040-ard.yaml
+++ b/tests/components/tm1621/test.rp2040-ard.yaml
@@ -1,7 +1,10 @@
 substitutions:
   cs_pin: GPIO6
   data_pin: GPIO7
-  read_pin: GPIO2
-  write_pin: GPIO3
+  read_pin: GPIO8
+  write_pin: GPIO9
+
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tm1637/test.esp32-ard.yaml b/tests/components/tm1637/test.esp32-ard.yaml
deleted file mode 100644
index 2c5786c47c..0000000000
--- a/tests/components/tm1637/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clk_pin: GPIO14
-  dio_pin: GPIO13
-
-<<: !include common.yaml
diff --git a/tests/components/tm1637/test.esp32-c3-ard.yaml b/tests/components/tm1637/test.esp32-c3-ard.yaml
deleted file mode 100644
index 96f6708a3b..0000000000
--- a/tests/components/tm1637/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clk_pin: GPIO4
-  dio_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/tm1637/test.esp32-c3-idf.yaml b/tests/components/tm1637/test.esp32-c3-idf.yaml
index 96f6708a3b..0c4d4a9a7a 100644
--- a/tests/components/tm1637/test.esp32-c3-idf.yaml
+++ b/tests/components/tm1637/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clk_pin: GPIO4
+  clk_pin: GPIO7
   dio_pin: GPIO3
 
 <<: !include common.yaml
diff --git a/tests/components/tm1637/test.esp8266-ard.yaml b/tests/components/tm1637/test.esp8266-ard.yaml
index 2c5786c47c..0a4221cbc0 100644
--- a/tests/components/tm1637/test.esp8266-ard.yaml
+++ b/tests/components/tm1637/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clk_pin: GPIO14
-  dio_pin: GPIO13
+  clk_pin: GPIO0
+  dio_pin: GPIO2
 
 <<: !include common.yaml
diff --git a/tests/components/tm1638/test.esp32-ard.yaml b/tests/components/tm1638/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/tm1638/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/tm1638/test.esp32-c3-ard.yaml b/tests/components/tm1638/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/tm1638/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/tm1651/test.esp32-ard.yaml b/tests/components/tm1651/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/tm1651/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/tm1651/test.esp32-c3-ard.yaml b/tests/components/tm1651/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/tm1651/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/display/test.esp32-ard.yaml b/tests/components/tm1651/test.esp32-c3-idf.yaml
similarity index 100%
rename from tests/components/display/test.esp32-ard.yaml
rename to tests/components/tm1651/test.esp32-c3-idf.yaml
diff --git a/tests/components/duty_cycle/test.esp32-ard.yaml b/tests/components/tm1651/test.esp32-idf.yaml
similarity index 100%
rename from tests/components/duty_cycle/test.esp32-ard.yaml
rename to tests/components/tm1651/test.esp32-idf.yaml
diff --git a/tests/components/tmp102/common.yaml b/tests/components/tmp102/common.yaml
index afc4a27fad..049a8067da 100644
--- a/tests/components/tmp102/common.yaml
+++ b/tests/components/tmp102/common.yaml
@@ -1,8 +1,4 @@
-i2c:
-  - id: i2c_tmp102
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: tmp102
+    i2c_id: i2c_bus
     name: TMP102 Temperature
diff --git a/tests/components/tmp102/test.esp32-ard.yaml b/tests/components/tmp102/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/tmp102/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/tmp102/test.esp32-c3-ard.yaml b/tests/components/tmp102/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/tmp102/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/tmp102/test.esp32-c3-idf.yaml b/tests/components/tmp102/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/tmp102/test.esp32-c3-idf.yaml
+++ b/tests/components/tmp102/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tmp102/test.esp32-idf.yaml b/tests/components/tmp102/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/tmp102/test.esp32-idf.yaml
+++ b/tests/components/tmp102/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tmp102/test.esp8266-ard.yaml b/tests/components/tmp102/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/tmp102/test.esp8266-ard.yaml
+++ b/tests/components/tmp102/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tmp102/test.rp2040-ard.yaml b/tests/components/tmp102/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/tmp102/test.rp2040-ard.yaml
+++ b/tests/components/tmp102/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tmp1075/common.yaml b/tests/components/tmp1075/common.yaml
index 4c4c6c6f35..90025f231e 100644
--- a/tests/components/tmp1075/common.yaml
+++ b/tests/components/tmp1075/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_tmp1075
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: tmp1075
+    i2c_id: i2c_bus
     name: Temperature TMP1075
     conversion_rate: 27.5ms
     alert:
diff --git a/tests/components/tmp1075/test.esp32-ard.yaml b/tests/components/tmp1075/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/tmp1075/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/tmp1075/test.esp32-c3-ard.yaml b/tests/components/tmp1075/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/tmp1075/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/tmp1075/test.esp32-c3-idf.yaml b/tests/components/tmp1075/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/tmp1075/test.esp32-c3-idf.yaml
+++ b/tests/components/tmp1075/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tmp1075/test.esp32-idf.yaml b/tests/components/tmp1075/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/tmp1075/test.esp32-idf.yaml
+++ b/tests/components/tmp1075/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tmp1075/test.esp8266-ard.yaml b/tests/components/tmp1075/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/tmp1075/test.esp8266-ard.yaml
+++ b/tests/components/tmp1075/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tmp1075/test.rp2040-ard.yaml b/tests/components/tmp1075/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/tmp1075/test.rp2040-ard.yaml
+++ b/tests/components/tmp1075/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tmp117/common.yaml b/tests/components/tmp117/common.yaml
index f4a5688933..58419c2134 100644
--- a/tests/components/tmp117/common.yaml
+++ b/tests/components/tmp117/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_tmp117
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: tmp117
+    i2c_id: i2c_bus
     name: TMP117 Temperature
     update_interval: 5s
diff --git a/tests/components/tmp117/test.esp32-ard.yaml b/tests/components/tmp117/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/tmp117/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/tmp117/test.esp32-c3-ard.yaml b/tests/components/tmp117/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/tmp117/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/tmp117/test.esp32-c3-idf.yaml b/tests/components/tmp117/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/tmp117/test.esp32-c3-idf.yaml
+++ b/tests/components/tmp117/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tmp117/test.esp32-idf.yaml b/tests/components/tmp117/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/tmp117/test.esp32-idf.yaml
+++ b/tests/components/tmp117/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tmp117/test.esp8266-ard.yaml b/tests/components/tmp117/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/tmp117/test.esp8266-ard.yaml
+++ b/tests/components/tmp117/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tmp117/test.rp2040-ard.yaml b/tests/components/tmp117/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/tmp117/test.rp2040-ard.yaml
+++ b/tests/components/tmp117/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tof10120/common.yaml b/tests/components/tof10120/common.yaml
index 67643323d9..b360a27248 100644
--- a/tests/components/tof10120/common.yaml
+++ b/tests/components/tof10120/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_tof10120
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: tof10120
+    i2c_id: i2c_bus
     name: Distance sensor
     update_interval: 5s
diff --git a/tests/components/tof10120/test.esp32-ard.yaml b/tests/components/tof10120/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/tof10120/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/tof10120/test.esp32-c3-ard.yaml b/tests/components/tof10120/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/tof10120/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/tof10120/test.esp32-c3-idf.yaml b/tests/components/tof10120/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/tof10120/test.esp32-c3-idf.yaml
+++ b/tests/components/tof10120/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tof10120/test.esp32-idf.yaml b/tests/components/tof10120/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/tof10120/test.esp32-idf.yaml
+++ b/tests/components/tof10120/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tof10120/test.esp8266-ard.yaml b/tests/components/tof10120/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/tof10120/test.esp8266-ard.yaml
+++ b/tests/components/tof10120/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tof10120/test.rp2040-ard.yaml b/tests/components/tof10120/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/tof10120/test.rp2040-ard.yaml
+++ b/tests/components/tof10120/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tormatic/common.yaml b/tests/components/tormatic/common.yaml
index 0f1b33ac12..712bf7569b 100644
--- a/tests/components/tormatic/common.yaml
+++ b/tests/components/tormatic/common.yaml
@@ -1,12 +1,5 @@
-uart:
-  - id: uart_tormatic
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 cover:
   - platform: tormatic
-    uart_id: uart_tormatic
     id: tormatic_garage_door
     name: Tormatic Garage Door
     open_duration: 15s
diff --git a/tests/components/tormatic/test.esp32-ard.yaml b/tests/components/tormatic/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/tormatic/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/tormatic/test.esp32-c3-ard.yaml b/tests/components/tormatic/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/tormatic/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/tormatic/test.esp32-c3-idf.yaml b/tests/components/tormatic/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/tormatic/test.esp32-c3-idf.yaml
+++ b/tests/components/tormatic/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tormatic/test.esp32-idf.yaml b/tests/components/tormatic/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/tormatic/test.esp32-idf.yaml
+++ b/tests/components/tormatic/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tormatic/test.esp8266-ard.yaml b/tests/components/tormatic/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/tormatic/test.esp8266-ard.yaml
+++ b/tests/components/tormatic/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tormatic/test.rp2040-ard.yaml b/tests/components/tormatic/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/tormatic/test.rp2040-ard.yaml
+++ b/tests/components/tormatic/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/toshiba/common.yaml b/tests/components/toshiba/common.yaml
index 79a833980e..ba96c0628d 100644
--- a/tests/components/toshiba/common.yaml
+++ b/tests/components/toshiba/common.yaml
@@ -1,7 +1,4 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: toshiba
     name: Toshiba Climate
+    transmitter_id: xmitr
diff --git a/tests/components/toshiba/common_ras2819t.yaml b/tests/components/toshiba/common_ras2819t.yaml
new file mode 100644
index 0000000000..32081fca98
--- /dev/null
+++ b/tests/components/toshiba/common_ras2819t.yaml
@@ -0,0 +1,13 @@
+remote_transmitter:
+  pin: ${tx_pin}
+  carrier_duty_percent: 50%
+
+remote_receiver:
+  id: rcvr
+  pin: ${rx_pin}
+
+climate:
+  - platform: toshiba
+    name: "RAS-2819T Climate"
+    model: RAS-2819T
+    receiver_id: rcvr
diff --git a/tests/components/toshiba/test.esp32-ard.yaml b/tests/components/toshiba/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/toshiba/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/toshiba/test.esp32-c3-ard.yaml b/tests/components/toshiba/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/toshiba/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/toshiba/test.esp32-c3-idf.yaml b/tests/components/toshiba/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/toshiba/test.esp32-c3-idf.yaml
+++ b/tests/components/toshiba/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/toshiba/test.esp32-idf.yaml b/tests/components/toshiba/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/toshiba/test.esp32-idf.yaml
+++ b/tests/components/toshiba/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/toshiba/test.esp8266-ard.yaml b/tests/components/toshiba/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/toshiba/test.esp8266-ard.yaml
+++ b/tests/components/toshiba/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/toshiba/test_ras2819t.esp32-ard.yaml b/tests/components/toshiba/test_ras2819t.esp32-ard.yaml
new file mode 100644
index 0000000000..00805baa01
--- /dev/null
+++ b/tests/components/toshiba/test_ras2819t.esp32-ard.yaml
@@ -0,0 +1,5 @@
+substitutions:
+  tx_pin: GPIO5
+  rx_pin: GPIO4
+
+<<: !include common_ras2819t.yaml
diff --git a/tests/components/toshiba/test_ras2819t.esp32-c3-ard.yaml b/tests/components/toshiba/test_ras2819t.esp32-c3-ard.yaml
new file mode 100644
index 0000000000..00805baa01
--- /dev/null
+++ b/tests/components/toshiba/test_ras2819t.esp32-c3-ard.yaml
@@ -0,0 +1,5 @@
+substitutions:
+  tx_pin: GPIO5
+  rx_pin: GPIO4
+
+<<: !include common_ras2819t.yaml
diff --git a/tests/components/toshiba/test_ras2819t.esp32-c3-idf.yaml b/tests/components/toshiba/test_ras2819t.esp32-c3-idf.yaml
new file mode 100644
index 0000000000..00805baa01
--- /dev/null
+++ b/tests/components/toshiba/test_ras2819t.esp32-c3-idf.yaml
@@ -0,0 +1,5 @@
+substitutions:
+  tx_pin: GPIO5
+  rx_pin: GPIO4
+
+<<: !include common_ras2819t.yaml
diff --git a/tests/components/toshiba/test_ras2819t.esp32-idf.yaml b/tests/components/toshiba/test_ras2819t.esp32-idf.yaml
new file mode 100644
index 0000000000..00805baa01
--- /dev/null
+++ b/tests/components/toshiba/test_ras2819t.esp32-idf.yaml
@@ -0,0 +1,5 @@
+substitutions:
+  tx_pin: GPIO5
+  rx_pin: GPIO4
+
+<<: !include common_ras2819t.yaml
diff --git a/tests/components/toshiba/test_ras2819t.esp8266-ard.yaml b/tests/components/toshiba/test_ras2819t.esp8266-ard.yaml
new file mode 100644
index 0000000000..00805baa01
--- /dev/null
+++ b/tests/components/toshiba/test_ras2819t.esp8266-ard.yaml
@@ -0,0 +1,5 @@
+substitutions:
+  tx_pin: GPIO5
+  rx_pin: GPIO4
+
+<<: !include common_ras2819t.yaml
diff --git a/tests/components/total_daily_energy/common.yaml b/tests/components/total_daily_energy/common.yaml
index ae4d30408b..dd7e648da6 100644
--- a/tests/components/total_daily_energy/common.yaml
+++ b/tests/components/total_daily_energy/common.yaml
@@ -17,10 +17,10 @@ sensor:
       name: HLW8012 Voltage
     power:
       name: HLW8012 Power
-      id: hlw8012_power
+      id: total_daily_energy_hlw8012_power
     energy:
       name: HLW8012 Energy
-      id: hlw8012_energy
+      id: total_daily_energy_hlw8012_energy
     update_interval: 15s
     current_resistor: 0.001 ohm
     voltage_divider: 2351
@@ -29,4 +29,4 @@ sensor:
     model: hlw8012
   - platform: total_daily_energy
     name: HLW8012 Total Daily Energy
-    power_id: hlw8012_power
+    power_id: total_daily_energy_hlw8012_power
diff --git a/tests/components/total_daily_energy/test.esp32-ard.yaml b/tests/components/total_daily_energy/test.esp32-ard.yaml
deleted file mode 100644
index 8b42b21b54..0000000000
--- a/tests/components/total_daily_energy/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  sel_pin: GPIO12
-  cf_pin: GPIO13
-  cf1_pin: GPIO14
-
-<<: !include common.yaml
diff --git a/tests/components/total_daily_energy/test.esp32-c3-ard.yaml b/tests/components/total_daily_energy/test.esp32-c3-ard.yaml
deleted file mode 100644
index 8b0d069ce2..0000000000
--- a/tests/components/total_daily_energy/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  sel_pin: GPIO2
-  cf_pin: GPIO3
-  cf1_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/total_daily_energy/test.esp8266-ard.yaml b/tests/components/total_daily_energy/test.esp8266-ard.yaml
index 8b42b21b54..ec9c0e43dc 100644
--- a/tests/components/total_daily_energy/test.esp8266-ard.yaml
+++ b/tests/components/total_daily_energy/test.esp8266-ard.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  sel_pin: GPIO12
-  cf_pin: GPIO13
-  cf1_pin: GPIO14
+  sel_pin: GPIO0
+  cf_pin: GPIO2
+  cf1_pin: GPIO15
 
 <<: !include common.yaml
diff --git a/tests/components/tsl2561/common.yaml b/tests/components/tsl2561/common.yaml
index d2b4f75df3..132bdb890e 100644
--- a/tests/components/tsl2561/common.yaml
+++ b/tests/components/tsl2561/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_tsl2561
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: tsl2561
+    i2c_id: i2c_bus
     name: TSL2561 Ambient Light
     address: 0x39
     is_cs_package: true
diff --git a/tests/components/tsl2561/test.esp32-ard.yaml b/tests/components/tsl2561/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/tsl2561/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/tsl2561/test.esp32-c3-ard.yaml b/tests/components/tsl2561/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/tsl2561/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/tsl2561/test.esp32-c3-idf.yaml b/tests/components/tsl2561/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/tsl2561/test.esp32-c3-idf.yaml
+++ b/tests/components/tsl2561/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tsl2561/test.esp32-idf.yaml b/tests/components/tsl2561/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/tsl2561/test.esp32-idf.yaml
+++ b/tests/components/tsl2561/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tsl2561/test.esp8266-ard.yaml b/tests/components/tsl2561/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/tsl2561/test.esp8266-ard.yaml
+++ b/tests/components/tsl2561/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tsl2561/test.rp2040-ard.yaml b/tests/components/tsl2561/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/tsl2561/test.rp2040-ard.yaml
+++ b/tests/components/tsl2561/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tsl2591/common.yaml b/tests/components/tsl2591/common.yaml
index d58c46fb48..93433cf968 100644
--- a/tests/components/tsl2591/common.yaml
+++ b/tests/components/tsl2591/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_tsl2591
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: tsl2591
+    i2c_id: i2c_bus
     id: test_tsl2591
     address: 0x29
     integration_time: 600ms
diff --git a/tests/components/tsl2591/test.esp32-ard.yaml b/tests/components/tsl2591/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/tsl2591/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/tsl2591/test.esp32-c3-ard.yaml b/tests/components/tsl2591/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/tsl2591/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/tsl2591/test.esp32-c3-idf.yaml b/tests/components/tsl2591/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/tsl2591/test.esp32-c3-idf.yaml
+++ b/tests/components/tsl2591/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tsl2591/test.esp32-idf.yaml b/tests/components/tsl2591/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/tsl2591/test.esp32-idf.yaml
+++ b/tests/components/tsl2591/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tsl2591/test.esp8266-ard.yaml b/tests/components/tsl2591/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/tsl2591/test.esp8266-ard.yaml
+++ b/tests/components/tsl2591/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tsl2591/test.rp2040-ard.yaml b/tests/components/tsl2591/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/tsl2591/test.rp2040-ard.yaml
+++ b/tests/components/tsl2591/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tt21100/common.yaml b/tests/components/tt21100/common.yaml
index a5d7970429..56089aed1e 100644
--- a/tests/components/tt21100/common.yaml
+++ b/tests/components/tt21100/common.yaml
@@ -1,25 +1,24 @@
-i2c:
-  - id: i2c_tt21100
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 display:
   - platform: ssd1306_i2c
-    id: ssd1306_display
+    i2c_id: i2c_bus
+    id: ssd1306_i2c_display
     model: SSD1306_128X64
     reset_pin: ${disp_reset_pin}
     pages:
-      - id: page1
+      - id: tt21100_page1
         lambda: |-
           it.rectangle(0, 0, it.get_width(), it.get_height());
 
 touchscreen:
   - platform: tt21100
-    display: ssd1306_display
+    i2c_id: i2c_bus
+    id: tt21100_touchscreen
+    display: ssd1306_i2c_display
     interrupt_pin: ${interrupt_pin}
     reset_pin: ${reset_pin}
 
 binary_sensor:
   - platform: tt21100
+    id: tt21100_button
     name: Home Button
     index: 1
diff --git a/tests/components/tt21100/test.esp32-ard.yaml b/tests/components/tt21100/test.esp32-ard.yaml
deleted file mode 100644
index 05598719f9..0000000000
--- a/tests/components/tt21100/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  disp_reset_pin: GPIO12
-  scl_pin: GPIO13
-  sda_pin: GPIO14
-  interrupt_pin: GPIO15
-  reset_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/tt21100/test.esp32-c3-ard.yaml b/tests/components/tt21100/test.esp32-c3-ard.yaml
deleted file mode 100644
index 36a8ce2778..0000000000
--- a/tests/components/tt21100/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  disp_reset_pin: GPIO10
-  scl_pin: GPIO0
-  sda_pin: GPIO1
-  interrupt_pin: GPIO2
-  reset_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/tt21100/test.esp32-c3-idf.yaml b/tests/components/tt21100/test.esp32-c3-idf.yaml
index 36a8ce2778..a7265e10b2 100644
--- a/tests/components/tt21100/test.esp32-c3-idf.yaml
+++ b/tests/components/tt21100/test.esp32-c3-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  disp_reset_pin: GPIO10
-  scl_pin: GPIO0
-  sda_pin: GPIO1
+  disp_reset_pin: GPIO7
   interrupt_pin: GPIO2
   reset_pin: GPIO3
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/tt21100/test.esp32-idf.yaml b/tests/components/tt21100/test.esp32-idf.yaml
index 05598719f9..033aafb73c 100644
--- a/tests/components/tt21100/test.esp32-idf.yaml
+++ b/tests/components/tt21100/test.esp32-idf.yaml
@@ -1,8 +1,9 @@
 substitutions:
   disp_reset_pin: GPIO12
-  scl_pin: GPIO13
-  sda_pin: GPIO14
   interrupt_pin: GPIO15
-  reset_pin: GPIO16
+  reset_pin: GPIO4
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tt21100/test.esp8266-ard.yaml b/tests/components/tt21100/test.esp8266-ard.yaml
index 05598719f9..25d1ff82e3 100644
--- a/tests/components/tt21100/test.esp8266-ard.yaml
+++ b/tests/components/tt21100/test.esp8266-ard.yaml
@@ -1,8 +1,9 @@
 substitutions:
-  disp_reset_pin: GPIO12
-  scl_pin: GPIO13
-  sda_pin: GPIO14
+  disp_reset_pin: GPIO0
   interrupt_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/tt21100/test.rp2040-ard.yaml b/tests/components/tt21100/test.rp2040-ard.yaml
index 36a8ce2778..0d13628294 100644
--- a/tests/components/tt21100/test.rp2040-ard.yaml
+++ b/tests/components/tt21100/test.rp2040-ard.yaml
@@ -1,8 +1,9 @@
 substitutions:
   disp_reset_pin: GPIO10
-  scl_pin: GPIO0
-  sda_pin: GPIO1
   interrupt_pin: GPIO2
   reset_pin: GPIO3
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ttp229_bsf/common.yaml b/tests/components/ttp229_bsf/common.yaml
index 42c26a5d51..91134f5098 100644
--- a/tests/components/ttp229_bsf/common.yaml
+++ b/tests/components/ttp229_bsf/common.yaml
@@ -1,6 +1,6 @@
 ttp229_bsf:
-  scl_pin: ${scl_pin}
-  sdo_pin: ${sdo_pin}
+  scl_pin: ${ttp229_scl_pin}
+  sdo_pin: ${ttp229_sdo_pin}
 
 binary_sensor:
   - platform: ttp229_bsf
diff --git a/tests/components/ttp229_bsf/test.esp32-ard.yaml b/tests/components/ttp229_bsf/test.esp32-ard.yaml
deleted file mode 100644
index 80ed75293f..0000000000
--- a/tests/components/ttp229_bsf/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sdo_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ttp229_bsf/test.esp32-c3-ard.yaml b/tests/components/ttp229_bsf/test.esp32-c3-ard.yaml
deleted file mode 100644
index 135b213edc..0000000000
--- a/tests/components/ttp229_bsf/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sdo_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ttp229_bsf/test.esp32-c3-idf.yaml b/tests/components/ttp229_bsf/test.esp32-c3-idf.yaml
index 135b213edc..ad1c58b40e 100644
--- a/tests/components/ttp229_bsf/test.esp32-c3-idf.yaml
+++ b/tests/components/ttp229_bsf/test.esp32-c3-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  scl_pin: GPIO5
-  sdo_pin: GPIO4
+  ttp229_scl_pin: GPIO7
+  ttp229_sdo_pin: GPIO4
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ttp229_bsf/test.esp32-idf.yaml b/tests/components/ttp229_bsf/test.esp32-idf.yaml
index 80ed75293f..8b7f6fe6ea 100644
--- a/tests/components/ttp229_bsf/test.esp32-idf.yaml
+++ b/tests/components/ttp229_bsf/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  scl_pin: GPIO16
-  sdo_pin: GPIO17
+  ttp229_scl_pin: GPIO14
+  ttp229_sdo_pin: GPIO5
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ttp229_bsf/test.esp8266-ard.yaml b/tests/components/ttp229_bsf/test.esp8266-ard.yaml
index 135b213edc..1832fde07f 100644
--- a/tests/components/ttp229_bsf/test.esp8266-ard.yaml
+++ b/tests/components/ttp229_bsf/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  scl_pin: GPIO5
-  sdo_pin: GPIO4
+  ttp229_scl_pin: GPIO0
+  ttp229_sdo_pin: GPIO2
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ttp229_bsf/test.rp2040-ard.yaml b/tests/components/ttp229_bsf/test.rp2040-ard.yaml
index 135b213edc..1448c82347 100644
--- a/tests/components/ttp229_bsf/test.rp2040-ard.yaml
+++ b/tests/components/ttp229_bsf/test.rp2040-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  scl_pin: GPIO5
-  sdo_pin: GPIO4
+  ttp229_scl_pin: GPIO2
+  ttp229_sdo_pin: GPIO6
+
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ttp229_lsf/common.yaml b/tests/components/ttp229_lsf/common.yaml
index 5c0dbf9517..41402f5aac 100644
--- a/tests/components/ttp229_lsf/common.yaml
+++ b/tests/components/ttp229_lsf/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_ttp229_lsf
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 ttp229_lsf:
+  i2c_id: i2c_bus
 
 binary_sensor:
   - platform: ttp229_lsf
diff --git a/tests/components/ttp229_lsf/test.esp32-ard.yaml b/tests/components/ttp229_lsf/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ttp229_lsf/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ttp229_lsf/test.esp32-c3-ard.yaml b/tests/components/ttp229_lsf/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ttp229_lsf/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ttp229_lsf/test.esp32-c3-idf.yaml b/tests/components/ttp229_lsf/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ttp229_lsf/test.esp32-c3-idf.yaml
+++ b/tests/components/ttp229_lsf/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ttp229_lsf/test.esp32-idf.yaml b/tests/components/ttp229_lsf/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ttp229_lsf/test.esp32-idf.yaml
+++ b/tests/components/ttp229_lsf/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ttp229_lsf/test.esp8266-ard.yaml b/tests/components/ttp229_lsf/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ttp229_lsf/test.esp8266-ard.yaml
+++ b/tests/components/ttp229_lsf/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ttp229_lsf/test.rp2040-ard.yaml b/tests/components/ttp229_lsf/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ttp229_lsf/test.rp2040-ard.yaml
+++ b/tests/components/ttp229_lsf/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tuya/common.yaml b/tests/components/tuya/common.yaml
index 2c40628139..e177b7d056 100644
--- a/tests/components/tuya/common.yaml
+++ b/tests/components/tuya/common.yaml
@@ -2,12 +2,6 @@ wifi:
   ssid: MySSID
   password: password1
 
-uart:
-  - id: uart_tuya
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 tuya:
   status_pin:
     number: ${status_pin}
diff --git a/tests/components/tuya/test.esp32-ard.yaml b/tests/components/tuya/test.esp32-ard.yaml
deleted file mode 100644
index 926a46cf73..0000000000
--- a/tests/components/tuya/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-  status_pin: GPIO12
-
-<<: !include common.yaml
diff --git a/tests/components/tuya/test.esp32-c3-ard.yaml b/tests/components/tuya/test.esp32-c3-ard.yaml
deleted file mode 100644
index c62a0b10f6..0000000000
--- a/tests/components/tuya/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  status_pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/tuya/test.esp32-c3-idf.yaml b/tests/components/tuya/test.esp32-c3-idf.yaml
index c62a0b10f6..43c28ba7b3 100644
--- a/tests/components/tuya/test.esp32-c3-idf.yaml
+++ b/tests/components/tuya/test.esp32-c3-idf.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
   status_pin: GPIO2
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tuya/test.esp32-idf.yaml b/tests/components/tuya/test.esp32-idf.yaml
index 926a46cf73..0baa48a6c5 100644
--- a/tests/components/tuya/test.esp32-idf.yaml
+++ b/tests/components/tuya/test.esp32-idf.yaml
@@ -1,6 +1,9 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
   status_pin: GPIO12
 
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/tuya/test.esp8266-ard.yaml b/tests/components/tuya/test.esp8266-ard.yaml
index 11d46ed50e..1565495cf8 100644
--- a/tests/components/tuya/test.esp8266-ard.yaml
+++ b/tests/components/tuya/test.esp8266-ard.yaml
@@ -1,6 +1,9 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-  status_pin: GPIO12
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+  status_pin: GPIO15
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/tuya/test.rp2040-ard.yaml b/tests/components/tuya/test.rp2040-ard.yaml
index 11d46ed50e..0c3db54644 100644
--- a/tests/components/tuya/test.rp2040-ard.yaml
+++ b/tests/components/tuya/test.rp2040-ard.yaml
@@ -3,4 +3,7 @@ substitutions:
   rx_pin: GPIO5
   status_pin: GPIO12
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/tx20/test.esp32-ard.yaml b/tests/components/tx20/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/tx20/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/tx20/test.esp32-c3-ard.yaml b/tests/components/tx20/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/tx20/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/uart/test-uart_max_with_usb_cdc.esp32-c3-ard.yaml b/tests/components/uart/test-uart_max_with_usb_cdc.esp32-c3-ard.yaml
deleted file mode 100644
index 2a73826c51..0000000000
--- a/tests/components/uart/test-uart_max_with_usb_cdc.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,30 +0,0 @@
-<<: !include ../logger/common-usb_cdc.yaml
-
-esphome:
-  on_boot:
-    then:
-      - uart.write:
-          id: uart_1
-          data: 'Hello World'
-      - uart.write:
-          id: uart_1
-          data: [0x00, 0x20, 0x42]
-
-uart:
-  - id: uart_1
-    tx_pin: 4
-    rx_pin: 5
-    baud_rate: 9600
-    data_bits: 8
-    rx_buffer_size: 512
-    parity: EVEN
-    stop_bits: 2
-
-  - id: uart_2
-    tx_pin: 6
-    rx_pin: 7
-    baud_rate: 9600
-    data_bits: 8
-    rx_buffer_size: 512
-    parity: EVEN
-    stop_bits: 2
diff --git a/tests/components/uart/test-uart_max_with_usb_cdc.esp32-s2-ard.yaml b/tests/components/uart/test-uart_max_with_usb_cdc.esp32-s2-ard.yaml
deleted file mode 100644
index 2a73826c51..0000000000
--- a/tests/components/uart/test-uart_max_with_usb_cdc.esp32-s2-ard.yaml
+++ /dev/null
@@ -1,30 +0,0 @@
-<<: !include ../logger/common-usb_cdc.yaml
-
-esphome:
-  on_boot:
-    then:
-      - uart.write:
-          id: uart_1
-          data: 'Hello World'
-      - uart.write:
-          id: uart_1
-          data: [0x00, 0x20, 0x42]
-
-uart:
-  - id: uart_1
-    tx_pin: 4
-    rx_pin: 5
-    baud_rate: 9600
-    data_bits: 8
-    rx_buffer_size: 512
-    parity: EVEN
-    stop_bits: 2
-
-  - id: uart_2
-    tx_pin: 6
-    rx_pin: 7
-    baud_rate: 9600
-    data_bits: 8
-    rx_buffer_size: 512
-    parity: EVEN
-    stop_bits: 2
diff --git a/tests/components/uart/test-uart_max_with_usb_cdc.esp32-s2-idf.yaml b/tests/components/uart/test-uart_max_with_usb_cdc.esp32-s2-idf.yaml
index 2a73826c51..602766869c 100644
--- a/tests/components/uart/test-uart_max_with_usb_cdc.esp32-s2-idf.yaml
+++ b/tests/components/uart/test-uart_max_with_usb_cdc.esp32-s2-idf.yaml
@@ -14,17 +14,23 @@ uart:
   - id: uart_1
     tx_pin: 4
     rx_pin: 5
+    flow_control_pin: 6
     baud_rate: 9600
     data_bits: 8
     rx_buffer_size: 512
+    rx_full_threshold: 10
+    rx_timeout: 1
     parity: EVEN
     stop_bits: 2
 
   - id: uart_2
-    tx_pin: 6
-    rx_pin: 7
+    tx_pin: 7
+    rx_pin: 8
+    flow_control_pin: 9
     baud_rate: 9600
     data_bits: 8
     rx_buffer_size: 512
+    rx_full_threshold: 10
+    rx_timeout: 1
     parity: EVEN
     stop_bits: 2
diff --git a/tests/components/uart/test-uart_max_with_usb_cdc.esp32-s3-ard.yaml b/tests/components/uart/test-uart_max_with_usb_cdc.esp32-s3-ard.yaml
deleted file mode 100644
index 2a73826c51..0000000000
--- a/tests/components/uart/test-uart_max_with_usb_cdc.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,30 +0,0 @@
-<<: !include ../logger/common-usb_cdc.yaml
-
-esphome:
-  on_boot:
-    then:
-      - uart.write:
-          id: uart_1
-          data: 'Hello World'
-      - uart.write:
-          id: uart_1
-          data: [0x00, 0x20, 0x42]
-
-uart:
-  - id: uart_1
-    tx_pin: 4
-    rx_pin: 5
-    baud_rate: 9600
-    data_bits: 8
-    rx_buffer_size: 512
-    parity: EVEN
-    stop_bits: 2
-
-  - id: uart_2
-    tx_pin: 6
-    rx_pin: 7
-    baud_rate: 9600
-    data_bits: 8
-    rx_buffer_size: 512
-    parity: EVEN
-    stop_bits: 2
diff --git a/tests/components/uart/test-uart_max_with_usb_serial_jtag.esp32-c3-idf.yaml b/tests/components/uart/test-uart_max_with_usb_serial_jtag.esp32-c3-idf.yaml
index e0a07dde91..3151403896 100644
--- a/tests/components/uart/test-uart_max_with_usb_serial_jtag.esp32-c3-idf.yaml
+++ b/tests/components/uart/test-uart_max_with_usb_serial_jtag.esp32-c3-idf.yaml
@@ -14,17 +14,23 @@ uart:
   - id: uart_1
     tx_pin: 4
     rx_pin: 5
+    flow_control_pin: 6
     baud_rate: 9600
     data_bits: 8
     rx_buffer_size: 512
+    rx_full_threshold: 10
+    rx_timeout: 1
     parity: EVEN
     stop_bits: 2
 
   - id: uart_2
-    tx_pin: 6
-    rx_pin: 7
+    tx_pin: 7
+    rx_pin: 8
+    flow_control_pin: 9
     baud_rate: 9600
     data_bits: 8
     rx_buffer_size: 512
+    rx_full_threshold: 10
+    rx_timeout: 1
     parity: EVEN
     stop_bits: 2
diff --git a/tests/components/uart/test-uart_max_with_usb_serial_jtag.esp32-s3-idf.yaml b/tests/components/uart/test-uart_max_with_usb_serial_jtag.esp32-s3-idf.yaml
index e0a07dde91..88a806eb92 100644
--- a/tests/components/uart/test-uart_max_with_usb_serial_jtag.esp32-s3-idf.yaml
+++ b/tests/components/uart/test-uart_max_with_usb_serial_jtag.esp32-s3-idf.yaml
@@ -14,17 +14,35 @@ uart:
   - id: uart_1
     tx_pin: 4
     rx_pin: 5
+    flow_control_pin: 6
     baud_rate: 9600
     data_bits: 8
     rx_buffer_size: 512
+    rx_full_threshold: 10
+    rx_timeout: 1
     parity: EVEN
     stop_bits: 2
 
   - id: uart_2
-    tx_pin: 6
-    rx_pin: 7
+    tx_pin: 7
+    rx_pin: 8
+    flow_control_pin: 9
     baud_rate: 9600
     data_bits: 8
     rx_buffer_size: 512
+    rx_full_threshold: 10
+    rx_timeout: 1
+    parity: EVEN
+    stop_bits: 2
+
+  - id: uart_3
+    tx_pin: 10
+    rx_pin: 11
+    flow_control_pin: 12
+    baud_rate: 9600
+    data_bits: 8
+    rx_buffer_size: 512
+    rx_full_threshold: 10
+    rx_timeout: 1
     parity: EVEN
     stop_bits: 2
diff --git a/tests/components/uart/test.esp32-ard.yaml b/tests/components/uart/test.esp32-ard.yaml
deleted file mode 100644
index bef5b460ab..0000000000
--- a/tests/components/uart/test.esp32-ard.yaml
+++ /dev/null
@@ -1,15 +0,0 @@
-esphome:
-  on_boot:
-    then:
-      - uart.write: 'Hello World'
-      - uart.write: [0x00, 0x20, 0x42]
-
-uart:
-  - id: uart_uart
-    tx_pin: 17
-    rx_pin: 16
-    baud_rate: 9600
-    data_bits: 8
-    rx_buffer_size: 512
-    parity: EVEN
-    stop_bits: 2
diff --git a/tests/components/uart/test.esp32-c3-ard.yaml b/tests/components/uart/test.esp32-c3-ard.yaml
deleted file mode 100644
index 09178f1663..0000000000
--- a/tests/components/uart/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,15 +0,0 @@
-esphome:
-  on_boot:
-    then:
-      - uart.write: 'Hello World'
-      - uart.write: [0x00, 0x20, 0x42]
-
-uart:
-  - id: uart_uart
-    tx_pin: 4
-    rx_pin: 5
-    baud_rate: 9600
-    data_bits: 8
-    rx_buffer_size: 512
-    parity: EVEN
-    stop_bits: 2
diff --git a/tests/components/uart/test.esp32-c3-idf.yaml b/tests/components/uart/test.esp32-c3-idf.yaml
index 09178f1663..b053290a8b 100644
--- a/tests/components/uart/test.esp32-c3-idf.yaml
+++ b/tests/components/uart/test.esp32-c3-idf.yaml
@@ -8,8 +8,11 @@ uart:
   - id: uart_uart
     tx_pin: 4
     rx_pin: 5
+    flow_control_pin: 6
     baud_rate: 9600
     data_bits: 8
     rx_buffer_size: 512
+    rx_full_threshold: 10
+    rx_timeout: 1
     parity: EVEN
     stop_bits: 2
diff --git a/tests/components/uart/test.esp32-idf.yaml b/tests/components/uart/test.esp32-idf.yaml
index 5a0ed7eba7..5634c5c6f6 100644
--- a/tests/components/uart/test.esp32-idf.yaml
+++ b/tests/components/uart/test.esp32-idf.yaml
@@ -8,9 +8,12 @@ uart:
   - id: uart_uart
     tx_pin: 17
     rx_pin: 16
+    flow_control_pin: 4
     baud_rate: 9600
     data_bits: 8
     rx_buffer_size: 512
+    rx_full_threshold: 10
+    rx_timeout: 1
     parity: EVEN
     stop_bits: 2
 
diff --git a/tests/components/udp/test.esp32-ard.yaml b/tests/components/udp/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/udp/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/udp/test.esp32-c3-ard.yaml b/tests/components/udp/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/udp/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/udp/test.esp32-c3-idf.yaml b/tests/components/udp/test.esp32-c3-idf.yaml
index dade44d145..9990d96d29 100644
--- a/tests/components/udp/test.esp32-c3-idf.yaml
+++ b/tests/components/udp/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/udp/test.esp32-idf.yaml b/tests/components/udp/test.esp32-idf.yaml
index dade44d145..b47e39c389 100644
--- a/tests/components/udp/test.esp32-idf.yaml
+++ b/tests/components/udp/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/udp/test.esp8266-ard.yaml b/tests/components/udp/test.esp8266-ard.yaml
index dade44d145..4a98b9388a 100644
--- a/tests/components/udp/test.esp8266-ard.yaml
+++ b/tests/components/udp/test.esp8266-ard.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/udp/test.host.yaml b/tests/components/udp/test.host.yaml
index e735c37e4d..84e78894e5 100644
--- a/tests/components/udp/test.host.yaml
+++ b/tests/components/udp/test.host.yaml
@@ -1,4 +1,15 @@
-packages:
-  common: !include common.yaml
-
-wifi: !remove
+udp:
+  id: my_udp
+  listen_address: 239.0.60.53
+  addresses: ["239.0.60.53"]
+  on_receive:
+    - logger.log:
+        format: "Received %d bytes"
+        args: [data.size()]
+    - udp.write:
+        id: my_udp
+        data: "hello world"
+    - udp.write:
+        id: my_udp
+        data: !lambda |-
+          return std::vector{1,3,4,5,6};
diff --git a/tests/components/udp/test.rp2040-ard.yaml b/tests/components/udp/test.rp2040-ard.yaml
index dade44d145..319a7c71a6 100644
--- a/tests/components/udp/test.rp2040-ard.yaml
+++ b/tests/components/udp/test.rp2040-ard.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/ufire_ec/common.yaml b/tests/components/ufire_ec/common.yaml
index dcc957aaee..4260f0ab4c 100644
--- a/tests/components/ufire_ec/common.yaml
+++ b/tests/components/ufire_ec/common.yaml
@@ -7,16 +7,12 @@ esphome:
           temperature: !lambda "return id(test_sensor).state;"
       - ufire_ec.reset:
 
-i2c:
-  - id: i2c_ufire_ec
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: template
     id: test_sensor
     lambda: "return 21;"
   - platform: ufire_ec
+    i2c_id: i2c_bus
     id: ufire_ec_board
     ec:
       name: Ufire EC
diff --git a/tests/components/ufire_ec/test.esp32-ard.yaml b/tests/components/ufire_ec/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ufire_ec/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ufire_ec/test.esp32-c3-ard.yaml b/tests/components/ufire_ec/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ufire_ec/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ufire_ec/test.esp32-c3-idf.yaml b/tests/components/ufire_ec/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ufire_ec/test.esp32-c3-idf.yaml
+++ b/tests/components/ufire_ec/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ufire_ec/test.esp32-idf.yaml b/tests/components/ufire_ec/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ufire_ec/test.esp32-idf.yaml
+++ b/tests/components/ufire_ec/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ufire_ec/test.esp8266-ard.yaml b/tests/components/ufire_ec/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ufire_ec/test.esp8266-ard.yaml
+++ b/tests/components/ufire_ec/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ufire_ec/test.rp2040-ard.yaml b/tests/components/ufire_ec/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ufire_ec/test.rp2040-ard.yaml
+++ b/tests/components/ufire_ec/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ufire_ise/common.yaml b/tests/components/ufire_ise/common.yaml
index d6ead8c479..f7865ea87b 100644
--- a/tests/components/ufire_ise/common.yaml
+++ b/tests/components/ufire_ise/common.yaml
@@ -9,16 +9,12 @@ esphome:
           solution: 4.0
       - ufire_ise.reset:
 
-i2c:
-  - id: i2c_ufire_ise
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: template
     id: test_sensor
     lambda: "return 21;"
   - platform: ufire_ise
+    i2c_id: i2c_bus
     id: ufire_ise_sensor
     temperature_sensor: test_sensor
     ph:
diff --git a/tests/components/ufire_ise/test.esp32-ard.yaml b/tests/components/ufire_ise/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/ufire_ise/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/ufire_ise/test.esp32-c3-ard.yaml b/tests/components/ufire_ise/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/ufire_ise/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/ufire_ise/test.esp32-c3-idf.yaml b/tests/components/ufire_ise/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/ufire_ise/test.esp32-c3-idf.yaml
+++ b/tests/components/ufire_ise/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ufire_ise/test.esp32-idf.yaml b/tests/components/ufire_ise/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/ufire_ise/test.esp32-idf.yaml
+++ b/tests/components/ufire_ise/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ufire_ise/test.esp8266-ard.yaml b/tests/components/ufire_ise/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/ufire_ise/test.esp8266-ard.yaml
+++ b/tests/components/ufire_ise/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/ufire_ise/test.rp2040-ard.yaml b/tests/components/ufire_ise/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/ufire_ise/test.rp2040-ard.yaml
+++ b/tests/components/ufire_ise/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/uln2003/test.esp32-ard.yaml b/tests/components/uln2003/test.esp32-ard.yaml
deleted file mode 100644
index ee4cff0923..0000000000
--- a/tests/components/uln2003/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  pin_a: GPIO12
-  pin_b: GPIO13
-  pin_c: GPIO14
-  pin_d: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/uln2003/test.esp32-c3-ard.yaml b/tests/components/uln2003/test.esp32-c3-ard.yaml
deleted file mode 100644
index 11d16a4d5d..0000000000
--- a/tests/components/uln2003/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  pin_a: GPIO0
-  pin_b: GPIO1
-  pin_c: GPIO2
-  pin_d: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/ultrasonic/test.esp32-ard.yaml b/tests/components/ultrasonic/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/ultrasonic/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/ultrasonic/test.esp32-c3-ard.yaml b/tests/components/ultrasonic/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/ultrasonic/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/update/common.yaml b/tests/components/update/common.yaml
index 45ed110352..521a0a6a5c 100644
--- a/tests/components/update/common.yaml
+++ b/tests/components/update/common.yaml
@@ -21,10 +21,12 @@ http_request:
 
 ota:
   - platform: http_request
+    id: update_http_request_ota
 
 update:
   - platform: http_request
     name: Firmware Update
+    ota_id: update_http_request_ota
     source: http://example.com/manifest.json
     on_update_available:
       - logger.log: "A new update is available"
diff --git a/tests/components/update/test.esp32-ard.yaml b/tests/components/update/test.esp32-ard.yaml
deleted file mode 100644
index c1937b5a10..0000000000
--- a/tests/components/update/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  verify_ssl: "false"
-
-<<: !include common.yaml
diff --git a/tests/components/uponor_smatrix/common.yaml b/tests/components/uponor_smatrix/common.yaml
index 8ee92bdfc5..786a604aec 100644
--- a/tests/components/uponor_smatrix/common.yaml
+++ b/tests/components/uponor_smatrix/common.yaml
@@ -2,12 +2,6 @@ wifi:
   ssid: MySSID
   password: password1
 
-uart:
-  - id: uponor_uart
-    baud_rate: 19200
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-
 time:
   - platform: sntp
     id: sntp_time
@@ -17,7 +11,6 @@ time:
       - 192.168.178.1
 
 uponor_smatrix:
-  uart_id: uponor_uart
   address: 0x110B
   time_id: sntp_time
   time_device_address: 0xDE13
diff --git a/tests/components/uponor_smatrix/test.esp32-ard.yaml b/tests/components/uponor_smatrix/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/uponor_smatrix/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/uponor_smatrix/test.esp32-c3-ard.yaml b/tests/components/uponor_smatrix/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/uponor_smatrix/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/uponor_smatrix/test.esp32-c3-idf.yaml b/tests/components/uponor_smatrix/test.esp32-c3-idf.yaml
index b516342f3b..cd26c783c2 100644
--- a/tests/components/uponor_smatrix/test.esp32-c3-idf.yaml
+++ b/tests/components/uponor_smatrix/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart_19200: !include ../../test_build_components/common/uart_19200/esp32-c3-idf.yaml
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/uponor_smatrix/test.esp32-idf.yaml b/tests/components/uponor_smatrix/test.esp32-idf.yaml
index f486544afa..76222997a8 100644
--- a/tests/components/uponor_smatrix/test.esp32-idf.yaml
+++ b/tests/components/uponor_smatrix/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+packages:
+  uart_19200: !include ../../test_build_components/common/uart_19200/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/uponor_smatrix/test.esp8266-ard.yaml b/tests/components/uponor_smatrix/test.esp8266-ard.yaml
index b516342f3b..1f4483954b 100644
--- a/tests/components/uponor_smatrix/test.esp8266-ard.yaml
+++ b/tests/components/uponor_smatrix/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart_19200: !include ../../test_build_components/common/uart_19200/esp8266-ard.yaml
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/uponor_smatrix/test.rp2040-ard.yaml b/tests/components/uponor_smatrix/test.rp2040-ard.yaml
index b516342f3b..65ba185aef 100644
--- a/tests/components/uponor_smatrix/test.rp2040-ard.yaml
+++ b/tests/components/uponor_smatrix/test.rp2040-ard.yaml
@@ -1,5 +1,5 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart_19200: !include ../../test_build_components/common/uart_19200/rp2040-ard.yaml
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/uptime/test.esp32-ard.yaml b/tests/components/uptime/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/uptime/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/uptime/test.esp32-c3-ard.yaml b/tests/components/uptime/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/uptime/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/uptime/test.nrf52-adafruit.yaml b/tests/components/uptime/test.nrf52-adafruit.yaml
new file mode 100644
index 0000000000..3c3c814813
--- /dev/null
+++ b/tests/components/uptime/test.nrf52-adafruit.yaml
@@ -0,0 +1,10 @@
+sensor:
+  - platform: uptime
+    name: Uptime Sensor
+  - platform: uptime
+    name: Uptime Sensor Seconds
+    type: seconds
+
+text_sensor:
+  - platform: uptime
+    name: Uptime Text
diff --git a/tests/components/uptime/test.nrf52-mcumgr.yaml b/tests/components/uptime/test.nrf52-mcumgr.yaml
new file mode 100644
index 0000000000..3c3c814813
--- /dev/null
+++ b/tests/components/uptime/test.nrf52-mcumgr.yaml
@@ -0,0 +1,10 @@
+sensor:
+  - platform: uptime
+    name: Uptime Sensor
+  - platform: uptime
+    name: Uptime Sensor Seconds
+    type: seconds
+
+text_sensor:
+  - platform: uptime
+    name: Uptime Text
diff --git a/tests/components/usb_host/test.esp32-s3-idf.yaml b/tests/components/usb_host/test.esp32-s3-idf.yaml
index a2892872e5..5360d1f6ff 100644
--- a/tests/components/usb_host/test.esp32-s3-idf.yaml
+++ b/tests/components/usb_host/test.esp32-s3-idf.yaml
@@ -1,4 +1,5 @@
 usb_host:
+  max_transfer_requests: 32  # Test uint32_t bitmask path (17-32 requests)
   devices:
     - id: device_1
       vid: 0x1234
diff --git a/tests/components/vbus/common.yaml b/tests/components/vbus/common.yaml
index a1f94cd839..33d9e2935d 100644
--- a/tests/components/vbus/common.yaml
+++ b/tests/components/vbus/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_vbus
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 vbus:
 
 binary_sensor:
diff --git a/tests/components/vbus/test.esp32-ard.yaml b/tests/components/vbus/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/vbus/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/vbus/test.esp32-c3-ard.yaml b/tests/components/vbus/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/vbus/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/vbus/test.esp32-c3-idf.yaml b/tests/components/vbus/test.esp32-c3-idf.yaml
index b516342f3b..a19013bf54 100644
--- a/tests/components/vbus/test.esp32-c3-idf.yaml
+++ b/tests/components/vbus/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/vbus/test.esp32-idf.yaml b/tests/components/vbus/test.esp32-idf.yaml
index f486544afa..2d29656c94 100644
--- a/tests/components/vbus/test.esp32-idf.yaml
+++ b/tests/components/vbus/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/vbus/test.esp8266-ard.yaml b/tests/components/vbus/test.esp8266-ard.yaml
index b516342f3b..5a05efa259 100644
--- a/tests/components/vbus/test.esp8266-ard.yaml
+++ b/tests/components/vbus/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/vbus/test.rp2040-ard.yaml b/tests/components/vbus/test.rp2040-ard.yaml
index b516342f3b..f1df2daf83 100644
--- a/tests/components/vbus/test.rp2040-ard.yaml
+++ b/tests/components/vbus/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/veml3235/common.yaml b/tests/components/veml3235/common.yaml
index b89a9e12c7..98ffb0729c 100644
--- a/tests/components/veml3235/common.yaml
+++ b/tests/components/veml3235/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_veml3235
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: veml3235
+    i2c_id: i2c_bus
     id: veml3235_sensor
     name: VEML3235 Light Sensor
     auto_gain: true
diff --git a/tests/components/veml3235/test.esp32-ard.yaml b/tests/components/veml3235/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/veml3235/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/veml3235/test.esp32-c3-ard.yaml b/tests/components/veml3235/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/veml3235/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/veml3235/test.esp32-c3-idf.yaml b/tests/components/veml3235/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/veml3235/test.esp32-c3-idf.yaml
+++ b/tests/components/veml3235/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/veml3235/test.esp32-idf.yaml b/tests/components/veml3235/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/veml3235/test.esp32-idf.yaml
+++ b/tests/components/veml3235/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/veml3235/test.esp8266-ard.yaml b/tests/components/veml3235/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/veml3235/test.esp8266-ard.yaml
+++ b/tests/components/veml3235/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/veml3235/test.rp2040-ard.yaml b/tests/components/veml3235/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/veml3235/test.rp2040-ard.yaml
+++ b/tests/components/veml3235/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/veml7700/common.yaml b/tests/components/veml7700/common.yaml
index af4ebee6e7..06c1d304c3 100644
--- a/tests/components/veml7700/common.yaml
+++ b/tests/components/veml7700/common.yaml
@@ -1,12 +1,7 @@
-i2c:
-  - id: i2c_veml7700
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: veml7700
+    i2c_id: i2c_bus
     address: 0x10
-    i2c_id: i2c_veml7700
     ambient_light: Ambient light
     ambient_light_counts: Ambient light counts
     full_spectrum: Full spectrum
diff --git a/tests/components/veml7700/test.esp32-ard.yaml b/tests/components/veml7700/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/veml7700/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/veml7700/test.esp32-c3-ard.yaml b/tests/components/veml7700/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/veml7700/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/veml7700/test.esp32-c3-idf.yaml b/tests/components/veml7700/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/veml7700/test.esp32-c3-idf.yaml
+++ b/tests/components/veml7700/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/veml7700/test.esp32-idf.yaml b/tests/components/veml7700/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/veml7700/test.esp32-idf.yaml
+++ b/tests/components/veml7700/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/veml7700/test.esp8266-ard.yaml b/tests/components/veml7700/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/veml7700/test.esp8266-ard.yaml
+++ b/tests/components/veml7700/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/veml7700/test.rp2040-ard.yaml b/tests/components/veml7700/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/veml7700/test.rp2040-ard.yaml
+++ b/tests/components/veml7700/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/version/test.esp32-ard.yaml b/tests/components/version/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/version/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/version/test.esp32-c3-ard.yaml b/tests/components/version/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/version/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/duty_cycle/test.esp32-c3-ard.yaml b/tests/components/version/test.nrf52-adafruit.yaml
similarity index 100%
rename from tests/components/duty_cycle/test.esp32-c3-ard.yaml
rename to tests/components/version/test.nrf52-adafruit.yaml
diff --git a/tests/components/duty_time/test.esp32-ard.yaml b/tests/components/version/test.nrf52-mcumgr.yaml
similarity index 100%
rename from tests/components/duty_time/test.esp32-ard.yaml
rename to tests/components/version/test.nrf52-mcumgr.yaml
diff --git a/tests/components/vl53l0x/common.yaml b/tests/components/vl53l0x/common.yaml
index 8346eae854..98277639cf 100644
--- a/tests/components/vl53l0x/common.yaml
+++ b/tests/components/vl53l0x/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_vl53l0x
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: vl53l0x
+    i2c_id: i2c_bus
     name: VL53L0x Distance
     address: 0x29
     enable_pin: 3
diff --git a/tests/components/vl53l0x/test.esp32-ard.yaml b/tests/components/vl53l0x/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/vl53l0x/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/vl53l0x/test.esp32-c3-ard.yaml b/tests/components/vl53l0x/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/vl53l0x/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/vl53l0x/test.esp32-c3-idf.yaml b/tests/components/vl53l0x/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/vl53l0x/test.esp32-c3-idf.yaml
+++ b/tests/components/vl53l0x/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/vl53l0x/test.esp32-idf.yaml b/tests/components/vl53l0x/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/vl53l0x/test.esp32-idf.yaml
+++ b/tests/components/vl53l0x/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/vl53l0x/test.esp8266-ard.yaml b/tests/components/vl53l0x/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/vl53l0x/test.esp8266-ard.yaml
+++ b/tests/components/vl53l0x/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/vl53l0x/test.rp2040-ard.yaml b/tests/components/vl53l0x/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/vl53l0x/test.rp2040-ard.yaml
+++ b/tests/components/vl53l0x/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/voice_assistant/common-idf.yaml b/tests/components/voice_assistant/common-idf.yaml
index b1d249d5b4..ab8cbf2434 100644
--- a/tests/components/voice_assistant/common-idf.yaml
+++ b/tests/components/voice_assistant/common-idf.yaml
@@ -18,6 +18,7 @@ i2s_audio:
 
 micro_wake_word:
   id: mww_id
+  microphone: mic_id_external
   on_wake_word_detected:
     - voice_assistant.start:
         wake_word: !lambda return wake_word;
diff --git a/tests/components/voice_assistant/test.esp32-ard.yaml b/tests/components/voice_assistant/test.esp32-ard.yaml
deleted file mode 100644
index f6e553f9dc..0000000000
--- a/tests/components/voice_assistant/test.esp32-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  i2s_lrclk_pin: GPIO16
-  i2s_bclk_pin: GPIO17
-  i2s_mclk_pin: GPIO15
-  i2s_din_pin: GPIO13
-  i2s_dout_pin: GPIO12
-
-<<: !include common.yaml
diff --git a/tests/components/voice_assistant/test.esp32-c3-ard.yaml b/tests/components/voice_assistant/test.esp32-c3-ard.yaml
deleted file mode 100644
index f596d927cb..0000000000
--- a/tests/components/voice_assistant/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,8 +0,0 @@
-substitutions:
-  i2s_lrclk_pin: GPIO6
-  i2s_bclk_pin: GPIO7
-  i2s_mclk_pin: GPIO5
-  i2s_din_pin: GPIO3
-  i2s_dout_pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/voice_assistant/test.esp32-idf.yaml b/tests/components/voice_assistant/test.esp32-idf.yaml
index 0fe5d347be..1c5c9ddf99 100644
--- a/tests/components/voice_assistant/test.esp32-idf.yaml
+++ b/tests/components/voice_assistant/test.esp32-idf.yaml
@@ -1,6 +1,6 @@
 substitutions:
-  i2s_lrclk_pin: GPIO16
-  i2s_bclk_pin: GPIO17
+  i2s_lrclk_pin: GPIO4
+  i2s_bclk_pin: GPIO5
   i2s_mclk_pin: GPIO15
   i2s_din_pin: GPIO13
   i2s_dout_pin: GPIO12
diff --git a/tests/components/wake_on_lan/test.esp32-ard.yaml b/tests/components/wake_on_lan/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/wake_on_lan/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/wake_on_lan/test.esp32-c3-ard.yaml b/tests/components/wake_on_lan/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/wake_on_lan/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/wake_on_lan/test.esp32-c3-idf.yaml b/tests/components/wake_on_lan/test.esp32-c3-idf.yaml
index dade44d145..9990d96d29 100644
--- a/tests/components/wake_on_lan/test.esp32-c3-idf.yaml
+++ b/tests/components/wake_on_lan/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wake_on_lan/test.esp32-idf.yaml b/tests/components/wake_on_lan/test.esp32-idf.yaml
index dade44d145..b47e39c389 100644
--- a/tests/components/wake_on_lan/test.esp32-idf.yaml
+++ b/tests/components/wake_on_lan/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wake_on_lan/test.esp8266-ard.yaml b/tests/components/wake_on_lan/test.esp8266-ard.yaml
index dade44d145..4a98b9388a 100644
--- a/tests/components/wake_on_lan/test.esp8266-ard.yaml
+++ b/tests/components/wake_on_lan/test.esp8266-ard.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wake_on_lan/test.rp2040-ard.yaml b/tests/components/wake_on_lan/test.rp2040-ard.yaml
index dade44d145..319a7c71a6 100644
--- a/tests/components/wake_on_lan/test.rp2040-ard.yaml
+++ b/tests/components/wake_on_lan/test.rp2040-ard.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/waveshare_epaper/common.yaml b/tests/components/waveshare_epaper/common.yaml
index a2aa3134b5..b80a352c1f 100644
--- a/tests/components/waveshare_epaper/common.yaml
+++ b/tests/components/waveshare_epaper/common.yaml
@@ -1,14 +1,8 @@
-spi:
-  - id: spi_waveshare_epaper
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-
 display:
   # 1.54 inch displays
   - platform: waveshare_epaper
     id: epd_1_54
     model: 1.54in
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -28,7 +22,6 @@ display:
   - platform: waveshare_epaper
     id: epd_1_54v2
     model: 1.54inv2
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -48,7 +41,6 @@ display:
   - platform: waveshare_epaper
     id: epd_1_54v2b
     model: 1.54inv2-b
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -67,7 +59,6 @@ display:
   - platform: waveshare_epaper
     id: epd_1_54m09
     model: 1.54in-m5coreink-m09
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -87,7 +78,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_13
     model: 2.13in
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -107,7 +97,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_13v2
     model: 2.13inv2
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -127,7 +116,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_13ttgo
     model: 2.13in-ttgo
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -147,7 +135,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_13ttgo_b1
     model: 2.13in-ttgo-b1
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -167,7 +154,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_13ttgo_b73
     model: 2.13in-ttgo-b73
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -187,7 +173,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_13ttgo_b74
     model: 2.13in-ttgo-b74
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -207,7 +192,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_13dke
     model: 2.13in-ttgo-dke
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -227,7 +211,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_13v3
     model: 2.13inv3
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -248,7 +231,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_70
     model: 2.70in
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -267,7 +249,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_70b
     model: 2.70in-b
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -286,7 +267,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_70bv2
     model: 2.70in-bv2
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -305,7 +285,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_70v2
     model: 2.70inv2
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -325,7 +304,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_90
     model: 2.90in
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -346,7 +324,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_90v2
     model: 2.90inv2
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -367,7 +344,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_90b
     model: 2.90in-b
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -386,7 +362,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_90bv3
     model: 2.90in-bv3
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -405,7 +380,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_90v2r2
     model: 2.90inv2-r2
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -425,7 +399,6 @@ display:
   - platform: waveshare_epaper
     id: epd_2_90dke
     model: 2.90in-dke
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -446,7 +419,6 @@ display:
   - platform: waveshare_epaper
     id: epd_gdew029t5
     model: gdew029t5
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -466,7 +438,6 @@ display:
   - platform: waveshare_epaper
     id: epd_gdew042t81
     model: gdey042t81
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -487,7 +458,6 @@ display:
   - platform: waveshare_epaper
     id: epd_4_20
     model: 4.20in
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -506,7 +476,6 @@ display:
   - platform: waveshare_epaper
     id: epd_4_20bv2
     model: 4.20in-bv2
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -525,7 +494,6 @@ display:
   - platform: waveshare_epaper
     id: epd_4_20in_bv2_bwr
     model: 4.20in-bv2-bwr
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -545,7 +513,6 @@ display:
   - platform: waveshare_epaper
     id: epd_5_65
     model: 5.65in-f
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -565,7 +532,6 @@ display:
   - platform: waveshare_epaper
     id: epd_5_83
     model: 5.83in
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -584,7 +550,6 @@ display:
   - platform: waveshare_epaper
     id: epd_5_83v2
     model: 5.83inv2
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -604,7 +569,6 @@ display:
   - platform: waveshare_epaper
     id: epd_7_50
     model: 7.50in
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -623,7 +587,6 @@ display:
   - platform: waveshare_epaper
     id: epd_7_50bv2
     model: 7.50in-bv2
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -642,7 +605,6 @@ display:
   - platform: waveshare_epaper
     id: epd_7_50bv3
     model: 7.50in-bv3
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -661,7 +623,6 @@ display:
   - platform: waveshare_epaper
     id: epd_7_50bv3_bwr
     model: 7.50in-bv3-bwr
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -680,7 +641,6 @@ display:
   - platform: waveshare_epaper
     id: epd_7_50bc
     model: 7.50in-bc
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -699,7 +659,6 @@ display:
   - platform: waveshare_epaper
     id: epd_7_50v2
     model: 7.50inv2
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -718,7 +677,6 @@ display:
   - platform: waveshare_epaper
     id: epd_7_50v2alt
     model: 7.50inv2alt
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -737,7 +695,6 @@ display:
   - platform: waveshare_epaper
     id: epd_7_50inv2p
     model: 7.50inv2p
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -757,7 +714,6 @@ display:
   - platform: waveshare_epaper
     id: epd_7_50hdb
     model: 7.50in-hd-b
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -777,7 +733,6 @@ display:
   - platform: waveshare_epaper
     id: epd_13_3k
     model: 13.3in-k
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -795,7 +750,6 @@ display:
 
   - platform: waveshare_epaper
     model: 2.90in-d
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -814,7 +768,6 @@ display:
 
   - platform: waveshare_epaper
     model: 2.90in
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -834,7 +787,6 @@ display:
 
   - platform: waveshare_epaper
     model: 2.90inv2
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -853,7 +805,6 @@ display:
 
   - platform: waveshare_epaper
     model: 2.70in-b
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
@@ -871,7 +822,6 @@ display:
 
   - platform: waveshare_epaper
     model: 2.70in-bv2
-    spi_id: spi_waveshare_epaper
     cs_pin:
       allow_other_uses: true
       number: ${cs_pin}
diff --git a/tests/components/waveshare_epaper/test.esp32-ard.yaml b/tests/components/waveshare_epaper/test.esp32-ard.yaml
deleted file mode 100644
index c36345b984..0000000000
--- a/tests/components/waveshare_epaper/test.esp32-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO18
-  mosi_pin: GPIO23
-  cs_pin: GPIO25
-  dc_pin: GPIO26
-  busy_pin: GPIO27
-  reset_pin: GPIO32
-
-<<: !include common.yaml
diff --git a/tests/components/waveshare_epaper/test.esp32-c3-ard.yaml b/tests/components/waveshare_epaper/test.esp32-c3-ard.yaml
deleted file mode 100644
index 4cb230f6f2..0000000000
--- a/tests/components/waveshare_epaper/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,9 +0,0 @@
-substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  cs_pin: GPIO4
-  dc_pin: GPIO1
-  busy_pin: GPIO2
-  reset_pin: GPIO3
-
-<<: !include common.yaml
diff --git a/tests/components/waveshare_epaper/test.esp32-c3-idf.yaml b/tests/components/waveshare_epaper/test.esp32-c3-idf.yaml
index 4cb230f6f2..bdd7e1d350 100644
--- a/tests/components/waveshare_epaper/test.esp32-c3-idf.yaml
+++ b/tests/components/waveshare_epaper/test.esp32-c3-idf.yaml
@@ -1,9 +1,9 @@
 substitutions:
-  clk_pin: GPIO6
-  mosi_pin: GPIO7
-  cs_pin: GPIO4
+  cs_pin: GPIO7
   dc_pin: GPIO1
   busy_pin: GPIO2
   reset_pin: GPIO3
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/waveshare_epaper/test.esp32-idf.yaml b/tests/components/waveshare_epaper/test.esp32-idf.yaml
index 9e8b2fdec8..b7b12c8c66 100644
--- a/tests/components/waveshare_epaper/test.esp32-idf.yaml
+++ b/tests/components/waveshare_epaper/test.esp32-idf.yaml
@@ -1,9 +1,10 @@
 substitutions:
-  clk_pin: GPIO16
-  mosi_pin: GPIO17
   cs_pin: GPIO4
   dc_pin: GPIO5
-  busy_pin: GPIO18
-  reset_pin: GPIO19
+  busy_pin: GPIO14
+  reset_pin: GPIO15
+
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/waveshare_epaper/test.esp8266-ard.yaml b/tests/components/waveshare_epaper/test.esp8266-ard.yaml
index ee8199bcc0..92452c840a 100644
--- a/tests/components/waveshare_epaper/test.esp8266-ard.yaml
+++ b/tests/components/waveshare_epaper/test.esp8266-ard.yaml
@@ -1,9 +1,12 @@
 substitutions:
-  clk_pin: GPIO14
-  mosi_pin: GPIO13
+  clk_pin: GPIO0
+  mosi_pin: GPIO2
   cs_pin: GPIO4
   dc_pin: GPIO5
   busy_pin: GPIO15
   reset_pin: GPIO16
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/waveshare_epaper/test.rp2040-ard.yaml b/tests/components/waveshare_epaper/test.rp2040-ard.yaml
index e92f6d421d..e9e001cdd4 100644
--- a/tests/components/waveshare_epaper/test.rp2040-ard.yaml
+++ b/tests/components/waveshare_epaper/test.rp2040-ard.yaml
@@ -6,4 +6,7 @@ substitutions:
   busy_pin: GPIO7
   reset_pin: GPIO8
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/web_server/test.esp32-c3-ard.yaml b/tests/components/web_server/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7e6658e20e..0000000000
--- a/tests/components/web_server/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common_v2.yaml
diff --git a/tests/components/web_server/test.esp32-idf.yaml b/tests/components/web_server/test.esp32-idf.yaml
index 7e6658e20e..24b292d0d6 100644
--- a/tests/components/web_server/test.esp32-idf.yaml
+++ b/tests/components/web_server/test.esp32-idf.yaml
@@ -1 +1,6 @@
 <<: !include common_v2.yaml
+
+web_server:
+  auth:
+    username: admin
+    password: password
diff --git a/tests/components/web_server/test_ota.esp32-idf.yaml b/tests/components/web_server/test_ota.esp32-idf.yaml
index 99873aa27b..98b080386b 100644
--- a/tests/components/web_server/test_ota.esp32-idf.yaml
+++ b/tests/components/web_server/test_ota.esp32-idf.yaml
@@ -12,8 +12,10 @@ packages:
 # Enable OTA for multipart upload testing
 ota:
   - platform: esphome
+    id: web_server_esphome_ota
     password: "test_ota_password"
   - platform: web_server
+    id: web_server_web_server_ota
 
 # Web server configuration
 web_server:
diff --git a/tests/components/whirlpool/common.yaml b/tests/components/whirlpool/common.yaml
index 804c1aac26..6d55db1f08 100644
--- a/tests/components/whirlpool/common.yaml
+++ b/tests/components/whirlpool/common.yaml
@@ -1,7 +1,4 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: whirlpool
     name: Whirlpool Climate
+    transmitter_id: xmitr
diff --git a/tests/components/whirlpool/test.esp32-ard.yaml b/tests/components/whirlpool/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/whirlpool/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/whirlpool/test.esp32-c3-ard.yaml b/tests/components/whirlpool/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/whirlpool/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/whirlpool/test.esp32-c3-idf.yaml b/tests/components/whirlpool/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/whirlpool/test.esp32-c3-idf.yaml
+++ b/tests/components/whirlpool/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/whirlpool/test.esp32-idf.yaml b/tests/components/whirlpool/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/whirlpool/test.esp32-idf.yaml
+++ b/tests/components/whirlpool/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/whirlpool/test.esp8266-ard.yaml b/tests/components/whirlpool/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/whirlpool/test.esp8266-ard.yaml
+++ b/tests/components/whirlpool/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/whynter/common.yaml b/tests/components/whynter/common.yaml
index 04ad6bed54..63df11dd91 100644
--- a/tests/components/whynter/common.yaml
+++ b/tests/components/whynter/common.yaml
@@ -1,7 +1,4 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: whynter
     name: Whynter Climate
+    transmitter_id: xmitr
diff --git a/tests/components/whynter/test.esp32-ard.yaml b/tests/components/whynter/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/whynter/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/whynter/test.esp32-c3-ard.yaml b/tests/components/whynter/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/whynter/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/whynter/test.esp32-c3-idf.yaml b/tests/components/whynter/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/whynter/test.esp32-c3-idf.yaml
+++ b/tests/components/whynter/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/whynter/test.esp32-idf.yaml b/tests/components/whynter/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/whynter/test.esp32-idf.yaml
+++ b/tests/components/whynter/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/whynter/test.esp8266-ard.yaml b/tests/components/whynter/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/whynter/test.esp8266-ard.yaml
+++ b/tests/components/whynter/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/wiegand/test.esp32-ard.yaml b/tests/components/wiegand/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/wiegand/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/wiegand/test.esp32-c3-ard.yaml b/tests/components/wiegand/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/wiegand/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/wifi/test-eap.esp32-ard.yaml b/tests/components/wifi/test-eap.esp32-ard.yaml
deleted file mode 100644
index 9177e5de10..0000000000
--- a/tests/components/wifi/test-eap.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common-eap.yaml
diff --git a/tests/components/wifi/test.esp32-ard.yaml b/tests/components/wifi/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/wifi/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/wifi/test.esp32-c3-ard.yaml b/tests/components/wifi/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/wifi/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/wifi/test.esp32-idf.yaml b/tests/components/wifi/test.esp32-idf.yaml
index dade44d145..91e235b9ce 100644
--- a/tests/components/wifi/test.esp32-idf.yaml
+++ b/tests/components/wifi/test.esp32-idf.yaml
@@ -1 +1,7 @@
-<<: !include common.yaml
+psram:
+
+wifi:
+  use_psram: true
+
+packages:
+  - !include common.yaml
diff --git a/tests/components/wifi_info/test.esp32-ard.yaml b/tests/components/wifi_info/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/wifi_info/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/wifi_info/test.esp32-c3-ard.yaml b/tests/components/wifi_info/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/wifi_info/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/wifi_info/test.esp32-c3-idf.yaml b/tests/components/wifi_info/test.esp32-c3-idf.yaml
index dade44d145..9990d96d29 100644
--- a/tests/components/wifi_info/test.esp32-c3-idf.yaml
+++ b/tests/components/wifi_info/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wifi_info/test.esp32-idf.yaml b/tests/components/wifi_info/test.esp32-idf.yaml
index dade44d145..b47e39c389 100644
--- a/tests/components/wifi_info/test.esp32-idf.yaml
+++ b/tests/components/wifi_info/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wifi_info/test.esp8266-ard.yaml b/tests/components/wifi_info/test.esp8266-ard.yaml
index dade44d145..4a98b9388a 100644
--- a/tests/components/wifi_info/test.esp8266-ard.yaml
+++ b/tests/components/wifi_info/test.esp8266-ard.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wifi_info/test.rp2040-ard.yaml b/tests/components/wifi_info/test.rp2040-ard.yaml
index dade44d145..319a7c71a6 100644
--- a/tests/components/wifi_info/test.rp2040-ard.yaml
+++ b/tests/components/wifi_info/test.rp2040-ard.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wifi_signal/test.esp32-ard.yaml b/tests/components/wifi_signal/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/wifi_signal/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/wifi_signal/test.esp32-c3-ard.yaml b/tests/components/wifi_signal/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/wifi_signal/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/wireguard/test.esp32-ard.yaml b/tests/components/wireguard/test.esp32-ard.yaml
deleted file mode 100644
index 2798f8e566..0000000000
--- a/tests/components/wireguard/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-<<: !include common.yaml
-
-network:
-  enable_ipv6: true
diff --git a/tests/components/wireguard/test.esp32-c3-ard.yaml b/tests/components/wireguard/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/wireguard/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/wireguard/test.esp32-c3-idf.yaml b/tests/components/wireguard/test.esp32-c3-idf.yaml
index dade44d145..9990d96d29 100644
--- a/tests/components/wireguard/test.esp32-c3-idf.yaml
+++ b/tests/components/wireguard/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wireguard/test.esp32-idf.yaml b/tests/components/wireguard/test.esp32-idf.yaml
index 2798f8e566..90dbc1cf7d 100644
--- a/tests/components/wireguard/test.esp32-idf.yaml
+++ b/tests/components/wireguard/test.esp32-idf.yaml
@@ -1,3 +1,6 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
 <<: !include common.yaml
 
 network:
diff --git a/tests/components/wireguard/test.esp8266-ard.yaml b/tests/components/wireguard/test.esp8266-ard.yaml
index 2798f8e566..ae6a3d6ce0 100644
--- a/tests/components/wireguard/test.esp8266-ard.yaml
+++ b/tests/components/wireguard/test.esp8266-ard.yaml
@@ -1,3 +1,6 @@
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
+
 <<: !include common.yaml
 
 network:
diff --git a/tests/components/wk2132_i2c/common.yaml b/tests/components/wk2132_i2c/common.yaml
index 942e01aafc..39013baeb2 100644
--- a/tests/components/wk2132_i2c/common.yaml
+++ b/tests/components/wk2132_i2c/common.yaml
@@ -1,14 +1,7 @@
-i2c:
-  id: i2c_bus
-  scl: ${scl_pin}
-  sda: ${sda_pin}
-  scan: true
-  frequency: 600kHz
-
 wk2132_i2c:
   - id: wk2132_i2c_id
-    address: 0x70
     i2c_id: i2c_bus
+    address: 0x70
     uart:
       - id: wk2132_id_0
         channel: 0
diff --git a/tests/components/wk2132_i2c/test.esp32-ard.yaml b/tests/components/wk2132_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 3b761d3fc1..0000000000
--- a/tests/components/wk2132_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
-
-<<: !include common.yaml
diff --git a/tests/components/wk2132_i2c/test.esp32-idf.yaml b/tests/components/wk2132_i2c/test.esp32-idf.yaml
index 3b761d3fc1..6b748a8f20 100644
--- a/tests/components/wk2132_i2c/test.esp32-idf.yaml
+++ b/tests/components/wk2132_i2c/test.esp32-idf.yaml
@@ -1,5 +1,5 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+  uart_bridge_2: !include ../../test_build_components/common/uart_bridge_2/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/wk2132_i2c/test.esp32-s3-ard.yaml b/tests/components/wk2132_i2c/test.esp32-s3-ard.yaml
deleted file mode 100644
index 4942e3c2b3..0000000000
--- a/tests/components/wk2132_i2c/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO40
-  sda_pin: GPIO41
-
-<<: !include common.yaml
diff --git a/tests/components/wk2132_i2c/test.esp32-s3-idf.yaml b/tests/components/wk2132_i2c/test.esp32-s3-idf.yaml
index 4942e3c2b3..d7b149a6fd 100644
--- a/tests/components/wk2132_i2c/test.esp32-s3-idf.yaml
+++ b/tests/components/wk2132_i2c/test.esp32-s3-idf.yaml
@@ -2,4 +2,8 @@ substitutions:
   scl_pin: GPIO40
   sda_pin: GPIO41
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-s3-idf.yaml
+  uart_bridge_2: !include ../../test_build_components/common/uart_bridge_2/esp32-s3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wk2132_spi/common.yaml b/tests/components/wk2132_spi/common.yaml
index a077b36998..18294974b9 100644
--- a/tests/components/wk2132_spi/common.yaml
+++ b/tests/components/wk2132_spi/common.yaml
@@ -1,27 +1,20 @@
-spi:
-  id: spi_bus
-  clk_pin: ${clk_pin}
-  mosi_pin: ${mosi_pin}
-  miso_pin: ${miso_pin}
-
 wk2132_spi:
-  - id: wk2132_spi_id
+  - id: wk2132_spi_bridge
     cs_pin: ${cs_pin}
-    spi_id: spi_bus
     crystal: 11059200
     data_rate: 1MHz
     uart:
-      - id: wk2132_spi_id0
+      - id: wk2132_spi_uart0
         channel: 0
         baud_rate: 115200
         stop_bits: 1
         parity: none
-      - id: wk2132_spi_id1
+      - id: wk2132_spi_uart1
         channel: 1
         baud_rate: 9600
 
 # Ensures a sensor doesn't break validation
 sensor:
   - platform: a02yyuw
-    uart_id: wk2132_spi_id1
+    uart_id: wk2132_spi_uart1
     id: distance_sensor
diff --git a/tests/components/wk2132_spi/test.esp32-ard.yaml b/tests/components/wk2132_spi/test.esp32-ard.yaml
deleted file mode 100644
index 76e7138ab0..0000000000
--- a/tests/components/wk2132_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO18
-  miso_pin: GPIO19
-  mosi_pin: GPIO23
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/wk2132_spi/test.esp32-idf.yaml b/tests/components/wk2132_spi/test.esp32-idf.yaml
index 76e7138ab0..9202a691ba 100644
--- a/tests/components/wk2132_spi/test.esp32-idf.yaml
+++ b/tests/components/wk2132_spi/test.esp32-idf.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  clk_pin: GPIO18
-  miso_pin: GPIO19
-  mosi_pin: GPIO23
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+  uart_bridge_2: !include ../../test_build_components/common/uart_bridge_2/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wk2132_spi/test.esp32-s3-ard.yaml b/tests/components/wk2132_spi/test.esp32-s3-ard.yaml
deleted file mode 100644
index b0aadf620a..0000000000
--- a/tests/components/wk2132_spi/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO40
-  miso_pin: GPIO41
-  mosi_pin: GPIO6
-  cs_pin: GPIO19
-
-<<: !include common.yaml
diff --git a/tests/components/wk2132_spi/test.esp32-s3-idf.yaml b/tests/components/wk2132_spi/test.esp32-s3-idf.yaml
index b0aadf620a..9c7d36996e 100644
--- a/tests/components/wk2132_spi/test.esp32-s3-idf.yaml
+++ b/tests/components/wk2132_spi/test.esp32-s3-idf.yaml
@@ -4,4 +4,8 @@ substitutions:
   mosi_pin: GPIO6
   cs_pin: GPIO19
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-s3-idf.yaml
+  uart_bridge_2: !include ../../test_build_components/common/uart_bridge_2/esp32-s3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wk2168_i2c/common.yaml b/tests/components/wk2168_i2c/common.yaml
index 10463e8abf..49f0d1ec6b 100644
--- a/tests/components/wk2168_i2c/common.yaml
+++ b/tests/components/wk2168_i2c/common.yaml
@@ -1,35 +1,28 @@
-i2c:
-  id: i2c_bus
-  scl: ${scl_pin}
-  sda: ${sda_pin}
-  scan: true
-  frequency: 600kHz
-
 #  component declaration
 wk2168_i2c:
-  - id: bridge_i2c
+  - id: wk2168_i2c_bridge
     i2c_id: i2c_bus
     address: 0x70
     uart:
-      - id: id0
+      - id: wk2168_i2c_uart0
         channel: 0
         baud_rate: 115200
         stop_bits: 1
         parity: none
-      - id: id1
+      - id: wk2168_i2c_uart1
         channel: 1
         baud_rate: 115200
-      - id: id2
+      - id: wk2168_i2c_uart2
         channel: 2
         baud_rate: 115200
-      - id: id3
+      - id: wk2168_i2c_uart3
         channel: 3
         baud_rate: 9600
 
 # Ensures a sensor doesn't break validation
 sensor:
   - platform: a02yyuw
-    uart_id: id3
+    uart_id: wk2168_i2c_uart3
     id: distance_sensor
 
 # individual binary_sensor inputs
@@ -37,14 +30,14 @@ binary_sensor:
   - platform: gpio
     name: "pin_0"
     pin:
-      wk2168_i2c: bridge_i2c
+      wk2168_i2c: wk2168_i2c_bridge
       number: 0
       mode:
         input: true
   - platform: gpio
     name: "pin_1"
     pin:
-      wk2168_i2c: bridge_i2c
+      wk2168_i2c: wk2168_i2c_bridge
       number: 1
       mode:
         input: true
@@ -55,14 +48,14 @@ switch:
   - platform: gpio
     name: "pin_2"
     pin:
-      wk2168_i2c: bridge_i2c
+      wk2168_i2c: wk2168_i2c_bridge
       number: 2
       mode:
         output: true
   - platform: gpio
     name: "pin_3"
     pin:
-      wk2168_i2c: bridge_i2c
+      wk2168_i2c: wk2168_i2c_bridge
       number: 3
       mode:
         output: true
diff --git a/tests/components/wk2168_i2c/test.esp32-ard.yaml b/tests/components/wk2168_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 3b761d3fc1..0000000000
--- a/tests/components/wk2168_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
-
-<<: !include common.yaml
diff --git a/tests/components/wk2168_i2c/test.esp32-idf.yaml b/tests/components/wk2168_i2c/test.esp32-idf.yaml
index 3b761d3fc1..9d9f0d4931 100644
--- a/tests/components/wk2168_i2c/test.esp32-idf.yaml
+++ b/tests/components/wk2168_i2c/test.esp32-idf.yaml
@@ -1,5 +1,5 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+  uart_bridge_4: !include ../../test_build_components/common/uart_bridge_4/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/wk2168_i2c/test.esp32-s3-ard.yaml b/tests/components/wk2168_i2c/test.esp32-s3-ard.yaml
deleted file mode 100644
index 4942e3c2b3..0000000000
--- a/tests/components/wk2168_i2c/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO40
-  sda_pin: GPIO41
-
-<<: !include common.yaml
diff --git a/tests/components/wk2168_i2c/test.esp32-s3-idf.yaml b/tests/components/wk2168_i2c/test.esp32-s3-idf.yaml
index 4942e3c2b3..115812be97 100644
--- a/tests/components/wk2168_i2c/test.esp32-s3-idf.yaml
+++ b/tests/components/wk2168_i2c/test.esp32-s3-idf.yaml
@@ -2,4 +2,8 @@ substitutions:
   scl_pin: GPIO40
   sda_pin: GPIO41
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-s3-idf.yaml
+  uart_bridge_4: !include ../../test_build_components/common/uart_bridge_4/esp32-s3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wk2168_spi/common.yaml b/tests/components/wk2168_spi/common.yaml
index fb126193fc..b402077aa3 100644
--- a/tests/components/wk2168_spi/common.yaml
+++ b/tests/components/wk2168_spi/common.yaml
@@ -1,35 +1,28 @@
-spi:
-  id: spi_bus
-  clk_pin: ${clk_pin}
-  mosi_pin: ${mosi_pin}
-  miso_pin: ${miso_pin}
-
 wk2168_spi:
-  - id: bridge_spi
+  - id: wk2168_spi_bridge
     cs_pin: ${cs_pin}
-    spi_id: spi_bus
     crystal: 11059200
     data_rate: 1MHz
     uart:
-      - id: id0
+      - id: wk2168_spi_uart0
         channel: 0
         baud_rate: 115200
         stop_bits: 1
         parity: none
-      - id: id1
+      - id: wk2168_spi_uart1
         channel: 1
         baud_rate: 115200
-      - id: id2
+      - id: wk2168_spi_uart2
         channel: 2
         baud_rate: 115200
-      - id: id3
+      - id: wk2168_spi_uart3
         channel: 3
         baud_rate: 9600
 
 # Ensures a sensor doesn't break validation
 sensor:
   - platform: a02yyuw
-    uart_id: id3
+    uart_id: wk2168_spi_uart3
     id: distance_sensor
 
 # individual binary_sensor inputs
@@ -37,14 +30,14 @@ binary_sensor:
   - platform: gpio
     name: "pin_0"
     pin:
-      wk2168_spi: bridge_spi
+      wk2168_spi: wk2168_spi_bridge
       number: 0
       mode:
         input: true
   - platform: gpio
     name: "pin_1"
     pin:
-      wk2168_spi: bridge_spi
+      wk2168_spi: wk2168_spi_bridge
       number: 1
       mode:
         input: true
@@ -55,14 +48,14 @@ switch:
   - platform: gpio
     name: "pin_2"
     pin:
-      wk2168_spi: bridge_spi
+      wk2168_spi: wk2168_spi_bridge
       number: 2
       mode:
         output: true
   - platform: gpio
     name: "pin_3"
     pin:
-      wk2168_spi: bridge_spi
+      wk2168_spi: wk2168_spi_bridge
       number: 3
       mode:
         output: true
diff --git a/tests/components/wk2168_spi/test.esp32-ard.yaml b/tests/components/wk2168_spi/test.esp32-ard.yaml
deleted file mode 100644
index 76e7138ab0..0000000000
--- a/tests/components/wk2168_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO18
-  miso_pin: GPIO19
-  mosi_pin: GPIO23
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/wk2168_spi/test.esp32-idf.yaml b/tests/components/wk2168_spi/test.esp32-idf.yaml
index 76e7138ab0..2b56a46b70 100644
--- a/tests/components/wk2168_spi/test.esp32-idf.yaml
+++ b/tests/components/wk2168_spi/test.esp32-idf.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  clk_pin: GPIO18
-  miso_pin: GPIO19
-  mosi_pin: GPIO23
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+  uart_bridge_4: !include ../../test_build_components/common/uart_bridge_4/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wk2168_spi/test.esp32-s3-ard.yaml b/tests/components/wk2168_spi/test.esp32-s3-ard.yaml
deleted file mode 100644
index b0aadf620a..0000000000
--- a/tests/components/wk2168_spi/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO40
-  miso_pin: GPIO41
-  mosi_pin: GPIO6
-  cs_pin: GPIO19
-
-<<: !include common.yaml
diff --git a/tests/components/wk2168_spi/test.esp32-s3-idf.yaml b/tests/components/wk2168_spi/test.esp32-s3-idf.yaml
index b0aadf620a..374fe64d16 100644
--- a/tests/components/wk2168_spi/test.esp32-s3-idf.yaml
+++ b/tests/components/wk2168_spi/test.esp32-s3-idf.yaml
@@ -4,4 +4,8 @@ substitutions:
   mosi_pin: GPIO6
   cs_pin: GPIO19
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-s3-idf.yaml
+  uart_bridge_4: !include ../../test_build_components/common/uart_bridge_4/esp32-s3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wk2204_i2c/common.yaml b/tests/components/wk2204_i2c/common.yaml
index 70c0f4babf..863633937b 100644
--- a/tests/components/wk2204_i2c/common.yaml
+++ b/tests/components/wk2204_i2c/common.yaml
@@ -1,10 +1,3 @@
-i2c:
-  id: i2c_bus
-  scl: ${scl_pin}
-  sda: ${sda_pin}
-  scan: true
-  frequency: 600kHz
-
 wk2204_i2c:
   - id: wk2204_i2c_id
     i2c_id: i2c_bus
diff --git a/tests/components/wk2204_i2c/test.esp32-ard.yaml b/tests/components/wk2204_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 3b761d3fc1..0000000000
--- a/tests/components/wk2204_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
-
-<<: !include common.yaml
diff --git a/tests/components/wk2204_i2c/test.esp32-idf.yaml b/tests/components/wk2204_i2c/test.esp32-idf.yaml
index 3b761d3fc1..9d9f0d4931 100644
--- a/tests/components/wk2204_i2c/test.esp32-idf.yaml
+++ b/tests/components/wk2204_i2c/test.esp32-idf.yaml
@@ -1,5 +1,5 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+  uart_bridge_4: !include ../../test_build_components/common/uart_bridge_4/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/wk2204_i2c/test.esp32-s3-ard.yaml b/tests/components/wk2204_i2c/test.esp32-s3-ard.yaml
deleted file mode 100644
index 4942e3c2b3..0000000000
--- a/tests/components/wk2204_i2c/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO40
-  sda_pin: GPIO41
-
-<<: !include common.yaml
diff --git a/tests/components/wk2204_i2c/test.esp32-s3-idf.yaml b/tests/components/wk2204_i2c/test.esp32-s3-idf.yaml
index 4942e3c2b3..115812be97 100644
--- a/tests/components/wk2204_i2c/test.esp32-s3-idf.yaml
+++ b/tests/components/wk2204_i2c/test.esp32-s3-idf.yaml
@@ -2,4 +2,8 @@ substitutions:
   scl_pin: GPIO40
   sda_pin: GPIO41
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-s3-idf.yaml
+  uart_bridge_4: !include ../../test_build_components/common/uart_bridge_4/esp32-s3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wk2204_spi/common.yaml b/tests/components/wk2204_spi/common.yaml
index a08cdb906f..0b62a7a009 100644
--- a/tests/components/wk2204_spi/common.yaml
+++ b/tests/components/wk2204_spi/common.yaml
@@ -1,35 +1,28 @@
-spi:
-  id: spi_bus
-  clk_pin: ${clk_pin}
-  mosi_pin: ${mosi_pin}
-  miso_pin: ${miso_pin}
-
 wk2204_spi:
-  - id: wk2204_spi_id
+  - id: wk2204_spi_bridge
     cs_pin: ${cs_pin}
-    spi_id: spi_bus
     crystal: 11059200
     data_rate: 1MHz
     uart:
-      - id: wk2204_spi_id0
+      - id: wk2204_spi_uart0
         channel: 0
         baud_rate: 115200
         stop_bits: 1
         parity: none
-      - id: wk2204_spi_id1
+      - id: wk2204_spi_uart1
         channel: 1
         baud_rate: 921600
-      - id: wk2204_spi_id2
+      - id: wk2204_spi_uart2
         channel: 2
         baud_rate: 115200
         stop_bits: 1
         parity: none
-      - id: wk2204_spi_id3
+      - id: wk2204_spi_uart3
         channel: 3
         baud_rate: 9600
 
 # Ensures a sensor doesn't break validation
 sensor:
   - platform: a02yyuw
-    uart_id: wk2204_spi_id3
+    uart_id: wk2204_spi_uart3
     id: distance_sensor
diff --git a/tests/components/wk2204_spi/test.esp32-ard.yaml b/tests/components/wk2204_spi/test.esp32-ard.yaml
deleted file mode 100644
index 76e7138ab0..0000000000
--- a/tests/components/wk2204_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO18
-  miso_pin: GPIO19
-  mosi_pin: GPIO23
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/wk2204_spi/test.esp32-idf.yaml b/tests/components/wk2204_spi/test.esp32-idf.yaml
index 76e7138ab0..2b56a46b70 100644
--- a/tests/components/wk2204_spi/test.esp32-idf.yaml
+++ b/tests/components/wk2204_spi/test.esp32-idf.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  clk_pin: GPIO18
-  miso_pin: GPIO19
-  mosi_pin: GPIO23
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+  uart_bridge_4: !include ../../test_build_components/common/uart_bridge_4/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wk2204_spi/test.esp32-s3-ard.yaml b/tests/components/wk2204_spi/test.esp32-s3-ard.yaml
deleted file mode 100644
index b0aadf620a..0000000000
--- a/tests/components/wk2204_spi/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO40
-  miso_pin: GPIO41
-  mosi_pin: GPIO6
-  cs_pin: GPIO19
-
-<<: !include common.yaml
diff --git a/tests/components/wk2204_spi/test.esp32-s3-idf.yaml b/tests/components/wk2204_spi/test.esp32-s3-idf.yaml
index b0aadf620a..374fe64d16 100644
--- a/tests/components/wk2204_spi/test.esp32-s3-idf.yaml
+++ b/tests/components/wk2204_spi/test.esp32-s3-idf.yaml
@@ -4,4 +4,8 @@ substitutions:
   mosi_pin: GPIO6
   cs_pin: GPIO19
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-s3-idf.yaml
+  uart_bridge_4: !include ../../test_build_components/common/uart_bridge_4/esp32-s3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wk2212_i2c/common.yaml b/tests/components/wk2212_i2c/common.yaml
index 0759ef8688..a754bec5c7 100644
--- a/tests/components/wk2212_i2c/common.yaml
+++ b/tests/components/wk2212_i2c/common.yaml
@@ -1,13 +1,6 @@
-i2c:
-  id: i2c_bus
-  scl: ${scl_pin}
-  sda: ${sda_pin}
-  scan: true
-  frequency: 600kHz
-
 #  component declaration
 wk2212_i2c:
-  - id: bridge_i2c
+  - id: wk2212_i2c_bridge
     i2c_id: i2c_bus
     address: 0x70
     uart:
@@ -33,14 +26,14 @@ binary_sensor:
   - platform: gpio
     name: "pin_0"
     pin:
-      wk2212_i2c: bridge_i2c
+      wk2212_i2c: wk2212_i2c_bridge
       number: 0
       mode:
         input: true
   - platform: gpio
     name: "pin_1"
     pin:
-      wk2212_i2c: bridge_i2c
+      wk2212_i2c: wk2212_i2c_bridge
       number: 1
       mode:
         input: true
@@ -51,14 +44,14 @@ switch:
   - platform: gpio
     name: "pin_2"
     pin:
-      wk2212_i2c: bridge_i2c
+      wk2212_i2c: wk2212_i2c_bridge
       number: 2
       mode:
         output: true
   - platform: gpio
     name: "pin_3"
     pin:
-      wk2212_i2c: bridge_i2c
+      wk2212_i2c: wk2212_i2c_bridge
       number: 3
       mode:
         output: true
diff --git a/tests/components/wk2212_i2c/test.esp32-ard.yaml b/tests/components/wk2212_i2c/test.esp32-ard.yaml
deleted file mode 100644
index 3b761d3fc1..0000000000
--- a/tests/components/wk2212_i2c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
-
-<<: !include common.yaml
diff --git a/tests/components/wk2212_i2c/test.esp32-idf.yaml b/tests/components/wk2212_i2c/test.esp32-idf.yaml
index 3b761d3fc1..9d9f0d4931 100644
--- a/tests/components/wk2212_i2c/test.esp32-idf.yaml
+++ b/tests/components/wk2212_i2c/test.esp32-idf.yaml
@@ -1,5 +1,5 @@
-substitutions:
-  scl_pin: GPIO22
-  sda_pin: GPIO21
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+  uart_bridge_4: !include ../../test_build_components/common/uart_bridge_4/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/wk2212_i2c/test.esp32-s3-ard.yaml b/tests/components/wk2212_i2c/test.esp32-s3-ard.yaml
deleted file mode 100644
index 4942e3c2b3..0000000000
--- a/tests/components/wk2212_i2c/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO40
-  sda_pin: GPIO41
-
-<<: !include common.yaml
diff --git a/tests/components/wk2212_i2c/test.esp32-s3-idf.yaml b/tests/components/wk2212_i2c/test.esp32-s3-idf.yaml
index 4942e3c2b3..115812be97 100644
--- a/tests/components/wk2212_i2c/test.esp32-s3-idf.yaml
+++ b/tests/components/wk2212_i2c/test.esp32-s3-idf.yaml
@@ -2,4 +2,8 @@ substitutions:
   scl_pin: GPIO40
   sda_pin: GPIO41
 
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-s3-idf.yaml
+  uart_bridge_4: !include ../../test_build_components/common/uart_bridge_4/esp32-s3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wk2212_spi/common.yaml b/tests/components/wk2212_spi/common.yaml
index 693d2a9ab2..969f16bb12 100644
--- a/tests/components/wk2212_spi/common.yaml
+++ b/tests/components/wk2212_spi/common.yaml
@@ -1,29 +1,22 @@
-spi:
-  id: spi_bus
-  clk_pin: ${clk_pin}
-  mosi_pin: ${mosi_pin}
-  miso_pin: ${miso_pin}
-
 wk2212_spi:
-  - id: bridge_spi
+  - id: wk2212_spi_bridge
     cs_pin: ${cs_pin}
-    spi_id: spi_bus
     crystal: 11059200
     data_rate: 1MHz
     uart:
-      - id: id0
+      - id: wk2212_spi_uart0
         channel: 0
         baud_rate: 115200
         stop_bits: 1
         parity: none
-      - id: id1
+      - id: wk2212_spi_uart1
         channel: 1
         baud_rate: 9600
 
 # Ensures a sensor doesn't break validation
 sensor:
   - platform: a02yyuw
-    uart_id: id1
+    uart_id: wk2212_spi_uart1
     id: distance_sensor
 
 # individual binary_sensor inputs
@@ -31,14 +24,14 @@ binary_sensor:
   - platform: gpio
     name: "pin_0"
     pin:
-      wk2212_spi: bridge_spi
+      wk2212_spi: wk2212_spi_bridge
       number: 0
       mode:
         input: true
   - platform: gpio
     name: "pin_1"
     pin:
-      wk2212_spi: bridge_spi
+      wk2212_spi: wk2212_spi_bridge
       number: 1
       mode:
         input: true
@@ -49,14 +42,14 @@ switch:
   - platform: gpio
     name: "pin_2"
     pin:
-      wk2212_spi: bridge_spi
+      wk2212_spi: wk2212_spi_bridge
       number: 2
       mode:
         output: true
   - platform: gpio
     name: "pin_3"
     pin:
-      wk2212_spi: bridge_spi
+      wk2212_spi: wk2212_spi_bridge
       number: 3
       mode:
         output: true
diff --git a/tests/components/wk2212_spi/test.esp32-ard.yaml b/tests/components/wk2212_spi/test.esp32-ard.yaml
deleted file mode 100644
index 76e7138ab0..0000000000
--- a/tests/components/wk2212_spi/test.esp32-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO18
-  miso_pin: GPIO19
-  mosi_pin: GPIO23
-  cs_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/wk2212_spi/test.esp32-idf.yaml b/tests/components/wk2212_spi/test.esp32-idf.yaml
index 76e7138ab0..2b56a46b70 100644
--- a/tests/components/wk2212_spi/test.esp32-idf.yaml
+++ b/tests/components/wk2212_spi/test.esp32-idf.yaml
@@ -1,7 +1,8 @@
 substitutions:
-  clk_pin: GPIO18
-  miso_pin: GPIO19
-  mosi_pin: GPIO23
   cs_pin: GPIO5
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+  uart_bridge_4: !include ../../test_build_components/common/uart_bridge_4/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wk2212_spi/test.esp32-s3-ard.yaml b/tests/components/wk2212_spi/test.esp32-s3-ard.yaml
deleted file mode 100644
index b0aadf620a..0000000000
--- a/tests/components/wk2212_spi/test.esp32-s3-ard.yaml
+++ /dev/null
@@ -1,7 +0,0 @@
-substitutions:
-  clk_pin: GPIO40
-  miso_pin: GPIO41
-  mosi_pin: GPIO6
-  cs_pin: GPIO19
-
-<<: !include common.yaml
diff --git a/tests/components/wk2212_spi/test.esp32-s3-idf.yaml b/tests/components/wk2212_spi/test.esp32-s3-idf.yaml
index b0aadf620a..374fe64d16 100644
--- a/tests/components/wk2212_spi/test.esp32-s3-idf.yaml
+++ b/tests/components/wk2212_spi/test.esp32-s3-idf.yaml
@@ -4,4 +4,8 @@ substitutions:
   mosi_pin: GPIO6
   cs_pin: GPIO19
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-s3-idf.yaml
+  uart_bridge_4: !include ../../test_build_components/common/uart_bridge_4/esp32-s3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wl_134/common.yaml b/tests/components/wl_134/common.yaml
index 71c50be79b..e79331cb52 100644
--- a/tests/components/wl_134/common.yaml
+++ b/tests/components/wl_134/common.yaml
@@ -1,9 +1,3 @@
-uart:
-  - id: uart_wl_134
-    tx_pin: ${tx_pin}
-    rx_pin: ${rx_pin}
-    baud_rate: 9600
-
 text_sensor:
   - platform: wl_134
     name: Transponder Code
diff --git a/tests/components/wl_134/test.esp32-ard.yaml b/tests/components/wl_134/test.esp32-ard.yaml
deleted file mode 100644
index f486544afa..0000000000
--- a/tests/components/wl_134/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
-
-<<: !include common.yaml
diff --git a/tests/components/wl_134/test.esp32-c3-ard.yaml b/tests/components/wl_134/test.esp32-c3-ard.yaml
deleted file mode 100644
index b516342f3b..0000000000
--- a/tests/components/wl_134/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/wl_134/test.esp32-c3-idf.yaml b/tests/components/wl_134/test.esp32-c3-idf.yaml
index b516342f3b..4b7c8351a7 100644
--- a/tests/components/wl_134/test.esp32-c3-idf.yaml
+++ b/tests/components/wl_134/test.esp32-c3-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/wl_134/test.esp32-idf.yaml b/tests/components/wl_134/test.esp32-idf.yaml
index f486544afa..b415125e84 100644
--- a/tests/components/wl_134/test.esp32-idf.yaml
+++ b/tests/components/wl_134/test.esp32-idf.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO17
-  rx_pin: GPIO16
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/wl_134/test.esp8266-ard.yaml b/tests/components/wl_134/test.esp8266-ard.yaml
index b516342f3b..96ab4ef6ac 100644
--- a/tests/components/wl_134/test.esp8266-ard.yaml
+++ b/tests/components/wl_134/test.esp8266-ard.yaml
@@ -1,5 +1,8 @@
 substitutions:
-  tx_pin: GPIO4
-  rx_pin: GPIO5
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/wl_134/test.rp2040-ard.yaml b/tests/components/wl_134/test.rp2040-ard.yaml
index b516342f3b..b28f2b5e05 100644
--- a/tests/components/wl_134/test.rp2040-ard.yaml
+++ b/tests/components/wl_134/test.rp2040-ard.yaml
@@ -2,4 +2,7 @@ substitutions:
   tx_pin: GPIO4
   rx_pin: GPIO5
 
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/wled/test.esp32-c3-ard.yaml b/tests/components/wled/test.esp32-c3-ard.yaml
deleted file mode 100644
index 156b31181e..0000000000
--- a/tests/components/wled/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,16 +0,0 @@
-wifi:
-  ssid: MySSID
-  password: password1
-
-wled:
-
-light:
-  - platform: esp32_rmt_led_strip
-    id: led_matrix_32x8
-    default_transition_length: 500ms
-    chipset: ws2812
-    rgb_order: GRB
-    num_leds: 256
-    pin: 2
-    effects:
-      - wled:
diff --git a/tests/components/wts01/common.yaml b/tests/components/wts01/common.yaml
new file mode 100644
index 0000000000..966588c82a
--- /dev/null
+++ b/tests/components/wts01/common.yaml
@@ -0,0 +1,3 @@
+sensor:
+  - platform: wts01
+    id: wts01_sensor
diff --git a/tests/components/wts01/test.esp32-c3-idf.yaml b/tests/components/wts01/test.esp32-c3-idf.yaml
new file mode 100644
index 0000000000..4b7c8351a7
--- /dev/null
+++ b/tests/components/wts01/test.esp32-c3-idf.yaml
@@ -0,0 +1,5 @@
+substitutions:
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/wts01/test.esp32-idf.yaml b/tests/components/wts01/test.esp32-idf.yaml
new file mode 100644
index 0000000000..b415125e84
--- /dev/null
+++ b/tests/components/wts01/test.esp32-idf.yaml
@@ -0,0 +1,8 @@
+substitutions:
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/wts01/test.esp8266-ard.yaml b/tests/components/wts01/test.esp8266-ard.yaml
new file mode 100644
index 0000000000..89ca3ab5ae
--- /dev/null
+++ b/tests/components/wts01/test.esp8266-ard.yaml
@@ -0,0 +1,8 @@
+substitutions:
+  tx_pin: GPIO1
+  rx_pin: GPIO3
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/wts01/test.rp2040-ard.yaml b/tests/components/wts01/test.rp2040-ard.yaml
new file mode 100644
index 0000000000..9246c39f08
--- /dev/null
+++ b/tests/components/wts01/test.rp2040-ard.yaml
@@ -0,0 +1,8 @@
+substitutions:
+  tx_pin: GPIO0
+  rx_pin: GPIO1
+
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/x9c/test.esp32-ard.yaml b/tests/components/x9c/test.esp32-ard.yaml
deleted file mode 100644
index 6dfe3a67eb..0000000000
--- a/tests/components/x9c/test.esp32-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  cs_pin: GPIO13
-  inc_pin: GPIO14
-  ud_pin: GPIO15
-
-<<: !include common.yaml
diff --git a/tests/components/x9c/test.esp32-c3-ard.yaml b/tests/components/x9c/test.esp32-c3-ard.yaml
deleted file mode 100644
index b06e15a98c..0000000000
--- a/tests/components/x9c/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
-substitutions:
-  cs_pin: GPIO3
-  inc_pin: GPIO4
-  ud_pin: GPIO5
-
-<<: !include common.yaml
diff --git a/tests/components/x9c/test.esp32-idf.yaml b/tests/components/x9c/test.esp32-idf.yaml
index 6dfe3a67eb..7beb6e67cb 100644
--- a/tests/components/x9c/test.esp32-idf.yaml
+++ b/tests/components/x9c/test.esp32-idf.yaml
@@ -3,4 +3,7 @@ substitutions:
   inc_pin: GPIO14
   ud_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/x9c/test.esp8266-ard.yaml b/tests/components/x9c/test.esp8266-ard.yaml
index 6dfe3a67eb..ae6b775ff7 100644
--- a/tests/components/x9c/test.esp8266-ard.yaml
+++ b/tests/components/x9c/test.esp8266-ard.yaml
@@ -1,6 +1,9 @@
 substitutions:
-  cs_pin: GPIO13
-  inc_pin: GPIO14
+  cs_pin: GPIO0
+  inc_pin: GPIO2
   ud_pin: GPIO15
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/x9c/test.rp2040-ard.yaml b/tests/components/x9c/test.rp2040-ard.yaml
index b06e15a98c..f17627baf3 100644
--- a/tests/components/x9c/test.rp2040-ard.yaml
+++ b/tests/components/x9c/test.rp2040-ard.yaml
@@ -1,6 +1,9 @@
 substitutions:
-  cs_pin: GPIO3
-  inc_pin: GPIO4
-  ud_pin: GPIO5
+  cs_pin: GPIO6
+  inc_pin: GPIO7
+  ud_pin: GPIO8
+
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/xgzp68xx/common.yaml b/tests/components/xgzp68xx/common.yaml
index 224dd9ed14..00e51e764c 100644
--- a/tests/components/xgzp68xx/common.yaml
+++ b/tests/components/xgzp68xx/common.yaml
@@ -1,12 +1,9 @@
-i2c:
-  - id: i2c_xgzp68xx
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: xgzp68xx
+    i2c_id: i2c_bus
     k_value: 4096
     temperature:
       name: Pressure Temperature
     pressure:
       name: Differential pressure
+      oversampling: 1024x
diff --git a/tests/components/xgzp68xx/test.esp32-ard.yaml b/tests/components/xgzp68xx/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/xgzp68xx/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/xgzp68xx/test.esp32-c3-ard.yaml b/tests/components/xgzp68xx/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/xgzp68xx/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/xgzp68xx/test.esp32-c3-idf.yaml b/tests/components/xgzp68xx/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/xgzp68xx/test.esp32-c3-idf.yaml
+++ b/tests/components/xgzp68xx/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/xgzp68xx/test.esp32-idf.yaml b/tests/components/xgzp68xx/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/xgzp68xx/test.esp32-idf.yaml
+++ b/tests/components/xgzp68xx/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/xgzp68xx/test.esp8266-ard.yaml b/tests/components/xgzp68xx/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/xgzp68xx/test.esp8266-ard.yaml
+++ b/tests/components/xgzp68xx/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/xgzp68xx/test.rp2040-ard.yaml b/tests/components/xgzp68xx/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/xgzp68xx/test.rp2040-ard.yaml
+++ b/tests/components/xgzp68xx/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_ble/test.esp32-ard.yaml b/tests/components/xiaomi_ble/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_ble/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_ble/test.esp32-c3-ard.yaml b/tests/components/xiaomi_ble/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_ble/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_ble/test.esp32-c3-idf.yaml b/tests/components/xiaomi_ble/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_ble/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_ble/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_ble/test.esp32-idf.yaml b/tests/components/xiaomi_ble/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_ble/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_ble/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_cgd1/test.esp32-ard.yaml b/tests/components/xiaomi_cgd1/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_cgd1/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_cgd1/test.esp32-c3-ard.yaml b/tests/components/xiaomi_cgd1/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_cgd1/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_cgd1/test.esp32-c3-idf.yaml b/tests/components/xiaomi_cgd1/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_cgd1/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_cgd1/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_cgd1/test.esp32-idf.yaml b/tests/components/xiaomi_cgd1/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_cgd1/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_cgd1/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_cgdk2/test.esp32-ard.yaml b/tests/components/xiaomi_cgdk2/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_cgdk2/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_cgdk2/test.esp32-c3-ard.yaml b/tests/components/xiaomi_cgdk2/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_cgdk2/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_cgdk2/test.esp32-c3-idf.yaml b/tests/components/xiaomi_cgdk2/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_cgdk2/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_cgdk2/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_cgdk2/test.esp32-idf.yaml b/tests/components/xiaomi_cgdk2/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_cgdk2/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_cgdk2/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_cgg1/test.esp32-ard.yaml b/tests/components/xiaomi_cgg1/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_cgg1/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_cgg1/test.esp32-c3-ard.yaml b/tests/components/xiaomi_cgg1/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_cgg1/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_cgg1/test.esp32-c3-idf.yaml b/tests/components/xiaomi_cgg1/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_cgg1/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_cgg1/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_cgg1/test.esp32-idf.yaml b/tests/components/xiaomi_cgg1/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_cgg1/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_cgg1/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_cgpr1/test.esp32-ard.yaml b/tests/components/xiaomi_cgpr1/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_cgpr1/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_cgpr1/test.esp32-c3-ard.yaml b/tests/components/xiaomi_cgpr1/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_cgpr1/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_cgpr1/test.esp32-c3-idf.yaml b/tests/components/xiaomi_cgpr1/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_cgpr1/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_cgpr1/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_cgpr1/test.esp32-idf.yaml b/tests/components/xiaomi_cgpr1/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_cgpr1/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_cgpr1/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_gcls002/test.esp32-ard.yaml b/tests/components/xiaomi_gcls002/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_gcls002/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_gcls002/test.esp32-c3-ard.yaml b/tests/components/xiaomi_gcls002/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_gcls002/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_gcls002/test.esp32-c3-idf.yaml b/tests/components/xiaomi_gcls002/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_gcls002/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_gcls002/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_gcls002/test.esp32-idf.yaml b/tests/components/xiaomi_gcls002/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_gcls002/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_gcls002/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_hhccjcy01/test.esp32-ard.yaml b/tests/components/xiaomi_hhccjcy01/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_hhccjcy01/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_hhccjcy01/test.esp32-c3-ard.yaml b/tests/components/xiaomi_hhccjcy01/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_hhccjcy01/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_hhccjcy01/test.esp32-c3-idf.yaml b/tests/components/xiaomi_hhccjcy01/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_hhccjcy01/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_hhccjcy01/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_hhccjcy01/test.esp32-idf.yaml b/tests/components/xiaomi_hhccjcy01/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_hhccjcy01/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_hhccjcy01/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_hhccpot002/test.esp32-ard.yaml b/tests/components/xiaomi_hhccpot002/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_hhccpot002/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_hhccpot002/test.esp32-c3-ard.yaml b/tests/components/xiaomi_hhccpot002/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_hhccpot002/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_hhccpot002/test.esp32-c3-idf.yaml b/tests/components/xiaomi_hhccpot002/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_hhccpot002/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_hhccpot002/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_hhccpot002/test.esp32-idf.yaml b/tests/components/xiaomi_hhccpot002/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_hhccpot002/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_hhccpot002/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_jqjcy01ym/test.esp32-ard.yaml b/tests/components/xiaomi_jqjcy01ym/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_jqjcy01ym/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_jqjcy01ym/test.esp32-c3-ard.yaml b/tests/components/xiaomi_jqjcy01ym/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_jqjcy01ym/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_jqjcy01ym/test.esp32-c3-idf.yaml b/tests/components/xiaomi_jqjcy01ym/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_jqjcy01ym/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_jqjcy01ym/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_jqjcy01ym/test.esp32-idf.yaml b/tests/components/xiaomi_jqjcy01ym/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_jqjcy01ym/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_jqjcy01ym/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsd02/test.esp32-ard.yaml b/tests/components/xiaomi_lywsd02/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_lywsd02/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsd02/test.esp32-c3-ard.yaml b/tests/components/xiaomi_lywsd02/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_lywsd02/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsd02/test.esp32-c3-idf.yaml b/tests/components/xiaomi_lywsd02/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_lywsd02/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_lywsd02/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsd02/test.esp32-idf.yaml b/tests/components/xiaomi_lywsd02/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_lywsd02/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_lywsd02/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsd02mmc/test.esp32-ard.yaml b/tests/components/xiaomi_lywsd02mmc/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_lywsd02mmc/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsd02mmc/test.esp32-c3-ard.yaml b/tests/components/xiaomi_lywsd02mmc/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_lywsd02mmc/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsd02mmc/test.esp32-c3-idf.yaml b/tests/components/xiaomi_lywsd02mmc/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_lywsd02mmc/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_lywsd02mmc/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsd02mmc/test.esp32-idf.yaml b/tests/components/xiaomi_lywsd02mmc/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_lywsd02mmc/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_lywsd02mmc/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsd03mmc/test.esp32-ard.yaml b/tests/components/xiaomi_lywsd03mmc/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_lywsd03mmc/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsd03mmc/test.esp32-c3-ard.yaml b/tests/components/xiaomi_lywsd03mmc/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_lywsd03mmc/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsd03mmc/test.esp32-c3-idf.yaml b/tests/components/xiaomi_lywsd03mmc/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_lywsd03mmc/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_lywsd03mmc/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsd03mmc/test.esp32-idf.yaml b/tests/components/xiaomi_lywsd03mmc/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_lywsd03mmc/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_lywsd03mmc/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsdcgq/test.esp32-ard.yaml b/tests/components/xiaomi_lywsdcgq/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_lywsdcgq/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsdcgq/test.esp32-c3-ard.yaml b/tests/components/xiaomi_lywsdcgq/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_lywsdcgq/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsdcgq/test.esp32-c3-idf.yaml b/tests/components/xiaomi_lywsdcgq/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_lywsdcgq/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_lywsdcgq/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_lywsdcgq/test.esp32-idf.yaml b/tests/components/xiaomi_lywsdcgq/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_lywsdcgq/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_lywsdcgq/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_mhoc303/test.esp32-ard.yaml b/tests/components/xiaomi_mhoc303/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_mhoc303/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_mhoc303/test.esp32-c3-ard.yaml b/tests/components/xiaomi_mhoc303/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_mhoc303/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_mhoc303/test.esp32-c3-idf.yaml b/tests/components/xiaomi_mhoc303/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_mhoc303/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_mhoc303/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_mhoc303/test.esp32-idf.yaml b/tests/components/xiaomi_mhoc303/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_mhoc303/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_mhoc303/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_mhoc401/test.esp32-ard.yaml b/tests/components/xiaomi_mhoc401/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_mhoc401/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_mhoc401/test.esp32-c3-ard.yaml b/tests/components/xiaomi_mhoc401/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_mhoc401/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_mhoc401/test.esp32-c3-idf.yaml b/tests/components/xiaomi_mhoc401/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_mhoc401/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_mhoc401/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_mhoc401/test.esp32-idf.yaml b/tests/components/xiaomi_mhoc401/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_mhoc401/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_mhoc401/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_miscale copy/common.yaml b/tests/components/xiaomi_miscale copy/common.yaml
deleted file mode 100644
index 89f32ad199..0000000000
--- a/tests/components/xiaomi_miscale copy/common.yaml	
+++ /dev/null
@@ -1,9 +0,0 @@
-esp32_ble_tracker:
-
-sensor:
-  - platform: xiaomi_miscale
-    mac_address: '5C:CA:D3:70:D4:A2'
-    weight:
-      name: "Xiaomi Mi Scale Weight"
-    impedance:
-      name: "Xiaomi Mi Scale Impedance"
diff --git a/tests/components/xiaomi_miscale copy/test.esp32-ard.yaml b/tests/components/xiaomi_miscale copy/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_miscale copy/test.esp32-ard.yaml	
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_miscale copy/test.esp32-c3-ard.yaml b/tests/components/xiaomi_miscale copy/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_miscale copy/test.esp32-c3-ard.yaml	
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_miscale copy/test.esp32-c3-idf.yaml b/tests/components/xiaomi_miscale copy/test.esp32-c3-idf.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_miscale copy/test.esp32-c3-idf.yaml	
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_miscale copy/test.esp32-idf.yaml b/tests/components/xiaomi_miscale copy/test.esp32-idf.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_miscale copy/test.esp32-idf.yaml	
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_miscale/test.esp32-ard.yaml b/tests/components/xiaomi_miscale/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_miscale/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_miscale/test.esp32-c3-ard.yaml b/tests/components/xiaomi_miscale/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_miscale/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_miscale/test.esp32-c3-idf.yaml b/tests/components/xiaomi_miscale/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_miscale/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_miscale/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_miscale/test.esp32-idf.yaml b/tests/components/xiaomi_miscale/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_miscale/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_miscale/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_mjyd02yla/test.esp32-ard.yaml b/tests/components/xiaomi_mjyd02yla/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_mjyd02yla/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_mjyd02yla/test.esp32-c3-ard.yaml b/tests/components/xiaomi_mjyd02yla/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_mjyd02yla/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_mjyd02yla/test.esp32-c3-idf.yaml b/tests/components/xiaomi_mjyd02yla/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_mjyd02yla/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_mjyd02yla/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_mjyd02yla/test.esp32-idf.yaml b/tests/components/xiaomi_mjyd02yla/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_mjyd02yla/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_mjyd02yla/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_mue4094rt/test.esp32-ard.yaml b/tests/components/xiaomi_mue4094rt/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_mue4094rt/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_mue4094rt/test.esp32-c3-ard.yaml b/tests/components/xiaomi_mue4094rt/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_mue4094rt/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_mue4094rt/test.esp32-c3-idf.yaml b/tests/components/xiaomi_mue4094rt/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_mue4094rt/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_mue4094rt/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_mue4094rt/test.esp32-idf.yaml b/tests/components/xiaomi_mue4094rt/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_mue4094rt/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_mue4094rt/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_rtcgq02lm/test.esp32-ard.yaml b/tests/components/xiaomi_rtcgq02lm/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_rtcgq02lm/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_rtcgq02lm/test.esp32-c3-ard.yaml b/tests/components/xiaomi_rtcgq02lm/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_rtcgq02lm/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_rtcgq02lm/test.esp32-c3-idf.yaml b/tests/components/xiaomi_rtcgq02lm/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_rtcgq02lm/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_rtcgq02lm/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_rtcgq02lm/test.esp32-idf.yaml b/tests/components/xiaomi_rtcgq02lm/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_rtcgq02lm/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_rtcgq02lm/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_wx08zm/test.esp32-ard.yaml b/tests/components/xiaomi_wx08zm/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_wx08zm/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_wx08zm/test.esp32-c3-ard.yaml b/tests/components/xiaomi_wx08zm/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_wx08zm/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_wx08zm/test.esp32-c3-idf.yaml b/tests/components/xiaomi_wx08zm/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_wx08zm/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_wx08zm/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_wx08zm/test.esp32-idf.yaml b/tests/components/xiaomi_wx08zm/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_wx08zm/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_wx08zm/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_xmwsdj04mmc/test.esp32-ard.yaml b/tests/components/xiaomi_xmwsdj04mmc/test.esp32-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_xmwsdj04mmc/test.esp32-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_xmwsdj04mmc/test.esp32-c3-ard.yaml b/tests/components/xiaomi_xmwsdj04mmc/test.esp32-c3-ard.yaml
deleted file mode 100644
index dade44d145..0000000000
--- a/tests/components/xiaomi_xmwsdj04mmc/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1 +0,0 @@
-<<: !include common.yaml
diff --git a/tests/components/xiaomi_xmwsdj04mmc/test.esp32-c3-idf.yaml b/tests/components/xiaomi_xmwsdj04mmc/test.esp32-c3-idf.yaml
index dade44d145..9f2634f967 100644
--- a/tests/components/xiaomi_xmwsdj04mmc/test.esp32-c3-idf.yaml
+++ b/tests/components/xiaomi_xmwsdj04mmc/test.esp32-c3-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-c3-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xiaomi_xmwsdj04mmc/test.esp32-idf.yaml b/tests/components/xiaomi_xmwsdj04mmc/test.esp32-idf.yaml
index dade44d145..7a6541ae76 100644
--- a/tests/components/xiaomi_xmwsdj04mmc/test.esp32-idf.yaml
+++ b/tests/components/xiaomi_xmwsdj04mmc/test.esp32-idf.yaml
@@ -1 +1,4 @@
+packages:
+  ble: !include ../../test_build_components/common/ble/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xl9535/common.yaml b/tests/components/xl9535/common.yaml
index e01163cf12..81e96131ab 100644
--- a/tests/components/xl9535/common.yaml
+++ b/tests/components/xl9535/common.yaml
@@ -1,10 +1,6 @@
-i2c:
-  - id: i2c_xl9535
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 xl9535:
   - id: xl9535_hub
+    i2c_id: i2c_bus
     address: 0x20
 
 binary_sensor:
diff --git a/tests/components/xl9535/test.esp32-ard.yaml b/tests/components/xl9535/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/xl9535/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/xl9535/test.esp32-c3-ard.yaml b/tests/components/xl9535/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/xl9535/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/xl9535/test.esp32-c3-idf.yaml b/tests/components/xl9535/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/xl9535/test.esp32-c3-idf.yaml
+++ b/tests/components/xl9535/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/xl9535/test.esp32-idf.yaml b/tests/components/xl9535/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/xl9535/test.esp32-idf.yaml
+++ b/tests/components/xl9535/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/xl9535/test.esp8266-ard.yaml b/tests/components/xl9535/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/xl9535/test.esp8266-ard.yaml
+++ b/tests/components/xl9535/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/xl9535/test.rp2040-ard.yaml b/tests/components/xl9535/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/xl9535/test.rp2040-ard.yaml
+++ b/tests/components/xl9535/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/xpt2046/common.yaml b/tests/components/xpt2046/common.yaml
index 9ef680cff4..3a8e3286a6 100644
--- a/tests/components/xpt2046/common.yaml
+++ b/tests/components/xpt2046/common.yaml
@@ -1,9 +1,3 @@
-spi:
-  - id: spi_xpt2046
-    clk_pin: ${clk_pin}
-    mosi_pin: ${mosi_pin}
-    miso_pin: ${miso_pin}
-
 display:
   - platform: ili9xxx
     id: xpt_display
diff --git a/tests/components/xpt2046/test.esp32-ard.yaml b/tests/components/xpt2046/test.esp32-ard.yaml
deleted file mode 100644
index b39174947b..0000000000
--- a/tests/components/xpt2046/test.esp32-ard.yaml
+++ /dev/null
@@ -1,11 +0,0 @@
-substitutions:
-  clk_pin: GPIO17
-  mosi_pin: GPIO18
-  miso_pin: GPIO19
-  dc_pin: GPIO13
-  cs_pin: GPIO14
-  disp_cs_pin: GPIO4
-  interrupt_pin: GPIO21
-  reset_pin: GPIO22
-
-<<: !include common.yaml
diff --git a/tests/components/xpt2046/test.esp32-c3-ard.yaml b/tests/components/xpt2046/test.esp32-c3-ard.yaml
deleted file mode 100644
index 79b84902ac..0000000000
--- a/tests/components/xpt2046/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,11 +0,0 @@
-substitutions:
-  clk_pin: GPIO4
-  mosi_pin: GPIO5
-  miso_pin: GPIO6
-  dc_pin: GPIO7
-  cs_pin: GPIO0
-  disp_cs_pin: GPIO1
-  interrupt_pin: GPIO3
-  reset_pin: GPIO10
-
-<<: !include common.yaml
diff --git a/tests/components/xpt2046/test.esp32-c3-idf.yaml b/tests/components/xpt2046/test.esp32-c3-idf.yaml
index 79b84902ac..ff7a32c26c 100644
--- a/tests/components/xpt2046/test.esp32-c3-idf.yaml
+++ b/tests/components/xpt2046/test.esp32-c3-idf.yaml
@@ -1,11 +1,10 @@
 substitutions:
-  clk_pin: GPIO4
-  mosi_pin: GPIO5
-  miso_pin: GPIO6
   dc_pin: GPIO7
   cs_pin: GPIO0
   disp_cs_pin: GPIO1
   interrupt_pin: GPIO3
   reset_pin: GPIO10
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/xpt2046/test.esp32-idf.yaml b/tests/components/xpt2046/test.esp32-idf.yaml
index b39174947b..608e135d74 100644
--- a/tests/components/xpt2046/test.esp32-idf.yaml
+++ b/tests/components/xpt2046/test.esp32-idf.yaml
@@ -1,11 +1,11 @@
 substitutions:
-  clk_pin: GPIO17
-  mosi_pin: GPIO18
-  miso_pin: GPIO19
   dc_pin: GPIO13
   cs_pin: GPIO14
   disp_cs_pin: GPIO4
   interrupt_pin: GPIO21
   reset_pin: GPIO22
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp32-idf.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xpt2046/test.esp8266-ard.yaml b/tests/components/xpt2046/test.esp8266-ard.yaml
index 246c5c8953..e2779a03b9 100644
--- a/tests/components/xpt2046/test.esp8266-ard.yaml
+++ b/tests/components/xpt2046/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clk_pin: GPIO14
+  clk_pin: GPIO0
   mosi_pin: GPIO13
   miso_pin: GPIO12
   dc_pin: GPIO15
@@ -8,4 +8,7 @@ substitutions:
   interrupt_pin: GPIO5
   reset_pin: GPIO2
 
+packages:
+  spi: !include ../../test_build_components/common/spi/esp8266-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/xpt2046/test.rp2040-ard.yaml b/tests/components/xpt2046/test.rp2040-ard.yaml
index e693b363d9..c547bdc0c9 100644
--- a/tests/components/xpt2046/test.rp2040-ard.yaml
+++ b/tests/components/xpt2046/test.rp2040-ard.yaml
@@ -8,4 +8,7 @@ substitutions:
   interrupt_pin: GPIO2
   reset_pin: GPIO3
 
+packages:
+  spi: !include ../../test_build_components/common/spi/rp2040-ard.yaml
+
 <<: !include common.yaml
diff --git a/tests/components/yashima/common.yaml b/tests/components/yashima/common.yaml
index bfe181f1a6..431c27ebb3 100644
--- a/tests/components/yashima/common.yaml
+++ b/tests/components/yashima/common.yaml
@@ -1,7 +1,4 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: yashima
     name: Yashima Climate
+    transmitter_id: xmitr
diff --git a/tests/components/yashima/test.esp32-ard.yaml b/tests/components/yashima/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/yashima/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/yashima/test.esp32-c3-ard.yaml b/tests/components/yashima/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/yashima/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/yashima/test.esp32-c3-idf.yaml b/tests/components/yashima/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/yashima/test.esp32-c3-idf.yaml
+++ b/tests/components/yashima/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/yashima/test.esp32-idf.yaml b/tests/components/yashima/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/yashima/test.esp32-idf.yaml
+++ b/tests/components/yashima/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/yashima/test.esp8266-ard.yaml b/tests/components/yashima/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/yashima/test.esp8266-ard.yaml
+++ b/tests/components/yashima/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/zhlt01/common.yaml b/tests/components/zhlt01/common.yaml
index 0adbe77325..d0fd531c87 100644
--- a/tests/components/zhlt01/common.yaml
+++ b/tests/components/zhlt01/common.yaml
@@ -1,7 +1,4 @@
-remote_transmitter:
-  pin: ${pin}
-  carrier_duty_percent: 50%
-
 climate:
   - platform: zhlt01
     name: ZH/LT-01 Climate
+    transmitter_id: xmitr
diff --git a/tests/components/zhlt01/test.esp32-ard.yaml b/tests/components/zhlt01/test.esp32-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/zhlt01/test.esp32-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/zhlt01/test.esp32-c3-ard.yaml b/tests/components/zhlt01/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7b012aa64c..0000000000
--- a/tests/components/zhlt01/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,4 +0,0 @@
-substitutions:
-  pin: GPIO2
-
-<<: !include common.yaml
diff --git a/tests/components/zhlt01/test.esp32-c3-idf.yaml b/tests/components/zhlt01/test.esp32-c3-idf.yaml
index 7b012aa64c..43d5343715 100644
--- a/tests/components/zhlt01/test.esp32-c3-idf.yaml
+++ b/tests/components/zhlt01/test.esp32-c3-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/zhlt01/test.esp32-idf.yaml b/tests/components/zhlt01/test.esp32-idf.yaml
index 7b012aa64c..e891f9dc85 100644
--- a/tests/components/zhlt01/test.esp32-idf.yaml
+++ b/tests/components/zhlt01/test.esp32-idf.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO2
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/zhlt01/test.esp8266-ard.yaml b/tests/components/zhlt01/test.esp8266-ard.yaml
index f5097fcf5f..4bed2f03e5 100644
--- a/tests/components/zhlt01/test.esp8266-ard.yaml
+++ b/tests/components/zhlt01/test.esp8266-ard.yaml
@@ -1,4 +1,4 @@
-substitutions:
-  pin: GPIO5
+packages:
+  remote_transmitter: !include ../../test_build_components/common/remote_transmitter/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/zio_ultrasonic/common.yaml b/tests/components/zio_ultrasonic/common.yaml
index e13853d8f1..34a3144db9 100644
--- a/tests/components/zio_ultrasonic/common.yaml
+++ b/tests/components/zio_ultrasonic/common.yaml
@@ -1,9 +1,5 @@
-i2c:
-  - id: i2c_zio_ultrasonic
-    scl: ${scl_pin}
-    sda: ${sda_pin}
-
 sensor:
   - platform: zio_ultrasonic
+    i2c_id: i2c_bus
     name: "Distance"
     update_interval: 60s
diff --git a/tests/components/zio_ultrasonic/test.esp32-ard.yaml b/tests/components/zio_ultrasonic/test.esp32-ard.yaml
deleted file mode 100644
index 63c3bd6afd..0000000000
--- a/tests/components/zio_ultrasonic/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/zio_ultrasonic/test.esp32-c3-ard.yaml b/tests/components/zio_ultrasonic/test.esp32-c3-ard.yaml
deleted file mode 100644
index ee2c29ca4e..0000000000
--- a/tests/components/zio_ultrasonic/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/zio_ultrasonic/test.esp32-c3-idf.yaml b/tests/components/zio_ultrasonic/test.esp32-c3-idf.yaml
index ee2c29ca4e..9990d96d29 100644
--- a/tests/components/zio_ultrasonic/test.esp32-c3-idf.yaml
+++ b/tests/components/zio_ultrasonic/test.esp32-c3-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-c3-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/zio_ultrasonic/test.esp32-idf.yaml b/tests/components/zio_ultrasonic/test.esp32-idf.yaml
index 63c3bd6afd..b47e39c389 100644
--- a/tests/components/zio_ultrasonic/test.esp32-idf.yaml
+++ b/tests/components/zio_ultrasonic/test.esp32-idf.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO16
-  sda_pin: GPIO17
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/zio_ultrasonic/test.esp8266-ard.yaml b/tests/components/zio_ultrasonic/test.esp8266-ard.yaml
index ee2c29ca4e..4a98b9388a 100644
--- a/tests/components/zio_ultrasonic/test.esp8266-ard.yaml
+++ b/tests/components/zio_ultrasonic/test.esp8266-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp8266-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/zio_ultrasonic/test.rp2040-ard.yaml b/tests/components/zio_ultrasonic/test.rp2040-ard.yaml
index ee2c29ca4e..319a7c71a6 100644
--- a/tests/components/zio_ultrasonic/test.rp2040-ard.yaml
+++ b/tests/components/zio_ultrasonic/test.rp2040-ard.yaml
@@ -1,5 +1,4 @@
-substitutions:
-  scl_pin: GPIO5
-  sda_pin: GPIO4
+packages:
+  i2c: !include ../../test_build_components/common/i2c/rp2040-ard.yaml
 
 <<: !include common.yaml
diff --git a/tests/components/zwave_proxy/common.yaml b/tests/components/zwave_proxy/common.yaml
new file mode 100644
index 0000000000..097e4b43a5
--- /dev/null
+++ b/tests/components/zwave_proxy/common.yaml
@@ -0,0 +1,9 @@
+wifi:
+  ssid: MySSID
+  password: password1
+  power_save_mode: none
+
+api:
+
+zwave_proxy:
+  id: zw_proxy
diff --git a/tests/components/zwave_proxy/test.esp32-c3-idf.yaml b/tests/components/zwave_proxy/test.esp32-c3-idf.yaml
new file mode 100644
index 0000000000..4b7c8351a7
--- /dev/null
+++ b/tests/components/zwave_proxy/test.esp32-c3-idf.yaml
@@ -0,0 +1,5 @@
+substitutions:
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-c3-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/zwave_proxy/test.esp32-idf.yaml b/tests/components/zwave_proxy/test.esp32-idf.yaml
new file mode 100644
index 0000000000..b415125e84
--- /dev/null
+++ b/tests/components/zwave_proxy/test.esp32-idf.yaml
@@ -0,0 +1,8 @@
+substitutions:
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp32-idf.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/zwave_proxy/test.esp8266-ard.yaml b/tests/components/zwave_proxy/test.esp8266-ard.yaml
new file mode 100644
index 0000000000..96ab4ef6ac
--- /dev/null
+++ b/tests/components/zwave_proxy/test.esp8266-ard.yaml
@@ -0,0 +1,8 @@
+substitutions:
+  tx_pin: GPIO0
+  rx_pin: GPIO2
+
+packages:
+  uart: !include ../../test_build_components/common/uart/esp8266-ard.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/zwave_proxy/test.rp2040-ard.yaml b/tests/components/zwave_proxy/test.rp2040-ard.yaml
new file mode 100644
index 0000000000..b28f2b5e05
--- /dev/null
+++ b/tests/components/zwave_proxy/test.rp2040-ard.yaml
@@ -0,0 +1,8 @@
+substitutions:
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+packages:
+  uart: !include ../../test_build_components/common/uart/rp2040-ard.yaml
+
+<<: !include common.yaml
diff --git a/tests/components/zyaura/test.esp32-ard.yaml b/tests/components/zyaura/test.esp32-ard.yaml
deleted file mode 100644
index d295973e3f..0000000000
--- a/tests/components/zyaura/test.esp32-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
-
-<<: !include common.yaml
diff --git a/tests/components/zyaura/test.esp32-c3-ard.yaml b/tests/components/zyaura/test.esp32-c3-ard.yaml
deleted file mode 100644
index 7808481215..0000000000
--- a/tests/components/zyaura/test.esp32-c3-ard.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
-substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
-
-<<: !include common.yaml
diff --git a/tests/components/zyaura/test.esp32-idf.yaml b/tests/components/zyaura/test.esp32-idf.yaml
index d295973e3f..a4ecdb6c49 100644
--- a/tests/components/zyaura/test.esp32-idf.yaml
+++ b/tests/components/zyaura/test.esp32-idf.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO16
-  data_pin: GPIO17
+  clock_pin: GPIO4
+  data_pin: GPIO5
 
 <<: !include common.yaml
diff --git a/tests/components/zyaura/test.esp8266-ard.yaml b/tests/components/zyaura/test.esp8266-ard.yaml
index 7808481215..7c7f1e1a11 100644
--- a/tests/components/zyaura/test.esp8266-ard.yaml
+++ b/tests/components/zyaura/test.esp8266-ard.yaml
@@ -1,5 +1,5 @@
 substitutions:
-  clock_pin: GPIO5
-  data_pin: GPIO4
+  clock_pin: GPIO0
+  data_pin: GPIO2
 
 <<: !include common.yaml
diff --git a/tests/dashboard/conftest.py b/tests/dashboard/conftest.py
new file mode 100644
index 0000000000..f95adef749
--- /dev/null
+++ b/tests/dashboard/conftest.py
@@ -0,0 +1,43 @@
+"""Common fixtures for dashboard tests."""
+
+from __future__ import annotations
+
+from pathlib import Path
+from unittest.mock import MagicMock, Mock
+
+import pytest
+import pytest_asyncio
+
+from esphome.dashboard.core import ESPHomeDashboard
+from esphome.dashboard.entries import DashboardEntries
+
+
+@pytest.fixture
+def mock_settings(tmp_path: Path) -> MagicMock:
+    """Create mock dashboard settings."""
+    settings = MagicMock()
+    settings.config_dir = str(tmp_path)
+    settings.absolute_config_dir = tmp_path
+    return settings
+
+
+@pytest.fixture
+def mock_dashboard(mock_settings: MagicMock) -> Mock:
+    """Create a mock dashboard."""
+    dashboard = Mock(spec=ESPHomeDashboard)
+    dashboard.settings = mock_settings
+    dashboard.entries = Mock()
+    dashboard.entries.async_all.return_value = []
+    dashboard.stop_event = Mock()
+    dashboard.stop_event.is_set.return_value = True
+    dashboard.ping_request = Mock()
+    dashboard.ignored_devices = set()
+    dashboard.bus = Mock()
+    dashboard.bus.async_fire = Mock()
+    return dashboard
+
+
+@pytest_asyncio.fixture
+async def dashboard_entries(mock_dashboard: Mock) -> DashboardEntries:
+    """Create a DashboardEntries instance for testing."""
+    return DashboardEntries(mock_dashboard)
diff --git a/tests/dashboard/status/__init__.py b/tests/dashboard/status/__init__.py
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/tests/dashboard/status/test_dns.py b/tests/dashboard/status/test_dns.py
new file mode 100644
index 0000000000..9ca48ba2d8
--- /dev/null
+++ b/tests/dashboard/status/test_dns.py
@@ -0,0 +1,121 @@
+"""Unit tests for esphome.dashboard.dns module."""
+
+from __future__ import annotations
+
+import time
+from unittest.mock import patch
+
+import pytest
+
+from esphome.dashboard.dns import DNSCache
+
+
+@pytest.fixture
+def dns_cache_fixture() -> DNSCache:
+    """Create a DNSCache instance."""
+    return DNSCache()
+
+
+def test_get_cached_addresses_not_in_cache(dns_cache_fixture: DNSCache) -> None:
+    """Test get_cached_addresses when hostname is not in cache."""
+    now = time.monotonic()
+    result = dns_cache_fixture.get_cached_addresses("unknown.example.com", now)
+    assert result is None
+
+
+def test_get_cached_addresses_expired(dns_cache_fixture: DNSCache) -> None:
+    """Test get_cached_addresses when cache entry is expired."""
+    now = time.monotonic()
+    # Add entry that's already expired
+    dns_cache_fixture._cache["example.com"] = (now - 1, ["192.168.1.10"])
+
+    result = dns_cache_fixture.get_cached_addresses("example.com", now)
+    assert result is None
+    # Expired entry should still be in cache (not removed by get_cached_addresses)
+    assert "example.com" in dns_cache_fixture._cache
+
+
+def test_get_cached_addresses_valid(dns_cache_fixture: DNSCache) -> None:
+    """Test get_cached_addresses with valid cache entry."""
+    now = time.monotonic()
+    # Add entry that expires in 60 seconds
+    dns_cache_fixture._cache["example.com"] = (
+        now + 60,
+        ["192.168.1.10", "192.168.1.11"],
+    )
+
+    result = dns_cache_fixture.get_cached_addresses("example.com", now)
+    assert result == ["192.168.1.10", "192.168.1.11"]
+    # Entry should still be in cache
+    assert "example.com" in dns_cache_fixture._cache
+
+
+def test_get_cached_addresses_hostname_normalization(
+    dns_cache_fixture: DNSCache,
+) -> None:
+    """Test get_cached_addresses normalizes hostname."""
+    now = time.monotonic()
+    # Add entry with lowercase hostname
+    dns_cache_fixture._cache["example.com"] = (now + 60, ["192.168.1.10"])
+
+    # Test with various forms
+    assert dns_cache_fixture.get_cached_addresses("EXAMPLE.COM", now) == [
+        "192.168.1.10"
+    ]
+    assert dns_cache_fixture.get_cached_addresses("example.com.", now) == [
+        "192.168.1.10"
+    ]
+    assert dns_cache_fixture.get_cached_addresses("EXAMPLE.COM.", now) == [
+        "192.168.1.10"
+    ]
+
+
+def test_get_cached_addresses_ipv6(dns_cache_fixture: DNSCache) -> None:
+    """Test get_cached_addresses with IPv6 addresses."""
+    now = time.monotonic()
+    dns_cache_fixture._cache["example.com"] = (now + 60, ["2001:db8::1", "fe80::1"])
+
+    result = dns_cache_fixture.get_cached_addresses("example.com", now)
+    assert result == ["2001:db8::1", "fe80::1"]
+
+
+def test_get_cached_addresses_empty_list(dns_cache_fixture: DNSCache) -> None:
+    """Test get_cached_addresses with empty address list."""
+    now = time.monotonic()
+    dns_cache_fixture._cache["example.com"] = (now + 60, [])
+
+    result = dns_cache_fixture.get_cached_addresses("example.com", now)
+    assert result == []
+
+
+def test_get_cached_addresses_exception_in_cache(dns_cache_fixture: DNSCache) -> None:
+    """Test get_cached_addresses when cache contains an exception."""
+    now = time.monotonic()
+    # Store an exception (from failed resolution)
+    dns_cache_fixture._cache["example.com"] = (now + 60, OSError("Resolution failed"))
+
+    result = dns_cache_fixture.get_cached_addresses("example.com", now)
+    assert result is None  # Should return None for exceptions
+
+
+def test_async_resolve_not_called(dns_cache_fixture: DNSCache) -> None:
+    """Test that get_cached_addresses never calls async_resolve."""
+    now = time.monotonic()
+
+    with patch.object(dns_cache_fixture, "async_resolve") as mock_resolve:
+        # Test non-cached
+        result = dns_cache_fixture.get_cached_addresses("uncached.com", now)
+        assert result is None
+        mock_resolve.assert_not_called()
+
+        # Test expired
+        dns_cache_fixture._cache["expired.com"] = (now - 1, ["192.168.1.10"])
+        result = dns_cache_fixture.get_cached_addresses("expired.com", now)
+        assert result is None
+        mock_resolve.assert_not_called()
+
+        # Test valid
+        dns_cache_fixture._cache["valid.com"] = (now + 60, ["192.168.1.10"])
+        result = dns_cache_fixture.get_cached_addresses("valid.com", now)
+        assert result == ["192.168.1.10"]
+        mock_resolve.assert_not_called()
diff --git a/tests/dashboard/status/test_mdns.py b/tests/dashboard/status/test_mdns.py
new file mode 100644
index 0000000000..56c6d254cf
--- /dev/null
+++ b/tests/dashboard/status/test_mdns.py
@@ -0,0 +1,240 @@
+"""Unit tests for esphome.dashboard.status.mdns module."""
+
+from __future__ import annotations
+
+from unittest.mock import Mock, patch
+
+import pytest
+import pytest_asyncio
+from zeroconf import AddressResolver, IPVersion
+
+from esphome.dashboard.const import DashboardEvent
+from esphome.dashboard.status.mdns import MDNSStatus
+from esphome.zeroconf import DiscoveredImport
+
+
+@pytest_asyncio.fixture
+async def mdns_status(mock_dashboard: Mock) -> MDNSStatus:
+    """Create an MDNSStatus instance in async context."""
+    # We're in an async context so get_running_loop will work
+    return MDNSStatus(mock_dashboard)
+
+
+@pytest.mark.asyncio
+async def test_get_cached_addresses_no_zeroconf(mdns_status: MDNSStatus) -> None:
+    """Test get_cached_addresses when no zeroconf instance is available."""
+    mdns_status.aiozc = None
+    result = mdns_status.get_cached_addresses("device.local")
+    assert result is None
+
+
+@pytest.mark.asyncio
+async def test_get_cached_addresses_not_in_cache(mdns_status: MDNSStatus) -> None:
+    """Test get_cached_addresses when address is not in cache."""
+    mdns_status.aiozc = Mock()
+    mdns_status.aiozc.zeroconf = Mock()
+
+    with patch("esphome.dashboard.status.mdns.AddressResolver") as mock_resolver:
+        mock_info = Mock(spec=AddressResolver)
+        mock_info.load_from_cache.return_value = False
+        mock_resolver.return_value = mock_info
+
+        result = mdns_status.get_cached_addresses("device.local")
+        assert result is None
+        mock_info.load_from_cache.assert_called_once_with(mdns_status.aiozc.zeroconf)
+
+
+@pytest.mark.asyncio
+async def test_get_cached_addresses_found_in_cache(mdns_status: MDNSStatus) -> None:
+    """Test get_cached_addresses when address is found in cache."""
+    mdns_status.aiozc = Mock()
+    mdns_status.aiozc.zeroconf = Mock()
+
+    with patch("esphome.dashboard.status.mdns.AddressResolver") as mock_resolver:
+        mock_info = Mock(spec=AddressResolver)
+        mock_info.load_from_cache.return_value = True
+        mock_info.parsed_scoped_addresses.return_value = ["192.168.1.10", "fe80::1"]
+        mock_resolver.return_value = mock_info
+
+        result = mdns_status.get_cached_addresses("device.local")
+        assert result == ["192.168.1.10", "fe80::1"]
+        mock_info.load_from_cache.assert_called_once_with(mdns_status.aiozc.zeroconf)
+        mock_info.parsed_scoped_addresses.assert_called_once_with(IPVersion.All)
+
+
+@pytest.mark.asyncio
+async def test_get_cached_addresses_with_trailing_dot(mdns_status: MDNSStatus) -> None:
+    """Test get_cached_addresses with hostname having trailing dot."""
+    mdns_status.aiozc = Mock()
+    mdns_status.aiozc.zeroconf = Mock()
+
+    with patch("esphome.dashboard.status.mdns.AddressResolver") as mock_resolver:
+        mock_info = Mock(spec=AddressResolver)
+        mock_info.load_from_cache.return_value = True
+        mock_info.parsed_scoped_addresses.return_value = ["192.168.1.10"]
+        mock_resolver.return_value = mock_info
+
+        result = mdns_status.get_cached_addresses("device.local.")
+        assert result == ["192.168.1.10"]
+        # Should normalize to device.local. for zeroconf
+        mock_resolver.assert_called_once_with("device.local.")
+
+
+@pytest.mark.asyncio
+async def test_get_cached_addresses_uppercase_hostname(mdns_status: MDNSStatus) -> None:
+    """Test get_cached_addresses with uppercase hostname."""
+    mdns_status.aiozc = Mock()
+    mdns_status.aiozc.zeroconf = Mock()
+
+    with patch("esphome.dashboard.status.mdns.AddressResolver") as mock_resolver:
+        mock_info = Mock(spec=AddressResolver)
+        mock_info.load_from_cache.return_value = True
+        mock_info.parsed_scoped_addresses.return_value = ["192.168.1.10"]
+        mock_resolver.return_value = mock_info
+
+        result = mdns_status.get_cached_addresses("DEVICE.LOCAL")
+        assert result == ["192.168.1.10"]
+        # Should normalize to device.local. for zeroconf
+        mock_resolver.assert_called_once_with("device.local.")
+
+
+@pytest.mark.asyncio
+async def test_get_cached_addresses_simple_hostname(mdns_status: MDNSStatus) -> None:
+    """Test get_cached_addresses with simple hostname (no domain)."""
+    mdns_status.aiozc = Mock()
+    mdns_status.aiozc.zeroconf = Mock()
+
+    with patch("esphome.dashboard.status.mdns.AddressResolver") as mock_resolver:
+        mock_info = Mock(spec=AddressResolver)
+        mock_info.load_from_cache.return_value = True
+        mock_info.parsed_scoped_addresses.return_value = ["192.168.1.10"]
+        mock_resolver.return_value = mock_info
+
+        result = mdns_status.get_cached_addresses("device")
+        assert result == ["192.168.1.10"]
+        # Should append .local. for zeroconf
+        mock_resolver.assert_called_once_with("device.local.")
+
+
+@pytest.mark.asyncio
+async def test_get_cached_addresses_ipv6_only(mdns_status: MDNSStatus) -> None:
+    """Test get_cached_addresses returning only IPv6 addresses."""
+    mdns_status.aiozc = Mock()
+    mdns_status.aiozc.zeroconf = Mock()
+
+    with patch("esphome.dashboard.status.mdns.AddressResolver") as mock_resolver:
+        mock_info = Mock(spec=AddressResolver)
+        mock_info.load_from_cache.return_value = True
+        mock_info.parsed_scoped_addresses.return_value = ["fe80::1", "2001:db8::1"]
+        mock_resolver.return_value = mock_info
+
+        result = mdns_status.get_cached_addresses("device.local")
+        assert result == ["fe80::1", "2001:db8::1"]
+
+
+@pytest.mark.asyncio
+async def test_get_cached_addresses_empty_list(mdns_status: MDNSStatus) -> None:
+    """Test get_cached_addresses returning empty list from cache."""
+    mdns_status.aiozc = Mock()
+    mdns_status.aiozc.zeroconf = Mock()
+
+    with patch("esphome.dashboard.status.mdns.AddressResolver") as mock_resolver:
+        mock_info = Mock(spec=AddressResolver)
+        mock_info.load_from_cache.return_value = True
+        mock_info.parsed_scoped_addresses.return_value = []
+        mock_resolver.return_value = mock_info
+
+        result = mdns_status.get_cached_addresses("device.local")
+        assert result == []
+
+
+@pytest.mark.asyncio
+async def test_async_setup_success(mock_dashboard: Mock) -> None:
+    """Test successful async_setup."""
+    mdns_status = MDNSStatus(mock_dashboard)
+    with patch("esphome.dashboard.status.mdns.AsyncEsphomeZeroconf") as mock_zc:
+        mock_zc.return_value = Mock()
+        result = mdns_status.async_setup()
+        assert result is True
+        assert mdns_status.aiozc is not None
+
+
+@pytest.mark.asyncio
+async def test_async_setup_failure(mock_dashboard: Mock) -> None:
+    """Test async_setup with OSError."""
+    mdns_status = MDNSStatus(mock_dashboard)
+    with patch("esphome.dashboard.status.mdns.AsyncEsphomeZeroconf") as mock_zc:
+        mock_zc.side_effect = OSError("Network error")
+        result = mdns_status.async_setup()
+        assert result is False
+        assert mdns_status.aiozc is None
+
+
+@pytest.mark.asyncio
+async def test_on_import_update_device_added(mdns_status: MDNSStatus) -> None:
+    """Test _on_import_update when a device is added."""
+    # Create a DiscoveredImport object
+    discovered = DiscoveredImport(
+        device_name="test_device",
+        friendly_name="Test Device",
+        package_import_url="https://example.com/package",
+        project_name="test_project",
+        project_version="1.0.0",
+        network="wifi",
+    )
+
+    # Call _on_import_update with a device
+    mdns_status._on_import_update("test_device", discovered)
+
+    # Should fire IMPORTABLE_DEVICE_ADDED event
+    mock_dashboard = mdns_status.dashboard
+    mock_dashboard.bus.async_fire.assert_called_once()
+    call_args = mock_dashboard.bus.async_fire.call_args
+    assert call_args[0][0] == DashboardEvent.IMPORTABLE_DEVICE_ADDED
+    assert "device" in call_args[0][1]
+    device_data = call_args[0][1]["device"]
+    assert device_data["name"] == "test_device"
+    assert device_data["friendly_name"] == "Test Device"
+    assert device_data["project_name"] == "test_project"
+    assert device_data["ignored"] is False
+
+
+@pytest.mark.asyncio
+async def test_on_import_update_device_ignored(mdns_status: MDNSStatus) -> None:
+    """Test _on_import_update when a device is ignored."""
+    # Add device to ignored list
+    mdns_status.dashboard.ignored_devices.add("ignored_device")
+
+    # Create a DiscoveredImport object for ignored device
+    discovered = DiscoveredImport(
+        device_name="ignored_device",
+        friendly_name="Ignored Device",
+        package_import_url="https://example.com/package",
+        project_name="test_project",
+        project_version="1.0.0",
+        network="ethernet",
+    )
+
+    # Call _on_import_update with an ignored device
+    mdns_status._on_import_update("ignored_device", discovered)
+
+    # Should fire IMPORTABLE_DEVICE_ADDED event with ignored=True
+    mock_dashboard = mdns_status.dashboard
+    mock_dashboard.bus.async_fire.assert_called_once()
+    call_args = mock_dashboard.bus.async_fire.call_args
+    assert call_args[0][0] == DashboardEvent.IMPORTABLE_DEVICE_ADDED
+    device_data = call_args[0][1]["device"]
+    assert device_data["name"] == "ignored_device"
+    assert device_data["ignored"] is True
+
+
+@pytest.mark.asyncio
+async def test_on_import_update_device_removed(mdns_status: MDNSStatus) -> None:
+    """Test _on_import_update when a device is removed."""
+    # Call _on_import_update with None (device removed)
+    mdns_status._on_import_update("removed_device", None)
+
+    # Should fire IMPORTABLE_DEVICE_REMOVED event
+    mdns_status.dashboard.bus.async_fire.assert_called_once_with(
+        DashboardEvent.IMPORTABLE_DEVICE_REMOVED, {"name": "removed_device"}
+    )
diff --git a/tests/dashboard/test_entries.py b/tests/dashboard/test_entries.py
new file mode 100644
index 0000000000..9a3a776b28
--- /dev/null
+++ b/tests/dashboard/test_entries.py
@@ -0,0 +1,288 @@
+"""Tests for dashboard entries Path-related functionality."""
+
+from __future__ import annotations
+
+import os
+from pathlib import Path
+import tempfile
+from unittest.mock import Mock
+
+import pytest
+
+from esphome.core import CORE
+from esphome.dashboard.const import DashboardEvent
+from esphome.dashboard.entries import DashboardEntries, DashboardEntry
+
+
+def create_cache_key() -> tuple[int, int, float, int]:
+    """Helper to create a valid DashboardCacheKeyType."""
+    return (0, 0, 0.0, 0)
+
+
+@pytest.fixture(autouse=True)
+def setup_core():
+    """Set up CORE for testing."""
+    with tempfile.TemporaryDirectory() as tmpdir:
+        CORE.config_path = Path(tmpdir) / "test.yaml"
+        yield
+        CORE.reset()
+
+
+def test_dashboard_entry_path_initialization() -> None:
+    """Test DashboardEntry initializes with path correctly."""
+    test_path = Path("/test/config/device.yaml")
+    cache_key = create_cache_key()
+
+    entry = DashboardEntry(test_path, cache_key)
+
+    assert entry.path == test_path
+    assert entry.cache_key == cache_key
+
+
+def test_dashboard_entry_path_with_absolute_path() -> None:
+    """Test DashboardEntry handles absolute paths."""
+    # Use a truly absolute path for the platform
+    test_path = Path.cwd() / "absolute" / "path" / "to" / "config.yaml"
+    cache_key = create_cache_key()
+
+    entry = DashboardEntry(test_path, cache_key)
+
+    assert entry.path == test_path
+    assert entry.path.is_absolute()
+
+
+def test_dashboard_entry_path_with_relative_path() -> None:
+    """Test DashboardEntry handles relative paths."""
+    test_path = Path("configs/device.yaml")
+    cache_key = create_cache_key()
+
+    entry = DashboardEntry(test_path, cache_key)
+
+    assert entry.path == test_path
+    assert not entry.path.is_absolute()
+
+
+@pytest.mark.asyncio
+async def test_dashboard_entries_get_by_path(
+    dashboard_entries: DashboardEntries, tmp_path: Path
+) -> None:
+    """Test getting entry by path."""
+    # Create a test file
+    test_file = tmp_path / "device.yaml"
+    test_file.write_text("test config")
+
+    # Update entries to load the file
+    await dashboard_entries.async_update_entries()
+
+    # Verify the entry was loaded
+    all_entries = dashboard_entries.async_all()
+    assert len(all_entries) == 1
+    entry = all_entries[0]
+    assert entry.path == test_file
+
+    # Also verify get() works with Path
+    result = dashboard_entries.get(test_file)
+    assert result == entry
+
+
+@pytest.mark.asyncio
+async def test_dashboard_entries_get_nonexistent_path(
+    dashboard_entries: DashboardEntries,
+) -> None:
+    """Test getting non-existent entry returns None."""
+    result = dashboard_entries.get("/nonexistent/path.yaml")
+    assert result is None
+
+
+@pytest.mark.asyncio
+async def test_dashboard_entries_path_normalization(
+    dashboard_entries: DashboardEntries, tmp_path: Path
+) -> None:
+    """Test that paths are handled consistently."""
+    # Create a test file
+    test_file = tmp_path / "device.yaml"
+    test_file.write_text("test config")
+
+    # Update entries to load the file
+    await dashboard_entries.async_update_entries()
+
+    # Get the entry by path
+    result = dashboard_entries.get(test_file)
+    assert result is not None
+
+
+@pytest.mark.asyncio
+async def test_dashboard_entries_path_with_spaces(
+    dashboard_entries: DashboardEntries, tmp_path: Path
+) -> None:
+    """Test handling paths with spaces."""
+    # Create a test file with spaces in name
+    test_file = tmp_path / "my device.yaml"
+    test_file.write_text("test config")
+
+    # Update entries to load the file
+    await dashboard_entries.async_update_entries()
+
+    # Get the entry by path
+    result = dashboard_entries.get(test_file)
+    assert result is not None
+    assert result.path == test_file
+
+
+@pytest.mark.asyncio
+async def test_dashboard_entries_path_with_special_chars(
+    dashboard_entries: DashboardEntries, tmp_path: Path
+) -> None:
+    """Test handling paths with special characters."""
+    # Create a test file with special characters
+    test_file = tmp_path / "device-01_test.yaml"
+    test_file.write_text("test config")
+
+    # Update entries to load the file
+    await dashboard_entries.async_update_entries()
+
+    # Get the entry by path
+    result = dashboard_entries.get(test_file)
+    assert result is not None
+
+
+def test_dashboard_entries_windows_path() -> None:
+    """Test handling Windows-style paths."""
+    test_path = Path(r"C:\Users\test\esphome\device.yaml")
+    cache_key = create_cache_key()
+
+    entry = DashboardEntry(test_path, cache_key)
+
+    assert entry.path == test_path
+
+
+@pytest.mark.asyncio
+async def test_dashboard_entries_path_to_cache_key_mapping(
+    dashboard_entries: DashboardEntries, tmp_path: Path
+) -> None:
+    """Test internal entries storage with paths and cache keys."""
+    # Create test files
+    file1 = tmp_path / "device1.yaml"
+    file2 = tmp_path / "device2.yaml"
+    file1.write_text("test config 1")
+    file2.write_text("test config 2")
+
+    # Update entries to load the files
+    await dashboard_entries.async_update_entries()
+
+    # Get entries and verify they have different cache keys
+    entry1 = dashboard_entries.get(file1)
+    entry2 = dashboard_entries.get(file2)
+
+    assert entry1 is not None
+    assert entry2 is not None
+    assert entry1.cache_key != entry2.cache_key
+
+
+def test_dashboard_entry_path_property() -> None:
+    """Test that path property returns expected value."""
+    test_path = Path("/test/config/device.yaml")
+    entry = DashboardEntry(test_path, create_cache_key())
+
+    assert entry.path == test_path
+    assert isinstance(entry.path, Path)
+
+
+@pytest.mark.asyncio
+async def test_dashboard_entries_all_returns_entries_with_paths(
+    dashboard_entries: DashboardEntries, tmp_path: Path
+) -> None:
+    """Test that all() returns entries with their paths intact."""
+    # Create test files
+    files = [
+        tmp_path / "device1.yaml",
+        tmp_path / "device2.yaml",
+        tmp_path / "device3.yaml",
+    ]
+
+    for file in files:
+        file.write_text("test config")
+
+    # Update entries to load the files
+    await dashboard_entries.async_update_entries()
+
+    all_entries = dashboard_entries.async_all()
+
+    assert len(all_entries) == len(files)
+    retrieved_paths = [entry.path for entry in all_entries]
+    assert set(retrieved_paths) == set(files)
+
+
+@pytest.mark.asyncio
+async def test_async_update_entries_removed_path(
+    dashboard_entries: DashboardEntries, mock_dashboard: Mock, tmp_path: Path
+) -> None:
+    """Test that removed files trigger ENTRY_REMOVED event."""
+
+    # Create a test file
+    test_file = tmp_path / "device.yaml"
+    test_file.write_text("test config")
+
+    # First update to add the entry
+    await dashboard_entries.async_update_entries()
+
+    # Verify entry was added
+    all_entries = dashboard_entries.async_all()
+    assert len(all_entries) == 1
+    entry = all_entries[0]
+
+    # Delete the file
+    test_file.unlink()
+
+    # Second update to detect removal
+    await dashboard_entries.async_update_entries()
+
+    # Verify entry was removed
+    all_entries = dashboard_entries.async_all()
+    assert len(all_entries) == 0
+
+    # Verify ENTRY_REMOVED event was fired
+    mock_dashboard.bus.async_fire.assert_any_call(
+        DashboardEvent.ENTRY_REMOVED, {"entry": entry}
+    )
+
+
+@pytest.mark.asyncio
+async def test_async_update_entries_updated_path(
+    dashboard_entries: DashboardEntries, mock_dashboard: Mock, tmp_path: Path
+) -> None:
+    """Test that modified files trigger ENTRY_UPDATED event."""
+
+    # Create a test file
+    test_file = tmp_path / "device.yaml"
+    test_file.write_text("test config")
+
+    # First update to add the entry
+    await dashboard_entries.async_update_entries()
+
+    # Verify entry was added
+    all_entries = dashboard_entries.async_all()
+    assert len(all_entries) == 1
+    entry = all_entries[0]
+    original_cache_key = entry.cache_key
+
+    # Modify the file to change its mtime
+    test_file.write_text("updated config")
+    # Explicitly change the mtime to ensure it's different
+    stat = test_file.stat()
+    os.utime(test_file, (stat.st_atime, stat.st_mtime + 1))
+
+    # Second update to detect modification
+    await dashboard_entries.async_update_entries()
+
+    # Verify entry is still there with updated cache key
+    all_entries = dashboard_entries.async_all()
+    assert len(all_entries) == 1
+    updated_entry = all_entries[0]
+    assert updated_entry == entry  # Same entry object
+    assert updated_entry.cache_key != original_cache_key  # But cache key updated
+
+    # Verify ENTRY_UPDATED event was fired
+    mock_dashboard.bus.async_fire.assert_any_call(
+        DashboardEvent.ENTRY_UPDATED, {"entry": entry}
+    )
diff --git a/tests/dashboard/test_settings.py b/tests/dashboard/test_settings.py
new file mode 100644
index 0000000000..91a8ec70c3
--- /dev/null
+++ b/tests/dashboard/test_settings.py
@@ -0,0 +1,223 @@
+"""Tests for dashboard settings Path-related functionality."""
+
+from __future__ import annotations
+
+from argparse import Namespace
+from pathlib import Path
+import tempfile
+
+import pytest
+
+from esphome.core import CORE
+from esphome.dashboard.settings import DashboardSettings
+
+
+@pytest.fixture
+def dashboard_settings(tmp_path: Path) -> DashboardSettings:
+    """Create DashboardSettings instance with temp directory."""
+    settings = DashboardSettings()
+    # Resolve symlinks to ensure paths match
+    resolved_dir = tmp_path.resolve()
+    settings.config_dir = resolved_dir
+    settings.absolute_config_dir = resolved_dir
+    return settings
+
+
+def test_rel_path_simple(dashboard_settings: DashboardSettings) -> None:
+    """Test rel_path with simple relative path."""
+    result = dashboard_settings.rel_path("config.yaml")
+
+    expected = dashboard_settings.config_dir / "config.yaml"
+    assert result == expected
+
+
+def test_rel_path_multiple_components(dashboard_settings: DashboardSettings) -> None:
+    """Test rel_path with multiple path components."""
+    result = dashboard_settings.rel_path("subfolder", "device", "config.yaml")
+
+    expected = dashboard_settings.config_dir / "subfolder" / "device" / "config.yaml"
+    assert result == expected
+
+
+def test_rel_path_with_dots(dashboard_settings: DashboardSettings) -> None:
+    """Test rel_path prevents directory traversal."""
+    # This should raise ValueError as it tries to go outside config_dir
+    with pytest.raises(ValueError):
+        dashboard_settings.rel_path("..", "outside.yaml")
+
+
+def test_rel_path_absolute_path_within_config(
+    dashboard_settings: DashboardSettings,
+) -> None:
+    """Test rel_path with absolute path that's within config dir."""
+    internal_path = dashboard_settings.absolute_config_dir / "internal.yaml"
+
+    internal_path.touch()
+    result = dashboard_settings.rel_path("internal.yaml")
+    expected = dashboard_settings.config_dir / "internal.yaml"
+    assert result == expected
+
+
+def test_rel_path_absolute_path_outside_config(
+    dashboard_settings: DashboardSettings,
+) -> None:
+    """Test rel_path with absolute path outside config dir raises error."""
+    outside_path = "/tmp/outside/config.yaml"
+
+    with pytest.raises(ValueError):
+        dashboard_settings.rel_path(outside_path)
+
+
+def test_rel_path_empty_args(dashboard_settings: DashboardSettings) -> None:
+    """Test rel_path with no arguments returns config_dir."""
+    result = dashboard_settings.rel_path()
+    assert result == dashboard_settings.config_dir
+
+
+def test_rel_path_with_pathlib_path(dashboard_settings: DashboardSettings) -> None:
+    """Test rel_path works with Path objects as arguments."""
+    path_obj = Path("subfolder") / "config.yaml"
+    result = dashboard_settings.rel_path(path_obj)
+
+    expected = dashboard_settings.config_dir / "subfolder" / "config.yaml"
+    assert result == expected
+
+
+def test_rel_path_normalizes_slashes(dashboard_settings: DashboardSettings) -> None:
+    """Test rel_path normalizes path separators."""
+    # os.path.join normalizes slashes on Windows but preserves them on Unix
+    # Test that providing components separately gives same result
+    result1 = dashboard_settings.rel_path("folder", "subfolder", "file.yaml")
+    result2 = dashboard_settings.rel_path("folder", "subfolder", "file.yaml")
+    assert result1 == result2
+
+    # Also test that the result is as expected
+    expected = dashboard_settings.config_dir / "folder" / "subfolder" / "file.yaml"
+    assert result1 == expected
+
+
+def test_rel_path_handles_spaces(dashboard_settings: DashboardSettings) -> None:
+    """Test rel_path handles paths with spaces."""
+    result = dashboard_settings.rel_path("my folder", "my config.yaml")
+
+    expected = dashboard_settings.config_dir / "my folder" / "my config.yaml"
+    assert result == expected
+
+
+def test_rel_path_handles_special_chars(dashboard_settings: DashboardSettings) -> None:
+    """Test rel_path handles paths with special characters."""
+    result = dashboard_settings.rel_path("device-01_test", "config.yaml")
+
+    expected = dashboard_settings.config_dir / "device-01_test" / "config.yaml"
+    assert result == expected
+
+
+def test_config_dir_as_path_property(dashboard_settings: DashboardSettings) -> None:
+    """Test that config_dir can be accessed and used with Path operations."""
+    config_path = dashboard_settings.config_dir
+
+    assert config_path.exists()
+    assert config_path.is_dir()
+    assert config_path.is_absolute()
+
+
+def test_absolute_config_dir_property(dashboard_settings: DashboardSettings) -> None:
+    """Test absolute_config_dir is a Path object."""
+    assert isinstance(dashboard_settings.absolute_config_dir, Path)
+    assert dashboard_settings.absolute_config_dir.exists()
+    assert dashboard_settings.absolute_config_dir.is_dir()
+    assert dashboard_settings.absolute_config_dir.is_absolute()
+
+
+def test_rel_path_symlink_inside_config(dashboard_settings: DashboardSettings) -> None:
+    """Test rel_path with symlink that points inside config dir."""
+    target = dashboard_settings.absolute_config_dir / "target.yaml"
+    target.touch()
+    symlink = dashboard_settings.absolute_config_dir / "link.yaml"
+    symlink.symlink_to(target)
+    result = dashboard_settings.rel_path("link.yaml")
+    expected = dashboard_settings.config_dir / "link.yaml"
+    assert result == expected
+
+
+def test_rel_path_symlink_outside_config(dashboard_settings: DashboardSettings) -> None:
+    """Test rel_path with symlink that points outside config dir."""
+    with tempfile.NamedTemporaryFile(suffix=".yaml") as tmp:
+        symlink = dashboard_settings.absolute_config_dir / "external_link.yaml"
+        symlink.symlink_to(tmp.name)
+        with pytest.raises(ValueError):
+            dashboard_settings.rel_path("external_link.yaml")
+
+
+def test_rel_path_with_none_arg(dashboard_settings: DashboardSettings) -> None:
+    """Test rel_path handles None arguments gracefully."""
+    result = dashboard_settings.rel_path("None")
+    expected = dashboard_settings.config_dir / "None"
+    assert result == expected
+
+
+def test_rel_path_with_numeric_args(dashboard_settings: DashboardSettings) -> None:
+    """Test rel_path handles numeric arguments."""
+    result = dashboard_settings.rel_path("123", "456.789")
+    expected = dashboard_settings.config_dir / "123" / "456.789"
+    assert result == expected
+
+
+def test_config_path_parent_resolves_to_config_dir(tmp_path: Path) -> None:
+    """Test that CORE.config_path.parent resolves to config_dir after parse_args.
+
+    This is a regression test for issue #11280 where binary download failed
+    when using packages with secrets after the Path migration in 2025.10.0.
+
+    The issue was that after switching from os.path to Path:
+    - Before: os.path.dirname("/config/.") → "/config"
+    - After: Path("/config/.").parent → Path("/") (normalized first!)
+
+    The fix uses a sentinel file so .parent returns the correct directory:
+    - Fixed: Path("/config/___DASHBOARD_SENTINEL___.yaml").parent → Path("/config")
+    """
+    # Create test directory structure with secrets and packages
+    config_dir = tmp_path / "config"
+    config_dir.mkdir()
+
+    # Create secrets.yaml with obviously fake test values
+    secrets_file = config_dir / "secrets.yaml"
+    secrets_file.write_text(
+        "wifi_ssid: TEST-DUMMY-SSID\n"
+        "wifi_password: not-a-real-password-just-for-testing\n"
+    )
+
+    # Create package file that uses secrets
+    package_file = config_dir / "common.yaml"
+    package_file.write_text(
+        "wifi:\n  ssid: !secret wifi_ssid\n  password: !secret wifi_password\n"
+    )
+
+    # Create main device config that includes the package
+    device_config = config_dir / "test-device.yaml"
+    device_config.write_text(
+        "esphome:\n  name: test-device\n\npackages:\n  common: !include common.yaml\n"
+    )
+
+    # Set up dashboard settings with our test config directory
+    settings = DashboardSettings()
+    args = Namespace(
+        configuration=str(config_dir),
+        password=None,
+        username=None,
+        ha_addon=False,
+        verbose=False,
+    )
+    settings.parse_args(args)
+
+    # Verify that CORE.config_path.parent correctly points to the config directory
+    # This is critical for secret resolution in yaml_util.py which does:
+    #   main_config_dir = CORE.config_path.parent
+    #   main_secret_yml = main_config_dir / "secrets.yaml"
+    assert CORE.config_path.parent == config_dir.resolve()
+    assert (CORE.config_path.parent / "secrets.yaml").exists()
+    assert (CORE.config_path.parent / "common.yaml").exists()
+
+    # Verify that CORE.config_path itself uses the sentinel file
+    assert CORE.config_path.name == "___DASHBOARD_SENTINEL___.yaml"
+    assert not CORE.config_path.exists()  # Sentinel file doesn't actually exist
diff --git a/tests/dashboard/test_web_server.py b/tests/dashboard/test_web_server.py
index cd02200d0b..385841b1c8 100644
--- a/tests/dashboard/test_web_server.py
+++ b/tests/dashboard/test_web_server.py
@@ -1,23 +1,60 @@
 from __future__ import annotations
 
+from argparse import Namespace
 import asyncio
+from collections.abc import Generator
+from contextlib import asynccontextmanager
+import gzip
 import json
 import os
-from unittest.mock import Mock
+from pathlib import Path
+from unittest.mock import AsyncMock, MagicMock, Mock, patch
 
 import pytest
 import pytest_asyncio
-from tornado.httpclient import AsyncHTTPClient, HTTPResponse
+from tornado.httpclient import AsyncHTTPClient, HTTPClientError, HTTPResponse
 from tornado.httpserver import HTTPServer
 from tornado.ioloop import IOLoop
 from tornado.testing import bind_unused_port
+from tornado.websocket import WebSocketClientConnection, websocket_connect
 
+from esphome import yaml_util
+from esphome.core import CORE
 from esphome.dashboard import web_server
+from esphome.dashboard.const import DashboardEvent
 from esphome.dashboard.core import DASHBOARD
+from esphome.dashboard.entries import (
+    DashboardEntry,
+    EntryStateSource,
+    bool_to_entry_state,
+)
+from esphome.dashboard.models import build_importable_device_dict
+from esphome.dashboard.web_server import DashboardSubscriber
+from esphome.zeroconf import DiscoveredImport
 
 from .common import get_fixture_path
 
 
+def get_build_path(base_path: Path, device_name: str) -> Path:
+    """Get the build directory path for a device.
+
+    This is a test helper that constructs the standard ESPHome build directory
+    structure. Note: This helper does NOT perform path traversal sanitization
+    because it's only used in tests where we control the inputs. The actual
+    web_server.py code handles sanitization in DownloadBinaryRequestHandler.get()
+    via file_name.replace("..", "").lstrip("/").
+
+    Args:
+        base_path: The base temporary path (typically tmp_path from pytest)
+        device_name: The name of the device (should not contain path separators
+                     in production use, but tests may use it for specific scenarios)
+
+    Returns:
+        Path to the build directory (.esphome/build/device_name)
+    """
+    return base_path / ".esphome" / "build" / device_name
+
+
 class DashboardTestHelper:
     def __init__(self, io_loop: IOLoop, client: AsyncHTTPClient, port: int) -> None:
         self.io_loop = io_loop
@@ -31,8 +68,67 @@ class DashboardTestHelper:
         else:
             url = f"http://127.0.0.1:{self.port}{path}"
         future = self.client.fetch(url, raise_error=True, **kwargs)
-        result = await future
-        return result
+        return await future
+
+
+@pytest.fixture
+def mock_async_run_system_command() -> Generator[MagicMock]:
+    """Fixture to mock async_run_system_command."""
+    with patch("esphome.dashboard.web_server.async_run_system_command") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_trash_storage_path(tmp_path: Path) -> Generator[MagicMock]:
+    """Fixture to mock trash_storage_path."""
+    trash_dir = tmp_path / "trash"
+    with patch(
+        "esphome.dashboard.web_server.trash_storage_path", return_value=trash_dir
+    ) as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_archive_storage_path(tmp_path: Path) -> Generator[MagicMock]:
+    """Fixture to mock archive_storage_path."""
+    archive_dir = tmp_path / "archive"
+    with patch(
+        "esphome.dashboard.web_server.archive_storage_path",
+        return_value=archive_dir,
+    ) as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_dashboard_settings() -> Generator[MagicMock]:
+    """Fixture to mock dashboard settings."""
+    with patch("esphome.dashboard.web_server.settings") as mock_settings:
+        # Set default auth settings to avoid authentication issues
+        mock_settings.using_auth = False
+        mock_settings.on_ha_addon = False
+        yield mock_settings
+
+
+@pytest.fixture
+def mock_ext_storage_path(tmp_path: Path) -> Generator[MagicMock]:
+    """Fixture to mock ext_storage_path."""
+    with patch("esphome.dashboard.web_server.ext_storage_path") as mock:
+        mock.return_value = str(tmp_path / "storage.json")
+        yield mock
+
+
+@pytest.fixture
+def mock_storage_json() -> Generator[MagicMock]:
+    """Fixture to mock StorageJSON."""
+    with patch("esphome.dashboard.web_server.StorageJSON") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_idedata() -> Generator[MagicMock]:
+    """Fixture to mock platformio_api.IDEData."""
+    with patch("esphome.dashboard.web_server.platformio_api.IDEData") as mock:
+        yield mock
 
 
 @pytest_asyncio.fixture()
@@ -64,6 +160,33 @@ async def dashboard() -> DashboardTestHelper:
     io_loop.close()
 
 
+@asynccontextmanager
+async def websocket_connection(dashboard: DashboardTestHelper):
+    """Async context manager for WebSocket connections."""
+    url = f"ws://127.0.0.1:{dashboard.port}/events"
+    ws = await websocket_connect(url)
+    try:
+        yield ws
+    finally:
+        if ws:
+            ws.close()
+
+
+@pytest_asyncio.fixture
+async def websocket_client(dashboard: DashboardTestHelper) -> WebSocketClientConnection:
+    """Create a WebSocket connection for testing."""
+    url = f"ws://127.0.0.1:{dashboard.port}/events"
+    ws = await websocket_connect(url)
+
+    # Read and discard initial state message
+    await ws.read_message()
+
+    yield ws
+
+    if ws:
+        ws.close()
+
+
 @pytest.mark.asyncio
 async def test_main_page(dashboard: DashboardTestHelper) -> None:
     response = await dashboard.fetch("/")
@@ -81,3 +204,1366 @@ async def test_devices_page(dashboard: DashboardTestHelper) -> None:
     first_device = configured_devices[0]
     assert first_device["name"] == "pico"
     assert first_device["configuration"] == "pico.yaml"
+
+
+@pytest.mark.asyncio
+async def test_wizard_handler_invalid_input(dashboard: DashboardTestHelper) -> None:
+    """Test the WizardRequestHandler.post method with invalid inputs."""
+    # Test with missing name (should fail with 422)
+    body_no_name = json.dumps(
+        {
+            "name": "",  # Empty name
+            "platform": "ESP32",
+            "board": "esp32dev",
+        }
+    )
+    with pytest.raises(HTTPClientError) as exc_info:
+        await dashboard.fetch(
+            "/wizard",
+            method="POST",
+            body=body_no_name,
+            headers={"Content-Type": "application/json"},
+        )
+    assert exc_info.value.code == 422
+
+    # Test with invalid wizard type (should fail with 422)
+    body_invalid_type = json.dumps(
+        {
+            "name": "test_device",
+            "type": "invalid_type",
+            "platform": "ESP32",
+            "board": "esp32dev",
+        }
+    )
+    with pytest.raises(HTTPClientError) as exc_info:
+        await dashboard.fetch(
+            "/wizard",
+            method="POST",
+            body=body_invalid_type,
+            headers={"Content-Type": "application/json"},
+        )
+    assert exc_info.value.code == 422
+
+
+@pytest.mark.asyncio
+async def test_wizard_handler_conflict(dashboard: DashboardTestHelper) -> None:
+    """Test the WizardRequestHandler.post when config already exists."""
+    # Try to create a wizard for existing pico.yaml (should conflict)
+    body = json.dumps(
+        {
+            "name": "pico",  # This already exists in fixtures
+            "platform": "ESP32",
+            "board": "esp32dev",
+        }
+    )
+    with pytest.raises(HTTPClientError) as exc_info:
+        await dashboard.fetch(
+            "/wizard",
+            method="POST",
+            body=body,
+            headers={"Content-Type": "application/json"},
+        )
+    assert exc_info.value.code == 409
+
+
+@pytest.mark.asyncio
+async def test_download_binary_handler_not_found(
+    dashboard: DashboardTestHelper,
+) -> None:
+    """Test the DownloadBinaryRequestHandler.get with non-existent config."""
+    with pytest.raises(HTTPClientError) as exc_info:
+        await dashboard.fetch(
+            "/download.bin?configuration=nonexistent.yaml",
+            method="GET",
+        )
+    assert exc_info.value.code == 404
+
+
+@pytest.mark.asyncio
+@pytest.mark.usefixtures("mock_ext_storage_path")
+async def test_download_binary_handler_no_file_param(
+    dashboard: DashboardTestHelper,
+    tmp_path: Path,
+    mock_storage_json: MagicMock,
+) -> None:
+    """Test the DownloadBinaryRequestHandler.get without file parameter."""
+    # Mock storage to exist, but still should fail without file param
+    mock_storage = Mock()
+    mock_storage.name = "test_device"
+    mock_storage.firmware_bin_path = str(tmp_path / "firmware.bin")
+    mock_storage_json.load.return_value = mock_storage
+
+    with pytest.raises(HTTPClientError) as exc_info:
+        await dashboard.fetch(
+            "/download.bin?configuration=pico.yaml",
+            method="GET",
+        )
+    assert exc_info.value.code == 400
+
+
+@pytest.mark.asyncio
+@pytest.mark.usefixtures("mock_ext_storage_path")
+async def test_download_binary_handler_with_file(
+    dashboard: DashboardTestHelper,
+    tmp_path: Path,
+    mock_storage_json: MagicMock,
+) -> None:
+    """Test the DownloadBinaryRequestHandler.get with existing binary file."""
+    # Create a fake binary file
+    build_dir = tmp_path / ".esphome" / "build" / "test"
+    build_dir.mkdir(parents=True)
+    firmware_file = build_dir / "firmware.bin"
+    firmware_file.write_bytes(b"fake firmware content")
+
+    # Mock storage JSON
+    mock_storage = Mock()
+    mock_storage.name = "test_device"
+    mock_storage.firmware_bin_path = firmware_file
+    mock_storage_json.load.return_value = mock_storage
+
+    response = await dashboard.fetch(
+        "/download.bin?configuration=test.yaml&file=firmware.bin",
+        method="GET",
+    )
+    assert response.code == 200
+    assert response.body == b"fake firmware content"
+    assert response.headers["Content-Type"] == "application/octet-stream"
+    assert "attachment" in response.headers["Content-Disposition"]
+    assert "test_device-firmware.bin" in response.headers["Content-Disposition"]
+
+
+@pytest.mark.asyncio
+@pytest.mark.usefixtures("mock_ext_storage_path")
+async def test_download_binary_handler_compressed(
+    dashboard: DashboardTestHelper,
+    tmp_path: Path,
+    mock_storage_json: MagicMock,
+) -> None:
+    """Test the DownloadBinaryRequestHandler.get with compression."""
+    # Create a fake binary file
+    build_dir = tmp_path / ".esphome" / "build" / "test"
+    build_dir.mkdir(parents=True)
+    firmware_file = build_dir / "firmware.bin"
+    original_content = b"fake firmware content for compression test"
+    firmware_file.write_bytes(original_content)
+
+    # Mock storage JSON
+    mock_storage = Mock()
+    mock_storage.name = "test_device"
+    mock_storage.firmware_bin_path = firmware_file
+    mock_storage_json.load.return_value = mock_storage
+
+    response = await dashboard.fetch(
+        "/download.bin?configuration=test.yaml&file=firmware.bin&compressed=1",
+        method="GET",
+    )
+    assert response.code == 200
+    # Decompress and verify content
+    decompressed = gzip.decompress(response.body)
+    assert decompressed == original_content
+    assert response.headers["Content-Type"] == "application/octet-stream"
+    assert "firmware.bin.gz" in response.headers["Content-Disposition"]
+
+
+@pytest.mark.asyncio
+@pytest.mark.usefixtures("mock_ext_storage_path")
+async def test_download_binary_handler_custom_download_name(
+    dashboard: DashboardTestHelper,
+    tmp_path: Path,
+    mock_storage_json: MagicMock,
+) -> None:
+    """Test the DownloadBinaryRequestHandler.get with custom download name."""
+    # Create a fake binary file
+    build_dir = tmp_path / ".esphome" / "build" / "test"
+    build_dir.mkdir(parents=True)
+    firmware_file = build_dir / "firmware.bin"
+    firmware_file.write_bytes(b"content")
+
+    # Mock storage JSON
+    mock_storage = Mock()
+    mock_storage.name = "test_device"
+    mock_storage.firmware_bin_path = firmware_file
+    mock_storage_json.load.return_value = mock_storage
+
+    response = await dashboard.fetch(
+        "/download.bin?configuration=test.yaml&file=firmware.bin&download=custom_name.bin",
+        method="GET",
+    )
+    assert response.code == 200
+    assert "custom_name.bin" in response.headers["Content-Disposition"]
+
+
+@pytest.mark.asyncio
+@pytest.mark.usefixtures("mock_ext_storage_path")
+async def test_download_binary_handler_idedata_fallback(
+    dashboard: DashboardTestHelper,
+    tmp_path: Path,
+    mock_async_run_system_command: MagicMock,
+    mock_storage_json: MagicMock,
+    mock_idedata: MagicMock,
+) -> None:
+    """Test the DownloadBinaryRequestHandler.get falling back to idedata for extra images."""
+    # Create build directory but no bootloader file initially
+    build_dir = tmp_path / ".esphome" / "build" / "test"
+    build_dir.mkdir(parents=True)
+    firmware_file = build_dir / "firmware.bin"
+    firmware_file.write_bytes(b"firmware")
+
+    # Create bootloader file that idedata will find
+    bootloader_file = tmp_path / "bootloader.bin"
+    bootloader_file.write_bytes(b"bootloader content")
+
+    # Mock storage JSON
+    mock_storage = Mock()
+    mock_storage.name = "test_device"
+    mock_storage.firmware_bin_path = firmware_file
+    mock_storage_json.load.return_value = mock_storage
+
+    # Mock idedata response
+    mock_image = Mock()
+    mock_image.path = str(bootloader_file)
+    mock_idedata_instance = Mock()
+    mock_idedata_instance.extra_flash_images = [mock_image]
+    mock_idedata.return_value = mock_idedata_instance
+
+    # Mock async_run_system_command to return idedata JSON
+    mock_async_run_system_command.return_value = (0, '{"extra_flash_images": []}', "")
+
+    response = await dashboard.fetch(
+        "/download.bin?configuration=test.yaml&file=bootloader.bin",
+        method="GET",
+    )
+    assert response.code == 200
+    assert response.body == b"bootloader content"
+
+
+@pytest.mark.asyncio
+@pytest.mark.usefixtures("mock_ext_storage_path")
+async def test_download_binary_handler_subdirectory_file(
+    dashboard: DashboardTestHelper,
+    tmp_path: Path,
+    mock_storage_json: MagicMock,
+) -> None:
+    """Test the DownloadBinaryRequestHandler.get with file in subdirectory (nRF52 case).
+
+    This is a regression test for issue #11343 where the Path migration broke
+    downloads for nRF52 firmware files in subdirectories like 'zephyr/zephyr.uf2'.
+
+    The issue was that with_name() doesn't accept path separators:
+    - Before: path = storage_json.firmware_bin_path.with_name(file_name)
+      ValueError: Invalid name 'zephyr/zephyr.uf2'
+    - After: path = storage_json.firmware_bin_path.parent.joinpath(file_name)
+      Works correctly with subdirectory paths
+    """
+    # Create a fake nRF52 build structure with firmware in subdirectory
+    build_dir = get_build_path(tmp_path, "nrf52-device")
+    zephyr_dir = build_dir / "zephyr"
+    zephyr_dir.mkdir(parents=True)
+
+    # Create the main firmware binary (would be in build root)
+    firmware_file = build_dir / "firmware.bin"
+    firmware_file.write_bytes(b"main firmware")
+
+    # Create the UF2 file in zephyr subdirectory (nRF52 specific)
+    uf2_file = zephyr_dir / "zephyr.uf2"
+    uf2_file.write_bytes(b"nRF52 UF2 firmware content")
+
+    # Mock storage JSON
+    mock_storage = Mock()
+    mock_storage.name = "nrf52-device"
+    mock_storage.firmware_bin_path = firmware_file
+    mock_storage_json.load.return_value = mock_storage
+
+    # Request the UF2 file with subdirectory path
+    response = await dashboard.fetch(
+        "/download.bin?configuration=nrf52-device.yaml&file=zephyr/zephyr.uf2",
+        method="GET",
+    )
+    assert response.code == 200
+    assert response.body == b"nRF52 UF2 firmware content"
+    assert response.headers["Content-Type"] == "application/octet-stream"
+    assert "attachment" in response.headers["Content-Disposition"]
+    # Download name should be device-name + full file path
+    assert "nrf52-device-zephyr/zephyr.uf2" in response.headers["Content-Disposition"]
+
+
+@pytest.mark.asyncio
+@pytest.mark.usefixtures("mock_ext_storage_path")
+async def test_download_binary_handler_subdirectory_file_url_encoded(
+    dashboard: DashboardTestHelper,
+    tmp_path: Path,
+    mock_storage_json: MagicMock,
+) -> None:
+    """Test the DownloadBinaryRequestHandler.get with URL-encoded subdirectory path.
+
+    Verifies that URL-encoded paths (e.g., zephyr%2Fzephyr.uf2) are correctly
+    decoded and handled, and that custom download names work with subdirectories.
+    """
+    # Create a fake build structure with firmware in subdirectory
+    build_dir = get_build_path(tmp_path, "test")
+    zephyr_dir = build_dir / "zephyr"
+    zephyr_dir.mkdir(parents=True)
+
+    firmware_file = build_dir / "firmware.bin"
+    firmware_file.write_bytes(b"content")
+
+    uf2_file = zephyr_dir / "zephyr.uf2"
+    uf2_file.write_bytes(b"content")
+
+    # Mock storage JSON
+    mock_storage = Mock()
+    mock_storage.name = "test_device"
+    mock_storage.firmware_bin_path = firmware_file
+    mock_storage_json.load.return_value = mock_storage
+
+    # Request with URL-encoded path and custom download name
+    response = await dashboard.fetch(
+        "/download.bin?configuration=test.yaml&file=zephyr%2Fzephyr.uf2&download=custom_name.bin",
+        method="GET",
+    )
+    assert response.code == 200
+    assert "custom_name.bin" in response.headers["Content-Disposition"]
+
+
+@pytest.mark.asyncio
+@pytest.mark.usefixtures("mock_ext_storage_path")
+@pytest.mark.parametrize(
+    "attack_path",
+    [
+        pytest.param("../../../secrets.yaml", id="basic_traversal"),
+        pytest.param("..%2F..%2F..%2Fsecrets.yaml", id="url_encoded"),
+        pytest.param("zephyr/../../../secrets.yaml", id="traversal_with_prefix"),
+        pytest.param("/etc/passwd", id="absolute_path"),
+        pytest.param("//etc/passwd", id="double_slash_absolute"),
+        pytest.param("....//secrets.yaml", id="multiple_dots"),
+    ],
+)
+async def test_download_binary_handler_path_traversal_protection(
+    dashboard: DashboardTestHelper,
+    tmp_path: Path,
+    mock_storage_json: MagicMock,
+    attack_path: str,
+) -> None:
+    """Test that DownloadBinaryRequestHandler prevents path traversal attacks.
+
+    Verifies that attempts to use '..' in file paths are sanitized to prevent
+    accessing files outside the build directory. Tests multiple attack vectors.
+    """
+    # Create build structure
+    build_dir = get_build_path(tmp_path, "test")
+    build_dir.mkdir(parents=True)
+    firmware_file = build_dir / "firmware.bin"
+    firmware_file.write_bytes(b"firmware content")
+
+    # Create a sensitive file outside the build directory that should NOT be accessible
+    sensitive_file = tmp_path / "secrets.yaml"
+    sensitive_file.write_bytes(b"secret: my_secret_password")
+
+    # Mock storage JSON
+    mock_storage = Mock()
+    mock_storage.name = "test_device"
+    mock_storage.firmware_bin_path = firmware_file
+    mock_storage_json.load.return_value = mock_storage
+
+    # Attempt path traversal attack - should be blocked
+    with pytest.raises(HTTPClientError) as exc_info:
+        await dashboard.fetch(
+            f"/download.bin?configuration=test.yaml&file={attack_path}",
+            method="GET",
+        )
+    # Should get 404 (file not found after sanitization) or 500 (idedata fails)
+    assert exc_info.value.code in (404, 500)
+
+
+@pytest.mark.asyncio
+@pytest.mark.usefixtures("mock_ext_storage_path")
+async def test_download_binary_handler_multiple_subdirectory_levels(
+    dashboard: DashboardTestHelper,
+    tmp_path: Path,
+    mock_storage_json: MagicMock,
+) -> None:
+    """Test downloading files from multiple subdirectory levels.
+
+    Verifies that joinpath correctly handles multi-level paths like 'build/output/firmware.bin'.
+    """
+    # Create nested directory structure
+    build_dir = get_build_path(tmp_path, "test")
+    nested_dir = build_dir / "build" / "output"
+    nested_dir.mkdir(parents=True)
+
+    firmware_file = build_dir / "firmware.bin"
+    firmware_file.write_bytes(b"main")
+
+    nested_file = nested_dir / "firmware.bin"
+    nested_file.write_bytes(b"nested firmware content")
+
+    # Mock storage JSON
+    mock_storage = Mock()
+    mock_storage.name = "test_device"
+    mock_storage.firmware_bin_path = firmware_file
+    mock_storage_json.load.return_value = mock_storage
+
+    response = await dashboard.fetch(
+        "/download.bin?configuration=test.yaml&file=build/output/firmware.bin",
+        method="GET",
+    )
+    assert response.code == 200
+    assert response.body == b"nested firmware content"
+
+
+@pytest.mark.asyncio
+async def test_edit_request_handler_post_invalid_file(
+    dashboard: DashboardTestHelper,
+) -> None:
+    """Test the EditRequestHandler.post with non-yaml file."""
+    with pytest.raises(HTTPClientError) as exc_info:
+        await dashboard.fetch(
+            "/edit?configuration=test.txt",
+            method="POST",
+            body=b"content",
+        )
+    assert exc_info.value.code == 404
+
+
+@pytest.mark.asyncio
+async def test_edit_request_handler_post_existing(
+    dashboard: DashboardTestHelper,
+    tmp_path: Path,
+    mock_dashboard_settings: MagicMock,
+) -> None:
+    """Test the EditRequestHandler.post with existing yaml file."""
+    # Create a temporary yaml file to edit (don't modify fixtures)
+    test_file = tmp_path / "test_edit.yaml"
+    test_file.write_text("esphome:\n  name: original\n")
+
+    # Configure the mock settings
+    mock_dashboard_settings.rel_path.return_value = test_file
+    mock_dashboard_settings.absolute_config_dir = test_file.parent
+
+    new_content = "esphome:\n  name: modified\n"
+    response = await dashboard.fetch(
+        "/edit?configuration=test_edit.yaml",
+        method="POST",
+        body=new_content.encode(),
+    )
+    assert response.code == 200
+
+    # Verify the file was actually modified
+    assert test_file.read_text() == new_content
+
+
+@pytest.mark.asyncio
+async def test_unarchive_request_handler(
+    dashboard: DashboardTestHelper,
+    mock_archive_storage_path: MagicMock,
+    mock_dashboard_settings: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test the UnArchiveRequestHandler.post method."""
+    # Set up an archived file
+    archive_dir = mock_archive_storage_path.return_value
+    archive_dir.mkdir(parents=True, exist_ok=True)
+    archived_file = archive_dir / "archived.yaml"
+    archived_file.write_text("test content")
+
+    # Set up the destination path where the file should be moved
+    config_dir = tmp_path / "config"
+    config_dir.mkdir(parents=True, exist_ok=True)
+    destination_file = config_dir / "archived.yaml"
+    mock_dashboard_settings.rel_path.return_value = destination_file
+
+    response = await dashboard.fetch(
+        "/unarchive?configuration=archived.yaml",
+        method="POST",
+        body=b"",
+    )
+    assert response.code == 200
+
+    # Verify the file was actually moved from archive to config
+    assert not archived_file.exists()  # File should be gone from archive
+    assert destination_file.exists()  # File should now be in config
+    assert destination_file.read_text() == "test content"  # Content preserved
+
+
+@pytest.mark.asyncio
+async def test_secret_keys_handler_no_file(dashboard: DashboardTestHelper) -> None:
+    """Test the SecretKeysRequestHandler.get when no secrets file exists."""
+    # By default, there's no secrets file in the test fixtures
+    with pytest.raises(HTTPClientError) as exc_info:
+        await dashboard.fetch("/secret_keys", method="GET")
+    assert exc_info.value.code == 404
+
+
+@pytest.mark.asyncio
+async def test_secret_keys_handler_with_file(
+    dashboard: DashboardTestHelper,
+    tmp_path: Path,
+    mock_dashboard_settings: MagicMock,
+) -> None:
+    """Test the SecretKeysRequestHandler.get when secrets file exists."""
+    # Create a secrets file in temp directory
+    secrets_file = tmp_path / "secrets.yaml"
+    secrets_file.write_text(
+        "wifi_ssid: TestNetwork\nwifi_password: TestPass123\napi_key: test_key\n"
+    )
+
+    # Configure mock to return our temp secrets file
+    # Since the file actually exists, os.path.isfile will return True naturally
+    mock_dashboard_settings.rel_path.return_value = secrets_file
+
+    response = await dashboard.fetch("/secret_keys", method="GET")
+    assert response.code == 200
+    data = json.loads(response.body.decode())
+    assert "wifi_ssid" in data
+    assert "wifi_password" in data
+    assert "api_key" in data
+
+
+@pytest.mark.asyncio
+async def test_json_config_handler(
+    dashboard: DashboardTestHelper,
+    mock_async_run_system_command: MagicMock,
+) -> None:
+    """Test the JsonConfigRequestHandler.get method."""
+    # This will actually run the esphome config command on pico.yaml
+    mock_output = json.dumps(
+        {
+            "esphome": {"name": "pico"},
+            "esp32": {"board": "esp32dev"},
+        }
+    )
+    mock_async_run_system_command.return_value = (0, mock_output, "")
+
+    response = await dashboard.fetch(
+        "/json-config?configuration=pico.yaml", method="GET"
+    )
+    assert response.code == 200
+    data = json.loads(response.body.decode())
+    assert data["esphome"]["name"] == "pico"
+
+
+@pytest.mark.asyncio
+async def test_json_config_handler_invalid_config(
+    dashboard: DashboardTestHelper,
+    mock_async_run_system_command: MagicMock,
+) -> None:
+    """Test the JsonConfigRequestHandler.get with invalid config."""
+    # Simulate esphome config command failure
+    mock_async_run_system_command.return_value = (1, "", "Error: Invalid configuration")
+
+    with pytest.raises(HTTPClientError) as exc_info:
+        await dashboard.fetch("/json-config?configuration=pico.yaml", method="GET")
+    assert exc_info.value.code == 422
+
+
+@pytest.mark.asyncio
+async def test_json_config_handler_not_found(dashboard: DashboardTestHelper) -> None:
+    """Test the JsonConfigRequestHandler.get with non-existent file."""
+    with pytest.raises(HTTPClientError) as exc_info:
+        await dashboard.fetch(
+            "/json-config?configuration=nonexistent.yaml", method="GET"
+        )
+    assert exc_info.value.code == 404
+
+
+def test_start_web_server_with_address_port(
+    tmp_path: Path,
+    mock_trash_storage_path: MagicMock,
+    mock_archive_storage_path: MagicMock,
+) -> None:
+    """Test the start_web_server function with address and port."""
+    app = Mock()
+    trash_dir = mock_trash_storage_path.return_value
+    archive_dir = mock_archive_storage_path.return_value
+
+    # Create trash dir to test migration
+    trash_dir.mkdir()
+    (trash_dir / "old.yaml").write_text("old")
+
+    web_server.start_web_server(app, None, "127.0.0.1", 6052, str(tmp_path / "config"))
+
+    # The function calls app.listen directly for non-socket mode
+    app.listen.assert_called_once_with(6052, "127.0.0.1")
+
+    # Verify trash was moved to archive
+    assert not trash_dir.exists()
+    assert archive_dir.exists()
+    assert (archive_dir / "old.yaml").exists()
+
+
+@pytest.mark.asyncio
+async def test_edit_request_handler_get(dashboard: DashboardTestHelper) -> None:
+    """Test EditRequestHandler.get method."""
+    # Test getting a valid yaml file
+    response = await dashboard.fetch("/edit?configuration=pico.yaml")
+    assert response.code == 200
+    assert response.headers["content-type"] == "application/yaml"
+    content = response.body.decode()
+    assert "esphome:" in content  # Verify it's a valid ESPHome config
+
+    # Test getting a non-existent file
+    with pytest.raises(HTTPClientError) as exc_info:
+        await dashboard.fetch("/edit?configuration=nonexistent.yaml")
+    assert exc_info.value.code == 404
+
+    # Test getting a non-yaml file
+    with pytest.raises(HTTPClientError) as exc_info:
+        await dashboard.fetch("/edit?configuration=test.txt")
+    assert exc_info.value.code == 404
+
+    # Test path traversal attempt
+    with pytest.raises(HTTPClientError) as exc_info:
+        await dashboard.fetch("/edit?configuration=../../../etc/passwd")
+    assert exc_info.value.code == 404
+
+
+@pytest.mark.asyncio
+async def test_archive_request_handler_post(
+    dashboard: DashboardTestHelper,
+    mock_archive_storage_path: MagicMock,
+    mock_ext_storage_path: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test ArchiveRequestHandler.post method without storage_json."""
+
+    # Set up temp directories
+    config_dir = Path(get_fixture_path("conf"))
+    archive_dir = tmp_path / "archive"
+
+    # Create a test configuration file
+    test_config = config_dir / "test_archive.yaml"
+    test_config.write_text("esphome:\n  name: test_archive\n")
+
+    # Archive the configuration
+    response = await dashboard.fetch(
+        "/archive",
+        method="POST",
+        body="configuration=test_archive.yaml",
+        headers={"Content-Type": "application/x-www-form-urlencoded"},
+    )
+    assert response.code == 200
+
+    # Verify file was moved to archive
+    assert not test_config.exists()
+    assert (archive_dir / "test_archive.yaml").exists()
+    assert (
+        archive_dir / "test_archive.yaml"
+    ).read_text() == "esphome:\n  name: test_archive\n"
+
+
+@pytest.mark.asyncio
+async def test_archive_handler_with_build_folder(
+    dashboard: DashboardTestHelper,
+    mock_archive_storage_path: MagicMock,
+    mock_ext_storage_path: MagicMock,
+    mock_dashboard_settings: MagicMock,
+    mock_storage_json: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test ArchiveRequestHandler.post with storage_json and build folder."""
+    config_dir = tmp_path / "config"
+    config_dir.mkdir()
+    archive_dir = tmp_path / "archive"
+    archive_dir.mkdir()
+    build_dir = tmp_path / "build"
+    build_dir.mkdir()
+
+    configuration = "test_device.yaml"
+    test_config = config_dir / configuration
+    test_config.write_text("esphome:\n  name: test_device\n")
+
+    build_folder = build_dir / "test_device"
+    build_folder.mkdir()
+    (build_folder / "firmware.bin").write_text("binary content")
+    (build_folder / ".pioenvs").mkdir()
+
+    mock_dashboard_settings.config_dir = str(config_dir)
+    mock_dashboard_settings.rel_path.return_value = test_config
+    mock_archive_storage_path.return_value = archive_dir
+
+    mock_storage = MagicMock()
+    mock_storage.name = "test_device"
+    mock_storage.build_path = build_folder
+    mock_storage_json.load.return_value = mock_storage
+
+    response = await dashboard.fetch(
+        "/archive",
+        method="POST",
+        body=f"configuration={configuration}",
+        headers={"Content-Type": "application/x-www-form-urlencoded"},
+    )
+    assert response.code == 200
+
+    assert not test_config.exists()
+    assert (archive_dir / configuration).exists()
+
+    assert not build_folder.exists()
+    assert not (archive_dir / "test_device").exists()
+
+
+@pytest.mark.asyncio
+async def test_archive_handler_no_build_folder(
+    dashboard: DashboardTestHelper,
+    mock_archive_storage_path: MagicMock,
+    mock_ext_storage_path: MagicMock,
+    mock_dashboard_settings: MagicMock,
+    mock_storage_json: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test ArchiveRequestHandler.post with storage_json but no build folder."""
+    config_dir = tmp_path / "config"
+    config_dir.mkdir()
+    archive_dir = tmp_path / "archive"
+    archive_dir.mkdir()
+
+    configuration = "test_device.yaml"
+    test_config = config_dir / configuration
+    test_config.write_text("esphome:\n  name: test_device\n")
+
+    mock_dashboard_settings.config_dir = str(config_dir)
+    mock_dashboard_settings.rel_path.return_value = test_config
+    mock_archive_storage_path.return_value = archive_dir
+
+    mock_storage = MagicMock()
+    mock_storage.name = "test_device"
+    mock_storage.build_path = None
+    mock_storage_json.load.return_value = mock_storage
+
+    response = await dashboard.fetch(
+        "/archive",
+        method="POST",
+        body=f"configuration={configuration}",
+        headers={"Content-Type": "application/x-www-form-urlencoded"},
+    )
+    assert response.code == 200
+
+    assert not test_config.exists()
+    assert (archive_dir / configuration).exists()
+    assert not (archive_dir / "test_device").exists()
+
+
+@pytest.mark.skipif(os.name == "nt", reason="Unix sockets are not supported on Windows")
+@pytest.mark.usefixtures("mock_trash_storage_path", "mock_archive_storage_path")
+def test_start_web_server_with_unix_socket(tmp_path: Path) -> None:
+    """Test the start_web_server function with unix socket."""
+    app = Mock()
+    socket_path = tmp_path / "test.sock"
+
+    # Don't create trash_dir - it doesn't exist, so no migration needed
+    with (
+        patch("tornado.httpserver.HTTPServer") as mock_server_class,
+        patch("tornado.netutil.bind_unix_socket") as mock_bind,
+    ):
+        server = Mock()
+        mock_server_class.return_value = server
+        mock_bind.return_value = Mock()
+
+        web_server.start_web_server(
+            app, str(socket_path), None, None, str(tmp_path / "config")
+        )
+
+        mock_server_class.assert_called_once_with(app)
+        mock_bind.assert_called_once_with(str(socket_path), mode=0o666)
+        server.add_socket.assert_called_once()
+
+
+def test_build_cache_arguments_no_entry(mock_dashboard: Mock) -> None:
+    """Test with no entry returns empty list."""
+    result = web_server.build_cache_arguments(None, mock_dashboard, 0.0)
+    assert result == []
+
+
+def test_build_cache_arguments_no_address_no_name(mock_dashboard: Mock) -> None:
+    """Test with entry but no address or name."""
+    entry = Mock(spec=web_server.DashboardEntry)
+    entry.address = None
+    entry.name = None
+    result = web_server.build_cache_arguments(entry, mock_dashboard, 0.0)
+    assert result == []
+
+
+def test_build_cache_arguments_mdns_address_cached(mock_dashboard: Mock) -> None:
+    """Test with .local address that has cached mDNS results."""
+    entry = Mock(spec=web_server.DashboardEntry)
+    entry.address = "device.local"
+    entry.name = None
+    mock_dashboard.mdns_status = Mock()
+    mock_dashboard.mdns_status.get_cached_addresses.return_value = [
+        "192.168.1.10",
+        "fe80::1",
+    ]
+
+    result = web_server.build_cache_arguments(entry, mock_dashboard, 0.0)
+
+    assert result == [
+        "--mdns-address-cache",
+        "device.local=192.168.1.10,fe80::1",
+    ]
+    mock_dashboard.mdns_status.get_cached_addresses.assert_called_once_with(
+        "device.local"
+    )
+
+
+def test_build_cache_arguments_dns_address_cached(mock_dashboard: Mock) -> None:
+    """Test with non-.local address that has cached DNS results."""
+    entry = Mock(spec=web_server.DashboardEntry)
+    entry.address = "example.com"
+    entry.name = None
+    mock_dashboard.dns_cache = Mock()
+    mock_dashboard.dns_cache.get_cached_addresses.return_value = [
+        "93.184.216.34",
+        "2606:2800:220:1:248:1893:25c8:1946",
+    ]
+
+    now = 100.0
+    result = web_server.build_cache_arguments(entry, mock_dashboard, now)
+
+    # IPv6 addresses are sorted before IPv4
+    assert result == [
+        "--dns-address-cache",
+        "example.com=2606:2800:220:1:248:1893:25c8:1946,93.184.216.34",
+    ]
+    mock_dashboard.dns_cache.get_cached_addresses.assert_called_once_with(
+        "example.com", now
+    )
+
+
+def test_build_cache_arguments_name_without_address(mock_dashboard: Mock) -> None:
+    """Test with name but no address - should check mDNS with .local suffix."""
+    entry = Mock(spec=web_server.DashboardEntry)
+    entry.name = "my-device"
+    entry.address = None
+    mock_dashboard.mdns_status = Mock()
+    mock_dashboard.mdns_status.get_cached_addresses.return_value = ["192.168.1.20"]
+
+    result = web_server.build_cache_arguments(entry, mock_dashboard, 0.0)
+
+    assert result == [
+        "--mdns-address-cache",
+        "my-device.local=192.168.1.20",
+    ]
+    mock_dashboard.mdns_status.get_cached_addresses.assert_called_once_with(
+        "my-device.local"
+    )
+
+
+@pytest.mark.asyncio
+async def test_websocket_connection_initial_state(
+    dashboard: DashboardTestHelper,
+) -> None:
+    """Test WebSocket connection and initial state."""
+    async with websocket_connection(dashboard) as ws:
+        # Should receive initial state with configured and importable devices
+        msg = await ws.read_message()
+        assert msg is not None
+        data = json.loads(msg)
+        assert data["event"] == "initial_state"
+        assert "devices" in data["data"]
+        assert "configured" in data["data"]["devices"]
+        assert "importable" in data["data"]["devices"]
+
+        # Check configured devices
+        configured = data["data"]["devices"]["configured"]
+        assert len(configured) > 0
+        assert configured[0]["name"] == "pico"  # From test fixtures
+
+
+@pytest.mark.asyncio
+async def test_websocket_ping_pong(
+    dashboard: DashboardTestHelper, websocket_client: WebSocketClientConnection
+) -> None:
+    """Test WebSocket ping/pong mechanism."""
+    # Send ping
+    await websocket_client.write_message(json.dumps({"event": "ping"}))
+
+    # Should receive pong
+    msg = await websocket_client.read_message()
+    assert msg is not None
+    data = json.loads(msg)
+    assert data["event"] == "pong"
+
+
+@pytest.mark.asyncio
+async def test_websocket_invalid_json(
+    dashboard: DashboardTestHelper, websocket_client: WebSocketClientConnection
+) -> None:
+    """Test WebSocket handling of invalid JSON."""
+    # Send invalid JSON
+    await websocket_client.write_message("not valid json {]")
+
+    # Send a valid ping to verify connection is still alive
+    await websocket_client.write_message(json.dumps({"event": "ping"}))
+
+    # Should receive pong, confirming the connection wasn't closed by invalid JSON
+    msg = await websocket_client.read_message()
+    assert msg is not None
+    data = json.loads(msg)
+    assert data["event"] == "pong"
+
+
+@pytest.mark.asyncio
+async def test_websocket_authentication_required(
+    dashboard: DashboardTestHelper,
+) -> None:
+    """Test WebSocket authentication when auth is required."""
+    with patch(
+        "esphome.dashboard.web_server.is_authenticated"
+    ) as mock_is_authenticated:
+        mock_is_authenticated.return_value = False
+
+        # Try to connect - should be rejected with 401
+        url = f"ws://127.0.0.1:{dashboard.port}/events"
+        with pytest.raises(HTTPClientError) as exc_info:
+            await websocket_connect(url)
+        # Should get HTTP 401 Unauthorized
+        assert exc_info.value.code == 401
+
+
+@pytest.mark.asyncio
+async def test_websocket_authentication_not_required(
+    dashboard: DashboardTestHelper,
+) -> None:
+    """Test WebSocket connection when no auth is required."""
+    with patch(
+        "esphome.dashboard.web_server.is_authenticated"
+    ) as mock_is_authenticated:
+        mock_is_authenticated.return_value = True
+
+        # Should be able to connect successfully
+        async with websocket_connection(dashboard) as ws:
+            msg = await ws.read_message()
+            assert msg is not None
+            data = json.loads(msg)
+            assert data["event"] == "initial_state"
+
+
+@pytest.mark.asyncio
+async def test_websocket_entry_state_changed(
+    dashboard: DashboardTestHelper, websocket_client: WebSocketClientConnection
+) -> None:
+    """Test WebSocket entry state changed event."""
+    # Simulate entry state change
+    entry = DASHBOARD.entries.async_all()[0]
+    state = bool_to_entry_state(True, EntryStateSource.MDNS)
+    DASHBOARD.bus.async_fire(
+        DashboardEvent.ENTRY_STATE_CHANGED, {"entry": entry, "state": state}
+    )
+
+    # Should receive state change event
+    msg = await websocket_client.read_message()
+    assert msg is not None
+    data = json.loads(msg)
+    assert data["event"] == "entry_state_changed"
+    assert data["data"]["filename"] == entry.filename
+    assert data["data"]["name"] == entry.name
+    assert data["data"]["state"] is True
+
+
+@pytest.mark.asyncio
+async def test_websocket_entry_added(
+    dashboard: DashboardTestHelper, websocket_client: WebSocketClientConnection
+) -> None:
+    """Test WebSocket entry added event."""
+    # Create a mock entry
+    mock_entry = Mock(spec=DashboardEntry)
+    mock_entry.filename = "test.yaml"
+    mock_entry.name = "test_device"
+    mock_entry.to_dict.return_value = {
+        "name": "test_device",
+        "filename": "test.yaml",
+        "configuration": "test.yaml",
+    }
+
+    # Simulate entry added
+    DASHBOARD.bus.async_fire(DashboardEvent.ENTRY_ADDED, {"entry": mock_entry})
+
+    # Should receive entry added event
+    msg = await websocket_client.read_message()
+    assert msg is not None
+    data = json.loads(msg)
+    assert data["event"] == "entry_added"
+    assert data["data"]["device"]["name"] == "test_device"
+    assert data["data"]["device"]["filename"] == "test.yaml"
+
+
+@pytest.mark.asyncio
+async def test_websocket_entry_removed(
+    dashboard: DashboardTestHelper, websocket_client: WebSocketClientConnection
+) -> None:
+    """Test WebSocket entry removed event."""
+    # Create a mock entry
+    mock_entry = Mock(spec=DashboardEntry)
+    mock_entry.filename = "removed.yaml"
+    mock_entry.name = "removed_device"
+    mock_entry.to_dict.return_value = {
+        "name": "removed_device",
+        "filename": "removed.yaml",
+        "configuration": "removed.yaml",
+    }
+
+    # Simulate entry removed
+    DASHBOARD.bus.async_fire(DashboardEvent.ENTRY_REMOVED, {"entry": mock_entry})
+
+    # Should receive entry removed event
+    msg = await websocket_client.read_message()
+    assert msg is not None
+    data = json.loads(msg)
+    assert data["event"] == "entry_removed"
+    assert data["data"]["device"]["name"] == "removed_device"
+    assert data["data"]["device"]["filename"] == "removed.yaml"
+
+
+@pytest.mark.asyncio
+async def test_websocket_importable_device_added(
+    dashboard: DashboardTestHelper, websocket_client: WebSocketClientConnection
+) -> None:
+    """Test WebSocket importable device added event with real DiscoveredImport."""
+    # Create a real DiscoveredImport object
+    discovered = DiscoveredImport(
+        device_name="new_import_device",
+        friendly_name="New Import Device",
+        package_import_url="https://example.com/package",
+        project_name="test_project",
+        project_version="1.0.0",
+        network="wifi",
+    )
+
+    # Directly fire the event as the mDNS system would
+    device_dict = build_importable_device_dict(DASHBOARD, discovered)
+    DASHBOARD.bus.async_fire(
+        DashboardEvent.IMPORTABLE_DEVICE_ADDED, {"device": device_dict}
+    )
+
+    # Should receive importable device added event
+    msg = await websocket_client.read_message()
+    assert msg is not None
+    data = json.loads(msg)
+    assert data["event"] == "importable_device_added"
+    assert data["data"]["device"]["name"] == "new_import_device"
+    assert data["data"]["device"]["friendly_name"] == "New Import Device"
+    assert data["data"]["device"]["project_name"] == "test_project"
+    assert data["data"]["device"]["network"] == "wifi"
+    assert data["data"]["device"]["ignored"] is False
+
+
+@pytest.mark.asyncio
+async def test_websocket_importable_device_added_ignored(
+    dashboard: DashboardTestHelper, websocket_client: WebSocketClientConnection
+) -> None:
+    """Test WebSocket importable device added event for ignored device."""
+    # Add device to ignored list
+    DASHBOARD.ignored_devices.add("ignored_device")
+
+    # Create a real DiscoveredImport object
+    discovered = DiscoveredImport(
+        device_name="ignored_device",
+        friendly_name="Ignored Device",
+        package_import_url="https://example.com/package",
+        project_name="test_project",
+        project_version="1.0.0",
+        network="ethernet",
+    )
+
+    # Directly fire the event as the mDNS system would
+    device_dict = build_importable_device_dict(DASHBOARD, discovered)
+    DASHBOARD.bus.async_fire(
+        DashboardEvent.IMPORTABLE_DEVICE_ADDED, {"device": device_dict}
+    )
+
+    # Should receive importable device added event with ignored=True
+    msg = await websocket_client.read_message()
+    assert msg is not None
+    data = json.loads(msg)
+    assert data["event"] == "importable_device_added"
+    assert data["data"]["device"]["name"] == "ignored_device"
+    assert data["data"]["device"]["friendly_name"] == "Ignored Device"
+    assert data["data"]["device"]["network"] == "ethernet"
+    assert data["data"]["device"]["ignored"] is True
+
+
+@pytest.mark.asyncio
+async def test_websocket_importable_device_removed(
+    dashboard: DashboardTestHelper, websocket_client: WebSocketClientConnection
+) -> None:
+    """Test WebSocket importable device removed event."""
+    # Simulate importable device removed
+    DASHBOARD.bus.async_fire(
+        DashboardEvent.IMPORTABLE_DEVICE_REMOVED,
+        {"name": "removed_import_device"},
+    )
+
+    # Should receive importable device removed event
+    msg = await websocket_client.read_message()
+    assert msg is not None
+    data = json.loads(msg)
+    assert data["event"] == "importable_device_removed"
+    assert data["data"]["name"] == "removed_import_device"
+
+
+@pytest.mark.asyncio
+async def test_websocket_importable_device_already_configured(
+    dashboard: DashboardTestHelper, websocket_client: WebSocketClientConnection
+) -> None:
+    """Test that importable device event is not sent if device is already configured."""
+    # Get an existing configured device name
+    existing_entry = DASHBOARD.entries.async_all()[0]
+
+    # Simulate importable device added with same name as configured device
+    DASHBOARD.bus.async_fire(
+        DashboardEvent.IMPORTABLE_DEVICE_ADDED,
+        {
+            "device": {
+                "name": existing_entry.name,
+                "friendly_name": "Should Not Be Sent",
+                "package_import_url": "https://example.com/package",
+                "project_name": "test_project",
+                "project_version": "1.0.0",
+                "network": "wifi",
+            }
+        },
+    )
+
+    # Send a ping to ensure connection is still alive
+    await websocket_client.write_message(json.dumps({"event": "ping"}))
+
+    # Should only receive pong, not the importable device event
+    msg = await websocket_client.read_message()
+    assert msg is not None
+    data = json.loads(msg)
+    assert data["event"] == "pong"
+
+
+@pytest.mark.asyncio
+async def test_websocket_multiple_connections(dashboard: DashboardTestHelper) -> None:
+    """Test multiple WebSocket connections."""
+    async with (
+        websocket_connection(dashboard) as ws1,
+        websocket_connection(dashboard) as ws2,
+    ):
+        # Both should receive initial state
+        msg1 = await ws1.read_message()
+        assert msg1 is not None
+        data1 = json.loads(msg1)
+        assert data1["event"] == "initial_state"
+
+        msg2 = await ws2.read_message()
+        assert msg2 is not None
+        data2 = json.loads(msg2)
+        assert data2["event"] == "initial_state"
+
+        # Fire an event - both should receive it
+        entry = DASHBOARD.entries.async_all()[0]
+        state = bool_to_entry_state(False, EntryStateSource.MDNS)
+        DASHBOARD.bus.async_fire(
+            DashboardEvent.ENTRY_STATE_CHANGED, {"entry": entry, "state": state}
+        )
+
+        msg1 = await ws1.read_message()
+        assert msg1 is not None
+        data1 = json.loads(msg1)
+        assert data1["event"] == "entry_state_changed"
+
+        msg2 = await ws2.read_message()
+        assert msg2 is not None
+        data2 = json.loads(msg2)
+        assert data2["event"] == "entry_state_changed"
+
+
+@pytest.mark.asyncio
+async def test_dashboard_subscriber_lifecycle(dashboard: DashboardTestHelper) -> None:
+    """Test DashboardSubscriber lifecycle."""
+    subscriber = DashboardSubscriber()
+
+    # Initially no subscribers
+    assert len(subscriber._subscribers) == 0
+    assert subscriber._event_loop_task is None
+
+    # Add a subscriber
+    mock_websocket = Mock()
+    unsubscribe = subscriber.subscribe(mock_websocket)
+
+    # Should have started the event loop task
+    assert len(subscriber._subscribers) == 1
+    assert subscriber._event_loop_task is not None
+
+    # Unsubscribe
+    unsubscribe()
+
+    # Should have stopped the task
+    assert len(subscriber._subscribers) == 0
+
+
+@pytest.mark.asyncio
+async def test_dashboard_subscriber_entries_update_interval(
+    dashboard: DashboardTestHelper,
+) -> None:
+    """Test DashboardSubscriber entries update interval."""
+    # Patch the constants to make the test run faster
+    with (
+        patch("esphome.dashboard.web_server.DASHBOARD_POLL_INTERVAL", 0.01),
+        patch("esphome.dashboard.web_server.DASHBOARD_ENTRIES_UPDATE_ITERATIONS", 2),
+        patch("esphome.dashboard.web_server.settings") as mock_settings,
+        patch("esphome.dashboard.web_server.DASHBOARD") as mock_dashboard,
+    ):
+        mock_settings.status_use_mqtt = False
+
+        # Mock dashboard dependencies
+        mock_dashboard.ping_request = Mock()
+        mock_dashboard.ping_request.set = Mock()
+        mock_dashboard.entries = Mock()
+        mock_dashboard.entries.async_request_update_entries = Mock()
+
+        subscriber = DashboardSubscriber()
+        mock_websocket = Mock()
+
+        # Subscribe to start the event loop
+        unsubscribe = subscriber.subscribe(mock_websocket)
+
+        # Wait for a few iterations to ensure entries update is called
+        await asyncio.sleep(0.05)  # Should be enough for 2+ iterations
+
+        # Unsubscribe to stop the task
+        unsubscribe()
+
+        # Verify entries update was called
+        assert mock_dashboard.entries.async_request_update_entries.call_count >= 1
+        # Verify ping request was set multiple times
+        assert mock_dashboard.ping_request.set.call_count >= 2
+
+
+@pytest.mark.asyncio
+async def test_websocket_refresh_command(
+    dashboard: DashboardTestHelper, websocket_client: WebSocketClientConnection
+) -> None:
+    """Test WebSocket refresh command triggers dashboard update."""
+    with patch("esphome.dashboard.web_server.DASHBOARD_SUBSCRIBER") as mock_subscriber:
+        mock_subscriber.request_refresh = Mock()
+
+        # Send refresh command
+        await websocket_client.write_message(json.dumps({"event": "refresh"}))
+
+        # Give it a moment to process
+        await asyncio.sleep(0.01)
+
+        # Verify request_refresh was called
+        mock_subscriber.request_refresh.assert_called_once()
+
+
+@pytest.mark.asyncio
+async def test_dashboard_subscriber_refresh_event(
+    dashboard: DashboardTestHelper,
+) -> None:
+    """Test DashboardSubscriber refresh event triggers immediate update."""
+    # Patch the constants to make the test run faster
+    with (
+        patch(
+            "esphome.dashboard.web_server.DASHBOARD_POLL_INTERVAL", 1.0
+        ),  # Long timeout
+        patch(
+            "esphome.dashboard.web_server.DASHBOARD_ENTRIES_UPDATE_ITERATIONS", 100
+        ),  # Won't reach naturally
+        patch("esphome.dashboard.web_server.settings") as mock_settings,
+        patch("esphome.dashboard.web_server.DASHBOARD") as mock_dashboard,
+    ):
+        mock_settings.status_use_mqtt = False
+
+        # Mock dashboard dependencies
+        mock_dashboard.ping_request = Mock()
+        mock_dashboard.ping_request.set = Mock()
+        mock_dashboard.entries = Mock()
+        mock_dashboard.entries.async_request_update_entries = AsyncMock()
+
+        subscriber = DashboardSubscriber()
+        mock_websocket = Mock()
+
+        # Subscribe to start the event loop
+        unsubscribe = subscriber.subscribe(mock_websocket)
+
+        # Wait a bit to ensure loop is running
+        await asyncio.sleep(0.01)
+
+        # Verify entries update hasn't been called yet (iterations not reached)
+        assert mock_dashboard.entries.async_request_update_entries.call_count == 0
+
+        # Request refresh
+        subscriber.request_refresh()
+
+        # Wait for the refresh to be processed
+        await asyncio.sleep(0.01)
+
+        # Now entries update should have been called
+        assert mock_dashboard.entries.async_request_update_entries.call_count == 1
+
+        # Unsubscribe to stop the task
+        unsubscribe()
+
+        # Give it a moment to clean up
+        await asyncio.sleep(0.01)
+
+
+@pytest.mark.asyncio
+async def test_dashboard_yaml_loading_with_packages_and_secrets(
+    tmp_path: Path,
+) -> None:
+    """Test dashboard YAML loading with packages referencing secrets.
+
+    This is a regression test for issue #11280 where binary download failed
+    when using packages with secrets after the Path migration in 2025.10.0.
+
+    This test verifies that CORE.config_path initialization in the dashboard
+    allows yaml_util.load_yaml() to correctly resolve secrets from packages.
+    """
+    # Create test directory structure with secrets and packages
+    config_dir = tmp_path / "config"
+    config_dir.mkdir()
+
+    # Create secrets.yaml with obviously fake test values
+    secrets_file = config_dir / "secrets.yaml"
+    secrets_file.write_text(
+        "wifi_ssid: TEST-DUMMY-SSID\n"
+        "wifi_password: not-a-real-password-just-for-testing\n"
+    )
+
+    # Create package file that uses secrets
+    package_file = config_dir / "common.yaml"
+    package_file.write_text(
+        "wifi:\n  ssid: !secret wifi_ssid\n  password: !secret wifi_password\n"
+    )
+
+    # Create main device config that includes the package
+    device_config = config_dir / "test-download-secrets.yaml"
+    device_config.write_text(
+        "esphome:\n  name: test-download-secrets\n  platform: ESP32\n  board: esp32dev\n\n"
+        "packages:\n  common: !include common.yaml\n"
+    )
+
+    # Initialize DASHBOARD settings with our test config directory
+    # This is what sets CORE.config_path - the critical code path for the bug
+    args = Namespace(
+        configuration=str(config_dir),
+        password=None,
+        username=None,
+        ha_addon=False,
+        verbose=False,
+    )
+    DASHBOARD.settings.parse_args(args)
+
+    # With the fix: CORE.config_path should be config_dir / "___DASHBOARD_SENTINEL___.yaml"
+    # so CORE.config_path.parent would be config_dir
+    # Without the fix: CORE.config_path is config_dir / "." which normalizes to config_dir
+    # so CORE.config_path.parent would be tmp_path (the parent of config_dir)
+
+    # The fix ensures CORE.config_path.parent points to config_dir
+    assert CORE.config_path.parent == config_dir.resolve(), (
+        f"CORE.config_path.parent should point to config_dir. "
+        f"Got {CORE.config_path.parent}, expected {config_dir.resolve()}. "
+        f"CORE.config_path is {CORE.config_path}"
+    )
+
+    # Now load the YAML with packages that reference secrets
+    # This is where the bug would manifest - yaml_util.load_yaml would fail
+    # to find secrets.yaml because CORE.config_path.parent pointed to the wrong place
+    config = yaml_util.load_yaml(device_config)
+    # If we get here, secret resolution worked!
+    assert "esphome" in config
+    assert config["esphome"]["name"] == "test-download-secrets"
diff --git a/tests/dashboard/test_web_server_paths.py b/tests/dashboard/test_web_server_paths.py
new file mode 100644
index 0000000000..b596ebb581
--- /dev/null
+++ b/tests/dashboard/test_web_server_paths.py
@@ -0,0 +1,223 @@
+"""Tests for dashboard web_server Path-related functionality."""
+
+from __future__ import annotations
+
+import gzip
+import os
+from pathlib import Path
+from unittest.mock import MagicMock, patch
+
+from esphome.dashboard import web_server
+
+
+def test_get_base_frontend_path_production() -> None:
+    """Test get_base_frontend_path in production mode."""
+    mock_module = MagicMock()
+    mock_module.where.return_value = Path("/usr/local/lib/esphome_dashboard")
+
+    with (
+        patch.dict(os.environ, {}, clear=True),
+        patch.dict("sys.modules", {"esphome_dashboard": mock_module}),
+    ):
+        result = web_server.get_base_frontend_path()
+        assert result == Path("/usr/local/lib/esphome_dashboard")
+        mock_module.where.assert_called_once()
+
+
+def test_get_base_frontend_path_dev_mode() -> None:
+    """Test get_base_frontend_path in development mode."""
+    test_path = "/home/user/esphome/dashboard"
+
+    with patch.dict(os.environ, {"ESPHOME_DASHBOARD_DEV": test_path}):
+        result = web_server.get_base_frontend_path()
+
+        # The function uses Path.resolve() which resolves symlinks
+        # The actual function adds "/" to the path, so we simulate that
+        test_path_with_slash = test_path if test_path.endswith("/") else test_path + "/"
+        expected = (
+            Path(os.getcwd()) / test_path_with_slash / "esphome_dashboard"
+        ).resolve()
+        assert result == expected
+
+
+def test_get_base_frontend_path_dev_mode_with_trailing_slash() -> None:
+    """Test get_base_frontend_path in dev mode with trailing slash."""
+    test_path = "/home/user/esphome/dashboard/"
+
+    with patch.dict(os.environ, {"ESPHOME_DASHBOARD_DEV": test_path}):
+        result = web_server.get_base_frontend_path()
+
+        # The function uses Path.resolve() which resolves symlinks
+        expected = (Path.cwd() / test_path / "esphome_dashboard").resolve()
+        assert result == expected
+
+
+def test_get_base_frontend_path_dev_mode_relative_path() -> None:
+    """Test get_base_frontend_path with relative dev path."""
+    test_path = "./dashboard"
+
+    with patch.dict(os.environ, {"ESPHOME_DASHBOARD_DEV": test_path}):
+        result = web_server.get_base_frontend_path()
+
+        # The function uses Path.resolve() which resolves symlinks
+        # The actual function adds "/" to the path, so we simulate that
+        test_path_with_slash = test_path if test_path.endswith("/") else test_path + "/"
+        expected = (
+            Path(os.getcwd()) / test_path_with_slash / "esphome_dashboard"
+        ).resolve()
+        assert result == expected
+        assert result.is_absolute()
+
+
+def test_get_static_path_single_component() -> None:
+    """Test get_static_path with single path component."""
+    with patch("esphome.dashboard.web_server.get_base_frontend_path") as mock_base:
+        mock_base.return_value = Path("/base/frontend")
+
+        result = web_server.get_static_path("file.js")
+
+        assert result == Path("/base/frontend") / "static" / "file.js"
+
+
+def test_get_static_path_multiple_components() -> None:
+    """Test get_static_path with multiple path components."""
+    with patch("esphome.dashboard.web_server.get_base_frontend_path") as mock_base:
+        mock_base.return_value = Path("/base/frontend")
+
+        result = web_server.get_static_path("js", "esphome", "index.js")
+
+        assert (
+            result == Path("/base/frontend") / "static" / "js" / "esphome" / "index.js"
+        )
+
+
+def test_get_static_path_empty_args() -> None:
+    """Test get_static_path with no arguments."""
+    with patch("esphome.dashboard.web_server.get_base_frontend_path") as mock_base:
+        mock_base.return_value = Path("/base/frontend")
+
+        result = web_server.get_static_path()
+
+        assert result == Path("/base/frontend") / "static"
+
+
+def test_get_static_path_with_pathlib_path() -> None:
+    """Test get_static_path with Path objects."""
+    with patch("esphome.dashboard.web_server.get_base_frontend_path") as mock_base:
+        mock_base.return_value = Path("/base/frontend")
+
+        path_obj = Path("js") / "app.js"
+        result = web_server.get_static_path(str(path_obj))
+
+        assert result == Path("/base/frontend") / "static" / "js" / "app.js"
+
+
+def test_get_static_file_url_production() -> None:
+    """Test get_static_file_url in production mode."""
+    web_server.get_static_file_url.cache_clear()
+    mock_module = MagicMock()
+    mock_path = MagicMock(spec=Path)
+    mock_path.read_bytes.return_value = b"test content"
+
+    with (
+        patch.dict(os.environ, {}, clear=True),
+        patch.dict("sys.modules", {"esphome_dashboard": mock_module}),
+        patch("esphome.dashboard.web_server.get_static_path") as mock_get_path,
+    ):
+        mock_get_path.return_value = mock_path
+        result = web_server.get_static_file_url("js/app.js")
+        assert result.startswith("./static/js/app.js?hash=")
+
+
+def test_get_static_file_url_dev_mode() -> None:
+    """Test get_static_file_url in development mode."""
+    with patch.dict(os.environ, {"ESPHOME_DASHBOARD_DEV": "/dev/path"}):
+        web_server.get_static_file_url.cache_clear()
+        result = web_server.get_static_file_url("js/app.js")
+
+        assert result == "./static/js/app.js"
+
+
+def test_get_static_file_url_index_js_special_case() -> None:
+    """Test get_static_file_url replaces index.js with entrypoint."""
+    web_server.get_static_file_url.cache_clear()
+    mock_module = MagicMock()
+    mock_module.entrypoint.return_value = "main.js"
+
+    with (
+        patch.dict(os.environ, {}, clear=True),
+        patch.dict("sys.modules", {"esphome_dashboard": mock_module}),
+    ):
+        result = web_server.get_static_file_url("js/esphome/index.js")
+        assert result == "./static/js/esphome/main.js"
+
+
+def test_load_file_path(tmp_path: Path) -> None:
+    """Test loading a file."""
+    test_file = tmp_path / "test.txt"
+    test_file.write_bytes(b"test content")
+
+    with open(test_file, "rb") as f:
+        content = f.read()
+    assert content == b"test content"
+
+
+def test_load_file_compressed_path(tmp_path: Path) -> None:
+    """Test loading a compressed file."""
+    test_file = tmp_path / "test.txt.gz"
+
+    with gzip.open(test_file, "wb") as gz:
+        gz.write(b"compressed content")
+
+    with gzip.open(test_file, "rb") as gz:
+        content = gz.read()
+    assert content == b"compressed content"
+
+
+def test_path_normalization_in_static_path() -> None:
+    """Test that paths are normalized correctly."""
+    with patch("esphome.dashboard.web_server.get_base_frontend_path") as mock_base:
+        mock_base.return_value = Path("/base/frontend")
+
+        # Test with separate components
+        result1 = web_server.get_static_path("js", "app.js")
+        result2 = web_server.get_static_path("js", "app.js")
+
+        assert result1 == result2
+        assert result1 == Path("/base/frontend") / "static" / "js" / "app.js"
+
+
+def test_windows_path_handling() -> None:
+    """Test handling of Windows-style paths."""
+    with patch("esphome.dashboard.web_server.get_base_frontend_path") as mock_base:
+        mock_base.return_value = Path(r"C:\Program Files\esphome\frontend")
+
+        result = web_server.get_static_path("js", "app.js")
+
+        # Path should handle this correctly on the platform
+        expected = (
+            Path(r"C:\Program Files\esphome\frontend") / "static" / "js" / "app.js"
+        )
+        assert result == expected
+
+
+def test_path_with_special_characters() -> None:
+    """Test paths with special characters."""
+    with patch("esphome.dashboard.web_server.get_base_frontend_path") as mock_base:
+        mock_base.return_value = Path("/base/frontend")
+
+        result = web_server.get_static_path("js-modules", "app_v1.0.js")
+
+        assert (
+            result == Path("/base/frontend") / "static" / "js-modules" / "app_v1.0.js"
+        )
+
+
+def test_path_with_spaces() -> None:
+    """Test paths with spaces."""
+    with patch("esphome.dashboard.web_server.get_base_frontend_path") as mock_base:
+        mock_base.return_value = Path("/base/my frontend")
+
+        result = web_server.get_static_path("my js", "my app.js")
+
+        assert result == Path("/base/my frontend") / "static" / "my js" / "my app.js"
diff --git a/tests/dashboard/util/test_file.py b/tests/dashboard/util/test_file.py
deleted file mode 100644
index 51ba10b328..0000000000
--- a/tests/dashboard/util/test_file.py
+++ /dev/null
@@ -1,56 +0,0 @@
-import os
-from pathlib import Path
-from unittest.mock import patch
-
-import py
-import pytest
-
-from esphome.dashboard.util.file import write_file, write_utf8_file
-
-
-def test_write_utf8_file(tmp_path: Path) -> None:
-    write_utf8_file(tmp_path.joinpath("foo.txt"), "foo")
-    assert tmp_path.joinpath("foo.txt").read_text() == "foo"
-
-    with pytest.raises(OSError):
-        write_utf8_file(Path("/dev/not-writable"), "bar")
-
-
-def test_write_file(tmp_path: Path) -> None:
-    write_file(tmp_path.joinpath("foo.txt"), b"foo")
-    assert tmp_path.joinpath("foo.txt").read_text() == "foo"
-
-
-def test_write_utf8_file_fails_at_rename(
-    tmpdir: py.path.local, caplog: pytest.LogCaptureFixture
-) -> None:
-    """Test that if rename fails not not remove, we do not log the failed cleanup."""
-    test_dir = tmpdir.mkdir("files")
-    test_file = Path(test_dir / "test.json")
-
-    with (
-        pytest.raises(OSError),
-        patch("esphome.dashboard.util.file.os.replace", side_effect=OSError),
-    ):
-        write_utf8_file(test_file, '{"some":"data"}', False)
-
-    assert not os.path.exists(test_file)
-
-    assert "File replacement cleanup failed" not in caplog.text
-
-
-def test_write_utf8_file_fails_at_rename_and_remove(
-    tmpdir: py.path.local, caplog: pytest.LogCaptureFixture
-) -> None:
-    """Test that if rename and remove both fail, we log the failed cleanup."""
-    test_dir = tmpdir.mkdir("files")
-    test_file = Path(test_dir / "test.json")
-
-    with (
-        pytest.raises(OSError),
-        patch("esphome.dashboard.util.file.os.remove", side_effect=OSError),
-        patch("esphome.dashboard.util.file.os.replace", side_effect=OSError),
-    ):
-        write_utf8_file(test_file, '{"some":"data"}', False)
-
-    assert "File replacement cleanup failed" in caplog.text
diff --git a/tests/integration/conftest.py b/tests/integration/conftest.py
index e3ba09de43..965363972f 100644
--- a/tests/integration/conftest.py
+++ b/tests/integration/conftest.py
@@ -58,6 +58,8 @@ def _get_platformio_env(cache_dir: Path) -> dict[str, str]:
     env["PLATFORMIO_CORE_DIR"] = str(cache_dir)
     env["PLATFORMIO_CACHE_DIR"] = str(cache_dir / ".cache")
     env["PLATFORMIO_LIBDEPS_DIR"] = str(cache_dir / "libdeps")
+    # Prevent cache cleaning during integration tests
+    env["ESPHOME_SKIP_CLEAN_BUILD"] = "1"
     return env
 
 
@@ -68,6 +70,11 @@ def shared_platformio_cache() -> Generator[Path]:
     test_cache_dir = Path.home() / ".esphome-integration-tests"
     cache_dir = test_cache_dir / "platformio"
 
+    # Create the temp directory that PlatformIO uses to avoid race conditions
+    # This ensures it exists and won't be deleted by parallel processes
+    platformio_tmp_dir = cache_dir / ".cache" / "tmp"
+    platformio_tmp_dir.mkdir(parents=True, exist_ok=True)
+
     # Use a lock file in the home directory to ensure only one process initializes the cache
     # This is needed when running with pytest-xdist
     # The lock file must be in a directory that already exists to avoid race conditions
@@ -83,17 +90,11 @@ def shared_platformio_cache() -> Generator[Path]:
             test_cache_dir.mkdir(exist_ok=True)
 
             with tempfile.TemporaryDirectory() as tmpdir:
-                # Create a basic host config
+                # Use the cache_init fixture for initialization
                 init_dir = Path(tmpdir)
+                fixture_path = Path(__file__).parent / "fixtures" / "cache_init.yaml"
                 config_path = init_dir / "cache_init.yaml"
-                config_path.write_text("""esphome:
-  name: cache-init
-host:
-api:
-  encryption:
-    key: "IIevImVI42I0FGos5nLqFK91jrJehrgidI0ArwMLr8w="
-logger:
-""")
+                config_path.write_text(fixture_path.read_text())
 
                 # Run compilation to populate the cache
                 # We must succeed here to avoid race conditions where multiple
@@ -105,6 +106,7 @@ logger:
                     check=True,
                     cwd=init_dir,
                     env=env,
+                    close_fds=False,
                 )
 
         # Lock is held until here, ensuring cache is fully populated before any test proceeds
@@ -183,19 +185,17 @@ async def yaml_config(request: pytest.FixtureRequest, unused_tcp_port: int) -> s
         content = content.replace("api:", f"api:\n  port: {unused_tcp_port}")
 
     # Add debug build flags for integration tests to enable assertions
-    if "esphome:" in content:
-        # Check if platformio_options already exists
-        if "platformio_options:" not in content:
-            # Add platformio_options with debug flags after esphome:
-            content = content.replace(
-                "esphome:",
-                "esphome:\n"
-                "  # Enable assertions for integration tests\n"
-                "  platformio_options:\n"
-                "    build_flags:\n"
-                '      - "-DDEBUG"  # Enable assert() statements\n'
-                '      - "-g"       # Add debug symbols',
-            )
+    if "esphome:" in content and "platformio_options:" not in content:
+        # Add platformio_options with debug flags after esphome:
+        content = content.replace(
+            "esphome:",
+            "esphome:\n"
+            "  # Enable assertions for integration tests\n"
+            "  platformio_options:\n"
+            "    build_flags:\n"
+            '      - "-DDEBUG"  # Enable assert() statements\n'
+            '      - "-g"       # Add debug symbols',
+        )
 
     return content
 
@@ -247,32 +247,32 @@ async def compile_esphome(
                 # Start in a new process group to isolate signal handling
                 start_new_session=True,
                 env=env,
+                close_fds=False,
             )
             await proc.wait()
 
             if proc.returncode == 0:
                 # Success!
                 break
-            elif proc.returncode == -11 and attempt < max_retries - 1:
+            if proc.returncode == -11 and attempt < max_retries - 1:
                 # Segfault (-11 = SIGSEGV), retry
                 print(
                     f"Compilation segfaulted (attempt {attempt + 1}/{max_retries}), retrying..."
                 )
                 await asyncio.sleep(1)  # Brief pause before retry
                 continue
-            else:
-                # Other error or final retry
-                raise RuntimeError(
-                    f"Failed to compile {config_path}, return code: {proc.returncode}. "
-                    f"Run with 'pytest -s' to see compilation output."
-                )
+            # Other error or final retry
+            raise RuntimeError(
+                f"Failed to compile {config_path}, return code: {proc.returncode}. "
+                f"Run with 'pytest -s' to see compilation output."
+            )
 
         # Load the config to get idedata (blocking call, must use executor)
         loop = asyncio.get_running_loop()
 
         def _read_config_and_get_binary():
             CORE.reset()  # Reset CORE state between test runs
-            CORE.config_path = str(config_path)
+            CORE.config_path = config_path
             config = esphome.config.read_config(
                 {"command": "compile", "config": str(config_path)}
             )
@@ -347,7 +347,8 @@ async def wait_and_connect_api_client(
     noise_psk: str | None = None,
     client_info: str = "integration-test",
     timeout: float = API_CONNECTION_TIMEOUT,
-) -> AsyncGenerator[APIClient]:
+    return_disconnect_event: bool = False,
+) -> AsyncGenerator[APIClient | tuple[APIClient, asyncio.Event]]:
     """Wait for API to be available and connect."""
     client = APIClient(
         address=address,
@@ -360,14 +361,17 @@ async def wait_and_connect_api_client(
     # Create a future to signal when connected
     loop = asyncio.get_running_loop()
     connected_future: asyncio.Future[None] = loop.create_future()
+    disconnect_event = asyncio.Event()
 
     async def on_connect() -> None:
         """Called when successfully connected."""
+        disconnect_event.clear()  # Clear the disconnect event on new connection
         if not connected_future.done():
             connected_future.set_result(None)
 
     async def on_disconnect(expected_disconnect: bool) -> None:
         """Called when disconnected."""
+        disconnect_event.set()
         if not connected_future.done() and not expected_disconnect:
             connected_future.set_exception(
                 APIConnectionError("Disconnected before fully connected")
@@ -395,10 +399,13 @@ async def wait_and_connect_api_client(
         # Wait for connection with timeout
         try:
             await asyncio.wait_for(connected_future, timeout=timeout)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             raise TimeoutError(f"Failed to connect to API after {timeout} seconds")
 
-        yield client
+        if return_disconnect_event:
+            yield client, disconnect_event
+        else:
+            yield client
     finally:
         # Stop reconnect logic and disconnect
         await reconnect_logic.stop()
@@ -431,6 +438,33 @@ async def api_client_connected(
     yield _connect_client
 
 
+@pytest_asyncio.fixture
+async def api_client_connected_with_disconnect(
+    unused_tcp_port: int,
+) -> AsyncGenerator:
+    """Factory for creating connected API client context managers with disconnect event."""
+
+    def _connect_client_with_disconnect(
+        address: str = LOCALHOST,
+        port: int | None = None,
+        password: str = "",
+        noise_psk: str | None = None,
+        client_info: str = "integration-test",
+        timeout: float = API_CONNECTION_TIMEOUT,
+    ):
+        return wait_and_connect_api_client(
+            address=address,
+            port=port if port is not None else unused_tcp_port,
+            password=password,
+            noise_psk=noise_psk,
+            client_info=client_info,
+            timeout=timeout,
+            return_disconnect_event=True,
+        )
+
+    yield _connect_client_with_disconnect
+
+
 async def _read_stream_lines(
     stream: asyncio.StreamReader,
     lines: list[str],
@@ -480,6 +514,7 @@ async def run_binary_and_wait_for_port(
         # Start in a new process group to isolate signal handling
         start_new_session=True,
         pass_fds=(device_fd,),
+        close_fds=False,
     )
 
     # Close the device end in the parent process
@@ -575,12 +610,12 @@ async def run_binary_and_wait_for_port(
             process.send_signal(signal.SIGINT)
             try:
                 await asyncio.wait_for(process.wait(), timeout=SIGINT_TIMEOUT)
-            except asyncio.TimeoutError:
+            except TimeoutError:
                 # If SIGINT didn't work, try SIGTERM
                 process.terminate()
                 try:
                     await asyncio.wait_for(process.wait(), timeout=SIGTERM_TIMEOUT)
-                except asyncio.TimeoutError:
+                except TimeoutError:
                     # Last resort: SIGKILL
                     process.kill()
                     await process.wait()
diff --git a/tests/integration/fixtures/api_homeassistant.yaml b/tests/integration/fixtures/api_homeassistant.yaml
new file mode 100644
index 0000000000..ce8628977a
--- /dev/null
+++ b/tests/integration/fixtures/api_homeassistant.yaml
@@ -0,0 +1,311 @@
+esphome:
+  name: test-ha-api
+  friendly_name: Home Assistant API Test
+
+host:
+
+api:
+  services:
+    - service: trigger_all_tests
+      then:
+        - logger.log: "=== Starting Home Assistant API Tests ==="
+        - button.press: test_basic_service
+        - button.press: test_templated_service
+        - button.press: test_empty_string_service
+        - button.press: test_multiple_fields_service
+        - button.press: test_complex_lambda_service
+        - button.press: test_all_empty_service
+        - button.press: test_rapid_service_calls
+        - button.press: test_read_ha_states
+        - number.set:
+            id: ha_number
+            value: 42.5
+        - switch.turn_on: ha_switch
+        - switch.turn_off: ha_switch
+        - logger.log: "=== All tests completed ==="
+
+logger:
+  level: DEBUG
+
+# Time component for templated values
+time:
+  - platform: homeassistant
+    id: homeassistant_time
+
+# Global variables for testing
+globals:
+  - id: test_brightness
+    type: int
+    initial_value: '75'
+  - id: test_string
+    type: std::string
+    initial_value: '"test_value"'
+
+# Sensors for testing state reading
+sensor:
+  - platform: template
+    name: "Test Sensor"
+    id: test_sensor
+    lambda: return 42.0;
+    update_interval: 0.1s
+
+  # Home Assistant sensor that reads external state
+  - platform: homeassistant
+    name: "HA Temperature"
+    entity_id: sensor.external_temperature
+    id: ha_temperature
+    on_value:
+      then:
+        - logger.log:
+            format: "HA Temperature state updated: %.1f"
+            args: ['x']
+
+  # Test multiple HA state sensors
+  - platform: homeassistant
+    name: "HA Humidity"
+    entity_id: sensor.external_humidity
+    id: ha_humidity
+    on_value:
+      then:
+        - logger.log:
+            format: "HA Humidity state updated: %.1f"
+            args: ['x']
+
+# Binary sensor from Home Assistant
+binary_sensor:
+  - platform: homeassistant
+    name: "HA Motion"
+    entity_id: binary_sensor.external_motion
+    id: ha_motion
+    on_state:
+      then:
+        - logger.log:
+            format: "HA Motion state changed: %s"
+            args: ['x ? "ON" : "OFF"']
+
+# Text sensor from Home Assistant
+text_sensor:
+  - platform: homeassistant
+    name: "HA Weather"
+    entity_id: weather.home
+    attribute: condition
+    id: ha_weather
+    on_value:
+      then:
+        - logger.log:
+            format: "HA Weather condition updated: %s"
+            args: ['x.c_str()']
+
+  # Test empty state handling
+  - platform: homeassistant
+    name: "HA Empty State"
+    entity_id: sensor.nonexistent_sensor
+    id: ha_empty_state
+    on_value:
+      then:
+        - logger.log:
+            format: "HA Empty state updated: %s"
+            args: ['x.c_str()']
+
+# Number component for testing HA number control
+number:
+  - platform: template
+    name: "HA Controlled Number"
+    id: ha_number
+    min_value: 0
+    max_value: 100
+    step: 1
+    optimistic: true
+    set_action:
+      - logger.log:
+          format: "Setting HA number to: %.1f"
+          args: ['x']
+      - homeassistant.action:
+          action: input_number.set_value
+          data:
+            entity_id: input_number.test_number
+            value: !lambda 'return to_string(x);'
+
+# Switch component for testing HA switch control
+switch:
+  - platform: template
+    name: "HA Controlled Switch"
+    id: ha_switch
+    optimistic: true
+    turn_on_action:
+      - logger.log: "Toggling HA switch: switch.test_switch ON"
+      - homeassistant.action:
+          action: switch.turn_on
+          data:
+            entity_id: switch.test_switch
+    turn_off_action:
+      - logger.log: "Toggling HA switch: switch.test_switch OFF"
+      - homeassistant.action:
+          action: switch.turn_off
+          data:
+            entity_id: switch.test_switch
+
+# Buttons for testing various service call scenarios
+button:
+  # Test 1: Basic service call with static values
+  - platform: template
+    name: "Test Basic Service"
+    id: test_basic_service
+    on_press:
+      - logger.log: "Sending HomeAssistant service call: light.turn_off"
+      - homeassistant.action:
+          action: light.turn_off
+          data:
+            entity_id: light.test_light
+      - logger.log: "Service data: entity_id=light.test_light"
+
+  # Test 2: Service call with templated/lambda values (main bug fix test)
+  - platform: template
+    name: "Test Templated Service"
+    id: test_templated_service
+    on_press:
+      - logger.log: "Testing templated service call"
+      - lambda: |-
+          int brightness_percent = id(test_brightness);
+          std::string computed = to_string(brightness_percent * 255 / 100);
+          ESP_LOGI("test", "Lambda computed value: %s", computed.c_str());
+      - homeassistant.action:
+          action: light.turn_on
+          data:
+            entity_id: light.test_light
+            # This creates a temporary string - the main test case
+            brightness: !lambda 'return to_string(id(test_brightness) * 255 / 100);'
+          data_template:
+            color_name: !lambda 'return id(test_string);'
+          variables:
+            transition: !lambda 'return "2.5";'
+
+  # Test 3: Service call with empty string values
+  - platform: template
+    name: "Test Empty String Service"
+    id: test_empty_string_service
+    on_press:
+      - logger.log: "Testing empty string values"
+      - homeassistant.action:
+          action: notify.test
+          data:
+            message: "Test message"
+            title: ""
+          data_template:
+            target: !lambda 'return "";'
+          variables:
+            sound: !lambda 'return "";'
+
+      - logger.log: "Empty value for key: title"
+      - logger.log: "Empty value for key: target"
+      - logger.log: "Empty value for key: sound"
+
+  # Test 4: Service call with multiple data fields
+  - platform: template
+    name: "Test Multiple Fields Service"
+    id: test_multiple_fields_service
+    on_press:
+      - logger.log: "Testing multiple data fields"
+      - homeassistant.action:
+          action: climate.set_temperature
+          data:
+            entity_id: climate.test_climate
+            temperature: "22"
+            hvac_mode: "heat"
+          data_template:
+            target_temp_high: !lambda 'return "24";'
+            target_temp_low: !lambda 'return "20";'
+          variables:
+            preset_mode: !lambda 'return "comfort";'
+
+  # Test 5: Complex lambda with string operations
+  - platform: template
+    name: "Test Complex Lambda Service"
+    id: test_complex_lambda_service
+    on_press:
+      - logger.log: "Testing complex lambda expressions"
+      - homeassistant.action:
+          action: script.test_script
+          data:
+            entity_id: !lambda |-
+              std::string base = "light.";
+              std::string room = "living_room";
+              return base + room;
+            brightness_pct: !lambda |-
+              float sensor_val = id(test_sensor).state;
+              int pct = (int)(sensor_val * 2.38);  // 42 * 2.38 ≈ 100
+              return to_string(pct);
+          data_template:
+            message: !lambda |-
+              char buffer[50];
+              snprintf(buffer, sizeof(buffer), "Sensor: %.1f, Time: %02d:%02d",
+                       id(test_sensor).state,
+                       id(homeassistant_time).now().hour,
+                       id(homeassistant_time).now().minute);
+              return std::string(buffer);
+
+  # Test 6: Service with only empty strings to verify size calculation
+  - platform: template
+    name: "Test All Empty Service"
+    id: test_all_empty_service
+    on_press:
+      - logger.log: "Testing all empty string values"
+      - homeassistant.action:
+          action: test.empty
+          data:
+            field1: ""
+            field2: ""
+          data_template:
+            field3: !lambda 'return "";'
+          variables:
+            field4: !lambda 'return "";'
+      - logger.log: "All empty service call completed"
+
+  # Test 7: Rapid successive service calls
+  - platform: template
+    name: "Test Rapid Service Calls"
+    id: test_rapid_service_calls
+    on_press:
+      - logger.log: "Testing rapid service calls"
+      - repeat:
+          count: 5
+          then:
+            - homeassistant.action:
+                action: counter.increment
+                data:
+                  entity_id: counter.test_counter
+            - delay: 10ms
+      - logger.log: "Rapid service calls completed"
+
+  # Test 8: Log current HA states
+  - platform: template
+    name: "Test Read HA States"
+    id: test_read_ha_states
+    on_press:
+      - logger.log: "Reading current HA states"
+      - lambda: |-
+          if (id(ha_temperature).has_state()) {
+            ESP_LOGI("test", "Current HA Temperature: %.1f", id(ha_temperature).state);
+          } else {
+            ESP_LOGI("test", "HA Temperature has no state");
+          }
+
+          if (id(ha_humidity).has_state()) {
+            ESP_LOGI("test", "Current HA Humidity: %.1f", id(ha_humidity).state);
+          } else {
+            ESP_LOGI("test", "HA Humidity has no state");
+          }
+
+          ESP_LOGI("test", "Current HA Motion: %s", id(ha_motion).state ? "ON" : "OFF");
+
+          if (id(ha_weather).has_state()) {
+            ESP_LOGI("test", "Current HA Weather: %s", id(ha_weather).state.c_str());
+          } else {
+            ESP_LOGI("test", "HA Weather has no state");
+          }
+
+          if (id(ha_empty_state).has_state()) {
+            ESP_LOGI("test", "HA Empty State value: %s", id(ha_empty_state).state.c_str());
+          } else {
+            ESP_LOGI("test", "HA Empty State has no value (expected)");
+          }
diff --git a/tests/integration/fixtures/api_string_lambda.yaml b/tests/integration/fixtures/api_string_lambda.yaml
new file mode 100644
index 0000000000..e2da4683c0
--- /dev/null
+++ b/tests/integration/fixtures/api_string_lambda.yaml
@@ -0,0 +1,87 @@
+esphome:
+  name: api-string-lambda-test
+host:
+
+api:
+  actions:
+    # Service that tests string lambda functionality
+    - action: test_string_lambda
+      variables:
+        input_string: string
+      then:
+        # Log the input to verify service was called
+        - logger.log:
+            format: "Service called with string: %s"
+            args: [input_string.c_str()]
+
+        # This is the key test - using a lambda that returns x.c_str()
+        # where x is already a string. This would fail to compile in 2025.7.0b5
+        # with "no matching function for call to 'to_string(std::string)'"
+        # This is the exact case from issue #9539
+        - homeassistant.tag_scanned: !lambda 'return input_string.c_str();'
+
+        # Also test with homeassistant.event to verify our fix works with data fields
+        - homeassistant.event:
+            event: esphome.test_string_lambda
+            data:
+              value: !lambda 'return input_string.c_str();'
+
+    # Service that tests int lambda functionality
+    - action: test_int_lambda
+      variables:
+        input_number: int
+      then:
+        # Log the input to verify service was called
+        - logger.log:
+            format: "Service called with int: %d"
+            args: [input_number]
+
+        # Test that int lambdas still work correctly with to_string
+        # The TemplatableStringValue should automatically convert int to string
+        - homeassistant.event:
+            event: esphome.test_int_lambda
+            data:
+              value: !lambda 'return input_number;'
+
+    # Service that tests float lambda functionality
+    - action: test_float_lambda
+      variables:
+        input_float: float
+      then:
+        # Log the input to verify service was called
+        - logger.log:
+            format: "Service called with float: %.2f"
+            args: [input_float]
+
+        # Test that float lambdas still work correctly with to_string
+        # The TemplatableStringValue should automatically convert float to string
+        - homeassistant.event:
+            event: esphome.test_float_lambda
+            data:
+              value: !lambda 'return input_float;'
+
+    # Service that tests char* lambda functionality (e.g., from itoa or sprintf)
+    - action: test_char_ptr_lambda
+      variables:
+        input_number: int
+        input_string: string
+      then:
+        # Log the input to verify service was called
+        - logger.log:
+            format: "Service called with number for char* test: %d"
+            args: [input_number]
+
+        # Test that char* lambdas work correctly
+        # This would fail in issue #9628 with "invalid conversion from 'char*' to 'long long unsigned int'"
+        - homeassistant.event:
+            event: esphome.test_char_ptr_lambda
+            data:
+              # Test snprintf returning char*
+              decimal_value: !lambda 'static char buffer[20]; snprintf(buffer, sizeof(buffer), "%d", input_number); return buffer;'
+              # Test strdup returning char* (dynamically allocated)
+              string_copy: !lambda 'return strdup(input_string.c_str());'
+              # Test string literal (const char*)
+              literal: !lambda 'return "test literal";'
+
+logger:
+  level: DEBUG
diff --git a/tests/integration/fixtures/areas_and_devices.yaml b/tests/integration/fixtures/areas_and_devices.yaml
index 6bf1519c79..08b02e6e1e 100644
--- a/tests/integration/fixtures/areas_and_devices.yaml
+++ b/tests/integration/fixtures/areas_and_devices.yaml
@@ -54,3 +54,71 @@ sensor:
     device_id: smart_switch_device
     lambda: return 4.0;
     update_interval: 0.1s
+
+  - platform: template
+    name: Living Room Sensor
+    device_id: ""
+    lambda: return 5.0;
+    update_interval: 0.1s
+
+# Switches with the same name on different devices to test device_id lookup
+switch:
+  # Switch with no device_id (defaults to 0)
+  - platform: template
+    name: Test Switch
+    id: test_switch_main
+    optimistic: true
+    turn_on_action:
+      - logger.log: "Turning on Test Switch on Main Device (no device_id)"
+    turn_off_action:
+      - logger.log: "Turning off Test Switch on Main Device (no device_id)"
+
+  - platform: template
+    name: Test Switch
+    device_id: light_controller_device
+    id: test_switch_light_controller
+    optimistic: true
+    turn_on_action:
+      - logger.log: "Turning on Test Switch on Light Controller"
+    turn_off_action:
+      - logger.log: "Turning off Test Switch on Light Controller"
+
+  - platform: template
+    name: Test Switch
+    device_id: temp_sensor_device
+    id: test_switch_temp_sensor
+    optimistic: true
+    turn_on_action:
+      - logger.log: "Turning on Test Switch on Temperature Sensor"
+    turn_off_action:
+      - logger.log: "Turning off Test Switch on Temperature Sensor"
+
+  - platform: template
+    name: Test Switch
+    device_id: motion_detector_device
+    id: test_switch_motion_detector
+    optimistic: true
+    turn_on_action:
+      - logger.log: "Turning on Test Switch on Motion Detector"
+    turn_off_action:
+      - logger.log: "Turning off Test Switch on Motion Detector"
+
+  - platform: template
+    name: Living Room Blank Switch
+    device_id: ""
+    id: test_switch_blank_living_room
+    optimistic: true
+    turn_on_action:
+      - logger.log: "Turning on Living Room Blank Switch"
+    turn_off_action:
+      - logger.log: "Turning off Living Room Blank Switch"
+
+  - platform: template
+    name: Living Room None Switch
+    device_id:
+    id: test_switch_none_living_room
+    optimistic: true
+    turn_on_action:
+      - logger.log: "Turning on Living Room None Switch"
+    turn_off_action:
+      - logger.log: "Turning off Living Room None Switch"
diff --git a/tests/integration/fixtures/batch_delay_zero_rapid_transitions.yaml b/tests/integration/fixtures/batch_delay_zero_rapid_transitions.yaml
index 32cacfaa79..f7b0fdcb63 100644
--- a/tests/integration/fixtures/batch_delay_zero_rapid_transitions.yaml
+++ b/tests/integration/fixtures/batch_delay_zero_rapid_transitions.yaml
@@ -34,10 +34,9 @@ binary_sensor:
           ESP_LOGD("test", "Button ON at %u", now);
         }
         return true;
-      } else {
-        // Only log state change
-        if (id(ir_remote_button).state) {
-          ESP_LOGD("test", "Button OFF at %u", now);
-        }
-        return false;
       }
+      // Only log state change
+      if (id(ir_remote_button).state) {
+        ESP_LOGD("test", "Button OFF at %u", now);
+      }
+      return false;
diff --git a/tests/integration/fixtures/cache_init.yaml b/tests/integration/fixtures/cache_init.yaml
new file mode 100644
index 0000000000..de208196cd
--- /dev/null
+++ b/tests/integration/fixtures/cache_init.yaml
@@ -0,0 +1,10 @@
+esphome:
+  name: cache-init
+
+host:
+
+api:
+  encryption:
+    key: "IIevImVI42I0FGos5nLqFK91jrJehrgidI0ArwMLr8w="
+
+logger:
diff --git a/tests/integration/fixtures/crc8_helper.yaml b/tests/integration/fixtures/crc8_helper.yaml
new file mode 100644
index 0000000000..e97e23eab0
--- /dev/null
+++ b/tests/integration/fixtures/crc8_helper.yaml
@@ -0,0 +1,17 @@
+esphome:
+  name: crc8-helper-test
+
+host:
+
+api:
+
+logger:
+  level: INFO
+
+external_components:
+  - source:
+      type: local
+      path: EXTERNAL_COMPONENT_PATH
+    components: [crc8_test_component]
+
+crc8_test_component:
diff --git a/tests/integration/fixtures/delay_action_cancellation.yaml b/tests/integration/fixtures/delay_action_cancellation.yaml
new file mode 100644
index 0000000000..e0dd427c2d
--- /dev/null
+++ b/tests/integration/fixtures/delay_action_cancellation.yaml
@@ -0,0 +1,24 @@
+esphome:
+  name: test-delay-action
+
+host:
+api:
+  actions:
+    - action: start_delay_then_restart
+      then:
+        - logger.log: "Starting first script execution"
+        - script.execute: test_delay_script
+        - delay: 250ms  # Give first script time to start delay
+        - logger.log: "Restarting script (should cancel first delay)"
+        - script.execute: test_delay_script
+
+logger:
+  level: DEBUG
+
+script:
+  - id: test_delay_script
+    mode: restart
+    then:
+      - logger.log: "Script started, beginning delay"
+      - delay: 500ms  # Long enough that it won't complete before restart
+      - logger.log: "Delay completed successfully"
diff --git a/tests/integration/fixtures/duplicate_entities_not_allowed_on_different_devices.yaml b/tests/integration/fixtures/duplicate_entities_on_different_devices.yaml
similarity index 61%
rename from tests/integration/fixtures/duplicate_entities_not_allowed_on_different_devices.yaml
rename to tests/integration/fixtures/duplicate_entities_on_different_devices.yaml
index f7d017a0ae..ecc502ad28 100644
--- a/tests/integration/fixtures/duplicate_entities_not_allowed_on_different_devices.yaml
+++ b/tests/integration/fixtures/duplicate_entities_on_different_devices.yaml
@@ -1,6 +1,6 @@
 esphome:
   name: duplicate-entities-test
-  # Define devices to test multi-device unique name validation
+  # Define devices to test multi-device duplicate handling
   devices:
     - id: controller_1
       name: Controller 1
@@ -13,31 +13,31 @@ host:
 api:  # Port will be automatically injected
 logger:
 
-# Test that duplicate entity names are NOT allowed on different devices
+# Test that duplicate entity names are allowed on different devices
 
-# Scenario 1: Different sensor names on different devices (allowed)
+# Scenario 1: Same sensor name on different devices (allowed)
 sensor:
   - platform: template
-    name: Temperature Controller 1
+    name: Temperature
     device_id: controller_1
     lambda: return 21.0;
     update_interval: 0.1s
 
   - platform: template
-    name: Temperature Controller 2
+    name: Temperature
     device_id: controller_2
     lambda: return 22.0;
     update_interval: 0.1s
 
   - platform: template
-    name: Temperature Controller 3
+    name: Temperature
     device_id: controller_3
     lambda: return 23.0;
     update_interval: 0.1s
 
   # Main device sensor (no device_id)
   - platform: template
-    name: Temperature Main
+    name: Temperature
     lambda: return 20.0;
     update_interval: 0.1s
 
@@ -47,20 +47,20 @@ sensor:
     lambda: return 60.0;
     update_interval: 0.1s
 
-# Scenario 2: Different binary sensor names on different devices
+# Scenario 2: Same binary sensor name on different devices (allowed)
 binary_sensor:
   - platform: template
-    name: Status Controller 1
+    name: Status
     device_id: controller_1
     lambda: return true;
 
   - platform: template
-    name: Status Controller 2
+    name: Status
     device_id: controller_2
     lambda: return false;
 
   - platform: template
-    name: Status Main
+    name: Status
     lambda: return true;  # Main device
 
   # Different platform can have same name as sensor
@@ -68,43 +68,43 @@ binary_sensor:
     name: Temperature
     lambda: return true;
 
-# Scenario 3: Different text sensor names on different devices
+# Scenario 3: Same text sensor name on different devices
 text_sensor:
   - platform: template
-    name: Device Info Controller 1
+    name: Device Info
     device_id: controller_1
     lambda: return {"Controller 1 Active"};
     update_interval: 0.1s
 
   - platform: template
-    name: Device Info Controller 2
+    name: Device Info
     device_id: controller_2
     lambda: return {"Controller 2 Active"};
     update_interval: 0.1s
 
   - platform: template
-    name: Device Info Main
+    name: Device Info
     lambda: return {"Main Device Active"};
     update_interval: 0.1s
 
-# Scenario 4: Different switch names on different devices
+# Scenario 4: Same switch name on different devices
 switch:
   - platform: template
-    name: Power Controller 1
+    name: Power
     device_id: controller_1
     lambda: return false;
     turn_on_action: []
     turn_off_action: []
 
   - platform: template
-    name: Power Controller 2
+    name: Power
     device_id: controller_2
     lambda: return true;
     turn_on_action: []
     turn_off_action: []
 
   - platform: template
-    name: Power Controller 3
+    name: Power
     device_id: controller_3
     lambda: return false;
     turn_on_action: []
@@ -117,54 +117,26 @@ switch:
     turn_on_action: []
     turn_off_action: []
 
-# Scenario 5: Buttons with unique names
+# Scenario 5: Empty names on different devices (should use device name)
 button:
   - platform: template
-    name: "Reset Controller 1"
+    name: ""
     device_id: controller_1
     on_press: []
 
   - platform: template
-    name: "Reset Controller 2"
+    name: ""
     device_id: controller_2
     on_press: []
 
   - platform: template
-    name: "Reset Main"
+    name: ""
     on_press: []  # Main device
 
-# Scenario 6: Empty names (should use device names)
-select:
-  - platform: template
-    name: ""
-    device_id: controller_1
-    options:
-      - "Option 1"
-      - "Option 2"
-    lambda: return {"Option 1"};
-    set_action: []
-
-  - platform: template
-    name: ""
-    device_id: controller_2
-    options:
-      - "Option 1"
-      - "Option 2"
-    lambda: return {"Option 1"};
-    set_action: []
-
-  - platform: template
-    name: ""  # Main device
-    options:
-      - "Option 1"
-      - "Option 2"
-    lambda: return {"Option 1"};
-    set_action: []
-
-# Scenario 7: Special characters in names - now with unique names
+# Scenario 6: Special characters in names
 number:
   - platform: template
-    name: "Temperature Setpoint! Controller 1"
+    name: "Temperature Setpoint!"
     device_id: controller_1
     min_value: 10.0
     max_value: 30.0
@@ -173,7 +145,7 @@ number:
     set_action: []
 
   - platform: template
-    name: "Temperature Setpoint! Controller 2"
+    name: "Temperature Setpoint!"
     device_id: controller_2
     min_value: 10.0
     max_value: 30.0
diff --git a/tests/integration/fixtures/external_components/crc8_test_component/__init__.py b/tests/integration/fixtures/external_components/crc8_test_component/__init__.py
new file mode 100644
index 0000000000..6032b0861f
--- /dev/null
+++ b/tests/integration/fixtures/external_components/crc8_test_component/__init__.py
@@ -0,0 +1,17 @@
+import esphome.codegen as cg
+import esphome.config_validation as cv
+from esphome.const import CONF_ID
+
+crc8_test_component_ns = cg.esphome_ns.namespace("crc8_test_component")
+CRC8TestComponent = crc8_test_component_ns.class_("CRC8TestComponent", cg.Component)
+
+CONFIG_SCHEMA = cv.Schema(
+    {
+        cv.GenerateID(): cv.declare_id(CRC8TestComponent),
+    }
+).extend(cv.COMPONENT_SCHEMA)
+
+
+async def to_code(config):
+    var = cg.new_Pvariable(config[CONF_ID])
+    await cg.register_component(var, config)
diff --git a/tests/integration/fixtures/external_components/crc8_test_component/crc8_test_component.cpp b/tests/integration/fixtures/external_components/crc8_test_component/crc8_test_component.cpp
new file mode 100644
index 0000000000..6c46af19fd
--- /dev/null
+++ b/tests/integration/fixtures/external_components/crc8_test_component/crc8_test_component.cpp
@@ -0,0 +1,170 @@
+#include "crc8_test_component.h"
+#include "esphome/core/helpers.h"
+#include "esphome/core/log.h"
+
+namespace esphome {
+namespace crc8_test_component {
+
+static const char *const TAG = "crc8_test";
+
+void CRC8TestComponent::setup() {
+  ESP_LOGI(TAG, "CRC8 Helper Function Integration Test Starting");
+
+  // Run all test suites
+  test_crc8_dallas_maxim();
+  test_crc8_sensirion_style();
+  test_crc8_pec_style();
+  test_crc8_parameter_equivalence();
+  test_crc8_edge_cases();
+  test_component_compatibility();
+
+  ESP_LOGI(TAG, "CRC8 Integration Test Complete");
+}
+
+void CRC8TestComponent::test_crc8_dallas_maxim() {
+  ESP_LOGI(TAG, "Testing Dallas/Maxim CRC8 (default parameters)");
+
+  // Test vectors for Dallas/Maxim CRC8 (polynomial 0x8C, LSB-first, init 0x00)
+  const uint8_t test1[] = {0x01};
+  const uint8_t test2[] = {0xFF};
+  const uint8_t test3[] = {0x12, 0x34};
+  const uint8_t test4[] = {0xAA, 0xBB, 0xCC};
+  const uint8_t test5[] = {0x01, 0x02, 0x03, 0x04, 0x05};
+
+  bool all_passed = true;
+  all_passed &= verify_crc8("Dallas [0x01]", test1, sizeof(test1), 0x5E);
+  all_passed &= verify_crc8("Dallas [0xFF]", test2, sizeof(test2), 0x35);
+  all_passed &= verify_crc8("Dallas [0x12, 0x34]", test3, sizeof(test3), 0xA2);
+  all_passed &= verify_crc8("Dallas [0xAA, 0xBB, 0xCC]", test4, sizeof(test4), 0xD4);
+  all_passed &= verify_crc8("Dallas [0x01...0x05]", test5, sizeof(test5), 0x2A);
+
+  log_test_result("Dallas/Maxim CRC8", all_passed);
+}
+
+void CRC8TestComponent::test_crc8_sensirion_style() {
+  ESP_LOGI(TAG, "Testing Sensirion CRC8 (0x31 poly, MSB-first, init 0xFF)");
+
+  const uint8_t test1[] = {0x00};
+  const uint8_t test2[] = {0x01};
+  const uint8_t test3[] = {0xFF};
+  const uint8_t test4[] = {0x12, 0x34};
+  const uint8_t test5[] = {0xBE, 0xEF};
+
+  bool all_passed = true;
+  all_passed &= verify_crc8("Sensirion [0x00]", test1, sizeof(test1), 0xAC, 0xFF, 0x31, true);
+  all_passed &= verify_crc8("Sensirion [0x01]", test2, sizeof(test2), 0x9D, 0xFF, 0x31, true);
+  all_passed &= verify_crc8("Sensirion [0xFF]", test3, sizeof(test3), 0x00, 0xFF, 0x31, true);
+  all_passed &= verify_crc8("Sensirion [0x12, 0x34]", test4, sizeof(test4), 0x37, 0xFF, 0x31, true);
+  all_passed &= verify_crc8("Sensirion [0xBE, 0xEF]", test5, sizeof(test5), 0x92, 0xFF, 0x31, true);
+
+  log_test_result("Sensirion CRC8", all_passed);
+}
+
+void CRC8TestComponent::test_crc8_pec_style() {
+  ESP_LOGI(TAG, "Testing PEC CRC8 (0x07 poly, MSB-first, init 0x00)");
+
+  const uint8_t test1[] = {0x00};
+  const uint8_t test2[] = {0x01};
+  const uint8_t test3[] = {0xFF};
+  const uint8_t test4[] = {0x12, 0x34};
+  const uint8_t test5[] = {0xAA, 0xBB};
+
+  bool all_passed = true;
+  all_passed &= verify_crc8("PEC [0x00]", test1, sizeof(test1), 0x00, 0x00, 0x07, true);
+  all_passed &= verify_crc8("PEC [0x01]", test2, sizeof(test2), 0x07, 0x00, 0x07, true);
+  all_passed &= verify_crc8("PEC [0xFF]", test3, sizeof(test3), 0xF3, 0x00, 0x07, true);
+  all_passed &= verify_crc8("PEC [0x12, 0x34]", test4, sizeof(test4), 0xF1, 0x00, 0x07, true);
+  all_passed &= verify_crc8("PEC [0xAA, 0xBB]", test5, sizeof(test5), 0xB2, 0x00, 0x07, true);
+
+  log_test_result("PEC CRC8", all_passed);
+}
+
+void CRC8TestComponent::test_crc8_parameter_equivalence() {
+  ESP_LOGI(TAG, "Testing parameter equivalence");
+
+  const uint8_t test_data[] = {0x12, 0x34, 0x56, 0x78};
+
+  // Test that default parameters work as expected
+  uint8_t default_result = crc8(test_data, sizeof(test_data));
+  uint8_t explicit_result = crc8(test_data, sizeof(test_data), 0x00, 0x8C, false);
+
+  bool passed = (default_result == explicit_result);
+  if (!passed) {
+    ESP_LOGE(TAG, "Parameter equivalence FAILED: default=0x%02X, explicit=0x%02X", default_result, explicit_result);
+  }
+
+  log_test_result("Parameter equivalence", passed);
+}
+
+void CRC8TestComponent::test_crc8_edge_cases() {
+  ESP_LOGI(TAG, "Testing edge cases");
+
+  bool all_passed = true;
+
+  // Empty array test
+  const uint8_t empty[] = {};
+  uint8_t empty_result = crc8(empty, 0);
+  bool empty_passed = (empty_result == 0x00);  // Should return init value
+  if (!empty_passed) {
+    ESP_LOGE(TAG, "Empty array test FAILED: expected 0x00, got 0x%02X", empty_result);
+  }
+  all_passed &= empty_passed;
+
+  // Single byte tests
+  const uint8_t single_zero[] = {0x00};
+  const uint8_t single_ff[] = {0xFF};
+  all_passed &= verify_crc8("Single [0x00]", single_zero, 1, 0x00);
+  all_passed &= verify_crc8("Single [0xFF]", single_ff, 1, 0x35);
+
+  log_test_result("Edge cases", all_passed);
+}
+
+void CRC8TestComponent::test_component_compatibility() {
+  ESP_LOGI(TAG, "Testing component compatibility");
+
+  // Test specific component use cases
+  bool all_passed = true;
+
+  // AGS10-style data (Sensirion CRC8)
+  const uint8_t ags10_data[] = {0x12, 0x34, 0x56};
+  uint8_t ags10_result = crc8(ags10_data, sizeof(ags10_data), 0xFF, 0x31, true);
+  ESP_LOGI(TAG, "AGS10-style CRC8: 0x%02X", ags10_result);
+
+  // LC709203F-style data (PEC CRC8)
+  const uint8_t lc_data[] = {0xAA, 0xBB};
+  uint8_t lc_result = crc8(lc_data, sizeof(lc_data), 0x00, 0x07, true);
+  ESP_LOGI(TAG, "LC709203F-style CRC8: 0x%02X", lc_result);
+
+  // DallasTemperature-style data (Dallas CRC8)
+  const uint8_t dallas_data[] = {0x28, 0x12, 0x34, 0x56, 0x78, 0x9A, 0xBC};
+  uint8_t dallas_result = crc8(dallas_data, sizeof(dallas_data));
+  ESP_LOGI(TAG, "Dallas-style CRC8: 0x%02X", dallas_result);
+
+  all_passed = true;  // These are just demonstration tests
+  log_test_result("Component compatibility", all_passed);
+}
+
+bool CRC8TestComponent::verify_crc8(const char *test_name, const uint8_t *data, uint8_t len, uint8_t expected,
+                                    uint8_t crc, uint8_t poly, bool msb_first) {
+  uint8_t result = esphome::crc8(data, len, crc, poly, msb_first);
+  bool passed = (result == expected);
+
+  if (passed) {
+    ESP_LOGI(TAG, "%s: PASS (0x%02X)", test_name, result);
+  } else {
+    ESP_LOGE(TAG, "%s: FAIL - expected 0x%02X, got 0x%02X", test_name, expected, result);
+  }
+
+  return passed;
+}
+
+void CRC8TestComponent::log_test_result(const char *test_name, bool passed) {
+  if (passed) {
+    ESP_LOGI(TAG, "%s: ALL TESTS PASSED", test_name);
+  } else {
+    ESP_LOGE(TAG, "%s: SOME TESTS FAILED", test_name);
+  }
+}
+
+}  // namespace crc8_test_component
+}  // namespace esphome
diff --git a/tests/integration/fixtures/external_components/crc8_test_component/crc8_test_component.h b/tests/integration/fixtures/external_components/crc8_test_component/crc8_test_component.h
new file mode 100644
index 0000000000..3b8847259c
--- /dev/null
+++ b/tests/integration/fixtures/external_components/crc8_test_component/crc8_test_component.h
@@ -0,0 +1,29 @@
+#pragma once
+
+#include "esphome/core/component.h"
+#include "esphome/core/helpers.h"
+#include "esphome/core/log.h"
+
+namespace esphome {
+namespace crc8_test_component {
+
+class CRC8TestComponent : public Component {
+ public:
+  void setup() override;
+
+ private:
+  void test_crc8_dallas_maxim();
+  void test_crc8_sensirion_style();
+  void test_crc8_pec_style();
+  void test_crc8_parameter_equivalence();
+  void test_crc8_edge_cases();
+  void test_component_compatibility();
+  void test_old_vs_new_implementations();
+
+  void log_test_result(const char *test_name, bool passed);
+  bool verify_crc8(const char *test_name, const uint8_t *data, uint8_t len, uint8_t expected, uint8_t crc = 0x00,
+                   uint8_t poly = 0x8C, bool msb_first = false);
+};
+
+}  // namespace crc8_test_component
+}  // namespace esphome
diff --git a/tests/integration/fixtures/external_components/gpio_expander_test_component/__init__.py b/tests/integration/fixtures/external_components/gpio_expander_test_component/__init__.py
new file mode 100644
index 0000000000..5672f80004
--- /dev/null
+++ b/tests/integration/fixtures/external_components/gpio_expander_test_component/__init__.py
@@ -0,0 +1,25 @@
+import esphome.codegen as cg
+import esphome.config_validation as cv
+from esphome.const import CONF_ID
+
+AUTO_LOAD = ["gpio_expander"]
+
+gpio_expander_test_component_ns = cg.esphome_ns.namespace(
+    "gpio_expander_test_component"
+)
+
+GPIOExpanderTestComponent = gpio_expander_test_component_ns.class_(
+    "GPIOExpanderTestComponent", cg.Component
+)
+
+
+CONFIG_SCHEMA = cv.Schema(
+    {
+        cv.GenerateID(): cv.declare_id(GPIOExpanderTestComponent),
+    }
+).extend(cv.COMPONENT_SCHEMA)
+
+
+async def to_code(config):
+    var = cg.new_Pvariable(config[CONF_ID])
+    await cg.register_component(var, config)
diff --git a/tests/integration/fixtures/external_components/gpio_expander_test_component/gpio_expander_test_component.cpp b/tests/integration/fixtures/external_components/gpio_expander_test_component/gpio_expander_test_component.cpp
new file mode 100644
index 0000000000..6e128687c4
--- /dev/null
+++ b/tests/integration/fixtures/external_components/gpio_expander_test_component/gpio_expander_test_component.cpp
@@ -0,0 +1,40 @@
+#include "gpio_expander_test_component.h"
+
+#include "esphome/core/application.h"
+#include "esphome/core/log.h"
+
+namespace esphome::gpio_expander_test_component {
+
+static const char *const TAG = "gpio_expander_test";
+
+void GPIOExpanderTestComponent::setup() {
+  for (uint8_t pin = 0; pin < 32; pin++) {
+    this->digital_read(pin);
+  }
+
+  this->digital_read(3);
+  this->digital_read(3);
+  this->digital_read(4);
+  this->digital_read(3);
+  this->digital_read(10);
+  this->reset_pin_cache_();  // Reset cache to ensure next read is from hardware
+  this->digital_read(15);
+  this->digital_read(14);
+  this->digital_read(14);
+
+  ESP_LOGD(TAG, "DONE");
+}
+
+bool GPIOExpanderTestComponent::digital_read_hw(uint8_t pin) {
+  ESP_LOGD(TAG, "digital_read_hw pin=%d", pin);
+  // Return true to indicate successful read operation
+  return true;
+}
+
+bool GPIOExpanderTestComponent::digital_read_cache(uint8_t pin) {
+  ESP_LOGD(TAG, "digital_read_cache pin=%d", pin);
+  // Return the pin state (always HIGH for testing)
+  return true;
+}
+
+}  // namespace esphome::gpio_expander_test_component
diff --git a/tests/integration/fixtures/external_components/gpio_expander_test_component/gpio_expander_test_component.h b/tests/integration/fixtures/external_components/gpio_expander_test_component/gpio_expander_test_component.h
new file mode 100644
index 0000000000..ffaee2cd65
--- /dev/null
+++ b/tests/integration/fixtures/external_components/gpio_expander_test_component/gpio_expander_test_component.h
@@ -0,0 +1,18 @@
+#pragma once
+
+#include "esphome/components/gpio_expander/cached_gpio.h"
+#include "esphome/core/component.h"
+
+namespace esphome::gpio_expander_test_component {
+
+class GPIOExpanderTestComponent : public Component, public esphome::gpio_expander::CachedGpioExpander {
+ public:
+  void setup() override;
+
+ protected:
+  bool digital_read_hw(uint8_t pin) override;
+  bool digital_read_cache(uint8_t pin) override;
+  void digital_write_hw(uint8_t pin, bool value) override{};
+};
+
+}  // namespace esphome::gpio_expander_test_component
diff --git a/tests/integration/fixtures/external_components/gpio_expander_test_component_uint16/__init__.py b/tests/integration/fixtures/external_components/gpio_expander_test_component_uint16/__init__.py
new file mode 100644
index 0000000000..76f20b942c
--- /dev/null
+++ b/tests/integration/fixtures/external_components/gpio_expander_test_component_uint16/__init__.py
@@ -0,0 +1,24 @@
+import esphome.codegen as cg
+import esphome.config_validation as cv
+from esphome.const import CONF_ID
+
+AUTO_LOAD = ["gpio_expander"]
+
+gpio_expander_test_component_uint16_ns = cg.esphome_ns.namespace(
+    "gpio_expander_test_component_uint16"
+)
+
+GPIOExpanderTestUint16Component = gpio_expander_test_component_uint16_ns.class_(
+    "GPIOExpanderTestUint16Component", cg.Component
+)
+
+CONFIG_SCHEMA = cv.Schema(
+    {
+        cv.GenerateID(): cv.declare_id(GPIOExpanderTestUint16Component),
+    }
+).extend(cv.COMPONENT_SCHEMA)
+
+
+async def to_code(config):
+    var = cg.new_Pvariable(config[CONF_ID])
+    await cg.register_component(var, config)
diff --git a/tests/integration/fixtures/external_components/gpio_expander_test_component_uint16/gpio_expander_test_component_uint16.cpp b/tests/integration/fixtures/external_components/gpio_expander_test_component_uint16/gpio_expander_test_component_uint16.cpp
new file mode 100644
index 0000000000..09537c81bb
--- /dev/null
+++ b/tests/integration/fixtures/external_components/gpio_expander_test_component_uint16/gpio_expander_test_component_uint16.cpp
@@ -0,0 +1,43 @@
+#include "gpio_expander_test_component_uint16.h"
+#include "esphome/core/log.h"
+
+namespace esphome::gpio_expander_test_component_uint16 {
+
+static const char *const TAG = "gpio_expander_test_uint16";
+
+void GPIOExpanderTestUint16Component::setup() {
+  ESP_LOGD(TAG, "Testing uint16_t bank (single 16-pin bank)");
+
+  // Test reading all 16 pins - first should trigger hw read, rest use cache
+  for (uint8_t pin = 0; pin < 16; pin++) {
+    this->digital_read(pin);
+  }
+
+  // Reset cache and test specific reads
+  ESP_LOGD(TAG, "Resetting cache for uint16_t test");
+  this->reset_pin_cache_();
+
+  // First read triggers hw for entire bank
+  this->digital_read(5);
+  // These should all use cache since they're in the same bank
+  this->digital_read(10);
+  this->digital_read(15);
+  this->digital_read(0);
+
+  ESP_LOGD(TAG, "DONE_UINT16");
+}
+
+bool GPIOExpanderTestUint16Component::digital_read_hw(uint8_t pin) {
+  ESP_LOGD(TAG, "uint16_digital_read_hw pin=%d", pin);
+  // In a real component, this would read from I2C/SPI into internal state
+  // For testing, we just return true to indicate successful read
+  return true;  // Return true to indicate successful read
+}
+
+bool GPIOExpanderTestUint16Component::digital_read_cache(uint8_t pin) {
+  ESP_LOGD(TAG, "uint16_digital_read_cache pin=%d", pin);
+  // Return the actual pin state from our test pattern
+  return (this->test_state_ >> pin) & 1;
+}
+
+}  // namespace esphome::gpio_expander_test_component_uint16
diff --git a/tests/integration/fixtures/external_components/gpio_expander_test_component_uint16/gpio_expander_test_component_uint16.h b/tests/integration/fixtures/external_components/gpio_expander_test_component_uint16/gpio_expander_test_component_uint16.h
new file mode 100644
index 0000000000..be102f9b57
--- /dev/null
+++ b/tests/integration/fixtures/external_components/gpio_expander_test_component_uint16/gpio_expander_test_component_uint16.h
@@ -0,0 +1,23 @@
+#pragma once
+
+#include "esphome/components/gpio_expander/cached_gpio.h"
+#include "esphome/core/component.h"
+
+namespace esphome::gpio_expander_test_component_uint16 {
+
+// Test component using uint16_t bank type (single 16-pin bank)
+class GPIOExpanderTestUint16Component : public Component,
+                                        public esphome::gpio_expander::CachedGpioExpander {
+ public:
+  void setup() override;
+
+ protected:
+  bool digital_read_hw(uint8_t pin) override;
+  bool digital_read_cache(uint8_t pin) override;
+  void digital_write_hw(uint8_t pin, bool value) override{};
+
+ private:
+  uint16_t test_state_{0xAAAA};  // Test pattern: alternating bits
+};
+
+}  // namespace esphome::gpio_expander_test_component_uint16
diff --git a/tests/integration/fixtures/external_components/loop_test_component/__init__.py b/tests/integration/fixtures/external_components/loop_test_component/__init__.py
index 3f3a40db09..a0b0f8c65a 100644
--- a/tests/integration/fixtures/external_components/loop_test_component/__init__.py
+++ b/tests/integration/fixtures/external_components/loop_test_component/__init__.py
@@ -72,8 +72,7 @@ DisableAction = loop_test_component_ns.class_("DisableAction", automation.Action
 )
 async def enable_to_code(config, action_id, template_arg, args):
     parent = await cg.get_variable(config[CONF_ID])
-    var = cg.new_Pvariable(action_id, template_arg, parent)
-    return var
+    return cg.new_Pvariable(action_id, template_arg, parent)
 
 
 @automation.register_action(
@@ -87,8 +86,7 @@ async def enable_to_code(config, action_id, template_arg, args):
 )
 async def disable_to_code(config, action_id, template_arg, args):
     parent = await cg.get_variable(config[CONF_ID])
-    var = cg.new_Pvariable(action_id, template_arg, parent)
-    return var
+    return cg.new_Pvariable(action_id, template_arg, parent)
 
 
 async def to_code(config):
diff --git a/tests/integration/fixtures/gpio_expander_cache.yaml b/tests/integration/fixtures/gpio_expander_cache.yaml
new file mode 100644
index 0000000000..8b5375af4c
--- /dev/null
+++ b/tests/integration/fixtures/gpio_expander_cache.yaml
@@ -0,0 +1,21 @@
+esphome:
+  name: gpio-expander-cache
+host:
+
+logger:
+  level: DEBUG
+
+api:
+
+# External component that uses gpio_expander::CachedGpioExpander
+external_components:
+  - source:
+      type: local
+      path: EXTERNAL_COMPONENT_PATH
+    components: [gpio_expander_test_component, gpio_expander_test_component_uint16]
+
+# Test with uint8_t (multiple banks)
+gpio_expander_test_component:
+
+# Test with uint16_t (single bank)
+gpio_expander_test_component_uint16:
diff --git a/tests/integration/fixtures/host_mode_api_password.yaml b/tests/integration/fixtures/host_mode_api_password.yaml
new file mode 100644
index 0000000000..038b6871e0
--- /dev/null
+++ b/tests/integration/fixtures/host_mode_api_password.yaml
@@ -0,0 +1,14 @@
+esphome:
+  name: host-mode-api-password
+host:
+api:
+  password: "test_password_123"
+logger:
+  level: DEBUG
+# Test sensor to verify connection works
+sensor:
+  - platform: template
+    name: Test Sensor
+    id: test_sensor
+    lambda: return 42.0;
+    update_interval: 0.1s
diff --git a/tests/integration/fixtures/host_mode_many_entities.yaml b/tests/integration/fixtures/host_mode_many_entities.yaml
index 3d1aa36196..612186507c 100644
--- a/tests/integration/fixtures/host_mode_many_entities.yaml
+++ b/tests/integration/fixtures/host_mode_many_entities.yaml
@@ -210,6 +210,15 @@ sensor:
     name: "Test Sensor 50"
     lambda: return 50.0;
     update_interval: 0.1s
+  # Temperature sensor for the thermostat
+  - platform: template
+    name: "Temperature Sensor"
+    id: temp_sensor
+    lambda: return 22.5;
+    unit_of_measurement: "°C"
+    device_class: temperature
+    state_class: measurement
+    update_interval: 5s
 
 # Mixed entity types for comprehensive batching test
 binary_sensor:
@@ -285,6 +294,50 @@ valve:
     stop_action:
       - logger.log: "Valve stopping"
 
+output:
+  - platform: template
+    id: heater_output
+    type: binary
+    write_action:
+      - logger.log: "Heater output changed"
+  - platform: template
+    id: cooler_output
+    type: binary
+    write_action:
+      - logger.log: "Cooler output changed"
+
+climate:
+  - platform: thermostat
+    name: "Test Thermostat"
+    sensor: temp_sensor
+    default_preset: Home
+    on_boot_restore_from: default_preset
+    min_heating_off_time: 1s
+    min_heating_run_time: 1s
+    min_cooling_off_time: 1s
+    min_cooling_run_time: 1s
+    min_idle_time: 1s
+    heat_action:
+      - output.turn_on: heater_output
+    cool_action:
+      - output.turn_on: cooler_output
+    idle_action:
+      - output.turn_off: heater_output
+      - output.turn_off: cooler_output
+    preset:
+      - name: Home
+        default_target_temperature_low: 20
+        default_target_temperature_high: 24
+        mode: heat_cool
+      - name: Away
+        default_target_temperature_low: 16
+        default_target_temperature_high: 26
+        mode: heat_cool
+      - name: Sleep
+        default_target_temperature_low: 18
+        default_target_temperature_high: 22
+        mode: heat_cool
+
 alarm_control_panel:
   - platform: template
     name: "Test Alarm"
@@ -320,3 +373,20 @@ button:
     name: "Test Button"
     on_press:
       - logger.log: "Button pressed"
+
+# Date, Time, and DateTime entities
+datetime:
+  - platform: template
+    type: date
+    name: "Test Date"
+    initial_value: "2023-05-13"
+    optimistic: true
+  - platform: template
+    type: time
+    name: "Test Time"
+    initial_value: "12:30:00"
+    optimistic: true
+  - platform: template
+    type: datetime
+    name: "Test DateTime"
+    optimistic: true
diff --git a/tests/integration/fixtures/host_preferences_save_load.yaml b/tests/integration/fixtures/host_preferences_save_load.yaml
new file mode 100644
index 0000000000..929c5f7ff0
--- /dev/null
+++ b/tests/integration/fixtures/host_preferences_save_load.yaml
@@ -0,0 +1,110 @@
+esphome:
+  name: test_device
+  on_boot:
+    - lambda: |-
+        ESP_LOGD("test", "Host preferences test starting");
+
+host:
+
+logger:
+  level: DEBUG
+
+api:
+
+preferences:
+  flash_write_interval: 0s  # Disable automatic saving for test control
+
+switch:
+  - platform: template
+    name: "Test Switch"
+    id: test_switch
+    optimistic: true
+    restore_mode: DISABLED  # Don't auto-restore for test control
+
+number:
+  - platform: template
+    name: "Test Number"
+    id: test_number
+    min_value: 0
+    max_value: 100
+    step: 0.1
+    optimistic: true
+    restore_value: false  # Don't auto-restore for test control
+
+button:
+  - platform: template
+    name: "Save Preferences"
+    on_press:
+      - lambda: |-
+          // Save current values to preferences
+          ESPPreferenceObject switch_pref = global_preferences->make_preference(0x1234);
+          ESPPreferenceObject number_pref = global_preferences->make_preference(0x5678);
+
+          bool switch_value = id(test_switch).state;
+          float number_value = id(test_number).state;
+
+          if (switch_pref.save(&switch_value)) {
+            ESP_LOGI("test", "Preference saved: key=switch, value=%.1f", switch_value ? 1.0 : 0.0);
+          }
+          if (number_pref.save(&number_value)) {
+            ESP_LOGI("test", "Preference saved: key=number, value=%.1f", number_value);
+          }
+
+          // Force sync to disk
+          global_preferences->sync();
+
+  - platform: template
+    name: "Load Preferences"
+    on_press:
+      - lambda: |-
+          // Load values from preferences
+          ESPPreferenceObject switch_pref = global_preferences->make_preference(0x1234);
+          ESPPreferenceObject number_pref = global_preferences->make_preference(0x5678);
+
+          // Also try to load non-existent preferences (tests our fix)
+          ESPPreferenceObject fake_pref1 = global_preferences->make_preference(0x9999);
+          ESPPreferenceObject fake_pref2 = global_preferences->make_preference(0xAAAA);
+
+          bool switch_value = false;
+          float number_value = 0.0;
+          uint32_t fake_value = 0;
+          int loaded_count = 0;
+
+          // These should not exist and shouldn't create map entries
+          fake_pref1.load(&fake_value);
+          fake_pref2.load(&fake_value);
+
+          if (switch_pref.load(&switch_value)) {
+            id(test_switch).publish_state(switch_value);
+            ESP_LOGI("test", "Preference loaded: key=switch, value=%.1f", switch_value ? 1.0 : 0.0);
+            loaded_count++;
+          } else {
+            ESP_LOGW("test", "Failed to load switch preference");
+          }
+
+          if (number_pref.load(&number_value)) {
+            id(test_number).publish_state(number_value);
+            ESP_LOGI("test", "Preference loaded: key=number, value=%.1f", number_value);
+            loaded_count++;
+          } else {
+            ESP_LOGW("test", "Failed to load number preference");
+          }
+
+          // Log completion message for the test to detect
+          ESP_LOGI("test", "Final load test: loaded %d preferences successfully", loaded_count);
+
+  - platform: template
+    name: "Verify Preferences"
+    on_press:
+      - lambda: |-
+          // Verify current values match what we expect
+          bool switch_value = id(test_switch).state;
+          float number_value = id(test_number).state;
+
+          // After loading, switch should be true (1.0) and number should be 42.5
+          if (switch_value == true && number_value == 42.5) {
+            ESP_LOGI("test", "Preferences verified: values match!");
+          } else {
+            ESP_LOGE("test", "Preferences mismatch: switch=%d, number=%.1f",
+                     switch_value, number_value);
+          }
diff --git a/tests/integration/fixtures/light_calls.yaml b/tests/integration/fixtures/light_calls.yaml
index d692a11765..2b7650526f 100644
--- a/tests/integration/fixtures/light_calls.yaml
+++ b/tests/integration/fixtures/light_calls.yaml
@@ -56,10 +56,29 @@ light:
     warm_white_color_temperature: 2000 K
     constant_brightness: true
     effects:
+      # Use default parameters:
       - random:
-          name: "Random Effect"
+      # Customize parameters - use longer names to potentially trigger buffer issues
+      - random:
+          name: "My Very Slow Random Effect With Long Name"
+          transition_length: 30ms
+          update_interval: 30ms
+      - random:
+          name: "My Fast Random Effect That Changes Quickly"
+          transition_length: 4ms
+          update_interval: 5ms
+      - random:
+          name: "Random Effect With Medium Length Name Here"
           transition_length: 100ms
           update_interval: 200ms
+      - random:
+          name: "Another Random Effect With Different Parameters"
+          transition_length: 2ms
+          update_interval: 3ms
+      - random:
+          name: "Yet Another Random Effect To Test Memory"
+          transition_length: 15ms
+          update_interval: 20ms
       - strobe:
           name: "Strobe Effect"
       - pulse:
@@ -73,6 +92,17 @@ light:
     red: test_red
     green: test_green
     blue: test_blue
+    effects:
+      # Same random effects to test for cross-contamination
+      - random:
+      - random:
+          name: "RGB Slow Random"
+          transition_length: 20ms
+          update_interval: 25ms
+      - random:
+          name: "RGB Fast Random"
+          transition_length: 2ms
+          update_interval: 3ms
 
   - platform: binary
     name: "Test Binary Light"
diff --git a/tests/integration/fixtures/multi_device_preferences.yaml b/tests/integration/fixtures/multi_device_preferences.yaml
new file mode 100644
index 0000000000..634d7157b2
--- /dev/null
+++ b/tests/integration/fixtures/multi_device_preferences.yaml
@@ -0,0 +1,165 @@
+esphome:
+  name: multi-device-preferences-test
+  # Define multiple devices for testing preference storage
+  devices:
+    - id: device_a
+      name: Device A
+    - id: device_b
+      name: Device B
+
+host:
+api:  # Port will be automatically injected
+logger:
+  level: DEBUG
+
+# Test entities with restore modes to verify preference storage
+
+# Switches with same name on different devices - test restore mode
+switch:
+  - platform: template
+    name: Light
+    id: light_device_a
+    device_id: device_a
+    restore_mode: RESTORE_DEFAULT_OFF
+    turn_on_action:
+      - lambda: |-
+          ESP_LOGI("test", "Device A Light turned ON");
+    turn_off_action:
+      - lambda: |-
+          ESP_LOGI("test", "Device A Light turned OFF");
+
+  - platform: template
+    name: Light
+    id: light_device_b
+    device_id: device_b
+    restore_mode: RESTORE_DEFAULT_ON  # Different default to test uniqueness
+    turn_on_action:
+      - lambda: |-
+          ESP_LOGI("test", "Device B Light turned ON");
+    turn_off_action:
+      - lambda: |-
+          ESP_LOGI("test", "Device B Light turned OFF");
+
+  - platform: template
+    name: Light
+    id: light_main
+    restore_mode: RESTORE_DEFAULT_OFF
+    turn_on_action:
+      - lambda: |-
+          ESP_LOGI("test", "Main Light turned ON");
+    turn_off_action:
+      - lambda: |-
+          ESP_LOGI("test", "Main Light turned OFF");
+
+# Numbers with restore to test preference storage
+number:
+  - platform: template
+    name: Setpoint
+    id: setpoint_device_a
+    device_id: device_a
+    min_value: 10.0
+    max_value: 30.0
+    step: 0.5
+    restore_value: true
+    initial_value: 20.0
+    set_action:
+      - lambda: |-
+          ESP_LOGI("test", "Device A Setpoint set to %.1f", x);
+          id(setpoint_device_a).state = x;
+
+  - platform: template
+    name: Setpoint
+    id: setpoint_device_b
+    device_id: device_b
+    min_value: 10.0
+    max_value: 30.0
+    step: 0.5
+    restore_value: true
+    initial_value: 25.0  # Different initial to test uniqueness
+    set_action:
+      - lambda: |-
+          ESP_LOGI("test", "Device B Setpoint set to %.1f", x);
+          id(setpoint_device_b).state = x;
+
+  - platform: template
+    name: Setpoint
+    id: setpoint_main
+    min_value: 10.0
+    max_value: 30.0
+    step: 0.5
+    restore_value: true
+    initial_value: 22.0
+    set_action:
+      - lambda: |-
+          ESP_LOGI("test", "Main Setpoint set to %.1f", x);
+          id(setpoint_main).state = x;
+
+# Selects with restore to test preference storage
+select:
+  - platform: template
+    name: Mode
+    id: mode_device_a
+    device_id: device_a
+    options:
+      - "Auto"
+      - "Manual"
+      - "Off"
+    restore_value: true
+    initial_option: "Auto"
+    set_action:
+      - lambda: |-
+          ESP_LOGI("test", "Device A Mode set to %s", x.c_str());
+          id(mode_device_a).state = x;
+
+  - platform: template
+    name: Mode
+    id: mode_device_b
+    device_id: device_b
+    options:
+      - "Auto"
+      - "Manual"
+      - "Off"
+    restore_value: true
+    initial_option: "Manual"  # Different initial to test uniqueness
+    set_action:
+      - lambda: |-
+          ESP_LOGI("test", "Device B Mode set to %s", x.c_str());
+          id(mode_device_b).state = x;
+
+  - platform: template
+    name: Mode
+    id: mode_main
+    options:
+      - "Auto"
+      - "Manual"
+      - "Off"
+    restore_value: true
+    initial_option: "Off"
+    set_action:
+      - lambda: |-
+          ESP_LOGI("test", "Main Mode set to %s", x.c_str());
+          id(mode_main).state = x;
+
+# Button to trigger preference logging test
+button:
+  - platform: template
+    name: Test Preferences
+    on_press:
+      - lambda: |-
+          ESP_LOGI("test", "Testing preference storage uniqueness:");
+          ESP_LOGI("test", "Device A Light state: %s", id(light_device_a).state ? "ON" : "OFF");
+          ESP_LOGI("test", "Device B Light state: %s", id(light_device_b).state ? "ON" : "OFF");
+          ESP_LOGI("test", "Main Light state: %s", id(light_main).state ? "ON" : "OFF");
+          ESP_LOGI("test", "Device A Setpoint: %.1f", id(setpoint_device_a).state);
+          ESP_LOGI("test", "Device B Setpoint: %.1f", id(setpoint_device_b).state);
+          ESP_LOGI("test", "Main Setpoint: %.1f", id(setpoint_main).state);
+          ESP_LOGI("test", "Device A Mode: %s", id(mode_device_a).state.c_str());
+          ESP_LOGI("test", "Device B Mode: %s", id(mode_device_b).state.c_str());
+          ESP_LOGI("test", "Main Mode: %s", id(mode_main).state.c_str());
+          // Log preference hashes for entities that actually store preferences
+          ESP_LOGI("test", "Device A Switch Pref Hash: %u", id(light_device_a).get_preference_hash());
+          ESP_LOGI("test", "Device B Switch Pref Hash: %u", id(light_device_b).get_preference_hash());
+          ESP_LOGI("test", "Main Switch Pref Hash: %u", id(light_main).get_preference_hash());
+          ESP_LOGI("test", "Device A Number Pref Hash: %u", id(setpoint_device_a).get_preference_hash());
+          ESP_LOGI("test", "Device B Number Pref Hash: %u", id(setpoint_device_b).get_preference_hash());
+          ESP_LOGI("test", "Main Number Pref Hash: %u", id(setpoint_main).get_preference_hash());
diff --git a/tests/integration/fixtures/noise_corrupt_encrypted_frame.yaml b/tests/integration/fixtures/noise_corrupt_encrypted_frame.yaml
new file mode 100644
index 0000000000..6f0266c6fd
--- /dev/null
+++ b/tests/integration/fixtures/noise_corrupt_encrypted_frame.yaml
@@ -0,0 +1,11 @@
+esphome:
+  name: oversized-noise
+
+host:
+
+api:
+  encryption:
+    key: N4Yle5YirwZhPiHHsdZLdOA73ndj/84veVaLhTvxCuU=
+
+logger:
+  level: VERY_VERBOSE
diff --git a/tests/integration/fixtures/noise_encryption_key_protection.yaml b/tests/integration/fixtures/noise_encryption_key_protection.yaml
new file mode 100644
index 0000000000..3ce84cd373
--- /dev/null
+++ b/tests/integration/fixtures/noise_encryption_key_protection.yaml
@@ -0,0 +1,10 @@
+esphome:
+  name: noise-key-test
+
+host:
+
+api:
+  encryption:
+    key: "zX9/JHxMKwpP0jUGsF0iESCm1wRvNgR6NkKVOhn7kSs="
+
+logger:
diff --git a/tests/integration/fixtures/oversized_payload_noise.yaml b/tests/integration/fixtures/oversized_payload_noise.yaml
new file mode 100644
index 0000000000..6f0266c6fd
--- /dev/null
+++ b/tests/integration/fixtures/oversized_payload_noise.yaml
@@ -0,0 +1,11 @@
+esphome:
+  name: oversized-noise
+
+host:
+
+api:
+  encryption:
+    key: N4Yle5YirwZhPiHHsdZLdOA73ndj/84veVaLhTvxCuU=
+
+logger:
+  level: VERY_VERBOSE
diff --git a/tests/integration/fixtures/oversized_payload_plaintext.yaml b/tests/integration/fixtures/oversized_payload_plaintext.yaml
new file mode 100644
index 0000000000..44ece4f770
--- /dev/null
+++ b/tests/integration/fixtures/oversized_payload_plaintext.yaml
@@ -0,0 +1,9 @@
+esphome:
+  name: oversized-plaintext
+
+host:
+
+api:
+
+logger:
+  level: VERY_VERBOSE
diff --git a/tests/integration/fixtures/oversized_protobuf_message_id_noise.yaml b/tests/integration/fixtures/oversized_protobuf_message_id_noise.yaml
new file mode 100644
index 0000000000..6f0266c6fd
--- /dev/null
+++ b/tests/integration/fixtures/oversized_protobuf_message_id_noise.yaml
@@ -0,0 +1,11 @@
+esphome:
+  name: oversized-noise
+
+host:
+
+api:
+  encryption:
+    key: N4Yle5YirwZhPiHHsdZLdOA73ndj/84veVaLhTvxCuU=
+
+logger:
+  level: VERY_VERBOSE
diff --git a/tests/integration/fixtures/oversized_protobuf_message_id_plaintext.yaml b/tests/integration/fixtures/oversized_protobuf_message_id_plaintext.yaml
new file mode 100644
index 0000000000..1e9eadfdc5
--- /dev/null
+++ b/tests/integration/fixtures/oversized_protobuf_message_id_plaintext.yaml
@@ -0,0 +1,9 @@
+esphome:
+  name: oversized-protobuf-plaintext
+
+host:
+
+api:
+
+logger:
+  level: VERY_VERBOSE
diff --git a/tests/integration/fixtures/parallel_script_delays.yaml b/tests/integration/fixtures/parallel_script_delays.yaml
new file mode 100644
index 0000000000..71d5b904e9
--- /dev/null
+++ b/tests/integration/fixtures/parallel_script_delays.yaml
@@ -0,0 +1,45 @@
+esphome:
+  name: test-parallel-delays
+
+host:
+
+logger:
+  level: VERY_VERBOSE
+
+api:
+  actions:
+    - action: test_parallel_delays
+      then:
+        # Start three parallel script instances with small delays between starts
+        - globals.set:
+            id: instance_counter
+            value: '1'
+        - script.execute: parallel_delay_script
+        - delay: 10ms
+        - globals.set:
+            id: instance_counter
+            value: '2'
+        - script.execute: parallel_delay_script
+        - delay: 10ms
+        - globals.set:
+            id: instance_counter
+            value: '3'
+        - script.execute: parallel_delay_script
+
+globals:
+  - id: instance_counter
+    type: int
+    initial_value: '0'
+
+script:
+  - id: parallel_delay_script
+    mode: parallel
+    then:
+      - lambda: !lambda |-
+          int instance = id(instance_counter);
+          ESP_LOGI("TEST", "Parallel script instance %d started", instance);
+      - delay: 1s
+      - lambda: !lambda |-
+          static int completed_counter = 0;
+          completed_counter++;
+          ESP_LOGI("TEST", "Parallel script instance %d completed after delay", completed_counter);
diff --git a/tests/integration/fixtures/runtime_stats.yaml b/tests/integration/fixtures/runtime_stats.yaml
new file mode 100644
index 0000000000..aad1c275fb
--- /dev/null
+++ b/tests/integration/fixtures/runtime_stats.yaml
@@ -0,0 +1,39 @@
+esphome:
+  name: runtime-stats-test
+
+host:
+
+api:
+
+logger:
+  level: DEBUG
+  logs:
+    runtime_stats: INFO
+
+runtime_stats:
+  log_interval: 1s
+
+# Add some components that will execute periodically to generate stats
+sensor:
+  - platform: template
+    name: "Test Sensor 1"
+    id: test_sensor_1
+    lambda: return 42.0;
+    update_interval: 0.1s
+
+  - platform: template
+    name: "Test Sensor 2"
+    id: test_sensor_2
+    lambda: return 24.0;
+    update_interval: 0.2s
+
+switch:
+  - platform: template
+    name: "Test Switch"
+    id: test_switch
+    optimistic: true
+
+interval:
+  - interval: 0.5s
+    then:
+      - switch.toggle: test_switch
diff --git a/tests/integration/fixtures/scheduler_pool.yaml b/tests/integration/fixtures/scheduler_pool.yaml
new file mode 100644
index 0000000000..5389125188
--- /dev/null
+++ b/tests/integration/fixtures/scheduler_pool.yaml
@@ -0,0 +1,282 @@
+esphome:
+  name: scheduler-pool-test
+  on_boot:
+    priority: -100
+    then:
+      - logger.log: "Starting scheduler pool tests"
+  debug_scheduler: true  # Enable scheduler debug logging
+
+host:
+api:
+  services:
+    - service: run_phase_1
+      then:
+        - script.execute: test_pool_recycling
+    - service: run_phase_2
+      then:
+        - script.execute: test_sensor_polling
+    - service: run_phase_3
+      then:
+        - script.execute: test_communication_patterns
+    - service: run_phase_4
+      then:
+        - script.execute: test_defer_patterns
+    - service: run_phase_5
+      then:
+        - script.execute: test_pool_reuse_verification
+    - service: run_phase_6
+      then:
+        - script.execute: test_full_pool_reuse
+    - service: run_phase_7
+      then:
+        - script.execute: test_same_defer_optimization
+    - service: run_complete
+      then:
+        - script.execute: complete_test
+logger:
+  level: VERY_VERBOSE  # Need VERY_VERBOSE to see pool debug messages
+
+globals:
+  - id: create_count
+    type: int
+    initial_value: '0'
+  - id: cancel_count
+    type: int
+    initial_value: '0'
+  - id: interval_counter
+    type: int
+    initial_value: '0'
+  - id: pool_test_done
+    type: bool
+    initial_value: 'false'
+
+script:
+  - id: test_pool_recycling
+    then:
+      - logger.log: "Testing scheduler pool recycling with realistic usage patterns"
+      - lambda: |-
+          auto *component = id(test_sensor);
+
+          // Simulate realistic component behavior with timeouts that complete naturally
+          ESP_LOGI("test", "Phase 1: Simulating normal component lifecycle");
+
+          // Sensor update timeouts (common pattern)
+          App.scheduler.set_timeout(component, "sensor_init", 10, []() {
+            ESP_LOGD("test", "Sensor initialized");
+            id(create_count)++;
+          });
+
+          // Retry timeout (gets cancelled if successful)
+          App.scheduler.set_timeout(component, "retry_timeout", 50, []() {
+            ESP_LOGD("test", "Retry timeout executed");
+            id(create_count)++;
+          });
+
+          // Simulate successful operation - cancel retry
+          App.scheduler.set_timeout(component, "success_sim", 20, []() {
+            ESP_LOGD("test", "Operation succeeded, cancelling retry");
+            App.scheduler.cancel_timeout(id(test_sensor), "retry_timeout");
+            id(cancel_count)++;
+          });
+
+          id(create_count) += 3;
+          ESP_LOGI("test", "Phase 1 complete");
+
+  - id: test_sensor_polling
+    then:
+      - lambda: |-
+          // Simulate sensor polling pattern
+          ESP_LOGI("test", "Phase 2: Simulating sensor polling patterns");
+          auto *component = id(test_sensor);
+
+          // Multiple sensors with different update intervals
+          // These should only allocate once and reuse the same item for each interval execution
+          App.scheduler.set_interval(component, "temp_sensor", 10, []() {
+            ESP_LOGD("test", "Temperature sensor update");
+            id(interval_counter)++;
+            if (id(interval_counter) >= 3) {
+              App.scheduler.cancel_interval(id(test_sensor), "temp_sensor");
+              ESP_LOGD("test", "Temperature sensor stopped");
+            }
+          });
+
+          App.scheduler.set_interval(component, "humidity_sensor", 15, []() {
+            ESP_LOGD("test", "Humidity sensor update");
+            id(interval_counter)++;
+            if (id(interval_counter) >= 5) {
+              App.scheduler.cancel_interval(id(test_sensor), "humidity_sensor");
+              ESP_LOGD("test", "Humidity sensor stopped");
+            }
+          });
+
+          // Only 2 allocations for the intervals, no matter how many times they execute
+          id(create_count) += 2;
+          ESP_LOGD("test", "Created 2 intervals - they will reuse same items for each execution");
+          ESP_LOGI("test", "Phase 2 complete");
+
+  - id: test_communication_patterns
+    then:
+      - lambda: |-
+          // Simulate communication patterns (WiFi/API reconnects, etc)
+          ESP_LOGI("test", "Phase 3: Simulating communication patterns");
+          auto *component = id(test_sensor);
+
+          // Connection timeout pattern
+          App.scheduler.set_timeout(component, "connect_timeout", 200, []() {
+            ESP_LOGD("test", "Connection timeout - would retry");
+            id(create_count)++;
+
+            // Schedule retry
+            App.scheduler.set_timeout(id(test_sensor), "connect_retry", 100, []() {
+              ESP_LOGD("test", "Retrying connection");
+              id(create_count)++;
+            });
+          });
+
+          // Heartbeat pattern
+          App.scheduler.set_interval(component, "heartbeat", 50, []() {
+            ESP_LOGD("test", "Heartbeat");
+            id(interval_counter)++;
+            if (id(interval_counter) >= 10) {
+              App.scheduler.cancel_interval(id(test_sensor), "heartbeat");
+              ESP_LOGD("test", "Heartbeat stopped");
+            }
+          });
+
+          id(create_count) += 2;
+          ESP_LOGI("test", "Phase 3 complete");
+
+  - id: test_defer_patterns
+    then:
+      - lambda: |-
+          // Simulate defer patterns (state changes, async operations)
+          ESP_LOGI("test", "Phase 4: Simulating heavy defer patterns like ratgdo");
+
+          auto *component = id(test_sensor);
+
+          // Simulate a burst of defer operations like ratgdo does with state updates
+          // These should execute immediately and recycle quickly to the pool
+          for (int i = 0; i < 10; i++) {
+            std::string defer_name = "defer_" + std::to_string(i);
+            App.scheduler.set_timeout(component, defer_name, 0, [i]() {
+              ESP_LOGD("test", "Defer %d executed", i);
+              // Force a small delay between defer executions to see recycling
+              if (i == 5) {
+                ESP_LOGI("test", "Half of defers executed, checking pool status");
+              }
+            });
+          }
+
+          id(create_count) += 10;
+          ESP_LOGD("test", "Created 10 defer operations (0ms timeouts)");
+
+          // Also create some named defers that might get replaced
+          App.scheduler.set_timeout(component, "state_update", 0, []() {
+            ESP_LOGD("test", "State update 1");
+          });
+
+          // Replace the same named defer (should cancel previous)
+          App.scheduler.set_timeout(component, "state_update", 0, []() {
+            ESP_LOGD("test", "State update 2 (replaced)");
+          });
+
+          id(create_count) += 2;
+          id(cancel_count) += 1; // One cancelled due to replacement
+
+          ESP_LOGI("test", "Phase 4 complete");
+
+  - id: test_pool_reuse_verification
+    then:
+      - lambda: |-
+          ESP_LOGI("test", "Phase 5: Verifying pool reuse after everything settles");
+
+          // Cancel any remaining intervals
+          auto *component = id(test_sensor);
+          App.scheduler.cancel_interval(component, "temp_sensor");
+          App.scheduler.cancel_interval(component, "humidity_sensor");
+          App.scheduler.cancel_interval(component, "heartbeat");
+
+          ESP_LOGD("test", "Cancelled any remaining intervals");
+
+          // The pool should have items from completed timeouts in earlier phases.
+          // Phase 1 had 3 timeouts that completed and were recycled.
+          // Phase 3 had 1 timeout that completed and was recycled.
+          // Phase 4 had 3 defers that completed and were recycled.
+          // So we should have a decent pool size already from naturally completed items.
+
+          // Now create 8 new timeouts - they should reuse from pool when available
+          int reuse_test_count = 8;
+
+          for (int i = 0; i < reuse_test_count; i++) {
+            std::string name = "reuse_test_" + std::to_string(i);
+            App.scheduler.set_timeout(component, name, 10 + i * 5, [i]() {
+              ESP_LOGD("test", "Reuse test %d completed", i);
+            });
+          }
+
+          ESP_LOGI("test", "Created %d items for reuse verification", reuse_test_count);
+          id(create_count) += reuse_test_count;
+          ESP_LOGI("test", "Phase 5 complete");
+
+  - id: test_full_pool_reuse
+    then:
+      - lambda: |-
+          ESP_LOGI("test", "Phase 6: Testing pool size limits after Phase 5 items complete");
+
+          // At this point, all Phase 5 timeouts should have completed and been recycled.
+          // The pool should be at its maximum size (5).
+          // Creating 10 new items tests that:
+          // - First 5 items reuse from the pool
+          // - Remaining 5 items allocate new (pool empty)
+          // - Pool doesn't grow beyond MAX_POOL_SIZE of 5
+
+          auto *component = id(test_sensor);
+          int full_reuse_count = 10;
+
+          for (int i = 0; i < full_reuse_count; i++) {
+            std::string name = "full_reuse_" + std::to_string(i);
+            App.scheduler.set_timeout(component, name, 10 + i * 5, [i]() {
+              ESP_LOGD("test", "Full reuse test %d completed", i);
+            });
+          }
+
+          ESP_LOGI("test", "Created %d items for full pool reuse verification", full_reuse_count);
+          id(create_count) += full_reuse_count;
+          ESP_LOGI("test", "Phase 6 complete");
+
+  - id: test_same_defer_optimization
+    then:
+      - lambda: |-
+          ESP_LOGI("test", "Phase 7: Testing same-named defer optimization");
+
+          auto *component = id(test_sensor);
+
+          // Create 10 defers with the same name - should optimize to update callback in-place
+          // This pattern is common in components like ratgdo that repeatedly defer state updates
+          for (int i = 0; i < 10; i++) {
+            App.scheduler.set_timeout(component, "repeated_defer", 0, [i]() {
+              ESP_LOGD("test", "Repeated defer executed with value: %d", i);
+            });
+          }
+
+          // Only the first should allocate, the rest should update in-place
+          // We expect only 1 allocation for all 10 operations
+          id(create_count) += 1;  // Only count 1 since others should be optimized
+
+          ESP_LOGD("test", "Created 10 same-named defers (should only allocate once)");
+          ESP_LOGI("test", "Phase 7 complete");
+
+  - id: complete_test
+    then:
+      - lambda: |-
+          ESP_LOGI("test", "Pool recycling test complete - created %d items, cancelled %d, intervals %d",
+                   id(create_count), id(cancel_count), id(interval_counter));
+
+sensor:
+  - platform: template
+    name: Test Sensor
+    id: test_sensor
+    lambda: return 1.0;
+    update_interval: never
+
+# No interval - tests will be triggered from Python via API services
diff --git a/tests/integration/fixtures/scheduler_removed_item_race.yaml b/tests/integration/fixtures/scheduler_removed_item_race.yaml
new file mode 100644
index 0000000000..2f8a7fb987
--- /dev/null
+++ b/tests/integration/fixtures/scheduler_removed_item_race.yaml
@@ -0,0 +1,139 @@
+esphome:
+  name: scheduler-removed-item-race
+
+host:
+
+api:
+  services:
+    - service: run_test
+      then:
+        - script.execute: run_test_script
+
+logger:
+  level: DEBUG
+
+globals:
+  - id: test_passed
+    type: bool
+    initial_value: 'true'
+  - id: removed_item_executed
+    type: int
+    initial_value: '0'
+  - id: normal_item_executed
+    type: int
+    initial_value: '0'
+
+sensor:
+  - platform: template
+    id: test_sensor
+    name: "Test Sensor"
+    update_interval: never
+    lambda: return 0.0;
+
+script:
+  - id: run_test_script
+    then:
+      - logger.log: "=== Starting Removed Item Race Test ==="
+
+      # This test creates a scenario where:
+      # 1. First item in heap is NOT cancelled (cleanup stops immediately)
+      # 2. Items behind it ARE cancelled (remain in heap after cleanup)
+      # 3. All items execute at the same time, including cancelled ones
+
+      - lambda: |-
+          // The key to hitting the race:
+          // 1. Add items in a specific order to control heap structure
+          // 2. Cancel ONLY items that won't be at the front
+          // 3. Ensure the first item stays non-cancelled so cleanup_() stops immediately
+
+          // Schedule all items to execute at the SAME time (1ms from now)
+          // Using 1ms instead of 0 to avoid defer queue on multi-core platforms
+          // This ensures they'll all be ready together and go through the heap
+          const uint32_t exec_time = 1;
+
+          // CRITICAL: Add a non-cancellable item FIRST
+          // This will be at the front of the heap and block cleanup_()
+          App.scheduler.set_timeout(id(test_sensor), "blocker", exec_time, []() {
+            ESP_LOGD("test", "Blocker timeout executed (expected) - was at front of heap");
+            id(normal_item_executed)++;
+          });
+
+          // Now add items that we WILL cancel
+          // These will be behind the blocker in the heap
+          App.scheduler.set_timeout(id(test_sensor), "cancel_1", exec_time, []() {
+            ESP_LOGE("test", "RACE: Cancelled timeout 1 executed after being cancelled!");
+            id(removed_item_executed)++;
+            id(test_passed) = false;
+          });
+
+          App.scheduler.set_timeout(id(test_sensor), "cancel_2", exec_time, []() {
+            ESP_LOGE("test", "RACE: Cancelled timeout 2 executed after being cancelled!");
+            id(removed_item_executed)++;
+            id(test_passed) = false;
+          });
+
+          App.scheduler.set_timeout(id(test_sensor), "cancel_3", exec_time, []() {
+            ESP_LOGE("test", "RACE: Cancelled timeout 3 executed after being cancelled!");
+            id(removed_item_executed)++;
+            id(test_passed) = false;
+          });
+
+          // Add some more normal items
+          App.scheduler.set_timeout(id(test_sensor), "normal_1", exec_time, []() {
+            ESP_LOGD("test", "Normal timeout 1 executed (expected)");
+            id(normal_item_executed)++;
+          });
+
+          App.scheduler.set_timeout(id(test_sensor), "normal_2", exec_time, []() {
+            ESP_LOGD("test", "Normal timeout 2 executed (expected)");
+            id(normal_item_executed)++;
+          });
+
+          App.scheduler.set_timeout(id(test_sensor), "normal_3", exec_time, []() {
+            ESP_LOGD("test", "Normal timeout 3 executed (expected)");
+            id(normal_item_executed)++;
+          });
+
+          // Force items into the heap before cancelling
+          App.scheduler.process_to_add();
+
+          // NOW cancel the items - they're behind "blocker" in the heap
+          // When cleanup_() runs, it will see "blocker" (not removed) at the front
+          // and stop immediately, leaving cancel_1, cancel_2, cancel_3 in the heap
+          bool c1 = App.scheduler.cancel_timeout(id(test_sensor), "cancel_1");
+          bool c2 = App.scheduler.cancel_timeout(id(test_sensor), "cancel_2");
+          bool c3 = App.scheduler.cancel_timeout(id(test_sensor), "cancel_3");
+
+          ESP_LOGD("test", "Cancelled items (behind blocker): %s, %s, %s",
+                   c1 ? "true" : "false",
+                   c2 ? "true" : "false",
+                   c3 ? "true" : "false");
+
+          // The heap now has:
+          // - "blocker" at front (not cancelled)
+          // - cancelled items behind it (marked remove=true but still in heap)
+          // - When all execute at once, cleanup_() stops at "blocker"
+          // - The loop then executes ALL ready items including cancelled ones
+
+          ESP_LOGD("test", "Setup complete. Blocker at front prevents cleanup of cancelled items behind it");
+
+      # Wait for all timeouts to execute (or not)
+      - delay: 20ms
+
+      # Check results
+      - lambda: |-
+          ESP_LOGI("test", "=== Test Results ===");
+          ESP_LOGI("test", "Normal items executed: %d (expected 4)", id(normal_item_executed));
+          ESP_LOGI("test", "Removed items executed: %d (expected 0)", id(removed_item_executed));
+
+          if (id(removed_item_executed) > 0) {
+            ESP_LOGE("test", "TEST FAILED: %d cancelled items were executed!", id(removed_item_executed));
+            id(test_passed) = false;
+          } else if (id(normal_item_executed) != 4) {
+            ESP_LOGE("test", "TEST FAILED: Expected 4 normal items, got %d", id(normal_item_executed));
+            id(test_passed) = false;
+          } else {
+            ESP_LOGI("test", "TEST PASSED: No cancelled items were executed");
+          }
+
+          ESP_LOGI("test", "=== Test Complete ===");
diff --git a/tests/integration/fixtures/scheduler_retry_test.yaml b/tests/integration/fixtures/scheduler_retry_test.yaml
new file mode 100644
index 0000000000..11fff6c395
--- /dev/null
+++ b/tests/integration/fixtures/scheduler_retry_test.yaml
@@ -0,0 +1,307 @@
+esphome:
+  name: scheduler-retry-test
+  on_boot:
+    priority: -100
+    then:
+      - logger.log: "Starting scheduler retry tests"
+      # Run all tests sequentially with delays
+      - script.execute: run_all_tests
+
+host:
+api:
+logger:
+  level: VERY_VERBOSE
+
+globals:
+  - id: simple_retry_counter
+    type: int
+    initial_value: '0'
+  - id: backoff_retry_counter
+    type: int
+    initial_value: '0'
+  - id: backoff_last_attempt_time
+    type: uint32_t
+    initial_value: '0'
+  - id: immediate_done_counter
+    type: int
+    initial_value: '0'
+  - id: cancel_retry_counter
+    type: int
+    initial_value: '0'
+  - id: empty_name_retry_counter
+    type: int
+    initial_value: '0'
+  - id: script_retry_counter
+    type: int
+    initial_value: '0'
+  - id: multiple_same_name_counter
+    type: int
+    initial_value: '0'
+  - id: const_char_retry_counter
+    type: int
+    initial_value: '0'
+  - id: static_char_retry_counter
+    type: int
+    initial_value: '0'
+  - id: mixed_cancel_result
+    type: bool
+    initial_value: 'false'
+
+# Using different component types for each test to ensure isolation
+sensor:
+  - platform: template
+    name: Simple Retry Test Sensor
+    id: simple_retry_sensor
+    lambda: return 1.0;
+    update_interval: never
+
+  - platform: template
+    name: Backoff Retry Test Sensor
+    id: backoff_retry_sensor
+    lambda: return 2.0;
+    update_interval: never
+
+  - platform: template
+    name: Immediate Done Test Sensor
+    id: immediate_done_sensor
+    lambda: return 3.0;
+    update_interval: never
+
+binary_sensor:
+  - platform: template
+    name: Cancel Retry Test Binary Sensor
+    id: cancel_retry_binary_sensor
+    lambda: return false;
+
+  - platform: template
+    name: Empty Name Test Binary Sensor
+    id: empty_name_binary_sensor
+    lambda: return true;
+
+switch:
+  - platform: template
+    name: Script Retry Test Switch
+    id: script_retry_switch
+    optimistic: true
+
+  - platform: template
+    name: Multiple Same Name Test Switch
+    id: multiple_same_name_switch
+    optimistic: true
+
+script:
+  - id: run_all_tests
+    then:
+      # Test 1: Simple retry
+      - logger.log: "=== Test 1: Simple retry ==="
+      - lambda: |-
+          auto *component = id(simple_retry_sensor);
+          App.scheduler.set_retry(component, "simple_retry", 50, 3,
+            [](uint8_t retry_countdown) {
+              id(simple_retry_counter)++;
+              ESP_LOGI("test", "Simple retry attempt %d (countdown=%d)",
+                       id(simple_retry_counter), retry_countdown);
+
+              if (id(simple_retry_counter) >= 2) {
+                ESP_LOGI("test", "Simple retry succeeded on attempt %d", id(simple_retry_counter));
+                return RetryResult::DONE;
+              }
+              return RetryResult::RETRY;
+            });
+
+      # Test 2: Backoff retry
+      - logger.log: "=== Test 2: Retry with backoff ==="
+      - lambda: |-
+          auto *component = id(backoff_retry_sensor);
+
+          App.scheduler.set_retry(component, "backoff_retry", 50, 4,
+            [](uint8_t retry_countdown) {
+              id(backoff_retry_counter)++;
+              uint32_t now = millis();
+              uint32_t interval = 0;
+
+              // Only calculate interval after first attempt
+              if (id(backoff_retry_counter) > 1) {
+                interval = now - id(backoff_last_attempt_time);
+              }
+              id(backoff_last_attempt_time) = now;
+
+              ESP_LOGI("test", "Backoff retry attempt %d (countdown=%d, interval=%dms)",
+                       id(backoff_retry_counter), retry_countdown, interval);
+
+              if (id(backoff_retry_counter) == 1) {
+                ESP_LOGI("test", "First call was immediate");
+              } else if (id(backoff_retry_counter) == 2) {
+                ESP_LOGI("test", "Second call interval: %dms (expected ~50ms)", interval);
+              } else if (id(backoff_retry_counter) == 3) {
+                ESP_LOGI("test", "Third call interval: %dms (expected ~100ms)", interval);
+              } else if (id(backoff_retry_counter) == 4) {
+                ESP_LOGI("test", "Fourth call interval: %dms (expected ~200ms)", interval);
+                ESP_LOGI("test", "Backoff retry completed");
+                return RetryResult::DONE;
+              }
+
+              return RetryResult::RETRY;
+            }, 2.0f);
+
+      # Test 3: Immediate done
+      - logger.log: "=== Test 3: Immediate done ==="
+      - lambda: |-
+          auto *component = id(immediate_done_sensor);
+          App.scheduler.set_retry(component, "immediate_done", 50, 5,
+            [](uint8_t retry_countdown) {
+              id(immediate_done_counter)++;
+              ESP_LOGI("test", "Immediate done retry called (countdown=%d)", retry_countdown);
+              return RetryResult::DONE;
+            });
+
+      # Test 4: Cancel retry
+      - logger.log: "=== Test 4: Cancel retry ==="
+      - lambda: |-
+          auto *component = id(cancel_retry_binary_sensor);
+          App.scheduler.set_retry(component, "cancel_test", 30, 10,
+            [](uint8_t retry_countdown) {
+              id(cancel_retry_counter)++;
+              ESP_LOGI("test", "Cancel test retry attempt %d", id(cancel_retry_counter));
+              return RetryResult::RETRY;
+            });
+
+          // Cancel it after 100ms
+          App.scheduler.set_timeout(component, "cancel_timer", 100, []() {
+            bool cancelled = App.scheduler.cancel_retry(id(cancel_retry_binary_sensor), "cancel_test");
+            ESP_LOGI("test", "Retry cancellation result: %s", cancelled ? "true" : "false");
+            ESP_LOGI("test", "Cancel retry ran %d times before cancellation", id(cancel_retry_counter));
+          });
+
+      # Test 5: Empty name retry
+      - logger.log: "=== Test 5: Empty name retry ==="
+      - lambda: |-
+          auto *component = id(empty_name_binary_sensor);
+          App.scheduler.set_retry(component, "", 100, 5,
+            [](uint8_t retry_countdown) {
+              id(empty_name_retry_counter)++;
+              ESP_LOGI("test", "Empty name retry attempt %d", id(empty_name_retry_counter));
+              return RetryResult::RETRY;
+            });
+
+          // Try to cancel after 150ms
+          App.scheduler.set_timeout(component, "empty_cancel_timer", 150, []() {
+            bool cancelled = App.scheduler.cancel_retry(id(empty_name_binary_sensor), "");
+            ESP_LOGI("test", "Empty name retry cancel result: %s",
+                     cancelled ? "true" : "false");
+            ESP_LOGI("test", "Empty name retry ran %d times", id(empty_name_retry_counter));
+          });
+
+      # Test 6: Component method
+      - logger.log: "=== Test 6: Component::set_retry method ==="
+      - lambda: |-
+          class TestRetryComponent : public Component {
+          public:
+            void test_retry() {
+              this->set_retry(50, 3,
+                [](uint8_t retry_countdown) {
+                  id(script_retry_counter)++;
+                  ESP_LOGI("test", "Component retry attempt %d", id(script_retry_counter));
+                  if (id(script_retry_counter) >= 2) {
+                    return RetryResult::DONE;
+                  }
+                  return RetryResult::RETRY;
+                }, 1.5f);
+            }
+          };
+
+          static TestRetryComponent test_component;
+          test_component.test_retry();
+
+      # Test 7: Multiple same name
+      - logger.log: "=== Test 7: Multiple retries with same name ==="
+      - lambda: |-
+          auto *component = id(multiple_same_name_switch);
+
+          // Set first retry
+          App.scheduler.set_retry(component, "duplicate_retry", 100, 5,
+            [](uint8_t retry_countdown) {
+              id(multiple_same_name_counter) += 1;
+              ESP_LOGI("test", "First duplicate retry - should not run");
+              return RetryResult::RETRY;
+            });
+
+          // Set second retry with same name (should cancel first)
+          App.scheduler.set_retry(component, "duplicate_retry", 50, 3,
+            [](uint8_t retry_countdown) {
+              id(multiple_same_name_counter) += 10;
+              ESP_LOGI("test", "Second duplicate retry attempt (counter=%d)",
+                       id(multiple_same_name_counter));
+              if (id(multiple_same_name_counter) >= 20) {
+                return RetryResult::DONE;
+              }
+              return RetryResult::RETRY;
+            });
+
+      # Test 8: Const char* overloads
+      - logger.log: "=== Test 8: Const char* overloads ==="
+      - lambda: |-
+          auto *component = id(simple_retry_sensor);
+
+          // Test 8a: Direct string literal
+          App.scheduler.set_retry(component, "const_char_test", 30, 2,
+            [](uint8_t retry_countdown) {
+              id(const_char_retry_counter)++;
+              ESP_LOGI("test", "Const char retry %d", id(const_char_retry_counter));
+              return RetryResult::DONE;
+            });
+
+      # Test 9: Static const char* variable
+      - logger.log: "=== Test 9: Static const char* ==="
+      - lambda: |-
+          auto *component = id(backoff_retry_sensor);
+
+          static const char* STATIC_NAME = "static_retry_test";
+          App.scheduler.set_retry(component, STATIC_NAME, 20, 1,
+            [](uint8_t retry_countdown) {
+              id(static_char_retry_counter)++;
+              ESP_LOGI("test", "Static const char retry %d", id(static_char_retry_counter));
+              return RetryResult::DONE;
+            });
+
+          // Cancel with same static const char*
+          App.scheduler.set_timeout(component, "static_cancel", 10, []() {
+            static const char* STATIC_NAME = "static_retry_test";
+            bool result = App.scheduler.cancel_retry(id(backoff_retry_sensor), STATIC_NAME);
+            ESP_LOGI("test", "Static cancel result: %s", result ? "true" : "false");
+          });
+
+      # Test 10: Mix string and const char* cancel
+      - logger.log: "=== Test 10: Mixed string/const char* ==="
+      - lambda: |-
+          auto *component = id(immediate_done_sensor);
+
+          // Set with std::string
+          std::string str_name = "mixed_retry";
+          App.scheduler.set_retry(component, str_name, 40, 3,
+            [](uint8_t retry_countdown) {
+              ESP_LOGI("test", "Mixed retry - should be cancelled");
+              return RetryResult::RETRY;
+            });
+
+          // Cancel with const char*
+          id(mixed_cancel_result) = App.scheduler.cancel_retry(component, "mixed_retry");
+          ESP_LOGI("test", "Mixed cancel result: %s", id(mixed_cancel_result) ? "true" : "false");
+
+      # Wait for all tests to complete before reporting
+      - delay: 500ms
+
+      # Final report
+      - logger.log: "=== Retry Test Results ==="
+      - lambda: |-
+          ESP_LOGI("test", "Simple retry counter: %d (expected 2)", id(simple_retry_counter));
+          ESP_LOGI("test", "Backoff retry counter: %d (expected 4)", id(backoff_retry_counter));
+          ESP_LOGI("test", "Immediate done counter: %d (expected 1)", id(immediate_done_counter));
+          ESP_LOGI("test", "Cancel retry counter: %d (expected 2-4)", id(cancel_retry_counter));
+          ESP_LOGI("test", "Empty name retry counter: %d (expected 1-2)", id(empty_name_retry_counter));
+          ESP_LOGI("test", "Component retry counter: %d (expected 2)", id(script_retry_counter));
+          ESP_LOGI("test", "Multiple same name counter: %d (expected 20+)", id(multiple_same_name_counter));
+          ESP_LOGI("test", "Const char retry counter: %d (expected 1)", id(const_char_retry_counter));
+          ESP_LOGI("test", "Static char retry counter: %d (expected 1)", id(static_char_retry_counter));
+          ESP_LOGI("test", "Mixed cancel result: %s (expected true)", id(mixed_cancel_result) ? "true" : "false");
+          ESP_LOGI("test", "All retry tests completed");
diff --git a/tests/integration/fixtures/scheduler_string_test.yaml b/tests/integration/fixtures/scheduler_string_test.yaml
index 3dfe891370..c53ec392df 100644
--- a/tests/integration/fixtures/scheduler_string_test.yaml
+++ b/tests/integration/fixtures/scheduler_string_test.yaml
@@ -4,9 +4,7 @@ esphome:
     priority: -100
     then:
       - logger.log: "Starting scheduler string tests"
-  platformio_options:
-    build_flags:
-      - "-DESPHOME_DEBUG_SCHEDULER"  # Enable scheduler debug logging
+  debug_scheduler: true  # Enable scheduler debug logging
 
 host:
 api:
@@ -32,6 +30,12 @@ globals:
   - id: results_reported
     type: bool
     initial_value: 'false'
+  - id: edge_tests_done
+    type: bool
+    initial_value: 'false'
+  - id: empty_cancel_failed
+    type: bool
+    initial_value: 'false'
 
 script:
   - id: test_static_strings
@@ -147,12 +151,106 @@ script:
           static TestDynamicDeferComponent test_dynamic_defer_component;
           test_dynamic_defer_component.test_dynamic_defer();
 
+  - id: test_cancellation_edge_cases
+    then:
+      - logger.log: "Testing cancellation edge cases"
+      - lambda: |-
+          auto *component1 = id(test_sensor1);
+          // Use a different component for empty string tests to avoid interference
+          auto *component2 = id(test_sensor2);
+
+          // Test 12: Cancel with empty string - regression test for issue #9599
+          // First create a timeout with empty name on component2 to avoid interference
+          App.scheduler.set_timeout(component2, "", 500, []() {
+            ESP_LOGE("test", "ERROR: Empty name timeout fired - it should have been cancelled!");
+            id(empty_cancel_failed) = true;
+          });
+
+          // Now cancel it - this should work after our fix
+          bool cancelled_empty = App.scheduler.cancel_timeout(component2, "");
+          ESP_LOGI("test", "Cancel empty string result: %s (should be true)", cancelled_empty ? "true" : "false");
+          if (!cancelled_empty) {
+            ESP_LOGE("test", "ERROR: Failed to cancel empty string timeout!");
+            id(empty_cancel_failed) = true;
+          }
+
+          // Test 13: Cancel non-existent timeout
+          bool cancelled_nonexistent = App.scheduler.cancel_timeout(component1, "does_not_exist");
+          ESP_LOGI("test", "Cancel non-existent timeout result: %s",
+                   cancelled_nonexistent ? "true (unexpected!)" : "false (expected)");
+
+          // Test 14: Multiple timeouts with same name - only last should execute
+          for (int i = 0; i < 5; i++) {
+            App.scheduler.set_timeout(component1, "duplicate_timeout", 200 + i*10, [i]() {
+              ESP_LOGI("test", "Duplicate timeout %d fired", i);
+              id(timeout_counter) += 1;
+            });
+          }
+          ESP_LOGI("test", "Created 5 timeouts with same name 'duplicate_timeout'");
+
+          // Test 15: Multiple intervals with same name - only last should run
+          for (int i = 0; i < 3; i++) {
+            App.scheduler.set_interval(component1, "duplicate_interval", 300, [i]() {
+              ESP_LOGI("test", "Duplicate interval %d fired", i);
+              id(interval_counter) += 10; // Large increment to detect multiple
+              // Cancel after first execution
+              App.scheduler.cancel_interval(id(test_sensor1), "duplicate_interval");
+            });
+          }
+          ESP_LOGI("test", "Created 3 intervals with same name 'duplicate_interval'");
+
+          // Test 16: Cancel with nullptr protection (via empty const char*)
+          const char* null_name = "";
+          App.scheduler.set_timeout(component2, null_name, 600, []() {
+            ESP_LOGE("test", "ERROR: Const char* empty timeout fired - should have been cancelled!");
+            id(empty_cancel_failed) = true;
+          });
+          bool cancelled_const_empty = App.scheduler.cancel_timeout(component2, null_name);
+          ESP_LOGI("test", "Cancel const char* empty result: %s (should be true)",
+                   cancelled_const_empty ? "true" : "false");
+          if (!cancelled_const_empty) {
+            ESP_LOGE("test", "ERROR: Failed to cancel const char* empty timeout!");
+            id(empty_cancel_failed) = true;
+          }
+
+          // Test 17: Rapid create/cancel/create with same name
+          App.scheduler.set_timeout(component1, "rapid_test", 5000, []() {
+            ESP_LOGI("test", "First rapid timeout - should not fire");
+            id(timeout_counter) += 100;
+          });
+          App.scheduler.cancel_timeout(component1, "rapid_test");
+          App.scheduler.set_timeout(component1, "rapid_test", 250, []() {
+            ESP_LOGI("test", "Second rapid timeout - should fire");
+            id(timeout_counter) += 1;
+          });
+
+          // Test 18: Cancel all with a specific name (multiple instances)
+          // Create multiple with same name
+          App.scheduler.set_timeout(component1, "multi_cancel", 300, []() {
+            ESP_LOGI("test", "Multi-cancel timeout 1");
+          });
+          App.scheduler.set_timeout(component1, "multi_cancel", 350, []() {
+            ESP_LOGI("test", "Multi-cancel timeout 2");
+          });
+          App.scheduler.set_timeout(component1, "multi_cancel", 400, []() {
+            ESP_LOGI("test", "Multi-cancel timeout 3 - only this should fire");
+            id(timeout_counter) += 1;
+          });
+          // Note: Each set_timeout with same name cancels the previous one automatically
+
   - id: report_results
     then:
       - lambda: |-
           ESP_LOGI("test", "Final results - Timeouts: %d, Intervals: %d",
                    id(timeout_counter), id(interval_counter));
 
+          // Check if empty string cancellation test passed
+          if (id(empty_cancel_failed)) {
+            ESP_LOGE("test", "ERROR: Empty string cancellation test FAILED!");
+          } else {
+            ESP_LOGI("test", "Empty string cancellation test PASSED");
+          }
+
 sensor:
   - platform: template
     name: Test Sensor 1
@@ -189,12 +287,23 @@ interval:
             - delay: 0.2s
             - script.execute: test_dynamic_strings
 
+  # Run cancellation edge case tests after dynamic tests
+  - interval: 0.2s
+    then:
+      - if:
+          condition:
+            lambda: 'return id(dynamic_tests_done) && !id(edge_tests_done);'
+          then:
+            - lambda: 'id(edge_tests_done) = true;'
+            - delay: 0.5s
+            - script.execute: test_cancellation_edge_cases
+
   # Report results after all tests
   - interval: 0.2s
     then:
       - if:
           condition:
-            lambda: 'return id(dynamic_tests_done) && !id(results_reported);'
+            lambda: 'return id(edge_tests_done) && !id(results_reported);'
           then:
             - lambda: 'id(results_reported) = true;'
             - delay: 1s
diff --git a/tests/integration/test_api_custom_services.py b/tests/integration/test_api_custom_services.py
index 2862dcc900..9ae4cdcb5d 100644
--- a/tests/integration/test_api_custom_services.py
+++ b/tests/integration/test_api_custom_services.py
@@ -59,86 +59,86 @@ async def test_api_custom_services(
             custom_arrays_future.set_result(True)
 
     # Run with log monitoring
-    async with run_compiled(yaml_config, line_callback=check_output):
-        async with api_client_connected() as client:
-            # Verify device info
-            device_info = await client.device_info()
-            assert device_info is not None
-            assert device_info.name == "api-custom-services-test"
+    async with (
+        run_compiled(yaml_config, line_callback=check_output),
+        api_client_connected() as client,
+    ):
+        # Verify device info
+        device_info = await client.device_info()
+        assert device_info is not None
+        assert device_info.name == "api-custom-services-test"
 
-            # List services
-            _, services = await client.list_entities_services()
+        # List services
+        _, services = await client.list_entities_services()
 
-            # Should have 4 services: 1 YAML + 3 CustomAPIDevice
-            assert len(services) == 4, f"Expected 4 services, found {len(services)}"
+        # Should have 4 services: 1 YAML + 3 CustomAPIDevice
+        assert len(services) == 4, f"Expected 4 services, found {len(services)}"
 
-            # Find our services
-            yaml_service: UserService | None = None
-            custom_service: UserService | None = None
-            custom_args_service: UserService | None = None
-            custom_arrays_service: UserService | None = None
+        # Find our services
+        yaml_service: UserService | None = None
+        custom_service: UserService | None = None
+        custom_args_service: UserService | None = None
+        custom_arrays_service: UserService | None = None
 
-            for service in services:
-                if service.name == "test_yaml_service":
-                    yaml_service = service
-                elif service.name == "custom_test_service":
-                    custom_service = service
-                elif service.name == "custom_service_with_args":
-                    custom_args_service = service
-                elif service.name == "custom_service_with_arrays":
-                    custom_arrays_service = service
+        for service in services:
+            if service.name == "test_yaml_service":
+                yaml_service = service
+            elif service.name == "custom_test_service":
+                custom_service = service
+            elif service.name == "custom_service_with_args":
+                custom_args_service = service
+            elif service.name == "custom_service_with_arrays":
+                custom_arrays_service = service
 
-            assert yaml_service is not None, "test_yaml_service not found"
-            assert custom_service is not None, "custom_test_service not found"
-            assert custom_args_service is not None, "custom_service_with_args not found"
-            assert custom_arrays_service is not None, (
-                "custom_service_with_arrays not found"
-            )
+        assert yaml_service is not None, "test_yaml_service not found"
+        assert custom_service is not None, "custom_test_service not found"
+        assert custom_args_service is not None, "custom_service_with_args not found"
+        assert custom_arrays_service is not None, "custom_service_with_arrays not found"
 
-            # Test YAML service
-            client.execute_service(yaml_service, {})
-            await asyncio.wait_for(yaml_service_future, timeout=5.0)
+        # Test YAML service
+        client.execute_service(yaml_service, {})
+        await asyncio.wait_for(yaml_service_future, timeout=5.0)
 
-            # Test simple CustomAPIDevice service
-            client.execute_service(custom_service, {})
-            await asyncio.wait_for(custom_service_future, timeout=5.0)
+        # Test simple CustomAPIDevice service
+        client.execute_service(custom_service, {})
+        await asyncio.wait_for(custom_service_future, timeout=5.0)
 
-            # Verify custom_args_service arguments
-            assert len(custom_args_service.args) == 4
-            arg_types = {arg.name: arg.type for arg in custom_args_service.args}
-            assert arg_types["arg_string"] == UserServiceArgType.STRING
-            assert arg_types["arg_int"] == UserServiceArgType.INT
-            assert arg_types["arg_bool"] == UserServiceArgType.BOOL
-            assert arg_types["arg_float"] == UserServiceArgType.FLOAT
+        # Verify custom_args_service arguments
+        assert len(custom_args_service.args) == 4
+        arg_types = {arg.name: arg.type for arg in custom_args_service.args}
+        assert arg_types["arg_string"] == UserServiceArgType.STRING
+        assert arg_types["arg_int"] == UserServiceArgType.INT
+        assert arg_types["arg_bool"] == UserServiceArgType.BOOL
+        assert arg_types["arg_float"] == UserServiceArgType.FLOAT
 
-            # Test CustomAPIDevice service with arguments
-            client.execute_service(
-                custom_args_service,
-                {
-                    "arg_string": "test_string",
-                    "arg_int": 456,
-                    "arg_bool": True,
-                    "arg_float": 78.9,
-                },
-            )
-            await asyncio.wait_for(custom_args_future, timeout=5.0)
+        # Test CustomAPIDevice service with arguments
+        client.execute_service(
+            custom_args_service,
+            {
+                "arg_string": "test_string",
+                "arg_int": 456,
+                "arg_bool": True,
+                "arg_float": 78.9,
+            },
+        )
+        await asyncio.wait_for(custom_args_future, timeout=5.0)
 
-            # Verify array service arguments
-            assert len(custom_arrays_service.args) == 4
-            array_arg_types = {arg.name: arg.type for arg in custom_arrays_service.args}
-            assert array_arg_types["bool_array"] == UserServiceArgType.BOOL_ARRAY
-            assert array_arg_types["int_array"] == UserServiceArgType.INT_ARRAY
-            assert array_arg_types["float_array"] == UserServiceArgType.FLOAT_ARRAY
-            assert array_arg_types["string_array"] == UserServiceArgType.STRING_ARRAY
+        # Verify array service arguments
+        assert len(custom_arrays_service.args) == 4
+        array_arg_types = {arg.name: arg.type for arg in custom_arrays_service.args}
+        assert array_arg_types["bool_array"] == UserServiceArgType.BOOL_ARRAY
+        assert array_arg_types["int_array"] == UserServiceArgType.INT_ARRAY
+        assert array_arg_types["float_array"] == UserServiceArgType.FLOAT_ARRAY
+        assert array_arg_types["string_array"] == UserServiceArgType.STRING_ARRAY
 
-            # Test CustomAPIDevice service with arrays
-            client.execute_service(
-                custom_arrays_service,
-                {
-                    "bool_array": [True, False],
-                    "int_array": [1, 2, 3],
-                    "float_array": [1.1, 2.2],
-                    "string_array": ["hello", "world"],
-                },
-            )
-            await asyncio.wait_for(custom_arrays_future, timeout=5.0)
+        # Test CustomAPIDevice service with arrays
+        client.execute_service(
+            custom_arrays_service,
+            {
+                "bool_array": [True, False],
+                "int_array": [1, 2, 3],
+                "float_array": [1.1, 2.2],
+                "string_array": ["hello", "world"],
+            },
+        )
+        await asyncio.wait_for(custom_arrays_future, timeout=5.0)
diff --git a/tests/integration/test_api_homeassistant.py b/tests/integration/test_api_homeassistant.py
new file mode 100644
index 0000000000..f69838396d
--- /dev/null
+++ b/tests/integration/test_api_homeassistant.py
@@ -0,0 +1,305 @@
+"""Integration test for Home Assistant API functionality.
+
+Tests:
+- Home Assistant service calls with templated values (main bug fix)
+- Service calls with empty string values
+- Home Assistant state reading (sensors, binary sensors, text sensors)
+- Home Assistant number and switch component control
+- Complex lambda expressions and string handling
+"""
+
+from __future__ import annotations
+
+import asyncio
+import re
+
+from aioesphomeapi import HomeassistantServiceCall
+import pytest
+
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_api_homeassistant(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Comprehensive test for Home Assistant API functionality."""
+    loop = asyncio.get_running_loop()
+
+    # Create futures for patterns that capture values
+    lambda_computed_future = loop.create_future()
+    ha_temp_state_future = loop.create_future()
+    ha_humidity_state_future = loop.create_future()
+    ha_motion_state_future = loop.create_future()
+    ha_weather_state_future = loop.create_future()
+
+    # State update futures
+    temp_update_future = loop.create_future()
+    humidity_update_future = loop.create_future()
+    motion_update_future = loop.create_future()
+    weather_update_future = loop.create_future()
+
+    # Number future
+    ha_number_future = loop.create_future()
+
+    tests_complete_future = loop.create_future()
+
+    # Patterns to match in logs - only keeping patterns that capture values
+    lambda_computed_pattern = re.compile(r"Lambda computed value: (\d+)")
+    ha_temp_state_pattern = re.compile(r"Current HA Temperature: ([\d.]+)")
+    ha_humidity_state_pattern = re.compile(r"Current HA Humidity: ([\d.]+)")
+    ha_motion_state_pattern = re.compile(r"Current HA Motion: (ON|OFF)")
+    ha_weather_state_pattern = re.compile(r"Current HA Weather: (\w+)")
+
+    # State update patterns
+    temp_update_pattern = re.compile(r"HA Temperature state updated: ([\d.]+)")
+    humidity_update_pattern = re.compile(r"HA Humidity state updated: ([\d.]+)")
+    motion_update_pattern = re.compile(r"HA Motion state changed: (ON|OFF)")
+    weather_update_pattern = re.compile(r"HA Weather condition updated: (\w+)")
+
+    # Number pattern
+    ha_number_pattern = re.compile(r"Setting HA number to: ([\d.]+)")
+
+    tests_complete_pattern = re.compile(r"=== All tests completed ===")
+
+    # Track all log lines for debugging
+    log_lines: list[str] = []
+
+    # Track HomeAssistant service calls
+    ha_service_calls: list[HomeassistantServiceCall] = []
+
+    # Service call futures organized by service name
+    service_call_futures = {
+        "light.turn_off": loop.create_future(),  # basic_service_call
+        "light.turn_on": loop.create_future(),  # templated_service_call
+        "notify.test": loop.create_future(),  # empty_string_service_call
+        "climate.set_temperature": loop.create_future(),  # multiple_fields_service_call
+        "script.test_script": loop.create_future(),  # complex_lambda_service_call
+        "test.empty": loop.create_future(),  # all_empty_service_call
+        "input_number.set_value": loop.create_future(),  # ha_number_service_call
+        "switch.turn_on": loop.create_future(),  # ha_switch_on_service_call
+        "switch.turn_off": loop.create_future(),  # ha_switch_off_service_call
+    }
+
+    def on_service_call(service_call: HomeassistantServiceCall) -> None:
+        """Capture HomeAssistant service calls."""
+        ha_service_calls.append(service_call)
+
+        # Check if this service call is one we're waiting for
+        if service_call.service in service_call_futures:
+            future = service_call_futures[service_call.service]
+            if not future.done():
+                future.set_result(service_call)
+
+    def check_output(line: str) -> None:
+        """Check log output for expected messages."""
+        log_lines.append(line)
+
+        # Check for patterns that capture values
+        if not lambda_computed_future.done():
+            match = lambda_computed_pattern.search(line)
+            if match:
+                lambda_computed_future.set_result(match.group(1))
+        elif not ha_temp_state_future.done() and ha_temp_state_pattern.search(line):
+            ha_temp_state_future.set_result(line)
+        elif not ha_humidity_state_future.done() and ha_humidity_state_pattern.search(
+            line
+        ):
+            ha_humidity_state_future.set_result(line)
+        elif not ha_motion_state_future.done() and ha_motion_state_pattern.search(line):
+            ha_motion_state_future.set_result(line)
+        elif not ha_weather_state_future.done() and ha_weather_state_pattern.search(
+            line
+        ):
+            ha_weather_state_future.set_result(line)
+
+        # Check state update patterns
+        elif not temp_update_future.done() and temp_update_pattern.search(line):
+            temp_update_future.set_result(line)
+        elif not humidity_update_future.done() and humidity_update_pattern.search(line):
+            humidity_update_future.set_result(line)
+        elif not motion_update_future.done() and motion_update_pattern.search(line):
+            motion_update_future.set_result(line)
+        elif not weather_update_future.done() and weather_update_pattern.search(line):
+            weather_update_future.set_result(line)
+
+        # Check number pattern
+        elif not ha_number_future.done() and ha_number_pattern.search(line):
+            match = ha_number_pattern.search(line)
+            if match:
+                ha_number_future.set_result(match.group(1))
+
+        elif not tests_complete_future.done() and tests_complete_pattern.search(line):
+            tests_complete_future.set_result(True)
+
+    # Run with log monitoring
+    async with (
+        run_compiled(yaml_config, line_callback=check_output),
+        api_client_connected() as client,
+    ):
+        # Verify device info
+        device_info = await client.device_info()
+        assert device_info is not None
+        assert device_info.name == "test-ha-api"
+
+        # Subscribe to HomeAssistant service calls
+        client.subscribe_service_calls(on_service_call)
+
+        # Send some Home Assistant states for our sensors to read
+        client.send_home_assistant_state("sensor.external_temperature", "", "22.5")
+        client.send_home_assistant_state("sensor.external_humidity", "", "65.0")
+        client.send_home_assistant_state("binary_sensor.external_motion", "", "ON")
+        client.send_home_assistant_state("weather.home", "condition", "sunny")
+
+        # List entities and services
+        _, services = await client.list_entities_services()
+
+        # Find the trigger service
+        trigger_service = next(
+            (s for s in services if s.name == "trigger_all_tests"), None
+        )
+        assert trigger_service is not None, "trigger_all_tests service not found"
+
+        # Execute all tests
+        client.execute_service(trigger_service, {})
+
+        # Wait for all tests to complete with appropriate timeouts
+        try:
+            # Templated service test - the main bug fix
+            computed_value = await asyncio.wait_for(lambda_computed_future, timeout=5.0)
+            # Verify the computed value is reasonable (75 * 255 / 100 = 191.25 -> 191)
+            assert computed_value in ["191", "192"], (
+                f"Unexpected computed value: {computed_value}"
+            )
+
+            # Check state reads - verify we received the mocked values
+            temp_line = await asyncio.wait_for(ha_temp_state_future, timeout=5.0)
+            assert "Current HA Temperature: 22.5" in temp_line
+
+            humidity_line = await asyncio.wait_for(
+                ha_humidity_state_future, timeout=5.0
+            )
+            assert "Current HA Humidity: 65.0" in humidity_line
+
+            motion_line = await asyncio.wait_for(ha_motion_state_future, timeout=5.0)
+            assert "Current HA Motion: ON" in motion_line
+
+            weather_line = await asyncio.wait_for(ha_weather_state_future, timeout=5.0)
+            assert "Current HA Weather: sunny" in weather_line
+
+            # Number test
+            number_value = await asyncio.wait_for(ha_number_future, timeout=5.0)
+            assert number_value == "42.5", f"Unexpected number value: {number_value}"
+
+            # Wait for completion
+            await asyncio.wait_for(tests_complete_future, timeout=5.0)
+
+            # Now verify the protobuf messages
+            # 1. Basic service call
+            basic_call = await asyncio.wait_for(
+                service_call_futures["light.turn_off"], timeout=2.0
+            )
+            assert basic_call.service == "light.turn_off"
+            assert "entity_id" in basic_call.data, (
+                f"entity_id not found in data: {basic_call.data}"
+            )
+            assert basic_call.data["entity_id"] == "light.test_light", (
+                f"Wrong entity_id: {basic_call.data['entity_id']}"
+            )
+
+            # 2. Templated service call - verify the temporary string issue is fixed
+            templated_call = await asyncio.wait_for(
+                service_call_futures["light.turn_on"], timeout=2.0
+            )
+            assert templated_call.service == "light.turn_on"
+            # Check the computed brightness value
+            assert "brightness" in templated_call.data
+            assert templated_call.data["brightness"] in ["191", "192"]  # 75 * 255 / 100
+            # Check data_template
+            assert "color_name" in templated_call.data_template
+            assert templated_call.data_template["color_name"] == "test_value"
+            # Check variables
+            assert "transition" in templated_call.variables
+            assert templated_call.variables["transition"] == "2.5"
+
+            # 3. Empty string service call
+            empty_call = await asyncio.wait_for(
+                service_call_futures["notify.test"], timeout=2.0
+            )
+            assert empty_call.service == "notify.test"
+            # Verify empty strings are properly handled
+            assert "title" in empty_call.data and empty_call.data["title"] == ""
+            assert (
+                "target" in empty_call.data_template
+                and empty_call.data_template["target"] == ""
+            )
+            assert (
+                "sound" in empty_call.variables and empty_call.variables["sound"] == ""
+            )
+
+            # 4. Multiple fields service call
+            multi_call = await asyncio.wait_for(
+                service_call_futures["climate.set_temperature"], timeout=2.0
+            )
+            assert multi_call.service == "climate.set_temperature"
+            assert multi_call.data["temperature"] == "22"
+            assert multi_call.data["hvac_mode"] == "heat"
+            assert multi_call.data_template["target_temp_high"] == "24"
+            assert multi_call.variables["preset_mode"] == "comfort"
+
+            # 5. Complex lambda service call
+            complex_call = await asyncio.wait_for(
+                service_call_futures["script.test_script"], timeout=2.0
+            )
+            assert complex_call.service == "script.test_script"
+            assert complex_call.data["entity_id"] == "light.living_room"
+            assert complex_call.data["brightness_pct"] == "99"  # 42 * 2.38 ≈ 99
+            # Check message includes sensor value
+            assert "message" in complex_call.data_template
+            assert "Sensor: 42.0" in complex_call.data_template["message"]
+
+            # 6. All empty service call
+            all_empty_call = await asyncio.wait_for(
+                service_call_futures["test.empty"], timeout=2.0
+            )
+            assert all_empty_call.service == "test.empty"
+            # All fields should be empty strings
+            assert all(v == "" for v in all_empty_call.data.values())
+            assert all(v == "" for v in all_empty_call.data_template.values())
+            assert all(v == "" for v in all_empty_call.variables.values())
+
+            # 7. HA Number service call
+            number_call = await asyncio.wait_for(
+                service_call_futures["input_number.set_value"], timeout=2.0
+            )
+            assert number_call.service == "input_number.set_value"
+            assert number_call.data["entity_id"] == "input_number.test_number"
+            # The value might be formatted with trailing zeros
+            assert float(number_call.data["value"]) == 42.5
+
+            # 8. HA Switch service calls
+            switch_on_call = await asyncio.wait_for(
+                service_call_futures["switch.turn_on"], timeout=2.0
+            )
+            assert switch_on_call.service == "switch.turn_on"
+            assert switch_on_call.data["entity_id"] == "switch.test_switch"
+
+            switch_off_call = await asyncio.wait_for(
+                service_call_futures["switch.turn_off"], timeout=2.0
+            )
+            assert switch_off_call.service == "switch.turn_off"
+            assert switch_off_call.data["entity_id"] == "switch.test_switch"
+
+        except TimeoutError as e:
+            # Show recent log lines for debugging
+            recent_logs = "\n".join(log_lines[-20:])
+            service_calls_summary = "\n".join(
+                f"- {call.service}" for call in ha_service_calls
+            )
+            pytest.fail(
+                f"Test timed out waiting for expected log pattern or service call. Error: {e}\n\n"
+                f"Recent log lines:\n{recent_logs}\n\n"
+                f"Received service calls:\n{service_calls_summary}"
+            )
diff --git a/tests/integration/test_api_message_size_batching.py b/tests/integration/test_api_message_size_batching.py
index 631e64825e..f7859eb902 100644
--- a/tests/integration/test_api_message_size_batching.py
+++ b/tests/integration/test_api_message_size_batching.py
@@ -177,7 +177,7 @@ async def test_api_message_size_batching(
         # Wait for states with timeout
         try:
             await asyncio.wait_for(states_future, timeout=5.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             missing_keys = expected_keys - received_keys
             pytest.fail(
                 f"Did not receive states from all entities within 5 seconds. "
diff --git a/tests/integration/test_api_reboot_timeout.py b/tests/integration/test_api_reboot_timeout.py
index dd9f5fbd1e..9cada0a296 100644
--- a/tests/integration/test_api_reboot_timeout.py
+++ b/tests/integration/test_api_reboot_timeout.py
@@ -29,7 +29,7 @@ async def test_api_reboot_timeout(
         # (0.5s reboot timeout + some margin for processing)
         try:
             await asyncio.wait_for(reboot_future, timeout=2.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Device did not reboot within expected timeout")
 
     # Test passes if we get here - reboot was detected
diff --git a/tests/integration/test_api_string_lambda.py b/tests/integration/test_api_string_lambda.py
new file mode 100644
index 0000000000..f4ef77bad8
--- /dev/null
+++ b/tests/integration/test_api_string_lambda.py
@@ -0,0 +1,100 @@
+"""Integration test for TemplatableStringValue with string lambdas."""
+
+from __future__ import annotations
+
+import asyncio
+import re
+
+import pytest
+
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_api_string_lambda(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Test TemplatableStringValue works with lambdas that return different types."""
+    loop = asyncio.get_running_loop()
+
+    # Track log messages for all four service calls
+    string_called_future = loop.create_future()
+    int_called_future = loop.create_future()
+    float_called_future = loop.create_future()
+    char_ptr_called_future = loop.create_future()
+
+    # Patterns to match in logs - confirms the lambdas compiled and executed
+    string_pattern = re.compile(r"Service called with string: STRING_FROM_LAMBDA")
+    int_pattern = re.compile(r"Service called with int: 42")
+    float_pattern = re.compile(r"Service called with float: 3\.14")
+    char_ptr_pattern = re.compile(r"Service called with number for char\* test: 123")
+
+    def check_output(line: str) -> None:
+        """Check log output for expected messages."""
+        if not string_called_future.done() and string_pattern.search(line):
+            string_called_future.set_result(True)
+        if not int_called_future.done() and int_pattern.search(line):
+            int_called_future.set_result(True)
+        if not float_called_future.done() and float_pattern.search(line):
+            float_called_future.set_result(True)
+        if not char_ptr_called_future.done() and char_ptr_pattern.search(line):
+            char_ptr_called_future.set_result(True)
+
+    # Run with log monitoring
+    async with (
+        run_compiled(yaml_config, line_callback=check_output),
+        api_client_connected() as client,
+    ):
+        # Verify device info
+        device_info = await client.device_info()
+        assert device_info is not None
+        assert device_info.name == "api-string-lambda-test"
+
+        # List services to find our test services
+        _, services = await client.list_entities_services()
+
+        # Find all test services
+        string_service = next(
+            (s for s in services if s.name == "test_string_lambda"), None
+        )
+        assert string_service is not None, "test_string_lambda service not found"
+
+        int_service = next((s for s in services if s.name == "test_int_lambda"), None)
+        assert int_service is not None, "test_int_lambda service not found"
+
+        float_service = next(
+            (s for s in services if s.name == "test_float_lambda"), None
+        )
+        assert float_service is not None, "test_float_lambda service not found"
+
+        char_ptr_service = next(
+            (s for s in services if s.name == "test_char_ptr_lambda"), None
+        )
+        assert char_ptr_service is not None, "test_char_ptr_lambda service not found"
+
+        # Execute all four services to test different lambda return types
+        client.execute_service(string_service, {"input_string": "STRING_FROM_LAMBDA"})
+        client.execute_service(int_service, {"input_number": 42})
+        client.execute_service(float_service, {"input_float": 3.14})
+        client.execute_service(
+            char_ptr_service, {"input_number": 123, "input_string": "test_string"}
+        )
+
+        # Wait for all service log messages
+        # This confirms the lambdas compiled successfully and executed
+        try:
+            await asyncio.wait_for(
+                asyncio.gather(
+                    string_called_future,
+                    int_called_future,
+                    float_called_future,
+                    char_ptr_called_future,
+                ),
+                timeout=5.0,
+            )
+        except TimeoutError:
+            pytest.fail(
+                "One or more service log messages not received - lambda may have failed to compile or execute"
+            )
diff --git a/tests/integration/test_areas_and_devices.py b/tests/integration/test_areas_and_devices.py
index 4184255724..93326de0a9 100644
--- a/tests/integration/test_areas_and_devices.py
+++ b/tests/integration/test_areas_and_devices.py
@@ -4,7 +4,7 @@ from __future__ import annotations
 
 import asyncio
 
-from aioesphomeapi import EntityState
+from aioesphomeapi import EntityState, SwitchInfo, SwitchState
 import pytest
 
 from .types import APIClientConnectedFactory, RunCompiledFunction
@@ -84,23 +84,45 @@ async def test_areas_and_devices(
 
         # Subscribe to states to get sensor values
         loop = asyncio.get_running_loop()
-        states: dict[int, EntityState] = {}
-        states_future: asyncio.Future[bool] = loop.create_future()
+        states: dict[tuple[int, int], EntityState] = {}
+        # Subscribe to all switch states
+        switch_state_futures: dict[
+            tuple[int, int], asyncio.Future[EntityState]
+        ] = {}  # (device_id, key) -> future
+        initial_states_received: set[tuple[int, int]] = set()
+        initial_states_future: asyncio.Future[bool] = loop.create_future()
 
         def on_state(state: EntityState) -> None:
-            states[state.key] = state
-            # Check if we have all expected sensor states
-            if len(states) >= 4 and not states_future.done():
-                states_future.set_result(True)
+            state_key = (state.device_id, state.key)
+            states[state_key] = state
+
+            initial_states_received.add(state_key)
+            # Check if we have all initial states
+            if (
+                len(initial_states_received) >= 8  # 8 entities expected
+                and not initial_states_future.done()
+            ):
+                initial_states_future.set_result(True)
+
+            if not initial_states_future.done():
+                return
+
+            # Resolve the future for this switch if it exists
+            if (
+                state_key in switch_state_futures
+                and not switch_state_futures[state_key].done()
+                and isinstance(state, SwitchState)
+            ):
+                switch_state_futures[state_key].set_result(state)
 
         client.subscribe_states(on_state)
 
         # Wait for sensor states
         try:
-            await asyncio.wait_for(states_future, timeout=10.0)
-        except asyncio.TimeoutError:
+            await asyncio.wait_for(initial_states_future, timeout=10.0)
+        except TimeoutError:
             pytest.fail(
-                f"Did not receive all sensor states within 10 seconds. "
+                f"Did not receive all states within 10 seconds. "
                 f"Received {len(states)} states"
             )
 
@@ -110,6 +132,7 @@ async def test_areas_and_devices(
             "Temperature Sensor Reading": temp_sensor.device_id,
             "Motion Detector Status": motion_detector.device_id,
             "Smart Switch Power": smart_switch.device_id,
+            "Living Room Sensor": 0,  # Main device
         }
 
         for entity in sensor_entities:
@@ -119,3 +142,133 @@ async def test_areas_and_devices(
                     f"{entity.name} has device_id {entity.device_id}, "
                     f"expected {expected_device_id}"
                 )
+
+        all_entities, _ = entities  # Unpack the tuple
+        switch_entities = [e for e in all_entities if isinstance(e, SwitchInfo)]
+
+        # Find all switches named "Test Switch"
+        test_switches = [e for e in switch_entities if e.name == "Test Switch"]
+        assert len(test_switches) == 4, (
+            f"Expected 4 'Test Switch' entities, got {len(test_switches)}"
+        )
+
+        # Verify we have switches with different device_ids including one with 0 (main)
+        switch_device_ids = {s.device_id for s in test_switches}
+        assert len(switch_device_ids) == 4, (
+            "All Test Switch entities should have different device_ids"
+        )
+        assert 0 in switch_device_ids, (
+            "Should have a switch with device_id 0 (main device)"
+        )
+
+        # Verify extra switches with blank and none device_id are correctly available
+        extra_switches = [
+            e for e in switch_entities if e.name.startswith("Living Room")
+        ]
+        assert len(extra_switches) == 2, (
+            f"Expected 2 extra switches for Living Room, got {len(extra_switches)}"
+        )
+        extra_switch_device_ids = [e.device_id for e in extra_switches]
+        assert all(d == 0 for d in extra_switch_device_ids), (
+            "All extra switches should have device_id 0 (main device)"
+        )
+
+        # Wait for initial states to be received for all switches
+        await asyncio.wait_for(initial_states_future, timeout=2.0)
+
+        # Test controlling each switch specifically by device_id
+        for device_name, device in [
+            ("Light Controller", light_controller),
+            ("Temperature Sensor", temp_sensor),
+            ("Motion Detector", motion_detector),
+        ]:
+            # Find the switch for this specific device
+            device_switch = next(
+                (s for s in test_switches if s.device_id == device.device_id), None
+            )
+            assert device_switch is not None, f"No Test Switch found for {device_name}"
+
+            # Create future for this switch's state change
+            state_key = (device_switch.device_id, device_switch.key)
+            switch_state_futures[state_key] = loop.create_future()
+
+            # Turn on the switch with device_id
+            client.switch_command(
+                device_switch.key, True, device_id=device_switch.device_id
+            )
+
+            # Wait for state to change
+            await asyncio.wait_for(switch_state_futures[state_key], timeout=2.0)
+
+            # Verify the correct switch was turned on
+            assert states[state_key].state is True, f"{device_name} switch should be on"
+
+            # Create new future for turning off
+            switch_state_futures[state_key] = loop.create_future()
+
+            # Turn off the switch with device_id
+            client.switch_command(
+                device_switch.key, False, device_id=device_switch.device_id
+            )
+
+            # Wait for state to change
+            await asyncio.wait_for(switch_state_futures[state_key], timeout=2.0)
+
+            # Verify the correct switch was turned off
+            assert states[state_key].state is False, (
+                f"{device_name} switch should be off"
+            )
+
+        # Test that controlling a switch with device_id doesn't affect main switch
+        # Find the main switch (device_id = 0)
+        main_switch = next((s for s in test_switches if s.device_id == 0), None)
+        assert main_switch is not None, "No main switch (device_id=0) found"
+
+        # Find a switch with a device_id
+        device_switch = next(
+            (s for s in test_switches if s.device_id == light_controller.device_id),
+            None,
+        )
+        assert device_switch is not None, "No device switch found"
+
+        # Create futures for both switches
+        main_key = (main_switch.device_id, main_switch.key)
+        device_key = (device_switch.device_id, device_switch.key)
+
+        # Turn on the main switch first
+        switch_state_futures[main_key] = loop.create_future()
+        client.switch_command(main_switch.key, True, device_id=main_switch.device_id)
+        await asyncio.wait_for(switch_state_futures[main_key], timeout=2.0)
+        assert states[main_key].state is True, "Main switch should be on"
+
+        # Now turn on the device switch
+        switch_state_futures[device_key] = loop.create_future()
+        client.switch_command(
+            device_switch.key, True, device_id=device_switch.device_id
+        )
+        await asyncio.wait_for(switch_state_futures[device_key], timeout=2.0)
+
+        # Verify device switch is on and main switch is still on
+        assert states[device_key].state is True, "Device switch should be on"
+        assert states[main_key].state is True, (
+            "Main switch should still be on after turning on device switch"
+        )
+
+        # Turn off the device switch
+        switch_state_futures[device_key] = loop.create_future()
+        client.switch_command(
+            device_switch.key, False, device_id=device_switch.device_id
+        )
+        await asyncio.wait_for(switch_state_futures[device_key], timeout=2.0)
+
+        # Verify device switch is off and main switch is still on
+        assert states[device_key].state is False, "Device switch should be off"
+        assert states[main_key].state is True, (
+            "Main switch should still be on after turning off device switch"
+        )
+
+        # Clean up - turn off main switch
+        switch_state_futures[main_key] = loop.create_future()
+        client.switch_command(main_switch.key, False, device_id=main_switch.device_id)
+        await asyncio.wait_for(switch_state_futures[main_key], timeout=2.0)
+        assert states[main_key].state is False, "Main switch should be off"
diff --git a/tests/integration/test_automations.py b/tests/integration/test_automations.py
new file mode 100644
index 0000000000..83268c1eea
--- /dev/null
+++ b/tests/integration/test_automations.py
@@ -0,0 +1,161 @@
+"""Test ESPHome automations functionality."""
+
+from __future__ import annotations
+
+import asyncio
+import re
+
+import pytest
+
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_delay_action_cancellation(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Test that delay actions can be properly cancelled when script restarts."""
+    loop = asyncio.get_running_loop()
+
+    # Track log messages with timestamps
+    log_entries: list[tuple[float, str]] = []
+    script_starts: list[float] = []
+    delay_completions: list[float] = []
+    script_restart_logged = False
+    test_started_time = None
+
+    # Patterns to match
+    test_start_pattern = re.compile(r"Starting first script execution")
+    script_start_pattern = re.compile(r"Script started, beginning delay")
+    restart_pattern = re.compile(r"Restarting script \(should cancel first delay\)")
+    delay_complete_pattern = re.compile(r"Delay completed successfully")
+
+    # Future to track when we can check results
+    second_script_started = loop.create_future()
+
+    def check_output(line: str) -> None:
+        """Check log output for expected messages."""
+        nonlocal script_restart_logged, test_started_time
+
+        current_time = loop.time()
+        log_entries.append((current_time, line))
+
+        if test_start_pattern.search(line):
+            test_started_time = current_time
+        elif script_start_pattern.search(line) and test_started_time:
+            script_starts.append(current_time)
+            if len(script_starts) == 2 and not second_script_started.done():
+                second_script_started.set_result(True)
+        elif restart_pattern.search(line):
+            script_restart_logged = True
+        elif delay_complete_pattern.search(line):
+            delay_completions.append(current_time)
+
+    async with (
+        run_compiled(yaml_config, line_callback=check_output),
+        api_client_connected() as client,
+    ):
+        # Get services
+        entities, services = await client.list_entities_services()
+
+        # Find our test service
+        test_service = next(
+            (s for s in services if s.name == "start_delay_then_restart"), None
+        )
+        assert test_service is not None, "start_delay_then_restart service not found"
+
+        # Execute the test sequence
+        client.execute_service(test_service, {})
+
+        # Wait for the second script to start
+        await asyncio.wait_for(second_script_started, timeout=5.0)
+
+        # Wait for potential delay completion
+        await asyncio.sleep(0.75)  # Original delay was 500ms
+
+        # Check results
+        assert len(script_starts) == 2, (
+            f"Script should have started twice, but started {len(script_starts)} times"
+        )
+        assert script_restart_logged, "Script restart was not logged"
+
+        # Verify we got exactly one completion and it happened ~500ms after the second start
+        assert len(delay_completions) == 1, (
+            f"Expected 1 delay completion, got {len(delay_completions)}"
+        )
+        time_from_second_start = delay_completions[0] - script_starts[1]
+        assert 0.4 < time_from_second_start < 0.6, (
+            f"Delay completed {time_from_second_start:.3f}s after second start, expected ~0.5s"
+        )
+
+
+@pytest.mark.asyncio
+async def test_parallel_script_delays(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Test that parallel scripts with delays don't interfere with each other."""
+    loop = asyncio.get_running_loop()
+
+    # Track script executions
+    script_starts: list[float] = []
+    script_ends: list[float] = []
+
+    # Patterns to match
+    start_pattern = re.compile(r"Parallel script instance \d+ started")
+    end_pattern = re.compile(r"Parallel script instance \d+ completed after delay")
+
+    # Future to track when all scripts have completed
+    all_scripts_completed = loop.create_future()
+
+    def check_output(line: str) -> None:
+        """Check log output for parallel script messages."""
+        current_time = loop.time()
+
+        if start_pattern.search(line):
+            script_starts.append(current_time)
+
+        if end_pattern.search(line):
+            script_ends.append(current_time)
+            # Check if we have all 3 completions
+            if len(script_ends) == 3 and not all_scripts_completed.done():
+                all_scripts_completed.set_result(True)
+
+    async with (
+        run_compiled(yaml_config, line_callback=check_output),
+        api_client_connected() as client,
+    ):
+        # Get services
+        entities, services = await client.list_entities_services()
+
+        # Find our test service
+        test_service = next(
+            (s for s in services if s.name == "test_parallel_delays"), None
+        )
+        assert test_service is not None, "test_parallel_delays service not found"
+
+        # Execute the test - this will start 3 parallel scripts with 1 second delays
+        client.execute_service(test_service, {})
+
+        # Wait for all scripts to complete (should take ~1 second, not 3)
+        await asyncio.wait_for(all_scripts_completed, timeout=2.0)
+
+        # Verify we had 3 starts and 3 ends
+        assert len(script_starts) == 3, (
+            f"Expected 3 script starts, got {len(script_starts)}"
+        )
+        assert len(script_ends) == 3, f"Expected 3 script ends, got {len(script_ends)}"
+
+        # Verify they ran in parallel - all should complete within ~1.5 seconds
+        first_start = min(script_starts)
+        last_end = max(script_ends)
+        total_time = last_end - first_start
+
+        # If running in parallel, total time should be close to 1 second
+        # If they were interfering (running sequentially), it would take 3+ seconds
+        assert total_time < 1.5, (
+            f"Parallel scripts took {total_time:.2f}s total, should be ~1s if running in parallel"
+        )
diff --git a/tests/integration/test_crc8_helper.py b/tests/integration/test_crc8_helper.py
new file mode 100644
index 0000000000..ffe6244598
--- /dev/null
+++ b/tests/integration/test_crc8_helper.py
@@ -0,0 +1,100 @@
+"""Integration test for CRC8 helper function."""
+
+from __future__ import annotations
+
+import asyncio
+from pathlib import Path
+
+import pytest
+
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_crc8_helper(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Test the CRC8 helper function through integration testing."""
+    # Get the path to the external components directory
+    external_components_path = str(
+        Path(__file__).parent / "fixtures" / "external_components"
+    )
+
+    # Replace the placeholder in the YAML config with the actual path
+    yaml_config = yaml_config.replace(
+        "EXTERNAL_COMPONENT_PATH", external_components_path
+    )
+
+    # Track test completion with asyncio.Event
+    test_complete = asyncio.Event()
+
+    # Track test results
+    test_results = {
+        "dallas_maxim": False,
+        "sensirion": False,
+        "pec": False,
+        "parameter_equivalence": False,
+        "edge_cases": False,
+        "component_compatibility": False,
+        "setup_started": False,
+    }
+
+    def on_log_line(line):
+        """Process log lines to track test progress and results."""
+        # Track test start
+        if "CRC8 Helper Function Integration Test Starting" in line:
+            test_results["setup_started"] = True
+
+        # Track test completion
+        elif "CRC8 Integration Test Complete" in line:
+            test_complete.set()
+
+        # Track individual test results
+        elif "ALL TESTS PASSED" in line:
+            if "Dallas/Maxim CRC8" in line:
+                test_results["dallas_maxim"] = True
+            elif "Sensirion CRC8" in line:
+                test_results["sensirion"] = True
+            elif "PEC CRC8" in line:
+                test_results["pec"] = True
+            elif "Parameter equivalence" in line:
+                test_results["parameter_equivalence"] = True
+            elif "Edge cases" in line:
+                test_results["edge_cases"] = True
+            elif "Component compatibility" in line:
+                test_results["component_compatibility"] = True
+
+        # Log failures for debugging
+        elif "TEST FAILED:" in line or "SUBTEST FAILED:" in line:
+            print(f"CRC8 Test Failure: {line}")
+
+    # Compile and run the test
+    async with (
+        run_compiled(yaml_config, line_callback=on_log_line),
+        api_client_connected() as client,
+    ):
+        # Verify device info
+        device_info = await client.device_info()
+        assert device_info is not None
+        assert device_info.name == "crc8-helper-test"
+
+        # Wait for tests to complete with timeout
+        try:
+            await asyncio.wait_for(test_complete.wait(), timeout=5.0)
+        except TimeoutError:
+            pytest.fail("CRC8 integration test timed out after 5 seconds")
+
+        # Verify all tests passed
+        assert test_results["setup_started"], "CRC8 test setup never started"
+        assert test_results["dallas_maxim"], "Dallas/Maxim CRC8 test failed"
+        assert test_results["sensirion"], "Sensirion CRC8 test failed"
+        assert test_results["pec"], "PEC CRC8 test failed"
+        assert test_results["parameter_equivalence"], (
+            "Parameter equivalence test failed"
+        )
+        assert test_results["edge_cases"], "Edge cases test failed"
+        assert test_results["component_compatibility"], (
+            "Component compatibility test failed"
+        )
diff --git a/tests/integration/test_device_id_in_state.py b/tests/integration/test_device_id_in_state.py
index eaa91ec92e..51088bcbf7 100644
--- a/tests/integration/test_device_id_in_state.py
+++ b/tests/integration/test_device_id_in_state.py
@@ -47,9 +47,7 @@ async def test_device_id_in_state(
                 entity_device_mapping[entity.key] = device_ids["Humidity Monitor"]
             elif entity.name == "Motion Detected":
                 entity_device_mapping[entity.key] = device_ids["Motion Sensor"]
-            elif entity.name == "Temperature Monitor Power":
-                entity_device_mapping[entity.key] = device_ids["Temperature Monitor"]
-            elif entity.name == "Temperature Status":
+            elif entity.name in {"Temperature Monitor Power", "Temperature Status"}:
                 entity_device_mapping[entity.key] = device_ids["Temperature Monitor"]
             elif entity.name == "Motion Light":
                 entity_device_mapping[entity.key] = device_ids["Motion Sensor"]
@@ -77,7 +75,7 @@ async def test_device_id_in_state(
         # Wait for states
         try:
             await asyncio.wait_for(states_future, timeout=10.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(
                 f"Did not receive all entity states within 10 seconds. "
                 f"Received {len(states)} states, expected {len(entity_device_mapping)}"
diff --git a/tests/integration/test_duplicate_entities.py b/tests/integration/test_duplicate_entities.py
index b7ee8dd478..9b2670c4ae 100644
--- a/tests/integration/test_duplicate_entities.py
+++ b/tests/integration/test_duplicate_entities.py
@@ -11,12 +11,12 @@ from .types import APIClientConnectedFactory, RunCompiledFunction
 
 
 @pytest.mark.asyncio
-async def test_duplicate_entities_not_allowed_on_different_devices(
+async def test_duplicate_entities_on_different_devices(
     yaml_config: str,
     run_compiled: RunCompiledFunction,
     api_client_connected: APIClientConnectedFactory,
 ) -> None:
-    """Test that duplicate entity names are NOT allowed on different devices."""
+    """Test that duplicate entity names are allowed on different devices."""
     async with run_compiled(yaml_config), api_client_connected() as client:
         # Get device info
         device_info = await client.device_info()
@@ -38,8 +38,7 @@ async def test_duplicate_entities_not_allowed_on_different_devices(
         # Get entity list
         entities = await client.list_entities_services()
         all_entities: list[EntityInfo] = []
-        for entity_list in entities[0]:
-            all_entities.append(entity_list)
+        all_entities.extend(entities[0])
 
         # Group entities by type for easier testing
         sensors = [e for e in all_entities if e.__class__.__name__ == "SensorInfo"]
@@ -52,46 +51,42 @@ async def test_duplicate_entities_not_allowed_on_different_devices(
         switches = [e for e in all_entities if e.__class__.__name__ == "SwitchInfo"]
         buttons = [e for e in all_entities if e.__class__.__name__ == "ButtonInfo"]
         numbers = [e for e in all_entities if e.__class__.__name__ == "NumberInfo"]
-        selects = [e for e in all_entities if e.__class__.__name__ == "SelectInfo"]
 
-        # Scenario 1: Check that temperature sensors have unique names per device
-        temp_sensors = [s for s in sensors if "Temperature" in s.name]
+        # Scenario 1: Check sensors with same "Temperature" name on different devices
+        temp_sensors = [s for s in sensors if s.name == "Temperature"]
         assert len(temp_sensors) == 4, (
             f"Expected exactly 4 temperature sensors, got {len(temp_sensors)}"
         )
 
-        # Verify each sensor has a unique name
-        temp_names = set()
+        # Verify each sensor is on a different device
+        temp_device_ids = set()
         temp_object_ids = set()
 
         for sensor in temp_sensors:
-            temp_names.add(sensor.name)
+            temp_device_ids.add(sensor.device_id)
             temp_object_ids.add(sensor.object_id)
 
-        # Should have 4 unique names
-        assert len(temp_names) == 4, (
-            f"Temperature sensors should have unique names, got {temp_names}"
+            # All should have object_id "temperature" (no suffix)
+            assert sensor.object_id == "temperature", (
+                f"Expected object_id 'temperature', got '{sensor.object_id}'"
+            )
+
+        # Should have 4 different device IDs (including None for main device)
+        assert len(temp_device_ids) == 4, (
+            f"Temperature sensors should be on different devices, got {temp_device_ids}"
         )
 
-        # Object IDs should also be unique
-        assert len(temp_object_ids) == 4, (
-            f"Temperature sensors should have unique object_ids, got {temp_object_ids}"
-        )
-
-        # Scenario 2: Check binary sensors have unique names
-        status_binary = [b for b in binary_sensors if "Status" in b.name]
+        # Scenario 2: Check binary sensors "Status" on different devices
+        status_binary = [b for b in binary_sensors if b.name == "Status"]
         assert len(status_binary) == 3, (
             f"Expected exactly 3 status binary sensors, got {len(status_binary)}"
         )
 
-        # All should have unique object_ids
-        status_names = set()
+        # All should have object_id "status"
         for binary in status_binary:
-            status_names.add(binary.name)
-
-        assert len(status_names) == 3, (
-            f"Status binary sensors should have unique names, got {status_names}"
-        )
+            assert binary.object_id == "status", (
+                f"Expected object_id 'status', got '{binary.object_id}'"
+            )
 
         # Scenario 3: Check that sensor and binary_sensor can have same name
         temp_binary = [b for b in binary_sensors if b.name == "Temperature"]
@@ -100,86 +95,62 @@ async def test_duplicate_entities_not_allowed_on_different_devices(
         )
         assert temp_binary[0].object_id == "temperature"
 
-        # Scenario 4: Check text sensors have unique names
-        info_text = [t for t in text_sensors if "Device Info" in t.name]
+        # Scenario 4: Check text sensors "Device Info" on different devices
+        info_text = [t for t in text_sensors if t.name == "Device Info"]
         assert len(info_text) == 3, (
             f"Expected exactly 3 device info text sensors, got {len(info_text)}"
         )
 
-        # All should have unique names and object_ids
-        info_names = set()
+        # All should have object_id "device_info"
         for text in info_text:
-            info_names.add(text.name)
+            assert text.object_id == "device_info", (
+                f"Expected object_id 'device_info', got '{text.object_id}'"
+            )
 
-        assert len(info_names) == 3, (
-            f"Device info text sensors should have unique names, got {info_names}"
+        # Scenario 5: Check switches "Power" on different devices
+        power_switches = [s for s in switches if s.name == "Power"]
+        assert len(power_switches) == 3, (
+            f"Expected exactly 3 power switches, got {len(power_switches)}"
         )
 
-        # Scenario 5: Check switches have unique names
-        power_switches = [s for s in switches if "Power" in s.name]
-        assert len(power_switches) == 4, (
-            f"Expected exactly 4 power switches, got {len(power_switches)}"
-        )
-
-        # All should have unique names
-        power_names = set()
+        # All should have object_id "power"
         for switch in power_switches:
-            power_names.add(switch.name)
+            assert switch.object_id == "power", (
+                f"Expected object_id 'power', got '{switch.object_id}'"
+            )
 
-        assert len(power_names) == 4, (
-            f"Power switches should have unique names, got {power_names}"
-        )
-
-        # Scenario 6: Check reset buttons have unique names
-        reset_buttons = [b for b in buttons if "Reset" in b.name]
-        assert len(reset_buttons) == 3, (
-            f"Expected exactly 3 reset buttons, got {len(reset_buttons)}"
-        )
-
-        # All should have unique names
-        reset_names = set()
-        for button in reset_buttons:
-            reset_names.add(button.name)
-
-        assert len(reset_names) == 3, (
-            f"Reset buttons should have unique names, got {reset_names}"
-        )
-
-        # Scenario 7: Check empty name selects (should use device names)
-        empty_selects = [s for s in selects if s.name == ""]
-        assert len(empty_selects) == 3, (
-            f"Expected exactly 3 empty name selects, got {len(empty_selects)}"
+        # Scenario 6: Check empty name buttons (should use device name)
+        empty_buttons = [b for b in buttons if b.name == ""]
+        assert len(empty_buttons) == 3, (
+            f"Expected exactly 3 empty name buttons, got {len(empty_buttons)}"
         )
 
         # Group by device
-        c1_selects = [s for s in empty_selects if s.device_id == controller_1.device_id]
-        c2_selects = [s for s in empty_selects if s.device_id == controller_2.device_id]
+        c1_buttons = [b for b in empty_buttons if b.device_id == controller_1.device_id]
+        c2_buttons = [b for b in empty_buttons if b.device_id == controller_2.device_id]
 
         # For main device, device_id is 0
-        main_selects = [s for s in empty_selects if s.device_id == 0]
+        main_buttons = [b for b in empty_buttons if b.device_id == 0]
 
-        # Check object IDs for empty name entities - they should use device names
-        assert len(c1_selects) == 1 and c1_selects[0].object_id == "controller_1"
-        assert len(c2_selects) == 1 and c2_selects[0].object_id == "controller_2"
+        # Check object IDs for empty name entities
+        assert len(c1_buttons) == 1 and c1_buttons[0].object_id == "controller_1"
+        assert len(c2_buttons) == 1 and c2_buttons[0].object_id == "controller_2"
         assert (
-            len(main_selects) == 1
-            and main_selects[0].object_id == "duplicate-entities-test"
+            len(main_buttons) == 1
+            and main_buttons[0].object_id == "duplicate-entities-test"
         )
 
-        # Scenario 8: Check special characters in number names - now unique
-        temp_numbers = [n for n in numbers if "Temperature Setpoint!" in n.name]
+        # Scenario 7: Check special characters in number names
+        temp_numbers = [n for n in numbers if n.name == "Temperature Setpoint!"]
         assert len(temp_numbers) == 2, (
             f"Expected exactly 2 temperature setpoint numbers, got {len(temp_numbers)}"
         )
 
-        # Should have unique names
-        setpoint_names = set()
+        # Special characters should be sanitized to _ in object_id
         for number in temp_numbers:
-            setpoint_names.add(number.name)
-
-        assert len(setpoint_names) == 2, (
-            f"Temperature setpoint numbers should have unique names, got {setpoint_names}"
-        )
+            assert number.object_id == "temperature_setpoint_", (
+                f"Expected object_id 'temperature_setpoint_', got '{number.object_id}'"
+            )
 
         # Verify we can get states for all entities (ensures they're functional)
         loop = asyncio.get_running_loop()
@@ -192,7 +163,6 @@ async def test_duplicate_entities_not_allowed_on_different_devices(
             + len(switches)
             + len(buttons)
             + len(numbers)
-            + len(selects)
         )
 
         def on_state(state) -> None:
@@ -206,7 +176,7 @@ async def test_duplicate_entities_not_allowed_on_different_devices(
         # Wait for all entity states
         try:
             await asyncio.wait_for(states_future, timeout=10.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(
                 f"Did not receive all entity states within 10 seconds. "
                 f"Expected {expected_count}, received {state_count}"
diff --git a/tests/integration/test_entity_icon.py b/tests/integration/test_entity_icon.py
index aec7168165..a634ae385e 100644
--- a/tests/integration/test_entity_icon.py
+++ b/tests/integration/test_entity_icon.py
@@ -82,7 +82,7 @@ async def test_entity_icon(
         # Wait for states
         try:
             await asyncio.wait_for(state_received.wait(), timeout=5.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("No states received within 5 seconds")
 
         # Verify we received states
diff --git a/tests/integration/test_gpio_expander_cache.py b/tests/integration/test_gpio_expander_cache.py
new file mode 100644
index 0000000000..e5f0f2818f
--- /dev/null
+++ b/tests/integration/test_gpio_expander_cache.py
@@ -0,0 +1,146 @@
+"""Integration test for CachedGPIOExpander to ensure correct behavior."""
+
+from __future__ import annotations
+
+import asyncio
+from pathlib import Path
+import re
+
+import pytest
+
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_gpio_expander_cache(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Test gpio_expander::CachedGpioExpander correctly calls hardware functions."""
+    # Get the path to the external components directory
+    external_components_path = str(
+        Path(__file__).parent / "fixtures" / "external_components"
+    )
+
+    # Replace the placeholder in the YAML config with the actual path
+    yaml_config = yaml_config.replace(
+        "EXTERNAL_COMPONENT_PATH", external_components_path
+    )
+
+    logs_done = asyncio.Event()
+
+    # Patterns to match in logs - match any variation of digital_read
+    read_hw_pattern = re.compile(r"(?:uint16_)?digital_read_hw pin=(\d+)")
+    read_cache_pattern = re.compile(r"(?:uint16_)?digital_read_cache pin=(\d+)")
+
+    # Keep specific patterns for building the expected order
+    digital_read_hw_pattern = re.compile(r"^digital_read_hw pin=(\d+)")
+    digital_read_cache_pattern = re.compile(r"^digital_read_cache pin=(\d+)")
+    uint16_read_hw_pattern = re.compile(r"^uint16_digital_read_hw pin=(\d+)")
+    uint16_read_cache_pattern = re.compile(r"^uint16_digital_read_cache pin=(\d+)")
+
+    # ensure logs are in the expected order
+    log_order = [
+        (digital_read_hw_pattern, 0),
+        [(digital_read_cache_pattern, i) for i in range(0, 8)],
+        (digital_read_hw_pattern, 8),
+        [(digital_read_cache_pattern, i) for i in range(8, 16)],
+        (digital_read_hw_pattern, 16),
+        [(digital_read_cache_pattern, i) for i in range(16, 24)],
+        (digital_read_hw_pattern, 24),
+        [(digital_read_cache_pattern, i) for i in range(24, 32)],
+        (digital_read_hw_pattern, 3),
+        (digital_read_cache_pattern, 3),
+        (digital_read_hw_pattern, 3),
+        (digital_read_cache_pattern, 3),
+        (digital_read_cache_pattern, 4),
+        (digital_read_hw_pattern, 3),
+        (digital_read_cache_pattern, 3),
+        (digital_read_hw_pattern, 10),
+        (digital_read_cache_pattern, 10),
+        # full cache reset here for testing
+        (digital_read_hw_pattern, 15),
+        (digital_read_cache_pattern, 15),
+        (digital_read_cache_pattern, 14),
+        (digital_read_hw_pattern, 14),
+        (digital_read_cache_pattern, 14),
+        # uint16_t component tests (single bank of 16 pins)
+        (uint16_read_hw_pattern, 0),  # First pin triggers hw read
+        [
+            (uint16_read_cache_pattern, i) for i in range(0, 16)
+        ],  # All 16 pins return via cache
+        # After cache reset
+        (uint16_read_hw_pattern, 5),  # First read after reset triggers hw
+        (uint16_read_cache_pattern, 5),
+        (uint16_read_cache_pattern, 10),  # These use cache (same bank)
+        (uint16_read_cache_pattern, 15),
+        (uint16_read_cache_pattern, 0),
+    ]
+    # Flatten the log order for easier processing
+    log_order: list[tuple[re.Pattern, int]] = [
+        item
+        for sublist in log_order
+        for item in (sublist if isinstance(sublist, list) else [sublist])
+    ]
+
+    index = 0
+
+    def check_output(line: str) -> None:
+        """Check log output for expected messages."""
+        nonlocal index
+        if logs_done.is_set():
+            return
+
+        clean_line = re.sub(r"\x1b\[[0-9;]*m", "", line)
+
+        # Extract just the log message part (after the log level)
+        msg = clean_line.split(": ", 1)[-1] if ": " in clean_line else clean_line
+
+        # Check if this line contains a read operation we're tracking
+        if read_hw_pattern.search(msg) or read_cache_pattern.search(msg):
+            if index >= len(log_order):
+                print(f"Received unexpected log line: {msg}")
+                logs_done.set()
+                return
+
+            pattern, expected_pin = log_order[index]
+            match = pattern.search(msg)
+
+            if not match:
+                print(f"Log line did not match next expected pattern: {msg}")
+                print(f"Expected pattern: {pattern.pattern}")
+                logs_done.set()
+                return
+
+            pin = int(match.group(1))
+            if pin != expected_pin:
+                print(f"Unexpected pin number. Expected {expected_pin}, got {pin}")
+                logs_done.set()
+                return
+
+            index += 1
+
+        elif "DONE_UINT16" in clean_line:
+            # uint16 component is done, check if we've seen all expected logs
+            if index == len(log_order):
+                logs_done.set()
+
+    # Run with log monitoring
+    async with (
+        run_compiled(yaml_config, line_callback=check_output),
+        api_client_connected() as client,
+    ):
+        # Verify device info
+        device_info = await client.device_info()
+        assert device_info is not None
+        assert device_info.name == "gpio-expander-cache"
+
+        try:
+            await asyncio.wait_for(logs_done.wait(), timeout=5.0)
+        except TimeoutError:
+            pytest.fail("Timeout waiting for logs to complete")
+
+        assert index == len(log_order), (
+            f"Expected {len(log_order)} log entries, but got {index}"
+        )
diff --git a/tests/integration/test_host_mode_api_password.py b/tests/integration/test_host_mode_api_password.py
new file mode 100644
index 0000000000..5c5e689e45
--- /dev/null
+++ b/tests/integration/test_host_mode_api_password.py
@@ -0,0 +1,69 @@
+"""Integration test for API password authentication."""
+
+from __future__ import annotations
+
+import asyncio
+
+from aioesphomeapi import APIConnectionError, InvalidAuthAPIError
+import pytest
+
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_host_mode_api_password(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Test API authentication with password."""
+    async with run_compiled(yaml_config):
+        # Connect with correct password
+        async with api_client_connected(password="test_password_123") as client:
+            # Verify we can get device info
+            device_info = await client.device_info()
+            assert device_info is not None
+            assert device_info.uses_password is True
+            assert device_info.name == "host-mode-api-password"
+
+            # Subscribe to states to ensure authenticated connection works
+            loop = asyncio.get_running_loop()
+            state_future: asyncio.Future[bool] = loop.create_future()
+            states = {}
+
+            def on_state(state):
+                states[state.key] = state
+                if not state_future.done():
+                    state_future.set_result(True)
+
+            client.subscribe_states(on_state)
+
+            # Wait for at least one state with timeout
+            try:
+                await asyncio.wait_for(state_future, timeout=5.0)
+            except TimeoutError:
+                pytest.fail("No states received within timeout")
+
+            # Should have received at least one state (the test sensor)
+            assert len(states) > 0
+
+        # Test with wrong password - should fail
+        # Try connecting with wrong password
+        try:
+            async with api_client_connected(
+                password="wrong_password", timeout=5
+            ) as client:
+                # If we get here without exception, try to use the connection
+                # which should fail if auth failed
+                await client.device_info_and_list_entities()
+                # If we successfully got device info and entities, auth didn't fail properly
+                pytest.fail("Connection succeeded with wrong password")
+        except (InvalidAuthAPIError, APIConnectionError) as e:
+            # Expected - auth should fail
+            # Accept either InvalidAuthAPIError or generic APIConnectionError
+            # since the client might not always distinguish
+            assert (
+                "password" in str(e).lower()
+                or "auth" in str(e).lower()
+                or "invalid" in str(e).lower()
+            )
diff --git a/tests/integration/test_host_mode_batch_delay.py b/tests/integration/test_host_mode_batch_delay.py
index 5165b90e47..a3f666fa21 100644
--- a/tests/integration/test_host_mode_batch_delay.py
+++ b/tests/integration/test_host_mode_batch_delay.py
@@ -44,7 +44,7 @@ async def test_host_mode_batch_delay(
         # Wait for states from all entities with timeout
         try:
             entity_count = await asyncio.wait_for(entity_count_future, timeout=5.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(
                 f"Did not receive states from at least 7 entities within 5 seconds. "
                 f"Received {len(states)} states"
diff --git a/tests/integration/test_host_mode_empty_string_options.py b/tests/integration/test_host_mode_empty_string_options.py
index 16399dcfb8..242db2d40f 100644
--- a/tests/integration/test_host_mode_empty_string_options.py
+++ b/tests/integration/test_host_mode_empty_string_options.py
@@ -99,7 +99,7 @@ async def test_host_mode_empty_string_options(
         # Wait for initial states with timeout
         try:
             await asyncio.wait_for(states_received_future, timeout=5.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(
                 f"Did not receive states for all select entities. "
                 f"Expected keys: {expected_select_keys}, Received: {received_select_keys}"
diff --git a/tests/integration/test_host_mode_entity_fields.py b/tests/integration/test_host_mode_entity_fields.py
index b9fa3e9746..5ec1b64a99 100644
--- a/tests/integration/test_host_mode_entity_fields.py
+++ b/tests/integration/test_host_mode_entity_fields.py
@@ -86,7 +86,7 @@ async def test_host_mode_entity_fields(
         # Wait for at least one state
         try:
             await asyncio.wait_for(state_received.wait(), timeout=5.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("No states received within 5 seconds")
 
         # Verify we received states (which means has_state flag is working)
diff --git a/tests/integration/test_host_mode_fan_preset.py b/tests/integration/test_host_mode_fan_preset.py
index d18b9f08ad..d44fdc0a3b 100644
--- a/tests/integration/test_host_mode_fan_preset.py
+++ b/tests/integration/test_host_mode_fan_preset.py
@@ -23,9 +23,7 @@ async def test_host_mode_fan_preset(
         entities = await client.list_entities_services()
         fans: list[FanInfo] = []
         for entity_list in entities:
-            for entity in entity_list:
-                if isinstance(entity, FanInfo):
-                    fans.append(entity)
+            fans.extend(entity for entity in entity_list if isinstance(entity, FanInfo))
 
         # Create a map of fan names to entity info
         fan_map = {fan.name: fan for fan in fans}
diff --git a/tests/integration/test_host_mode_many_entities.py b/tests/integration/test_host_mode_many_entities.py
index 19d1ee315f..fbe3dc25c8 100644
--- a/tests/integration/test_host_mode_many_entities.py
+++ b/tests/integration/test_host_mode_many_entities.py
@@ -4,7 +4,17 @@ from __future__ import annotations
 
 import asyncio
 
-from aioesphomeapi import EntityState, SensorState
+from aioesphomeapi import (
+    ClimateInfo,
+    DateInfo,
+    DateState,
+    DateTimeInfo,
+    DateTimeState,
+    EntityState,
+    SensorState,
+    TimeInfo,
+    TimeState,
+)
 import pytest
 
 from .types import APIClientConnectedFactory, RunCompiledFunction
@@ -22,34 +32,56 @@ async def test_host_mode_many_entities(
     async with run_compiled(yaml_config), api_client_connected() as client:
         # Subscribe to state changes
         states: dict[int, EntityState] = {}
-        sensor_count_future: asyncio.Future[int] = loop.create_future()
+        minimum_states_future: asyncio.Future[None] = loop.create_future()
 
         def on_state(state: EntityState) -> None:
             states[state.key] = state
-            # Count sensor states specifically
+            # Check if we have received minimum expected states
             sensor_states = [
                 s
                 for s in states.values()
                 if isinstance(s, SensorState) and isinstance(s.state, float)
             ]
-            # When we have received states from at least 50 sensors, resolve the future
-            if len(sensor_states) >= 50 and not sensor_count_future.done():
-                sensor_count_future.set_result(len(sensor_states))
+            date_states = [s for s in states.values() if isinstance(s, DateState)]
+            time_states = [s for s in states.values() if isinstance(s, TimeState)]
+            datetime_states = [
+                s for s in states.values() if isinstance(s, DateTimeState)
+            ]
+
+            # We expect at least 50 sensors and 1 of each datetime entity type
+            if (
+                len(sensor_states) >= 50
+                and len(date_states) >= 1
+                and len(time_states) >= 1
+                and len(datetime_states) >= 1
+                and not minimum_states_future.done()
+            ):
+                minimum_states_future.set_result(None)
 
         client.subscribe_states(on_state)
 
-        # Wait for states from at least 50 sensors with timeout
+        # Wait for minimum states with timeout
         try:
-            sensor_count = await asyncio.wait_for(sensor_count_future, timeout=10.0)
-        except asyncio.TimeoutError:
+            await asyncio.wait_for(minimum_states_future, timeout=10.0)
+        except TimeoutError:
             sensor_states = [
                 s
                 for s in states.values()
                 if isinstance(s, SensorState) and isinstance(s.state, float)
             ]
+            date_states = [s for s in states.values() if isinstance(s, DateState)]
+            time_states = [s for s in states.values() if isinstance(s, TimeState)]
+            datetime_states = [
+                s for s in states.values() if isinstance(s, DateTimeState)
+            ]
+
             pytest.fail(
-                f"Did not receive states from at least 50 sensors within 10 seconds. "
-                f"Received {len(sensor_states)} sensor states out of {len(states)} total states"
+                f"Did not receive expected states within 10 seconds. "
+                f"Received: {len(sensor_states)} sensor states (expected >=50), "
+                f"{len(date_states)} date states (expected >=1), "
+                f"{len(time_states)} time states (expected >=1), "
+                f"{len(datetime_states)} datetime states (expected >=1). "
+                f"Total states: {len(states)}"
             )
 
         # Verify we received a good number of entity states
@@ -64,9 +96,65 @@ async def test_host_mode_many_entities(
             if isinstance(s, SensorState) and isinstance(s.state, float)
         ]
 
-        assert sensor_count >= 50, (
-            f"Expected at least 50 sensor states, got {sensor_count}"
-        )
         assert len(sensor_states) >= 50, (
             f"Expected at least 50 sensor states, got {len(sensor_states)}"
         )
+
+        # Verify we received datetime entity states
+        date_states = [s for s in states.values() if isinstance(s, DateState)]
+        time_states = [s for s in states.values() if isinstance(s, TimeState)]
+        datetime_states = [s for s in states.values() if isinstance(s, DateTimeState)]
+
+        assert len(date_states) >= 1, (
+            f"Expected at least 1 date state, got {len(date_states)}"
+        )
+        assert len(time_states) >= 1, (
+            f"Expected at least 1 time state, got {len(time_states)}"
+        )
+        assert len(datetime_states) >= 1, (
+            f"Expected at least 1 datetime state, got {len(datetime_states)}"
+        )
+
+        # Get entity info to verify climate entity details
+        entities = await client.list_entities_services()
+        climate_infos = [e for e in entities[0] if isinstance(e, ClimateInfo)]
+        assert len(climate_infos) >= 1, "Expected at least 1 climate entity"
+
+        climate_info = climate_infos[0]
+        # Verify the thermostat has presets
+        assert len(climate_info.supported_presets) > 0, (
+            "Expected climate to have presets"
+        )
+        # The thermostat platform uses standard presets (Home, Away, Sleep)
+        # which should be transmitted properly without string copies
+
+        # Verify specific presets exist
+        preset_names = [p.name for p in climate_info.supported_presets]
+        assert "HOME" in preset_names, f"Expected 'HOME' preset, got {preset_names}"
+        assert "AWAY" in preset_names, f"Expected 'AWAY' preset, got {preset_names}"
+        assert "SLEEP" in preset_names, f"Expected 'SLEEP' preset, got {preset_names}"
+
+        # Verify datetime entities exist
+        date_infos = [e for e in entities[0] if isinstance(e, DateInfo)]
+        time_infos = [e for e in entities[0] if isinstance(e, TimeInfo)]
+        datetime_infos = [e for e in entities[0] if isinstance(e, DateTimeInfo)]
+
+        assert len(date_infos) >= 1, "Expected at least 1 date entity"
+        assert len(time_infos) >= 1, "Expected at least 1 time entity"
+        assert len(datetime_infos) >= 1, "Expected at least 1 datetime entity"
+
+        # Verify the entity names
+        date_info = date_infos[0]
+        assert date_info.name == "Test Date", (
+            f"Expected date entity name 'Test Date', got {date_info.name}"
+        )
+
+        time_info = time_infos[0]
+        assert time_info.name == "Test Time", (
+            f"Expected time entity name 'Test Time', got {time_info.name}"
+        )
+
+        datetime_info = datetime_infos[0]
+        assert datetime_info.name == "Test DateTime", (
+            f"Expected datetime entity name 'Test DateTime', got {datetime_info.name}"
+        )
diff --git a/tests/integration/test_host_mode_many_entities_multiple_connections.py b/tests/integration/test_host_mode_many_entities_multiple_connections.py
index a4e5f8a45c..a7939bb277 100644
--- a/tests/integration/test_host_mode_many_entities_multiple_connections.py
+++ b/tests/integration/test_host_mode_many_entities_multiple_connections.py
@@ -50,7 +50,7 @@ async def test_host_mode_many_entities_multiple_connections(
                 asyncio.wait_for(client1_ready, timeout=10.0),
                 asyncio.wait_for(client2_ready, timeout=10.0),
             )
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(
                 f"One or both clients did not receive enough states within 10 seconds. "
                 f"Client1: {len(states1)}, Client2: {len(states2)}"
diff --git a/tests/integration/test_host_mode_sensor.py b/tests/integration/test_host_mode_sensor.py
index 8c1e9f5d51..e28d3419e6 100644
--- a/tests/integration/test_host_mode_sensor.py
+++ b/tests/integration/test_host_mode_sensor.py
@@ -40,7 +40,7 @@ async def test_host_mode_with_sensor(
         # Wait for sensor with specific value (42.0) with timeout
         try:
             test_sensor_state = await asyncio.wait_for(sensor_future, timeout=5.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(
                 f"Sensor with value 42.0 not received within 5 seconds. "
                 f"Received states: {list(states.values())}"
diff --git a/tests/integration/test_host_preferences.py b/tests/integration/test_host_preferences.py
new file mode 100644
index 0000000000..38c6460cf1
--- /dev/null
+++ b/tests/integration/test_host_preferences.py
@@ -0,0 +1,167 @@
+"""Test host preferences save and load functionality."""
+
+from __future__ import annotations
+
+import asyncio
+import re
+from typing import Any
+
+from aioesphomeapi import ButtonInfo, EntityInfo, NumberInfo, SwitchInfo
+import pytest
+
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+def find_entity_by_name(
+    entities: list[EntityInfo], entity_type: type, name: str
+) -> Any:
+    """Helper to find an entity by type and name."""
+    return next(
+        (e for e in entities if isinstance(e, entity_type) and e.name == name), None
+    )
+
+
+@pytest.mark.asyncio
+async def test_host_preferences_save_load(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Test that preferences are correctly saved and loaded after our optimization fix."""
+    loop = asyncio.get_running_loop()
+    log_lines: list[str] = []
+    preferences_saved = loop.create_future()
+    preferences_loaded = loop.create_future()
+    values_match = loop.create_future()
+    final_load_complete = loop.create_future()
+
+    # Patterns to match preference logs
+    save_pattern = re.compile(r"Preference saved: key=(\w+), value=([0-9.]+)")
+    load_pattern = re.compile(r"Preference loaded: key=(\w+), value=([0-9.]+)")
+    verify_pattern = re.compile(r"Preferences verified: values match!")
+    final_load_success_pattern = re.compile(
+        r"Final load test: loaded \d+ preferences successfully"
+    )
+
+    saved_values: dict[str, float] = {}
+    loaded_values: dict[str, float] = {}
+
+    def check_output(line: str) -> None:
+        """Check log output for preference operations."""
+        log_lines.append(line)
+
+        # Look for save operations
+        match = save_pattern.search(line)
+        if match:
+            key = match.group(1)
+            value = float(match.group(2))
+            saved_values[key] = value
+            if len(saved_values) >= 2 and not preferences_saved.done():
+                preferences_saved.set_result(True)
+
+        # Look for load operations
+        match = load_pattern.search(line)
+        if match:
+            key = match.group(1)
+            value = float(match.group(2))
+            loaded_values[key] = value
+            if len(loaded_values) >= 2 and not preferences_loaded.done():
+                preferences_loaded.set_result(True)
+
+        # Look for verification
+        if verify_pattern.search(line) and not values_match.done():
+            values_match.set_result(True)
+
+        # Look for final load test completion
+        if final_load_success_pattern.search(line) and not final_load_complete.done():
+            final_load_complete.set_result(True)
+
+    async with (
+        run_compiled(yaml_config, line_callback=check_output),
+        api_client_connected() as client,
+    ):
+        # Get entity list
+        entities, _ = await client.list_entities_services()
+
+        # Find our test entities using helper
+        test_switch = find_entity_by_name(entities, SwitchInfo, "Test Switch")
+        test_number = find_entity_by_name(entities, NumberInfo, "Test Number")
+        save_button = find_entity_by_name(entities, ButtonInfo, "Save Preferences")
+        load_button = find_entity_by_name(entities, ButtonInfo, "Load Preferences")
+        verify_button = find_entity_by_name(entities, ButtonInfo, "Verify Preferences")
+
+        assert test_switch is not None, "Test Switch not found"
+        assert test_number is not None, "Test Number not found"
+        assert save_button is not None, "Save Preferences button not found"
+        assert load_button is not None, "Load Preferences button not found"
+        assert verify_button is not None, "Verify Preferences button not found"
+
+        # Set initial values
+        client.switch_command(test_switch.key, True)
+        client.number_command(test_number.key, 42.5)
+
+        # Save preferences
+        client.button_command(save_button.key)
+
+        # Wait for save to complete
+        try:
+            await asyncio.wait_for(preferences_saved, timeout=5.0)
+        except TimeoutError:
+            pytest.fail("Preferences not saved within timeout")
+
+        # Verify we saved the expected values
+        assert "switch" in saved_values, f"Switch preference not saved: {saved_values}"
+        assert "number" in saved_values, f"Number preference not saved: {saved_values}"
+        assert saved_values["switch"] == 1.0, (
+            f"Switch value incorrect: {saved_values['switch']}"
+        )
+        assert saved_values["number"] == 42.5, (
+            f"Number value incorrect: {saved_values['number']}"
+        )
+
+        # Change the values to something else
+        client.switch_command(test_switch.key, False)
+        client.number_command(test_number.key, 13.7)
+
+        # Load preferences (should restore the saved values)
+        client.button_command(load_button.key)
+
+        # Wait for load to complete
+        try:
+            await asyncio.wait_for(preferences_loaded, timeout=5.0)
+        except TimeoutError:
+            pytest.fail("Preferences not loaded within timeout")
+
+        # Verify loaded values match saved values
+        assert "switch" in loaded_values, (
+            f"Switch preference not loaded: {loaded_values}"
+        )
+        assert "number" in loaded_values, (
+            f"Number preference not loaded: {loaded_values}"
+        )
+        assert loaded_values["switch"] == saved_values["switch"], (
+            f"Loaded switch value {loaded_values['switch']} doesn't match saved {saved_values['switch']}"
+        )
+        assert loaded_values["number"] == saved_values["number"], (
+            f"Loaded number value {loaded_values['number']} doesn't match saved {saved_values['number']}"
+        )
+
+        # Verify the values were actually restored
+        client.button_command(verify_button.key)
+
+        # Wait for verification
+        try:
+            await asyncio.wait_for(values_match, timeout=5.0)
+        except TimeoutError:
+            pytest.fail("Preference verification failed within timeout")
+
+        # Test that non-existent preferences don't crash (tests our fix)
+        # This will trigger load attempts for keys that don't exist
+        # Our fix should prevent map entries from being created
+        client.button_command(load_button.key)
+
+        # Wait for the final load test to complete
+        try:
+            await asyncio.wait_for(final_load_complete, timeout=5.0)
+        except TimeoutError:
+            pytest.fail("Final load test did not complete within timeout")
diff --git a/tests/integration/test_light_calls.py b/tests/integration/test_light_calls.py
index 1c56bbbf9e..af90ddbe86 100644
--- a/tests/integration/test_light_calls.py
+++ b/tests/integration/test_light_calls.py
@@ -108,14 +108,51 @@ async def test_light_calls(
         # Wait for flash to end
         state = await wait_for_state_change(rgbcw_light.key)
 
-        # Test 13: effect only
+        # Test 13: effect only - test all random effects
         # First ensure light is on
         client.light_command(key=rgbcw_light.key, state=True)
         state = await wait_for_state_change(rgbcw_light.key)
-        # Now set effect
-        client.light_command(key=rgbcw_light.key, effect="Random Effect")
+
+        # Test 13a: Default random effect (no name, gets default name "Random")
+        client.light_command(key=rgbcw_light.key, effect="Random")
         state = await wait_for_state_change(rgbcw_light.key)
-        assert state.effect == "Random Effect"
+        assert state.effect == "Random"
+
+        # Test 13b: Slow random effect with long name
+        client.light_command(
+            key=rgbcw_light.key, effect="My Very Slow Random Effect With Long Name"
+        )
+        state = await wait_for_state_change(rgbcw_light.key)
+        assert state.effect == "My Very Slow Random Effect With Long Name"
+
+        # Test 13c: Fast random effect with long name
+        client.light_command(
+            key=rgbcw_light.key, effect="My Fast Random Effect That Changes Quickly"
+        )
+        state = await wait_for_state_change(rgbcw_light.key)
+        assert state.effect == "My Fast Random Effect That Changes Quickly"
+
+        # Test 13d: Random effect with medium length name
+        client.light_command(
+            key=rgbcw_light.key, effect="Random Effect With Medium Length Name Here"
+        )
+        state = await wait_for_state_change(rgbcw_light.key)
+        assert state.effect == "Random Effect With Medium Length Name Here"
+
+        # Test 13e: Another random effect
+        client.light_command(
+            key=rgbcw_light.key,
+            effect="Another Random Effect With Different Parameters",
+        )
+        state = await wait_for_state_change(rgbcw_light.key)
+        assert state.effect == "Another Random Effect With Different Parameters"
+
+        # Test 13f: Yet another random effect
+        client.light_command(
+            key=rgbcw_light.key, effect="Yet Another Random Effect To Test Memory"
+        )
+        state = await wait_for_state_change(rgbcw_light.key)
+        assert state.effect == "Yet Another Random Effect To Test Memory"
 
         # Test 14: stop effect
         client.light_command(key=rgbcw_light.key, effect="None")
@@ -180,6 +217,69 @@ async def test_light_calls(
         state = await wait_for_state_change(rgb_light.key)
         assert state.state is False
 
+        # Test color mode combinations to verify get_suitable_color_modes optimization
+
+        # Test 22: White only mode
+        client.light_command(key=rgbcw_light.key, state=True, white=0.5)
+        state = await wait_for_state_change(rgbcw_light.key)
+        assert state.state is True
+
+        # Test 23: Color temperature only mode
+        client.light_command(key=rgbcw_light.key, state=True, color_temperature=300)
+        state = await wait_for_state_change(rgbcw_light.key)
+        assert state.color_temperature == pytest.approx(300)
+
+        # Test 24: Cold/warm white only mode
+        client.light_command(
+            key=rgbcw_light.key, state=True, cold_white=0.6, warm_white=0.4
+        )
+        state = await wait_for_state_change(rgbcw_light.key)
+        assert state.cold_white == pytest.approx(0.6)
+        assert state.warm_white == pytest.approx(0.4)
+
+        # Test 25: RGB only mode
+        client.light_command(key=rgb_light.key, state=True, rgb=(0.5, 0.5, 0.5))
+        state = await wait_for_state_change(rgb_light.key)
+        assert state.state is True
+
+        # Test 26: RGB + white combination
+        client.light_command(
+            key=rgbcw_light.key, state=True, rgb=(0.3, 0.3, 0.3), white=0.5
+        )
+        state = await wait_for_state_change(rgbcw_light.key)
+        assert state.state is True
+
+        # Test 27: RGB + color temperature combination
+        client.light_command(
+            key=rgbcw_light.key, state=True, rgb=(0.4, 0.4, 0.4), color_temperature=280
+        )
+        state = await wait_for_state_change(rgbcw_light.key)
+        assert state.state is True
+
+        # Test 28: RGB + cold/warm white combination
+        client.light_command(
+            key=rgbcw_light.key,
+            state=True,
+            rgb=(0.2, 0.2, 0.2),
+            cold_white=0.5,
+            warm_white=0.5,
+        )
+        state = await wait_for_state_change(rgbcw_light.key)
+        assert state.state is True
+
+        # Test 29: White + color temperature combination
+        client.light_command(
+            key=rgbcw_light.key, state=True, white=0.6, color_temperature=320
+        )
+        state = await wait_for_state_change(rgbcw_light.key)
+        assert state.state is True
+
+        # Test 30: No specific color parameters (tests default mode selection)
+        client.light_command(key=rgbcw_light.key, state=True, brightness=0.75)
+        state = await wait_for_state_change(rgbcw_light.key)
+        assert state.state is True
+        assert state.brightness == pytest.approx(0.75)
+
         # Final cleanup - turn all lights off
         for light in lights:
             client.light_command(
diff --git a/tests/integration/test_loop_disable_enable.py b/tests/integration/test_loop_disable_enable.py
index e93fc32178..2a866b1574 100644
--- a/tests/integration/test_loop_disable_enable.py
+++ b/tests/integration/test_loop_disable_enable.py
@@ -150,7 +150,7 @@ async def test_loop_disable_enable(
         # Wait for self_disable_10 to disable itself
         try:
             await asyncio.wait_for(self_disable_10_disabled.wait(), timeout=10.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("self_disable_10 did not disable itself within 10 seconds")
 
         # Verify it ran at least 10 times before disabling
@@ -164,7 +164,7 @@ async def test_loop_disable_enable(
         # Wait for normal_component to run at least 10 times
         try:
             await asyncio.wait_for(normal_component_10_loops.wait(), timeout=10.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(
                 f"normal_component did not reach 10 loops within timeout, got {len(normal_component_counts)}"
             )
@@ -172,12 +172,12 @@ async def test_loop_disable_enable(
         # Wait for redundant operation tests
         try:
             await asyncio.wait_for(redundant_enable_tested.wait(), timeout=10.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("redundant_enable did not test enabling when already enabled")
 
         try:
             await asyncio.wait_for(redundant_disable_tested.wait(), timeout=10.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(
                 "redundant_disable did not test disabling when will be disabled"
             )
@@ -185,7 +185,7 @@ async def test_loop_disable_enable(
         # Wait to see if self_disable_10 gets re-enabled
         try:
             await asyncio.wait_for(self_disable_10_re_enabled.wait(), timeout=5.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("self_disable_10 was not re-enabled within 5 seconds")
 
         # Component was re-enabled - verify it ran more times
@@ -198,7 +198,7 @@ async def test_loop_disable_enable(
         # Wait for ISR component to disable itself after 5 loops
         try:
             await asyncio.wait_for(isr_component_disabled.wait(), timeout=3.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("ISR component did not disable itself within 3 seconds")
 
         # Verify it ran exactly 5 times before disabling
@@ -210,7 +210,7 @@ async def test_loop_disable_enable(
         # Wait for component to be re-enabled by periodic ISR simulation and run again
         try:
             await asyncio.wait_for(isr_component_re_enabled.wait(), timeout=2.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("ISR component was not re-enabled after ISR call")
 
         # Verify it's running again after ISR enable
@@ -222,7 +222,7 @@ async def test_loop_disable_enable(
         # Wait for pure ISR enable (no main loop enable) to work
         try:
             await asyncio.wait_for(isr_component_pure_re_enabled.wait(), timeout=2.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("ISR component was not re-enabled by pure ISR call")
 
         # Verify it ran after pure ISR enable
@@ -235,7 +235,7 @@ async def test_loop_disable_enable(
         # Wait for update component to disable its loop
         try:
             await asyncio.wait_for(update_component_loop_disabled.wait(), timeout=3.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Update component did not disable its loop within 3 seconds")
 
         # Verify it ran exactly 3 loops before disabling
@@ -248,7 +248,7 @@ async def test_loop_disable_enable(
             await asyncio.wait_for(
                 update_component_manual_update_called.wait(), timeout=5.0
             )
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Manual component.update was not called within 5 seconds")
 
         # The key test: verify that manual component.update worked after loop was disabled
diff --git a/tests/integration/test_multi_device_preferences.py b/tests/integration/test_multi_device_preferences.py
new file mode 100644
index 0000000000..625f83f16e
--- /dev/null
+++ b/tests/integration/test_multi_device_preferences.py
@@ -0,0 +1,144 @@
+"""Test multi-device preference storage functionality."""
+
+from __future__ import annotations
+
+import asyncio
+import re
+
+from aioesphomeapi import ButtonInfo, NumberInfo, SelectInfo, SwitchInfo
+import pytest
+
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_multi_device_preferences(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Test that entities with same names on different devices have unique preference storage."""
+    loop = asyncio.get_running_loop()
+    log_lines: list[str] = []
+    preferences_logged = loop.create_future()
+
+    # Patterns to match preference hash logs
+    switch_hash_pattern_device = re.compile(r"Device ([AB]) Switch Pref Hash: (\d+)")
+    switch_hash_pattern_main = re.compile(r"Main Switch Pref Hash: (\d+)")
+    number_hash_pattern_device = re.compile(r"Device ([AB]) Number Pref Hash: (\d+)")
+    number_hash_pattern_main = re.compile(r"Main Number Pref Hash: (\d+)")
+    switch_hashes: dict[str, int] = {}
+    number_hashes: dict[str, int] = {}
+
+    def check_output(line: str) -> None:
+        """Check log output for preference hash information."""
+        log_lines.append(line)
+
+        # Look for device switch preference hash logs
+        match = switch_hash_pattern_device.search(line)
+        if match:
+            device = match.group(1)
+            hash_value = int(match.group(2))
+            switch_hashes[device] = hash_value
+
+        # Look for main switch preference hash
+        match = switch_hash_pattern_main.search(line)
+        if match:
+            hash_value = int(match.group(1))
+            switch_hashes["Main"] = hash_value
+
+        # Look for device number preference hash logs
+        match = number_hash_pattern_device.search(line)
+        if match:
+            device = match.group(1)
+            hash_value = int(match.group(2))
+            number_hashes[device] = hash_value
+
+        # Look for main number preference hash
+        match = number_hash_pattern_main.search(line)
+        if match:
+            hash_value = int(match.group(1))
+            number_hashes["Main"] = hash_value
+
+        # If we have all hashes, complete the future
+        if (
+            len(switch_hashes) == 3
+            and len(number_hashes) == 3
+            and not preferences_logged.done()
+        ):
+            preferences_logged.set_result(True)
+
+    async with (
+        run_compiled(yaml_config, line_callback=check_output),
+        api_client_connected() as client,
+    ):
+        # Get entity list
+        entities, _ = await client.list_entities_services()
+
+        # Verify we have the expected entities with duplicate names on different devices
+
+        # Check switches (3 with name "Light")
+        switches = [
+            e for e in entities if isinstance(e, SwitchInfo) and e.name == "Light"
+        ]
+        assert len(switches) == 3, f"Expected 3 'Light' switches, got {len(switches)}"
+
+        # Check numbers (3 with name "Setpoint")
+        numbers = [
+            e for e in entities if isinstance(e, NumberInfo) and e.name == "Setpoint"
+        ]
+        assert len(numbers) == 3, f"Expected 3 'Setpoint' numbers, got {len(numbers)}"
+
+        # Check selects (3 with name "Mode")
+        selects = [
+            e for e in entities if isinstance(e, SelectInfo) and e.name == "Mode"
+        ]
+        assert len(selects) == 3, f"Expected 3 'Mode' selects, got {len(selects)}"
+
+        # Find the test button entity to trigger preference logging
+        buttons = [e for e in entities if isinstance(e, ButtonInfo)]
+        test_button = next((b for b in buttons if b.name == "Test Preferences"), None)
+        assert test_button is not None, "Test Preferences button not found"
+
+        # Press the button to trigger logging
+        client.button_command(test_button.key)
+
+        # Wait for preference hashes to be logged
+        try:
+            await asyncio.wait_for(preferences_logged, timeout=5.0)
+        except TimeoutError:
+            pytest.fail("Preference hashes not logged within timeout")
+
+        # Verify all switch preference hashes are unique
+        assert len(switch_hashes) == 3, (
+            f"Expected 3 devices with switches, got {switch_hashes}"
+        )
+        switch_hash_values = list(switch_hashes.values())
+        assert len(switch_hash_values) == len(set(switch_hash_values)), (
+            f"Switch preference hashes are not unique: {switch_hashes}"
+        )
+
+        # Verify all number preference hashes are unique
+        assert len(number_hashes) == 3, (
+            f"Expected 3 devices with numbers, got {number_hashes}"
+        )
+        number_hash_values = list(number_hashes.values())
+        assert len(number_hash_values) == len(set(number_hash_values)), (
+            f"Number preference hashes are not unique: {number_hashes}"
+        )
+
+        # Verify Device A and Device B have different hashes (they have device_id set)
+        assert switch_hashes["A"] != switch_hashes["B"], (
+            f"Device A and B switches should have different hashes: A={switch_hashes['A']}, B={switch_hashes['B']}"
+        )
+        assert number_hashes["A"] != number_hashes["B"], (
+            f"Device A and B numbers should have different hashes: A={number_hashes['A']}, B={number_hashes['B']}"
+        )
+
+        # Verify Main device hash is different from both A and B
+        assert switch_hashes["Main"] != switch_hashes["A"], (
+            f"Main and Device A switches should have different hashes: Main={switch_hashes['Main']}, A={switch_hashes['A']}"
+        )
+        assert switch_hashes["Main"] != switch_hashes["B"], (
+            f"Main and Device B switches should have different hashes: Main={switch_hashes['Main']}, B={switch_hashes['B']}"
+        )
diff --git a/tests/integration/test_noise_encryption_key_protection.py b/tests/integration/test_noise_encryption_key_protection.py
new file mode 100644
index 0000000000..03c43ca8d3
--- /dev/null
+++ b/tests/integration/test_noise_encryption_key_protection.py
@@ -0,0 +1,51 @@
+"""Integration test for noise encryption key protection from YAML."""
+
+from __future__ import annotations
+
+import base64
+
+from aioesphomeapi import InvalidEncryptionKeyAPIError
+import pytest
+
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_noise_encryption_key_protection(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Test that noise encryption key set in YAML cannot be changed via API."""
+    # The key that's set in the YAML fixture
+    noise_psk = "zX9/JHxMKwpP0jUGsF0iESCm1wRvNgR6NkKVOhn7kSs="
+
+    # Keep ESPHome process running throughout all tests
+    async with run_compiled(yaml_config):
+        # First connection - test key change attempt
+        async with api_client_connected(noise_psk=noise_psk) as client:
+            # Verify connection is established
+            device_info = await client.device_info()
+            assert device_info is not None
+
+            # Try to set a new encryption key via API
+            new_key = base64.b64encode(
+                b"x" * 32
+            )  # Valid 32-byte key in base64 as bytes
+
+            # This should fail since key was set in YAML
+            success = await client.noise_encryption_set_key(new_key)
+            assert success is False
+
+        # Reconnect with the original key to verify it still works
+        async with api_client_connected(noise_psk=noise_psk) as client:
+            # Verify connection is still successful with original key
+            device_info = await client.device_info()
+            assert device_info is not None
+            assert device_info.name == "noise-key-test"
+
+        # Verify that connecting with a wrong key fails
+        wrong_key = base64.b64encode(b"y" * 32).decode()  # Different key
+        with pytest.raises(InvalidEncryptionKeyAPIError):
+            async with api_client_connected(noise_psk=wrong_key) as client:
+                await client.device_info()
diff --git a/tests/integration/test_oversized_payloads.py b/tests/integration/test_oversized_payloads.py
new file mode 100644
index 0000000000..ba18e3d348
--- /dev/null
+++ b/tests/integration/test_oversized_payloads.py
@@ -0,0 +1,337 @@
+"""Integration tests for oversized payloads and headers that should cause disconnection."""
+
+from __future__ import annotations
+
+import asyncio
+
+import pytest
+
+from .types import APIClientConnectedWithDisconnectFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_oversized_payload_plaintext(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected_with_disconnect: APIClientConnectedWithDisconnectFactory,
+) -> None:
+    """Test that oversized payloads (>32768 bytes) from client cause disconnection without crashing."""
+    process_exited = False
+    helper_log_found = False
+
+    def check_logs(line: str) -> None:
+        nonlocal process_exited, helper_log_found
+        # Check for signs that the process exited/crashed
+        if "Segmentation fault" in line or "core dumped" in line:
+            process_exited = True
+        # Check for HELPER_LOG message about message size exceeding maximum
+        if (
+            "[VV]" in line
+            and "Bad packet: message size" in line
+            and "exceeds maximum" in line
+        ):
+            helper_log_found = True
+
+    async with run_compiled(yaml_config, line_callback=check_logs):
+        async with api_client_connected_with_disconnect() as (client, disconnect_event):
+            # Verify basic connection works first
+            device_info = await client.device_info()
+            assert device_info is not None
+            assert device_info.name == "oversized-plaintext"
+
+            # Create an oversized payload (>32768 bytes which is our new limit)
+            oversized_data = b"X" * 40000  # ~40KiB, exceeds the 32768 byte limit
+
+            # Access the internal connection to send raw data
+            frame_helper = client._connection._frame_helper
+            # Create a message with oversized payload
+            # Using message type 1 (DeviceInfoRequest) as an example
+            message_type = 1
+            frame_helper.write_packets([(message_type, oversized_data)], True)
+
+            # Wait for the connection to be closed by ESPHome
+            await asyncio.wait_for(disconnect_event.wait(), timeout=5.0)
+
+        # After disconnection, verify process didn't crash
+        assert not process_exited, "ESPHome process should not crash"
+        # Verify we saw the expected HELPER_LOG message
+        assert helper_log_found, (
+            "Expected to see HELPER_LOG about message size exceeding maximum"
+        )
+
+        # Try to reconnect to verify the process is still running
+        async with api_client_connected_with_disconnect() as (client2, _):
+            device_info = await client2.device_info()
+            assert device_info is not None
+            assert device_info.name == "oversized-plaintext"
+
+
+@pytest.mark.asyncio
+async def test_oversized_protobuf_message_id_plaintext(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected_with_disconnect: APIClientConnectedWithDisconnectFactory,
+) -> None:
+    """Test that protobuf messages with ID > UINT16_MAX cause disconnection without crashing.
+
+    This tests the message type limit - message IDs must fit in a uint16_t (0-65535).
+    """
+    process_exited = False
+    helper_log_found = False
+
+    def check_logs(line: str) -> None:
+        nonlocal process_exited, helper_log_found
+        # Check for signs that the process exited/crashed
+        if "Segmentation fault" in line or "core dumped" in line:
+            process_exited = True
+        # Check for HELPER_LOG message about message type exceeding maximum
+        if (
+            "[VV]" in line
+            and "Bad packet: message type" in line
+            and "exceeds maximum" in line
+        ):
+            helper_log_found = True
+
+    async with run_compiled(yaml_config, line_callback=check_logs):
+        async with api_client_connected_with_disconnect() as (client, disconnect_event):
+            # Verify basic connection works first
+            device_info = await client.device_info()
+            assert device_info is not None
+            assert device_info.name == "oversized-protobuf-plaintext"
+
+            # Access the internal connection to send raw message with large ID
+            frame_helper = client._connection._frame_helper
+            # Message ID that exceeds uint16_t limit (> 65535)
+            large_message_id = 65536  # 2^16, exceeds UINT16_MAX
+            # Small payload for the test
+            payload = b"test"
+
+            # This should cause disconnection due to oversized varint
+            frame_helper.write_packets([(large_message_id, payload)], True)
+
+            # Wait for the connection to be closed by ESPHome
+            await asyncio.wait_for(disconnect_event.wait(), timeout=5.0)
+
+        # After disconnection, verify process didn't crash
+        assert not process_exited, "ESPHome process should not crash"
+        # Verify we saw the expected HELPER_LOG message
+        assert helper_log_found, (
+            "Expected to see HELPER_LOG about message type exceeding maximum"
+        )
+
+        # Try to reconnect to verify the process is still running
+        async with api_client_connected_with_disconnect() as (client2, _):
+            device_info = await client2.device_info()
+            assert device_info is not None
+            assert device_info.name == "oversized-protobuf-plaintext"
+
+
+@pytest.mark.asyncio
+async def test_oversized_payload_noise(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected_with_disconnect: APIClientConnectedWithDisconnectFactory,
+) -> None:
+    """Test that oversized payloads from client cause disconnection without crashing with noise encryption."""
+    noise_key = "N4Yle5YirwZhPiHHsdZLdOA73ndj/84veVaLhTvxCuU="
+    process_exited = False
+    helper_log_found = False
+
+    def check_logs(line: str) -> None:
+        nonlocal process_exited, helper_log_found
+        # Check for signs that the process exited/crashed
+        if "Segmentation fault" in line or "core dumped" in line:
+            process_exited = True
+        # Check for HELPER_LOG message about message size exceeding maximum
+        # With our new protection, oversized messages are rejected at frame level
+        if (
+            "[VV]" in line
+            and "Bad packet: message size" in line
+            and "exceeds maximum" in line
+        ):
+            helper_log_found = True
+
+    async with run_compiled(yaml_config, line_callback=check_logs):
+        async with api_client_connected_with_disconnect(noise_psk=noise_key) as (
+            client,
+            disconnect_event,
+        ):
+            # Verify basic connection works first
+            device_info = await client.device_info()
+            assert device_info is not None
+            assert device_info.name == "oversized-noise"
+
+            # Create an oversized payload (>32768 bytes which is our new limit)
+            oversized_data = b"Y" * 40000  # ~40KiB, exceeds the 32768 byte limit
+
+            # Access the internal connection to send raw data
+            frame_helper = client._connection._frame_helper
+            # For noise connections, we still send through write_packets
+            # but the frame helper will handle encryption
+            # Using message type 1 (DeviceInfoRequest) as an example
+            message_type = 1
+            frame_helper.write_packets([(message_type, oversized_data)], True)
+
+            # Wait for the connection to be closed by ESPHome
+            await asyncio.wait_for(disconnect_event.wait(), timeout=5.0)
+
+        # After disconnection, verify process didn't crash
+        assert not process_exited, "ESPHome process should not crash"
+        # Verify we saw the expected HELPER_LOG message
+        assert helper_log_found, (
+            "Expected to see HELPER_LOG about message size exceeding maximum"
+        )
+
+        # Try to reconnect to verify the process is still running
+        async with api_client_connected_with_disconnect(noise_psk=noise_key) as (
+            client2,
+            _,
+        ):
+            device_info = await client2.device_info()
+            assert device_info is not None
+            assert device_info.name == "oversized-noise"
+
+
+@pytest.mark.asyncio
+async def test_oversized_protobuf_message_id_noise(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected_with_disconnect: APIClientConnectedWithDisconnectFactory,
+) -> None:
+    """Test that the noise protocol handles unknown message types correctly.
+
+    With noise encryption, message types are stored as uint16_t (2 bytes) after decryption.
+    Unknown message types should be ignored without disconnecting, as ESPHome needs to
+    read the full message to maintain encryption stream continuity.
+    """
+    noise_key = "N4Yle5YirwZhPiHHsdZLdOA73ndj/84veVaLhTvxCuU="
+    process_exited = False
+
+    def check_logs(line: str) -> None:
+        nonlocal process_exited
+        # Check for signs that the process exited/crashed
+        if "Segmentation fault" in line or "core dumped" in line:
+            process_exited = True
+
+    async with run_compiled(yaml_config, line_callback=check_logs):
+        async with api_client_connected_with_disconnect(noise_psk=noise_key) as (
+            client,
+            disconnect_event,
+        ):
+            # Verify basic connection works first
+            device_info = await client.device_info()
+            assert device_info is not None
+            assert device_info.name == "oversized-noise"
+
+            # With noise, message types are uint16_t, so we test with an unknown but valid value
+            frame_helper = client._connection._frame_helper
+
+            # Test with an unknown message type (65535 is not used by ESPHome)
+            unknown_message_id = 65535  # Valid uint16_t but unknown to ESPHome
+            payload = b"test"
+
+            # Send the unknown message type - ESPHome should read and ignore it
+            frame_helper.write_packets([(unknown_message_id, payload)], True)
+
+            # Give ESPHome a moment to process (but expect no disconnection)
+            # The connection should stay alive as ESPHome ignores unknown message types
+            with pytest.raises(asyncio.TimeoutError):
+                await asyncio.wait_for(disconnect_event.wait(), timeout=0.5)
+
+            # Connection should still be alive - unknown types are ignored, not fatal
+            assert client._connection.is_connected, (
+                "Connection should remain open for unknown message types"
+            )
+
+            # Verify we can still communicate by sending a valid request
+            device_info2 = await client.device_info()
+            assert device_info2 is not None
+            assert device_info2.name == "oversized-noise"
+
+        # After test, verify process didn't crash
+        assert not process_exited, "ESPHome process should not crash"
+
+        # Verify we can still reconnect
+        async with api_client_connected_with_disconnect(noise_psk=noise_key) as (
+            client2,
+            _,
+        ):
+            device_info = await client2.device_info()
+            assert device_info is not None
+            assert device_info.name == "oversized-noise"
+
+
+@pytest.mark.asyncio
+async def test_noise_corrupt_encrypted_frame(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected_with_disconnect: APIClientConnectedWithDisconnectFactory,
+) -> None:
+    """Test that noise protocol properly handles corrupt encrypted frames.
+
+    Send a frame with valid size but corrupt encrypted content (garbage bytes).
+    This should fail decryption and cause disconnection.
+    """
+    noise_key = "N4Yle5YirwZhPiHHsdZLdOA73ndj/84veVaLhTvxCuU="
+    process_exited = False
+    cipherstate_failed = False
+
+    def check_logs(line: str) -> None:
+        nonlocal process_exited, cipherstate_failed
+        # Check for signs that the process exited/crashed
+        if "Segmentation fault" in line or "core dumped" in line:
+            process_exited = True
+        # Check for the expected warning about decryption failure
+        if (
+            "[W][api.connection" in line
+            and "Reading failed CIPHERSTATE_DECRYPT_FAILED" in line
+        ):
+            cipherstate_failed = True
+
+    async with run_compiled(yaml_config, line_callback=check_logs):
+        async with api_client_connected_with_disconnect(noise_psk=noise_key) as (
+            client,
+            disconnect_event,
+        ):
+            # Verify basic connection works first
+            device_info = await client.device_info()
+            assert device_info is not None
+            assert device_info.name == "oversized-noise"
+
+            # Get the socket to send raw corrupt data
+            socket = client._connection._socket
+
+            # Send a corrupt noise frame directly to the socket
+            # Format: [indicator=0x01][size_high][size_low][garbage_encrypted_data]
+            # Size of 32 bytes (reasonable size for a noise frame with MAC)
+            corrupt_frame = bytes(
+                [
+                    0x01,  # Noise indicator
+                    0x00,  # Size high byte
+                    0x20,  # Size low byte (32 bytes)
+                ]
+            ) + bytes(32)  # 32 bytes of zeros (invalid encrypted data)
+
+            # Send the corrupt frame
+            socket.sendall(corrupt_frame)
+
+            # Wait for ESPHome to disconnect due to decryption failure
+            await asyncio.wait_for(disconnect_event.wait(), timeout=5.0)
+
+        # After disconnection, verify process didn't crash
+        assert not process_exited, (
+            "ESPHome process should not crash on corrupt encrypted frames"
+        )
+        # Verify we saw the expected warning message
+        assert cipherstate_failed, (
+            "Expected to see warning about CIPHERSTATE_DECRYPT_FAILED"
+        )
+
+        # Verify we can still reconnect after handling the corrupt frame
+        async with api_client_connected_with_disconnect(noise_psk=noise_key) as (
+            client2,
+            _,
+        ):
+            device_info = await client2.device_info()
+            assert device_info is not None
+            assert device_info.name == "oversized-noise"
diff --git a/tests/integration/test_runtime_stats.py b/tests/integration/test_runtime_stats.py
new file mode 100644
index 0000000000..9e93035d83
--- /dev/null
+++ b/tests/integration/test_runtime_stats.py
@@ -0,0 +1,88 @@
+"""Test runtime statistics component."""
+
+from __future__ import annotations
+
+import asyncio
+import re
+
+import pytest
+
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_runtime_stats(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Test runtime stats logs statistics at configured interval and tracks components."""
+    loop = asyncio.get_running_loop()
+
+    # Track how many times we see the total stats
+    stats_count = 0
+    first_stats_future = loop.create_future()
+    second_stats_future = loop.create_future()
+
+    # Track component stats
+    component_stats_found = set()
+
+    # Patterns to match - need to handle ANSI color codes and timestamps
+    # The log format is: [HH:MM:SS][color codes][I][tag]: message
+    total_stats_pattern = re.compile(r"Total stats \(since boot\):")
+    # Match component names that may include dots (e.g., template.sensor)
+    component_pattern = re.compile(
+        r"^\[[^\]]+\].*?\s+([\w.]+):\s+count=(\d+),\s+avg=([\d.]+)ms"
+    )
+
+    def check_output(line: str) -> None:
+        """Check log output for runtime stats messages."""
+        nonlocal stats_count
+
+        # Check for total stats line
+        if total_stats_pattern.search(line):
+            stats_count += 1
+
+            if stats_count == 1 and not first_stats_future.done():
+                first_stats_future.set_result(True)
+            elif stats_count == 2 and not second_stats_future.done():
+                second_stats_future.set_result(True)
+
+        # Check for component stats
+        match = component_pattern.match(line)
+        if match:
+            component_name = match.group(1)
+            component_stats_found.add(component_name)
+
+    async with (
+        run_compiled(yaml_config, line_callback=check_output),
+        api_client_connected() as client,
+    ):
+        # Verify device is connected
+        device_info = await client.device_info()
+        assert device_info is not None
+
+        # Wait for first "Total stats" log (should happen at 1s)
+        try:
+            await asyncio.wait_for(first_stats_future, timeout=5.0)
+        except TimeoutError:
+            pytest.fail("First 'Total stats' log not seen within 5 seconds")
+
+        # Wait for second "Total stats" log (should happen at 2s)
+        try:
+            await asyncio.wait_for(second_stats_future, timeout=5.0)
+        except TimeoutError:
+            pytest.fail(f"Second 'Total stats' log not seen. Total seen: {stats_count}")
+
+        # Verify we got at least 2 stats logs
+        assert stats_count >= 2, (
+            f"Expected at least 2 'Total stats' logs, got {stats_count}"
+        )
+
+        # Verify we found stats for our components
+        assert "template.sensor" in component_stats_found, (
+            f"Expected template.sensor stats, found: {component_stats_found}"
+        )
+        assert "template.switch" in component_stats_found, (
+            f"Expected template.switch stats, found: {component_stats_found}"
+        )
diff --git a/tests/integration/test_scheduler_bulk_cleanup.py b/tests/integration/test_scheduler_bulk_cleanup.py
index 08ff293b84..b52a4a3496 100644
--- a/tests/integration/test_scheduler_bulk_cleanup.py
+++ b/tests/integration/test_scheduler_bulk_cleanup.py
@@ -103,7 +103,7 @@ async def test_scheduler_bulk_cleanup(
         # Wait for test completion
         try:
             await asyncio.wait_for(test_complete_future, timeout=10.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Bulk cleanup test timed out")
 
         # Verify bulk cleanup was triggered
diff --git a/tests/integration/test_scheduler_defer_cancel.py b/tests/integration/test_scheduler_defer_cancel.py
index 923cf946c4..34c46bab82 100644
--- a/tests/integration/test_scheduler_defer_cancel.py
+++ b/tests/integration/test_scheduler_defer_cancel.py
@@ -70,11 +70,13 @@ async def test_scheduler_defer_cancel(
                 test_complete_future.set_result(True)
                 return
 
-            if state.key == test_result_entity.key and not test_result_future.done():
-                # Event type should be "defer_executed_X" where X is the defer number
-                if state.event_type.startswith("defer_executed_"):
-                    defer_num = int(state.event_type.split("_")[-1])
-                    test_result_future.set_result(defer_num)
+            if (
+                state.key == test_result_entity.key
+                and not test_result_future.done()
+                and state.event_type.startswith("defer_executed_")
+            ):
+                defer_num = int(state.event_type.split("_")[-1])
+                test_result_future.set_result(defer_num)
 
         client.subscribe_states(on_state)
 
@@ -85,7 +87,7 @@ async def test_scheduler_defer_cancel(
         try:
             await asyncio.wait_for(test_complete_future, timeout=10.0)
             executed_defer = await asyncio.wait_for(test_result_future, timeout=1.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Test did not complete within timeout")
 
         # Verify that only defer 10 was executed
diff --git a/tests/integration/test_scheduler_defer_cancel_regular.py b/tests/integration/test_scheduler_defer_cancel_regular.py
index 57b7134feb..c93d814fbe 100644
--- a/tests/integration/test_scheduler_defer_cancel_regular.py
+++ b/tests/integration/test_scheduler_defer_cancel_regular.py
@@ -64,7 +64,7 @@ async def test_scheduler_defer_cancels_regular(
         # Wait for test completion
         try:
             await asyncio.wait_for(test_complete_future, timeout=5.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(f"Test timed out. Log messages: {log_messages}")
 
         # Verify results
diff --git a/tests/integration/test_scheduler_defer_fifo_simple.py b/tests/integration/test_scheduler_defer_fifo_simple.py
index eb4058fedd..3502302368 100644
--- a/tests/integration/test_scheduler_defer_fifo_simple.py
+++ b/tests/integration/test_scheduler_defer_fifo_simple.py
@@ -90,7 +90,7 @@ async def test_scheduler_defer_fifo_simple(
         try:
             await asyncio.wait_for(test_complete_future, timeout=5.0)
             test1_passed = await asyncio.wait_for(test_result_future, timeout=1.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Test set_timeout(0) did not complete within 5 seconds")
 
         assert test1_passed is True, (
@@ -108,7 +108,7 @@ async def test_scheduler_defer_fifo_simple(
         try:
             await asyncio.wait_for(test_complete_future, timeout=5.0)
             test2_passed = await asyncio.wait_for(test_result_future, timeout=1.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Test defer() did not complete within 5 seconds")
 
         # Verify the test passed
diff --git a/tests/integration/test_scheduler_defer_stress.py b/tests/integration/test_scheduler_defer_stress.py
index d546b7132f..6f4d997307 100644
--- a/tests/integration/test_scheduler_defer_stress.py
+++ b/tests/integration/test_scheduler_defer_stress.py
@@ -97,7 +97,7 @@ async def test_scheduler_defer_stress(
         # Wait for all defers to execute (should be quick)
         try:
             await asyncio.wait_for(test_complete_future, timeout=5.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             # Report how many we got
             pytest.fail(
                 f"Stress test timed out. Only {len(executed_defers)} of "
diff --git a/tests/integration/test_scheduler_heap_stress.py b/tests/integration/test_scheduler_heap_stress.py
index 3c757bfc9d..2d55b8ae89 100644
--- a/tests/integration/test_scheduler_heap_stress.py
+++ b/tests/integration/test_scheduler_heap_stress.py
@@ -103,13 +103,14 @@ async def test_scheduler_heap_stress(
 
         # Wait for all callbacks to execute (should be quick, but give more time for scheduling)
         try:
-            await asyncio.wait_for(test_complete_future, timeout=60.0)
-        except asyncio.TimeoutError:
+            await asyncio.wait_for(test_complete_future, timeout=10.0)
+        except TimeoutError:
             # Report how many we got
+            missing_ids = sorted(set(range(1000)) - executed_callbacks)
             pytest.fail(
                 f"Stress test timed out. Only {len(executed_callbacks)} of "
                 f"1000 callbacks executed. Missing IDs: "
-                f"{sorted(set(range(1000)) - executed_callbacks)[:10]}..."
+                f"{missing_ids[:20]}... (total missing: {len(missing_ids)})"
             )
 
         # Verify all callbacks executed
diff --git a/tests/integration/test_scheduler_null_name.py b/tests/integration/test_scheduler_null_name.py
index 41bcd8aed7..75864ea2d2 100644
--- a/tests/integration/test_scheduler_null_name.py
+++ b/tests/integration/test_scheduler_null_name.py
@@ -27,33 +27,33 @@ async def test_scheduler_null_name(
         if not test_complete_future.done() and test_complete_pattern.search(line):
             test_complete_future.set_result(True)
 
-    async with run_compiled(yaml_config, line_callback=check_output):
-        async with api_client_connected() as client:
-            # Verify we can connect
-            device_info = await client.device_info()
-            assert device_info is not None
-            assert device_info.name == "scheduler-null-name"
+    async with (
+        run_compiled(yaml_config, line_callback=check_output),
+        api_client_connected() as client,
+    ):
+        # Verify we can connect
+        device_info = await client.device_info()
+        assert device_info is not None
+        assert device_info.name == "scheduler-null-name"
 
-            # List services
-            _, services = await asyncio.wait_for(
-                client.list_entities_services(), timeout=5.0
+        # List services
+        _, services = await asyncio.wait_for(
+            client.list_entities_services(), timeout=5.0
+        )
+
+        # Find our test service
+        test_null_name_service = next(
+            (s for s in services if s.name == "test_null_name"), None
+        )
+        assert test_null_name_service is not None, "test_null_name service not found"
+
+        # Execute the test
+        client.execute_service(test_null_name_service, {})
+
+        # Wait for test completion
+        try:
+            await asyncio.wait_for(test_complete_future, timeout=10.0)
+        except TimeoutError:
+            pytest.fail(
+                "Test did not complete within timeout - likely crashed due to NULL name"
             )
-
-            # Find our test service
-            test_null_name_service = next(
-                (s for s in services if s.name == "test_null_name"), None
-            )
-            assert test_null_name_service is not None, (
-                "test_null_name service not found"
-            )
-
-            # Execute the test
-            client.execute_service(test_null_name_service, {})
-
-            # Wait for test completion
-            try:
-                await asyncio.wait_for(test_complete_future, timeout=10.0)
-            except asyncio.TimeoutError:
-                pytest.fail(
-                    "Test did not complete within timeout - likely crashed due to NULL name"
-                )
diff --git a/tests/integration/test_scheduler_pool.py b/tests/integration/test_scheduler_pool.py
new file mode 100644
index 0000000000..b5f9f12631
--- /dev/null
+++ b/tests/integration/test_scheduler_pool.py
@@ -0,0 +1,209 @@
+"""Integration test for scheduler memory pool functionality."""
+
+from __future__ import annotations
+
+import asyncio
+import re
+
+import pytest
+
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_scheduler_pool(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Test that the scheduler memory pool is working correctly with realistic usage.
+
+    This test simulates real-world scheduler usage patterns and verifies that:
+    1. Items are recycled to the pool when timeouts complete naturally
+    2. Items are recycled when intervals/timeouts are cancelled
+    3. Items are reused from the pool for new scheduler operations
+    4. The pool grows gradually based on actual usage patterns
+    5. Pool operations are logged correctly with debug scheduler enabled
+    """
+    # Track log messages to verify pool behavior
+    log_lines: list[str] = []
+    pool_reuse_count = 0
+    pool_recycle_count = 0
+    pool_full_count = 0
+    new_alloc_count = 0
+
+    # Patterns to match pool operations
+    reuse_pattern = re.compile(r"Reused item from pool \(pool size now: (\d+)\)")
+    recycle_pattern = re.compile(r"Recycled item to pool \(pool size now: (\d+)\)")
+    pool_full_pattern = re.compile(r"Pool full \(size: (\d+)\), deleting item")
+    new_alloc_pattern = re.compile(r"Allocated new item \(pool empty\)")
+
+    # Futures to track when test phases complete
+    loop = asyncio.get_running_loop()
+    test_complete_future: asyncio.Future[bool] = loop.create_future()
+    phase_futures = {
+        1: loop.create_future(),
+        2: loop.create_future(),
+        3: loop.create_future(),
+        4: loop.create_future(),
+        5: loop.create_future(),
+        6: loop.create_future(),
+        7: loop.create_future(),
+    }
+
+    def check_output(line: str) -> None:
+        """Check log output for pool operations and phase completion."""
+        nonlocal pool_reuse_count, pool_recycle_count, pool_full_count, new_alloc_count
+        log_lines.append(line)
+
+        # Track pool operations
+        if reuse_pattern.search(line):
+            pool_reuse_count += 1
+
+        elif recycle_pattern.search(line):
+            pool_recycle_count += 1
+
+        elif pool_full_pattern.search(line):
+            pool_full_count += 1
+
+        elif new_alloc_pattern.search(line):
+            new_alloc_count += 1
+
+        # Track phase completion
+        for phase_num in range(1, 8):
+            if (
+                f"Phase {phase_num} complete" in line
+                and phase_num in phase_futures
+                and not phase_futures[phase_num].done()
+            ):
+                phase_futures[phase_num].set_result(True)
+
+        # Check for test completion
+        if "Pool recycling test complete" in line and not test_complete_future.done():
+            test_complete_future.set_result(True)
+
+    # Run the test with log monitoring
+    async with (
+        run_compiled(yaml_config, line_callback=check_output),
+        api_client_connected() as client,
+    ):
+        # Verify device is running
+        device_info = await client.device_info()
+        assert device_info is not None
+        assert device_info.name == "scheduler-pool-test"
+
+        # Get list of services
+        entities, services = await client.list_entities_services()
+        service_names = {s.name for s in services}
+
+        # Verify all test services are available
+        expected_services = {
+            "run_phase_1",
+            "run_phase_2",
+            "run_phase_3",
+            "run_phase_4",
+            "run_phase_5",
+            "run_phase_6",
+            "run_phase_7",
+            "run_complete",
+        }
+        assert expected_services.issubset(service_names), (
+            f"Missing services: {expected_services - service_names}"
+        )
+
+        # Get service objects
+        phase_services = {
+            num: next(s for s in services if s.name == f"run_phase_{num}")
+            for num in range(1, 8)
+        }
+        complete_service = next(s for s in services if s.name == "run_complete")
+
+        try:
+            # Phase 1: Component lifecycle
+            client.execute_service(phase_services[1], {})
+            await asyncio.wait_for(phase_futures[1], timeout=1.0)
+            await asyncio.sleep(0.05)  # Let timeouts complete
+
+            # Phase 2: Sensor polling
+            client.execute_service(phase_services[2], {})
+            await asyncio.wait_for(phase_futures[2], timeout=1.0)
+            await asyncio.sleep(0.1)  # Let intervals run a bit
+
+            # Phase 3: Communication patterns
+            client.execute_service(phase_services[3], {})
+            await asyncio.wait_for(phase_futures[3], timeout=1.0)
+            await asyncio.sleep(0.1)  # Let heartbeat run
+
+            # Phase 4: Defer patterns
+            client.execute_service(phase_services[4], {})
+            await asyncio.wait_for(phase_futures[4], timeout=1.0)
+            await asyncio.sleep(0.2)  # Let everything settle and recycle
+
+            # Phase 5: Pool reuse verification
+            client.execute_service(phase_services[5], {})
+            await asyncio.wait_for(phase_futures[5], timeout=1.0)
+            await asyncio.sleep(0.1)  # Let Phase 5 timeouts complete and recycle
+
+            # Phase 6: Full pool reuse verification
+            client.execute_service(phase_services[6], {})
+            await asyncio.wait_for(phase_futures[6], timeout=1.0)
+            await asyncio.sleep(0.1)  # Let Phase 6 timeouts complete
+
+            # Phase 7: Same-named defer optimization
+            client.execute_service(phase_services[7], {})
+            await asyncio.wait_for(phase_futures[7], timeout=1.0)
+            await asyncio.sleep(0.05)  # Let the single defer execute
+
+            # Complete test
+            client.execute_service(complete_service, {})
+            await asyncio.wait_for(test_complete_future, timeout=0.5)
+
+        except TimeoutError as e:
+            # Print debug info if test times out
+            recent_logs = "\n".join(log_lines[-30:])
+            phases_completed = [num for num, fut in phase_futures.items() if fut.done()]
+            pytest.fail(
+                f"Test timed out waiting for phase/completion. Error: {e}\n"
+                f"  Phases completed: {phases_completed}\n"
+                f"  Pool stats:\n"
+                f"    Reuse count: {pool_reuse_count}\n"
+                f"    Recycle count: {pool_recycle_count}\n"
+                f"    Pool full count: {pool_full_count}\n"
+                f"    New alloc count: {new_alloc_count}\n"
+                f"Recent logs:\n{recent_logs}"
+            )
+
+    # Verify all test phases ran
+    for phase_num in range(1, 8):
+        assert phase_futures[phase_num].done(), f"Phase {phase_num} did not complete"
+
+    # Verify pool behavior
+    assert pool_recycle_count > 0, "Should have recycled items to pool"
+
+    # Check pool metrics
+    if pool_recycle_count > 0:
+        max_pool_size = 0
+        for line in log_lines:
+            if match := recycle_pattern.search(line):
+                size = int(match.group(1))
+                max_pool_size = max(max_pool_size, size)
+
+        # Pool can grow up to its maximum of 5
+        assert max_pool_size <= 5, f"Pool grew beyond maximum ({max_pool_size})"
+
+    # Log summary for debugging
+    print("\nScheduler Pool Test Summary (Python Orchestrated):")
+    print(f"  Items recycled to pool: {pool_recycle_count}")
+    print(f"  Items reused from pool: {pool_reuse_count}")
+    print(f"  Pool full events: {pool_full_count}")
+    print(f"  New allocations: {new_alloc_count}")
+    print("  All phases completed successfully")
+
+    # Verify reuse happened
+    if pool_reuse_count == 0 and pool_recycle_count > 3:
+        pytest.fail("Pool had items recycled but none were reused")
+
+    # Success - pool is working
+    assert pool_recycle_count > 0 or new_alloc_count < 15, (
+        "Pool should either recycle items or limit new allocations"
+    )
diff --git a/tests/integration/test_scheduler_rapid_cancellation.py b/tests/integration/test_scheduler_rapid_cancellation.py
index 90577f36f1..1b7da32aaa 100644
--- a/tests/integration/test_scheduler_rapid_cancellation.py
+++ b/tests/integration/test_scheduler_rapid_cancellation.py
@@ -61,9 +61,10 @@ async def test_scheduler_rapid_cancellation(
         elif "Total executed:" in line:
             if match := re.search(r"Total executed: (\d+)", line):
                 test_stats["final_executed"] = int(match.group(1))
-        elif "Implicit cancellations (replaced):" in line:
-            if match := re.search(r"Implicit cancellations \(replaced\): (\d+)", line):
-                test_stats["final_implicit_cancellations"] = int(match.group(1))
+        elif "Implicit cancellations (replaced):" in line and (
+            match := re.search(r"Implicit cancellations \(replaced\): (\d+)", line)
+        ):
+            test_stats["final_implicit_cancellations"] = int(match.group(1))
 
         # Check for crash indicators
         if any(
@@ -112,7 +113,7 @@ async def test_scheduler_rapid_cancellation(
         # Wait for test to complete with timeout
         try:
             await asyncio.wait_for(test_complete_future, timeout=10.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(f"Test timed out. Stats: {test_stats}")
 
         # Check for any errors
diff --git a/tests/integration/test_scheduler_recursive_timeout.py b/tests/integration/test_scheduler_recursive_timeout.py
index c015978e15..d98d2ac5ee 100644
--- a/tests/integration/test_scheduler_recursive_timeout.py
+++ b/tests/integration/test_scheduler_recursive_timeout.py
@@ -84,7 +84,7 @@ async def test_scheduler_recursive_timeout(
         # Wait for test to complete
         try:
             await asyncio.wait_for(test_complete_future, timeout=10.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(
                 f"Recursive timeout test timed out. Got sequence: {execution_sequence}"
             )
diff --git a/tests/integration/test_scheduler_removed_item_race.py b/tests/integration/test_scheduler_removed_item_race.py
new file mode 100644
index 0000000000..3e72bacc0d
--- /dev/null
+++ b/tests/integration/test_scheduler_removed_item_race.py
@@ -0,0 +1,102 @@
+"""Test for scheduler race condition where removed items still execute."""
+
+import asyncio
+import re
+
+import pytest
+
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_scheduler_removed_item_race(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Test that items marked for removal don't execute.
+
+    This test verifies the fix for a race condition where:
+    1. cleanup_() only removes items from the front of the heap
+    2. Items in the middle of the heap marked for removal still execute
+    3. This causes cancelled timeouts to run when they shouldn't
+    """
+
+    loop = asyncio.get_running_loop()
+    test_complete_future: asyncio.Future[bool] = loop.create_future()
+
+    # Track test results
+    test_passed = False
+    removed_executed = 0
+    normal_executed = 0
+
+    # Patterns to match
+    race_pattern = re.compile(r"RACE: .* executed after being cancelled!")
+    passed_pattern = re.compile(r"TEST PASSED")
+    failed_pattern = re.compile(r"TEST FAILED")
+    complete_pattern = re.compile(r"=== Test Complete ===")
+    normal_count_pattern = re.compile(r"Normal items executed: (\d+)")
+    removed_count_pattern = re.compile(r"Removed items executed: (\d+)")
+
+    def check_output(line: str) -> None:
+        """Check log output for test results."""
+        nonlocal test_passed, removed_executed, normal_executed
+
+        if race_pattern.search(line):
+            # Race condition detected - a cancelled item executed
+            test_passed = False
+
+        if passed_pattern.search(line):
+            test_passed = True
+        elif failed_pattern.search(line):
+            test_passed = False
+
+        normal_match = normal_count_pattern.search(line)
+        if normal_match:
+            normal_executed = int(normal_match.group(1))
+
+        removed_match = removed_count_pattern.search(line)
+        if removed_match:
+            removed_executed = int(removed_match.group(1))
+
+        if not test_complete_future.done() and complete_pattern.search(line):
+            test_complete_future.set_result(True)
+
+    async with (
+        run_compiled(yaml_config, line_callback=check_output),
+        api_client_connected() as client,
+    ):
+        # Verify we can connect
+        device_info = await client.device_info()
+        assert device_info is not None
+        assert device_info.name == "scheduler-removed-item-race"
+
+        # List services
+        _, services = await asyncio.wait_for(
+            client.list_entities_services(), timeout=5.0
+        )
+
+        # Find run_test service
+        run_test_service = next((s for s in services if s.name == "run_test"), None)
+        assert run_test_service is not None, "run_test service not found"
+
+        # Execute the test
+        client.execute_service(run_test_service, {})
+
+        # Wait for test completion
+        try:
+            await asyncio.wait_for(test_complete_future, timeout=5.0)
+        except TimeoutError:
+            pytest.fail("Test did not complete within timeout")
+
+        # Verify results
+        assert test_passed, (
+            f"Test failed! Removed items executed: {removed_executed}, "
+            f"Normal items executed: {normal_executed}"
+        )
+        assert removed_executed == 0, (
+            f"Cancelled items should not execute, but {removed_executed} did"
+        )
+        assert normal_executed == 4, (
+            f"Expected 4 normal items to execute, got {normal_executed}"
+        )
diff --git a/tests/integration/test_scheduler_retry_test.py b/tests/integration/test_scheduler_retry_test.py
new file mode 100644
index 0000000000..c04b7197c9
--- /dev/null
+++ b/tests/integration/test_scheduler_retry_test.py
@@ -0,0 +1,296 @@
+"""Test scheduler retry functionality."""
+
+import asyncio
+import re
+
+import pytest
+
+from .types import APIClientConnectedFactory, RunCompiledFunction
+
+
+@pytest.mark.asyncio
+async def test_scheduler_retry_test(
+    yaml_config: str,
+    run_compiled: RunCompiledFunction,
+    api_client_connected: APIClientConnectedFactory,
+) -> None:
+    """Test that scheduler retry functionality works correctly."""
+    # Track test progress
+    simple_retry_done = asyncio.Event()
+    backoff_retry_done = asyncio.Event()
+    immediate_done_done = asyncio.Event()
+    cancel_retry_done = asyncio.Event()
+    empty_name_retry_done = asyncio.Event()
+    component_retry_done = asyncio.Event()
+    multiple_name_done = asyncio.Event()
+    const_char_done = asyncio.Event()
+    static_char_done = asyncio.Event()
+    mixed_cancel_done = asyncio.Event()
+    test_complete = asyncio.Event()
+
+    # Track retry counts
+    simple_retry_count = 0
+    backoff_retry_count = 0
+    immediate_done_count = 0
+    cancel_retry_count = 0
+    empty_name_retry_count = 0
+    component_retry_count = 0
+    multiple_name_count = 0
+    const_char_retry_count = 0
+    static_char_retry_count = 0
+
+    # Track specific test results
+    cancel_result = None
+    empty_cancel_result = None
+    mixed_cancel_result = None
+    backoff_intervals = []
+
+    def on_log_line(line: str) -> None:
+        nonlocal simple_retry_count, backoff_retry_count, immediate_done_count
+        nonlocal cancel_retry_count, empty_name_retry_count, component_retry_count
+        nonlocal multiple_name_count, const_char_retry_count, static_char_retry_count
+        nonlocal cancel_result, empty_cancel_result, mixed_cancel_result
+
+        # Strip ANSI color codes
+        clean_line = re.sub(r"\x1b\[[0-9;]*m", "", line)
+
+        # Simple retry test
+        if "Simple retry attempt" in clean_line:
+            if match := re.search(r"Simple retry attempt (\d+)", clean_line):
+                simple_retry_count = int(match.group(1))
+
+        elif "Simple retry succeeded on attempt" in clean_line:
+            simple_retry_done.set()
+
+        # Backoff retry test
+        elif "Backoff retry attempt" in clean_line:
+            if match := re.search(
+                r"Backoff retry attempt (\d+).*interval=(\d+)ms", clean_line
+            ):
+                backoff_retry_count = int(match.group(1))
+                interval = int(match.group(2))
+                if backoff_retry_count > 1:  # Skip first (immediate) call
+                    backoff_intervals.append(interval)
+
+        elif "Backoff retry completed" in clean_line:
+            backoff_retry_done.set()
+
+        # Immediate done test
+        elif "Immediate done retry called" in clean_line:
+            immediate_done_count += 1
+            immediate_done_done.set()
+
+        # Cancel retry test
+        elif "Cancel test retry attempt" in clean_line:
+            cancel_retry_count += 1
+
+        elif "Retry cancellation result:" in clean_line:
+            cancel_result = "true" in clean_line
+            cancel_retry_done.set()
+
+        # Empty name retry test
+        elif "Empty name retry attempt" in clean_line:
+            if match := re.search(r"Empty name retry attempt (\d+)", clean_line):
+                empty_name_retry_count = int(match.group(1))
+
+        elif "Empty name retry cancel result:" in clean_line:
+            empty_cancel_result = "true" in clean_line
+
+        elif "Empty name retry ran" in clean_line:
+            empty_name_retry_done.set()
+
+        # Component retry test
+        elif "Component retry attempt" in clean_line:
+            if match := re.search(r"Component retry attempt (\d+)", clean_line):
+                component_retry_count = int(match.group(1))
+                if component_retry_count >= 2:
+                    component_retry_done.set()
+
+        # Multiple same name test
+        elif "Second duplicate retry attempt" in clean_line:
+            if match := re.search(r"counter=(\d+)", clean_line):
+                multiple_name_count = int(match.group(1))
+                if multiple_name_count >= 20:
+                    multiple_name_done.set()
+
+        # Const char retry test
+        elif "Const char retry" in clean_line:
+            if match := re.search(r"Const char retry (\d+)", clean_line):
+                const_char_retry_count = int(match.group(1))
+                const_char_done.set()
+
+        # Static const char retry test
+        elif "Static const char retry" in clean_line:
+            if match := re.search(r"Static const char retry (\d+)", clean_line):
+                static_char_retry_count = int(match.group(1))
+                static_char_done.set()
+
+        elif "Static cancel result:" in clean_line:
+            # This is part of test 9, but we don't track it separately
+            pass
+
+        # Mixed cancel test
+        elif "Mixed cancel result:" in clean_line:
+            mixed_cancel_result = "true" in clean_line
+            mixed_cancel_done.set()
+
+        # Test completion
+        elif "All retry tests completed" in clean_line:
+            test_complete.set()
+
+    async with (
+        run_compiled(yaml_config, line_callback=on_log_line),
+        api_client_connected() as client,
+    ):
+        # Verify we can connect
+        device_info = await client.device_info()
+        assert device_info is not None
+        assert device_info.name == "scheduler-retry-test"
+
+        # Wait for simple retry test
+        try:
+            await asyncio.wait_for(simple_retry_done.wait(), timeout=1.0)
+        except TimeoutError:
+            pytest.fail(
+                f"Simple retry test did not complete. Count: {simple_retry_count}"
+            )
+
+        assert simple_retry_count == 2, (
+            f"Expected 2 simple retry attempts, got {simple_retry_count}"
+        )
+
+        # Wait for backoff retry test
+        try:
+            await asyncio.wait_for(backoff_retry_done.wait(), timeout=3.0)
+        except TimeoutError:
+            pytest.fail(
+                f"Backoff retry test did not complete. Count: {backoff_retry_count}"
+            )
+
+        assert backoff_retry_count == 4, (
+            f"Expected 4 backoff retry attempts, got {backoff_retry_count}"
+        )
+
+        # Verify backoff intervals (allowing for timing variations)
+        assert len(backoff_intervals) >= 2, (
+            f"Expected at least 2 intervals, got {len(backoff_intervals)}"
+        )
+        if len(backoff_intervals) >= 3:
+            # First interval should be ~50ms (very wide tolerance for heavy system load)
+            assert 20 <= backoff_intervals[0] <= 150, (
+                f"First interval {backoff_intervals[0]}ms not ~50ms"
+            )
+            # Second interval should be ~100ms (50ms * 2.0)
+            assert 50 <= backoff_intervals[1] <= 250, (
+                f"Second interval {backoff_intervals[1]}ms not ~100ms"
+            )
+            # Third interval should be ~200ms (100ms * 2.0)
+            assert 100 <= backoff_intervals[2] <= 500, (
+                f"Third interval {backoff_intervals[2]}ms not ~200ms"
+            )
+
+        # Wait for immediate done test
+        try:
+            await asyncio.wait_for(immediate_done_done.wait(), timeout=3.0)
+        except TimeoutError:
+            pytest.fail(
+                f"Immediate done test did not complete. Count: {immediate_done_count}"
+            )
+
+        assert immediate_done_count == 1, (
+            f"Expected 1 immediate done call, got {immediate_done_count}"
+        )
+
+        # Wait for cancel retry test
+        try:
+            await asyncio.wait_for(cancel_retry_done.wait(), timeout=3.0)
+        except TimeoutError:
+            pytest.fail(
+                f"Cancel retry test did not complete. Count: {cancel_retry_count}"
+            )
+
+        assert cancel_result is True, "Retry cancellation should have succeeded"
+        assert 2 <= cancel_retry_count <= 5, (
+            f"Expected 2-5 cancel retry attempts before cancellation, got {cancel_retry_count}"
+        )
+
+        # Wait for empty name retry test
+        try:
+            await asyncio.wait_for(empty_name_retry_done.wait(), timeout=1.0)
+        except TimeoutError:
+            pytest.fail(
+                f"Empty name retry test did not complete. Count: {empty_name_retry_count}"
+            )
+
+        # Empty name retry should run at least once before being cancelled
+        assert 1 <= empty_name_retry_count <= 3, (
+            f"Expected 1-3 empty name retry attempts, got {empty_name_retry_count}"
+        )
+        assert empty_cancel_result is True, (
+            "Empty name retry cancel should have succeeded"
+        )
+
+        # Wait for component retry test
+        try:
+            await asyncio.wait_for(component_retry_done.wait(), timeout=1.0)
+        except TimeoutError:
+            pytest.fail(
+                f"Component retry test did not complete. Count: {component_retry_count}"
+            )
+
+        assert component_retry_count >= 2, (
+            f"Expected at least 2 component retry attempts, got {component_retry_count}"
+        )
+
+        # Wait for multiple same name test
+        try:
+            await asyncio.wait_for(multiple_name_done.wait(), timeout=1.0)
+        except TimeoutError:
+            pytest.fail(
+                f"Multiple same name test did not complete. Count: {multiple_name_count}"
+            )
+
+        # Should be 20+ (only second retry should run)
+        assert multiple_name_count >= 20, (
+            f"Expected multiple name count >= 20 (second retry only), got {multiple_name_count}"
+        )
+
+        # Wait for const char retry test
+        try:
+            await asyncio.wait_for(const_char_done.wait(), timeout=1.0)
+        except TimeoutError:
+            pytest.fail(
+                f"Const char retry test did not complete. Count: {const_char_retry_count}"
+            )
+
+        assert const_char_retry_count == 1, (
+            f"Expected 1 const char retry call, got {const_char_retry_count}"
+        )
+
+        # Wait for static char retry test
+        try:
+            await asyncio.wait_for(static_char_done.wait(), timeout=1.0)
+        except TimeoutError:
+            pytest.fail(
+                f"Static char retry test did not complete. Count: {static_char_retry_count}"
+            )
+
+        assert static_char_retry_count == 1, (
+            f"Expected 1 static char retry call, got {static_char_retry_count}"
+        )
+
+        # Wait for mixed cancel test
+        try:
+            await asyncio.wait_for(mixed_cancel_done.wait(), timeout=1.0)
+        except TimeoutError:
+            pytest.fail("Mixed cancel test did not complete")
+
+        assert mixed_cancel_result is True, (
+            "Mixed string/const char cancel should have succeeded"
+        )
+
+        # Wait for test completion
+        try:
+            await asyncio.wait_for(test_complete.wait(), timeout=1.0)
+        except TimeoutError:
+            pytest.fail("Test did not complete within timeout")
diff --git a/tests/integration/test_scheduler_simultaneous_callbacks.py b/tests/integration/test_scheduler_simultaneous_callbacks.py
index f5120ce4ce..82fd0fc01e 100644
--- a/tests/integration/test_scheduler_simultaneous_callbacks.py
+++ b/tests/integration/test_scheduler_simultaneous_callbacks.py
@@ -103,7 +103,7 @@ async def test_scheduler_simultaneous_callbacks(
         # Wait for test to complete
         try:
             await asyncio.wait_for(test_complete_future, timeout=30.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(f"Simultaneous callbacks test timed out. Stats: {test_stats}")
 
         # Check for any errors
diff --git a/tests/integration/test_scheduler_string_lifetime.py b/tests/integration/test_scheduler_string_lifetime.py
index 4d77abd954..7ec5a54373 100644
--- a/tests/integration/test_scheduler_string_lifetime.py
+++ b/tests/integration/test_scheduler_string_lifetime.py
@@ -157,7 +157,7 @@ async def test_scheduler_string_lifetime(
             client.execute_service(test_services["final"], {})
             await asyncio.wait_for(all_tests_complete.wait(), timeout=5.0)
 
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(f"String lifetime test timed out. Stats: {test_stats}")
 
         # Check for any errors
diff --git a/tests/integration/test_scheduler_string_name_stress.py b/tests/integration/test_scheduler_string_name_stress.py
index 3045842223..4c52913e63 100644
--- a/tests/integration/test_scheduler_string_name_stress.py
+++ b/tests/integration/test_scheduler_string_name_stress.py
@@ -97,7 +97,7 @@ async def test_scheduler_string_name_stress(
         # Wait for test to complete or crash
         try:
             await asyncio.wait_for(test_complete_future, timeout=30.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail(
                 f"String name stress test timed out. Executed {len(executed_callbacks)} callbacks. "
                 f"This might indicate a deadlock."
diff --git a/tests/integration/test_scheduler_string_test.py b/tests/integration/test_scheduler_string_test.py
index f3a36b2db7..783ed37c13 100644
--- a/tests/integration/test_scheduler_string_test.py
+++ b/tests/integration/test_scheduler_string_test.py
@@ -122,22 +122,22 @@ async def test_scheduler_string_test(
         # Wait for static string tests
         try:
             await asyncio.wait_for(static_timeout_1_fired.wait(), timeout=0.5)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Static timeout 1 did not fire within 0.5 seconds")
 
         try:
             await asyncio.wait_for(static_timeout_2_fired.wait(), timeout=0.5)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Static timeout 2 did not fire within 0.5 seconds")
 
         try:
             await asyncio.wait_for(static_interval_fired.wait(), timeout=1.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Static interval did not fire within 1 second")
 
         try:
             await asyncio.wait_for(static_interval_cancelled.wait(), timeout=2.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Static interval was not cancelled within 2 seconds")
 
         # Verify static interval ran at least 3 times
@@ -153,41 +153,41 @@ async def test_scheduler_string_test(
         # Wait for static defer tests
         try:
             await asyncio.wait_for(static_defer_1_fired.wait(), timeout=0.5)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Static defer 1 did not fire within 0.5 seconds")
 
         try:
             await asyncio.wait_for(static_defer_2_fired.wait(), timeout=0.5)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Static defer 2 did not fire within 0.5 seconds")
 
         # Wait for dynamic string tests
         try:
             await asyncio.wait_for(dynamic_timeout_fired.wait(), timeout=1.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Dynamic timeout did not fire within 1 second")
 
         try:
             await asyncio.wait_for(dynamic_interval_fired.wait(), timeout=1.5)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Dynamic interval did not fire within 1.5 seconds")
 
         # Wait for dynamic defer test
         try:
             await asyncio.wait_for(dynamic_defer_fired.wait(), timeout=1.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Dynamic defer did not fire within 1 second")
 
         # Wait for cancel test
         try:
             await asyncio.wait_for(cancel_test_done.wait(), timeout=1.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Cancel test did not complete within 1 second")
 
         # Wait for final results
         try:
             await asyncio.wait_for(final_results_logged.wait(), timeout=4.0)
-        except asyncio.TimeoutError:
+        except TimeoutError:
             pytest.fail("Final results were not logged within 4 seconds")
 
         # Verify results
diff --git a/tests/integration/types.py b/tests/integration/types.py
index 5e4bfaa29d..b6728a2fcb 100644
--- a/tests/integration/types.py
+++ b/tests/integration/types.py
@@ -54,3 +54,17 @@ class APIClientConnectedFactory(Protocol):
         client_info: str = "integration-test",
         timeout: float = 30,
     ) -> AbstractAsyncContextManager[APIClient]: ...
+
+
+class APIClientConnectedWithDisconnectFactory(Protocol):
+    """Protocol for connected API client factory that returns disconnect event."""
+
+    def __call__(  # noqa: E704
+        self,
+        address: str = "localhost",
+        port: int | None = None,
+        password: str = "",
+        noise_psk: str | None = None,
+        client_info: str = "integration-test",
+        timeout: float = 30,
+    ) -> AbstractAsyncContextManager[tuple[APIClient, asyncio.Event]]: ...
diff --git a/tests/script/test_clang_tidy_hash.py b/tests/script/test_clang_tidy_hash.py
index 7b66a69adb..b1690a6a2d 100644
--- a/tests/script/test_clang_tidy_hash.py
+++ b/tests/script/test_clang_tidy_hash.py
@@ -44,37 +44,53 @@ def test_get_clang_tidy_version_from_requirements(
     assert result == expected
 
 
-def test_calculate_clang_tidy_hash() -> None:
-    """Test calculating hash from all configuration sources."""
+def test_calculate_clang_tidy_hash_with_sdkconfig(tmp_path: Path) -> None:
+    """Test calculating hash from all configuration sources including sdkconfig.defaults."""
     clang_tidy_content = b"Checks: '-*,readability-*'\n"
     requirements_version = "clang-tidy==18.1.5"
     platformio_content = b"[env:esp32]\nplatform = espressif32\n"
+    sdkconfig_content = b"CONFIG_AUTOSTART_ARDUINO=y\n"
+    requirements_content = "clang-tidy==18.1.5\n"
+
+    # Create temporary files
+    (tmp_path / ".clang-tidy").write_bytes(clang_tidy_content)
+    (tmp_path / "platformio.ini").write_bytes(platformio_content)
+    (tmp_path / "sdkconfig.defaults").write_bytes(sdkconfig_content)
+    (tmp_path / "requirements_dev.txt").write_text(requirements_content)
 
     # Expected hash calculation
     expected_hasher = hashlib.sha256()
     expected_hasher.update(clang_tidy_content)
     expected_hasher.update(requirements_version.encode())
     expected_hasher.update(platformio_content)
+    expected_hasher.update(sdkconfig_content)
     expected_hash = expected_hasher.hexdigest()
 
-    # Mock the dependencies
-    with (
-        patch("clang_tidy_hash.read_file_bytes") as mock_read_bytes,
-        patch(
-            "clang_tidy_hash.get_clang_tidy_version_from_requirements",
-            return_value=requirements_version,
-        ),
-    ):
-        # Set up mock to return different content based on the file being read
-        def read_file_mock(path: Path) -> bytes:
-            if ".clang-tidy" in str(path):
-                return clang_tidy_content
-            elif "platformio.ini" in str(path):
-                return platformio_content
-            return b""
+    result = clang_tidy_hash.calculate_clang_tidy_hash(repo_root=tmp_path)
 
-        mock_read_bytes.side_effect = read_file_mock
-        result = clang_tidy_hash.calculate_clang_tidy_hash()
+    assert result == expected_hash
+
+
+def test_calculate_clang_tidy_hash_without_sdkconfig(tmp_path: Path) -> None:
+    """Test calculating hash without sdkconfig.defaults file."""
+    clang_tidy_content = b"Checks: '-*,readability-*'\n"
+    requirements_version = "clang-tidy==18.1.5"
+    platformio_content = b"[env:esp32]\nplatform = espressif32\n"
+    requirements_content = "clang-tidy==18.1.5\n"
+
+    # Create temporary files (without sdkconfig.defaults)
+    (tmp_path / ".clang-tidy").write_bytes(clang_tidy_content)
+    (tmp_path / "platformio.ini").write_bytes(platformio_content)
+    (tmp_path / "requirements_dev.txt").write_text(requirements_content)
+
+    # Expected hash calculation (no sdkconfig)
+    expected_hasher = hashlib.sha256()
+    expected_hasher.update(clang_tidy_content)
+    expected_hasher.update(requirements_version.encode())
+    expected_hasher.update(platformio_content)
+    expected_hash = expected_hasher.hexdigest()
+
+    result = clang_tidy_hash.calculate_clang_tidy_hash(repo_root=tmp_path)
 
     assert result == expected_hash
 
@@ -85,67 +101,63 @@ def test_read_stored_hash_exists(tmp_path: Path) -> None:
     hash_file = tmp_path / ".clang-tidy.hash"
     hash_file.write_text(f"{stored_hash}\n")
 
-    with (
-        patch("clang_tidy_hash.Path") as mock_path_class,
-        patch("clang_tidy_hash.read_file_lines", return_value=[f"{stored_hash}\n"]),
-    ):
-        # Mock the path calculation and exists check
-        mock_hash_file = Mock()
-        mock_hash_file.exists.return_value = True
-        mock_path_class.return_value.parent.parent.__truediv__.return_value = (
-            mock_hash_file
-        )
-
-        result = clang_tidy_hash.read_stored_hash()
+    result = clang_tidy_hash.read_stored_hash(repo_root=tmp_path)
 
     assert result == stored_hash
 
 
-def test_read_stored_hash_not_exists() -> None:
+def test_read_stored_hash_not_exists(tmp_path: Path) -> None:
     """Test reading hash when file doesn't exist."""
-    with patch("clang_tidy_hash.Path") as mock_path_class:
-        # Mock the path calculation and exists check
-        mock_hash_file = Mock()
-        mock_hash_file.exists.return_value = False
-        mock_path_class.return_value.parent.parent.__truediv__.return_value = (
-            mock_hash_file
-        )
-
-        result = clang_tidy_hash.read_stored_hash()
+    result = clang_tidy_hash.read_stored_hash(repo_root=tmp_path)
 
     assert result is None
 
 
-def test_write_hash() -> None:
+def test_write_hash(tmp_path: Path) -> None:
     """Test writing hash to file."""
     hash_value = "abc123def456"
+    hash_file = tmp_path / ".clang-tidy.hash"
 
-    with patch("clang_tidy_hash.write_file_content") as mock_write:
-        clang_tidy_hash.write_hash(hash_value)
+    clang_tidy_hash.write_hash(hash_value, repo_root=tmp_path)
 
-        # Verify write_file_content was called with correct parameters
-        mock_write.assert_called_once()
-        args = mock_write.call_args[0]
-        assert str(args[0]).endswith(".clang-tidy.hash")
-        assert args[1] == hash_value.strip() + "\n"
+    assert hash_file.exists()
+    assert hash_file.read_text() == hash_value.strip() + "\n"
 
 
 @pytest.mark.parametrize(
-    ("args", "current_hash", "stored_hash", "expected_exit"),
+    ("args", "current_hash", "stored_hash", "hash_file_in_changed", "expected_exit"),
     [
-        (["--check"], "abc123", "abc123", 1),  # Hashes match, no scan needed
-        (["--check"], "abc123", "def456", 0),  # Hashes differ, scan needed
-        (["--check"], "abc123", None, 0),  # No stored hash, scan needed
+        (["--check"], "abc123", "abc123", False, 1),  # Hashes match, no scan needed
+        (["--check"], "abc123", "def456", False, 0),  # Hashes differ, scan needed
+        (["--check"], "abc123", None, False, 0),  # No stored hash, scan needed
+        (
+            ["--check"],
+            "abc123",
+            "abc123",
+            True,
+            0,
+        ),  # Hash file updated in PR, scan needed
     ],
 )
 def test_main_check_mode(
-    args: list[str], current_hash: str, stored_hash: str | None, expected_exit: int
+    args: list[str],
+    current_hash: str,
+    stored_hash: str | None,
+    hash_file_in_changed: bool,
+    expected_exit: int,
 ) -> None:
     """Test main function in check mode."""
+    changed = [".clang-tidy.hash"] if hash_file_in_changed else []
+
+    # Create a mock module that can be imported
+    mock_helpers = Mock()
+    mock_helpers.changed_files = Mock(return_value=changed)
+
     with (
         patch("sys.argv", ["clang_tidy_hash.py"] + args),
         patch("clang_tidy_hash.calculate_clang_tidy_hash", return_value=current_hash),
         patch("clang_tidy_hash.read_stored_hash", return_value=stored_hash),
+        patch.dict("sys.modules", {"helpers": mock_helpers}),
         pytest.raises(SystemExit) as exc_info,
     ):
         clang_tidy_hash.main()
diff --git a/tests/script/test_determine_jobs.py b/tests/script/test_determine_jobs.py
index 4aaaadd80a..0559d116be 100644
--- a/tests/script/test_determine_jobs.py
+++ b/tests/script/test_determine_jobs.py
@@ -4,9 +4,10 @@ from collections.abc import Generator
 import importlib.util
 import json
 import os
+from pathlib import Path
 import subprocess
 import sys
-from unittest.mock import Mock, patch
+from unittest.mock import Mock, call, patch
 
 import pytest
 
@@ -72,9 +73,11 @@ def test_main_all_tests_should_run(
     mock_should_run_clang_format.return_value = True
     mock_should_run_python_linters.return_value = True
 
-    # Mock list-components.py output
+    # Mock list-components.py output (now returns JSON with --changed-with-deps)
     mock_result = Mock()
-    mock_result.stdout = "wifi\napi\nsensor\n"
+    mock_result.stdout = json.dumps(
+        {"directly_changed": ["wifi", "api"], "all_changed": ["wifi", "api", "sensor"]}
+    )
     mock_subprocess_run.return_value = mock_result
 
     # Run main function with mocked argv
@@ -90,7 +93,13 @@ def test_main_all_tests_should_run(
     assert output["clang_format"] is True
     assert output["python_linters"] is True
     assert output["changed_components"] == ["wifi", "api", "sensor"]
-    assert output["component_test_count"] == 3
+    # changed_components_with_tests will only include components that actually have test files
+    assert "changed_components_with_tests" in output
+    assert isinstance(output["changed_components_with_tests"], list)
+    # component_test_count matches number of components with tests
+    assert output["component_test_count"] == len(
+        output["changed_components_with_tests"]
+    )
 
 
 def test_main_no_tests_should_run(
@@ -109,7 +118,7 @@ def test_main_no_tests_should_run(
 
     # Mock empty list-components.py output
     mock_result = Mock()
-    mock_result.stdout = ""
+    mock_result.stdout = json.dumps({"directly_changed": [], "all_changed": []})
     mock_subprocess_run.return_value = mock_result
 
     # Run main function with mocked argv
@@ -125,6 +134,7 @@ def test_main_no_tests_should_run(
     assert output["clang_format"] is False
     assert output["python_linters"] is False
     assert output["changed_components"] == []
+    assert output["changed_components_with_tests"] == []
     assert output["component_test_count"] == 0
 
 
@@ -146,9 +156,11 @@ def test_main_list_components_fails(
     mock_subprocess_run.side_effect = subprocess.CalledProcessError(1, "cmd")
 
     # Run main function with mocked argv - should raise
-    with patch("sys.argv", ["determine-jobs.py"]):
-        with pytest.raises(subprocess.CalledProcessError):
-            determine_jobs.main()
+    with (
+        patch("sys.argv", ["determine-jobs.py"]),
+        pytest.raises(subprocess.CalledProcessError),
+    ):
+        determine_jobs.main()
 
 
 def test_main_with_branch_argument(
@@ -167,7 +179,9 @@ def test_main_with_branch_argument(
 
     # Mock list-components.py output
     mock_result = Mock()
-    mock_result.stdout = "mqtt\n"
+    mock_result.stdout = json.dumps(
+        {"directly_changed": ["mqtt"], "all_changed": ["mqtt"]}
+    )
     mock_subprocess_run.return_value = mock_result
 
     with patch("sys.argv", ["script.py", "-b", "main"]):
@@ -182,7 +196,7 @@ def test_main_with_branch_argument(
     # Check that list-components.py was called with branch
     mock_subprocess_run.assert_called_once()
     call_args = mock_subprocess_run.call_args[0][0]
-    assert "--changed" in call_args
+    assert "--changed-with-deps" in call_args
     assert "-b" in call_args
     assert "main" in call_args
 
@@ -195,7 +209,13 @@ def test_main_with_branch_argument(
     assert output["clang_format"] is False
     assert output["python_linters"] is True
     assert output["changed_components"] == ["mqtt"]
-    assert output["component_test_count"] == 1
+    # changed_components_with_tests will only include components that actually have test files
+    assert "changed_components_with_tests" in output
+    assert isinstance(output["changed_components_with_tests"], list)
+    # component_test_count matches number of components with tests
+    assert output["component_test_count"] == len(
+        output["changed_components_with_tests"]
+    )
 
 
 def test_should_run_integration_tests(
@@ -243,17 +263,21 @@ def test_should_run_integration_tests_with_branch() -> None:
 
 def test_should_run_integration_tests_component_dependency() -> None:
     """Test that integration tests run when components used in fixtures change."""
-    with patch.object(
-        determine_jobs, "changed_files", return_value=["esphome/components/api/api.cpp"]
-    ):
-        with patch.object(
+    with (
+        patch.object(
+            determine_jobs,
+            "changed_files",
+            return_value=["esphome/components/api/api.cpp"],
+        ),
+        patch.object(
             determine_jobs, "get_components_from_integration_fixtures"
-        ) as mock_fixtures:
-            mock_fixtures.return_value = {"api", "sensor"}
-            with patch.object(determine_jobs, "get_all_dependencies") as mock_deps:
-                mock_deps.return_value = {"api", "sensor", "network"}
-                result = determine_jobs.should_run_integration_tests()
-                assert result is True
+        ) as mock_fixtures,
+    ):
+        mock_fixtures.return_value = {"api", "sensor"}
+        with patch.object(determine_jobs, "get_all_dependencies") as mock_deps:
+            mock_deps.return_value = {"api", "sensor", "network"}
+            result = determine_jobs.should_run_integration_tests()
+            assert result is True
 
 
 @pytest.mark.parametrize(
@@ -262,6 +286,8 @@ def test_should_run_integration_tests_component_dependency() -> None:
         (0, [], True),  # Hash changed - need full scan
         (1, ["esphome/core.cpp"], True),  # C++ file changed
         (1, ["README.md"], False),  # No C++ files changed
+        (1, [".clang-tidy.hash"], True),  # Hash file itself changed
+        (1, ["platformio.ini", ".clang-tidy.hash"], True),  # Config + hash changed
     ],
 )
 def test_should_run_clang_tidy(
@@ -270,18 +296,35 @@ def test_should_run_clang_tidy(
     expected_result: bool,
 ) -> None:
     """Test should_run_clang_tidy function."""
-    with patch.object(determine_jobs, "changed_files", return_value=changed_files):
+    with (
+        patch.object(determine_jobs, "changed_files", return_value=changed_files),
+        patch("subprocess.run") as mock_run,
+    ):
         # Test with hash check returning specific code
-        with patch("subprocess.run") as mock_run:
-            mock_run.return_value = Mock(returncode=check_returncode)
-            result = determine_jobs.should_run_clang_tidy()
-            assert result == expected_result
+        mock_run.return_value = Mock(returncode=check_returncode)
+        result = determine_jobs.should_run_clang_tidy()
+        assert result == expected_result
 
-        # Test with hash check failing (exception)
-        if check_returncode != 0:
-            with patch("subprocess.run", side_effect=Exception("Failed")):
-                result = determine_jobs.should_run_clang_tidy()
-            assert result is True  # Fail safe - run clang-tidy
+
+def test_should_run_clang_tidy_hash_check_exception() -> None:
+    """Test should_run_clang_tidy when hash check fails with exception."""
+    # When hash check fails, clang-tidy should run as a safety measure
+    with (
+        patch.object(determine_jobs, "changed_files", return_value=["README.md"]),
+        patch("subprocess.run", side_effect=Exception("Hash check failed")),
+    ):
+        result = determine_jobs.should_run_clang_tidy()
+        assert result is True  # Fail safe - run clang-tidy
+
+    # Even with C++ files, exception should trigger clang-tidy
+    with (
+        patch.object(
+            determine_jobs, "changed_files", return_value=["esphome/core.cpp"]
+        ),
+        patch("subprocess.run", side_effect=Exception("Hash check failed")),
+    ):
+        result = determine_jobs.should_run_clang_tidy()
+        assert result is True
 
 
 def test_should_run_clang_tidy_with_branch() -> None:
@@ -291,7 +334,9 @@ def test_should_run_clang_tidy_with_branch() -> None:
         with patch("subprocess.run") as mock_run:
             mock_run.return_value = Mock(returncode=1)  # Hash unchanged
             determine_jobs.should_run_clang_tidy("release")
-            mock_changed.assert_called_once_with("release")
+            # Changed files is called twice now - once for hash check, once for .clang-tidy.hash check
+            assert mock_changed.call_count == 2
+            mock_changed.assert_has_calls([call("release"), call("release")])
 
 
 @pytest.mark.parametrize(
@@ -350,3 +395,67 @@ def test_should_run_clang_format_with_branch() -> None:
         mock_changed.return_value = []
         determine_jobs.should_run_clang_format("release")
         mock_changed.assert_called_once_with("release")
+
+
+def test_main_filters_components_without_tests(
+    mock_should_run_integration_tests: Mock,
+    mock_should_run_clang_tidy: Mock,
+    mock_should_run_clang_format: Mock,
+    mock_should_run_python_linters: Mock,
+    mock_subprocess_run: Mock,
+    capsys: pytest.CaptureFixture[str],
+    tmp_path: Path,
+) -> None:
+    """Test that components without test files are filtered out."""
+    mock_should_run_integration_tests.return_value = False
+    mock_should_run_clang_tidy.return_value = False
+    mock_should_run_clang_format.return_value = False
+    mock_should_run_python_linters.return_value = False
+
+    # Mock list-components.py output with 3 components
+    # wifi: has tests, sensor: has tests, airthings_ble: no tests
+    mock_result = Mock()
+    mock_result.stdout = json.dumps(
+        {
+            "directly_changed": ["wifi", "sensor"],
+            "all_changed": ["wifi", "sensor", "airthings_ble"],
+        }
+    )
+    mock_subprocess_run.return_value = mock_result
+
+    # Create test directory structure
+    tests_dir = tmp_path / "tests" / "components"
+
+    # wifi has tests
+    wifi_dir = tests_dir / "wifi"
+    wifi_dir.mkdir(parents=True)
+    (wifi_dir / "test.esp32.yaml").write_text("test: config")
+
+    # sensor has tests
+    sensor_dir = tests_dir / "sensor"
+    sensor_dir.mkdir(parents=True)
+    (sensor_dir / "test.esp8266.yaml").write_text("test: config")
+
+    # airthings_ble exists but has no test files
+    airthings_dir = tests_dir / "airthings_ble"
+    airthings_dir.mkdir(parents=True)
+
+    # Mock root_path to use tmp_path
+    with (
+        patch.object(determine_jobs, "root_path", str(tmp_path)),
+        patch("sys.argv", ["determine-jobs.py"]),
+    ):
+        # Clear the cache since we're mocking root_path
+        determine_jobs._component_has_tests.cache_clear()
+        determine_jobs.main()
+
+    # Check output
+    captured = capsys.readouterr()
+    output = json.loads(captured.out)
+
+    # changed_components should have all components
+    assert set(output["changed_components"]) == {"wifi", "sensor", "airthings_ble"}
+    # changed_components_with_tests should only have components with test files
+    assert set(output["changed_components_with_tests"]) == {"wifi", "sensor"}
+    # component_test_count should be based on components with tests
+    assert output["component_test_count"] == 2
diff --git a/tests/script/test_helpers.py b/tests/script/test_helpers.py
index d0db08e6f7..63f1f0e600 100644
--- a/tests/script/test_helpers.py
+++ b/tests/script/test_helpers.py
@@ -183,6 +183,61 @@ def test_get_changed_files_github_actions_pull_request(
         assert result == expected_files
 
 
+def test_get_changed_files_github_actions_pull_request_large_pr(
+    monkeypatch: MonkeyPatch,
+) -> None:
+    """Test _get_changed_files_github_actions fallback for PRs with >300 files."""
+    monkeypatch.setenv("GITHUB_EVENT_NAME", "pull_request")
+
+    expected_files = ["file1.py", "file2.cpp"]
+
+    with (
+        patch("helpers._get_pr_number_from_github_env", return_value="10214"),
+        patch("helpers._get_changed_files_from_command") as mock_get,
+    ):
+        # First call fails with too many files error, second succeeds with API method
+        mock_get.side_effect = [
+            Exception("Sorry, the diff exceeded the maximum number of files (300)"),
+            expected_files,
+        ]
+
+        result = _get_changed_files_github_actions()
+
+        assert mock_get.call_count == 2
+        mock_get.assert_any_call(["gh", "pr", "diff", "10214", "--name-only"])
+        mock_get.assert_any_call(
+            [
+                "gh",
+                "api",
+                "repos/esphome/esphome/pulls/10214/files",
+                "--paginate",
+                "--jq",
+                ".[].filename",
+            ]
+        )
+        assert result == expected_files
+
+
+def test_get_changed_files_github_actions_pull_request_other_error(
+    monkeypatch: MonkeyPatch,
+) -> None:
+    """Test _get_changed_files_github_actions re-raises non-file-limit errors."""
+    monkeypatch.setenv("GITHUB_EVENT_NAME", "pull_request")
+
+    with (
+        patch("helpers._get_pr_number_from_github_env", return_value="1234"),
+        patch("helpers._get_changed_files_from_command") as mock_get,
+    ):
+        # Error that is not about file limit
+        mock_get.side_effect = Exception("Command failed: authentication required")
+
+        with pytest.raises(Exception, match="authentication required"):
+            _get_changed_files_github_actions()
+
+        # Should only be called once (no retry with API)
+        mock_get.assert_called_once_with(["gh", "pr", "diff", "1234", "--name-only"])
+
+
 def test_get_changed_files_github_actions_pull_request_no_pr_number(
     monkeypatch: MonkeyPatch,
 ) -> None:
@@ -315,9 +370,8 @@ def test_local_development_no_remotes_configured(monkeypatch: MonkeyPatch) -> No
         def side_effect_func(*args):
             if args == ("git", "remote"):
                 return "origin\nupstream\n"
-            else:
-                # All merge-base attempts fail
-                raise Exception("Command failed")
+            # All merge-base attempts fail
+            raise Exception("Command failed")
 
         mock_output.side_effect = side_effect_func
 
@@ -986,8 +1040,7 @@ def test_get_components_from_integration_fixtures() -> None:
 
     with (
         patch("pathlib.Path.glob") as mock_glob,
-        patch("builtins.open", create=True),
-        patch("yaml.safe_load", return_value=yaml_content),
+        patch("esphome.yaml_util.load_yaml", return_value=yaml_content),
     ):
         mock_glob.return_value = [mock_yaml_file]
 
diff --git a/tests/test_build_components/build_components_base.bk72xx-ard.yaml b/tests/test_build_components/build_components_base.bk72xx-ard.yaml
index 9a4e15d5cf..817acc3c39 100644
--- a/tests/test_build_components/build_components_base.bk72xx-ard.yaml
+++ b/tests/test_build_components/build_components_base.bk72xx-ard.yaml
@@ -3,7 +3,7 @@ esphome:
   friendly_name: $component_name
 
 bk72xx:
-  board: generic-bk7231n-qfn32-tuya
+  board: generic-bk7252
 
 logger:
   level: VERY_VERBOSE
diff --git a/tests/test_build_components/build_components_base.esp32-c3-idf.yaml b/tests/test_build_components/build_components_base.esp32-c3-idf.yaml
index 18584497f4..73a85467d3 100644
--- a/tests/test_build_components/build_components_base.esp32-c3-idf.yaml
+++ b/tests/test_build_components/build_components_base.esp32-c3-idf.yaml
@@ -6,6 +6,9 @@ esp32:
   board: lolin_c3_mini
   framework:
     type: esp-idf
+  # Use custom partition table with larger app partition (3MB)
+  # Default IDF partitions only allow 1.75MB which is too small for grouped tests
+  partitions: ../partitions_testing.csv
 
 logger:
   level: VERY_VERBOSE
diff --git a/tests/test_build_components/build_components_base.esp32-idf.yaml b/tests/test_build_components/build_components_base.esp32-idf.yaml
index a62a995e68..dcb951c1ed 100644
--- a/tests/test_build_components/build_components_base.esp32-idf.yaml
+++ b/tests/test_build_components/build_components_base.esp32-idf.yaml
@@ -3,9 +3,13 @@ esphome:
   friendly_name: $component_name
 
 esp32:
-  board: nodemcu-32s
+  # Use board with 8MB flash for testing large component groups
+  board: esp32-pico-devkitm-2
   framework:
     type: esp-idf
+  # Use custom partition table with larger app partitions (3MB each)
+  # Default IDF partitions only allow 1.75MB which is too small for grouped tests
+  partitions: ../partitions_testing.csv
 
 logger:
   level: VERY_VERBOSE
diff --git a/tests/test_build_components/build_components_base.nrf52-adafruit.yaml b/tests/test_build_components/build_components_base.nrf52-adafruit.yaml
new file mode 100644
index 0000000000..05e3a6387c
--- /dev/null
+++ b/tests/test_build_components/build_components_base.nrf52-adafruit.yaml
@@ -0,0 +1,16 @@
+esphome:
+  name: componenttestnrf52
+  friendly_name: $component_name
+
+nrf52:
+  board: adafruit_itsybitsy_nrf52840
+  bootloader: adafruit_nrf52_sd140_v6
+
+logger:
+  level: VERY_VERBOSE
+
+packages:
+  component_under_test: !include
+    file: $component_test_file
+    vars:
+      component_test_file: $component_test_file
diff --git a/tests/test_build_components/build_components_base.nrf52-mcumgr.yaml b/tests/test_build_components/build_components_base.nrf52-mcumgr.yaml
new file mode 100644
index 0000000000..04211ffdfe
--- /dev/null
+++ b/tests/test_build_components/build_components_base.nrf52-mcumgr.yaml
@@ -0,0 +1,15 @@
+esphome:
+  name: componenttestnrf52
+  friendly_name: $component_name
+
+nrf52:
+  board: adafruit_feather_nrf52840
+
+logger:
+  level: VERY_VERBOSE
+
+packages:
+  component_under_test: !include
+    file: $component_test_file
+    vars:
+      component_test_file: $component_test_file
diff --git a/tests/test_build_components/build_components_base.nrf52-xiao-ble.yaml b/tests/test_build_components/build_components_base.nrf52-xiao-ble.yaml
new file mode 100644
index 0000000000..2f3f91d957
--- /dev/null
+++ b/tests/test_build_components/build_components_base.nrf52-xiao-ble.yaml
@@ -0,0 +1,15 @@
+esphome:
+  name: componenttestnrf52
+  friendly_name: $component_name
+
+nrf52:
+  board: xiao_ble
+
+logger:
+  level: VERY_VERBOSE
+
+packages:
+  component_under_test: !include
+    file: $component_test_file
+    vars:
+      component_test_file: $component_test_file
diff --git a/tests/test_build_components/common/README.md b/tests/test_build_components/common/README.md
new file mode 100644
index 0000000000..76f14b8664
--- /dev/null
+++ b/tests/test_build_components/common/README.md
@@ -0,0 +1,248 @@
+# Common Bus Configurations for Component Tests
+
+This directory contains standardized bus configurations (I2C, SPI, UART, Modbus, BLE) that component tests use, enabling multiple components to be tested together with intelligent grouping.
+
+## Purpose
+
+These common configs allow multiple components to **share a single bus**, dramatically reducing CI time by compiling multiple compatible components together. Components with identical bus configurations are automatically grouped and tested together.
+
+## Structure
+
+```
+common/
+├── i2c/              # Standard I2C (50kHz)
+│   ├── esp32-idf.yaml
+│   ├── esp32-ard.yaml
+│   ├── esp32-c3-idf.yaml
+│   ├── esp32-c3-ard.yaml
+│   ├── esp32-s2-idf.yaml
+│   ├── esp32-s2-ard.yaml
+│   ├── esp32-s3-idf.yaml
+│   ├── esp32-s3-ard.yaml
+│   ├── esp8266-ard.yaml
+│   ├── rp2040-ard.yaml
+│   └── bk72xx-ard.yaml
+├── i2c_low_freq/     # Low frequency I2C (10kHz)
+│   └── (same platform variants)
+├── spi/
+│   └── (same platform variants)
+├── uart/
+│   ├── esp32-idf.yaml
+│   ├── esp32-c3-idf.yaml
+│   ├── esp8266-ard.yaml
+│   └── rp2040-ard.yaml
+├── modbus/           # Modbus (includes uart via packages)
+│   ├── esp32-idf.yaml
+│   ├── esp32-c3-idf.yaml
+│   ├── esp8266-ard.yaml
+│   └── rp2040-ard.yaml
+└── ble/
+    ├── esp32-idf.yaml
+    ├── esp32-ard.yaml
+    └── esp32-c3-idf.yaml
+```
+
+## How It Works
+
+### Component Test Structure
+Each component test includes the common bus config:
+
+```yaml
+# tests/components/bh1750/test.esp32-idf.yaml
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
+<<: !include common.yaml
+```
+
+The common config provides:
+- Standardized pin assignments
+- A shared bus instance (`i2c_bus`, `spi_bus`, `uart_bus`, `modbus_bus`, etc.)
+
+The component's `common.yaml` references the shared bus:
+```yaml
+# tests/components/bh1750/common.yaml
+sensor:
+  - platform: bh1750
+    i2c_id: i2c_bus
+    name: Living Room Brightness
+    address: 0x23
+```
+
+### Intelligent Grouping (Implemented)
+Components with identical bus configurations are automatically grouped and tested together:
+
+```yaml
+# Auto-generated merged config (created by test_build_components.py)
+packages:
+  i2c: !include ../../test_build_components/common/i2c/esp32-idf.yaml
+
+sensor:
+  - platform: bme280_i2c
+    i2c_id: i2c_bus
+    temperature:
+      name: BME280 Temperature
+  - platform: bh1750
+    i2c_id: i2c_bus
+    name: BH1750 Illuminance
+  - platform: sht3xd
+    i2c_id: i2c_bus
+    temperature:
+      name: SHT3xD Temperature
+```
+
+**Result**: 3 components compile in one test instead of 3 separate tests!
+
+### Package Dependencies
+Some packages include other packages to avoid duplication:
+
+```yaml
+# tests/test_build_components/common/modbus/esp32-idf.yaml
+packages:
+  uart: !include ../uart/esp32-idf.yaml  # Modbus requires UART
+
+substitutions:
+  flow_control_pin: GPIO4
+
+modbus:
+  - id: modbus_bus
+    uart_id: uart_bus
+    flow_control_pin: ${flow_control_pin}
+```
+
+Components using `modbus` packages automatically get `uart` as well.
+
+## Pin Allocations
+
+### I2C (Standard - 50kHz)
+- **ESP32 IDF**: SCL=GPIO16, SDA=GPIO17
+- **ESP32 Arduino**: SCL=GPIO22, SDA=GPIO21
+- **ESP32-C3**: SCL=GPIO5, SDA=GPIO4
+- **ESP32-S2/S3**: SCL=GPIO9, SDA=GPIO8
+- **ESP8266**: SCL=GPIO5, SDA=GPIO4
+- **RP2040**: SCL=GPIO5, SDA=GPIO4
+- **BK72xx**: SCL=P20, SDA=P21
+
+### I2C Low Frequency (10kHz)
+Same pin allocations as standard I2C, but with 10kHz frequency for components requiring slower speeds.
+
+### SPI
+- **ESP32**: CLK=GPIO18, MOSI=GPIO23, MISO=GPIO19
+- **ESP32-C3**: CLK=GPIO6, MOSI=GPIO7, MISO=GPIO5
+- **ESP32-S2**: CLK=GPIO36, MOSI=GPIO35, MISO=GPIO37
+- **ESP32-S3**: CLK=GPIO40, MOSI=GPIO6, MISO=GPIO41
+- **ESP8266**: CLK=GPIO14, MOSI=GPIO13, MISO=GPIO12
+- **RP2040**: CLK=GPIO18, MOSI=GPIO19, MISO=GPIO16
+- CS pins are component-specific (each SPI device needs unique CS)
+
+### UART
+- **ESP32 IDF**: TX=GPIO17, RX=GPIO16 (baud: 19200)
+- **ESP32-C3 IDF**: TX=GPIO7, RX=GPIO6 (baud: 19200)
+- **ESP8266**: TX=GPIO1, RX=GPIO3 (baud: 19200)
+- **RP2040**: TX=GPIO0, RX=GPIO1 (baud: 19200)
+
+### Modbus (includes UART)
+Same UART pins as above, plus:
+- **flow_control_pin**: GPIO4 (all platforms)
+
+### BLE
+- **ESP32**: Shared `esp32_ble_tracker` infrastructure
+- Each component defines unique `ble_client` with different MAC addresses
+
+## Benefits
+
+1. **Shared bus = less duplication**
+   - 200+ I2C components use common bus configs
+   - 60+ SPI components use common bus configs
+   - 80+ UART components use common bus configs
+   - 6 Modbus components use common modbus configs (which include UART)
+
+2. **Intelligent grouping reduces CI time**
+   - Components with identical bus configs are automatically grouped
+   - Typical reduction: 80-90% fewer builds
+   - Example: 3 I2C components → 1 merged build (saves 2 builds)
+   - CI runs 5 component batches in parallel (configurable via `max-parallel` in `.github/workflows/ci.yml`)
+
+3. **Easier maintenance**
+   - Change bus pins for a platform once, affects all component tests
+   - Consistent pin assignments across all components
+   - Centralized bus configuration
+
+## Component Compatibility
+
+### Groupable Components
+Components are automatically grouped when they have:
+- Identical bus package references (e.g., all use `i2c/esp32-idf.yaml`)
+- No local file references (`$component_dir`)
+- No `!extend` or `!remove` directives
+- Proper bus ID references (`i2c_id: i2c_bus`, `spi_id: spi_bus`, etc.)
+
+### Non-Groupable Components
+Components tested individually when they:
+- Use different bus frequencies (e.g., `i2c` vs `i2c_low_freq`)
+- Reference local files with `$component_dir`
+- Are platform components (abstract base classes with `IS_PLATFORM_COMPONENT = True`)
+- Define buses directly (not migrated to packages yet)
+- Are in `ISOLATED_COMPONENTS` list (known build conflicts)
+
+### Bus Variants
+- **i2c** (50kHz): Standard I2C frequency for most sensors
+- **i2c_low_freq** (10kHz): For sensors requiring slower I2C speeds
+- **modbus**: Includes UART via package dependencies
+
+### Making Components Groupable
+
+**WARNING**: Using `!extend` or `!remove` directives in component test files prevents automatic component grouping in CI, making builds slower.
+
+When platform-specific parameters are needed, inline the full configuration rather than using `!extend` or `!remove`. This allows the component to be grouped with other compatible components, reducing CI build time.
+
+**Note**: Some components legitimately require `!extend` or `!remove` for platform-specific features (e.g., `adc` removing ESP32-only `attenuation` parameter on ESP8266). These are correctly identified as non-groupable.
+
+## Testing Components
+
+### Testing Individual Components
+Test specific components using `test_build_components`:
+```bash
+# Test a single component
+./script/test_build_components -c bme280_i2c -t esp32-idf -e config
+
+# Test multiple components
+./script/test_build_components -c bme280_i2c,bh1750,sht3xd -t esp32-idf -e compile
+```
+
+### Testing All Components Together
+To verify that all components can be tested together without ID conflicts or configuration issues:
+```bash
+./script/test_component_grouping.py -e config --all
+```
+
+This tests all components in a single build to catch conflicts that might not appear when testing components individually. This is useful for:
+- Detecting ID conflicts between components
+- Validating that components can coexist in the same configuration
+- Ensuring proper `i2c_id`, `spi_id`, `uart_id` specifications
+
+Use `-e config` for fast configuration validation, or `-e compile` for full compilation testing.
+
+### Testing Component Groups
+Test specific groups of components by bus signature:
+```bash
+# Test all I2C components together
+./script/test_component_grouping.py -s i2c -e config
+
+# Test with custom group sizes
+./script/test_component_grouping.py --min-size 5 --max-size 20 --max-groups 3
+```
+
+## Implementation Details
+
+### Scripts
+- `script/analyze_component_buses.py`: Analyzes components to detect bus usage and grouping compatibility
+- `script/merge_component_configs.py`: Merges multiple component configs into a single test file
+- `script/test_build_components.py`: Main test runner with intelligent grouping
+- `script/test_component_grouping.py`: Test component groups or all components together
+- `script/split_components_for_ci.py`: Splits components into batches for parallel CI execution
+
+### Configuration
+- `.github/workflows/ci.yml`: CI workflow with `max-parallel: 5` for component testing
+- Package dependencies defined in `PACKAGE_DEPENDENCIES` (e.g., modbus → uart)
+- Base bus components excluded from migration warnings: `i2c`, `spi`, `uart`, `modbus`, `canbus`
diff --git a/tests/test_build_components/common/ble/esp32-ard.yaml b/tests/test_build_components/common/ble/esp32-ard.yaml
new file mode 100644
index 0000000000..e1b5aec694
--- /dev/null
+++ b/tests/test_build_components/common/ble/esp32-ard.yaml
@@ -0,0 +1,9 @@
+# Common BLE tracker configuration for ESP32 Arduino tests
+# BLE client components share this tracker infrastructure
+# Each component defines its own ble_client with unique MAC address
+
+esp32_ble_tracker:
+  scan_parameters:
+    interval: 1100ms
+    window: 1100ms
+    active: true
diff --git a/tests/test_build_components/common/ble/esp32-c3-idf.yaml b/tests/test_build_components/common/ble/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..6671722f21
--- /dev/null
+++ b/tests/test_build_components/common/ble/esp32-c3-idf.yaml
@@ -0,0 +1,9 @@
+# Common BLE tracker configuration for ESP32-C3 IDF tests
+# BLE client components share this tracker infrastructure
+# Each component defines its own ble_client with unique MAC address
+
+esp32_ble_tracker:
+  scan_parameters:
+    interval: 1100ms
+    window: 1100ms
+    active: true
diff --git a/tests/test_build_components/common/ble/esp32-idf.yaml b/tests/test_build_components/common/ble/esp32-idf.yaml
new file mode 100644
index 0000000000..7cdcea71f0
--- /dev/null
+++ b/tests/test_build_components/common/ble/esp32-idf.yaml
@@ -0,0 +1,9 @@
+# Common BLE tracker configuration for ESP32 IDF tests
+# BLE client components share this tracker infrastructure
+# Each component defines its own ble_client with unique MAC address
+
+esp32_ble_tracker:
+  scan_parameters:
+    interval: 1100ms
+    window: 1100ms
+    active: true
diff --git a/tests/test_build_components/common/i2c/bk72xx-ard.yaml b/tests/test_build_components/common/i2c/bk72xx-ard.yaml
new file mode 100644
index 0000000000..7d00546721
--- /dev/null
+++ b/tests/test_build_components/common/i2c/bk72xx-ard.yaml
@@ -0,0 +1,11 @@
+# Common I2C configuration for BK72XX Arduino tests
+
+substitutions:
+  scl_pin: P26
+  sda_pin: P24
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    scan: true
diff --git a/tests/test_build_components/common/i2c/esp32-ard.yaml b/tests/test_build_components/common/i2c/esp32-ard.yaml
new file mode 100644
index 0000000000..fff09ab85c
--- /dev/null
+++ b/tests/test_build_components/common/i2c/esp32-ard.yaml
@@ -0,0 +1,11 @@
+# Common I2C configuration for ESP32 Arduino tests
+
+substitutions:
+  scl_pin: GPIO22
+  sda_pin: GPIO21
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    scan: true
diff --git a/tests/test_build_components/common/i2c/esp32-c3-ard.yaml b/tests/test_build_components/common/i2c/esp32-c3-ard.yaml
new file mode 100644
index 0000000000..0216dccbfa
--- /dev/null
+++ b/tests/test_build_components/common/i2c/esp32-c3-ard.yaml
@@ -0,0 +1,11 @@
+# Common I2C configuration for ESP32-C3 Arduino tests
+
+substitutions:
+  scl_pin: GPIO8
+  sda_pin: GPIO10
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    scan: true
diff --git a/tests/test_build_components/common/i2c/esp32-c3-idf.yaml b/tests/test_build_components/common/i2c/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..fd873b6db8
--- /dev/null
+++ b/tests/test_build_components/common/i2c/esp32-c3-idf.yaml
@@ -0,0 +1,11 @@
+# Common I2C configuration for ESP32-C3 IDF tests
+
+substitutions:
+  scl_pin: GPIO8
+  sda_pin: GPIO10
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    scan: true
diff --git a/tests/test_build_components/common/i2c/esp32-idf.yaml b/tests/test_build_components/common/i2c/esp32-idf.yaml
new file mode 100644
index 0000000000..bb2376e7d6
--- /dev/null
+++ b/tests/test_build_components/common/i2c/esp32-idf.yaml
@@ -0,0 +1,13 @@
+# Common I2C configuration for ESP32 IDF tests
+# Provides a shared I2C bus that all components can use
+# Components will auto-use this bus if they don't specify i2c_id
+
+substitutions:
+  scl_pin: GPIO16
+  sda_pin: GPIO17
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    scan: true
diff --git a/tests/test_build_components/common/i2c/esp32-p4-idf.yaml b/tests/test_build_components/common/i2c/esp32-p4-idf.yaml
new file mode 100644
index 0000000000..0c7d7e3414
--- /dev/null
+++ b/tests/test_build_components/common/i2c/esp32-p4-idf.yaml
@@ -0,0 +1,13 @@
+# Common I2C configuration for ESP32-P4 IDF tests
+# Provides a shared I2C bus that all components can use
+# Components will auto-use this bus if they don't specify i2c_id
+
+substitutions:
+  scl_pin: GPIO8
+  sda_pin: GPIO7
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    scan: true
diff --git a/tests/test_build_components/common/i2c/esp32-s2-ard.yaml b/tests/test_build_components/common/i2c/esp32-s2-ard.yaml
new file mode 100644
index 0000000000..d8201db042
--- /dev/null
+++ b/tests/test_build_components/common/i2c/esp32-s2-ard.yaml
@@ -0,0 +1,11 @@
+# Common I2C configuration for ESP32-S2 Arduino tests
+
+substitutions:
+  scl_pin: GPIO16
+  sda_pin: GPIO17
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    scan: true
diff --git a/tests/test_build_components/common/i2c/esp32-s2-idf.yaml b/tests/test_build_components/common/i2c/esp32-s2-idf.yaml
new file mode 100644
index 0000000000..f23f7e35cc
--- /dev/null
+++ b/tests/test_build_components/common/i2c/esp32-s2-idf.yaml
@@ -0,0 +1,11 @@
+# Common I2C configuration for ESP32-S2 IDF tests
+
+substitutions:
+  scl_pin: GPIO16
+  sda_pin: GPIO17
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    scan: true
diff --git a/tests/test_build_components/common/i2c/esp32-s3-ard.yaml b/tests/test_build_components/common/i2c/esp32-s3-ard.yaml
new file mode 100644
index 0000000000..3499770cb7
--- /dev/null
+++ b/tests/test_build_components/common/i2c/esp32-s3-ard.yaml
@@ -0,0 +1,11 @@
+# Common I2C configuration for ESP32-S3 Arduino tests
+
+substitutions:
+  scl_pin: GPIO9
+  sda_pin: GPIO8
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    scan: true
diff --git a/tests/test_build_components/common/i2c/esp32-s3-idf.yaml b/tests/test_build_components/common/i2c/esp32-s3-idf.yaml
new file mode 100644
index 0000000000..9f5b002eb3
--- /dev/null
+++ b/tests/test_build_components/common/i2c/esp32-s3-idf.yaml
@@ -0,0 +1,11 @@
+# Common I2C configuration for ESP32-S3 IDF tests
+
+substitutions:
+  scl_pin: GPIO9
+  sda_pin: GPIO8
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    scan: true
diff --git a/tests/test_build_components/common/i2c/esp8266-ard.yaml b/tests/test_build_components/common/i2c/esp8266-ard.yaml
new file mode 100644
index 0000000000..6aa28524b2
--- /dev/null
+++ b/tests/test_build_components/common/i2c/esp8266-ard.yaml
@@ -0,0 +1,11 @@
+# Common I2C configuration for ESP8266 Arduino tests
+
+substitutions:
+  scl_pin: GPIO5
+  sda_pin: GPIO4
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    scan: true
diff --git a/tests/test_build_components/common/i2c/rp2040-ard.yaml b/tests/test_build_components/common/i2c/rp2040-ard.yaml
new file mode 100644
index 0000000000..cb241aef76
--- /dev/null
+++ b/tests/test_build_components/common/i2c/rp2040-ard.yaml
@@ -0,0 +1,11 @@
+# Common I2C configuration for RP2040 Arduino tests
+
+substitutions:
+  scl_pin: GPIO5
+  sda_pin: GPIO4
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    scan: true
diff --git a/tests/test_build_components/common/i2c_camera/esp32-idf.yaml b/tests/test_build_components/common/i2c_camera/esp32-idf.yaml
new file mode 100644
index 0000000000..a6e7c264cb
--- /dev/null
+++ b/tests/test_build_components/common/i2c_camera/esp32-idf.yaml
@@ -0,0 +1,34 @@
+# I2C bus for camera sensor
+i2c:
+  - id: i2c_camera_bus
+    sda: 25
+    scl: 23
+    frequency: 400kHz
+
+esp32_camera:
+  name: ESP32 Camera
+  data_pins:
+    - number: 17
+    - number: 35
+    - number: 34
+    - number: 5
+    - number: 39
+    - number: 18
+    - number: 36
+    - number: 19
+  vsync_pin: 22
+  href_pin: 26
+  pixel_clock_pin: 21
+  external_clock:
+    pin: 27
+    frequency: 20MHz
+  i2c_id: i2c_camera_bus
+  reset_pin: 15
+  power_down_pin: 1
+  resolution: 640x480
+  jpeg_quality: 10
+  frame_buffer_location: PSRAM
+  on_image:
+    then:
+      - lambda: |-
+          ESP_LOGD("main", "image len=%d, data=%c", image.length, image.data[0]);
diff --git a/tests/test_build_components/common/i2c_low_freq/esp32-ard.yaml b/tests/test_build_components/common/i2c_low_freq/esp32-ard.yaml
new file mode 100644
index 0000000000..53d051f174
--- /dev/null
+++ b/tests/test_build_components/common/i2c_low_freq/esp32-ard.yaml
@@ -0,0 +1,12 @@
+# Common I2C configuration for ESP32 Arduino tests - Low Frequency (10kHz)
+
+substitutions:
+  scl_pin: GPIO22
+  sda_pin: GPIO21
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    frequency: 10kHz
+    scan: true
diff --git a/tests/test_build_components/common/i2c_low_freq/esp32-c3-ard.yaml b/tests/test_build_components/common/i2c_low_freq/esp32-c3-ard.yaml
new file mode 100644
index 0000000000..c9856c6b29
--- /dev/null
+++ b/tests/test_build_components/common/i2c_low_freq/esp32-c3-ard.yaml
@@ -0,0 +1,12 @@
+# Common I2C configuration for ESP32-C3 Arduino tests - Low Frequency (10kHz)
+
+substitutions:
+  scl_pin: GPIO6
+  sda_pin: GPIO5
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    frequency: 10kHz
+    scan: true
diff --git a/tests/test_build_components/common/i2c_low_freq/esp32-c3-idf.yaml b/tests/test_build_components/common/i2c_low_freq/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..e9d44b585e
--- /dev/null
+++ b/tests/test_build_components/common/i2c_low_freq/esp32-c3-idf.yaml
@@ -0,0 +1,12 @@
+# Common I2C configuration for ESP32-C3 IDF tests - Low Frequency (10kHz)
+
+substitutions:
+  scl_pin: GPIO5
+  sda_pin: GPIO4
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    frequency: 10kHz
+    scan: true
diff --git a/tests/test_build_components/common/i2c_low_freq/esp32-idf.yaml b/tests/test_build_components/common/i2c_low_freq/esp32-idf.yaml
new file mode 100644
index 0000000000..4afe220315
--- /dev/null
+++ b/tests/test_build_components/common/i2c_low_freq/esp32-idf.yaml
@@ -0,0 +1,13 @@
+# Common I2C configuration for ESP32 IDF tests - Low Frequency (10kHz)
+# For components that require I2C frequency <= 15kHz (ags10, ltr501, ltr_als_ps)
+
+substitutions:
+  scl_pin: GPIO16
+  sda_pin: GPIO17
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    frequency: 10kHz
+    scan: true
diff --git a/tests/test_build_components/common/i2c_low_freq/esp8266-ard.yaml b/tests/test_build_components/common/i2c_low_freq/esp8266-ard.yaml
new file mode 100644
index 0000000000..281177ebbd
--- /dev/null
+++ b/tests/test_build_components/common/i2c_low_freq/esp8266-ard.yaml
@@ -0,0 +1,12 @@
+# Common I2C configuration for ESP8266 Arduino tests - Low Frequency (10kHz)
+
+substitutions:
+  scl_pin: GPIO5
+  sda_pin: GPIO4
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    frequency: 10kHz
+    scan: true
diff --git a/tests/test_build_components/common/i2c_low_freq/rp2040-ard.yaml b/tests/test_build_components/common/i2c_low_freq/rp2040-ard.yaml
new file mode 100644
index 0000000000..67d893e733
--- /dev/null
+++ b/tests/test_build_components/common/i2c_low_freq/rp2040-ard.yaml
@@ -0,0 +1,12 @@
+# Common I2C configuration for RP2040 Arduino tests - Low Frequency (10kHz)
+
+substitutions:
+  scl_pin: GPIO5
+  sda_pin: GPIO4
+
+i2c:
+  - id: i2c_bus
+    scl: ${scl_pin}
+    sda: ${sda_pin}
+    frequency: 10kHz
+    scan: true
diff --git a/tests/test_build_components/common/modbus/bk72xx-ard.yaml b/tests/test_build_components/common/modbus/bk72xx-ard.yaml
new file mode 100644
index 0000000000..c428f0a7be
--- /dev/null
+++ b/tests/test_build_components/common/modbus/bk72xx-ard.yaml
@@ -0,0 +1,12 @@
+# Common Modbus configuration for BK72XX Arduino tests
+
+packages:
+  uart: !include ../uart/bk72xx-ard.yaml
+
+substitutions:
+  flow_control_pin: P6
+
+modbus:
+  - id: modbus_bus
+    uart_id: uart_bus
+    flow_control_pin: ${flow_control_pin}
diff --git a/tests/test_build_components/common/modbus/esp32-ard.yaml b/tests/test_build_components/common/modbus/esp32-ard.yaml
new file mode 100644
index 0000000000..f327cf266e
--- /dev/null
+++ b/tests/test_build_components/common/modbus/esp32-ard.yaml
@@ -0,0 +1,12 @@
+# Common Modbus configuration for ESP32 Arduino tests
+
+packages:
+  uart: !include ../uart/esp32-ard.yaml
+
+substitutions:
+  flow_control_pin: GPIO4
+
+modbus:
+  - id: modbus_bus
+    uart_id: uart_bus
+    flow_control_pin: ${flow_control_pin}
diff --git a/tests/test_build_components/common/modbus/esp32-c3-ard.yaml b/tests/test_build_components/common/modbus/esp32-c3-ard.yaml
new file mode 100644
index 0000000000..f12e5518c0
--- /dev/null
+++ b/tests/test_build_components/common/modbus/esp32-c3-ard.yaml
@@ -0,0 +1,12 @@
+# Common Modbus configuration for ESP32-C3 Arduino tests
+
+packages:
+  uart: !include ../uart/esp32-c3-ard.yaml
+
+substitutions:
+  flow_control_pin: GPIO4
+
+modbus:
+  - id: modbus_bus
+    uart_id: uart_bus
+    flow_control_pin: ${flow_control_pin}
diff --git a/tests/test_build_components/common/modbus/esp32-c3-idf.yaml b/tests/test_build_components/common/modbus/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..98fc11b0b7
--- /dev/null
+++ b/tests/test_build_components/common/modbus/esp32-c3-idf.yaml
@@ -0,0 +1,12 @@
+# Common Modbus configuration for ESP32-C3 IDF tests
+
+packages:
+  uart: !include ../uart/esp32-c3-idf.yaml
+
+substitutions:
+  flow_control_pin: GPIO4
+
+modbus:
+  - id: modbus_bus
+    uart_id: uart_bus
+    flow_control_pin: ${flow_control_pin}
diff --git a/tests/test_build_components/common/modbus/esp32-idf.yaml b/tests/test_build_components/common/modbus/esp32-idf.yaml
new file mode 100644
index 0000000000..c2d777c3d7
--- /dev/null
+++ b/tests/test_build_components/common/modbus/esp32-idf.yaml
@@ -0,0 +1,13 @@
+# Common Modbus configuration for ESP32 IDF tests
+# Provides a shared Modbus bus that all components can use
+
+packages:
+  uart: !include ../uart/esp32-idf.yaml
+
+substitutions:
+  flow_control_pin: GPIO4
+
+modbus:
+  - id: modbus_bus
+    uart_id: uart_bus
+    flow_control_pin: ${flow_control_pin}
diff --git a/tests/test_build_components/common/modbus/esp32-s2-ard.yaml b/tests/test_build_components/common/modbus/esp32-s2-ard.yaml
new file mode 100644
index 0000000000..a47036f379
--- /dev/null
+++ b/tests/test_build_components/common/modbus/esp32-s2-ard.yaml
@@ -0,0 +1,12 @@
+# Common Modbus configuration for ESP32-S2 Arduino tests
+
+packages:
+  uart: !include ../uart/esp32-s2-ard.yaml
+
+substitutions:
+  flow_control_pin: GPIO4
+
+modbus:
+  - id: modbus_bus
+    uart_id: uart_bus
+    flow_control_pin: ${flow_control_pin}
diff --git a/tests/test_build_components/common/modbus/esp32-s2-idf.yaml b/tests/test_build_components/common/modbus/esp32-s2-idf.yaml
new file mode 100644
index 0000000000..2cac82aa15
--- /dev/null
+++ b/tests/test_build_components/common/modbus/esp32-s2-idf.yaml
@@ -0,0 +1,12 @@
+# Common Modbus configuration for ESP32-S2 IDF tests
+
+packages:
+  uart: !include ../uart/esp32-s2-idf.yaml
+
+substitutions:
+  flow_control_pin: GPIO4
+
+modbus:
+  - id: modbus_bus
+    uart_id: uart_bus
+    flow_control_pin: ${flow_control_pin}
diff --git a/tests/test_build_components/common/modbus/esp32-s3-ard.yaml b/tests/test_build_components/common/modbus/esp32-s3-ard.yaml
new file mode 100644
index 0000000000..3031f57159
--- /dev/null
+++ b/tests/test_build_components/common/modbus/esp32-s3-ard.yaml
@@ -0,0 +1,12 @@
+# Common Modbus configuration for ESP32-S3 Arduino tests
+
+packages:
+  uart: !include ../uart/esp32-s3-ard.yaml
+
+substitutions:
+  flow_control_pin: GPIO4
+
+modbus:
+  - id: modbus_bus
+    uart_id: uart_bus
+    flow_control_pin: ${flow_control_pin}
diff --git a/tests/test_build_components/common/modbus/esp32-s3-idf.yaml b/tests/test_build_components/common/modbus/esp32-s3-idf.yaml
new file mode 100644
index 0000000000..0a0d4dbd07
--- /dev/null
+++ b/tests/test_build_components/common/modbus/esp32-s3-idf.yaml
@@ -0,0 +1,12 @@
+# Common Modbus configuration for ESP32-S3 IDF tests
+
+packages:
+  uart: !include ../uart/esp32-s3-idf.yaml
+
+substitutions:
+  flow_control_pin: GPIO4
+
+modbus:
+  - id: modbus_bus
+    uart_id: uart_bus
+    flow_control_pin: ${flow_control_pin}
diff --git a/tests/test_build_components/common/modbus/esp8266-ard.yaml b/tests/test_build_components/common/modbus/esp8266-ard.yaml
new file mode 100644
index 0000000000..fce4c6df1d
--- /dev/null
+++ b/tests/test_build_components/common/modbus/esp8266-ard.yaml
@@ -0,0 +1,12 @@
+# Common Modbus configuration for ESP8266 Arduino tests
+
+packages:
+  uart: !include ../uart/esp8266-ard.yaml
+
+substitutions:
+  flow_control_pin: GPIO5
+
+modbus:
+  - id: modbus_bus
+    uart_id: uart_bus
+    flow_control_pin: ${flow_control_pin}
diff --git a/tests/test_build_components/common/modbus/rp2040-ard.yaml b/tests/test_build_components/common/modbus/rp2040-ard.yaml
new file mode 100644
index 0000000000..264ad8944f
--- /dev/null
+++ b/tests/test_build_components/common/modbus/rp2040-ard.yaml
@@ -0,0 +1,12 @@
+# Common Modbus configuration for RP2040 Arduino tests
+
+packages:
+  uart: !include ../uart/rp2040-ard.yaml
+
+substitutions:
+  flow_control_pin: GPIO2
+
+modbus:
+  - id: modbus_bus
+    uart_id: uart_bus
+    flow_control_pin: ${flow_control_pin}
diff --git a/tests/test_build_components/common/qspi/esp32-s3-idf.yaml b/tests/test_build_components/common/qspi/esp32-s3-idf.yaml
new file mode 100644
index 0000000000..22c98ef664
--- /dev/null
+++ b/tests/test_build_components/common/qspi/esp32-s3-idf.yaml
@@ -0,0 +1,13 @@
+# Common QSPI configuration for ESP32-S3 IDF tests
+# For components that need QuadSPI (qspi_dbi displays)
+
+spi:
+  - id: quad_spi
+    type: quad
+    interface: spi3
+    clk_pin: 47
+    data_pins:
+      - 40
+      - 41
+      - 42
+      - 43
diff --git a/tests/test_build_components/common/remote_receiver/esp32-c3-idf.yaml b/tests/test_build_components/common/remote_receiver/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..ad5acedf55
--- /dev/null
+++ b/tests/test_build_components/common/remote_receiver/esp32-c3-idf.yaml
@@ -0,0 +1,16 @@
+# Common remote_receiver configuration for ESP32-C3 IDF tests
+# Provides a shared remote receiver that all components can use
+# Components will auto-use this receiver if they don't specify receiver_id
+
+substitutions:
+  remote_receiver_pin: GPIO5
+
+remote_receiver:
+  - id: rcvr
+    pin: ${remote_receiver_pin}
+    dump: all
+    tolerance: 25%
+    clock_resolution: 2000000
+    filter_symbols: 2
+    receive_symbols: 4
+    rmt_symbols: 64
diff --git a/tests/test_build_components/common/remote_receiver/esp32-idf.yaml b/tests/test_build_components/common/remote_receiver/esp32-idf.yaml
new file mode 100644
index 0000000000..2905e22233
--- /dev/null
+++ b/tests/test_build_components/common/remote_receiver/esp32-idf.yaml
@@ -0,0 +1,16 @@
+# Common remote_receiver configuration for ESP32 IDF tests
+# Provides a shared remote receiver that all components can use
+# Components will auto-use this receiver if they don't specify receiver_id
+
+substitutions:
+  remote_receiver_pin: GPIO32
+
+remote_receiver:
+  - id: rcvr
+    pin: ${remote_receiver_pin}
+    dump: all
+    tolerance: 25%
+    clock_resolution: 2000000
+    filter_symbols: 2
+    receive_symbols: 4
+    rmt_symbols: 64
diff --git a/tests/test_build_components/common/remote_receiver/esp8266-ard.yaml b/tests/test_build_components/common/remote_receiver/esp8266-ard.yaml
new file mode 100644
index 0000000000..e2472d00c5
--- /dev/null
+++ b/tests/test_build_components/common/remote_receiver/esp8266-ard.yaml
@@ -0,0 +1,12 @@
+# Common remote_receiver configuration for ESP8266 Arduino tests
+# Provides a shared remote receiver that all components can use
+# Components will auto-use this receiver if they don't specify receiver_id
+
+substitutions:
+  remote_receiver_pin: GPIO5
+
+remote_receiver:
+  id: rcvr
+  pin: ${remote_receiver_pin}
+  dump: all
+  tolerance: 25%
diff --git a/tests/test_build_components/common/remote_transmitter/bk72xx-ard.yaml b/tests/test_build_components/common/remote_transmitter/bk72xx-ard.yaml
new file mode 100644
index 0000000000..b951b8713f
--- /dev/null
+++ b/tests/test_build_components/common/remote_transmitter/bk72xx-ard.yaml
@@ -0,0 +1,11 @@
+# Common remote_transmitter configuration for BK72XX Arduino tests
+# Provides a shared remote transmitter that all components can use
+# Components will auto-use this transmitter if they don't specify transmitter_id
+
+substitutions:
+  remote_transmitter_pin: GPIO6
+
+remote_transmitter:
+  id: xmitr
+  pin: ${remote_transmitter_pin}
+  carrier_duty_percent: 50%
diff --git a/tests/test_build_components/common/remote_transmitter/esp32-ard.yaml b/tests/test_build_components/common/remote_transmitter/esp32-ard.yaml
new file mode 100644
index 0000000000..4378e328af
--- /dev/null
+++ b/tests/test_build_components/common/remote_transmitter/esp32-ard.yaml
@@ -0,0 +1,11 @@
+# Common remote_transmitter configuration for ESP32 Arduino tests
+# Provides a shared remote transmitter that all components can use
+# Components will auto-use this transmitter if they don't specify transmitter_id
+
+substitutions:
+  remote_transmitter_pin: GPIO2
+
+remote_transmitter:
+  id: xmitr
+  pin: ${remote_transmitter_pin}
+  carrier_duty_percent: 50%
diff --git a/tests/test_build_components/common/remote_transmitter/esp32-c3-idf.yaml b/tests/test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..b6b9a87fe1
--- /dev/null
+++ b/tests/test_build_components/common/remote_transmitter/esp32-c3-idf.yaml
@@ -0,0 +1,13 @@
+# Common remote_transmitter configuration for ESP32-C3 IDF tests
+# Provides a shared remote transmitter that all components can use
+# Components will auto-use this transmitter if they don't specify transmitter_id
+
+substitutions:
+  remote_transmitter_pin: GPIO2
+
+remote_transmitter:
+  - id: xmitr
+    pin: ${remote_transmitter_pin}
+    carrier_duty_percent: 50%
+    clock_resolution: 2000000
+    rmt_symbols: 64
diff --git a/tests/test_build_components/common/remote_transmitter/esp32-idf.yaml b/tests/test_build_components/common/remote_transmitter/esp32-idf.yaml
new file mode 100644
index 0000000000..1d771b3edd
--- /dev/null
+++ b/tests/test_build_components/common/remote_transmitter/esp32-idf.yaml
@@ -0,0 +1,13 @@
+# Common remote_transmitter configuration for ESP32 IDF tests
+# Provides a shared remote transmitter that all components can use
+# Components will auto-use this transmitter if they don't specify transmitter_id
+
+substitutions:
+  remote_transmitter_pin: GPIO2
+
+remote_transmitter:
+  - id: xmitr
+    pin: ${remote_transmitter_pin}
+    carrier_duty_percent: 50%
+    clock_resolution: 2000000
+    rmt_symbols: 64
diff --git a/tests/test_build_components/common/remote_transmitter/esp8266-ard.yaml b/tests/test_build_components/common/remote_transmitter/esp8266-ard.yaml
new file mode 100644
index 0000000000..3be59c7997
--- /dev/null
+++ b/tests/test_build_components/common/remote_transmitter/esp8266-ard.yaml
@@ -0,0 +1,11 @@
+# Common remote_transmitter configuration for ESP8266 Arduino tests
+# Provides a shared remote transmitter that all components can use
+# Components will auto-use this transmitter if they don't specify transmitter_id
+
+substitutions:
+  remote_transmitter_pin: GPIO2
+
+remote_transmitter:
+  id: xmitr
+  pin: ${remote_transmitter_pin}
+  carrier_duty_percent: 50%
diff --git a/tests/test_build_components/common/spi/bk72xx-ard.yaml b/tests/test_build_components/common/spi/bk72xx-ard.yaml
new file mode 100644
index 0000000000..471b147bfa
--- /dev/null
+++ b/tests/test_build_components/common/spi/bk72xx-ard.yaml
@@ -0,0 +1,12 @@
+# Common SPI configuration for BK72XX Arduino tests
+
+substitutions:
+  clk_pin: P10
+  mosi_pin: P11
+  miso_pin: P6
+
+spi:
+  - id: spi_bus
+    clk_pin: ${clk_pin}
+    mosi_pin: ${mosi_pin}
+    miso_pin: ${miso_pin}
diff --git a/tests/test_build_components/common/spi/esp32-ard.yaml b/tests/test_build_components/common/spi/esp32-ard.yaml
new file mode 100644
index 0000000000..816609688c
--- /dev/null
+++ b/tests/test_build_components/common/spi/esp32-ard.yaml
@@ -0,0 +1,12 @@
+# Common SPI configuration for ESP32 Arduino tests
+
+substitutions:
+  clk_pin: GPIO18
+  mosi_pin: GPIO23
+  miso_pin: GPIO19
+
+spi:
+  - id: spi_bus
+    clk_pin: ${clk_pin}
+    mosi_pin: ${mosi_pin}
+    miso_pin: ${miso_pin}
diff --git a/tests/test_build_components/common/spi/esp32-c3-ard.yaml b/tests/test_build_components/common/spi/esp32-c3-ard.yaml
new file mode 100644
index 0000000000..da3182f259
--- /dev/null
+++ b/tests/test_build_components/common/spi/esp32-c3-ard.yaml
@@ -0,0 +1,12 @@
+# Common SPI configuration for ESP32-C3 Arduino tests
+
+substitutions:
+  clk_pin: GPIO4
+  mosi_pin: GPIO6
+  miso_pin: GPIO5
+
+spi:
+  - id: spi_bus
+    clk_pin: ${clk_pin}
+    mosi_pin: ${mosi_pin}
+    miso_pin: ${miso_pin}
diff --git a/tests/test_build_components/common/spi/esp32-c3-idf.yaml b/tests/test_build_components/common/spi/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..6a8f76c38c
--- /dev/null
+++ b/tests/test_build_components/common/spi/esp32-c3-idf.yaml
@@ -0,0 +1,15 @@
+# Common SPI configuration for ESP32-C3 IDF tests
+# Provides a shared SPI bus that all components can use
+# Components will auto-use this bus if they don't specify spi_id
+# CS pins are component-specific
+
+substitutions:
+  clk_pin: GPIO4
+  mosi_pin: GPIO6
+  miso_pin: GPIO5
+
+spi:
+  - id: spi_bus
+    clk_pin: ${clk_pin}
+    mosi_pin: ${mosi_pin}
+    miso_pin: ${miso_pin}
diff --git a/tests/test_build_components/common/spi/esp32-idf.yaml b/tests/test_build_components/common/spi/esp32-idf.yaml
new file mode 100644
index 0000000000..c2c39a2bc0
--- /dev/null
+++ b/tests/test_build_components/common/spi/esp32-idf.yaml
@@ -0,0 +1,15 @@
+# Common SPI configuration for ESP32 IDF tests
+# Provides a shared SPI bus that all components can use
+# Components will auto-use this bus if they don't specify spi_id
+# CS pins are component-specific
+
+substitutions:
+  clk_pin: GPIO18
+  mosi_pin: GPIO23
+  miso_pin: GPIO19
+
+spi:
+  - id: spi_bus
+    clk_pin: ${clk_pin}
+    mosi_pin: ${mosi_pin}
+    miso_pin: ${miso_pin}
diff --git a/tests/test_build_components/common/spi/esp32-s2-ard.yaml b/tests/test_build_components/common/spi/esp32-s2-ard.yaml
new file mode 100644
index 0000000000..7c8997ae7f
--- /dev/null
+++ b/tests/test_build_components/common/spi/esp32-s2-ard.yaml
@@ -0,0 +1,12 @@
+# Common SPI configuration for ESP32-S2 Arduino tests
+
+substitutions:
+  clk_pin: GPIO36
+  mosi_pin: GPIO35
+  miso_pin: GPIO37
+
+spi:
+  - id: spi_bus
+    clk_pin: ${clk_pin}
+    mosi_pin: ${mosi_pin}
+    miso_pin: ${miso_pin}
diff --git a/tests/test_build_components/common/spi/esp32-s2-idf.yaml b/tests/test_build_components/common/spi/esp32-s2-idf.yaml
new file mode 100644
index 0000000000..afcd83e94e
--- /dev/null
+++ b/tests/test_build_components/common/spi/esp32-s2-idf.yaml
@@ -0,0 +1,12 @@
+# Common SPI configuration for ESP32-S2 IDF tests
+
+substitutions:
+  clk_pin: GPIO36
+  mosi_pin: GPIO35
+  miso_pin: GPIO37
+
+spi:
+  - id: spi_bus
+    clk_pin: ${clk_pin}
+    mosi_pin: ${mosi_pin}
+    miso_pin: ${miso_pin}
diff --git a/tests/test_build_components/common/spi/esp32-s3-ard.yaml b/tests/test_build_components/common/spi/esp32-s3-ard.yaml
new file mode 100644
index 0000000000..06d5f65771
--- /dev/null
+++ b/tests/test_build_components/common/spi/esp32-s3-ard.yaml
@@ -0,0 +1,12 @@
+# Common SPI configuration for ESP32-S3 Arduino tests
+
+substitutions:
+  clk_pin: GPIO40
+  mosi_pin: GPIO6
+  miso_pin: GPIO41
+
+spi:
+  - id: spi_bus
+    clk_pin: ${clk_pin}
+    mosi_pin: ${mosi_pin}
+    miso_pin: ${miso_pin}
diff --git a/tests/test_build_components/common/spi/esp32-s3-idf.yaml b/tests/test_build_components/common/spi/esp32-s3-idf.yaml
new file mode 100644
index 0000000000..ee47396ec7
--- /dev/null
+++ b/tests/test_build_components/common/spi/esp32-s3-idf.yaml
@@ -0,0 +1,12 @@
+# Common SPI configuration for ESP32-S3 IDF tests
+
+substitutions:
+  clk_pin: GPIO40
+  mosi_pin: GPIO6
+  miso_pin: GPIO41
+
+spi:
+  - id: spi_bus
+    clk_pin: ${clk_pin}
+    mosi_pin: ${mosi_pin}
+    miso_pin: ${miso_pin}
diff --git a/tests/test_build_components/common/spi/esp8266-ard.yaml b/tests/test_build_components/common/spi/esp8266-ard.yaml
new file mode 100644
index 0000000000..4320afebb9
--- /dev/null
+++ b/tests/test_build_components/common/spi/esp8266-ard.yaml
@@ -0,0 +1,12 @@
+# Common SPI configuration for ESP8266 Arduino tests
+
+substitutions:
+  clk_pin: GPIO14
+  mosi_pin: GPIO13
+  miso_pin: GPIO12
+
+spi:
+  - id: spi_bus
+    clk_pin: ${clk_pin}
+    mosi_pin: ${mosi_pin}
+    miso_pin: ${miso_pin}
diff --git a/tests/test_build_components/common/spi/rp2040-ard.yaml b/tests/test_build_components/common/spi/rp2040-ard.yaml
new file mode 100644
index 0000000000..916a636318
--- /dev/null
+++ b/tests/test_build_components/common/spi/rp2040-ard.yaml
@@ -0,0 +1,12 @@
+# Common SPI configuration for RP2040 Arduino tests
+
+substitutions:
+  clk_pin: GPIO18
+  mosi_pin: GPIO19
+  miso_pin: GPIO16
+
+spi:
+  - id: spi_bus
+    clk_pin: ${clk_pin}
+    mosi_pin: ${mosi_pin}
+    miso_pin: ${miso_pin}
diff --git a/tests/test_build_components/common/uart/bk72xx-ard.yaml b/tests/test_build_components/common/uart/bk72xx-ard.yaml
new file mode 100644
index 0000000000..8d1abca70b
--- /dev/null
+++ b/tests/test_build_components/common/uart/bk72xx-ard.yaml
@@ -0,0 +1,13 @@
+# Common UART configuration for BK72XX Arduino tests
+# Provides a shared UART bus that components can use
+# Components will auto-use this bus if they don't specify uart_id
+
+substitutions:
+  tx_pin: TX1
+  rx_pin: RX1
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 9600
diff --git a/tests/test_build_components/common/uart/esp32-ard.yaml b/tests/test_build_components/common/uart/esp32-ard.yaml
new file mode 100644
index 0000000000..805695def6
--- /dev/null
+++ b/tests/test_build_components/common/uart/esp32-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32 Arduino tests
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 9600
diff --git a/tests/test_build_components/common/uart/esp32-c3-ard.yaml b/tests/test_build_components/common/uart/esp32-c3-ard.yaml
new file mode 100644
index 0000000000..565b109f9a
--- /dev/null
+++ b/tests/test_build_components/common/uart/esp32-c3-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32-C3 Arduino tests
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 9600
diff --git a/tests/test_build_components/common/uart/esp32-c3-idf.yaml b/tests/test_build_components/common/uart/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..944aa013dd
--- /dev/null
+++ b/tests/test_build_components/common/uart/esp32-c3-idf.yaml
@@ -0,0 +1,13 @@
+# Common UART configuration for ESP32-C3 IDF tests
+# Provides a shared UART bus that components can use
+# Components will auto-use this bus if they don't specify uart_id
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 9600
diff --git a/tests/test_build_components/common/uart/esp32-idf.yaml b/tests/test_build_components/common/uart/esp32-idf.yaml
new file mode 100644
index 0000000000..95e5db9fb1
--- /dev/null
+++ b/tests/test_build_components/common/uart/esp32-idf.yaml
@@ -0,0 +1,13 @@
+# Common UART configuration for ESP32 IDF tests
+# Provides a shared UART bus that components can use
+# Components will auto-use this bus if they don't specify uart_id
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 9600
diff --git a/tests/test_build_components/common/uart/esp8266-ard.yaml b/tests/test_build_components/common/uart/esp8266-ard.yaml
new file mode 100644
index 0000000000..e326f4fc0d
--- /dev/null
+++ b/tests/test_build_components/common/uart/esp8266-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP8266 Arduino tests
+
+substitutions:
+  tx_pin: GPIO1
+  rx_pin: GPIO3
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 9600
diff --git a/tests/test_build_components/common/uart/rp2040-ard.yaml b/tests/test_build_components/common/uart/rp2040-ard.yaml
new file mode 100644
index 0000000000..cd1e54a13b
--- /dev/null
+++ b/tests/test_build_components/common/uart/rp2040-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for RP2040 Arduino tests
+
+substitutions:
+  tx_pin: GPIO0
+  rx_pin: GPIO1
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 9600
diff --git a/tests/test_build_components/common/uart_115200/esp32-ard.yaml b/tests/test_build_components/common/uart_115200/esp32-ard.yaml
new file mode 100644
index 0000000000..9102910f31
--- /dev/null
+++ b/tests/test_build_components/common/uart_115200/esp32-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32 Arduino tests - 115200 baud
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 115200
diff --git a/tests/test_build_components/common/uart_115200/esp32-c3-ard.yaml b/tests/test_build_components/common/uart_115200/esp32-c3-ard.yaml
new file mode 100644
index 0000000000..87a969c6a3
--- /dev/null
+++ b/tests/test_build_components/common/uart_115200/esp32-c3-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32-C3 Arduino tests - 115200 baud
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 115200
diff --git a/tests/test_build_components/common/uart_115200/esp32-c3-idf.yaml b/tests/test_build_components/common/uart_115200/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..f3768592e5
--- /dev/null
+++ b/tests/test_build_components/common/uart_115200/esp32-c3-idf.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32-C3 IDF tests - 115200 baud
+# For components that require UART baud rate 115200 (bl0906)
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 115200
diff --git a/tests/test_build_components/common/uart_115200/esp32-idf.yaml b/tests/test_build_components/common/uart_115200/esp32-idf.yaml
new file mode 100644
index 0000000000..e405f74fe7
--- /dev/null
+++ b/tests/test_build_components/common/uart_115200/esp32-idf.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32 IDF tests - 115200 baud
+# For components that require UART baud rate 115200 (bl0906)
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 115200
diff --git a/tests/test_build_components/common/uart_115200/esp8266-ard.yaml b/tests/test_build_components/common/uart_115200/esp8266-ard.yaml
new file mode 100644
index 0000000000..2dcf1c4a5d
--- /dev/null
+++ b/tests/test_build_components/common/uart_115200/esp8266-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP8266 Arduino tests - 115200 baud
+
+substitutions:
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 115200
diff --git a/tests/test_build_components/common/uart_115200/rp2040-ard.yaml b/tests/test_build_components/common/uart_115200/rp2040-ard.yaml
new file mode 100644
index 0000000000..62a7b5aed2
--- /dev/null
+++ b/tests/test_build_components/common/uart_115200/rp2040-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for RP2040 Arduino tests - 115200 baud
+
+substitutions:
+  tx_pin: GPIO0
+  rx_pin: GPIO1
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 115200
diff --git a/tests/test_build_components/common/uart_1200/esp32-ard.yaml b/tests/test_build_components/common/uart_1200/esp32-ard.yaml
new file mode 100644
index 0000000000..0ff5663d1f
--- /dev/null
+++ b/tests/test_build_components/common/uart_1200/esp32-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32 Arduino tests - 1200 baud
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 1200
diff --git a/tests/test_build_components/common/uart_1200/esp32-c3-ard.yaml b/tests/test_build_components/common/uart_1200/esp32-c3-ard.yaml
new file mode 100644
index 0000000000..81cad70d3c
--- /dev/null
+++ b/tests/test_build_components/common/uart_1200/esp32-c3-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32-C3 Arduino tests - 1200 baud
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 1200
diff --git a/tests/test_build_components/common/uart_1200/esp32-c3-idf.yaml b/tests/test_build_components/common/uart_1200/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..8f1dace337
--- /dev/null
+++ b/tests/test_build_components/common/uart_1200/esp32-c3-idf.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32-C3 IDF tests - 1200 baud
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 1200
diff --git a/tests/test_build_components/common/uart_1200/esp32-idf.yaml b/tests/test_build_components/common/uart_1200/esp32-idf.yaml
new file mode 100644
index 0000000000..38ad1b1459
--- /dev/null
+++ b/tests/test_build_components/common/uart_1200/esp32-idf.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32 IDF tests - 1200 baud
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 1200
diff --git a/tests/test_build_components/common/uart_1200/esp8266-ard.yaml b/tests/test_build_components/common/uart_1200/esp8266-ard.yaml
new file mode 100644
index 0000000000..84907a3a42
--- /dev/null
+++ b/tests/test_build_components/common/uart_1200/esp8266-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP8266 Arduino tests - 1200 baud
+
+substitutions:
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 1200
diff --git a/tests/test_build_components/common/uart_1200/rp2040-ard.yaml b/tests/test_build_components/common/uart_1200/rp2040-ard.yaml
new file mode 100644
index 0000000000..3a3b322ea8
--- /dev/null
+++ b/tests/test_build_components/common/uart_1200/rp2040-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for RP2040 Arduino tests - 1200 baud
+
+substitutions:
+  tx_pin: GPIO0
+  rx_pin: GPIO1
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 1200
diff --git a/tests/test_build_components/common/uart_1200_even/esp32-ard.yaml b/tests/test_build_components/common/uart_1200_even/esp32-ard.yaml
new file mode 100644
index 0000000000..f5f7f0669f
--- /dev/null
+++ b/tests/test_build_components/common/uart_1200_even/esp32-ard.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32 Arduino tests - 1200 baud even parity
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 1200
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_1200_even/esp32-c3-ard.yaml b/tests/test_build_components/common/uart_1200_even/esp32-c3-ard.yaml
new file mode 100644
index 0000000000..0b1e3ba61b
--- /dev/null
+++ b/tests/test_build_components/common/uart_1200_even/esp32-c3-ard.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32-C3 Arduino tests - 1200 baud even parity
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 1200
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_1200_even/esp32-c3-idf.yaml b/tests/test_build_components/common/uart_1200_even/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..1781babefb
--- /dev/null
+++ b/tests/test_build_components/common/uart_1200_even/esp32-c3-idf.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32-C3 IDF tests - 1200 baud, EVEN parity
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 1200
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_1200_even/esp32-idf.yaml b/tests/test_build_components/common/uart_1200_even/esp32-idf.yaml
new file mode 100644
index 0000000000..3b4b17f892
--- /dev/null
+++ b/tests/test_build_components/common/uart_1200_even/esp32-idf.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32 IDF tests - 1200 baud, EVEN parity
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 1200
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_1200_even/esp8266-ard.yaml b/tests/test_build_components/common/uart_1200_even/esp8266-ard.yaml
new file mode 100644
index 0000000000..54f7de1757
--- /dev/null
+++ b/tests/test_build_components/common/uart_1200_even/esp8266-ard.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP8266 Arduino tests - 1200 baud even parity
+
+substitutions:
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 1200
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_1200_even/rp2040-ard.yaml b/tests/test_build_components/common/uart_1200_even/rp2040-ard.yaml
new file mode 100644
index 0000000000..0e8bdeae1f
--- /dev/null
+++ b/tests/test_build_components/common/uart_1200_even/rp2040-ard.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for RP2040 Arduino tests - 1200 baud even parity
+
+substitutions:
+  tx_pin: GPIO0
+  rx_pin: GPIO1
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 1200
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_19200/esp32-ard.yaml b/tests/test_build_components/common/uart_19200/esp32-ard.yaml
new file mode 100644
index 0000000000..f4f04669da
--- /dev/null
+++ b/tests/test_build_components/common/uart_19200/esp32-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32 Arduino tests - 19200 baud
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 19200
diff --git a/tests/test_build_components/common/uart_19200/esp32-c3-ard.yaml b/tests/test_build_components/common/uart_19200/esp32-c3-ard.yaml
new file mode 100644
index 0000000000..925acdc34c
--- /dev/null
+++ b/tests/test_build_components/common/uart_19200/esp32-c3-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32-C3 Arduino tests - 19200 baud
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 19200
diff --git a/tests/test_build_components/common/uart_19200/esp32-c3-idf.yaml b/tests/test_build_components/common/uart_19200/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..0d765a88a4
--- /dev/null
+++ b/tests/test_build_components/common/uart_19200/esp32-c3-idf.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32-C3 IDF tests - 19200 baud
+# For components that require UART baud rate 19200 (bl0906)
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 19200
diff --git a/tests/test_build_components/common/uart_19200/esp32-idf.yaml b/tests/test_build_components/common/uart_19200/esp32-idf.yaml
new file mode 100644
index 0000000000..e7849508c7
--- /dev/null
+++ b/tests/test_build_components/common/uart_19200/esp32-idf.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32 IDF tests - 19200 baud
+# For components that require UART baud rate 19200 (bl0906)
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 19200
diff --git a/tests/test_build_components/common/uart_19200/esp8266-ard.yaml b/tests/test_build_components/common/uart_19200/esp8266-ard.yaml
new file mode 100644
index 0000000000..f01bc4590c
--- /dev/null
+++ b/tests/test_build_components/common/uart_19200/esp8266-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP8266 Arduino tests - 19200 baud
+
+substitutions:
+  tx_pin: GPIO1
+  rx_pin: GPIO3
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 19200
diff --git a/tests/test_build_components/common/uart_19200/rp2040-ard.yaml b/tests/test_build_components/common/uart_19200/rp2040-ard.yaml
new file mode 100644
index 0000000000..6ebd02d451
--- /dev/null
+++ b/tests/test_build_components/common/uart_19200/rp2040-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for RP2040 Arduino tests - 19200 baud
+
+substitutions:
+  tx_pin: GPIO0
+  rx_pin: GPIO1
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 19200
diff --git a/tests/test_build_components/common/uart_38400/esp32-ard.yaml b/tests/test_build_components/common/uart_38400/esp32-ard.yaml
new file mode 100644
index 0000000000..15da771ccc
--- /dev/null
+++ b/tests/test_build_components/common/uart_38400/esp32-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32 Arduino tests - 38400 baud
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 38400
diff --git a/tests/test_build_components/common/uart_38400/esp32-c3-ard.yaml b/tests/test_build_components/common/uart_38400/esp32-c3-ard.yaml
new file mode 100644
index 0000000000..8838f029dc
--- /dev/null
+++ b/tests/test_build_components/common/uart_38400/esp32-c3-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32-C3 Arduino tests - 38400 baud
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 38400
diff --git a/tests/test_build_components/common/uart_38400/esp32-c3-idf.yaml b/tests/test_build_components/common/uart_38400/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..d7d902af3d
--- /dev/null
+++ b/tests/test_build_components/common/uart_38400/esp32-c3-idf.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32-C3 IDF tests - 38400 baud
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 38400
diff --git a/tests/test_build_components/common/uart_38400/esp32-idf.yaml b/tests/test_build_components/common/uart_38400/esp32-idf.yaml
new file mode 100644
index 0000000000..f1c9587e27
--- /dev/null
+++ b/tests/test_build_components/common/uart_38400/esp32-idf.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32 IDF tests - 38400 baud
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 38400
diff --git a/tests/test_build_components/common/uart_38400/esp8266-ard.yaml b/tests/test_build_components/common/uart_38400/esp8266-ard.yaml
new file mode 100644
index 0000000000..b1a046ea5e
--- /dev/null
+++ b/tests/test_build_components/common/uart_38400/esp8266-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP8266 Arduino tests - 38400 baud
+
+substitutions:
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 38400
diff --git a/tests/test_build_components/common/uart_38400/rp2040-ard.yaml b/tests/test_build_components/common/uart_38400/rp2040-ard.yaml
new file mode 100644
index 0000000000..01b5e58ed7
--- /dev/null
+++ b/tests/test_build_components/common/uart_38400/rp2040-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for RP2040 Arduino tests - 38400 baud
+
+substitutions:
+  tx_pin: GPIO0
+  rx_pin: GPIO1
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 38400
diff --git a/tests/test_build_components/common/uart_4800/esp32-ard.yaml b/tests/test_build_components/common/uart_4800/esp32-ard.yaml
new file mode 100644
index 0000000000..7f7096e31d
--- /dev/null
+++ b/tests/test_build_components/common/uart_4800/esp32-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32 Arduino tests - 4800 baud
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 4800
diff --git a/tests/test_build_components/common/uart_4800/esp32-c3-ard.yaml b/tests/test_build_components/common/uart_4800/esp32-c3-ard.yaml
new file mode 100644
index 0000000000..3c814b76e4
--- /dev/null
+++ b/tests/test_build_components/common/uart_4800/esp32-c3-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32-C3 Arduino tests - 4800 baud
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 4800
diff --git a/tests/test_build_components/common/uart_4800/esp32-c3-idf.yaml b/tests/test_build_components/common/uart_4800/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..7b18789b19
--- /dev/null
+++ b/tests/test_build_components/common/uart_4800/esp32-c3-idf.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32-C3 IDF tests - 4800 baud
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 4800
diff --git a/tests/test_build_components/common/uart_4800/esp32-idf.yaml b/tests/test_build_components/common/uart_4800/esp32-idf.yaml
new file mode 100644
index 0000000000..c2bd5b3edd
--- /dev/null
+++ b/tests/test_build_components/common/uart_4800/esp32-idf.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP32 IDF tests - 4800 baud
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 4800
diff --git a/tests/test_build_components/common/uart_4800/esp8266-ard.yaml b/tests/test_build_components/common/uart_4800/esp8266-ard.yaml
new file mode 100644
index 0000000000..afdbf4a599
--- /dev/null
+++ b/tests/test_build_components/common/uart_4800/esp8266-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for ESP8266 Arduino tests - 4800 baud
+
+substitutions:
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 4800
diff --git a/tests/test_build_components/common/uart_4800/rp2040-ard.yaml b/tests/test_build_components/common/uart_4800/rp2040-ard.yaml
new file mode 100644
index 0000000000..3bf0d6ba47
--- /dev/null
+++ b/tests/test_build_components/common/uart_4800/rp2040-ard.yaml
@@ -0,0 +1,11 @@
+# Common UART configuration for RP2040 Arduino tests - 4800 baud
+
+substitutions:
+  tx_pin: GPIO0
+  rx_pin: GPIO1
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 4800
diff --git a/tests/test_build_components/common/uart_4800_even/esp32-ard.yaml b/tests/test_build_components/common/uart_4800_even/esp32-ard.yaml
new file mode 100644
index 0000000000..053848615b
--- /dev/null
+++ b/tests/test_build_components/common/uart_4800_even/esp32-ard.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32 Arduino tests - 4800 baud even parity
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 4800
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_4800_even/esp32-c3-ard.yaml b/tests/test_build_components/common/uart_4800_even/esp32-c3-ard.yaml
new file mode 100644
index 0000000000..b1370bc1cb
--- /dev/null
+++ b/tests/test_build_components/common/uart_4800_even/esp32-c3-ard.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32-C3 Arduino tests - 4800 baud even parity
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 4800
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_4800_even/esp32-c3-idf.yaml b/tests/test_build_components/common/uart_4800_even/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..173c768937
--- /dev/null
+++ b/tests/test_build_components/common/uart_4800_even/esp32-c3-idf.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32-C3 IDF tests - 4800 baud, EVEN parity
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 4800
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_4800_even/esp32-idf.yaml b/tests/test_build_components/common/uart_4800_even/esp32-idf.yaml
new file mode 100644
index 0000000000..eb850ec2dd
--- /dev/null
+++ b/tests/test_build_components/common/uart_4800_even/esp32-idf.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32 IDF tests - 4800 baud, EVEN parity
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 4800
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_4800_even/esp8266-ard.yaml b/tests/test_build_components/common/uart_4800_even/esp8266-ard.yaml
new file mode 100644
index 0000000000..0b6bd9eac1
--- /dev/null
+++ b/tests/test_build_components/common/uart_4800_even/esp8266-ard.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP8266 Arduino tests - 4800 baud even parity
+
+substitutions:
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 4800
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_4800_even/rp2040-ard.yaml b/tests/test_build_components/common/uart_4800_even/rp2040-ard.yaml
new file mode 100644
index 0000000000..c99421b791
--- /dev/null
+++ b/tests/test_build_components/common/uart_4800_even/rp2040-ard.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for RP2040 Arduino tests - 4800 baud even parity
+
+substitutions:
+  tx_pin: GPIO0
+  rx_pin: GPIO1
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 4800
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_9600_even/esp32-ard.yaml b/tests/test_build_components/common/uart_9600_even/esp32-ard.yaml
new file mode 100644
index 0000000000..0a04f10705
--- /dev/null
+++ b/tests/test_build_components/common/uart_9600_even/esp32-ard.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32 Arduino tests - 9600 baud even parity
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 9600
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_9600_even/esp32-c3-ard.yaml b/tests/test_build_components/common/uart_9600_even/esp32-c3-ard.yaml
new file mode 100644
index 0000000000..1341a91b4e
--- /dev/null
+++ b/tests/test_build_components/common/uart_9600_even/esp32-c3-ard.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32-C3 Arduino tests - 9600 baud even parity
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 9600
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_9600_even/esp32-c3-idf.yaml b/tests/test_build_components/common/uart_9600_even/esp32-c3-idf.yaml
new file mode 100644
index 0000000000..5a7bce2198
--- /dev/null
+++ b/tests/test_build_components/common/uart_9600_even/esp32-c3-idf.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32-C3 IDF tests - 9600 baud, EVEN parity
+
+substitutions:
+  tx_pin: GPIO20
+  rx_pin: GPIO21
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 9600
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_9600_even/esp32-idf.yaml b/tests/test_build_components/common/uart_9600_even/esp32-idf.yaml
new file mode 100644
index 0000000000..b60cf71b17
--- /dev/null
+++ b/tests/test_build_components/common/uart_9600_even/esp32-idf.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP32 IDF tests - 9600 baud, EVEN parity
+
+substitutions:
+  tx_pin: GPIO17
+  rx_pin: GPIO16
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 9600
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_9600_even/esp8266-ard.yaml b/tests/test_build_components/common/uart_9600_even/esp8266-ard.yaml
new file mode 100644
index 0000000000..300ec842df
--- /dev/null
+++ b/tests/test_build_components/common/uart_9600_even/esp8266-ard.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for ESP8266 Arduino tests - 9600 baud even parity
+
+substitutions:
+  tx_pin: GPIO4
+  rx_pin: GPIO5
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 9600
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_9600_even/rp2040-ard.yaml b/tests/test_build_components/common/uart_9600_even/rp2040-ard.yaml
new file mode 100644
index 0000000000..c281ae84b5
--- /dev/null
+++ b/tests/test_build_components/common/uart_9600_even/rp2040-ard.yaml
@@ -0,0 +1,12 @@
+# Common UART configuration for RP2040 Arduino tests - 9600 baud even parity
+
+substitutions:
+  tx_pin: GPIO0
+  rx_pin: GPIO1
+
+uart:
+  - id: uart_bus
+    tx_pin: ${tx_pin}
+    rx_pin: ${rx_pin}
+    baud_rate: 9600
+    parity: EVEN
diff --git a/tests/test_build_components/common/uart_bridge_2/esp32-idf.yaml b/tests/test_build_components/common/uart_bridge_2/esp32-idf.yaml
new file mode 100644
index 0000000000..ff8a2f8d13
--- /dev/null
+++ b/tests/test_build_components/common/uart_bridge_2/esp32-idf.yaml
@@ -0,0 +1,11 @@
+# Common configuration for 2-channel UART bridge/expander chips
+# Used by components like wk2132 that create 2 UART channels
+# Defines standardized UART IDs: uart_id_0, uart_id_1
+
+substitutions:
+  # These will be overridden by component-specific values
+  uart_bridge_address: "0x70"
+
+# Note: The actual UART instances are created by the bridge component
+# This package just ensures all bridge components use the same ID naming convention
+# so they can be grouped together without conflicts
diff --git a/tests/test_build_components/common/uart_bridge_2/esp32-s3-idf.yaml b/tests/test_build_components/common/uart_bridge_2/esp32-s3-idf.yaml
new file mode 100644
index 0000000000..ff8a2f8d13
--- /dev/null
+++ b/tests/test_build_components/common/uart_bridge_2/esp32-s3-idf.yaml
@@ -0,0 +1,11 @@
+# Common configuration for 2-channel UART bridge/expander chips
+# Used by components like wk2132 that create 2 UART channels
+# Defines standardized UART IDs: uart_id_0, uart_id_1
+
+substitutions:
+  # These will be overridden by component-specific values
+  uart_bridge_address: "0x70"
+
+# Note: The actual UART instances are created by the bridge component
+# This package just ensures all bridge components use the same ID naming convention
+# so they can be grouped together without conflicts
diff --git a/tests/test_build_components/common/uart_bridge_4/esp32-idf.yaml b/tests/test_build_components/common/uart_bridge_4/esp32-idf.yaml
new file mode 100644
index 0000000000..bb88037947
--- /dev/null
+++ b/tests/test_build_components/common/uart_bridge_4/esp32-idf.yaml
@@ -0,0 +1,11 @@
+# Common configuration for 4-channel UART bridge/expander chips
+# Used by components like wk2168, wk2204, wk2212 that create 4 UART channels
+# Defines standardized UART IDs: uart_id_0, uart_id_1, uart_id_2, uart_id_3
+
+substitutions:
+  # These will be overridden by component-specific values
+  uart_bridge_address: "0x70"
+
+# Note: The actual UART instances are created by the bridge component
+# This package just ensures all bridge components use the same ID naming convention
+# so they can be grouped together without conflicts
diff --git a/tests/test_build_components/common/uart_bridge_4/esp32-s3-idf.yaml b/tests/test_build_components/common/uart_bridge_4/esp32-s3-idf.yaml
new file mode 100644
index 0000000000..bb88037947
--- /dev/null
+++ b/tests/test_build_components/common/uart_bridge_4/esp32-s3-idf.yaml
@@ -0,0 +1,11 @@
+# Common configuration for 4-channel UART bridge/expander chips
+# Used by components like wk2168, wk2204, wk2212 that create 4 UART channels
+# Defines standardized UART IDs: uart_id_0, uart_id_1, uart_id_2, uart_id_3
+
+substitutions:
+  # These will be overridden by component-specific values
+  uart_bridge_address: "0x70"
+
+# Note: The actual UART instances are created by the bridge component
+# This package just ensures all bridge components use the same ID naming convention
+# so they can be grouped together without conflicts
diff --git a/tests/test_build_components/partitions_testing.csv b/tests/test_build_components/partitions_testing.csv
new file mode 100644
index 0000000000..0ca8c24e05
--- /dev/null
+++ b/tests/test_build_components/partitions_testing.csv
@@ -0,0 +1,10 @@
+# ESP-IDF Partition Table for ESPHome Component Testing
+# Single app partition to maximize space for large component group testing
+# Fits in 4MB flash
+# Name,   Type, SubType, Offset,  Size, Flags
+nvs,      data, nvs,     0x9000,  0x4000,
+otadata,  data, ota,     ,        0x2000,
+phy_init, data, phy,     ,        0x1000,
+factory,  app,  factory, 0x10000, 0x300000,
+nvs_key,  data, nvs_keys,,        0x1000,
+coredump, data, coredump,,        0xEB000,
diff --git a/tests/unit_tests/build_gen/test_platformio.py b/tests/unit_tests/build_gen/test_platformio.py
new file mode 100644
index 0000000000..a124dbc128
--- /dev/null
+++ b/tests/unit_tests/build_gen/test_platformio.py
@@ -0,0 +1,188 @@
+"""Tests for esphome.build_gen.platformio module."""
+
+from __future__ import annotations
+
+from collections.abc import Generator
+from pathlib import Path
+from unittest.mock import MagicMock, patch
+
+import pytest
+
+from esphome.build_gen import platformio
+from esphome.core import CORE
+
+
+@pytest.fixture
+def mock_update_storage_json() -> Generator[MagicMock]:
+    """Mock update_storage_json for all tests."""
+    with patch("esphome.build_gen.platformio.update_storage_json") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_write_file_if_changed() -> Generator[MagicMock]:
+    """Mock write_file_if_changed for tests."""
+    with patch("esphome.build_gen.platformio.write_file_if_changed") as mock:
+        yield mock
+
+
+def test_write_ini_creates_new_file(
+    tmp_path: Path, mock_update_storage_json: MagicMock
+) -> None:
+    """Test write_ini creates a new platformio.ini file."""
+    CORE.build_path = str(tmp_path)
+
+    content = """
+[env:test]
+platform = espressif32
+board = esp32dev
+framework = arduino
+"""
+
+    platformio.write_ini(content)
+
+    ini_file = tmp_path / "platformio.ini"
+    assert ini_file.exists()
+
+    file_content = ini_file.read_text()
+    assert content in file_content
+    assert platformio.INI_AUTO_GENERATE_BEGIN in file_content
+    assert platformio.INI_AUTO_GENERATE_END in file_content
+
+
+def test_write_ini_updates_existing_file(
+    tmp_path: Path, mock_update_storage_json: MagicMock
+) -> None:
+    """Test write_ini updates existing platformio.ini file."""
+    CORE.build_path = str(tmp_path)
+
+    # Create existing file with custom content
+    ini_file = tmp_path / "platformio.ini"
+    existing_content = f"""
+; Custom header
+[platformio]
+default_envs = test
+
+{platformio.INI_AUTO_GENERATE_BEGIN}
+; Old auto-generated content
+[env:old]
+platform = old
+{platformio.INI_AUTO_GENERATE_END}
+
+; Custom footer
+"""
+    ini_file.write_text(existing_content)
+
+    # New content to write
+    new_content = """
+[env:test]
+platform = espressif32
+board = esp32dev
+framework = arduino
+"""
+
+    platformio.write_ini(new_content)
+
+    file_content = ini_file.read_text()
+
+    # Check that custom parts are preserved
+    assert "; Custom header" in file_content
+    assert "[platformio]" in file_content
+    assert "default_envs = test" in file_content
+    assert "; Custom footer" in file_content
+
+    # Check that new content replaced old auto-generated content
+    assert new_content in file_content
+    assert "[env:old]" not in file_content
+    assert "platform = old" not in file_content
+
+
+def test_write_ini_preserves_custom_sections(
+    tmp_path: Path, mock_update_storage_json: MagicMock
+) -> None:
+    """Test write_ini preserves custom sections outside auto-generate markers."""
+    CORE.build_path = str(tmp_path)
+
+    # Create existing file with multiple custom sections
+    ini_file = tmp_path / "platformio.ini"
+    existing_content = f"""
+[platformio]
+src_dir = .
+include_dir = .
+
+[common]
+lib_deps =
+    Wire
+    SPI
+
+{platformio.INI_AUTO_GENERATE_BEGIN}
+[env:old]
+platform = old
+{platformio.INI_AUTO_GENERATE_END}
+
+[env:custom]
+upload_speed = 921600
+monitor_speed = 115200
+"""
+    ini_file.write_text(existing_content)
+
+    new_content = "[env:auto]\nplatform = new"
+
+    platformio.write_ini(new_content)
+
+    file_content = ini_file.read_text()
+
+    # All custom sections should be preserved
+    assert "[platformio]" in file_content
+    assert "src_dir = ." in file_content
+    assert "[common]" in file_content
+    assert "lib_deps" in file_content
+    assert "[env:custom]" in file_content
+    assert "upload_speed = 921600" in file_content
+
+    # New auto-generated content should replace old
+    assert "[env:auto]" in file_content
+    assert "platform = new" in file_content
+    assert "[env:old]" not in file_content
+
+
+def test_write_ini_no_change_when_content_same(
+    tmp_path: Path,
+    mock_update_storage_json: MagicMock,
+    mock_write_file_if_changed: MagicMock,
+) -> None:
+    """Test write_ini doesn't rewrite file when content is unchanged."""
+    CORE.build_path = str(tmp_path)
+
+    content = "[env:test]\nplatform = esp32"
+    full_content = (
+        f"{platformio.INI_BASE_FORMAT[0]}"
+        f"{platformio.INI_AUTO_GENERATE_BEGIN}\n"
+        f"{content}"
+        f"{platformio.INI_AUTO_GENERATE_END}"
+        f"{platformio.INI_BASE_FORMAT[1]}"
+    )
+
+    ini_file = tmp_path / "platformio.ini"
+    ini_file.write_text(full_content)
+
+    mock_write_file_if_changed.return_value = False  # Indicate no change
+    platformio.write_ini(content)
+
+    # write_file_if_changed should be called with the same content
+    mock_write_file_if_changed.assert_called_once()
+    call_args = mock_write_file_if_changed.call_args[0]
+    assert call_args[0] == ini_file
+    assert content in call_args[1]
+
+
+def test_write_ini_calls_update_storage_json(
+    tmp_path: Path, mock_update_storage_json: MagicMock
+) -> None:
+    """Test write_ini calls update_storage_json."""
+    CORE.build_path = str(tmp_path)
+
+    content = "[env:test]\nplatform = esp32"
+
+    platformio.write_ini(content)
+    mock_update_storage_json.assert_called_once()
diff --git a/tests/unit_tests/conftest.py b/tests/unit_tests/conftest.py
index aac5a642f6..fc61841500 100644
--- a/tests/unit_tests/conftest.py
+++ b/tests/unit_tests/conftest.py
@@ -9,8 +9,10 @@ not be part of a unit test suite.
 
 """
 
+from collections.abc import Generator
 from pathlib import Path
 import sys
+from unittest.mock import Mock, patch
 
 import pytest
 
@@ -36,3 +38,80 @@ def fixture_path() -> Path:
     Location of all fixture files.
     """
     return here / "fixtures"
+
+
+@pytest.fixture
+def setup_core(tmp_path: Path) -> Path:
+    """Set up CORE with test paths."""
+    CORE.config_path = tmp_path / "test.yaml"
+    return tmp_path
+
+
+@pytest.fixture
+def mock_write_file_if_changed() -> Generator[Mock, None, None]:
+    """Mock write_file_if_changed for storage_json."""
+    with patch("esphome.storage_json.write_file_if_changed") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_copy_file_if_changed() -> Generator[Mock, None, None]:
+    """Mock copy_file_if_changed for core.config."""
+    with patch("esphome.core.config.copy_file_if_changed") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_run_platformio_cli() -> Generator[Mock, None, None]:
+    """Mock run_platformio_cli for platformio_api."""
+    with patch("esphome.platformio_api.run_platformio_cli") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_run_platformio_cli_run() -> Generator[Mock, None, None]:
+    """Mock run_platformio_cli_run for platformio_api."""
+    with patch("esphome.platformio_api.run_platformio_cli_run") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_decode_pc() -> Generator[Mock, None, None]:
+    """Mock _decode_pc for platformio_api."""
+    with patch("esphome.platformio_api._decode_pc") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_run_external_command() -> Generator[Mock, None, None]:
+    """Mock run_external_command for platformio_api."""
+    with patch("esphome.platformio_api.run_external_command") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_run_git_command() -> Generator[Mock, None, None]:
+    """Mock run_git_command for git module."""
+    with patch("esphome.git.run_git_command") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_subprocess_run() -> Generator[Mock, None, None]:
+    """Mock subprocess.run for testing."""
+    with patch("subprocess.run") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_get_idedata() -> Generator[Mock, None, None]:
+    """Mock get_idedata for platformio_api."""
+    with patch("esphome.platformio_api.get_idedata") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_get_component() -> Generator[Mock, None, None]:
+    """Mock get_component for config module."""
+    with patch("esphome.config.get_component") as mock:
+        yield mock
diff --git a/tests/unit_tests/core/common.py b/tests/unit_tests/core/common.py
index 1848d5397b..daa429dc96 100644
--- a/tests/unit_tests/core/common.py
+++ b/tests/unit_tests/core/common.py
@@ -10,7 +10,7 @@ from esphome.core import CORE
 
 
 def load_config_from_yaml(
-    yaml_file: Callable[[str], str], yaml_content: str
+    yaml_file: Callable[[str], Path], yaml_content: str
 ) -> Config | None:
     """Load configuration from YAML content."""
     yaml_path = yaml_file(yaml_content)
@@ -25,7 +25,7 @@ def load_config_from_yaml(
 
 
 def load_config_from_fixture(
-    yaml_file: Callable[[str], str], fixture_name: str, fixtures_dir: Path
+    yaml_file: Callable[[str], Path], fixture_name: str, fixtures_dir: Path
 ) -> Config | None:
     """Load configuration from a fixture file."""
     fixture_path = fixtures_dir / fixture_name
diff --git a/tests/unit_tests/core/conftest.py b/tests/unit_tests/core/conftest.py
index 60d6738ce9..42e59c15e6 100644
--- a/tests/unit_tests/core/conftest.py
+++ b/tests/unit_tests/core/conftest.py
@@ -7,12 +7,12 @@ import pytest
 
 
 @pytest.fixture
-def yaml_file(tmp_path: Path) -> Callable[[str], str]:
+def yaml_file(tmp_path: Path) -> Callable[[str], Path]:
     """Create a temporary YAML file for testing."""
 
-    def _yaml_file(content: str) -> str:
+    def _yaml_file(content: str) -> Path:
         yaml_path = tmp_path / "test.yaml"
         yaml_path.write_text(content)
-        return str(yaml_path)
+        return yaml_path
 
     return _yaml_file
diff --git a/tests/unit_tests/core/test_config.py b/tests/unit_tests/core/test_config.py
index 46e3b513d7..4fddfc9678 100644
--- a/tests/unit_tests/core/test_config.py
+++ b/tests/unit_tests/core/test_config.py
@@ -1,20 +1,56 @@
 """Unit tests for core config functionality including areas and devices."""
 
 from collections.abc import Callable
+import os
 from pathlib import Path
+import types
 from typing import Any
+from unittest.mock import MagicMock, Mock, patch
 
 import pytest
 
 from esphome import config_validation as cv, core
-from esphome.const import CONF_AREA, CONF_AREAS, CONF_DEVICES
-from esphome.core.config import Area, validate_area_config
+from esphome.const import (
+    CONF_AREA,
+    CONF_AREAS,
+    CONF_BUILD_PATH,
+    CONF_DEVICES,
+    CONF_ESPHOME,
+    CONF_NAME,
+    CONF_NAME_ADD_MAC_SUFFIX,
+    KEY_CORE,
+)
+from esphome.core import CORE, config
+from esphome.core.config import (
+    Area,
+    preload_core_config,
+    valid_include,
+    valid_project_name,
+    validate_area_config,
+    validate_hostname,
+)
 
 from .common import load_config_from_fixture
 
 FIXTURES_DIR = Path(__file__).parent.parent / "fixtures" / "core" / "config"
 
 
+@pytest.fixture
+def mock_cg_with_include_capture() -> tuple[Mock, list[str]]:
+    """Mock code generation with include capture."""
+    includes_added: list[str] = []
+
+    with patch("esphome.core.config.cg") as mock_cg:
+        mock_raw_statement = MagicMock()
+
+        def capture_include(text: str) -> MagicMock:
+            includes_added.append(text)
+            return mock_raw_statement
+
+        mock_cg.RawStatement.side_effect = capture_include
+        yield mock_cg, includes_added
+
+
 def test_validate_area_config_with_string() -> None:
     """Test that string area config is converted to structured format."""
     result = validate_area_config("Living Room")
@@ -223,3 +259,596 @@ def test_device_duplicate_id(
     # Check for the specific error message from IDPassValidationStep
     captured = capsys.readouterr()
     assert "ID duplicate_device redefined!" in captured.out
+
+
+def test_add_platform_defines_priority() -> None:
+    """Test that _add_platform_defines runs after globals.
+
+    This ensures the fix for issue #10431 where sensor counts were incorrect
+    when lambdas were present. The function must run at a lower priority than
+    globals (-100.0) to ensure all components (including those using globals
+    in lambdas) have registered their entities before the count defines are
+    generated.
+
+    Regression test for https://github.com/esphome/esphome/issues/10431
+    """
+    # Import globals to check its priority
+    from esphome.components.globals import to_code as globals_to_code
+
+    # _add_platform_defines must run AFTER globals (lower priority number = runs later)
+    assert config._add_platform_defines.priority < globals_to_code.priority, (
+        f"_add_platform_defines priority ({config._add_platform_defines.priority}) must be lower than "
+        f"globals priority ({globals_to_code.priority}) to fix issue #10431 (sensor count bug with lambdas)"
+    )
+
+
+def test_valid_include_with_angle_brackets() -> None:
+    """Test valid_include accepts angle bracket includes."""
+    assert valid_include("") == ""
+
+
+def test_valid_include_with_valid_file(tmp_path: Path) -> None:
+    """Test valid_include accepts valid include files."""
+    CORE.config_path = tmp_path / "test.yaml"
+    include_file = tmp_path / "include.h"
+    include_file.touch()
+
+    assert valid_include(str(include_file)) == str(include_file)
+
+
+def test_valid_include_with_valid_directory(tmp_path: Path) -> None:
+    """Test valid_include accepts valid directories."""
+    CORE.config_path = tmp_path / "test.yaml"
+    include_dir = tmp_path / "includes"
+    include_dir.mkdir()
+
+    assert valid_include(str(include_dir)) == str(include_dir)
+
+
+def test_valid_include_invalid_extension(tmp_path: Path) -> None:
+    """Test valid_include rejects files with invalid extensions."""
+    CORE.config_path = tmp_path / "test.yaml"
+    invalid_file = tmp_path / "file.txt"
+    invalid_file.touch()
+
+    with pytest.raises(cv.Invalid, match="Include has invalid file extension"):
+        valid_include(str(invalid_file))
+
+
+def test_valid_project_name_valid() -> None:
+    """Test valid_project_name accepts valid project names."""
+    assert valid_project_name("esphome.my_project") == "esphome.my_project"
+
+
+def test_valid_project_name_no_namespace() -> None:
+    """Test valid_project_name rejects names without namespace."""
+    with pytest.raises(cv.Invalid, match="project name needs to have a namespace"):
+        valid_project_name("my_project")
+
+
+def test_valid_project_name_multiple_dots() -> None:
+    """Test valid_project_name rejects names with multiple dots."""
+    with pytest.raises(cv.Invalid, match="project name needs to have a namespace"):
+        valid_project_name("esphome.my.project")
+
+
+def test_validate_hostname_valid() -> None:
+    """Test validate_hostname accepts valid hostnames."""
+    config = {CONF_NAME: "my-device", CONF_NAME_ADD_MAC_SUFFIX: False}
+    assert validate_hostname(config) == config
+
+
+def test_validate_hostname_too_long() -> None:
+    """Test validate_hostname rejects hostnames that are too long."""
+    config = {
+        CONF_NAME: "a" * 32,  # 32 chars, max is 31
+        CONF_NAME_ADD_MAC_SUFFIX: False,
+    }
+    with pytest.raises(cv.Invalid, match="Hostnames can only be 31 characters long"):
+        validate_hostname(config)
+
+
+def test_validate_hostname_too_long_with_mac_suffix() -> None:
+    """Test validate_hostname accounts for MAC suffix length."""
+    config = {
+        CONF_NAME: "a" * 25,  # 25 chars, max is 24 with MAC suffix
+        CONF_NAME_ADD_MAC_SUFFIX: True,
+    }
+    with pytest.raises(cv.Invalid, match="Hostnames can only be 24 characters long"):
+        validate_hostname(config)
+
+
+def test_validate_hostname_with_underscore(caplog) -> None:
+    """Test validate_hostname warns about underscores."""
+    config = {CONF_NAME: "my_device", CONF_NAME_ADD_MAC_SUFFIX: False}
+    assert validate_hostname(config) == config
+    assert (
+        "Using the '_' (underscore) character in the hostname is discouraged"
+        in caplog.text
+    )
+
+
+def test_preload_core_config_basic(setup_core: Path) -> None:
+    """Test preload_core_config sets basic CORE attributes."""
+    config = {
+        CONF_ESPHOME: {
+            CONF_NAME: "test_device",
+        },
+        "esp32": {},
+    }
+    result = {}
+
+    platform = preload_core_config(config, result)
+
+    assert CORE.name == "test_device"
+    assert platform == "esp32"
+    assert KEY_CORE in CORE.data
+    assert CONF_BUILD_PATH in config[CONF_ESPHOME]
+    # Verify default build path is "build/"
+    build_path = config[CONF_ESPHOME][CONF_BUILD_PATH]
+    assert build_path.endswith(os.path.join("build", "test_device"))
+
+
+def test_preload_core_config_with_build_path(setup_core: Path) -> None:
+    """Test preload_core_config uses provided build path."""
+    config = {
+        CONF_ESPHOME: {
+            CONF_NAME: "test_device",
+            CONF_BUILD_PATH: "/custom/build/path",
+        },
+        "esp8266": {},
+    }
+    result = {}
+
+    platform = preload_core_config(config, result)
+
+    assert config[CONF_ESPHOME][CONF_BUILD_PATH] == "/custom/build/path"
+    assert platform == "esp8266"
+
+
+def test_preload_core_config_env_build_path(setup_core: Path) -> None:
+    """Test preload_core_config uses ESPHOME_BUILD_PATH env var."""
+    config = {
+        CONF_ESPHOME: {
+            CONF_NAME: "test_device",
+        },
+        "rp2040": {},
+    }
+    result = {}
+
+    with patch.dict(os.environ, {"ESPHOME_BUILD_PATH": "/env/build"}):
+        platform = preload_core_config(config, result)
+
+    assert CONF_BUILD_PATH in config[CONF_ESPHOME]
+    assert "test_device" in config[CONF_ESPHOME][CONF_BUILD_PATH]
+    # Verify it uses the env var path with device name appended
+    build_path = config[CONF_ESPHOME][CONF_BUILD_PATH]
+    expected_path = os.path.join("/env/build", "test_device")
+    assert build_path == expected_path or build_path == expected_path.replace(
+        "/", os.sep
+    )
+    assert platform == "rp2040"
+
+
+def test_preload_core_config_no_platform(setup_core: Path) -> None:
+    """Test preload_core_config raises when no platform is specified."""
+    config = {
+        CONF_ESPHOME: {
+            CONF_NAME: "test_device",
+        },
+    }
+    result = {}
+
+    # Mock _is_target_platform to avoid expensive component loading
+    with patch("esphome.core.config._is_target_platform") as mock_is_platform:
+        # Return True for known platforms
+        mock_is_platform.side_effect = lambda name: name in [
+            "esp32",
+            "esp8266",
+            "rp2040",
+        ]
+
+        with pytest.raises(cv.Invalid, match="Platform missing"):
+            preload_core_config(config, result)
+
+
+def test_preload_core_config_multiple_platforms(setup_core: Path) -> None:
+    """Test preload_core_config raises when multiple platforms are specified."""
+    config = {
+        CONF_ESPHOME: {
+            CONF_NAME: "test_device",
+        },
+        "esp32": {},
+        "esp8266": {},
+    }
+    result = {}
+
+    # Mock _is_target_platform to avoid expensive component loading
+    with patch("esphome.core.config._is_target_platform") as mock_is_platform:
+        # Return True for known platforms
+        mock_is_platform.side_effect = lambda name: name in [
+            "esp32",
+            "esp8266",
+            "rp2040",
+        ]
+
+        with pytest.raises(cv.Invalid, match="Found multiple target platform blocks"):
+            preload_core_config(config, result)
+
+
+def test_include_file_header(tmp_path: Path, mock_copy_file_if_changed: Mock) -> None:
+    """Test include_file adds include statement for header files."""
+    src_file = tmp_path / "source.h"
+    src_file.write_text("// Header content")
+
+    CORE.build_path = tmp_path / "build"
+
+    with patch("esphome.core.config.cg") as mock_cg:
+        # Mock RawStatement to capture the text
+        mock_raw_statement = MagicMock()
+        mock_raw_statement.text = ""
+
+        def raw_statement_side_effect(text):
+            mock_raw_statement.text = text
+            return mock_raw_statement
+
+        mock_cg.RawStatement.side_effect = raw_statement_side_effect
+
+        config.include_file(src_file, Path("test.h"))
+
+        mock_copy_file_if_changed.assert_called_once()
+        mock_cg.add_global.assert_called_once()
+        # Check that include statement was added
+        assert '#include "test.h"' in mock_raw_statement.text
+
+
+def test_include_file_cpp(tmp_path: Path, mock_copy_file_if_changed: Mock) -> None:
+    """Test include_file does not add include for cpp files."""
+    src_file = tmp_path / "source.cpp"
+    src_file.write_text("// CPP content")
+
+    CORE.build_path = tmp_path / "build"
+
+    with patch("esphome.core.config.cg") as mock_cg:
+        config.include_file(src_file, Path("test.cpp"))
+
+        mock_copy_file_if_changed.assert_called_once()
+        # Should not add include statement for .cpp files
+        mock_cg.add_global.assert_not_called()
+
+
+def test_get_usable_cpu_count() -> None:
+    """Test get_usable_cpu_count returns CPU count."""
+    count = config.get_usable_cpu_count()
+    assert isinstance(count, int)
+    assert count > 0
+
+
+def test_get_usable_cpu_count_with_process_cpu_count() -> None:
+    """Test get_usable_cpu_count uses process_cpu_count when available."""
+    # Test with process_cpu_count (Python 3.13+)
+    # Create a mock os module with process_cpu_count
+
+    mock_os = types.SimpleNamespace(process_cpu_count=lambda: 8, cpu_count=lambda: 4)
+
+    with patch("esphome.core.config.os", mock_os):
+        # When process_cpu_count exists, it should be used
+        count = config.get_usable_cpu_count()
+        assert count == 8
+
+    # Test fallback to cpu_count when process_cpu_count not available
+    mock_os_no_process = types.SimpleNamespace(cpu_count=lambda: 4)
+
+    with patch("esphome.core.config.os", mock_os_no_process):
+        count = config.get_usable_cpu_count()
+        assert count == 4
+
+
+def test_list_target_platforms(tmp_path: Path) -> None:
+    """Test _list_target_platforms returns available platforms."""
+    # Create mock components directory structure
+    components_dir = tmp_path / "components"
+    components_dir.mkdir()
+
+    # Create platform and non-platform directories with __init__.py
+    platforms = ["esp32", "esp8266", "rp2040", "libretiny", "host"]
+    non_platforms = ["sensor"]
+
+    for component in platforms + non_platforms:
+        component_dir = components_dir / component
+        component_dir.mkdir()
+        (component_dir / "__init__.py").touch()
+
+    # Create a file (not a directory)
+    (components_dir / "README.md").touch()
+
+    # Create a directory without __init__.py
+    (components_dir / "no_init").mkdir()
+
+    # Mock Path(__file__).parents[1] to return our tmp_path
+    with patch("esphome.core.config.Path") as mock_path:
+        mock_file_path = MagicMock()
+        mock_file_path.parents = [MagicMock(), tmp_path]
+        mock_path.return_value = mock_file_path
+
+        platforms = config._list_target_platforms()
+
+    assert isinstance(platforms, list)
+    # Should include platform components
+    assert "esp32" in platforms
+    assert "esp8266" in platforms
+    assert "rp2040" in platforms
+    assert "libretiny" in platforms
+    assert "host" in platforms
+    # Should not include non-platform components
+    assert "sensor" not in platforms
+    assert "README.md" not in platforms
+    assert "no_init" not in platforms
+
+
+def test_is_target_platform() -> None:
+    """Test _is_target_platform identifies valid platforms."""
+    assert config._is_target_platform("esp32") is True
+    assert config._is_target_platform("esp8266") is True
+    assert config._is_target_platform("rp2040") is True
+    assert config._is_target_platform("invalid_platform") is False
+    assert config._is_target_platform("api") is False  # Component but not platform
+
+
+@pytest.mark.asyncio
+async def test_add_includes_with_single_file(
+    tmp_path: Path,
+    mock_copy_file_if_changed: Mock,
+    mock_cg_with_include_capture: tuple[Mock, list[str]],
+) -> None:
+    """Test add_includes copies a single header file to build directory."""
+    CORE.config_path = tmp_path / "config.yaml"
+    CORE.build_path = tmp_path / "build"
+    os.makedirs(CORE.build_path, exist_ok=True)
+
+    # Create include file
+    include_file = tmp_path / "my_header.h"
+    include_file.write_text("#define MY_CONSTANT 42")
+
+    mock_cg, includes_added = mock_cg_with_include_capture
+
+    await config.add_includes([str(include_file)])
+
+    # Verify copy_file_if_changed was called to copy the file
+    # Note: add_includes adds files to a src/ subdirectory
+    mock_copy_file_if_changed.assert_called_once_with(
+        include_file, CORE.build_path / "src" / "my_header.h"
+    )
+
+    # Verify include statement was added
+    assert any('#include "my_header.h"' in inc for inc in includes_added)
+
+
+@pytest.mark.asyncio
+@pytest.mark.skipif(os.name == "nt", reason="Unix-specific test")
+async def test_add_includes_with_directory_unix(
+    tmp_path: Path,
+    mock_copy_file_if_changed: Mock,
+    mock_cg_with_include_capture: tuple[Mock, list[str]],
+) -> None:
+    """Test add_includes copies all files from a directory on Unix."""
+    CORE.config_path = tmp_path / "config.yaml"
+    CORE.build_path = tmp_path / "build"
+    os.makedirs(CORE.build_path, exist_ok=True)
+
+    # Create include directory with files
+    include_dir = tmp_path / "includes"
+    include_dir.mkdir()
+    (include_dir / "header1.h").write_text("#define HEADER1")
+    (include_dir / "header2.hpp").write_text("#define HEADER2")
+    (include_dir / "source.cpp").write_text("// Implementation")
+    (include_dir / "README.md").write_text(
+        "# Documentation"
+    )  # Should be copied but not included
+
+    # Create subdirectory with files
+    subdir = include_dir / "subdir"
+    subdir.mkdir()
+    (subdir / "nested.h").write_text("#define NESTED")
+
+    mock_cg, includes_added = mock_cg_with_include_capture
+
+    await config.add_includes([str(include_dir)])
+
+    # Verify copy_file_if_changed was called for all files
+    assert mock_copy_file_if_changed.call_count == 5  # 4 code files + 1 README
+
+    # Verify include statements were added for valid extensions
+    include_strings = " ".join(includes_added)
+    assert "includes/header1.h" in include_strings
+    assert "includes/header2.hpp" in include_strings
+    assert "includes/subdir/nested.h" in include_strings
+    # CPP files are copied but not included
+    assert "source.cpp" not in include_strings or "#include" not in include_strings
+    # README.md should not have an include statement
+    assert "README.md" not in include_strings
+
+
+@pytest.mark.asyncio
+@pytest.mark.skipif(os.name != "nt", reason="Windows-specific test")
+async def test_add_includes_with_directory_windows(
+    tmp_path: Path,
+    mock_copy_file_if_changed: Mock,
+    mock_cg_with_include_capture: tuple[Mock, list[str]],
+) -> None:
+    """Test add_includes copies all files from a directory on Windows."""
+    CORE.config_path = tmp_path / "config.yaml"
+    CORE.build_path = tmp_path / "build"
+    os.makedirs(CORE.build_path, exist_ok=True)
+
+    # Create include directory with files
+    include_dir = tmp_path / "includes"
+    include_dir.mkdir()
+    (include_dir / "header1.h").write_text("#define HEADER1")
+    (include_dir / "header2.hpp").write_text("#define HEADER2")
+    (include_dir / "source.cpp").write_text("// Implementation")
+    (include_dir / "README.md").write_text(
+        "# Documentation"
+    )  # Should be copied but not included
+
+    # Create subdirectory with files
+    subdir = include_dir / "subdir"
+    subdir.mkdir()
+    (subdir / "nested.h").write_text("#define NESTED")
+
+    mock_cg, includes_added = mock_cg_with_include_capture
+
+    await config.add_includes([str(include_dir)])
+
+    # Verify copy_file_if_changed was called for all files
+    assert mock_copy_file_if_changed.call_count == 5  # 4 code files + 1 README
+
+    # Verify include statements were added for valid extensions
+    include_strings = " ".join(includes_added)
+    assert "includes\\header1.h" in include_strings
+    assert "includes\\header2.hpp" in include_strings
+    assert "includes\\subdir\\nested.h" in include_strings
+    # CPP files are copied but not included
+    assert "source.cpp" not in include_strings or "#include" not in include_strings
+    # README.md should not have an include statement
+    assert "README.md" not in include_strings
+
+
+@pytest.mark.asyncio
+async def test_add_includes_with_multiple_sources(
+    tmp_path: Path, mock_copy_file_if_changed: Mock
+) -> None:
+    """Test add_includes with multiple files and directories."""
+    CORE.config_path = tmp_path / "config.yaml"
+    CORE.build_path = tmp_path / "build"
+    os.makedirs(CORE.build_path, exist_ok=True)
+
+    # Create various include sources
+    single_file = tmp_path / "single.h"
+    single_file.write_text("#define SINGLE")
+
+    dir1 = tmp_path / "dir1"
+    dir1.mkdir()
+    (dir1 / "file1.h").write_text("#define FILE1")
+
+    dir2 = tmp_path / "dir2"
+    dir2.mkdir()
+    (dir2 / "file2.cpp").write_text("// File2")
+
+    with patch("esphome.core.config.cg"):
+        await config.add_includes([str(single_file), str(dir1), str(dir2)])
+
+    # Verify copy_file_if_changed was called for all files
+    assert mock_copy_file_if_changed.call_count == 3  # 3 files total
+
+
+@pytest.mark.asyncio
+async def test_add_includes_empty_directory(
+    tmp_path: Path, mock_copy_file_if_changed: Mock
+) -> None:
+    """Test add_includes with an empty directory doesn't fail."""
+    CORE.config_path = tmp_path / "config.yaml"
+    CORE.build_path = tmp_path / "build"
+    os.makedirs(CORE.build_path, exist_ok=True)
+
+    # Create empty directory
+    empty_dir = tmp_path / "empty"
+    empty_dir.mkdir()
+
+    with patch("esphome.core.config.cg"):
+        # Should not raise any errors
+        await config.add_includes([str(empty_dir)])
+
+    # No files to copy from empty directory
+    mock_copy_file_if_changed.assert_not_called()
+
+
+@pytest.mark.asyncio
+@pytest.mark.skipif(os.name == "nt", reason="Unix-specific test")
+async def test_add_includes_preserves_directory_structure_unix(
+    tmp_path: Path, mock_copy_file_if_changed: Mock
+) -> None:
+    """Test that add_includes preserves relative directory structure on Unix."""
+    CORE.config_path = tmp_path / "config.yaml"
+    CORE.build_path = tmp_path / "build"
+    os.makedirs(CORE.build_path, exist_ok=True)
+
+    # Create nested directory structure
+    lib_dir = tmp_path / "lib"
+    lib_dir.mkdir()
+
+    src_dir = lib_dir / "src"
+    src_dir.mkdir()
+    (src_dir / "core.h").write_text("#define CORE")
+
+    utils_dir = lib_dir / "utils"
+    utils_dir.mkdir()
+    (utils_dir / "helper.h").write_text("#define HELPER")
+
+    with patch("esphome.core.config.cg"):
+        await config.add_includes([str(lib_dir)])
+
+    # Verify copy_file_if_changed was called with correct paths
+    calls = mock_copy_file_if_changed.call_args_list
+    dest_paths = [call[0][1] for call in calls]
+
+    # Check that relative paths are preserved
+    assert any("lib/src/core.h" in str(path) for path in dest_paths)
+    assert any("lib/utils/helper.h" in str(path) for path in dest_paths)
+
+
+@pytest.mark.asyncio
+@pytest.mark.skipif(os.name != "nt", reason="Windows-specific test")
+async def test_add_includes_preserves_directory_structure_windows(
+    tmp_path: Path, mock_copy_file_if_changed: Mock
+) -> None:
+    """Test that add_includes preserves relative directory structure on Windows."""
+    CORE.config_path = tmp_path / "config.yaml"
+    CORE.build_path = tmp_path / "build"
+    os.makedirs(CORE.build_path, exist_ok=True)
+
+    # Create nested directory structure
+    lib_dir = tmp_path / "lib"
+    lib_dir.mkdir()
+
+    src_dir = lib_dir / "src"
+    src_dir.mkdir()
+    (src_dir / "core.h").write_text("#define CORE")
+
+    utils_dir = lib_dir / "utils"
+    utils_dir.mkdir()
+    (utils_dir / "helper.h").write_text("#define HELPER")
+
+    with patch("esphome.core.config.cg"):
+        await config.add_includes([str(lib_dir)])
+
+    # Verify copy_file_if_changed was called with correct paths
+    calls = mock_copy_file_if_changed.call_args_list
+    dest_paths = [call[0][1] for call in calls]
+
+    # Check that relative paths are preserved
+    assert any("lib\\src\\core.h" in str(path) for path in dest_paths)
+    assert any("lib\\utils\\helper.h" in str(path) for path in dest_paths)
+
+
+@pytest.mark.asyncio
+async def test_add_includes_overwrites_existing_files(
+    tmp_path: Path, mock_copy_file_if_changed: Mock
+) -> None:
+    """Test that add_includes overwrites existing files in build directory."""
+    CORE.config_path = tmp_path / "config.yaml"
+    CORE.build_path = tmp_path / "build"
+    os.makedirs(CORE.build_path, exist_ok=True)
+
+    # Create include file
+    include_file = tmp_path / "header.h"
+    include_file.write_text("#define NEW_VALUE 42")
+
+    with patch("esphome.core.config.cg"):
+        await config.add_includes([str(include_file)])
+
+    # Verify copy_file_if_changed was called (it handles overwriting)
+    # Note: add_includes adds files to a src/ subdirectory
+    mock_copy_file_if_changed.assert_called_once_with(
+        include_file, CORE.build_path / "src" / "header.h"
+    )
diff --git a/tests/unit_tests/core/test_entity_helpers.py b/tests/unit_tests/core/test_entity_helpers.py
index 4f256ffb33..9ba5367413 100644
--- a/tests/unit_tests/core/test_entity_helpers.py
+++ b/tests/unit_tests/core/test_entity_helpers.py
@@ -12,6 +12,7 @@ from esphome.const import (
     CONF_DEVICE_ID,
     CONF_DISABLED_BY_DEFAULT,
     CONF_ICON,
+    CONF_ID,
     CONF_INTERNAL,
     CONF_NAME,
 )
@@ -510,13 +511,19 @@ def test_entity_duplicate_validator() -> None:
     config1 = {CONF_NAME: "Temperature"}
     validated1 = validator(config1)
     assert validated1 == config1
-    assert ("sensor", "temperature") in CORE.unique_ids
+    assert ("", "sensor", "temperature") in CORE.unique_ids
+    # Check metadata was stored
+    metadata = CORE.unique_ids[("", "sensor", "temperature")]
+    assert metadata["name"] == "Temperature"
+    assert metadata["platform"] == "sensor"
 
     # Second entity with different name should pass
     config2 = {CONF_NAME: "Humidity"}
     validated2 = validator(config2)
     assert validated2 == config2
-    assert ("sensor", "humidity") in CORE.unique_ids
+    assert ("", "sensor", "humidity") in CORE.unique_ids
+    metadata2 = CORE.unique_ids[("", "sensor", "humidity")]
+    assert metadata2["name"] == "Humidity"
 
     # Duplicate entity should fail
     config3 = {CONF_NAME: "Temperature"}
@@ -535,36 +542,28 @@ def test_entity_duplicate_validator_with_devices() -> None:
     device1 = ID("device1", type="Device")
     device2 = ID("device2", type="Device")
 
-    # First entity on device1 should pass
+    # Same name on different devices should pass
     config1 = {CONF_NAME: "Temperature", CONF_DEVICE_ID: device1}
     validated1 = validator(config1)
     assert validated1 == config1
-    assert ("sensor", "temperature") in CORE.unique_ids
+    assert ("device1", "sensor", "temperature") in CORE.unique_ids
+    metadata1 = CORE.unique_ids[("device1", "sensor", "temperature")]
+    assert metadata1["device_id"] == "device1"
 
-    # Same name on different device should now fail
     config2 = {CONF_NAME: "Temperature", CONF_DEVICE_ID: device2}
+    validated2 = validator(config2)
+    assert validated2 == config2
+    assert ("device2", "sensor", "temperature") in CORE.unique_ids
+    metadata2 = CORE.unique_ids[("device2", "sensor", "temperature")]
+    assert metadata2["device_id"] == "device2"
+
+    # Duplicate on same device should fail
+    config3 = {CONF_NAME: "Temperature", CONF_DEVICE_ID: device1}
     with pytest.raises(
         Invalid,
-        match=r"Duplicate sensor entity with name 'Temperature' found. Each entity must have a unique name within its platform across all devices.",
+        match=r"Duplicate sensor entity with name 'Temperature' found on device 'device1'",
     ):
-        validator(config2)
-
-    # Different name on device2 should pass
-    config3 = {CONF_NAME: "Humidity", CONF_DEVICE_ID: device2}
-    validated3 = validator(config3)
-    assert validated3 == config3
-    assert ("sensor", "humidity") in CORE.unique_ids
-
-    # Empty names should use device names and be allowed
-    config4 = {CONF_NAME: "", CONF_DEVICE_ID: device1}
-    validated4 = validator(config4)
-    assert validated4 == config4
-    assert ("sensor", "device1") in CORE.unique_ids
-
-    config5 = {CONF_NAME: "", CONF_DEVICE_ID: device2}
-    validated5 = validator(config5)
-    assert validated5 == config5
-    assert ("sensor", "device2") in CORE.unique_ids
+        validator(config3)
 
 
 def test_duplicate_entity_yaml_validation(
@@ -588,10 +587,10 @@ def test_duplicate_entity_with_devices_yaml_validation(
     )
     assert result is None
 
-    # Check for the duplicate entity error message
+    # Check for the duplicate entity error message with device
     captured = capsys.readouterr()
     assert (
-        "Duplicate sensor entity with name 'Temperature' found. Each entity must have a unique name within its platform across all devices."
+        "Duplicate sensor entity with name 'Temperature' found on device 'device1'"
         in captured.out
     )
 
@@ -607,6 +606,54 @@ def test_entity_different_platforms_yaml_validation(
     assert result is not None
 
 
+def test_entity_duplicate_validator_error_message() -> None:
+    """Test that duplicate entity error messages include helpful metadata."""
+    # Create validator for sensor platform
+    validator = entity_duplicate_validator("sensor")
+
+    # Set current component to simulate validation context for uptime sensor
+    CORE.current_component = "sensor.uptime"
+
+    # First entity should pass
+    config1 = {CONF_NAME: "Battery", CONF_ID: ID("battery_1")}
+    validated1 = validator(config1)
+    assert validated1 == config1
+
+    # Reset component to simulate template sensor
+    CORE.current_component = "sensor.template"
+
+    # Duplicate entity should fail with detailed error
+    config2 = {CONF_NAME: "Battery", CONF_ID: ID("battery_2")}
+    with pytest.raises(
+        Invalid,
+        match=r"Duplicate sensor entity with name 'Battery' found.*"
+        r"Conflicts with entity 'Battery' \(id: battery_1\) from component 'sensor\.uptime'",
+    ):
+        validator(config2)
+
+    # Clean up
+    CORE.current_component = None
+
+
+def test_entity_conflict_between_components_yaml(
+    yaml_file: Callable[[str], str], capsys: pytest.CaptureFixture[str]
+) -> None:
+    """Test that conflicts between different components show helpful error messages."""
+    result = load_config_from_fixture(
+        yaml_file, "entity_conflict_components.yaml", FIXTURES_DIR
+    )
+    assert result is None
+
+    # Check for the enhanced error message
+    captured = capsys.readouterr()
+    # The error should mention both the conflict and which component created it
+    assert "Duplicate sensor entity with name 'Battery' found" in captured.out
+    # Should mention it conflicts with an entity from a specific sensor platform
+    assert "from component 'sensor." in captured.out
+    # Should show it's a conflict between wifi_signal and template
+    assert "sensor.wifi_signal" in captured.out or "sensor.template" in captured.out
+
+
 def test_entity_duplicate_validator_internal_entities() -> None:
     """Test that internal entities are excluded from duplicate name validation."""
     # Create validator for sensor platform
@@ -616,21 +663,25 @@ def test_entity_duplicate_validator_internal_entities() -> None:
     config1 = {CONF_NAME: "Temperature"}
     validated1 = validator(config1)
     assert validated1 == config1
-    assert ("sensor", "temperature") in CORE.unique_ids
+    # New format includes device_id (empty string for main device)
+    assert ("", "sensor", "temperature") in CORE.unique_ids
 
     # Internal entity with same name should pass (not added to unique_ids)
     config2 = {CONF_NAME: "Temperature", CONF_INTERNAL: True}
     validated2 = validator(config2)
     assert validated2 == config2
     # Internal entity should not be added to unique_ids
-    assert len([k for k in CORE.unique_ids if k == ("sensor", "temperature")]) == 1
+    # Count how many times the key appears (should still be 1)
+    count = sum(1 for k in CORE.unique_ids if k == ("", "sensor", "temperature"))
+    assert count == 1
 
     # Another internal entity with same name should also pass
     config3 = {CONF_NAME: "Temperature", CONF_INTERNAL: True}
     validated3 = validator(config3)
     assert validated3 == config3
     # Still only one entry in unique_ids (from the non-internal entity)
-    assert len([k for k in CORE.unique_ids if k == ("sensor", "temperature")]) == 1
+    count = sum(1 for k in CORE.unique_ids if k == ("", "sensor", "temperature"))
+    assert count == 1
 
     # Non-internal entity with same name should fail
     config4 = {CONF_NAME: "Temperature"}
@@ -638,3 +689,64 @@ def test_entity_duplicate_validator_internal_entities() -> None:
         Invalid, match=r"Duplicate sensor entity with name 'Temperature' found"
     ):
         validator(config4)
+
+
+def test_empty_or_null_device_id_on_entity() -> None:
+    """Test that empty or null device IDs are handled correctly."""
+    # Create validator for sensor platform
+    validator = entity_duplicate_validator("sensor")
+
+    # Entity with empty device_id should pass
+    config1 = {CONF_NAME: "Battery", CONF_DEVICE_ID: ""}
+    validated1 = validator(config1)
+    assert validated1 == config1
+
+    # Entity with None device_id should pass
+    config2 = {CONF_NAME: "Temperature", CONF_DEVICE_ID: None}
+    validated2 = validator(config2)
+    assert validated2 == config2
+
+
+def test_entity_duplicate_validator_non_ascii_names() -> None:
+    """Test that non-ASCII names show helpful error messages."""
+    # Create validator for binary_sensor platform
+    validator = entity_duplicate_validator("binary_sensor")
+
+    # First Russian sensor should pass
+    config1 = {CONF_NAME: "Датчик открытия основного крана"}
+    validated1 = validator(config1)
+    assert validated1 == config1
+
+    # Second Russian sensor with different text but same ASCII conversion should fail
+    config2 = {CONF_NAME: "Датчик закрытия основного крана"}
+    with pytest.raises(
+        Invalid,
+        match=re.compile(
+            r"Duplicate binary_sensor entity with name 'Датчик закрытия основного крана' found.*"
+            r"Original names: 'Датчик закрытия основного крана' and 'Датчик открытия основного крана'.*"
+            r"Both convert to ASCII ID: '_______________________________'.*"
+            r"To fix: Add unique ASCII characters \(e\.g\., '1', '2', or 'A', 'B'\)",
+            re.DOTALL,
+        ),
+    ):
+        validator(config2)
+
+
+def test_entity_duplicate_validator_same_name_no_enhanced_message() -> None:
+    """Test that identical names don't show the enhanced message."""
+    # Create validator for sensor platform
+    validator = entity_duplicate_validator("sensor")
+
+    # First entity should pass
+    config1 = {CONF_NAME: "Temperature"}
+    validated1 = validator(config1)
+    assert validated1 == config1
+
+    # Second entity with exact same name should fail without enhanced message
+    config2 = {CONF_NAME: "Temperature"}
+    with pytest.raises(
+        Invalid,
+        match=r"Duplicate sensor entity with name 'Temperature' found.*"
+        r"Each entity on a device must have a unique name within its platform\.$",
+    ):
+        validator(config2)
diff --git a/tests/unit_tests/fixtures/auto_load_dynamic.yaml b/tests/unit_tests/fixtures/auto_load_dynamic.yaml
new file mode 100644
index 0000000000..b604a2a42b
--- /dev/null
+++ b/tests/unit_tests/fixtures/auto_load_dynamic.yaml
@@ -0,0 +1,10 @@
+esphome:
+  name: test-device
+
+esp32:
+  board: esp32dev
+
+# Test component with dynamic AUTO_LOAD
+test_component:
+  enable_logger: true
+  enable_api: false
diff --git a/tests/unit_tests/fixtures/auto_load_static.yaml b/tests/unit_tests/fixtures/auto_load_static.yaml
new file mode 100644
index 0000000000..c8f9e6222a
--- /dev/null
+++ b/tests/unit_tests/fixtures/auto_load_static.yaml
@@ -0,0 +1,8 @@
+esphome:
+  name: test-device
+
+esp32:
+  board: esp32dev
+
+# Test component with static AUTO_LOAD
+test_component:
diff --git a/tests/unit_tests/fixtures/core/entity_helpers/entity_conflict_components.yaml b/tests/unit_tests/fixtures/core/entity_helpers/entity_conflict_components.yaml
new file mode 100644
index 0000000000..6a1df0f7b4
--- /dev/null
+++ b/tests/unit_tests/fixtures/core/entity_helpers/entity_conflict_components.yaml
@@ -0,0 +1,20 @@
+esphome:
+  name: test-device
+
+esp32:
+  board: esp32dev
+
+# Uptime sensor
+sensor:
+  - platform: uptime
+    name: "Battery"
+    id: uptime_battery
+
+# Template sensor also named "Battery" - this should conflict
+  - platform: template
+    name: "Battery"
+    id: template_battery
+    lambda: |-
+      return 95.0;
+    unit_of_measurement: "%"
+    update_interval: 60s
diff --git a/tests/unit_tests/fixtures/ota_empty_dict.yaml b/tests/unit_tests/fixtures/ota_empty_dict.yaml
new file mode 100644
index 0000000000..cf9b166afa
--- /dev/null
+++ b/tests/unit_tests/fixtures/ota_empty_dict.yaml
@@ -0,0 +1,17 @@
+esphome:
+  name: test-device2
+
+esp32:
+  board: esp32dev
+  framework:
+    type: esp-idf
+
+# OTA with empty dict - should be normalized
+ota: {}
+
+wifi:
+  ssid: "test"
+  password: "test"
+
+# Captive portal auto-loads ota.web_server which triggers the issue
+captive_portal:
diff --git a/tests/unit_tests/fixtures/ota_no_platform.yaml b/tests/unit_tests/fixtures/ota_no_platform.yaml
new file mode 100644
index 0000000000..0b09c836fb
--- /dev/null
+++ b/tests/unit_tests/fixtures/ota_no_platform.yaml
@@ -0,0 +1,17 @@
+esphome:
+  name: test-device
+
+esp32:
+  board: esp32dev
+  framework:
+    type: esp-idf
+
+# OTA with no value - this should be normalized to empty list
+ota:
+
+wifi:
+  ssid: "test"
+  password: "test"
+
+# Captive portal auto-loads ota.web_server which triggers the issue
+captive_portal:
diff --git a/tests/unit_tests/fixtures/ota_with_platform_list.yaml b/tests/unit_tests/fixtures/ota_with_platform_list.yaml
new file mode 100644
index 0000000000..b1b03743ae
--- /dev/null
+++ b/tests/unit_tests/fixtures/ota_with_platform_list.yaml
@@ -0,0 +1,19 @@
+esphome:
+  name: test-device3
+
+esp32:
+  board: esp32dev
+  framework:
+    type: esp-idf
+
+# OTA with proper list format
+ota:
+  - platform: esphome
+    password: "test123"
+
+wifi:
+  ssid: "test"
+  password: "test"
+
+# Captive portal auto-loads ota.web_server
+captive_portal:
diff --git a/tests/unit_tests/fixtures/substitutions/00-simple_var.approved.yaml b/tests/unit_tests/fixtures/substitutions/00-simple_var.approved.yaml
index f5d2f8aa20..795a788f62 100644
--- a/tests/unit_tests/fixtures/substitutions/00-simple_var.approved.yaml
+++ b/tests/unit_tests/fixtures/substitutions/00-simple_var.approved.yaml
@@ -1,7 +1,14 @@
 substitutions:
+  substituted: 99
   var1: '1'
   var2: '2'
   var21: '79'
+  value: 33
+  values: 44
+  position:
+    x: 79
+    y: 82
+
 esphome:
   name: test
 test_list:
@@ -19,3 +26,10 @@ test_list:
   - ${ undefined_var }
   - key1: 1
     key2: 2
+  - Literal $values ${are not substituted}
+  - ["list $value", "${is not}", "${substituted}"]
+  - {"$dictionary": "$value", "${is not}": "${substituted}"}
+  - |-
+    {{{ "x", "79"}, { "y", "82"}}}
+  - '{{{"AA"}}}'
+  - '"HELLO"'
diff --git a/tests/unit_tests/fixtures/substitutions/00-simple_var.input.yaml b/tests/unit_tests/fixtures/substitutions/00-simple_var.input.yaml
index 5717433c7e..722e116d36 100644
--- a/tests/unit_tests/fixtures/substitutions/00-simple_var.input.yaml
+++ b/tests/unit_tests/fixtures/substitutions/00-simple_var.input.yaml
@@ -2,9 +2,15 @@ esphome:
   name: test
 
 substitutions:
+  substituted: 99
   var1: "1"
   var2: "2"
   var21: "79"
+  value: 33
+  values: 44
+  position:
+    x: 79
+    y: 82
 
 test_list:
   - "$var1"
@@ -21,3 +27,10 @@ test_list:
   - ${ undefined_var }
   - key${var1}: 1
     key${var2}: 2
+  - !literal Literal $values ${are not substituted}
+  - !literal ["list $value", "${is not}", "${substituted}"]
+  - !literal {"$dictionary": "$value", "${is not}": "${substituted}"}
+  - |- # Test parsing things that look like a python set of sets when rendered:
+    {{{ "x", "${ position.x }"}, { "y", "${ position.y }"}}}
+  - ${ '{{{"AA"}}}' }
+  - ${ '"HELLO"' }
diff --git a/tests/unit_tests/fixtures/substitutions/02-expressions.approved.yaml b/tests/unit_tests/fixtures/substitutions/02-expressions.approved.yaml
index 9e401ec5d6..443cba144e 100644
--- a/tests/unit_tests/fixtures/substitutions/02-expressions.approved.yaml
+++ b/tests/unit_tests/fixtures/substitutions/02-expressions.approved.yaml
@@ -22,3 +22,6 @@ test_list:
   - The pin number is 18
   - The square root is: 5.0
   - The number is 80
+  - ord("a") = 97
+  - chr(97) = a
+  - len([1,2,3]) = 3
diff --git a/tests/unit_tests/fixtures/substitutions/02-expressions.input.yaml b/tests/unit_tests/fixtures/substitutions/02-expressions.input.yaml
index 1777b46f67..07ad992f1f 100644
--- a/tests/unit_tests/fixtures/substitutions/02-expressions.input.yaml
+++ b/tests/unit_tests/fixtures/substitutions/02-expressions.input.yaml
@@ -20,3 +20,6 @@ test_list:
   - The pin number is ${pin.number}
   - The square root is: ${math.sqrt(area)}
   - The number is ${var${numberOne} + 1}
+  - ord("a") = ${ ord("a") }
+  - chr(97) = ${ chr(97) }
+  - len([1,2,3]) = ${ len([1,2,3]) }
diff --git a/tests/unit_tests/fixtures/yaml_util/named_dir/.hidden.yaml b/tests/unit_tests/fixtures/yaml_util/named_dir/.hidden.yaml
new file mode 100644
index 0000000000..75eb989ea5
--- /dev/null
+++ b/tests/unit_tests/fixtures/yaml_util/named_dir/.hidden.yaml
@@ -0,0 +1,3 @@
+# This file should be ignored
+platform: template
+name: "Hidden Sensor"
diff --git a/tests/unit_tests/fixtures/yaml_util/named_dir/not_yaml.txt b/tests/unit_tests/fixtures/yaml_util/named_dir/not_yaml.txt
new file mode 100644
index 0000000000..98efb74b0f
--- /dev/null
+++ b/tests/unit_tests/fixtures/yaml_util/named_dir/not_yaml.txt
@@ -0,0 +1 @@
+This is not a YAML file and should be ignored
diff --git a/tests/unit_tests/fixtures/yaml_util/named_dir/sensor1.yaml b/tests/unit_tests/fixtures/yaml_util/named_dir/sensor1.yaml
new file mode 100644
index 0000000000..a4b0a11916
--- /dev/null
+++ b/tests/unit_tests/fixtures/yaml_util/named_dir/sensor1.yaml
@@ -0,0 +1,4 @@
+platform: template
+name: "Sensor 1"
+lambda: |-
+  return 42.0;
diff --git a/tests/unit_tests/fixtures/yaml_util/named_dir/sensor2.yaml b/tests/unit_tests/fixtures/yaml_util/named_dir/sensor2.yaml
new file mode 100644
index 0000000000..72d4b714b6
--- /dev/null
+++ b/tests/unit_tests/fixtures/yaml_util/named_dir/sensor2.yaml
@@ -0,0 +1,4 @@
+platform: template
+name: "Sensor 2"
+lambda: |-
+  return 100.0;
diff --git a/tests/unit_tests/fixtures/yaml_util/named_dir/subdir/sensor3.yaml b/tests/unit_tests/fixtures/yaml_util/named_dir/subdir/sensor3.yaml
new file mode 100644
index 0000000000..bcb8dd320d
--- /dev/null
+++ b/tests/unit_tests/fixtures/yaml_util/named_dir/subdir/sensor3.yaml
@@ -0,0 +1,4 @@
+platform: template
+name: "Sensor 3 in subdir"
+lambda: |-
+  return 200.0;
diff --git a/tests/unit_tests/fixtures/yaml_util/secrets.yaml b/tests/unit_tests/fixtures/yaml_util/secrets.yaml
new file mode 100644
index 0000000000..4eef570926
--- /dev/null
+++ b/tests/unit_tests/fixtures/yaml_util/secrets.yaml
@@ -0,0 +1,4 @@
+test_secret: "my_secret_value"
+another_secret: "another_value"
+wifi_password: "super_secret_wifi"
+api_key: "0123456789abcdef"
diff --git a/tests/unit_tests/fixtures/yaml_util/test_secret.yaml b/tests/unit_tests/fixtures/yaml_util/test_secret.yaml
new file mode 100644
index 0000000000..c23afaee94
--- /dev/null
+++ b/tests/unit_tests/fixtures/yaml_util/test_secret.yaml
@@ -0,0 +1,17 @@
+esphome:
+  name: test_device
+  platform: ESP32
+  board: esp32dev
+
+wifi:
+  ssid: "TestNetwork"
+  password: !secret wifi_password
+
+api:
+  encryption:
+    key: !secret api_key
+
+sensor:
+  - platform: template
+    name: "Test Sensor"
+    id: !secret test_secret
diff --git a/tests/unit_tests/test_address_cache.py b/tests/unit_tests/test_address_cache.py
new file mode 100644
index 0000000000..de43830d53
--- /dev/null
+++ b/tests/unit_tests/test_address_cache.py
@@ -0,0 +1,305 @@
+"""Tests for the address_cache module."""
+
+from __future__ import annotations
+
+import logging
+
+import pytest
+from pytest import LogCaptureFixture
+
+from esphome.address_cache import AddressCache, normalize_hostname
+
+
+def test_normalize_simple_hostname() -> None:
+    """Test normalizing a simple hostname."""
+    assert normalize_hostname("device") == "device"
+    assert normalize_hostname("device.local") == "device.local"
+    assert normalize_hostname("server.example.com") == "server.example.com"
+
+
+def test_normalize_removes_trailing_dots() -> None:
+    """Test that trailing dots are removed."""
+    assert normalize_hostname("device.") == "device"
+    assert normalize_hostname("device.local.") == "device.local"
+    assert normalize_hostname("server.example.com.") == "server.example.com"
+    assert normalize_hostname("device...") == "device"
+
+
+def test_normalize_converts_to_lowercase() -> None:
+    """Test that hostnames are converted to lowercase."""
+    assert normalize_hostname("DEVICE") == "device"
+    assert normalize_hostname("Device.Local") == "device.local"
+    assert normalize_hostname("Server.Example.COM") == "server.example.com"
+
+
+def test_normalize_combined() -> None:
+    """Test combination of trailing dots and case conversion."""
+    assert normalize_hostname("DEVICE.LOCAL.") == "device.local"
+    assert normalize_hostname("Server.Example.COM...") == "server.example.com"
+
+
+def test_init_empty() -> None:
+    """Test initialization with empty caches."""
+    cache = AddressCache()
+    assert cache.mdns_cache == {}
+    assert cache.dns_cache == {}
+    assert not cache.has_cache()
+
+
+def test_init_with_caches() -> None:
+    """Test initialization with provided caches."""
+    mdns_cache: dict[str, list[str]] = {"device.local": ["192.168.1.10"]}
+    dns_cache: dict[str, list[str]] = {"server.com": ["10.0.0.1"]}
+    cache = AddressCache(mdns_cache=mdns_cache, dns_cache=dns_cache)
+    assert cache.mdns_cache == mdns_cache
+    assert cache.dns_cache == dns_cache
+    assert cache.has_cache()
+
+
+def test_get_mdns_addresses() -> None:
+    """Test getting mDNS addresses."""
+    cache = AddressCache(mdns_cache={"device.local": ["192.168.1.10", "192.168.1.11"]})
+
+    # Direct lookup
+    assert cache.get_mdns_addresses("device.local") == [
+        "192.168.1.10",
+        "192.168.1.11",
+    ]
+
+    # Case insensitive lookup
+    assert cache.get_mdns_addresses("Device.Local") == [
+        "192.168.1.10",
+        "192.168.1.11",
+    ]
+
+    # With trailing dot
+    assert cache.get_mdns_addresses("device.local.") == [
+        "192.168.1.10",
+        "192.168.1.11",
+    ]
+
+    # Not found
+    assert cache.get_mdns_addresses("unknown.local") is None
+
+
+def test_get_dns_addresses() -> None:
+    """Test getting DNS addresses."""
+    cache = AddressCache(dns_cache={"server.com": ["10.0.0.1", "10.0.0.2"]})
+
+    # Direct lookup
+    assert cache.get_dns_addresses("server.com") == ["10.0.0.1", "10.0.0.2"]
+
+    # Case insensitive lookup
+    assert cache.get_dns_addresses("Server.COM") == ["10.0.0.1", "10.0.0.2"]
+
+    # With trailing dot
+    assert cache.get_dns_addresses("server.com.") == ["10.0.0.1", "10.0.0.2"]
+
+    # Not found
+    assert cache.get_dns_addresses("unknown.com") is None
+
+
+def test_get_addresses_auto_detection() -> None:
+    """Test automatic cache selection based on hostname."""
+    cache = AddressCache(
+        mdns_cache={"device.local": ["192.168.1.10"]},
+        dns_cache={"server.com": ["10.0.0.1"]},
+    )
+
+    # Should use mDNS cache for .local domains
+    assert cache.get_addresses("device.local") == ["192.168.1.10"]
+    assert cache.get_addresses("device.local.") == ["192.168.1.10"]
+    assert cache.get_addresses("Device.Local") == ["192.168.1.10"]
+
+    # Should use DNS cache for non-.local domains
+    assert cache.get_addresses("server.com") == ["10.0.0.1"]
+    assert cache.get_addresses("server.com.") == ["10.0.0.1"]
+    assert cache.get_addresses("Server.COM") == ["10.0.0.1"]
+
+    # Not found
+    assert cache.get_addresses("unknown.local") is None
+    assert cache.get_addresses("unknown.com") is None
+
+
+def test_has_cache() -> None:
+    """Test checking if cache has entries."""
+    # Empty cache
+    cache = AddressCache()
+    assert not cache.has_cache()
+
+    # Only mDNS cache
+    cache = AddressCache(mdns_cache={"device.local": ["192.168.1.10"]})
+    assert cache.has_cache()
+
+    # Only DNS cache
+    cache = AddressCache(dns_cache={"server.com": ["10.0.0.1"]})
+    assert cache.has_cache()
+
+    # Both caches
+    cache = AddressCache(
+        mdns_cache={"device.local": ["192.168.1.10"]},
+        dns_cache={"server.com": ["10.0.0.1"]},
+    )
+    assert cache.has_cache()
+
+
+def test_from_cli_args_empty() -> None:
+    """Test creating cache from empty CLI arguments."""
+    cache = AddressCache.from_cli_args([], [])
+    assert cache.mdns_cache == {}
+    assert cache.dns_cache == {}
+
+
+def test_from_cli_args_single_entry() -> None:
+    """Test creating cache from single CLI argument."""
+    mdns_args: list[str] = ["device.local=192.168.1.10"]
+    dns_args: list[str] = ["server.com=10.0.0.1"]
+
+    cache = AddressCache.from_cli_args(mdns_args, dns_args)
+
+    assert cache.mdns_cache == {"device.local": ["192.168.1.10"]}
+    assert cache.dns_cache == {"server.com": ["10.0.0.1"]}
+
+
+def test_from_cli_args_multiple_ips() -> None:
+    """Test creating cache with multiple IPs per host."""
+    mdns_args: list[str] = ["device.local=192.168.1.10,192.168.1.11"]
+    dns_args: list[str] = ["server.com=10.0.0.1,10.0.0.2,10.0.0.3"]
+
+    cache = AddressCache.from_cli_args(mdns_args, dns_args)
+
+    assert cache.mdns_cache == {"device.local": ["192.168.1.10", "192.168.1.11"]}
+    assert cache.dns_cache == {"server.com": ["10.0.0.1", "10.0.0.2", "10.0.0.3"]}
+
+
+def test_from_cli_args_multiple_entries() -> None:
+    """Test creating cache with multiple host entries."""
+    mdns_args: list[str] = [
+        "device1.local=192.168.1.10",
+        "device2.local=192.168.1.20,192.168.1.21",
+    ]
+    dns_args: list[str] = ["server1.com=10.0.0.1", "server2.com=10.0.0.2"]
+
+    cache = AddressCache.from_cli_args(mdns_args, dns_args)
+
+    assert cache.mdns_cache == {
+        "device1.local": ["192.168.1.10"],
+        "device2.local": ["192.168.1.20", "192.168.1.21"],
+    }
+    assert cache.dns_cache == {
+        "server1.com": ["10.0.0.1"],
+        "server2.com": ["10.0.0.2"],
+    }
+
+
+def test_from_cli_args_normalization() -> None:
+    """Test that CLI arguments are normalized."""
+    mdns_args: list[str] = ["Device1.Local.=192.168.1.10", "DEVICE2.LOCAL=192.168.1.20"]
+    dns_args: list[str] = ["Server1.COM.=10.0.0.1", "SERVER2.com=10.0.0.2"]
+
+    cache = AddressCache.from_cli_args(mdns_args, dns_args)
+
+    # Hostnames should be normalized (lowercase, no trailing dots)
+    assert cache.mdns_cache == {
+        "device1.local": ["192.168.1.10"],
+        "device2.local": ["192.168.1.20"],
+    }
+    assert cache.dns_cache == {
+        "server1.com": ["10.0.0.1"],
+        "server2.com": ["10.0.0.2"],
+    }
+
+
+def test_from_cli_args_whitespace_handling() -> None:
+    """Test that whitespace in IPs is handled."""
+    mdns_args: list[str] = ["device.local= 192.168.1.10 , 192.168.1.11 "]
+    dns_args: list[str] = ["server.com= 10.0.0.1 , 10.0.0.2 "]
+
+    cache = AddressCache.from_cli_args(mdns_args, dns_args)
+
+    assert cache.mdns_cache == {"device.local": ["192.168.1.10", "192.168.1.11"]}
+    assert cache.dns_cache == {"server.com": ["10.0.0.1", "10.0.0.2"]}
+
+
+def test_from_cli_args_invalid_format(caplog: LogCaptureFixture) -> None:
+    """Test handling of invalid argument format."""
+    mdns_args: list[str] = ["invalid_format", "device.local=192.168.1.10"]
+    dns_args: list[str] = ["server.com=10.0.0.1", "also_invalid"]
+
+    cache = AddressCache.from_cli_args(mdns_args, dns_args)
+
+    # Valid entries should still be processed
+    assert cache.mdns_cache == {"device.local": ["192.168.1.10"]}
+    assert cache.dns_cache == {"server.com": ["10.0.0.1"]}
+
+    # Check that warnings were logged for invalid entries
+    assert "Invalid cache format: invalid_format" in caplog.text
+    assert "Invalid cache format: also_invalid" in caplog.text
+
+
+def test_from_cli_args_ipv6() -> None:
+    """Test handling of IPv6 addresses."""
+    mdns_args: list[str] = ["device.local=fe80::1,2001:db8::1"]
+    dns_args: list[str] = ["server.com=2001:db8::2,::1"]
+
+    cache = AddressCache.from_cli_args(mdns_args, dns_args)
+
+    assert cache.mdns_cache == {"device.local": ["fe80::1", "2001:db8::1"]}
+    assert cache.dns_cache == {"server.com": ["2001:db8::2", "::1"]}
+
+
+def test_logging_output(caplog: LogCaptureFixture) -> None:
+    """Test that appropriate debug logging occurs."""
+    caplog.set_level(logging.DEBUG)
+
+    cache = AddressCache(
+        mdns_cache={"device.local": ["192.168.1.10"]},
+        dns_cache={"server.com": ["10.0.0.1"]},
+    )
+
+    # Test successful lookups log at debug level
+    result: list[str] | None = cache.get_mdns_addresses("device.local")
+    assert result == ["192.168.1.10"]
+    assert "Using mDNS cache for device.local" in caplog.text
+
+    caplog.clear()
+    result = cache.get_dns_addresses("server.com")
+    assert result == ["10.0.0.1"]
+    assert "Using DNS cache for server.com" in caplog.text
+
+    # Test that failed lookups don't log
+    caplog.clear()
+    result = cache.get_mdns_addresses("unknown.local")
+    assert result is None
+    assert "Using mDNS cache" not in caplog.text
+
+
+@pytest.mark.parametrize(
+    "hostname,expected",
+    [
+        ("test.local", "test.local"),
+        ("Test.Local.", "test.local"),
+        ("TEST.LOCAL...", "test.local"),
+        ("example.com", "example.com"),
+        ("EXAMPLE.COM.", "example.com"),
+    ],
+)
+def test_normalize_hostname_parametrized(hostname: str, expected: str) -> None:
+    """Test hostname normalization with various inputs."""
+    assert normalize_hostname(hostname) == expected
+
+
+@pytest.mark.parametrize(
+    "mdns_arg,expected",
+    [
+        ("host=1.2.3.4", {"host": ["1.2.3.4"]}),
+        ("Host.Local=1.2.3.4,5.6.7.8", {"host.local": ["1.2.3.4", "5.6.7.8"]}),
+        ("HOST.LOCAL.=::1", {"host.local": ["::1"]}),
+    ],
+)
+def test_parse_cache_args_parametrized(
+    mdns_arg: str, expected: dict[str, list[str]]
+) -> None:
+    """Test parsing of cache arguments with various formats."""
+    cache = AddressCache.from_cli_args([mdns_arg], [])
+    assert cache.mdns_cache == expected
diff --git a/tests/unit_tests/test_config_auto_load.py b/tests/unit_tests/test_config_auto_load.py
new file mode 100644
index 0000000000..d31b17eeec
--- /dev/null
+++ b/tests/unit_tests/test_config_auto_load.py
@@ -0,0 +1,131 @@
+"""Tests for AUTO_LOAD functionality including dynamic AUTO_LOAD."""
+
+from pathlib import Path
+from typing import Any
+from unittest.mock import Mock
+
+import pytest
+
+from esphome import config, config_validation as cv, yaml_util
+from esphome.core import CORE
+
+
+@pytest.fixture
+def fixtures_dir() -> Path:
+    """Get the fixtures directory."""
+    return Path(__file__).parent / "fixtures"
+
+
+@pytest.fixture
+def default_component() -> Mock:
+    """Create a default mock component for unmocked components."""
+    return Mock(
+        auto_load=[],
+        is_platform_component=False,
+        is_platform=False,
+        multi_conf=False,
+        multi_conf_no_default=False,
+        dependencies=[],
+        conflicts_with=[],
+        config_schema=cv.Schema({}, extra=cv.ALLOW_EXTRA),
+    )
+
+
+@pytest.fixture
+def static_auto_load_component() -> Mock:
+    """Create a mock component with static AUTO_LOAD."""
+    return Mock(
+        auto_load=["logger"],
+        is_platform_component=False,
+        is_platform=False,
+        multi_conf=False,
+        multi_conf_no_default=False,
+        dependencies=[],
+        conflicts_with=[],
+        config_schema=cv.Schema({}, extra=cv.ALLOW_EXTRA),
+    )
+
+
+def test_static_auto_load_adds_components(
+    mock_get_component: Mock,
+    fixtures_dir: Path,
+    static_auto_load_component: Mock,
+    default_component: Mock,
+) -> None:
+    """Test that static AUTO_LOAD triggers loading of specified components."""
+    CORE.config_path = fixtures_dir / "auto_load_static.yaml"
+
+    config_file = fixtures_dir / "auto_load_static.yaml"
+    raw_config = yaml_util.load_yaml(config_file)
+
+    component_mocks = {"test_component": static_auto_load_component}
+    mock_get_component.side_effect = lambda name: component_mocks.get(
+        name, default_component
+    )
+
+    result = config.validate_config(raw_config, {})
+
+    # Check for validation errors
+    assert not result.errors, f"Validation errors: {result.errors}"
+
+    # Logger should have been auto-loaded by test_component
+    assert "logger" in result
+    assert "test_component" in result
+
+
+def test_dynamic_auto_load_with_config_param(
+    mock_get_component: Mock,
+    fixtures_dir: Path,
+    default_component: Mock,
+) -> None:
+    """Test that dynamic AUTO_LOAD evaluates based on configuration."""
+    CORE.config_path = fixtures_dir / "auto_load_dynamic.yaml"
+
+    config_file = fixtures_dir / "auto_load_dynamic.yaml"
+    raw_config = yaml_util.load_yaml(config_file)
+
+    # Track if auto_load was called with config
+    auto_load_calls = []
+
+    def dynamic_auto_load(conf: dict[str, Any]) -> list[str]:
+        """Dynamically load components based on config."""
+        auto_load_calls.append(conf)
+        component_map = {
+            "enable_logger": "logger",
+            "enable_api": "api",
+        }
+        return [comp for key, comp in component_map.items() if conf.get(key)]
+
+    dynamic_component = Mock(
+        auto_load=dynamic_auto_load,
+        is_platform_component=False,
+        is_platform=False,
+        multi_conf=False,
+        multi_conf_no_default=False,
+        dependencies=[],
+        conflicts_with=[],
+        config_schema=cv.Schema({}, extra=cv.ALLOW_EXTRA),
+    )
+
+    component_mocks = {"test_component": dynamic_component}
+    mock_get_component.side_effect = lambda name: component_mocks.get(
+        name, default_component
+    )
+
+    result = config.validate_config(raw_config, {})
+
+    # Check for validation errors
+    assert not result.errors, f"Validation errors: {result.errors}"
+
+    # Verify auto_load was called with the validated config
+    assert len(auto_load_calls) == 1, "auto_load should be called exactly once"
+    assert auto_load_calls[0].get("enable_logger") is True
+    assert auto_load_calls[0].get("enable_api") is False
+
+    # Only logger should be auto-loaded (enable_logger=true in YAML)
+    assert "logger" in result, (
+        f"Logger not found in result. Result keys: {list(result.keys())}"
+    )
+    # API should NOT be auto-loaded (enable_api=false in YAML)
+    assert "api" not in result
+    assert "test_component" in result
diff --git a/tests/unit_tests/test_config_normalization.py b/tests/unit_tests/test_config_normalization.py
new file mode 100644
index 0000000000..d70f3c24e0
--- /dev/null
+++ b/tests/unit_tests/test_config_normalization.py
@@ -0,0 +1,115 @@
+"""Unit tests for esphome.config module."""
+
+from collections.abc import Generator
+from pathlib import Path
+from unittest.mock import MagicMock, Mock, patch
+
+import pytest
+
+from esphome import config, yaml_util
+from esphome.core import CORE
+
+
+@pytest.fixture
+def mock_get_platform() -> Generator[Mock, None, None]:
+    """Fixture for mocking get_platform."""
+    with patch("esphome.config.get_platform") as mock_get_platform:
+        # Default mock platform
+        mock_get_platform.return_value = MagicMock()
+        yield mock_get_platform
+
+
+@pytest.fixture
+def fixtures_dir() -> Path:
+    """Get the fixtures directory."""
+    return Path(__file__).parent / "fixtures"
+
+
+def test_ota_component_configs_with_proper_platform_list(
+    mock_get_component: Mock,
+    mock_get_platform: Mock,
+) -> None:
+    """Test iter_component_configs handles OTA properly configured as a list."""
+    test_config = {
+        "ota": [
+            {"platform": "esphome", "password": "test123", "id": "my_ota"},
+        ],
+    }
+
+    mock_get_component.return_value = MagicMock(
+        is_platform_component=True, multi_conf=False
+    )
+
+    configs = list(config.iter_component_configs(test_config))
+    assert len(configs) == 2
+
+    assert configs[0][0] == "ota"
+    assert configs[0][2] == test_config["ota"]  # The list itself
+
+    assert configs[1][0] == "ota.esphome"
+    assert configs[1][2]["platform"] == "esphome"
+    assert configs[1][2]["password"] == "test123"
+
+
+def test_iter_component_configs_with_multi_conf(mock_get_component: Mock) -> None:
+    """Test that iter_component_configs handles multi_conf components correctly."""
+    test_config = {
+        "switch": [
+            {"name": "Switch 1"},
+            {"name": "Switch 2"},
+        ],
+    }
+
+    mock_get_component.return_value = MagicMock(
+        is_platform_component=False, multi_conf=True
+    )
+
+    configs = list(config.iter_component_configs(test_config))
+    assert len(configs) == 2
+
+    for domain, component, conf in configs:
+        assert domain == "switch"
+        assert "name" in conf
+
+
+def test_ota_no_platform_with_captive_portal(fixtures_dir: Path) -> None:
+    """Test OTA with no platform (ota:) gets normalized when captive_portal auto-loads."""
+    CORE.config_path = fixtures_dir / "dummy.yaml"
+
+    config_file = fixtures_dir / "ota_no_platform.yaml"
+    raw_config = yaml_util.load_yaml(config_file)
+    result = config.validate_config(raw_config, {})
+
+    assert "ota" in result
+    assert isinstance(result["ota"], list), f"Expected list, got {type(result['ota'])}"
+    platforms = {p.get("platform") for p in result["ota"]}
+    assert "web_server" in platforms, f"Expected web_server platform in {platforms}"
+
+
+def test_ota_empty_dict_with_captive_portal(fixtures_dir: Path) -> None:
+    """Test OTA with empty dict ({}) gets normalized when captive_portal auto-loads."""
+    CORE.config_path = fixtures_dir / "dummy.yaml"
+
+    config_file = fixtures_dir / "ota_empty_dict.yaml"
+    raw_config = yaml_util.load_yaml(config_file)
+    result = config.validate_config(raw_config, {})
+
+    assert "ota" in result
+    assert isinstance(result["ota"], list), f"Expected list, got {type(result['ota'])}"
+    platforms = {p.get("platform") for p in result["ota"]}
+    assert "web_server" in platforms, f"Expected web_server platform in {platforms}"
+
+
+def test_ota_with_platform_list_and_captive_portal(fixtures_dir: Path) -> None:
+    """Test OTA with proper platform list remains valid when captive_portal auto-loads."""
+    CORE.config_path = fixtures_dir / "dummy.yaml"
+
+    config_file = fixtures_dir / "ota_with_platform_list.yaml"
+    raw_config = yaml_util.load_yaml(config_file)
+    result = config.validate_config(raw_config, {})
+
+    assert "ota" in result
+    assert isinstance(result["ota"], list), f"Expected list, got {type(result['ota'])}"
+    platforms = {p.get("platform") for p in result["ota"]}
+    assert "esphome" in platforms, f"Expected esphome platform in {platforms}"
+    assert "web_server" in platforms, f"Expected web_server platform in {platforms}"
diff --git a/tests/unit_tests/test_config_validation_paths.py b/tests/unit_tests/test_config_validation_paths.py
new file mode 100644
index 0000000000..f327e9c443
--- /dev/null
+++ b/tests/unit_tests/test_config_validation_paths.py
@@ -0,0 +1,187 @@
+"""Tests for config_validation.py path-related functions."""
+
+from pathlib import Path
+
+import pytest
+import voluptuous as vol
+
+from esphome import config_validation as cv
+
+
+def test_directory_valid_path(setup_core: Path) -> None:
+    """Test directory validator with valid directory."""
+    test_dir = setup_core / "test_directory"
+    test_dir.mkdir()
+
+    result = cv.directory("test_directory")
+
+    assert result == test_dir
+
+
+def test_directory_absolute_path(setup_core: Path) -> None:
+    """Test directory validator with absolute path."""
+    test_dir = setup_core / "test_directory"
+    test_dir.mkdir()
+
+    result = cv.directory(str(test_dir))
+
+    assert result == test_dir
+
+
+def test_directory_nonexistent_path(setup_core: Path) -> None:
+    """Test directory validator raises error for non-existent directory."""
+    with pytest.raises(
+        vol.Invalid, match="Could not find directory.*nonexistent_directory"
+    ):
+        cv.directory("nonexistent_directory")
+
+
+def test_directory_file_instead_of_directory(setup_core: Path) -> None:
+    """Test directory validator raises error when path is a file."""
+    test_file = setup_core / "test_file.txt"
+    test_file.write_text("content")
+
+    with pytest.raises(vol.Invalid, match="is not a directory"):
+        cv.directory("test_file.txt")
+
+
+def test_directory_with_parent_directory(setup_core: Path) -> None:
+    """Test directory validator with nested directory structure."""
+    nested_dir = setup_core / "parent" / "child" / "grandchild"
+    nested_dir.mkdir(parents=True)
+
+    result = cv.directory("parent/child/grandchild")
+
+    assert result == nested_dir
+
+
+def test_file_valid_path(setup_core: Path) -> None:
+    """Test file_ validator with valid file."""
+    test_file = setup_core / "test_file.yaml"
+    test_file.write_text("test content")
+
+    result = cv.file_("test_file.yaml")
+
+    assert result == test_file
+
+
+def test_file_absolute_path(setup_core: Path) -> None:
+    """Test file_ validator with absolute path."""
+    test_file = setup_core / "test_file.yaml"
+    test_file.write_text("test content")
+
+    result = cv.file_(str(test_file))
+
+    assert result == test_file
+
+
+def test_file_nonexistent_path(setup_core: Path) -> None:
+    """Test file_ validator raises error for non-existent file."""
+    with pytest.raises(vol.Invalid, match="Could not find file.*nonexistent_file.yaml"):
+        cv.file_("nonexistent_file.yaml")
+
+
+def test_file_directory_instead_of_file(setup_core: Path) -> None:
+    """Test file_ validator raises error when path is a directory."""
+    test_dir = setup_core / "test_directory"
+    test_dir.mkdir()
+
+    with pytest.raises(vol.Invalid, match="is not a file"):
+        cv.file_("test_directory")
+
+
+def test_file_with_parent_directory(setup_core: Path) -> None:
+    """Test file_ validator with file in nested directory."""
+    nested_dir = setup_core / "configs" / "sensors"
+    nested_dir.mkdir(parents=True)
+    test_file = nested_dir / "temperature.yaml"
+    test_file.write_text("sensor config")
+
+    result = cv.file_("configs/sensors/temperature.yaml")
+
+    assert result == test_file
+
+
+def test_directory_handles_trailing_slash(setup_core: Path) -> None:
+    """Test directory validator handles trailing slashes correctly."""
+    test_dir = setup_core / "test_dir"
+    test_dir.mkdir()
+
+    result = cv.directory("test_dir/")
+    assert result == test_dir
+
+    result = cv.directory("test_dir")
+    assert result == test_dir
+
+
+def test_file_handles_various_extensions(setup_core: Path) -> None:
+    """Test file_ validator works with different file extensions."""
+    yaml_file = setup_core / "config.yaml"
+    yaml_file.write_text("yaml content")
+    assert cv.file_("config.yaml") == yaml_file
+
+    yml_file = setup_core / "config.yml"
+    yml_file.write_text("yml content")
+    assert cv.file_("config.yml") == yml_file
+
+    txt_file = setup_core / "readme.txt"
+    txt_file.write_text("text content")
+    assert cv.file_("readme.txt") == txt_file
+
+    no_ext_file = setup_core / "LICENSE"
+    no_ext_file.write_text("license content")
+    assert cv.file_("LICENSE") == no_ext_file
+
+
+def test_directory_with_symlink(setup_core: Path) -> None:
+    """Test directory validator follows symlinks."""
+    actual_dir = setup_core / "actual_directory"
+    actual_dir.mkdir()
+
+    symlink_dir = setup_core / "symlink_directory"
+    symlink_dir.symlink_to(actual_dir)
+
+    result = cv.directory("symlink_directory")
+    assert result == symlink_dir
+
+
+def test_file_with_symlink(setup_core: Path) -> None:
+    """Test file_ validator follows symlinks."""
+    actual_file = setup_core / "actual_file.txt"
+    actual_file.write_text("content")
+
+    symlink_file = setup_core / "symlink_file.txt"
+    symlink_file.symlink_to(actual_file)
+
+    result = cv.file_("symlink_file.txt")
+    assert result == symlink_file
+
+
+def test_directory_error_shows_full_path(setup_core: Path) -> None:
+    """Test directory validator error message includes full path."""
+    with pytest.raises(vol.Invalid, match=".*missing_dir.*full path:.*"):
+        cv.directory("missing_dir")
+
+
+def test_file_error_shows_full_path(setup_core: Path) -> None:
+    """Test file_ validator error message includes full path."""
+    with pytest.raises(vol.Invalid, match=".*missing_file.yaml.*full path:.*"):
+        cv.file_("missing_file.yaml")
+
+
+def test_directory_with_spaces_in_name(setup_core: Path) -> None:
+    """Test directory validator handles spaces in directory names."""
+    dir_with_spaces = setup_core / "my test directory"
+    dir_with_spaces.mkdir()
+
+    result = cv.directory("my test directory")
+    assert result == dir_with_spaces
+
+
+def test_file_with_spaces_in_name(setup_core: Path) -> None:
+    """Test file_ validator handles spaces in file names."""
+    file_with_spaces = setup_core / "my test file.yaml"
+    file_with_spaces.write_text("content")
+
+    result = cv.file_("my test file.yaml")
+    assert result == file_with_spaces
diff --git a/tests/unit_tests/test_core.py b/tests/unit_tests/test_core.py
index 4f2a6453b4..48eae06ea6 100644
--- a/tests/unit_tests/test_core.py
+++ b/tests/unit_tests/test_core.py
@@ -1,3 +1,7 @@
+import os
+from pathlib import Path
+from unittest.mock import patch
+
 from hypothesis import given
 import pytest
 from strategies import mac_addr_strings
@@ -473,13 +477,68 @@ class TestLibrary:
 
         assert actual == expected
 
+    @pytest.mark.parametrize(
+        "target, other, result, exception",
+        (
+            (core.Library("libfoo", None), core.Library("libfoo", None), True, None),
+            (
+                core.Library("libfoo", "1.2.3"),
+                core.Library("libfoo", "1.2.3"),
+                True,  # target is unchanged
+                None,
+            ),
+            (
+                core.Library("libfoo", None),
+                core.Library("libfoo", "1.2.3"),
+                False,  # Use version from other
+                None,
+            ),
+            (
+                core.Library("libfoo", "1.2.3"),
+                core.Library("libfoo", "1.2.4"),
+                False,
+                ValueError,  # Version mismatch
+            ),
+            (
+                core.Library("libfoo", "1.2.3"),
+                core.Library("libbar", "1.2.3"),
+                False,
+                ValueError,  # Name mismatch
+            ),
+            (
+                core.Library(
+                    "libfoo", "1.2.4", "https://github.com/esphome/ESPAsyncWebServer"
+                ),
+                core.Library("libfoo", "1.2.3"),
+                True,  # target is unchanged due to having a repository
+                None,
+            ),
+            (
+                core.Library("libfoo", "1.2.3"),
+                core.Library(
+                    "libfoo", "1.2.4", "https://github.com/esphome/ESPAsyncWebServer"
+                ),
+                False,  # use other due to having a repository
+                None,
+            ),
+        ),
+    )
+    def test_reconcile(self, target, other, result, exception):
+        if exception is not None:
+            with pytest.raises(exception):
+                target.reconcile_with(other)
+        else:
+            expected = target if result else other
+            actual = target.reconcile_with(other)
+            assert actual == expected
+
 
 class TestEsphomeCore:
     @pytest.fixture
     def target(self, fixture_path):
         target = core.EsphomeCore()
-        target.build_path = "foo/build"
-        target.config_path = "foo/config"
+        target.build_path = Path("foo/build")
+        target.config_path = Path("foo/config")
         return target
 
     def test_reset(self, target):
@@ -522,3 +581,83 @@ class TestEsphomeCore:
 
         assert target.is_esp32 is False
         assert target.is_esp8266 is True
+
+    @pytest.mark.skipif(os.name == "nt", reason="Unix-specific test")
+    def test_data_dir_default_unix(self, target):
+        """Test data_dir returns .esphome in config directory by default on Unix."""
+        target.config_path = Path("/home/user/config.yaml")
+        assert target.data_dir == Path("/home/user/.esphome")
+
+    @pytest.mark.skipif(os.name != "nt", reason="Windows-specific test")
+    def test_data_dir_default_windows(self, target):
+        """Test data_dir returns .esphome in config directory by default on Windows."""
+        target.config_path = Path("D:\\home\\user\\config.yaml")
+        assert target.data_dir == Path("D:\\home\\user\\.esphome")
+
+    def test_data_dir_ha_addon(self, target):
+        """Test data_dir returns /data when running as Home Assistant addon."""
+        target.config_path = Path("/config/test.yaml")
+
+        with patch.dict(os.environ, {"ESPHOME_IS_HA_ADDON": "true"}):
+            assert target.data_dir == Path("/data")
+
+    def test_data_dir_env_override(self, target):
+        """Test data_dir uses ESPHOME_DATA_DIR environment variable when set."""
+        target.config_path = Path("/home/user/config.yaml")
+
+        with patch.dict(os.environ, {"ESPHOME_DATA_DIR": "/custom/data/path"}):
+            assert target.data_dir == Path("/custom/data/path")
+
+    @pytest.mark.skipif(os.name == "nt", reason="Unix-specific test")
+    def test_data_dir_priority_unix(self, target):
+        """Test data_dir priority on Unix: HA addon > env var > default."""
+        target.config_path = Path("/config/test.yaml")
+        expected_default = "/config/.esphome"
+
+        # Test HA addon takes priority over env var
+        with patch.dict(
+            os.environ,
+            {"ESPHOME_IS_HA_ADDON": "true", "ESPHOME_DATA_DIR": "/custom/path"},
+        ):
+            assert target.data_dir == Path("/data")
+
+        # Test env var is used when not HA addon
+        with patch.dict(
+            os.environ,
+            {"ESPHOME_IS_HA_ADDON": "false", "ESPHOME_DATA_DIR": "/custom/path"},
+        ):
+            assert target.data_dir == Path("/custom/path")
+
+        # Test default when neither is set
+        with patch.dict(os.environ, {}, clear=True):
+            # Ensure these env vars are not set
+            os.environ.pop("ESPHOME_IS_HA_ADDON", None)
+            os.environ.pop("ESPHOME_DATA_DIR", None)
+            assert target.data_dir == Path(expected_default)
+
+    @pytest.mark.skipif(os.name != "nt", reason="Windows-specific test")
+    def test_data_dir_priority_windows(self, target):
+        """Test data_dir priority on Windows: HA addon > env var > default."""
+        target.config_path = Path("D:\\config\\test.yaml")
+        expected_default = "D:\\config\\.esphome"
+
+        # Test HA addon takes priority over env var
+        with patch.dict(
+            os.environ,
+            {"ESPHOME_IS_HA_ADDON": "true", "ESPHOME_DATA_DIR": "/custom/path"},
+        ):
+            assert target.data_dir == Path("/data")
+
+        # Test env var is used when not HA addon
+        with patch.dict(
+            os.environ,
+            {"ESPHOME_IS_HA_ADDON": "false", "ESPHOME_DATA_DIR": "/custom/path"},
+        ):
+            assert target.data_dir == Path("/custom/path")
+
+        # Test default when neither is set
+        with patch.dict(os.environ, {}, clear=True):
+            # Ensure these env vars are not set
+            os.environ.pop("ESPHOME_IS_HA_ADDON", None)
+            os.environ.pop("ESPHOME_DATA_DIR", None)
+            assert target.data_dir == Path(expected_default)
diff --git a/tests/unit_tests/test_coroutine.py b/tests/unit_tests/test_coroutine.py
new file mode 100644
index 0000000000..e12c273294
--- /dev/null
+++ b/tests/unit_tests/test_coroutine.py
@@ -0,0 +1,219 @@
+"""Tests for the coroutine module."""
+
+import pytest
+
+from esphome.coroutine import CoroPriority, FakeEventLoop, coroutine_with_priority
+
+
+def test_coro_priority_enum_values() -> None:
+    """Test that CoroPriority enum values match expected priorities."""
+    assert CoroPriority.PLATFORM == 1000
+    assert CoroPriority.NETWORK == 201
+    assert CoroPriority.NETWORK_TRANSPORT == 200
+    assert CoroPriority.CORE == 100
+    assert CoroPriority.DIAGNOSTICS == 90
+    assert CoroPriority.STATUS == 80
+    assert CoroPriority.WEB_SERVER_BASE == 65
+    assert CoroPriority.CAPTIVE_PORTAL == 64
+    assert CoroPriority.COMMUNICATION == 60
+    assert CoroPriority.NETWORK_SERVICES == 55
+    assert CoroPriority.OTA_UPDATES == 54
+    assert CoroPriority.WEB_SERVER_OTA == 52
+    assert CoroPriority.APPLICATION == 50
+    assert CoroPriority.WEB == 40
+    assert CoroPriority.AUTOMATION == 30
+    assert CoroPriority.BUS == 1
+    assert CoroPriority.COMPONENT == 0
+    assert CoroPriority.LATE == -100
+    assert CoroPriority.WORKAROUNDS == -999
+    assert CoroPriority.FINAL == -1000
+
+
+def test_coroutine_with_priority_accepts_float() -> None:
+    """Test that coroutine_with_priority accepts float values."""
+
+    @coroutine_with_priority(100.0)
+    def test_func() -> None:
+        pass
+
+    assert hasattr(test_func, "priority")
+    assert test_func.priority == 100.0
+
+
+def test_coroutine_with_priority_accepts_enum() -> None:
+    """Test that coroutine_with_priority accepts CoroPriority enum values."""
+
+    @coroutine_with_priority(CoroPriority.CORE)
+    def test_func() -> None:
+        pass
+
+    assert hasattr(test_func, "priority")
+    assert test_func.priority == 100.0
+
+
+def test_float_and_enum_are_interchangeable() -> None:
+    """Test that float and CoroPriority enum values produce the same priority."""
+
+    @coroutine_with_priority(100.0)
+    def func_with_float() -> None:
+        pass
+
+    @coroutine_with_priority(CoroPriority.CORE)
+    def func_with_enum() -> None:
+        pass
+
+    assert func_with_float.priority == func_with_enum.priority
+    assert func_with_float.priority == 100.0
+
+
+@pytest.mark.parametrize(
+    ("enum_value", "float_value"),
+    [
+        (CoroPriority.PLATFORM, 1000.0),
+        (CoroPriority.NETWORK, 201.0),
+        (CoroPriority.NETWORK_TRANSPORT, 200.0),
+        (CoroPriority.CORE, 100.0),
+        (CoroPriority.DIAGNOSTICS, 90.0),
+        (CoroPriority.STATUS, 80.0),
+        (CoroPriority.WEB_SERVER_BASE, 65.0),
+        (CoroPriority.CAPTIVE_PORTAL, 64.0),
+        (CoroPriority.COMMUNICATION, 60.0),
+        (CoroPriority.NETWORK_SERVICES, 55.0),
+        (CoroPriority.OTA_UPDATES, 54.0),
+        (CoroPriority.WEB_SERVER_OTA, 52.0),
+        (CoroPriority.APPLICATION, 50.0),
+        (CoroPriority.WEB, 40.0),
+        (CoroPriority.AUTOMATION, 30.0),
+        (CoroPriority.BUS, 1.0),
+        (CoroPriority.COMPONENT, 0.0),
+        (CoroPriority.LATE, -100.0),
+        (CoroPriority.WORKAROUNDS, -999.0),
+        (CoroPriority.FINAL, -1000.0),
+    ],
+)
+def test_all_priority_values_are_interchangeable(
+    enum_value: CoroPriority, float_value: float
+) -> None:
+    """Test that all CoroPriority values work correctly with coroutine_with_priority."""
+
+    @coroutine_with_priority(enum_value)
+    def func_with_enum() -> None:
+        pass
+
+    @coroutine_with_priority(float_value)
+    def func_with_float() -> None:
+        pass
+
+    assert func_with_enum.priority == float_value
+    assert func_with_float.priority == float_value
+    assert func_with_enum.priority == func_with_float.priority
+
+
+def test_execution_order_with_enum_priorities() -> None:
+    """Test that execution order is correct when using enum priorities."""
+    execution_order: list[str] = []
+
+    @coroutine_with_priority(CoroPriority.PLATFORM)
+    async def platform_func() -> None:
+        execution_order.append("platform")
+
+    @coroutine_with_priority(CoroPriority.CORE)
+    async def core_func() -> None:
+        execution_order.append("core")
+
+    @coroutine_with_priority(CoroPriority.FINAL)
+    async def final_func() -> None:
+        execution_order.append("final")
+
+    # Create event loop and add jobs
+    loop = FakeEventLoop()
+    loop.add_job(platform_func)
+    loop.add_job(core_func)
+    loop.add_job(final_func)
+
+    # Run all tasks
+    loop.flush_tasks()
+
+    # Check execution order (higher priority runs first)
+    assert execution_order == ["platform", "core", "final"]
+
+
+def test_mixed_float_and_enum_priorities() -> None:
+    """Test that mixing float and enum priorities works correctly."""
+    execution_order: list[str] = []
+
+    @coroutine_with_priority(1000.0)  # Same as PLATFORM
+    async def func1() -> None:
+        execution_order.append("func1")
+
+    @coroutine_with_priority(CoroPriority.CORE)
+    async def func2() -> None:
+        execution_order.append("func2")
+
+    @coroutine_with_priority(-1000.0)  # Same as FINAL
+    async def func3() -> None:
+        execution_order.append("func3")
+
+    # Create event loop and add jobs
+    loop = FakeEventLoop()
+    loop.add_job(func2)
+    loop.add_job(func3)
+    loop.add_job(func1)
+
+    # Run all tasks
+    loop.flush_tasks()
+
+    # Check execution order
+    assert execution_order == ["func1", "func2", "func3"]
+
+
+def test_enum_priority_comparison() -> None:
+    """Test that enum priorities can be compared directly."""
+    assert CoroPriority.PLATFORM > CoroPriority.NETWORK
+    assert CoroPriority.NETWORK > CoroPriority.NETWORK_TRANSPORT
+    assert CoroPriority.NETWORK_TRANSPORT > CoroPriority.CORE
+    assert CoroPriority.CORE > CoroPriority.DIAGNOSTICS
+    assert CoroPriority.DIAGNOSTICS > CoroPriority.STATUS
+    assert CoroPriority.STATUS > CoroPriority.WEB_SERVER_BASE
+    assert CoroPriority.WEB_SERVER_BASE > CoroPriority.CAPTIVE_PORTAL
+    assert CoroPriority.CAPTIVE_PORTAL > CoroPriority.COMMUNICATION
+    assert CoroPriority.COMMUNICATION > CoroPriority.NETWORK_SERVICES
+    assert CoroPriority.NETWORK_SERVICES > CoroPriority.OTA_UPDATES
+    assert CoroPriority.OTA_UPDATES > CoroPriority.WEB_SERVER_OTA
+    assert CoroPriority.WEB_SERVER_OTA > CoroPriority.APPLICATION
+    assert CoroPriority.APPLICATION > CoroPriority.WEB
+    assert CoroPriority.WEB > CoroPriority.AUTOMATION
+    assert CoroPriority.AUTOMATION > CoroPriority.BUS
+    assert CoroPriority.BUS > CoroPriority.COMPONENT
+    assert CoroPriority.COMPONENT > CoroPriority.LATE
+    assert CoroPriority.LATE > CoroPriority.WORKAROUNDS
+    assert CoroPriority.WORKAROUNDS > CoroPriority.FINAL
+
+
+def test_custom_priority_between_enum_values() -> None:
+    """Test that custom float priorities between enum values work correctly."""
+    execution_order: list[str] = []
+
+    @coroutine_with_priority(CoroPriority.CORE)  # 100
+    async def core_func() -> None:
+        execution_order.append("core")
+
+    @coroutine_with_priority(95.0)  # Between CORE and DIAGNOSTICS
+    async def custom_func() -> None:
+        execution_order.append("custom")
+
+    @coroutine_with_priority(CoroPriority.DIAGNOSTICS)  # 90
+    async def diag_func() -> None:
+        execution_order.append("diagnostics")
+
+    # Create event loop and add jobs
+    loop = FakeEventLoop()
+    loop.add_job(diag_func)
+    loop.add_job(core_func)
+    loop.add_job(custom_func)
+
+    # Run all tasks
+    loop.flush_tasks()
+
+    # Check execution order
+    assert execution_order == ["core", "custom", "diagnostics"]
diff --git a/tests/unit_tests/test_espota2.py b/tests/unit_tests/test_espota2.py
new file mode 100644
index 0000000000..02f965782b
--- /dev/null
+++ b/tests/unit_tests/test_espota2.py
@@ -0,0 +1,738 @@
+"""Unit tests for esphome.espota2 module."""
+
+from __future__ import annotations
+
+from collections.abc import Generator
+import gzip
+import hashlib
+import io
+from pathlib import Path
+import socket
+import struct
+from unittest.mock import Mock, call, patch
+
+import pytest
+from pytest import CaptureFixture
+
+from esphome import espota2
+from esphome.core import EsphomeError
+
+# Test constants
+MOCK_RANDOM_VALUE = 0.123456
+MOCK_RANDOM_BYTES = b"0.123456"
+MOCK_MD5_NONCE = b"12345678901234567890123456789012"  # 32 char nonce for MD5
+MOCK_SHA256_NONCE = b"1234567890123456789012345678901234567890123456789012345678901234"  # 64 char nonce for SHA256
+
+
+@pytest.fixture
+def mock_socket() -> Mock:
+    """Create a mock socket for testing."""
+    socket_mock = Mock()
+    socket_mock.close = Mock()
+    socket_mock.recv = Mock()
+    socket_mock.sendall = Mock()
+    socket_mock.settimeout = Mock()
+    socket_mock.connect = Mock()
+    socket_mock.setsockopt = Mock()
+    return socket_mock
+
+
+@pytest.fixture
+def mock_file() -> io.BytesIO:
+    """Create a mock firmware file for testing."""
+    return io.BytesIO(b"firmware content here")
+
+
+@pytest.fixture
+def mock_time() -> Generator[None]:
+    """Mock time-related functions for consistent testing."""
+    # Provide enough values for multiple calls (tests may call perform_ota multiple times)
+    with (
+        patch("time.sleep"),
+        patch("time.perf_counter", side_effect=[0, 1, 0, 1, 0, 1]),
+    ):
+        yield
+
+
+@pytest.fixture
+def mock_random() -> Generator[Mock]:
+    """Mock random for predictable test values."""
+    with patch("random.random", return_value=MOCK_RANDOM_VALUE) as mock_rand:
+        yield mock_rand
+
+
+@pytest.fixture
+def mock_resolve_ip() -> Generator[Mock]:
+    """Mock resolve_ip_address for testing."""
+    with patch("esphome.espota2.resolve_ip_address") as mock:
+        mock.return_value = [
+            (socket.AF_INET, socket.SOCK_STREAM, 0, "", ("192.168.1.100", 3232))
+        ]
+        yield mock
+
+
+@pytest.fixture
+def mock_perform_ota() -> Generator[Mock]:
+    """Mock perform_ota function for testing."""
+    with patch("esphome.espota2.perform_ota") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_run_ota_impl() -> Generator[Mock]:
+    """Mock run_ota_impl_ function for testing."""
+    with patch("esphome.espota2.run_ota_impl_") as mock:
+        mock.return_value = (0, "192.168.1.100")
+        yield mock
+
+
+@pytest.fixture
+def mock_socket_constructor(mock_socket: Mock) -> Generator[Mock]:
+    """Mock socket.socket constructor to return our mock socket."""
+    with patch("socket.socket", return_value=mock_socket) as mock_constructor:
+        yield mock_constructor
+
+
+def test_recv_decode_with_decode(mock_socket: Mock) -> None:
+    """Test recv_decode with decode=True returns list."""
+    mock_socket.recv.return_value = b"\x01\x02\x03"
+
+    result = espota2.recv_decode(mock_socket, 3, decode=True)
+
+    assert result == [1, 2, 3]
+    mock_socket.recv.assert_called_once_with(3)
+
+
+def test_recv_decode_without_decode(mock_socket: Mock) -> None:
+    """Test recv_decode with decode=False returns bytes."""
+    mock_socket.recv.return_value = b"\x01\x02\x03"
+
+    result = espota2.recv_decode(mock_socket, 3, decode=False)
+
+    assert result == b"\x01\x02\x03"
+    mock_socket.recv.assert_called_once_with(3)
+
+
+def test_receive_exactly_success(mock_socket: Mock) -> None:
+    """Test receive_exactly successfully receives expected data."""
+    mock_socket.recv.side_effect = [b"\x00", b"\x01\x02"]
+
+    result = espota2.receive_exactly(mock_socket, 3, "test", espota2.RESPONSE_OK)
+
+    assert result == [0, 1, 2]
+    assert mock_socket.recv.call_count == 2
+
+
+def test_receive_exactly_with_error_response(mock_socket: Mock) -> None:
+    """Test receive_exactly raises OTAError on error response."""
+    mock_socket.recv.return_value = bytes([espota2.RESPONSE_ERROR_AUTH_INVALID])
+
+    with pytest.raises(espota2.OTAError, match="Error auth:.*Authentication invalid"):
+        espota2.receive_exactly(mock_socket, 1, "auth", [espota2.RESPONSE_OK])
+
+    mock_socket.close.assert_called_once()
+
+
+def test_receive_exactly_socket_error(mock_socket: Mock) -> None:
+    """Test receive_exactly handles socket errors."""
+    mock_socket.recv.side_effect = OSError("Connection reset")
+
+    with pytest.raises(espota2.OTAError, match="Error receiving acknowledge test"):
+        espota2.receive_exactly(mock_socket, 1, "test", espota2.RESPONSE_OK)
+
+
+@pytest.mark.parametrize(
+    ("error_code", "expected_msg"),
+    [
+        (espota2.RESPONSE_ERROR_MAGIC, "Error: Invalid magic byte"),
+        (espota2.RESPONSE_ERROR_UPDATE_PREPARE, "Error: Couldn't prepare flash memory"),
+        (espota2.RESPONSE_ERROR_AUTH_INVALID, "Error: Authentication invalid"),
+        (
+            espota2.RESPONSE_ERROR_WRITING_FLASH,
+            "Error: Writing OTA data to flash memory failed",
+        ),
+        (espota2.RESPONSE_ERROR_UPDATE_END, "Error: Finishing update failed"),
+        (
+            espota2.RESPONSE_ERROR_INVALID_BOOTSTRAPPING,
+            "Error: Please press the reset button",
+        ),
+        (
+            espota2.RESPONSE_ERROR_WRONG_CURRENT_FLASH_CONFIG,
+            "Error: ESP has been flashed with wrong flash size",
+        ),
+        (
+            espota2.RESPONSE_ERROR_WRONG_NEW_FLASH_CONFIG,
+            "Error: ESP does not have the requested flash size",
+        ),
+        (
+            espota2.RESPONSE_ERROR_ESP8266_NOT_ENOUGH_SPACE,
+            "Error: ESP does not have enough space",
+        ),
+        (
+            espota2.RESPONSE_ERROR_ESP32_NOT_ENOUGH_SPACE,
+            "Error: The OTA partition on the ESP is too small",
+        ),
+        (
+            espota2.RESPONSE_ERROR_NO_UPDATE_PARTITION,
+            "Error: The OTA partition on the ESP couldn't be found",
+        ),
+        (espota2.RESPONSE_ERROR_MD5_MISMATCH, "Error: Application MD5 code mismatch"),
+        (espota2.RESPONSE_ERROR_UNKNOWN, "Unknown error from ESP"),
+    ],
+)
+def test_check_error_with_various_errors(error_code: int, expected_msg: str) -> None:
+    """Test check_error raises appropriate errors for different error codes."""
+    with pytest.raises(espota2.OTAError, match=expected_msg):
+        espota2.check_error([error_code], [espota2.RESPONSE_OK])
+
+
+def test_check_error_unexpected_response() -> None:
+    """Test check_error raises error for unexpected response."""
+    with pytest.raises(espota2.OTAError, match="Unexpected response from ESP: 0x7F"):
+        espota2.check_error([0x7F], [espota2.RESPONSE_OK, espota2.RESPONSE_AUTH_OK])
+
+
+def test_send_check_with_various_data_types(mock_socket: Mock) -> None:
+    """Test send_check handles different data types."""
+
+    # Test with list/tuple
+    espota2.send_check(mock_socket, [0x01, 0x02], "list")
+    mock_socket.sendall.assert_called_with(b"\x01\x02")
+
+    # Test with int
+    espota2.send_check(mock_socket, 0x42, "int")
+    mock_socket.sendall.assert_called_with(b"\x42")
+
+    # Test with string
+    espota2.send_check(mock_socket, "hello", "string")
+    mock_socket.sendall.assert_called_with(b"hello")
+
+    # Test with bytes (should pass through)
+    espota2.send_check(mock_socket, b"\xaa\xbb", "bytes")
+    mock_socket.sendall.assert_called_with(b"\xaa\xbb")
+
+
+def test_send_check_socket_error(mock_socket: Mock) -> None:
+    """Test send_check handles socket errors."""
+    mock_socket.sendall.side_effect = OSError("Broken pipe")
+
+    with pytest.raises(espota2.OTAError, match="Error sending test"):
+        espota2.send_check(mock_socket, b"data", "test")
+
+
+@pytest.mark.usefixtures("mock_time")
+def test_perform_ota_successful_md5_auth(
+    mock_socket: Mock, mock_file: io.BytesIO, mock_random: Mock
+) -> None:
+    """Test successful OTA with MD5 authentication."""
+    # Setup socket responses for recv calls
+    recv_responses = [
+        bytes([espota2.RESPONSE_OK]),  # First byte of version response
+        bytes([espota2.OTA_VERSION_2_0]),  # Version number
+        bytes([espota2.RESPONSE_HEADER_OK]),  # Features response
+        bytes([espota2.RESPONSE_REQUEST_AUTH]),  # Auth request
+        MOCK_MD5_NONCE,  # 32 char hex nonce
+        bytes([espota2.RESPONSE_AUTH_OK]),  # Auth result
+        bytes([espota2.RESPONSE_UPDATE_PREPARE_OK]),  # Binary size OK
+        bytes([espota2.RESPONSE_BIN_MD5_OK]),  # MD5 checksum OK
+        bytes([espota2.RESPONSE_CHUNK_OK]),  # Chunk OK
+        bytes([espota2.RESPONSE_RECEIVE_OK]),  # Receive OK
+        bytes([espota2.RESPONSE_UPDATE_END_OK]),  # Update end OK
+    ]
+
+    mock_socket.recv.side_effect = recv_responses
+
+    # Run OTA
+    espota2.perform_ota(mock_socket, "testpass", mock_file, "test.bin")
+
+    # Verify magic bytes were sent
+    assert mock_socket.sendall.call_args_list[0] == call(bytes(espota2.MAGIC_BYTES))
+
+    # Verify features were sent (compression + SHA256 support)
+    assert mock_socket.sendall.call_args_list[1] == call(
+        bytes(
+            [
+                espota2.FEATURE_SUPPORTS_COMPRESSION
+                | espota2.FEATURE_SUPPORTS_SHA256_AUTH
+            ]
+        )
+    )
+
+    # Verify cnonce was sent (MD5 of random.random())
+    cnonce = hashlib.md5(MOCK_RANDOM_BYTES).hexdigest()
+    assert mock_socket.sendall.call_args_list[2] == call(cnonce.encode())
+
+    # Verify auth result was computed correctly
+    expected_hash = hashlib.md5()
+    expected_hash.update(b"testpass")
+    expected_hash.update(MOCK_MD5_NONCE)
+    expected_hash.update(cnonce.encode())
+    expected_result = expected_hash.hexdigest()
+    assert mock_socket.sendall.call_args_list[3] == call(expected_result.encode())
+
+
+@pytest.mark.usefixtures("mock_time")
+def test_perform_ota_no_auth(mock_socket: Mock, mock_file: io.BytesIO) -> None:
+    """Test OTA without authentication."""
+    recv_responses = [
+        bytes([espota2.RESPONSE_OK]),  # First byte of version response
+        bytes([espota2.OTA_VERSION_1_0]),  # Version number
+        bytes([espota2.RESPONSE_HEADER_OK]),  # Features response
+        bytes([espota2.RESPONSE_AUTH_OK]),  # No auth required
+        bytes([espota2.RESPONSE_UPDATE_PREPARE_OK]),  # Binary size OK
+        bytes([espota2.RESPONSE_BIN_MD5_OK]),  # MD5 checksum OK
+        bytes([espota2.RESPONSE_RECEIVE_OK]),  # Receive OK
+        bytes([espota2.RESPONSE_UPDATE_END_OK]),  # Update end OK
+    ]
+
+    mock_socket.recv.side_effect = recv_responses
+
+    espota2.perform_ota(mock_socket, None, mock_file, "test.bin")
+
+    # Should not send any auth-related data
+    auth_calls = [
+        call
+        for call in mock_socket.sendall.call_args_list
+        if "cnonce" in str(call) or "result" in str(call)
+    ]
+    assert len(auth_calls) == 0
+
+
+@pytest.mark.usefixtures("mock_time")
+def test_perform_ota_with_compression(mock_socket: Mock) -> None:
+    """Test OTA with compression support."""
+    original_content = b"firmware" * 100  # Repeating content for compression
+    mock_file = io.BytesIO(original_content)
+    recv_responses = [
+        bytes([espota2.RESPONSE_OK]),  # First byte of version response
+        bytes([espota2.OTA_VERSION_2_0]),  # Version number
+        bytes([espota2.RESPONSE_SUPPORTS_COMPRESSION]),  # Device supports compression
+        bytes([espota2.RESPONSE_AUTH_OK]),  # No auth required
+        bytes([espota2.RESPONSE_UPDATE_PREPARE_OK]),  # Binary size OK
+        bytes([espota2.RESPONSE_BIN_MD5_OK]),  # MD5 checksum OK
+        bytes([espota2.RESPONSE_CHUNK_OK]),  # Chunk OK
+        bytes([espota2.RESPONSE_RECEIVE_OK]),  # Receive OK
+        bytes([espota2.RESPONSE_UPDATE_END_OK]),  # Update end OK
+    ]
+
+    mock_socket.recv.side_effect = recv_responses
+
+    espota2.perform_ota(mock_socket, None, mock_file, "test.bin")
+
+    # Verify compressed content was sent
+    # Get the binary size that was sent (4 bytes after features)
+    size_bytes = mock_socket.sendall.call_args_list[2][0][0]
+    sent_size = struct.unpack(">I", size_bytes)[0]
+
+    # Size should be less than original due to compression
+    assert sent_size < len(original_content)
+
+    # Verify the content sent was gzipped
+    compressed = gzip.compress(original_content, compresslevel=9)
+    assert sent_size == len(compressed)
+
+
+def test_perform_ota_auth_without_password(mock_socket: Mock) -> None:
+    """Test OTA fails when auth is required but no password provided."""
+    mock_file = io.BytesIO(b"firmware")
+
+    responses = [
+        bytes([espota2.RESPONSE_OK, espota2.OTA_VERSION_2_0]),
+        bytes([espota2.RESPONSE_HEADER_OK]),
+        bytes([espota2.RESPONSE_REQUEST_AUTH]),
+    ]
+
+    mock_socket.recv.side_effect = responses
+
+    with pytest.raises(
+        espota2.OTAError, match="ESP requests password, but no password given"
+    ):
+        espota2.perform_ota(mock_socket, None, mock_file, "test.bin")
+
+
+@pytest.mark.usefixtures("mock_time")
+def test_perform_ota_md5_auth_wrong_password(
+    mock_socket: Mock, mock_file: io.BytesIO, mock_random: Mock
+) -> None:
+    """Test OTA fails when MD5 authentication is rejected due to wrong password."""
+    # Setup socket responses for recv calls
+    recv_responses = [
+        bytes([espota2.RESPONSE_OK]),  # First byte of version response
+        bytes([espota2.OTA_VERSION_2_0]),  # Version number
+        bytes([espota2.RESPONSE_HEADER_OK]),  # Features response
+        bytes([espota2.RESPONSE_REQUEST_AUTH]),  # Auth request
+        MOCK_MD5_NONCE,  # 32 char hex nonce
+        bytes([espota2.RESPONSE_ERROR_AUTH_INVALID]),  # Auth rejected!
+    ]
+
+    mock_socket.recv.side_effect = recv_responses
+
+    with pytest.raises(espota2.OTAError, match="Error auth.*Authentication invalid"):
+        espota2.perform_ota(mock_socket, "wrongpassword", mock_file, "test.bin")
+
+    # Verify the socket was closed after auth failure
+    mock_socket.close.assert_called()
+
+
+@pytest.mark.usefixtures("mock_time")
+def test_perform_ota_sha256_auth_wrong_password(
+    mock_socket: Mock, mock_file: io.BytesIO, mock_random: Mock
+) -> None:
+    """Test OTA fails when SHA256 authentication is rejected due to wrong password."""
+    # Setup socket responses for recv calls
+    recv_responses = [
+        bytes([espota2.RESPONSE_OK]),  # First byte of version response
+        bytes([espota2.OTA_VERSION_2_0]),  # Version number
+        bytes([espota2.RESPONSE_HEADER_OK]),  # Features response
+        bytes([espota2.RESPONSE_REQUEST_SHA256_AUTH]),  # SHA256 Auth request
+        MOCK_SHA256_NONCE,  # 64 char hex nonce
+        bytes([espota2.RESPONSE_ERROR_AUTH_INVALID]),  # Auth rejected!
+    ]
+
+    mock_socket.recv.side_effect = recv_responses
+
+    with pytest.raises(espota2.OTAError, match="Error auth.*Authentication invalid"):
+        espota2.perform_ota(mock_socket, "wrongpassword", mock_file, "test.bin")
+
+    # Verify the socket was closed after auth failure
+    mock_socket.close.assert_called()
+
+
+def test_perform_ota_sha256_auth_without_password(mock_socket: Mock) -> None:
+    """Test OTA fails when SHA256 auth is required but no password provided."""
+    mock_file = io.BytesIO(b"firmware")
+
+    responses = [
+        bytes([espota2.RESPONSE_OK, espota2.OTA_VERSION_2_0]),
+        bytes([espota2.RESPONSE_HEADER_OK]),
+        bytes([espota2.RESPONSE_REQUEST_SHA256_AUTH]),
+    ]
+
+    mock_socket.recv.side_effect = responses
+
+    with pytest.raises(
+        espota2.OTAError, match="ESP requests password, but no password given"
+    ):
+        espota2.perform_ota(mock_socket, None, mock_file, "test.bin")
+
+
+def test_perform_ota_unexpected_auth_response(mock_socket: Mock) -> None:
+    """Test OTA fails when device sends an unexpected auth response."""
+    mock_file = io.BytesIO(b"firmware")
+
+    # Use 0x03 which is not in the expected auth responses
+    # This will be caught by check_error and raise "Unexpected response from ESP"
+    UNKNOWN_AUTH_METHOD = 0x03
+
+    responses = [
+        bytes([espota2.RESPONSE_OK, espota2.OTA_VERSION_2_0]),
+        bytes([espota2.RESPONSE_HEADER_OK]),
+        bytes([UNKNOWN_AUTH_METHOD]),  # Unknown auth method
+    ]
+
+    mock_socket.recv.side_effect = responses
+
+    # This will actually raise "Unexpected response from ESP" from check_error
+    with pytest.raises(
+        espota2.OTAError, match=r"Error auth: Unexpected response from ESP: 0x03"
+    ):
+        espota2.perform_ota(mock_socket, "password", mock_file, "test.bin")
+
+
+def test_perform_ota_unsupported_version(mock_socket: Mock) -> None:
+    """Test OTA fails with unsupported version."""
+    mock_file = io.BytesIO(b"firmware")
+
+    responses = [
+        bytes([espota2.RESPONSE_OK, 99]),  # Unsupported version
+    ]
+
+    mock_socket.recv.side_effect = responses
+
+    with pytest.raises(espota2.OTAError, match="Device uses unsupported OTA version"):
+        espota2.perform_ota(mock_socket, None, mock_file, "test.bin")
+
+
+@pytest.mark.usefixtures("mock_time")
+def test_perform_ota_upload_error(mock_socket: Mock, mock_file: io.BytesIO) -> None:
+    """Test OTA handles upload errors."""
+    # Setup responses - provide enough for the recv calls
+    recv_responses = [
+        bytes([espota2.RESPONSE_OK]),  # First byte of version response
+        bytes([espota2.OTA_VERSION_2_0]),  # Version number
+        bytes([espota2.RESPONSE_HEADER_OK]),  # Features response
+        bytes([espota2.RESPONSE_AUTH_OK]),  # No auth required
+        bytes([espota2.RESPONSE_UPDATE_PREPARE_OK]),  # Binary size OK
+        bytes([espota2.RESPONSE_BIN_MD5_OK]),  # MD5 checksum OK
+    ]
+    # Add OSError to recv to simulate connection loss during chunk read
+    recv_responses.append(OSError("Connection lost"))
+
+    mock_socket.recv.side_effect = recv_responses
+
+    with pytest.raises(espota2.OTAError, match="Error receiving acknowledge chunk OK"):
+        espota2.perform_ota(mock_socket, None, mock_file, "test.bin")
+
+
+@pytest.mark.usefixtures("mock_socket_constructor", "mock_resolve_ip")
+def test_run_ota_impl_successful(
+    mock_socket: Mock, tmp_path: Path, mock_perform_ota: Mock
+) -> None:
+    """Test run_ota_impl_ with successful upload."""
+    # Create a real firmware file
+    firmware_file = tmp_path / "firmware.bin"
+    firmware_file.write_bytes(b"firmware content")
+
+    # Run OTA with real file path
+    result_code, result_host = espota2.run_ota_impl_(
+        "test.local", 3232, "password", str(firmware_file)
+    )
+
+    # Verify success
+    assert result_code == 0
+    assert result_host == "192.168.1.100"
+
+    # Verify socket was configured correctly
+    mock_socket.settimeout.assert_called_with(20.0)
+    mock_socket.connect.assert_called_once_with(("192.168.1.100", 3232))
+    mock_socket.close.assert_called_once()
+
+    # Verify perform_ota was called with real file
+    mock_perform_ota.assert_called_once()
+    call_args = mock_perform_ota.call_args[0]
+    assert call_args[0] == mock_socket
+    assert call_args[1] == "password"
+    # Verify the file object is a proper file handle
+    assert isinstance(call_args[2], io.IOBase)
+    assert call_args[3] == str(firmware_file)
+
+
+@pytest.mark.usefixtures("mock_socket_constructor", "mock_resolve_ip")
+def test_run_ota_impl_connection_failed(mock_socket: Mock, tmp_path: Path) -> None:
+    """Test run_ota_impl_ when connection fails."""
+    mock_socket.connect.side_effect = OSError("Connection refused")
+
+    # Create a real firmware file
+    firmware_file = tmp_path / "firmware.bin"
+    firmware_file.write_bytes(b"firmware content")
+
+    result_code, result_host = espota2.run_ota_impl_(
+        "test.local", 3232, "password", str(firmware_file)
+    )
+
+    assert result_code == 1
+    assert result_host is None
+    mock_socket.close.assert_called_once()
+
+
+def test_run_ota_impl_resolve_failed(tmp_path: Path, mock_resolve_ip: Mock) -> None:
+    """Test run_ota_impl_ when DNS resolution fails."""
+    # Create a real firmware file
+    firmware_file = tmp_path / "firmware.bin"
+    firmware_file.write_bytes(b"firmware content")
+
+    mock_resolve_ip.side_effect = EsphomeError("DNS resolution failed")
+
+    with pytest.raises(espota2.OTAError, match="DNS resolution failed"):
+        result_code, result_host = espota2.run_ota_impl_(
+            "unknown.host", 3232, "password", str(firmware_file)
+        )
+
+
+def test_run_ota_wrapper(mock_run_ota_impl: Mock) -> None:
+    """Test run_ota wrapper function."""
+    # Test successful case
+    mock_run_ota_impl.return_value = (0, "192.168.1.100")
+    result = espota2.run_ota("test.local", 3232, "pass", "fw.bin")
+    assert result == (0, "192.168.1.100")
+
+    # Test error case
+    mock_run_ota_impl.side_effect = espota2.OTAError("Test error")
+    result = espota2.run_ota("test.local", 3232, "pass", "fw.bin")
+    assert result == (1, None)
+
+
+def test_progress_bar(capsys: CaptureFixture[str]) -> None:
+    """Test ProgressBar functionality."""
+    progress = espota2.ProgressBar()
+
+    # Test initial update
+    progress.update(0.0)
+    captured = capsys.readouterr()
+    assert "0%" in captured.err
+    assert "[" in captured.err
+
+    # Test progress update
+    progress.update(0.5)
+    captured = capsys.readouterr()
+    assert "50%" in captured.err
+
+    # Test completion
+    progress.update(1.0)
+    captured = capsys.readouterr()
+    assert "100%" in captured.err
+    assert "Done" in captured.err
+
+    # Test done method
+    progress.done()
+    captured = capsys.readouterr()
+    assert captured.err == "\n"
+
+    # Test same progress doesn't update
+    progress.update(0.5)
+    progress.update(0.5)
+    captured = capsys.readouterr()
+    # Should only see one update (second call shouldn't write)
+    assert captured.err.count("50%") == 1
+
+
+# Tests for SHA256 authentication
+@pytest.mark.usefixtures("mock_time")
+def test_perform_ota_successful_sha256_auth(
+    mock_socket: Mock, mock_file: io.BytesIO, mock_random: Mock
+) -> None:
+    """Test successful OTA with SHA256 authentication."""
+    # Setup socket responses for recv calls
+    recv_responses = [
+        bytes([espota2.RESPONSE_OK]),  # First byte of version response
+        bytes([espota2.OTA_VERSION_2_0]),  # Version number
+        bytes([espota2.RESPONSE_HEADER_OK]),  # Features response
+        bytes([espota2.RESPONSE_REQUEST_SHA256_AUTH]),  # SHA256 Auth request
+        MOCK_SHA256_NONCE,  # 64 char hex nonce
+        bytes([espota2.RESPONSE_AUTH_OK]),  # Auth result
+        bytes([espota2.RESPONSE_UPDATE_PREPARE_OK]),  # Binary size OK
+        bytes([espota2.RESPONSE_BIN_MD5_OK]),  # MD5 checksum OK
+        bytes([espota2.RESPONSE_CHUNK_OK]),  # Chunk OK
+        bytes([espota2.RESPONSE_RECEIVE_OK]),  # Receive OK
+        bytes([espota2.RESPONSE_UPDATE_END_OK]),  # Update end OK
+    ]
+
+    mock_socket.recv.side_effect = recv_responses
+
+    # Run OTA
+    espota2.perform_ota(mock_socket, "testpass", mock_file, "test.bin")
+
+    # Verify magic bytes were sent
+    assert mock_socket.sendall.call_args_list[0] == call(bytes(espota2.MAGIC_BYTES))
+
+    # Verify features were sent (compression + SHA256 support)
+    assert mock_socket.sendall.call_args_list[1] == call(
+        bytes(
+            [
+                espota2.FEATURE_SUPPORTS_COMPRESSION
+                | espota2.FEATURE_SUPPORTS_SHA256_AUTH
+            ]
+        )
+    )
+
+    # Verify cnonce was sent (SHA256 of random.random())
+    cnonce = hashlib.sha256(MOCK_RANDOM_BYTES).hexdigest()
+    assert mock_socket.sendall.call_args_list[2] == call(cnonce.encode())
+
+    # Verify auth result was computed correctly with SHA256
+    expected_hash = hashlib.sha256()
+    expected_hash.update(b"testpass")
+    expected_hash.update(MOCK_SHA256_NONCE)
+    expected_hash.update(cnonce.encode())
+    expected_result = expected_hash.hexdigest()
+    assert mock_socket.sendall.call_args_list[3] == call(expected_result.encode())
+
+
+@pytest.mark.usefixtures("mock_time")
+def test_perform_ota_sha256_fallback_to_md5(
+    mock_socket: Mock, mock_file: io.BytesIO, mock_random: Mock
+) -> None:
+    """Test SHA256-capable client falls back to MD5 for compatibility."""
+    # This test verifies the temporary backward compatibility
+    # where a SHA256-capable client can still authenticate with MD5
+    # This compatibility will be removed in 2026.1.0
+    recv_responses = [
+        bytes([espota2.RESPONSE_OK]),  # First byte of version response
+        bytes([espota2.OTA_VERSION_2_0]),  # Version number
+        bytes([espota2.RESPONSE_HEADER_OK]),  # Features response
+        bytes(
+            [espota2.RESPONSE_REQUEST_AUTH]
+        ),  # MD5 Auth request (device doesn't support SHA256)
+        MOCK_MD5_NONCE,  # 32 char hex nonce for MD5
+        bytes([espota2.RESPONSE_AUTH_OK]),  # Auth result
+        bytes([espota2.RESPONSE_UPDATE_PREPARE_OK]),  # Binary size OK
+        bytes([espota2.RESPONSE_BIN_MD5_OK]),  # MD5 checksum OK
+        bytes([espota2.RESPONSE_CHUNK_OK]),  # Chunk OK
+        bytes([espota2.RESPONSE_RECEIVE_OK]),  # Receive OK
+        bytes([espota2.RESPONSE_UPDATE_END_OK]),  # Update end OK
+    ]
+
+    mock_socket.recv.side_effect = recv_responses
+
+    # Run OTA - should work even though device requested MD5
+    espota2.perform_ota(mock_socket, "testpass", mock_file, "test.bin")
+
+    # Verify client still advertised SHA256 support
+    assert mock_socket.sendall.call_args_list[1] == call(
+        bytes(
+            [
+                espota2.FEATURE_SUPPORTS_COMPRESSION
+                | espota2.FEATURE_SUPPORTS_SHA256_AUTH
+            ]
+        )
+    )
+
+    # But authentication was done with MD5
+    cnonce = hashlib.md5(MOCK_RANDOM_BYTES).hexdigest()
+    expected_hash = hashlib.md5()
+    expected_hash.update(b"testpass")
+    expected_hash.update(MOCK_MD5_NONCE)
+    expected_hash.update(cnonce.encode())
+    expected_result = expected_hash.hexdigest()
+    assert mock_socket.sendall.call_args_list[3] == call(expected_result.encode())
+
+
+@pytest.mark.usefixtures("mock_time")
+def test_perform_ota_version_differences(
+    mock_socket: Mock, mock_file: io.BytesIO
+) -> None:
+    """Test OTA behavior differences between version 1.0 and 2.0."""
+    # Test version 1.0 - no chunk acknowledgments
+    recv_responses = [
+        bytes([espota2.RESPONSE_OK]),  # First byte of version response
+        bytes([espota2.OTA_VERSION_1_0]),  # Version number
+        bytes([espota2.RESPONSE_HEADER_OK]),  # Features response
+        bytes([espota2.RESPONSE_AUTH_OK]),  # No auth required
+        bytes([espota2.RESPONSE_UPDATE_PREPARE_OK]),  # Binary size OK
+        bytes([espota2.RESPONSE_BIN_MD5_OK]),  # MD5 checksum OK
+        # No RESPONSE_CHUNK_OK for v1
+        bytes([espota2.RESPONSE_RECEIVE_OK]),  # Receive OK
+        bytes([espota2.RESPONSE_UPDATE_END_OK]),  # Update end OK
+    ]
+
+    mock_socket.recv.side_effect = recv_responses
+    espota2.perform_ota(mock_socket, None, mock_file, "test.bin")
+
+    # For v1.0, verify that we only get the expected number of recv calls
+    # v1.0 doesn't have chunk acknowledgments, so fewer recv calls
+    assert mock_socket.recv.call_count == 8  # v1.0 has 8 recv calls
+
+    # Reset mock for v2.0 test
+    mock_socket.reset_mock()
+
+    # Reset file position for second test
+    mock_file.seek(0)
+
+    # Test version 2.0 - with chunk acknowledgments
+    recv_responses_v2 = [
+        bytes([espota2.RESPONSE_OK]),  # First byte of version response
+        bytes([espota2.OTA_VERSION_2_0]),  # Version number
+        bytes([espota2.RESPONSE_HEADER_OK]),  # Features response
+        bytes([espota2.RESPONSE_AUTH_OK]),  # No auth required
+        bytes([espota2.RESPONSE_UPDATE_PREPARE_OK]),  # Binary size OK
+        bytes([espota2.RESPONSE_BIN_MD5_OK]),  # MD5 checksum OK
+        bytes([espota2.RESPONSE_CHUNK_OK]),  # v2.0 has chunk acknowledgment
+        bytes([espota2.RESPONSE_RECEIVE_OK]),  # Receive OK
+        bytes([espota2.RESPONSE_UPDATE_END_OK]),  # Update end OK
+    ]
+
+    mock_socket.recv.side_effect = recv_responses_v2
+    espota2.perform_ota(mock_socket, None, mock_file, "test.bin")
+
+    # For v2.0, verify more recv calls due to chunk acknowledgments
+    assert mock_socket.recv.call_count == 9  # v2.0 has 9 recv calls (includes chunk OK)
diff --git a/tests/unit_tests/test_external_files.py b/tests/unit_tests/test_external_files.py
new file mode 100644
index 0000000000..05e0bd3523
--- /dev/null
+++ b/tests/unit_tests/test_external_files.py
@@ -0,0 +1,200 @@
+"""Tests for external_files.py functions."""
+
+from pathlib import Path
+import time
+from unittest.mock import MagicMock, patch
+
+import pytest
+import requests
+
+from esphome import external_files
+from esphome.config_validation import Invalid
+from esphome.core import CORE, TimePeriod
+
+
+def test_compute_local_file_dir(setup_core: Path) -> None:
+    """Test compute_local_file_dir creates and returns correct path."""
+    domain = "font"
+
+    result = external_files.compute_local_file_dir(domain)
+
+    assert isinstance(result, Path)
+    assert result == Path(CORE.data_dir) / domain
+    assert result.exists()
+    assert result.is_dir()
+
+
+def test_compute_local_file_dir_nested(setup_core: Path) -> None:
+    """Test compute_local_file_dir works with nested domains."""
+    domain = "images/icons"
+
+    result = external_files.compute_local_file_dir(domain)
+
+    assert result == Path(CORE.data_dir) / "images" / "icons"
+    assert result.exists()
+    assert result.is_dir()
+
+
+def test_is_file_recent_with_recent_file(setup_core: Path) -> None:
+    """Test is_file_recent returns True for recently created file."""
+    test_file = setup_core / "recent.txt"
+    test_file.write_text("content")
+
+    refresh = TimePeriod(seconds=3600)
+
+    result = external_files.is_file_recent(test_file, refresh)
+
+    assert result is True
+
+
+def test_is_file_recent_with_old_file(setup_core: Path) -> None:
+    """Test is_file_recent returns False for old file."""
+    test_file = setup_core / "old.txt"
+    test_file.write_text("content")
+
+    old_time = time.time() - 7200
+    mock_stat = MagicMock()
+    mock_stat.st_ctime = old_time
+
+    with patch.object(Path, "stat", return_value=mock_stat):
+        refresh = TimePeriod(seconds=3600)
+
+        result = external_files.is_file_recent(test_file, refresh)
+
+        assert result is False
+
+
+def test_is_file_recent_nonexistent_file(setup_core: Path) -> None:
+    """Test is_file_recent returns False for non-existent file."""
+    test_file = setup_core / "nonexistent.txt"
+    refresh = TimePeriod(seconds=3600)
+
+    result = external_files.is_file_recent(test_file, refresh)
+
+    assert result is False
+
+
+def test_is_file_recent_with_zero_refresh(setup_core: Path) -> None:
+    """Test is_file_recent with zero refresh period returns False."""
+    test_file = setup_core / "test.txt"
+    test_file.write_text("content")
+
+    # Mock stat to return a time 10 seconds ago
+    mock_stat = MagicMock()
+    mock_stat.st_ctime = time.time() - 10
+    with patch.object(Path, "stat", return_value=mock_stat):
+        refresh = TimePeriod(seconds=0)
+        result = external_files.is_file_recent(test_file, refresh)
+        assert result is False
+
+
+@patch("esphome.external_files.requests.head")
+def test_has_remote_file_changed_not_modified(
+    mock_head: MagicMock, setup_core: Path
+) -> None:
+    """Test has_remote_file_changed returns False when file not modified."""
+    test_file = setup_core / "cached.txt"
+    test_file.write_text("cached content")
+
+    mock_response = MagicMock()
+    mock_response.status_code = 304
+    mock_head.return_value = mock_response
+
+    url = "https://example.com/file.txt"
+    result = external_files.has_remote_file_changed(url, test_file)
+
+    assert result is False
+    mock_head.assert_called_once()
+
+    call_args = mock_head.call_args
+    headers = call_args[1]["headers"]
+    assert external_files.IF_MODIFIED_SINCE in headers
+    assert external_files.CACHE_CONTROL in headers
+
+
+@patch("esphome.external_files.requests.head")
+def test_has_remote_file_changed_modified(
+    mock_head: MagicMock, setup_core: Path
+) -> None:
+    """Test has_remote_file_changed returns True when file modified."""
+    test_file = setup_core / "cached.txt"
+    test_file.write_text("cached content")
+
+    mock_response = MagicMock()
+    mock_response.status_code = 200
+    mock_head.return_value = mock_response
+
+    url = "https://example.com/file.txt"
+    result = external_files.has_remote_file_changed(url, test_file)
+
+    assert result is True
+
+
+def test_has_remote_file_changed_no_local_file(setup_core: Path) -> None:
+    """Test has_remote_file_changed returns True when local file doesn't exist."""
+    test_file = setup_core / "nonexistent.txt"
+
+    url = "https://example.com/file.txt"
+    result = external_files.has_remote_file_changed(url, test_file)
+
+    assert result is True
+
+
+@patch("esphome.external_files.requests.head")
+def test_has_remote_file_changed_network_error(
+    mock_head: MagicMock, setup_core: Path
+) -> None:
+    """Test has_remote_file_changed handles network errors gracefully."""
+    test_file = setup_core / "cached.txt"
+    test_file.write_text("cached content")
+
+    mock_head.side_effect = requests.exceptions.RequestException("Network error")
+
+    url = "https://example.com/file.txt"
+
+    with pytest.raises(Invalid, match="Could not check if.*Network error"):
+        external_files.has_remote_file_changed(url, test_file)
+
+
+@patch("esphome.external_files.requests.head")
+def test_has_remote_file_changed_timeout(
+    mock_head: MagicMock, setup_core: Path
+) -> None:
+    """Test has_remote_file_changed respects timeout."""
+    test_file = setup_core / "cached.txt"
+    test_file.write_text("cached content")
+
+    mock_response = MagicMock()
+    mock_response.status_code = 304
+    mock_head.return_value = mock_response
+
+    url = "https://example.com/file.txt"
+    external_files.has_remote_file_changed(url, test_file)
+
+    call_args = mock_head.call_args
+    assert call_args[1]["timeout"] == external_files.NETWORK_TIMEOUT
+
+
+def test_compute_local_file_dir_creates_parent_dirs(setup_core: Path) -> None:
+    """Test compute_local_file_dir creates parent directories."""
+    domain = "level1/level2/level3/level4"
+
+    result = external_files.compute_local_file_dir(domain)
+
+    assert result.exists()
+    assert result.is_dir()
+    assert result.parent.name == "level3"
+    assert result.parent.parent.name == "level2"
+    assert result.parent.parent.parent.name == "level1"
+
+
+def test_is_file_recent_handles_float_seconds(setup_core: Path) -> None:
+    """Test is_file_recent works with float seconds in TimePeriod."""
+    test_file = setup_core / "test.txt"
+    test_file.write_text("content")
+
+    refresh = TimePeriod(seconds=3600.5)
+
+    result = external_files.is_file_recent(test_file, refresh)
+
+    assert result is True
diff --git a/tests/unit_tests/test_git.py b/tests/unit_tests/test_git.py
new file mode 100644
index 0000000000..0411fe5e43
--- /dev/null
+++ b/tests/unit_tests/test_git.py
@@ -0,0 +1,673 @@
+"""Tests for git.py module."""
+
+from datetime import datetime, timedelta
+import os
+from pathlib import Path
+from typing import Any
+from unittest.mock import Mock
+
+import pytest
+
+from esphome import git
+from esphome.core import CORE, TimePeriodSeconds
+from esphome.git import GitCommandError
+
+
+def _compute_repo_dir(url: str, ref: str | None, domain: str) -> Path:
+    """Helper to compute the expected repo directory path using git module's logic."""
+    key = f"{url}@{ref}"
+    return git._compute_destination_path(key, domain)
+
+
+def _setup_old_repo(repo_dir: Path, days_old: int = 2) -> None:
+    """Helper to set up a git repo directory structure with an old timestamp.
+
+    Args:
+        repo_dir: The repository directory path to create.
+        days_old: Number of days old to make the FETCH_HEAD file (default: 2).
+    """
+    # Create repo directory
+    repo_dir.mkdir(parents=True)
+    git_dir = repo_dir / ".git"
+    git_dir.mkdir()
+
+    # Create FETCH_HEAD file with old timestamp
+    fetch_head = git_dir / "FETCH_HEAD"
+    fetch_head.write_text("test")
+    old_time = datetime.now() - timedelta(days=days_old)
+    fetch_head.touch()
+    os.utime(fetch_head, (old_time.timestamp(), old_time.timestamp()))
+
+
+def _get_git_command_type(cmd: list[str]) -> str | None:
+    """Helper to determine the type of git command from a command list.
+
+    Args:
+        cmd: The git command list (e.g., ["git", "rev-parse", "HEAD"]).
+
+    Returns:
+        The command type ("rev-parse", "stash", "fetch", "reset", "clone") or None.
+    """
+    # Git commands are always in format ["git", "command", ...], so check index 1
+    if len(cmd) > 1:
+        return cmd[1]
+    return None
+
+
+def test_run_git_command_success(tmp_path: Path) -> None:
+    """Test that run_git_command returns output on success."""
+    # Create a simple git repo to test with
+    repo_dir = tmp_path / "test_repo"
+    repo_dir.mkdir()
+
+    # Initialize a git repo
+    result = git.run_git_command(["git", "init"], str(repo_dir))
+    assert "Initialized empty Git repository" in result or result == ""
+
+    # Verify we can run a command and get output
+    result = git.run_git_command(["git", "status", "--porcelain"], str(repo_dir))
+    # Empty repo should have empty status
+    assert isinstance(result, str)
+
+
+def test_run_git_command_with_git_dir_isolation(
+    tmp_path: Path, mock_subprocess_run: Mock
+) -> None:
+    """Test that git_dir parameter properly isolates git operations."""
+    repo_dir = tmp_path / "test_repo"
+    repo_dir.mkdir()
+    git_dir = repo_dir / ".git"
+    git_dir.mkdir()
+
+    # Configure mock to return success
+    mock_subprocess_run.return_value = Mock(
+        returncode=0,
+        stdout=b"test output",
+        stderr=b"",
+    )
+
+    result = git.run_git_command(
+        ["git", "rev-parse", "HEAD"],
+        git_dir=repo_dir,
+    )
+
+    # Verify subprocess.run was called
+    assert mock_subprocess_run.called
+    call_args = mock_subprocess_run.call_args
+
+    # Verify environment was set
+    env = call_args[1]["env"]
+    assert "GIT_DIR" in env
+    assert "GIT_WORK_TREE" in env
+    assert env["GIT_DIR"] == str(repo_dir / ".git")
+    assert env["GIT_WORK_TREE"] == str(repo_dir)
+
+    assert result == "test output"
+
+
+def test_run_git_command_raises_git_not_installed_error(
+    tmp_path: Path, mock_subprocess_run: Mock
+) -> None:
+    """Test that FileNotFoundError is converted to GitNotInstalledError."""
+    from esphome.git import GitNotInstalledError
+
+    repo_dir = tmp_path / "test_repo"
+
+    # Configure mock to raise FileNotFoundError
+    mock_subprocess_run.side_effect = FileNotFoundError("git not found")
+
+    with pytest.raises(GitNotInstalledError, match="git is not installed"):
+        git.run_git_command(["git", "status"], git_dir=repo_dir)
+
+
+def test_run_git_command_raises_git_command_error_on_failure(
+    tmp_path: Path, mock_subprocess_run: Mock
+) -> None:
+    """Test that failed git commands raise GitCommandError."""
+    repo_dir = tmp_path / "test_repo"
+
+    # Configure mock to return non-zero exit code
+    mock_subprocess_run.return_value = Mock(
+        returncode=1,
+        stdout=b"",
+        stderr=b"fatal: not a git repository",
+    )
+
+    with pytest.raises(GitCommandError, match="not a git repository"):
+        git.run_git_command(["git", "status"], git_dir=repo_dir)
+
+
+def test_run_git_command_strips_fatal_prefix(
+    tmp_path: Path, mock_subprocess_run: Mock
+) -> None:
+    """Test that 'fatal: ' prefix is stripped from error messages."""
+    repo_dir = tmp_path / "test_repo"
+
+    # Configure mock to return error with "fatal: " prefix
+    mock_subprocess_run.return_value = Mock(
+        returncode=128,
+        stdout=b"",
+        stderr=b"fatal: repository not found\n",
+    )
+
+    with pytest.raises(GitCommandError) as exc_info:
+        git.run_git_command(["git", "clone", "invalid-url"], git_dir=repo_dir)
+
+    # Error message should NOT include "fatal: " prefix
+    assert "fatal:" not in str(exc_info.value)
+    assert "repository not found" in str(exc_info.value)
+
+
+def test_run_git_command_without_git_dir(mock_subprocess_run: Mock) -> None:
+    """Test that run_git_command works without git_dir (clone case)."""
+    # Configure mock to return success
+    mock_subprocess_run.return_value = Mock(
+        returncode=0,
+        stdout=b"Cloning into 'test_repo'...",
+        stderr=b"",
+    )
+
+    result = git.run_git_command(["git", "clone", "https://github.com/test/repo"])
+
+    # Verify subprocess.run was called
+    assert mock_subprocess_run.called
+    call_args = mock_subprocess_run.call_args
+
+    # Verify environment does NOT have GIT_DIR or GIT_WORK_TREE set
+    # (it should use the default environment or None)
+    env = call_args[1].get("env")
+    if env is not None:
+        assert "GIT_DIR" not in env
+        assert "GIT_WORK_TREE" not in env
+
+    # Verify cwd is None (default)
+    assert call_args[1].get("cwd") is None
+
+    assert result == "Cloning into 'test_repo'..."
+
+
+def test_run_git_command_without_git_dir_raises_error(
+    mock_subprocess_run: Mock,
+) -> None:
+    """Test that run_git_command without git_dir can still raise errors."""
+    # Configure mock to return error
+    mock_subprocess_run.return_value = Mock(
+        returncode=128,
+        stdout=b"",
+        stderr=b"fatal: repository not found\n",
+    )
+
+    with pytest.raises(GitCommandError, match="repository not found"):
+        git.run_git_command(["git", "clone", "https://invalid.url/repo.git"])
+
+
+def test_clone_or_update_with_never_refresh(
+    tmp_path: Path, mock_run_git_command: Mock
+) -> None:
+    """Test that NEVER_REFRESH skips updates for existing repos."""
+    # Set up CORE.config_path so data_dir uses tmp_path
+    CORE.config_path = tmp_path / "test.yaml"
+
+    url = "https://github.com/test/repo"
+    ref = None
+    domain = "test"
+    repo_dir = _compute_repo_dir(url, ref, domain)
+
+    # Create the git repo directory structure
+    repo_dir.mkdir(parents=True)
+    git_dir = repo_dir / ".git"
+    git_dir.mkdir()
+
+    # Create FETCH_HEAD file with current timestamp
+    fetch_head = git_dir / "FETCH_HEAD"
+    fetch_head.write_text("test")
+
+    # Call with NEVER_REFRESH
+    result_dir, revert = git.clone_or_update(
+        url=url,
+        ref=ref,
+        refresh=git.NEVER_REFRESH,
+        domain=domain,
+    )
+
+    # Should NOT call git commands since NEVER_REFRESH and repo exists
+    mock_run_git_command.assert_not_called()
+    assert result_dir == repo_dir
+    assert revert is None
+
+
+def test_clone_or_update_with_refresh_updates_old_repo(
+    tmp_path: Path, mock_run_git_command: Mock
+) -> None:
+    """Test that refresh triggers update for old repos."""
+    # Set up CORE.config_path so data_dir uses tmp_path
+    CORE.config_path = tmp_path / "test.yaml"
+
+    url = "https://github.com/test/repo"
+    ref = None
+    domain = "test"
+    repo_dir = _compute_repo_dir(url, ref, domain)
+
+    # Create the git repo directory structure
+    repo_dir.mkdir(parents=True)
+    git_dir = repo_dir / ".git"
+    git_dir.mkdir()
+
+    # Create FETCH_HEAD file with old timestamp (2 days ago)
+    fetch_head = git_dir / "FETCH_HEAD"
+    fetch_head.write_text("test")
+    old_time = datetime.now() - timedelta(days=2)
+    fetch_head.touch()  # Create the file
+    # Set modification time to 2 days ago
+    os.utime(fetch_head, (old_time.timestamp(), old_time.timestamp()))
+
+    # Mock git command responses
+    mock_run_git_command.return_value = "abc123"  # SHA for rev-parse
+
+    # Call with refresh=1d (1 day)
+    refresh = TimePeriodSeconds(days=1)
+    result_dir, revert = git.clone_or_update(
+        url=url,
+        ref=ref,
+        refresh=refresh,
+        domain=domain,
+    )
+
+    # Should call git fetch and update commands since repo is older than refresh
+    assert mock_run_git_command.called
+    # Check for fetch command
+    fetch_calls = [
+        call
+        for call in mock_run_git_command.call_args_list
+        if len(call[0]) > 0 and "fetch" in call[0][0]
+    ]
+    assert len(fetch_calls) > 0
+
+
+def test_clone_or_update_with_refresh_skips_fresh_repo(
+    tmp_path: Path, mock_run_git_command: Mock
+) -> None:
+    """Test that refresh doesn't update fresh repos."""
+    # Set up CORE.config_path so data_dir uses tmp_path
+    CORE.config_path = tmp_path / "test.yaml"
+
+    url = "https://github.com/test/repo"
+    ref = None
+    domain = "test"
+    repo_dir = _compute_repo_dir(url, ref, domain)
+
+    # Create the git repo directory structure
+    repo_dir.mkdir(parents=True)
+    git_dir = repo_dir / ".git"
+    git_dir.mkdir()
+
+    # Create FETCH_HEAD file with recent timestamp (1 hour ago)
+    fetch_head = git_dir / "FETCH_HEAD"
+    fetch_head.write_text("test")
+    recent_time = datetime.now() - timedelta(hours=1)
+    fetch_head.touch()  # Create the file
+    # Set modification time to 1 hour ago
+    os.utime(fetch_head, (recent_time.timestamp(), recent_time.timestamp()))
+
+    # Call with refresh=1d (1 day)
+    refresh = TimePeriodSeconds(days=1)
+    result_dir, revert = git.clone_or_update(
+        url=url,
+        ref=ref,
+        refresh=refresh,
+        domain=domain,
+    )
+
+    # Should NOT call git fetch since repo is fresh
+    mock_run_git_command.assert_not_called()
+    assert result_dir == repo_dir
+    assert revert is None
+
+
+def test_clone_or_update_clones_missing_repo(
+    tmp_path: Path, mock_run_git_command: Mock
+) -> None:
+    """Test that missing repos are cloned regardless of refresh setting."""
+    # Set up CORE.config_path so data_dir uses tmp_path
+    CORE.config_path = tmp_path / "test.yaml"
+
+    url = "https://github.com/test/repo"
+    ref = None
+    domain = "test"
+    repo_dir = _compute_repo_dir(url, ref, domain)
+
+    # Create base directory but NOT the repo itself
+    base_dir = tmp_path / ".esphome" / domain
+    base_dir.mkdir(parents=True)
+    # repo_dir should NOT exist
+    assert not repo_dir.exists()
+
+    # Test with NEVER_REFRESH - should still clone since repo doesn't exist
+    result_dir, revert = git.clone_or_update(
+        url=url,
+        ref=ref,
+        refresh=git.NEVER_REFRESH,
+        domain=domain,
+    )
+
+    # Should call git clone
+    assert mock_run_git_command.called
+    clone_calls = [
+        call
+        for call in mock_run_git_command.call_args_list
+        if len(call[0]) > 0 and "clone" in call[0][0]
+    ]
+    assert len(clone_calls) > 0
+
+
+def test_clone_or_update_with_none_refresh_always_updates(
+    tmp_path: Path, mock_run_git_command: Mock
+) -> None:
+    """Test that refresh=None always updates existing repos."""
+    # Set up CORE.config_path so data_dir uses tmp_path
+    CORE.config_path = tmp_path / "test.yaml"
+
+    url = "https://github.com/test/repo"
+    ref = None
+    domain = "test"
+    repo_dir = _compute_repo_dir(url, ref, domain)
+
+    # Create the git repo directory structure
+    repo_dir.mkdir(parents=True)
+    git_dir = repo_dir / ".git"
+    git_dir.mkdir()
+
+    # Create FETCH_HEAD file with very recent timestamp (1 second ago)
+    fetch_head = git_dir / "FETCH_HEAD"
+    fetch_head.write_text("test")
+    recent_time = datetime.now() - timedelta(seconds=1)
+    fetch_head.touch()  # Create the file
+    # Set modification time to 1 second ago
+    os.utime(fetch_head, (recent_time.timestamp(), recent_time.timestamp()))
+
+    # Mock git command responses
+    mock_run_git_command.return_value = "abc123"  # SHA for rev-parse
+
+    # Call with refresh=None (default behavior)
+    result_dir, revert = git.clone_or_update(
+        url=url,
+        ref=ref,
+        refresh=None,
+        domain=domain,
+    )
+
+    # Should call git fetch and update commands since refresh=None means always update
+    assert mock_run_git_command.called
+    # Check for fetch command
+    fetch_calls = [
+        call
+        for call in mock_run_git_command.call_args_list
+        if len(call[0]) > 0 and "fetch" in call[0][0]
+    ]
+    assert len(fetch_calls) > 0
+
+
+@pytest.mark.parametrize(
+    ("fail_command", "error_message"),
+    [
+        (
+            "rev-parse",
+            "ambiguous argument 'HEAD': unknown revision or path not in the working tree.",
+        ),
+        ("stash", "fatal: unable to write new index file"),
+        (
+            "fetch",
+            "fatal: unable to access 'https://github.com/test/repo/': Could not resolve host",
+        ),
+        ("reset", "fatal: Could not reset index file to revision 'FETCH_HEAD'"),
+    ],
+)
+def test_clone_or_update_recovers_from_git_failures(
+    tmp_path: Path, mock_run_git_command: Mock, fail_command: str, error_message: str
+) -> None:
+    """Test that repos are re-cloned when various git commands fail."""
+    # Set up CORE.config_path so data_dir uses tmp_path
+    CORE.config_path = tmp_path / "test.yaml"
+
+    url = "https://github.com/test/repo"
+    ref = "main"
+    domain = "test"
+    repo_dir = _compute_repo_dir(url, ref, domain)
+
+    # Use helper to set up old repo
+    _setup_old_repo(repo_dir)
+
+    # Track command call counts to make first call fail, subsequent calls succeed
+    call_counts: dict[str, int] = {}
+
+    def git_command_side_effect(
+        cmd: list[str], cwd: str | None = None, **kwargs: Any
+    ) -> str:
+        # Determine which command this is
+        cmd_type = _get_git_command_type(cmd)
+
+        # Track call count for this command type
+        if cmd_type:
+            call_counts[cmd_type] = call_counts.get(cmd_type, 0) + 1
+
+        # Fail on first call to the specified command, succeed on subsequent calls
+        if cmd_type == fail_command and call_counts[cmd_type] == 1:
+            raise GitCommandError(error_message)
+
+        # Default successful responses
+        if cmd_type == "rev-parse":
+            return "abc123"
+        return ""
+
+    mock_run_git_command.side_effect = git_command_side_effect
+
+    refresh = TimePeriodSeconds(days=1)
+    result_dir, revert = git.clone_or_update(
+        url=url,
+        ref=ref,
+        refresh=refresh,
+        domain=domain,
+    )
+
+    # Verify recovery happened
+    call_list = mock_run_git_command.call_args_list
+
+    # Should have attempted the failing command
+    assert any(fail_command in str(c) for c in call_list)
+
+    # Should have called clone for recovery
+    assert any("clone" in str(c) for c in call_list)
+
+    # Verify the repo directory path is returned
+    assert result_dir == repo_dir
+
+
+def test_clone_or_update_fails_when_recovery_also_fails(
+    tmp_path: Path, mock_run_git_command: Mock
+) -> None:
+    """Test that we don't infinitely recurse when recovery also fails."""
+    # Set up CORE.config_path so data_dir uses tmp_path
+    CORE.config_path = tmp_path / "test.yaml"
+
+    url = "https://github.com/test/repo"
+    ref = "main"
+    domain = "test"
+    repo_dir = _compute_repo_dir(url, ref, domain)
+
+    # Use helper to set up old repo
+    _setup_old_repo(repo_dir)
+
+    # Mock git command to fail on clone (simulating network failure during recovery)
+    def git_command_side_effect(
+        cmd: list[str], cwd: str | None = None, **kwargs: Any
+    ) -> str:
+        cmd_type = _get_git_command_type(cmd)
+        if cmd_type == "rev-parse":
+            # First time fails (broken repo)
+            raise GitCommandError(
+                "ambiguous argument 'HEAD': unknown revision or path not in the working tree."
+            )
+        if cmd_type == "clone":
+            # Clone also fails (recovery fails)
+            raise GitCommandError("fatal: unable to access repository")
+        return ""
+
+    mock_run_git_command.side_effect = git_command_side_effect
+
+    refresh = TimePeriodSeconds(days=1)
+
+    # Should raise after one recovery attempt fails
+    with pytest.raises(GitCommandError, match="fatal: unable to access repository"):
+        git.clone_or_update(
+            url=url,
+            ref=ref,
+            refresh=refresh,
+            domain=domain,
+        )
+
+    # Verify we only tried to clone once (no infinite recursion)
+    call_list = mock_run_git_command.call_args_list
+    clone_calls = [c for c in call_list if "clone" in c[0][0]]
+    # Should have exactly one clone call (the recovery attempt that failed)
+    assert len(clone_calls) == 1
+    # Should have tried rev-parse once (which failed and triggered recovery)
+    rev_parse_calls = [c for c in call_list if "rev-parse" in c[0][0]]
+    assert len(rev_parse_calls) == 1
+
+
+def test_clone_or_update_recover_broken_flag_prevents_second_recovery(
+    tmp_path: Path, mock_run_git_command: Mock
+) -> None:
+    """Test that _recover_broken=False prevents a second recovery attempt (tests the raise path)."""
+    # Set up CORE.config_path so data_dir uses tmp_path
+    CORE.config_path = tmp_path / "test.yaml"
+
+    url = "https://github.com/test/repo"
+    ref = "main"
+    domain = "test"
+    repo_dir = _compute_repo_dir(url, ref, domain)
+
+    # Use helper to set up old repo
+    _setup_old_repo(repo_dir)
+
+    # Track fetch calls to differentiate between first (in clone) and second (in recovery update)
+    call_counts: dict[str, int] = {}
+
+    # Mock git command to fail on fetch during recovery's ref checkout
+    def git_command_side_effect(
+        cmd: list[str], cwd: str | None = None, **kwargs: Any
+    ) -> str:
+        cmd_type = _get_git_command_type(cmd)
+
+        if cmd_type:
+            call_counts[cmd_type] = call_counts.get(cmd_type, 0) + 1
+
+        # First attempt: rev-parse fails (broken repo)
+        if cmd_type == "rev-parse" and call_counts[cmd_type] == 1:
+            raise GitCommandError(
+                "ambiguous argument 'HEAD': unknown revision or path not in the working tree."
+            )
+
+        # Recovery: clone succeeds
+        if cmd_type == "clone":
+            return ""
+
+        # Recovery: fetch for ref checkout fails
+        # This happens in the clone path when ref is not None (line 80 in git.py)
+        if cmd_type == "fetch" and call_counts[cmd_type] == 1:
+            raise GitCommandError("fatal: couldn't find remote ref main")
+
+        # Default success
+        return "abc123" if cmd_type == "rev-parse" else ""
+
+    mock_run_git_command.side_effect = git_command_side_effect
+
+    refresh = TimePeriodSeconds(days=1)
+
+    # Should raise on the fetch during recovery (when _recover_broken=False)
+    # This tests the critical "if not _recover_broken: raise" path
+    with pytest.raises(GitCommandError, match="fatal: couldn't find remote ref main"):
+        git.clone_or_update(
+            url=url,
+            ref=ref,
+            refresh=refresh,
+            domain=domain,
+        )
+
+    # Verify the sequence of events
+    call_list = mock_run_git_command.call_args_list
+
+    # Should have: rev-parse (fail, triggers recovery), clone (success),
+    # fetch (fail during ref checkout, raises because _recover_broken=False)
+    rev_parse_calls = [c for c in call_list if "rev-parse" in c[0][0]]
+    # Should have exactly one rev-parse call that failed
+    assert len(rev_parse_calls) == 1
+
+    clone_calls = [c for c in call_list if "clone" in c[0][0]]
+    # Should have exactly one clone call (the recovery attempt)
+    assert len(clone_calls) == 1
+
+    fetch_calls = [c for c in call_list if "fetch" in c[0][0]]
+    # Should have exactly one fetch call that failed (during ref checkout in recovery)
+    assert len(fetch_calls) == 1
+
+
+def test_clone_or_update_recover_broken_flag_prevents_infinite_loop(
+    tmp_path: Path, mock_run_git_command: Mock
+) -> None:
+    """Test that _recover_broken=False prevents infinite recursion when repo persists."""
+    # This tests the critical "if not _recover_broken: raise" path at line 124-125
+    # Set up CORE.config_path so data_dir uses tmp_path
+    CORE.config_path = tmp_path / "test.yaml"
+
+    url = "https://github.com/test/repo"
+    ref = "main"
+    domain = "test"
+    repo_dir = _compute_repo_dir(url, ref, domain)
+
+    # Use helper to set up old repo
+    _setup_old_repo(repo_dir)
+
+    # Mock shutil.rmtree to NOT actually delete the directory
+    # This simulates a scenario where deletion fails (permissions, etc.)
+    import unittest.mock
+
+    def mock_rmtree(path, *args, **kwargs):
+        # Don't actually delete - this causes the recursive call to still see the repo
+        pass
+
+    # Mock git commands to always fail on stash
+    def git_command_side_effect(
+        cmd: list[str], cwd: str | None = None, **kwargs: Any
+    ) -> str:
+        cmd_type = _get_git_command_type(cmd)
+        if cmd_type == "rev-parse":
+            return "abc123"
+        if cmd_type == "stash":
+            # Always fails
+            raise GitCommandError("fatal: unable to write new index file")
+        return ""
+
+    mock_run_git_command.side_effect = git_command_side_effect
+
+    refresh = TimePeriodSeconds(days=1)
+
+    # Mock shutil.rmtree and test
+    # Should raise on the second attempt when _recover_broken=False
+    # This hits the "if not _recover_broken: raise" path
+    with (
+        unittest.mock.patch("esphome.git.shutil.rmtree", side_effect=mock_rmtree),
+        pytest.raises(GitCommandError, match="fatal: unable to write new index file"),
+    ):
+        git.clone_or_update(
+            url=url,
+            ref=ref,
+            refresh=refresh,
+            domain=domain,
+        )
+
+    # Verify the sequence: stash fails twice (once triggering recovery, once raising)
+    call_list = mock_run_git_command.call_args_list
+    stash_calls = [c for c in call_list if "stash" in c[0][0]]
+    # Should have exactly two stash calls
+    assert len(stash_calls) == 2
diff --git a/tests/unit_tests/test_helpers.py b/tests/unit_tests/test_helpers.py
index b353d1aa99..87ed901ecb 100644
--- a/tests/unit_tests/test_helpers.py
+++ b/tests/unit_tests/test_helpers.py
@@ -1,8 +1,18 @@
+import logging
+import os
+from pathlib import Path
+import socket
+import stat
+from unittest.mock import patch
+
+from aioesphomeapi.host_resolver import AddrInfo, IPv4Sockaddr, IPv6Sockaddr
 from hypothesis import given
 from hypothesis.strategies import ip_addresses
 import pytest
 
 from esphome import helpers
+from esphome.address_cache import AddressCache
+from esphome.core import EsphomeError
 
 
 @pytest.mark.parametrize(
@@ -144,11 +154,11 @@ def test_walk_files(fixture_path):
     actual = list(helpers.walk_files(path))
 
     # Ensure paths start with the root
-    assert all(p.startswith(str(path)) for p in actual)
+    assert all(p.is_relative_to(path) for p in actual)
 
 
 class Test_write_file_if_changed:
-    def test_src_and_dst_match(self, tmp_path):
+    def test_src_and_dst_match(self, tmp_path: Path):
         text = "A files are unique.\n"
         initial = text
         dst = tmp_path / "file-a.txt"
@@ -158,7 +168,7 @@ class Test_write_file_if_changed:
 
         assert dst.read_text() == text
 
-    def test_src_and_dst_do_not_match(self, tmp_path):
+    def test_src_and_dst_do_not_match(self, tmp_path: Path):
         text = "A files are unique.\n"
         initial = "B files are unique.\n"
         dst = tmp_path / "file-a.txt"
@@ -168,7 +178,7 @@ class Test_write_file_if_changed:
 
         assert dst.read_text() == text
 
-    def test_dst_does_not_exist(self, tmp_path):
+    def test_dst_does_not_exist(self, tmp_path: Path):
         text = "A files are unique.\n"
         dst = tmp_path / "file-a.txt"
 
@@ -178,7 +188,7 @@ class Test_write_file_if_changed:
 
 
 class Test_copy_file_if_changed:
-    def test_src_and_dst_match(self, tmp_path, fixture_path):
+    def test_src_and_dst_match(self, tmp_path: Path, fixture_path: Path):
         src = fixture_path / "helpers" / "file-a.txt"
         initial = fixture_path / "helpers" / "file-a.txt"
         dst = tmp_path / "file-a.txt"
@@ -187,7 +197,7 @@ class Test_copy_file_if_changed:
 
         helpers.copy_file_if_changed(src, dst)
 
-    def test_src_and_dst_do_not_match(self, tmp_path, fixture_path):
+    def test_src_and_dst_do_not_match(self, tmp_path: Path, fixture_path: Path):
         src = fixture_path / "helpers" / "file-a.txt"
         initial = fixture_path / "helpers" / "file-c.txt"
         dst = tmp_path / "file-a.txt"
@@ -198,7 +208,7 @@ class Test_copy_file_if_changed:
 
         assert src.read_text() == dst.read_text()
 
-    def test_dst_does_not_exist(self, tmp_path, fixture_path):
+    def test_dst_does_not_exist(self, tmp_path: Path, fixture_path: Path):
         src = fixture_path / "helpers" / "file-a.txt"
         dst = tmp_path / "file-a.txt"
 
@@ -277,3 +287,606 @@ def test_sort_ip_addresses(text: list[str], expected: list[str]) -> None:
     actual = helpers.sort_ip_addresses(text)
 
     assert actual == expected
+
+
+# DNS resolution tests
+def test_is_ip_address_ipv4() -> None:
+    """Test is_ip_address with IPv4 addresses."""
+    assert helpers.is_ip_address("192.168.1.1") is True
+    assert helpers.is_ip_address("127.0.0.1") is True
+    assert helpers.is_ip_address("255.255.255.255") is True
+    assert helpers.is_ip_address("0.0.0.0") is True
+
+
+def test_is_ip_address_ipv6() -> None:
+    """Test is_ip_address with IPv6 addresses."""
+    assert helpers.is_ip_address("::1") is True
+    assert helpers.is_ip_address("2001:db8::1") is True
+    assert helpers.is_ip_address("fe80::1") is True
+    assert helpers.is_ip_address("::") is True
+
+
+def test_is_ip_address_invalid() -> None:
+    """Test is_ip_address with non-IP strings."""
+    assert helpers.is_ip_address("hostname") is False
+    assert helpers.is_ip_address("hostname.local") is False
+    assert helpers.is_ip_address("256.256.256.256") is False
+    assert helpers.is_ip_address("192.168.1") is False
+    assert helpers.is_ip_address("") is False
+
+
+def test_resolve_ip_address_single_ipv4() -> None:
+    """Test resolving a single IPv4 address (fast path)."""
+    result = helpers.resolve_ip_address("192.168.1.100", 6053)
+
+    assert len(result) == 1
+    assert result[0][0] == socket.AF_INET  # family
+    assert result[0][1] in (
+        0,
+        socket.SOCK_STREAM,
+    )  # type (0 on Windows with AI_NUMERICHOST)
+    assert result[0][2] in (
+        0,
+        socket.IPPROTO_TCP,
+    )  # proto (0 on Windows with AI_NUMERICHOST)
+    assert result[0][3] == ""  # canonname
+    assert result[0][4] == ("192.168.1.100", 6053)  # sockaddr
+
+
+def test_resolve_ip_address_single_ipv6() -> None:
+    """Test resolving a single IPv6 address (fast path)."""
+    result = helpers.resolve_ip_address("::1", 6053)
+
+    assert len(result) == 1
+    assert result[0][0] == socket.AF_INET6  # family
+    assert result[0][1] in (
+        0,
+        socket.SOCK_STREAM,
+    )  # type (0 on Windows with AI_NUMERICHOST)
+    assert result[0][2] in (
+        0,
+        socket.IPPROTO_TCP,
+    )  # proto (0 on Windows with AI_NUMERICHOST)
+    assert result[0][3] == ""  # canonname
+    # IPv6 sockaddr has 4 elements
+    assert len(result[0][4]) == 4
+    assert result[0][4][0] == "::1"  # address
+    assert result[0][4][1] == 6053  # port
+
+
+def test_resolve_ip_address_list_of_ips() -> None:
+    """Test resolving a list of IP addresses (fast path)."""
+    ips = ["192.168.1.100", "10.0.0.1", "::1"]
+    result = helpers.resolve_ip_address(ips, 6053)
+
+    # Should return results sorted by preference (IPv6 first, then IPv4)
+    assert len(result) >= 2  # At least IPv4 addresses should work
+
+    # Check that results are properly formatted
+    for addr_info in result:
+        assert addr_info[0] in (socket.AF_INET, socket.AF_INET6)
+        assert addr_info[1] in (
+            0,
+            socket.SOCK_STREAM,
+        )  # 0 on Windows with AI_NUMERICHOST
+        assert addr_info[2] in (
+            0,
+            socket.IPPROTO_TCP,
+        )  # 0 on Windows with AI_NUMERICHOST
+        assert addr_info[3] == ""
+
+
+def test_resolve_ip_address_with_getaddrinfo_failure(caplog) -> None:
+    """Test that getaddrinfo OSError is handled gracefully in fast path."""
+    with (
+        caplog.at_level(logging.DEBUG),
+        patch("socket.getaddrinfo") as mock_getaddrinfo,
+    ):
+        # First IP succeeds
+        mock_getaddrinfo.side_effect = [
+            [
+                (
+                    socket.AF_INET,
+                    socket.SOCK_STREAM,
+                    socket.IPPROTO_TCP,
+                    "",
+                    ("192.168.1.100", 6053),
+                )
+            ],
+            OSError("Failed to resolve"),  # Second IP fails
+        ]
+
+        # Should continue despite one failure
+        result = helpers.resolve_ip_address(["192.168.1.100", "192.168.1.101"], 6053)
+
+        # Should have result from first IP only
+        assert len(result) == 1
+        assert result[0][4][0] == "192.168.1.100"
+
+        # Verify both IPs were attempted
+        assert mock_getaddrinfo.call_count == 2
+        mock_getaddrinfo.assert_any_call(
+            "192.168.1.100", 6053, proto=socket.IPPROTO_TCP, flags=socket.AI_NUMERICHOST
+        )
+        mock_getaddrinfo.assert_any_call(
+            "192.168.1.101", 6053, proto=socket.IPPROTO_TCP, flags=socket.AI_NUMERICHOST
+        )
+
+        # Verify the debug log was called for the failed IP
+        assert "Failed to parse IP address '192.168.1.101'" in caplog.text
+
+
+def test_resolve_ip_address_hostname() -> None:
+    """Test resolving a hostname (async resolver path)."""
+    mock_addr_info = AddrInfo(
+        family=socket.AF_INET,
+        type=socket.SOCK_STREAM,
+        proto=socket.IPPROTO_TCP,
+        sockaddr=IPv4Sockaddr(address="192.168.1.100", port=6053),
+    )
+
+    with patch("esphome.resolver.AsyncResolver") as MockResolver:
+        mock_resolver = MockResolver.return_value
+        mock_resolver.resolve.return_value = [mock_addr_info]
+
+        result = helpers.resolve_ip_address("test.local", 6053)
+
+        assert len(result) == 1
+        assert result[0][0] == socket.AF_INET
+        assert result[0][4] == ("192.168.1.100", 6053)
+        MockResolver.assert_called_once_with(["test.local"], 6053)
+        mock_resolver.resolve.assert_called_once()
+
+
+def test_resolve_ip_address_mixed_list() -> None:
+    """Test resolving a mix of IPs and hostnames."""
+    mock_addr_info = AddrInfo(
+        family=socket.AF_INET,
+        type=socket.SOCK_STREAM,
+        proto=socket.IPPROTO_TCP,
+        sockaddr=IPv4Sockaddr(address="192.168.1.200", port=6053),
+    )
+
+    with patch("esphome.resolver.AsyncResolver") as MockResolver:
+        mock_resolver = MockResolver.return_value
+        mock_resolver.resolve.return_value = [mock_addr_info]
+
+        # Mix of IP and hostname - should use async resolver
+        result = helpers.resolve_ip_address(["192.168.1.100", "test.local"], 6053)
+
+        assert len(result) == 1
+        assert result[0][4][0] == "192.168.1.200"
+        MockResolver.assert_called_once_with(["192.168.1.100", "test.local"], 6053)
+        mock_resolver.resolve.assert_called_once()
+
+
+def test_resolve_ip_address_url() -> None:
+    """Test extracting hostname from URL."""
+    mock_addr_info = AddrInfo(
+        family=socket.AF_INET,
+        type=socket.SOCK_STREAM,
+        proto=socket.IPPROTO_TCP,
+        sockaddr=IPv4Sockaddr(address="192.168.1.100", port=6053),
+    )
+
+    with patch("esphome.resolver.AsyncResolver") as MockResolver:
+        mock_resolver = MockResolver.return_value
+        mock_resolver.resolve.return_value = [mock_addr_info]
+
+        result = helpers.resolve_ip_address("http://test.local", 6053)
+
+        assert len(result) == 1
+        MockResolver.assert_called_once_with(["test.local"], 6053)
+        mock_resolver.resolve.assert_called_once()
+
+
+def test_resolve_ip_address_ipv6_conversion() -> None:
+    """Test proper IPv6 address info conversion."""
+    mock_addr_info = AddrInfo(
+        family=socket.AF_INET6,
+        type=socket.SOCK_STREAM,
+        proto=socket.IPPROTO_TCP,
+        sockaddr=IPv6Sockaddr(address="2001:db8::1", port=6053, flowinfo=1, scope_id=2),
+    )
+
+    with patch("esphome.resolver.AsyncResolver") as MockResolver:
+        mock_resolver = MockResolver.return_value
+        mock_resolver.resolve.return_value = [mock_addr_info]
+
+        result = helpers.resolve_ip_address("test.local", 6053)
+
+        assert len(result) == 1
+        assert result[0][0] == socket.AF_INET6
+        assert result[0][4] == ("2001:db8::1", 6053, 1, 2)
+
+
+def test_resolve_ip_address_error_handling() -> None:
+    """Test error handling from AsyncResolver."""
+    with patch("esphome.resolver.AsyncResolver") as MockResolver:
+        mock_resolver = MockResolver.return_value
+        mock_resolver.resolve.side_effect = EsphomeError("Resolution failed")
+
+        with pytest.raises(EsphomeError, match="Resolution failed"):
+            helpers.resolve_ip_address("test.local", 6053)
+
+
+def test_addr_preference_ipv4() -> None:
+    """Test address preference for IPv4."""
+    addr_info = (
+        socket.AF_INET,
+        socket.SOCK_STREAM,
+        socket.IPPROTO_TCP,
+        "",
+        ("192.168.1.1", 6053),
+    )
+    assert helpers.addr_preference_(addr_info) == 2
+
+
+def test_addr_preference_ipv6() -> None:
+    """Test address preference for regular IPv6."""
+    addr_info = (
+        socket.AF_INET6,
+        socket.SOCK_STREAM,
+        socket.IPPROTO_TCP,
+        "",
+        ("2001:db8::1", 6053, 0, 0),
+    )
+    assert helpers.addr_preference_(addr_info) == 1
+
+
+def test_addr_preference_ipv6_link_local_no_scope() -> None:
+    """Test address preference for link-local IPv6 without scope."""
+    addr_info = (
+        socket.AF_INET6,
+        socket.SOCK_STREAM,
+        socket.IPPROTO_TCP,
+        "",
+        ("fe80::1", 6053, 0, 0),  # link-local with scope_id=0
+    )
+    assert helpers.addr_preference_(addr_info) == 3
+
+
+def test_addr_preference_ipv6_link_local_with_scope() -> None:
+    """Test address preference for link-local IPv6 with scope."""
+    addr_info = (
+        socket.AF_INET6,
+        socket.SOCK_STREAM,
+        socket.IPPROTO_TCP,
+        "",
+        ("fe80::1", 6053, 0, 2),  # link-local with scope_id=2
+    )
+    assert helpers.addr_preference_(addr_info) == 1  # Has scope, so it's usable
+
+
+def test_mkdir_p(tmp_path: Path) -> None:
+    """Test mkdir_p creates directories recursively."""
+    # Test creating nested directories
+    nested_path = tmp_path / "level1" / "level2" / "level3"
+    helpers.mkdir_p(nested_path)
+    assert nested_path.exists()
+    assert nested_path.is_dir()
+
+    # Test that mkdir_p is idempotent (doesn't fail if directory exists)
+    helpers.mkdir_p(nested_path)
+    assert nested_path.exists()
+
+    # Test with empty path (should do nothing)
+    helpers.mkdir_p("")
+
+    # Test with existing directory
+    existing_dir = tmp_path / "existing"
+    existing_dir.mkdir()
+    helpers.mkdir_p(existing_dir)
+    assert existing_dir.exists()
+
+
+def test_mkdir_p_file_exists_error(tmp_path: Path) -> None:
+    """Test mkdir_p raises error when path is a file."""
+    # Create a file
+    file_path = tmp_path / "test_file.txt"
+    file_path.write_text("test content")
+
+    # Try to create directory with same name as existing file
+    with pytest.raises(EsphomeError, match=r"Error creating directories"):
+        helpers.mkdir_p(file_path)
+
+
+def test_mkdir_p_with_existing_file_raises_error(tmp_path: Path) -> None:
+    """Test mkdir_p raises error when trying to create dir over existing file."""
+    # Create a file where we want to create a directory
+    file_path = tmp_path / "existing_file"
+    file_path.write_text("content")
+
+    # Try to create a directory with a path that goes through the file
+    dir_path = file_path / "subdir"
+
+    with pytest.raises(EsphomeError, match=r"Error creating directories"):
+        helpers.mkdir_p(dir_path)
+
+
+def test_read_file(tmp_path: Path) -> None:
+    """Test read_file reads file content correctly."""
+    # Test reading regular file
+    test_file = tmp_path / "test.txt"
+    expected_content = "Test content\nLine 2\n"
+    test_file.write_text(expected_content)
+
+    content = helpers.read_file(test_file)
+    assert content == expected_content
+
+    # Test reading file with UTF-8 characters
+    utf8_file = tmp_path / "utf8.txt"
+    utf8_content = "Hello 世界 🌍"
+    utf8_file.write_text(utf8_content, encoding="utf-8")
+
+    content = helpers.read_file(utf8_file)
+    assert content == utf8_content
+
+
+def test_read_file_not_found() -> None:
+    """Test read_file raises error for non-existent file."""
+    with pytest.raises(EsphomeError, match=r"Error reading file"):
+        helpers.read_file(Path("/nonexistent/file.txt"))
+
+
+def test_read_file_unicode_decode_error(tmp_path: Path) -> None:
+    """Test read_file raises error for invalid UTF-8."""
+    test_file = tmp_path / "invalid.txt"
+    # Write invalid UTF-8 bytes
+    test_file.write_bytes(b"\xff\xfe")
+
+    with pytest.raises(EsphomeError, match=r"Error reading file"):
+        helpers.read_file(test_file)
+
+
+@pytest.mark.skipif(os.name == "nt", reason="Unix-specific test")
+def test_write_file_unix(tmp_path: Path) -> None:
+    """Test write_file writes content correctly on Unix."""
+    # Test writing string content
+    test_file = tmp_path / "test.txt"
+    content = "Test content\nLine 2"
+    helpers.write_file(test_file, content)
+
+    assert test_file.read_text() == content
+    # Check file permissions
+    assert oct(test_file.stat().st_mode)[-3:] == "644"
+
+    # Test overwriting existing file
+    new_content = "New content"
+    helpers.write_file(test_file, new_content)
+    assert test_file.read_text() == new_content
+
+    # Test writing to nested directories (should create them)
+    nested_file = tmp_path / "dir1" / "dir2" / "file.txt"
+    helpers.write_file(nested_file, content)
+    assert nested_file.read_text() == content
+
+
+@pytest.mark.skipif(os.name != "nt", reason="Windows-specific test")
+def test_write_file_windows(tmp_path: Path) -> None:
+    """Test write_file writes content correctly on Windows."""
+    # Test writing string content
+    test_file = tmp_path / "test.txt"
+    content = "Test content\nLine 2"
+    helpers.write_file(test_file, content)
+
+    assert test_file.read_text() == content
+    # Windows doesn't have Unix-style 644 permissions
+
+    # Test overwriting existing file
+    new_content = "New content"
+    helpers.write_file(test_file, new_content)
+    assert test_file.read_text() == new_content
+
+    # Test writing to nested directories (should create them)
+    nested_file = tmp_path / "dir1" / "dir2" / "file.txt"
+    helpers.write_file(nested_file, content)
+    assert nested_file.read_text() == content
+
+
+@pytest.mark.skipif(os.name == "nt", reason="Unix-specific permission test")
+def test_write_file_to_non_writable_directory_unix(tmp_path: Path) -> None:
+    """Test write_file raises error when directory is not writable on Unix."""
+    # Create a directory and make it read-only
+    read_only_dir = tmp_path / "readonly"
+    read_only_dir.mkdir()
+    test_file = read_only_dir / "test.txt"
+
+    # Make directory read-only (no write permission)
+    read_only_dir.chmod(0o555)
+
+    try:
+        with pytest.raises(EsphomeError, match=r"Could not write file"):
+            helpers.write_file(test_file, "content")
+    finally:
+        # Restore write permissions for cleanup
+        read_only_dir.chmod(0o755)
+
+
+@pytest.mark.skipif(os.name != "nt", reason="Windows-specific test")
+def test_write_file_to_non_writable_directory_windows(tmp_path: Path) -> None:
+    """Test write_file error handling on Windows."""
+    # Windows handles permissions differently - test a different error case
+    # Try to write to a file path that contains an existing file as a directory component
+    existing_file = tmp_path / "file.txt"
+    existing_file.write_text("content")
+
+    # Try to write to a path that treats the file as a directory
+    invalid_path = existing_file / "subdir" / "test.txt"
+
+    with pytest.raises(EsphomeError, match=r"Could not write file"):
+        helpers.write_file(invalid_path, "content")
+
+
+@pytest.mark.skipif(os.name == "nt", reason="Unix-specific permission test")
+def test_write_file_with_permission_bits_unix(tmp_path: Path) -> None:
+    """Test that write_file sets correct permissions on Unix."""
+    test_file = tmp_path / "test.txt"
+    helpers.write_file(test_file, "content")
+
+    # Check that file has 644 permissions
+    file_mode = test_file.stat().st_mode
+    assert stat.S_IMODE(file_mode) == 0o644
+
+
+@pytest.mark.skipif(os.name == "nt", reason="Unix-specific permission test")
+def test_copy_file_if_changed_permission_recovery_unix(tmp_path: Path) -> None:
+    """Test copy_file_if_changed handles permission errors correctly on Unix."""
+    # Test with read-only destination file
+    src = tmp_path / "source.txt"
+    dst = tmp_path / "dest.txt"
+    src.write_text("new content")
+    dst.write_text("old content")
+    dst.chmod(0o444)  # Make destination read-only
+
+    try:
+        # Should handle permission error by deleting and retrying
+        helpers.copy_file_if_changed(src, dst)
+        assert dst.read_text() == "new content"
+    finally:
+        # Restore write permissions for cleanup
+        if dst.exists():
+            dst.chmod(0o644)
+
+
+def test_copy_file_if_changed_creates_directories(tmp_path: Path) -> None:
+    """Test copy_file_if_changed creates missing directories."""
+    src = tmp_path / "source.txt"
+    dst = tmp_path / "subdir" / "nested" / "dest.txt"
+    src.write_text("content")
+
+    helpers.copy_file_if_changed(src, dst)
+    assert dst.exists()
+    assert dst.read_text() == "content"
+
+
+def test_copy_file_if_changed_nonexistent_source(tmp_path: Path) -> None:
+    """Test copy_file_if_changed with non-existent source."""
+    src = tmp_path / "nonexistent.txt"
+    dst = tmp_path / "dest.txt"
+
+    with pytest.raises(EsphomeError, match=r"Error copying file"):
+        helpers.copy_file_if_changed(src, dst)
+
+
+def test_resolve_ip_address_sorting() -> None:
+    """Test that results are sorted by preference."""
+    # Create multiple address infos with different preferences
+    mock_addr_infos = [
+        AddrInfo(
+            family=socket.AF_INET6,
+            type=socket.SOCK_STREAM,
+            proto=socket.IPPROTO_TCP,
+            sockaddr=IPv6Sockaddr(
+                address="fe80::1", port=6053, flowinfo=0, scope_id=0
+            ),  # Preference 3 (link-local no scope)
+        ),
+        AddrInfo(
+            family=socket.AF_INET,
+            type=socket.SOCK_STREAM,
+            proto=socket.IPPROTO_TCP,
+            sockaddr=IPv4Sockaddr(
+                address="192.168.1.100", port=6053
+            ),  # Preference 2 (IPv4)
+        ),
+        AddrInfo(
+            family=socket.AF_INET6,
+            type=socket.SOCK_STREAM,
+            proto=socket.IPPROTO_TCP,
+            sockaddr=IPv6Sockaddr(
+                address="2001:db8::1", port=6053, flowinfo=0, scope_id=0
+            ),  # Preference 1 (IPv6)
+        ),
+    ]
+
+    with patch("esphome.resolver.AsyncResolver") as MockResolver:
+        mock_resolver = MockResolver.return_value
+        mock_resolver.resolve.return_value = mock_addr_infos
+
+        result = helpers.resolve_ip_address("test.local", 6053)
+
+        # Should be sorted: IPv6 first, then IPv4, then link-local without scope
+        assert result[0][4][0] == "2001:db8::1"  # IPv6 (preference 1)
+        assert result[1][4][0] == "192.168.1.100"  # IPv4 (preference 2)
+        assert result[2][4][0] == "fe80::1"  # Link-local no scope (preference 3)
+
+
+def test_resolve_ip_address_with_cache() -> None:
+    """Test that the cache is used when provided."""
+    cache = AddressCache(
+        mdns_cache={"test.local": ["192.168.1.100", "192.168.1.101"]},
+        dns_cache={
+            "example.com": ["93.184.216.34", "2606:2800:220:1:248:1893:25c8:1946"]
+        },
+    )
+
+    # Test mDNS cache hit
+    result = helpers.resolve_ip_address("test.local", 6053, address_cache=cache)
+
+    # Should return cached addresses without calling resolver
+    assert len(result) == 2
+    assert result[0][4][0] == "192.168.1.100"
+    assert result[1][4][0] == "192.168.1.101"
+
+    # Test DNS cache hit
+    result = helpers.resolve_ip_address("example.com", 6053, address_cache=cache)
+
+    # Should return cached addresses with IPv6 first due to preference
+    assert len(result) == 2
+    assert result[0][4][0] == "2606:2800:220:1:248:1893:25c8:1946"  # IPv6 first
+    assert result[1][4][0] == "93.184.216.34"  # IPv4 second
+
+
+def test_resolve_ip_address_cache_miss() -> None:
+    """Test that resolver is called when not in cache."""
+    cache = AddressCache(mdns_cache={"other.local": ["192.168.1.200"]})
+
+    mock_addr_info = AddrInfo(
+        family=socket.AF_INET,
+        type=socket.SOCK_STREAM,
+        proto=socket.IPPROTO_TCP,
+        sockaddr=IPv4Sockaddr(address="192.168.1.100", port=6053),
+    )
+
+    with patch("esphome.resolver.AsyncResolver") as MockResolver:
+        mock_resolver = MockResolver.return_value
+        mock_resolver.resolve.return_value = [mock_addr_info]
+
+        result = helpers.resolve_ip_address("test.local", 6053, address_cache=cache)
+
+        # Should call resolver since test.local is not in cache
+        MockResolver.assert_called_once_with(["test.local"], 6053)
+        assert len(result) == 1
+        assert result[0][4][0] == "192.168.1.100"
+
+
+def test_resolve_ip_address_mixed_cached_uncached() -> None:
+    """Test resolution with mix of cached and uncached hosts."""
+    cache = AddressCache(mdns_cache={"cached.local": ["192.168.1.50"]})
+
+    mock_addr_info = AddrInfo(
+        family=socket.AF_INET,
+        type=socket.SOCK_STREAM,
+        proto=socket.IPPROTO_TCP,
+        sockaddr=IPv4Sockaddr(address="192.168.1.100", port=6053),
+    )
+
+    with patch("esphome.resolver.AsyncResolver") as MockResolver:
+        mock_resolver = MockResolver.return_value
+        mock_resolver.resolve.return_value = [mock_addr_info]
+
+        # Pass a list with cached IP, cached hostname, and uncached hostname
+        result = helpers.resolve_ip_address(
+            ["192.168.1.10", "cached.local", "uncached.local"],
+            6053,
+            address_cache=cache,
+        )
+
+        # Should only resolve uncached.local
+        MockResolver.assert_called_once_with(["uncached.local"], 6053)
+
+        # Results should include all addresses
+        addresses = [r[4][0] for r in result]
+        assert "192.168.1.10" in addresses  # Direct IP
+        assert "192.168.1.50" in addresses  # From cache
+        assert "192.168.1.100" in addresses  # From resolver
diff --git a/tests/unit_tests/test_log.py b/tests/unit_tests/test_log.py
new file mode 100644
index 0000000000..02798f1029
--- /dev/null
+++ b/tests/unit_tests/test_log.py
@@ -0,0 +1,80 @@
+import pytest
+
+from esphome.log import AnsiFore, AnsiStyle, color
+
+
+def test_color_keep_returns_unchanged_message() -> None:
+    """Test that AnsiFore.KEEP returns the message unchanged."""
+    msg = "test message"
+    result = color(AnsiFore.KEEP, msg)
+    assert result == msg
+
+
+def test_color_keep_ignores_reset_parameter() -> None:
+    """Test that reset parameter is ignored when using AnsiFore.KEEP."""
+    msg = "test message"
+    result_with_reset = color(AnsiFore.KEEP, msg, reset=True)
+    result_without_reset = color(AnsiFore.KEEP, msg, reset=False)
+    assert result_with_reset == msg
+    assert result_without_reset == msg
+
+
+def test_color_applies_color_code() -> None:
+    """Test that color codes are properly applied to messages."""
+    msg = "test message"
+    result = color(AnsiFore.RED, msg, reset=False)
+    assert result == f"{AnsiFore.RED.value}{msg}"
+
+
+def test_color_applies_reset_when_requested() -> None:
+    """Test that RESET_ALL is added when reset=True."""
+    msg = "test message"
+    result = color(AnsiFore.GREEN, msg, reset=True)
+    expected = f"{AnsiFore.GREEN.value}{msg}{AnsiStyle.RESET_ALL.value}"
+    assert result == expected
+
+
+def test_color_no_reset_when_not_requested() -> None:
+    """Test that RESET_ALL is not added when reset=False."""
+    msg = "test message"
+    result = color(AnsiFore.BLUE, msg, reset=False)
+    expected = f"{AnsiFore.BLUE.value}{msg}"
+    assert result == expected
+
+
+def test_color_with_empty_message() -> None:
+    """Test color function with empty message."""
+    result = color(AnsiFore.YELLOW, "", reset=True)
+    expected = f"{AnsiFore.YELLOW.value}{AnsiStyle.RESET_ALL.value}"
+    assert result == expected
+
+
+@pytest.mark.parametrize(
+    "col",
+    [
+        AnsiFore.BLACK,
+        AnsiFore.RED,
+        AnsiFore.GREEN,
+        AnsiFore.YELLOW,
+        AnsiFore.BLUE,
+        AnsiFore.MAGENTA,
+        AnsiFore.CYAN,
+        AnsiFore.WHITE,
+        AnsiFore.RESET,
+    ],
+)
+def test_all_ansi_colors(col: AnsiFore) -> None:
+    """Test that all AnsiFore colors work correctly."""
+    msg = "test"
+    result = color(col, msg, reset=True)
+    expected = f"{col.value}{msg}{AnsiStyle.RESET_ALL.value}"
+    assert result == expected
+
+
+def test_ansi_fore_keep_is_enum_member() -> None:
+    """Ensure AnsiFore.KEEP is an Enum member and evaluates to truthy."""
+    assert isinstance(AnsiFore.KEEP, AnsiFore)
+    # Enum members are truthy, even with empty string values
+    assert bool(AnsiFore.KEEP) is True
+    # But the value itself is still an empty string
+    assert AnsiFore.KEEP.value == ""
diff --git a/tests/unit_tests/test_main.py b/tests/unit_tests/test_main.py
new file mode 100644
index 0000000000..59d0433aa4
--- /dev/null
+++ b/tests/unit_tests/test_main.py
@@ -0,0 +1,2269 @@
+"""Unit tests for esphome.__main__ module."""
+
+from __future__ import annotations
+
+from collections.abc import Generator
+from dataclasses import dataclass
+import logging
+from pathlib import Path
+import re
+from typing import Any
+from unittest.mock import MagicMock, Mock, patch
+
+import pytest
+from pytest import CaptureFixture
+
+from esphome import platformio_api
+from esphome.__main__ import (
+    Purpose,
+    choose_upload_log_host,
+    command_clean_all,
+    command_rename,
+    command_update_all,
+    command_wizard,
+    get_port_type,
+    has_ip_address,
+    has_mqtt,
+    has_mqtt_ip_lookup,
+    has_mqtt_logging,
+    has_non_ip_address,
+    has_resolvable_address,
+    mqtt_get_ip,
+    show_logs,
+    upload_program,
+    upload_using_esptool,
+)
+from esphome.components.esp32.const import KEY_ESP32, KEY_VARIANT, VARIANT_ESP32
+from esphome.const import (
+    CONF_API,
+    CONF_BROKER,
+    CONF_DISABLED,
+    CONF_ESPHOME,
+    CONF_LEVEL,
+    CONF_LOG_TOPIC,
+    CONF_MDNS,
+    CONF_MQTT,
+    CONF_NAME,
+    CONF_OTA,
+    CONF_PASSWORD,
+    CONF_PLATFORM,
+    CONF_PORT,
+    CONF_SUBSTITUTIONS,
+    CONF_TOPIC,
+    CONF_USE_ADDRESS,
+    CONF_WIFI,
+    KEY_CORE,
+    KEY_TARGET_PLATFORM,
+    PLATFORM_BK72XX,
+    PLATFORM_ESP32,
+    PLATFORM_ESP8266,
+    PLATFORM_RP2040,
+)
+from esphome.core import CORE, EsphomeError
+
+
+def strip_ansi_codes(text: str) -> str:
+    """Remove ANSI escape codes from text.
+
+    This helps make test assertions cleaner by removing color codes and other
+    terminal formatting that can make tests brittle.
+    """
+    # Pattern to match ANSI escape sequences
+    ansi_escape = re.compile(r"\x1B(?:[@-Z\\-_]|\[[0-?]*[ -/]*[@-~])")
+    return ansi_escape.sub("", text)
+
+
+@dataclass
+class MockSerialPort:
+    """Mock serial port for testing.
+
+    Attributes:
+        path (str): The device path of the mock serial port (e.g., '/dev/ttyUSB0').
+        description (str): A human-readable description of the mock serial port.
+    """
+
+    path: str
+    description: str
+
+
+def setup_core(
+    config: dict[str, Any] | None = None,
+    address: str | None = None,
+    platform: str | None = None,
+    tmp_path: Path | None = None,
+    name: str = "test",
+) -> None:
+    """
+    Helper to set up CORE configuration with optional address.
+
+    Args:
+        config (dict[str, Any] | None): The configuration dictionary to set for CORE. If None, an empty dict is used.
+        address (str | None): Optional network address to set in the configuration. If provided, it is set under the wifi config.
+        platform (str | None): Optional target platform to set in CORE.data.
+        tmp_path (Path | None): Optional temp path for setting up build paths.
+        name (str): The name of the device (defaults to "test").
+    """
+    if config is None:
+        config = {}
+
+    if address is not None:
+        # Set address via wifi config (could also use ethernet)
+        config[CONF_WIFI] = {CONF_USE_ADDRESS: address}
+
+    CORE.config = config
+
+    if platform is not None:
+        CORE.data[KEY_CORE] = {}
+        CORE.data[KEY_CORE][KEY_TARGET_PLATFORM] = platform
+
+    if tmp_path is not None:
+        CORE.config_path = str(tmp_path / f"{name}.yaml")
+        CORE.name = name
+        CORE.build_path = str(tmp_path / ".esphome" / "build" / name)
+
+
+@pytest.fixture
+def mock_no_serial_ports() -> Generator[Mock]:
+    """Mock get_serial_ports to return no ports."""
+    with patch("esphome.__main__.get_serial_ports", return_value=[]) as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_get_port_type() -> Generator[Mock]:
+    """Mock get_port_type for testing."""
+    with patch("esphome.__main__.get_port_type") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_check_permissions() -> Generator[Mock]:
+    """Mock check_permissions for testing."""
+    with patch("esphome.__main__.check_permissions") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_run_miniterm() -> Generator[Mock]:
+    """Mock run_miniterm for testing."""
+    with patch("esphome.__main__.run_miniterm") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_upload_using_esptool() -> Generator[Mock]:
+    """Mock upload_using_esptool for testing."""
+    with patch("esphome.__main__.upload_using_esptool") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_upload_using_platformio() -> Generator[Mock]:
+    """Mock upload_using_platformio for testing."""
+    with patch("esphome.__main__.upload_using_platformio") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_run_ota() -> Generator[Mock]:
+    """Mock espota2.run_ota for testing."""
+    with patch("esphome.espota2.run_ota") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_is_ip_address() -> Generator[Mock]:
+    """Mock is_ip_address for testing."""
+    with patch("esphome.__main__.is_ip_address") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_mqtt_get_ip() -> Generator[Mock]:
+    """Mock mqtt_get_ip for testing."""
+    with patch("esphome.__main__.mqtt_get_ip") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_serial_ports() -> Generator[Mock]:
+    """Mock get_serial_ports to return test ports."""
+    mock_ports = [
+        MockSerialPort("/dev/ttyUSB0", "USB Serial"),
+        MockSerialPort("/dev/ttyUSB1", "Another USB Serial"),
+    ]
+    with patch("esphome.__main__.get_serial_ports", return_value=mock_ports) as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_choose_prompt() -> Generator[Mock]:
+    """Mock choose_prompt to return default selection."""
+    with patch("esphome.__main__.choose_prompt", return_value="/dev/ttyUSB0") as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_no_mqtt_logging() -> Generator[Mock]:
+    """Mock has_mqtt_logging to return False."""
+    with patch("esphome.__main__.has_mqtt_logging", return_value=False) as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_has_mqtt_logging() -> Generator[Mock]:
+    """Mock has_mqtt_logging to return True."""
+    with patch("esphome.__main__.has_mqtt_logging", return_value=True) as mock:
+        yield mock
+
+
+@pytest.fixture
+def mock_run_external_process() -> Generator[Mock]:
+    """Mock run_external_process for testing."""
+    with patch("esphome.__main__.run_external_process") as mock:
+        mock.return_value = 0  # Default to success
+        yield mock
+
+
+@pytest.fixture
+def mock_run_external_command() -> Generator[Mock]:
+    """Mock run_external_command for testing."""
+    with patch("esphome.__main__.run_external_command") as mock:
+        mock.return_value = 0  # Default to success
+        yield mock
+
+
+def test_choose_upload_log_host_with_string_default() -> None:
+    """Test with a single string default device."""
+    setup_core()
+    result = choose_upload_log_host(
+        default="192.168.1.100",
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == ["192.168.1.100"]
+
+
+def test_choose_upload_log_host_with_list_default() -> None:
+    """Test with a list of default devices."""
+    setup_core()
+    result = choose_upload_log_host(
+        default=["192.168.1.100", "192.168.1.101"],
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == ["192.168.1.100", "192.168.1.101"]
+
+
+def test_choose_upload_log_host_with_multiple_ip_addresses() -> None:
+    """Test with multiple IP addresses as defaults."""
+    setup_core()
+    result = choose_upload_log_host(
+        default=["1.2.3.4", "4.5.5.6"],
+        check_default=None,
+        purpose=Purpose.LOGGING,
+    )
+    assert result == ["1.2.3.4", "4.5.5.6"]
+
+
+def test_choose_upload_log_host_with_mixed_hostnames_and_ips() -> None:
+    """Test with a mix of hostnames and IP addresses."""
+    setup_core()
+    result = choose_upload_log_host(
+        default=["host.one", "host.one.local", "1.2.3.4"],
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == ["host.one", "host.one.local", "1.2.3.4"]
+
+
+def test_choose_upload_log_host_with_ota_list() -> None:
+    """Test with OTA as the only item in the list."""
+    setup_core(config={CONF_OTA: {}}, address="192.168.1.100")
+
+    result = choose_upload_log_host(
+        default=["OTA"],
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == ["192.168.1.100"]
+
+
+@pytest.mark.usefixtures("mock_has_mqtt_logging")
+def test_choose_upload_log_host_with_ota_list_mqtt_fallback() -> None:
+    """Test with OTA list falling back to MQTT when no address."""
+    setup_core(config={CONF_OTA: {}, "mqtt": {}})
+
+    result = choose_upload_log_host(
+        default=["OTA"],
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == ["MQTTIP"]
+
+
+@pytest.mark.usefixtures("mock_has_mqtt_logging")
+def test_choose_upload_log_host_with_ota_list_mqtt_fallback_logging() -> None:
+    """Test with OTA list with API and MQTT when no address."""
+    setup_core(config={CONF_API: {}, "mqtt": {}})
+
+    result = choose_upload_log_host(
+        default=["OTA"],
+        check_default=None,
+        purpose=Purpose.LOGGING,
+    )
+    assert result == ["MQTTIP", "MQTT"]
+
+
+@pytest.mark.usefixtures("mock_no_serial_ports")
+def test_choose_upload_log_host_with_serial_device_no_ports(
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test SERIAL device when no serial ports are found."""
+    setup_core()
+    result = choose_upload_log_host(
+        default="SERIAL",
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == []
+    assert "No serial ports found, skipping SERIAL device" in caplog.text
+
+
+@pytest.mark.usefixtures("mock_serial_ports")
+def test_choose_upload_log_host_with_serial_device_with_ports(
+    mock_choose_prompt: Mock,
+) -> None:
+    """Test SERIAL device when serial ports are available."""
+    setup_core()
+    result = choose_upload_log_host(
+        default="SERIAL",
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == ["/dev/ttyUSB0"]
+    mock_choose_prompt.assert_called_once_with(
+        [
+            ("/dev/ttyUSB0 (USB Serial)", "/dev/ttyUSB0"),
+            ("/dev/ttyUSB1 (Another USB Serial)", "/dev/ttyUSB1"),
+        ],
+        purpose=Purpose.UPLOADING,
+    )
+
+
+def test_choose_upload_log_host_with_ota_device_with_ota_config() -> None:
+    """Test OTA device when OTA is configured."""
+    setup_core(config={CONF_OTA: {}}, address="192.168.1.100")
+
+    result = choose_upload_log_host(
+        default="OTA",
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == ["192.168.1.100"]
+
+
+def test_choose_upload_log_host_with_ota_device_with_api_config() -> None:
+    """Test OTA device when API is configured (no upload without OTA in config)."""
+    setup_core(config={CONF_API: {}}, address="192.168.1.100")
+
+    result = choose_upload_log_host(
+        default="OTA",
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == []
+
+
+def test_choose_upload_log_host_with_ota_device_with_api_config_logging() -> None:
+    """Test OTA device when API is configured."""
+    setup_core(config={CONF_API: {}}, address="192.168.1.100")
+
+    result = choose_upload_log_host(
+        default="OTA",
+        check_default=None,
+        purpose=Purpose.LOGGING,
+    )
+    assert result == ["192.168.1.100"]
+
+
+@pytest.mark.usefixtures("mock_has_mqtt_logging")
+def test_choose_upload_log_host_with_ota_device_fallback_to_mqtt() -> None:
+    """Test OTA device fallback to MQTT when no OTA/API config."""
+    setup_core(config={"mqtt": {}})
+
+    result = choose_upload_log_host(
+        default="OTA",
+        check_default=None,
+        purpose=Purpose.LOGGING,
+    )
+    assert result == ["MQTT"]
+
+
+@pytest.mark.usefixtures("mock_no_mqtt_logging")
+def test_choose_upload_log_host_with_ota_device_no_fallback() -> None:
+    """Test OTA device with no valid fallback options."""
+    setup_core()
+
+    result = choose_upload_log_host(
+        default="OTA",
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == []
+
+
+@pytest.mark.usefixtures("mock_choose_prompt")
+def test_choose_upload_log_host_multiple_devices() -> None:
+    """Test with multiple devices including special identifiers."""
+    setup_core(config={CONF_OTA: {}}, address="192.168.1.100")
+
+    mock_ports = [MockSerialPort("/dev/ttyUSB0", "USB Serial")]
+
+    with patch("esphome.__main__.get_serial_ports", return_value=mock_ports):
+        result = choose_upload_log_host(
+            default=["192.168.1.50", "OTA", "SERIAL"],
+            check_default=None,
+            purpose=Purpose.UPLOADING,
+        )
+        assert result == ["192.168.1.50", "192.168.1.100", "/dev/ttyUSB0"]
+
+
+def test_choose_upload_log_host_no_defaults_with_serial_ports(
+    mock_choose_prompt: Mock,
+) -> None:
+    """Test interactive mode with serial ports available."""
+    mock_ports = [
+        MockSerialPort("/dev/ttyUSB0", "USB Serial"),
+    ]
+
+    setup_core()
+
+    with patch("esphome.__main__.get_serial_ports", return_value=mock_ports):
+        result = choose_upload_log_host(
+            default=None,
+            check_default=None,
+            purpose=Purpose.UPLOADING,
+        )
+        assert result == ["/dev/ttyUSB0"]
+        mock_choose_prompt.assert_called_once_with(
+            [("/dev/ttyUSB0 (USB Serial)", "/dev/ttyUSB0")],
+            purpose=Purpose.UPLOADING,
+        )
+
+
+@pytest.mark.usefixtures("mock_no_serial_ports")
+def test_choose_upload_log_host_no_defaults_with_ota() -> None:
+    """Test interactive mode with OTA option."""
+    setup_core(config={CONF_OTA: {}}, address="192.168.1.100")
+
+    with patch(
+        "esphome.__main__.choose_prompt", return_value="192.168.1.100"
+    ) as mock_prompt:
+        result = choose_upload_log_host(
+            default=None,
+            check_default=None,
+            purpose=Purpose.UPLOADING,
+        )
+        assert result == ["192.168.1.100"]
+        mock_prompt.assert_called_once_with(
+            [("Over The Air (192.168.1.100)", "192.168.1.100")],
+            purpose=Purpose.UPLOADING,
+        )
+
+
+@pytest.mark.usefixtures("mock_no_serial_ports")
+def test_choose_upload_log_host_no_defaults_with_api() -> None:
+    """Test interactive mode with API option."""
+    setup_core(config={CONF_API: {}}, address="192.168.1.100")
+
+    with patch(
+        "esphome.__main__.choose_prompt", return_value="192.168.1.100"
+    ) as mock_prompt:
+        result = choose_upload_log_host(
+            default=None,
+            check_default=None,
+            purpose=Purpose.LOGGING,
+        )
+        assert result == ["192.168.1.100"]
+        mock_prompt.assert_called_once_with(
+            [("Over The Air (192.168.1.100)", "192.168.1.100")],
+            purpose=Purpose.LOGGING,
+        )
+
+
+@pytest.mark.usefixtures("mock_no_serial_ports", "mock_has_mqtt_logging")
+def test_choose_upload_log_host_no_defaults_with_mqtt() -> None:
+    """Test interactive mode with MQTT option."""
+    setup_core(config={CONF_MQTT: {CONF_BROKER: "mqtt.local"}})
+
+    with patch("esphome.__main__.choose_prompt", return_value="MQTT") as mock_prompt:
+        result = choose_upload_log_host(
+            default=None,
+            check_default=None,
+            purpose=Purpose.LOGGING,
+        )
+        assert result == ["MQTT"]
+        mock_prompt.assert_called_once_with(
+            [("MQTT (mqtt.local)", "MQTT")],
+            purpose=Purpose.LOGGING,
+        )
+
+
+@pytest.mark.usefixtures("mock_has_mqtt_logging")
+def test_choose_upload_log_host_no_defaults_with_all_options(
+    mock_choose_prompt: Mock,
+) -> None:
+    """Test interactive mode with all options available."""
+    setup_core(
+        config={CONF_OTA: {}, CONF_API: {}, CONF_MQTT: {CONF_BROKER: "mqtt.local"}},
+        address="192.168.1.100",
+    )
+
+    mock_ports = [MockSerialPort("/dev/ttyUSB0", "USB Serial")]
+
+    with patch("esphome.__main__.get_serial_ports", return_value=mock_ports):
+        result = choose_upload_log_host(
+            default=None,
+            check_default=None,
+            purpose=Purpose.UPLOADING,
+        )
+        assert result == ["/dev/ttyUSB0"]
+
+        expected_options = [
+            ("/dev/ttyUSB0 (USB Serial)", "/dev/ttyUSB0"),
+            ("Over The Air (192.168.1.100)", "192.168.1.100"),
+            ("Over The Air (MQTT IP lookup)", "MQTTIP"),
+        ]
+        mock_choose_prompt.assert_called_once_with(
+            expected_options, purpose=Purpose.UPLOADING
+        )
+
+
+def test_choose_upload_log_host_no_defaults_with_all_options_logging(
+    mock_choose_prompt: Mock,
+) -> None:
+    """Test interactive mode with all options available."""
+    setup_core(
+        config={CONF_OTA: {}, CONF_API: {}, CONF_MQTT: {CONF_BROKER: "mqtt.local"}},
+        address="192.168.1.100",
+    )
+
+    mock_ports = [MockSerialPort("/dev/ttyUSB0", "USB Serial")]
+
+    with patch("esphome.__main__.get_serial_ports", return_value=mock_ports):
+        result = choose_upload_log_host(
+            default=None,
+            check_default=None,
+            purpose=Purpose.LOGGING,
+        )
+        assert result == ["/dev/ttyUSB0"]
+
+        expected_options = [
+            ("/dev/ttyUSB0 (USB Serial)", "/dev/ttyUSB0"),
+            ("MQTT (mqtt.local)", "MQTT"),
+            ("Over The Air (192.168.1.100)", "192.168.1.100"),
+            ("Over The Air (MQTT IP lookup)", "MQTTIP"),
+        ]
+        mock_choose_prompt.assert_called_once_with(
+            expected_options, purpose=Purpose.LOGGING
+        )
+
+
+@pytest.mark.usefixtures("mock_no_serial_ports")
+def test_choose_upload_log_host_check_default_matches() -> None:
+    """Test when check_default matches an available option."""
+    setup_core(config={CONF_OTA: {}}, address="192.168.1.100")
+
+    result = choose_upload_log_host(
+        default=None,
+        check_default="192.168.1.100",
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == ["192.168.1.100"]
+
+
+@pytest.mark.usefixtures("mock_no_serial_ports")
+def test_choose_upload_log_host_check_default_no_match() -> None:
+    """Test when check_default doesn't match any available option."""
+    setup_core()
+
+    with patch(
+        "esphome.__main__.choose_prompt", return_value="fallback"
+    ) as mock_prompt:
+        result = choose_upload_log_host(
+            default=None,
+            check_default="192.168.1.100",
+            purpose=Purpose.UPLOADING,
+        )
+        assert result == ["fallback"]
+        mock_prompt.assert_called_once()
+
+
+@pytest.mark.usefixtures("mock_no_serial_ports")
+def test_choose_upload_log_host_empty_defaults_list() -> None:
+    """Test with an empty list as default."""
+    setup_core()
+    with patch("esphome.__main__.choose_prompt", return_value="chosen") as mock_prompt:
+        result = choose_upload_log_host(
+            default=[],
+            check_default=None,
+            purpose=Purpose.UPLOADING,
+        )
+        assert result == ["chosen"]
+        mock_prompt.assert_called_once()
+
+
+@pytest.mark.usefixtures("mock_no_serial_ports", "mock_no_mqtt_logging")
+def test_choose_upload_log_host_all_devices_unresolved(
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test when all specified devices cannot be resolved."""
+    setup_core()
+
+    result = choose_upload_log_host(
+        default=["SERIAL", "OTA"],
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == []
+    assert (
+        "All specified devices: ['SERIAL', 'OTA'] could not be resolved." in caplog.text
+    )
+
+
+@pytest.mark.usefixtures("mock_no_serial_ports", "mock_no_mqtt_logging")
+def test_choose_upload_log_host_mixed_resolved_unresolved() -> None:
+    """Test with a mix of resolved and unresolved devices."""
+    setup_core()
+
+    result = choose_upload_log_host(
+        default=["192.168.1.50", "SERIAL", "OTA"],
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == ["192.168.1.50"]
+
+
+def test_choose_upload_log_host_ota_both_conditions() -> None:
+    """Test OTA device when both OTA and API are configured and enabled."""
+    setup_core(config={CONF_OTA: {}, CONF_API: {}}, address="192.168.1.100")
+
+    result = choose_upload_log_host(
+        default="OTA",
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == ["192.168.1.100"]
+
+
+@pytest.mark.usefixtures("mock_serial_ports")
+def test_choose_upload_log_host_ota_ip_all_options() -> None:
+    """Test OTA device when both static IP, OTA, API and MQTT are configured and enabled but MDNS not."""
+    setup_core(
+        config={
+            CONF_OTA: {},
+            CONF_API: {},
+            CONF_MQTT: {
+                CONF_BROKER: "mqtt.local",
+            },
+            CONF_MDNS: {
+                CONF_DISABLED: True,
+            },
+        },
+        address="192.168.1.100",
+    )
+
+    result = choose_upload_log_host(
+        default="OTA",
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == ["192.168.1.100", "MQTTIP"]
+
+
+@pytest.mark.usefixtures("mock_serial_ports")
+def test_choose_upload_log_host_ota_local_all_options() -> None:
+    """Test OTA device when both static IP, OTA, API and MQTT are configured and enabled but MDNS not."""
+    setup_core(
+        config={
+            CONF_OTA: {},
+            CONF_API: {},
+            CONF_MQTT: {
+                CONF_BROKER: "mqtt.local",
+            },
+            CONF_MDNS: {
+                CONF_DISABLED: True,
+            },
+        },
+        address="test.local",
+    )
+
+    result = choose_upload_log_host(
+        default="OTA",
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == ["MQTTIP", "test.local"]
+
+
+@pytest.mark.usefixtures("mock_serial_ports")
+def test_choose_upload_log_host_ota_ip_all_options_logging() -> None:
+    """Test OTA device when both static IP, OTA, API and MQTT are configured and enabled but MDNS not."""
+    setup_core(
+        config={
+            CONF_OTA: {},
+            CONF_API: {},
+            CONF_MQTT: {
+                CONF_BROKER: "mqtt.local",
+            },
+            CONF_MDNS: {
+                CONF_DISABLED: True,
+            },
+        },
+        address="192.168.1.100",
+    )
+
+    result = choose_upload_log_host(
+        default="OTA",
+        check_default=None,
+        purpose=Purpose.LOGGING,
+    )
+    assert result == ["192.168.1.100", "MQTTIP", "MQTT"]
+
+
+@pytest.mark.usefixtures("mock_serial_ports")
+def test_choose_upload_log_host_ota_local_all_options_logging() -> None:
+    """Test OTA device when both static IP, OTA, API and MQTT are configured and enabled but MDNS not."""
+    setup_core(
+        config={
+            CONF_OTA: {},
+            CONF_API: {},
+            CONF_MQTT: {
+                CONF_BROKER: "mqtt.local",
+            },
+            CONF_MDNS: {
+                CONF_DISABLED: True,
+            },
+        },
+        address="test.local",
+    )
+
+    result = choose_upload_log_host(
+        default="OTA",
+        check_default=None,
+        purpose=Purpose.LOGGING,
+    )
+    assert result == ["MQTTIP", "MQTT", "test.local"]
+
+
+@pytest.mark.usefixtures("mock_no_mqtt_logging")
+def test_choose_upload_log_host_no_address_with_ota_config() -> None:
+    """Test OTA device when OTA is configured but no address is set."""
+    setup_core(config={CONF_OTA: {}})
+
+    result = choose_upload_log_host(
+        default="OTA",
+        check_default=None,
+        purpose=Purpose.UPLOADING,
+    )
+    assert result == []
+
+
+@dataclass
+class MockArgs:
+    """Mock args for testing."""
+
+    file: str | None = None
+    upload_speed: int = 460800
+    username: str | None = None
+    password: str | None = None
+    client_id: str | None = None
+    topic: str | None = None
+    configuration: str | None = None
+    name: str | None = None
+    dashboard: bool = False
+
+
+def test_upload_program_serial_esp32(
+    mock_upload_using_esptool: Mock,
+    mock_get_port_type: Mock,
+    mock_check_permissions: Mock,
+) -> None:
+    """Test upload_program with serial port for ESP32."""
+    setup_core(platform=PLATFORM_ESP32)
+    mock_get_port_type.return_value = "SERIAL"
+    mock_upload_using_esptool.return_value = 0
+
+    config = {}
+    args = MockArgs()
+    devices = ["/dev/ttyUSB0"]
+
+    exit_code, host = upload_program(config, args, devices)
+
+    assert exit_code == 0
+    assert host == "/dev/ttyUSB0"
+    mock_check_permissions.assert_called_once_with("/dev/ttyUSB0")
+    mock_upload_using_esptool.assert_called_once()
+
+
+def test_upload_program_serial_esp8266_with_file(
+    mock_upload_using_esptool: Mock,
+    mock_get_port_type: Mock,
+    mock_check_permissions: Mock,
+) -> None:
+    """Test upload_program with serial port for ESP8266 with custom file."""
+    setup_core(platform=PLATFORM_ESP8266)
+    mock_get_port_type.return_value = "SERIAL"
+    mock_upload_using_esptool.return_value = 0
+
+    config = {}
+    args = MockArgs(file="firmware.bin")
+    devices = ["/dev/ttyUSB0"]
+
+    exit_code, host = upload_program(config, args, devices)
+
+    assert exit_code == 0
+    assert host == "/dev/ttyUSB0"
+    mock_check_permissions.assert_called_once_with("/dev/ttyUSB0")
+    mock_upload_using_esptool.assert_called_once_with(
+        config, "/dev/ttyUSB0", "firmware.bin", 460800
+    )
+
+
+def test_upload_using_esptool_path_conversion(
+    tmp_path: Path,
+    mock_run_external_command: Mock,
+    mock_get_idedata: Mock,
+) -> None:
+    """Test upload_using_esptool properly converts Path objects to strings for esptool.
+
+    This test ensures that img.path (Path object) is converted to string before
+    passing to esptool, preventing AttributeError.
+    """
+    setup_core(platform=PLATFORM_ESP32, tmp_path=tmp_path, name="test")
+
+    # Set up ESP32-specific data required by get_esp32_variant()
+    CORE.data[KEY_ESP32] = {KEY_VARIANT: VARIANT_ESP32}
+
+    # Create mock IDEData with Path objects
+    mock_idedata = MagicMock(spec=platformio_api.IDEData)
+    mock_idedata.firmware_bin_path = tmp_path / "firmware.bin"
+    mock_idedata.extra_flash_images = [
+        platformio_api.FlashImage(path=tmp_path / "bootloader.bin", offset="0x1000"),
+        platformio_api.FlashImage(path=tmp_path / "partitions.bin", offset="0x8000"),
+    ]
+
+    mock_get_idedata.return_value = mock_idedata
+
+    # Create the actual firmware files so they exist
+    (tmp_path / "firmware.bin").touch()
+    (tmp_path / "bootloader.bin").touch()
+    (tmp_path / "partitions.bin").touch()
+
+    config = {CONF_ESPHOME: {"platformio_options": {}}}
+
+    # Call upload_using_esptool without custom file argument
+    result = upload_using_esptool(config, "/dev/ttyUSB0", None, None)
+
+    assert result == 0
+
+    # Verify that run_external_command was called
+    assert mock_run_external_command.call_count == 1
+
+    # Get the actual call arguments
+    call_args = mock_run_external_command.call_args[0]
+
+    # The first argument should be esptool.main function,
+    # followed by the command arguments
+    assert len(call_args) > 1
+
+    # Find the indices of the flash image arguments
+    # They should come after "write-flash" and "-z"
+    cmd_list = list(call_args[1:])  # Skip the esptool.main function
+
+    # Verify all paths are strings, not Path objects
+    # The firmware and flash images should be at specific positions
+    write_flash_idx = cmd_list.index("write-flash")
+
+    # After write-flash we have: -z, --flash-size, detect, then offset/path pairs
+    # Check firmware at offset 0x10000 (ESP32)
+    firmware_offset_idx = write_flash_idx + 4
+    assert cmd_list[firmware_offset_idx] == "0x10000"
+    firmware_path = cmd_list[firmware_offset_idx + 1]
+    assert isinstance(firmware_path, str)
+    assert firmware_path.endswith("firmware.bin")
+
+    # Check bootloader
+    bootloader_offset_idx = firmware_offset_idx + 2
+    assert cmd_list[bootloader_offset_idx] == "0x1000"
+    bootloader_path = cmd_list[bootloader_offset_idx + 1]
+    assert isinstance(bootloader_path, str)
+    assert bootloader_path.endswith("bootloader.bin")
+
+    # Check partitions
+    partitions_offset_idx = bootloader_offset_idx + 2
+    assert cmd_list[partitions_offset_idx] == "0x8000"
+    partitions_path = cmd_list[partitions_offset_idx + 1]
+    assert isinstance(partitions_path, str)
+    assert partitions_path.endswith("partitions.bin")
+
+
+def test_upload_using_esptool_with_file_path(
+    tmp_path: Path,
+    mock_run_external_command: Mock,
+) -> None:
+    """Test upload_using_esptool with a custom file that's a Path object."""
+    setup_core(platform=PLATFORM_ESP8266, tmp_path=tmp_path, name="test")
+
+    # Create a test firmware file
+    firmware_file = tmp_path / "custom_firmware.bin"
+    firmware_file.touch()
+
+    config = {CONF_ESPHOME: {"platformio_options": {}}}
+
+    # Call with a Path object as the file argument (though usually it's a string)
+    result = upload_using_esptool(config, "/dev/ttyUSB0", str(firmware_file), None)
+
+    assert result == 0
+
+    # Verify that run_external_command was called
+    mock_run_external_command.assert_called_once()
+
+    # Get the actual call arguments
+    call_args = mock_run_external_command.call_args[0]
+    cmd_list = list(call_args[1:])  # Skip the esptool.main function
+
+    # Find the firmware path in the command
+    write_flash_idx = cmd_list.index("write-flash")
+
+    # For custom file, it should be at offset 0x0
+    firmware_offset_idx = write_flash_idx + 4
+    assert cmd_list[firmware_offset_idx] == "0x0"
+    firmware_path = cmd_list[firmware_offset_idx + 1]
+
+    # Verify it's a string, not a Path object
+    assert isinstance(firmware_path, str)
+    assert firmware_path.endswith("custom_firmware.bin")
+
+
+@pytest.mark.parametrize(
+    "platform,device",
+    [
+        (PLATFORM_RP2040, "/dev/ttyACM0"),
+        (PLATFORM_BK72XX, "/dev/ttyUSB0"),  # LibreTiny platform
+    ],
+)
+def test_upload_program_serial_platformio_platforms(
+    mock_upload_using_platformio: Mock,
+    mock_get_port_type: Mock,
+    mock_check_permissions: Mock,
+    platform: str,
+    device: str,
+) -> None:
+    """Test upload_program with serial port for platformio platforms (RP2040/LibreTiny)."""
+    setup_core(platform=platform)
+    mock_get_port_type.return_value = "SERIAL"
+    mock_upload_using_platformio.return_value = 0
+
+    config = {}
+    args = MockArgs()
+    devices = [device]
+
+    exit_code, host = upload_program(config, args, devices)
+
+    assert exit_code == 0
+    assert host == device
+    mock_check_permissions.assert_called_once_with(device)
+    mock_upload_using_platformio.assert_called_once_with(config, device)
+
+
+def test_upload_program_serial_upload_failed(
+    mock_upload_using_esptool: Mock,
+    mock_get_port_type: Mock,
+    mock_check_permissions: Mock,
+) -> None:
+    """Test upload_program when serial upload fails."""
+    setup_core(platform=PLATFORM_ESP32)
+    mock_get_port_type.return_value = "SERIAL"
+    mock_upload_using_esptool.return_value = 1  # Failed
+
+    config = {}
+    args = MockArgs()
+    devices = ["/dev/ttyUSB0"]
+
+    exit_code, host = upload_program(config, args, devices)
+
+    assert exit_code == 1
+    assert host is None
+    mock_check_permissions.assert_called_once_with("/dev/ttyUSB0")
+    mock_upload_using_esptool.assert_called_once()
+
+
+def test_upload_program_ota_success(
+    mock_run_ota: Mock,
+    mock_get_port_type: Mock,
+    tmp_path: Path,
+) -> None:
+    """Test upload_program with OTA."""
+    setup_core(platform=PLATFORM_ESP32, tmp_path=tmp_path)
+
+    mock_get_port_type.return_value = "NETWORK"
+    mock_run_ota.return_value = (0, "192.168.1.100")
+
+    config = {
+        CONF_OTA: [
+            {
+                CONF_PLATFORM: CONF_ESPHOME,
+                CONF_PORT: 3232,
+                CONF_PASSWORD: "secret",
+            }
+        ]
+    }
+    args = MockArgs()
+    devices = ["192.168.1.100"]
+
+    exit_code, host = upload_program(config, args, devices)
+
+    assert exit_code == 0
+    assert host == "192.168.1.100"
+    expected_firmware = (
+        tmp_path / ".esphome" / "build" / "test" / ".pioenvs" / "test" / "firmware.bin"
+    )
+    mock_run_ota.assert_called_once_with(
+        ["192.168.1.100"], 3232, "secret", expected_firmware
+    )
+
+
+def test_upload_program_ota_with_file_arg(
+    mock_run_ota: Mock,
+    mock_get_port_type: Mock,
+    tmp_path: Path,
+) -> None:
+    """Test upload_program with OTA and custom file."""
+    setup_core(platform=PLATFORM_ESP32, tmp_path=tmp_path)
+
+    mock_get_port_type.return_value = "NETWORK"
+    mock_run_ota.return_value = (0, "192.168.1.100")
+
+    config = {
+        CONF_OTA: [
+            {
+                CONF_PLATFORM: CONF_ESPHOME,
+                CONF_PORT: 3232,
+            }
+        ]
+    }
+    args = MockArgs(file="custom.bin")
+    devices = ["192.168.1.100"]
+
+    exit_code, host = upload_program(config, args, devices)
+
+    assert exit_code == 0
+    assert host == "192.168.1.100"
+    mock_run_ota.assert_called_once_with(
+        ["192.168.1.100"], 3232, None, Path("custom.bin")
+    )
+
+
+def test_upload_program_ota_no_config(
+    mock_get_port_type: Mock,
+) -> None:
+    """Test upload_program with OTA but no OTA config."""
+    setup_core(platform=PLATFORM_ESP32)
+    mock_get_port_type.return_value = "NETWORK"
+
+    config = {}  # No OTA config
+    args = MockArgs()
+    devices = ["192.168.1.100"]
+
+    with pytest.raises(EsphomeError, match="Cannot upload Over the Air"):
+        upload_program(config, args, devices)
+
+
+def test_upload_program_ota_with_mqtt_resolution(
+    mock_mqtt_get_ip: Mock,
+    mock_is_ip_address: Mock,
+    mock_run_ota: Mock,
+    tmp_path: Path,
+) -> None:
+    """Test upload_program with OTA using MQTT for address resolution."""
+    setup_core(address="device.local", platform=PLATFORM_ESP32, tmp_path=tmp_path)
+
+    mock_is_ip_address.return_value = False
+    mock_mqtt_get_ip.return_value = ["192.168.1.100"]
+    mock_run_ota.return_value = (0, "192.168.1.100")
+
+    config = {
+        CONF_OTA: [
+            {
+                CONF_PLATFORM: CONF_ESPHOME,
+                CONF_PORT: 3232,
+            }
+        ],
+        CONF_MQTT: {
+            CONF_BROKER: "mqtt.local",
+        },
+        CONF_MDNS: {
+            CONF_DISABLED: True,
+        },
+    }
+    args = MockArgs(username="user", password="pass", client_id="client")
+    devices = ["MQTT"]
+
+    exit_code, host = upload_program(config, args, devices)
+
+    assert exit_code == 0
+    assert host == "192.168.1.100"
+    mock_mqtt_get_ip.assert_called_once_with(config, "user", "pass", "client")
+    expected_firmware = (
+        tmp_path / ".esphome" / "build" / "test" / ".pioenvs" / "test" / "firmware.bin"
+    )
+    mock_run_ota.assert_called_once_with(
+        ["192.168.1.100"], 3232, None, expected_firmware
+    )
+
+
+@patch("esphome.__main__.importlib.import_module")
+def test_upload_program_platform_specific_handler(
+    mock_import: Mock,
+    mock_get_port_type: Mock,
+) -> None:
+    """Test upload_program with platform-specific upload handler."""
+    setup_core(platform="custom_platform")
+    mock_get_port_type.return_value = "CUSTOM"
+
+    mock_module = MagicMock()
+    mock_module.upload_program.return_value = True
+    mock_import.return_value = mock_module
+
+    config = {}
+    args = MockArgs()
+    devices = ["custom_device"]
+
+    exit_code, host = upload_program(config, args, devices)
+
+    assert exit_code == 0
+    assert host == "custom_device"
+    mock_import.assert_called_once_with("esphome.components.custom_platform")
+    mock_module.upload_program.assert_called_once_with(config, args, "custom_device")
+
+
+def test_show_logs_serial(
+    mock_get_port_type: Mock,
+    mock_check_permissions: Mock,
+    mock_run_miniterm: Mock,
+) -> None:
+    """Test show_logs with serial port."""
+    setup_core(config={"logger": {}}, platform=PLATFORM_ESP32)
+    mock_get_port_type.return_value = "SERIAL"
+    mock_run_miniterm.return_value = 0
+
+    args = MockArgs()
+    devices = ["/dev/ttyUSB0"]
+
+    result = show_logs(CORE.config, args, devices)
+
+    assert result == 0
+    mock_check_permissions.assert_called_once_with("/dev/ttyUSB0")
+    mock_run_miniterm.assert_called_once_with(CORE.config, "/dev/ttyUSB0", args)
+
+
+def test_show_logs_no_logger() -> None:
+    """Test show_logs when logger is not configured."""
+    setup_core(config={}, platform=PLATFORM_ESP32)  # No logger config
+    args = MockArgs()
+    devices = ["/dev/ttyUSB0"]
+
+    with pytest.raises(EsphomeError, match="Logger is not configured"):
+        show_logs(CORE.config, args, devices)
+
+
+@patch("esphome.components.api.client.run_logs")
+def test_show_logs_api(
+    mock_run_logs: Mock,
+) -> None:
+    """Test show_logs with API."""
+    setup_core(
+        config={
+            "logger": {},
+            CONF_API: {},
+            CONF_MDNS: {CONF_DISABLED: False},
+        },
+        platform=PLATFORM_ESP32,
+    )
+    mock_run_logs.return_value = 0
+
+    args = MockArgs()
+    devices = ["192.168.1.100", "192.168.1.101"]
+
+    result = show_logs(CORE.config, args, devices)
+
+    assert result == 0
+    mock_run_logs.assert_called_once_with(
+        CORE.config, ["192.168.1.100", "192.168.1.101"]
+    )
+
+
+@patch("esphome.components.api.client.run_logs")
+def test_show_logs_api_with_fqdn_mdns_disabled(
+    mock_run_logs: Mock,
+) -> None:
+    """Test show_logs with API using FQDN when mDNS is disabled."""
+    setup_core(
+        config={
+            "logger": {},
+            CONF_API: {},
+            CONF_MDNS: {CONF_DISABLED: True},
+        },
+        platform=PLATFORM_ESP32,
+    )
+    mock_run_logs.return_value = 0
+
+    args = MockArgs()
+    devices = ["device.example.com"]
+
+    result = show_logs(CORE.config, args, devices)
+
+    assert result == 0
+    # Should use the FQDN directly, not try MQTT lookup
+    mock_run_logs.assert_called_once_with(CORE.config, ["device.example.com"])
+
+
+@patch("esphome.components.api.client.run_logs")
+def test_show_logs_api_with_mqtt_fallback(
+    mock_run_logs: Mock,
+    mock_mqtt_get_ip: Mock,
+) -> None:
+    """Test show_logs with API using MQTT for address resolution."""
+    setup_core(
+        config={
+            "logger": {},
+            CONF_API: {},
+            CONF_MDNS: {CONF_DISABLED: True},
+            CONF_MQTT: {CONF_BROKER: "mqtt.local"},
+        },
+        platform=PLATFORM_ESP32,
+    )
+    mock_run_logs.return_value = 0
+    mock_mqtt_get_ip.return_value = ["192.168.1.200"]
+
+    args = MockArgs(username="user", password="pass", client_id="client")
+    devices = ["MQTTIP"]
+
+    result = show_logs(CORE.config, args, devices)
+
+    assert result == 0
+    mock_mqtt_get_ip.assert_called_once_with(CORE.config, "user", "pass", "client")
+    mock_run_logs.assert_called_once_with(CORE.config, ["192.168.1.200"])
+
+
+@patch("esphome.mqtt.show_logs")
+def test_show_logs_mqtt(
+    mock_mqtt_show_logs: Mock,
+) -> None:
+    """Test show_logs with MQTT."""
+    setup_core(
+        config={
+            "logger": {},
+            "mqtt": {CONF_BROKER: "mqtt.local"},
+        },
+        platform=PLATFORM_ESP32,
+    )
+    mock_mqtt_show_logs.return_value = 0
+
+    args = MockArgs(
+        topic="esphome/logs",
+        username="user",
+        password="pass",
+        client_id="client",
+    )
+    devices = ["MQTT"]
+
+    result = show_logs(CORE.config, args, devices)
+
+    assert result == 0
+    mock_mqtt_show_logs.assert_called_once_with(
+        CORE.config, "esphome/logs", "user", "pass", "client"
+    )
+
+
+@patch("esphome.mqtt.show_logs")
+def test_show_logs_network_with_mqtt_only(
+    mock_mqtt_show_logs: Mock,
+) -> None:
+    """Test show_logs with network port but only MQTT configured."""
+    setup_core(
+        config={
+            "logger": {},
+            "mqtt": {CONF_BROKER: "mqtt.local"},
+            # No API configured
+        },
+        platform=PLATFORM_ESP32,
+    )
+    mock_mqtt_show_logs.return_value = 0
+
+    args = MockArgs(
+        topic="esphome/logs",
+        username="user",
+        password="pass",
+        client_id="client",
+    )
+    devices = ["192.168.1.100"]
+
+    result = show_logs(CORE.config, args, devices)
+
+    assert result == 0
+    mock_mqtt_show_logs.assert_called_once_with(
+        CORE.config, "esphome/logs", "user", "pass", "client"
+    )
+
+
+def test_show_logs_no_method_configured() -> None:
+    """Test show_logs when no remote logging method is configured."""
+    setup_core(
+        config={
+            "logger": {},
+            # No API or MQTT configured
+        },
+        platform=PLATFORM_ESP32,
+    )
+
+    args = MockArgs()
+    devices = ["192.168.1.100"]
+
+    with pytest.raises(
+        EsphomeError, match="No remote or local logging method configured"
+    ):
+        show_logs(CORE.config, args, devices)
+
+
+@patch("esphome.__main__.importlib.import_module")
+def test_show_logs_platform_specific_handler(
+    mock_import: Mock,
+) -> None:
+    """Test show_logs with platform-specific logs handler."""
+    setup_core(platform="custom_platform", config={"logger": {}})
+
+    mock_module = MagicMock()
+    mock_module.show_logs.return_value = True
+    mock_import.return_value = mock_module
+
+    config = {"logger": {}}
+    args = MockArgs()
+    devices = ["custom_device"]
+
+    result = show_logs(config, args, devices)
+
+    assert result == 0
+    mock_import.assert_called_once_with("esphome.components.custom_platform")
+    mock_module.show_logs.assert_called_once_with(config, args, devices)
+
+
+def test_has_mqtt_logging_no_log_topic() -> None:
+    """Test has_mqtt_logging returns True when CONF_LOG_TOPIC is not in mqtt_config."""
+
+    # Setup MQTT config without CONF_LOG_TOPIC (defaults to enabled - this is the missing test case)
+    setup_core(config={CONF_MQTT: {CONF_BROKER: "mqtt.local"}})
+    assert has_mqtt_logging() is True
+
+    # Setup MQTT config with CONF_LOG_TOPIC set to None (explicitly disabled)
+    setup_core(config={CONF_MQTT: {CONF_BROKER: "mqtt.local", CONF_LOG_TOPIC: None}})
+    assert has_mqtt_logging() is False
+
+    # Setup MQTT config with CONF_LOG_TOPIC set with topic and level (explicitly enabled)
+    setup_core(
+        config={
+            CONF_MQTT: {
+                CONF_BROKER: "mqtt.local",
+                CONF_LOG_TOPIC: {CONF_TOPIC: "esphome/logs", CONF_LEVEL: "DEBUG"},
+            }
+        }
+    )
+    assert has_mqtt_logging() is True
+
+    # Setup MQTT config with CONF_LOG_TOPIC set but level is NONE (disabled)
+    setup_core(
+        config={
+            CONF_MQTT: {
+                CONF_BROKER: "mqtt.local",
+                CONF_LOG_TOPIC: {CONF_TOPIC: "esphome/logs", CONF_LEVEL: "NONE"},
+            }
+        }
+    )
+    assert has_mqtt_logging() is False
+
+    # Setup without MQTT config at all
+    setup_core(config={})
+    assert has_mqtt_logging() is False
+
+    # Setup MQTT config with CONF_LOG_TOPIC but no CONF_LEVEL (regression test for #10771)
+    # This simulates the default configuration created by validate_config in the MQTT component
+    setup_core(
+        config={
+            CONF_MQTT: {
+                CONF_BROKER: "mqtt.local",
+                CONF_LOG_TOPIC: {CONF_TOPIC: "esphome/debug"},
+            }
+        }
+    )
+    assert has_mqtt_logging() is True
+
+
+def test_has_mqtt() -> None:
+    """Test has_mqtt function."""
+
+    # Test with MQTT configured
+    setup_core(config={CONF_MQTT: {CONF_BROKER: "mqtt.local"}})
+    assert has_mqtt() is True
+
+    # Test without MQTT configured
+    setup_core(config={})
+    assert has_mqtt() is False
+
+    # Test with other components but no MQTT
+    setup_core(config={CONF_API: {}, CONF_OTA: {}})
+    assert has_mqtt() is False
+
+
+def test_get_port_type() -> None:
+    """Test get_port_type function."""
+
+    assert get_port_type("/dev/ttyUSB0") == "SERIAL"
+    assert get_port_type("/dev/ttyACM0") == "SERIAL"
+    assert get_port_type("COM1") == "SERIAL"
+    assert get_port_type("COM10") == "SERIAL"
+
+    assert get_port_type("MQTT") == "MQTT"
+    assert get_port_type("MQTTIP") == "MQTTIP"
+
+    assert get_port_type("192.168.1.100") == "NETWORK"
+    assert get_port_type("esphome-device.local") == "NETWORK"
+    assert get_port_type("10.0.0.1") == "NETWORK"
+
+
+def test_has_mqtt_ip_lookup() -> None:
+    """Test has_mqtt_ip_lookup function."""
+
+    CONF_DISCOVER_IP = "discover_ip"
+
+    setup_core(config={})
+    assert has_mqtt_ip_lookup() is False
+
+    setup_core(config={CONF_MQTT: {CONF_BROKER: "mqtt.local"}})
+    assert has_mqtt_ip_lookup() is True
+
+    setup_core(config={CONF_MQTT: {CONF_BROKER: "mqtt.local", CONF_DISCOVER_IP: True}})
+    assert has_mqtt_ip_lookup() is True
+
+    setup_core(config={CONF_MQTT: {CONF_BROKER: "mqtt.local", CONF_DISCOVER_IP: False}})
+    assert has_mqtt_ip_lookup() is False
+
+
+def test_has_non_ip_address() -> None:
+    """Test has_non_ip_address function."""
+
+    setup_core(address=None)
+    assert has_non_ip_address() is False
+
+    setup_core(address="192.168.1.100")
+    assert has_non_ip_address() is False
+
+    setup_core(address="10.0.0.1")
+    assert has_non_ip_address() is False
+
+    setup_core(address="esphome-device.local")
+    assert has_non_ip_address() is True
+
+    setup_core(address="my-device")
+    assert has_non_ip_address() is True
+
+
+def test_has_ip_address() -> None:
+    """Test has_ip_address function."""
+
+    setup_core(address=None)
+    assert has_ip_address() is False
+
+    setup_core(address="192.168.1.100")
+    assert has_ip_address() is True
+
+    setup_core(address="10.0.0.1")
+    assert has_ip_address() is True
+
+    setup_core(address="esphome-device.local")
+    assert has_ip_address() is False
+
+    setup_core(address="my-device")
+    assert has_ip_address() is False
+
+
+def test_mqtt_get_ip() -> None:
+    """Test mqtt_get_ip function."""
+    config = {CONF_MQTT: {CONF_BROKER: "mqtt.local"}}
+
+    with patch("esphome.mqtt.get_esphome_device_ip") as mock_get_ip:
+        mock_get_ip.return_value = ["192.168.1.100", "192.168.1.101"]
+
+        result = mqtt_get_ip(config, "user", "pass", "client-id")
+
+        assert result == ["192.168.1.100", "192.168.1.101"]
+        mock_get_ip.assert_called_once_with(config, "user", "pass", "client-id")
+
+
+def test_has_resolvable_address() -> None:
+    """Test has_resolvable_address function."""
+
+    # Test with mDNS enabled and .local hostname address
+    setup_core(config={}, address="esphome-device.local")
+    assert has_resolvable_address() is True
+
+    # Test with mDNS disabled and .local hostname address (still resolvable via DNS)
+    setup_core(
+        config={CONF_MDNS: {CONF_DISABLED: True}}, address="esphome-device.local"
+    )
+    assert has_resolvable_address() is True
+
+    # Test with mDNS disabled and regular DNS hostname (resolvable)
+    setup_core(config={CONF_MDNS: {CONF_DISABLED: True}}, address="device.example.com")
+    assert has_resolvable_address() is True
+
+    # Test with IP address (always resolvable, mDNS doesn't matter)
+    setup_core(config={}, address="192.168.1.100")
+    assert has_resolvable_address() is True
+
+    # Test with IP address and mDNS disabled (still resolvable)
+    setup_core(config={CONF_MDNS: {CONF_DISABLED: True}}, address="192.168.1.100")
+    assert has_resolvable_address() is True
+
+    # Test with no address
+    setup_core(config={}, address=None)
+    assert has_resolvable_address() is False
+
+    # Test with no address and mDNS disabled
+    setup_core(config={CONF_MDNS: {CONF_DISABLED: True}}, address=None)
+    assert has_resolvable_address() is False
+
+
+def test_command_wizard(tmp_path: Path) -> None:
+    """Test command_wizard function."""
+    config_file = tmp_path / "test.yaml"
+
+    # Mock wizard.wizard to avoid interactive prompts
+    with patch("esphome.wizard.wizard") as mock_wizard:
+        mock_wizard.return_value = 0
+
+        args = MockArgs(configuration=str(config_file))
+        result = command_wizard(args)
+
+        assert result == 0
+        mock_wizard.assert_called_once_with(config_file)
+
+
+def test_command_rename_invalid_characters(
+    tmp_path: Path, capfd: CaptureFixture[str]
+) -> None:
+    """Test command_rename with invalid characters in name."""
+    setup_core(tmp_path=tmp_path)
+
+    # Test with invalid character (space)
+    args = MockArgs(name="invalid name")
+    result = command_rename(args, {})
+
+    assert result == 1
+    captured = capfd.readouterr()
+    assert "invalid character" in captured.out.lower()
+
+
+def test_command_rename_complex_yaml(
+    tmp_path: Path, capfd: CaptureFixture[str]
+) -> None:
+    """Test command_rename with complex YAML that cannot be renamed."""
+    config_file = tmp_path / "test.yaml"
+    config_file.write_text("# Complex YAML without esphome section\nsome_key: value\n")
+    setup_core(tmp_path=tmp_path)
+    CORE.config_path = config_file
+
+    args = MockArgs(name="newname")
+    result = command_rename(args, {})
+
+    assert result == 1
+    captured = capfd.readouterr()
+    assert "complex yaml" in captured.out.lower()
+
+
+def test_command_rename_success(
+    tmp_path: Path,
+    capfd: CaptureFixture[str],
+    mock_run_external_process: Mock,
+) -> None:
+    """Test successful rename of a simple configuration."""
+    config_file = tmp_path / "oldname.yaml"
+    config_file.write_text("""
+esphome:
+  name: oldname
+
+esp32:
+  board: nodemcu-32s
+
+wifi:
+  ssid: "test"
+  password: "test1234"
+""")
+    setup_core(tmp_path=tmp_path)
+    CORE.config_path = config_file
+
+    # Set up CORE.config to avoid ValueError when accessing CORE.address
+    CORE.config = {CONF_ESPHOME: {CONF_NAME: "oldname"}}
+
+    args = MockArgs(name="newname", dashboard=False)
+
+    # Simulate successful validation and upload
+    mock_run_external_process.return_value = 0
+
+    result = command_rename(args, {})
+
+    assert result == 0
+
+    # Verify new file was created
+    new_file = tmp_path / "newname.yaml"
+    assert new_file.exists()
+
+    # Verify old file was removed
+    assert not config_file.exists()
+
+    # Verify content was updated
+    content = new_file.read_text()
+    assert (
+        'name: "newname"' in content
+        or "name: 'newname'" in content
+        or "name: newname" in content
+    )
+
+    captured = capfd.readouterr()
+    assert "SUCCESS" in captured.out
+
+
+def test_command_rename_with_substitutions(
+    tmp_path: Path,
+    mock_run_external_process: Mock,
+) -> None:
+    """Test rename with substitutions in YAML."""
+    config_file = tmp_path / "oldname.yaml"
+    config_file.write_text("""
+substitutions:
+  device_name: oldname
+
+esphome:
+  name: ${device_name}
+
+esp32:
+  board: nodemcu-32s
+""")
+    setup_core(tmp_path=tmp_path)
+    CORE.config_path = config_file
+
+    # Set up CORE.config to avoid ValueError when accessing CORE.address
+    CORE.config = {
+        CONF_ESPHOME: {CONF_NAME: "oldname"},
+        CONF_SUBSTITUTIONS: {"device_name": "oldname"},
+    }
+
+    args = MockArgs(name="newname", dashboard=False)
+
+    mock_run_external_process.return_value = 0
+
+    result = command_rename(args, {})
+
+    assert result == 0
+
+    # Verify substitution was updated
+    new_file = tmp_path / "newname.yaml"
+    content = new_file.read_text()
+    assert 'device_name: "newname"' in content
+
+
+def test_command_rename_validation_failure(
+    tmp_path: Path,
+    capfd: CaptureFixture[str],
+    mock_run_external_process: Mock,
+) -> None:
+    """Test rename when validation fails."""
+    config_file = tmp_path / "oldname.yaml"
+    config_file.write_text("""
+esphome:
+  name: oldname
+
+esp32:
+  board: nodemcu-32s
+""")
+    setup_core(tmp_path=tmp_path)
+    CORE.config_path = config_file
+
+    args = MockArgs(name="newname", dashboard=False)
+
+    # First call for validation fails
+    mock_run_external_process.return_value = 1
+
+    result = command_rename(args, {})
+
+    assert result == 1
+
+    # Verify new file was created but then removed due to failure
+    new_file = tmp_path / "newname.yaml"
+    assert not new_file.exists()
+
+    # Verify old file still exists (not removed on failure)
+    assert config_file.exists()
+
+    captured = capfd.readouterr()
+    assert "Rename failed" in captured.out
+
+
+def test_command_update_all_path_string_conversion(
+    tmp_path: Path,
+    mock_run_external_process: Mock,
+    capfd: CaptureFixture[str],
+) -> None:
+    """Test that command_update_all properly converts Path objects to strings in output."""
+    yaml1 = tmp_path / "device1.yaml"
+    yaml1.write_text("""
+esphome:
+  name: device1
+
+esp32:
+  board: nodemcu-32s
+""")
+
+    yaml2 = tmp_path / "device2.yaml"
+    yaml2.write_text("""
+esphome:
+  name: device2
+
+esp8266:
+  board: nodemcuv2
+""")
+
+    setup_core(tmp_path=tmp_path)
+    mock_run_external_process.return_value = 0
+
+    assert command_update_all(MockArgs(configuration=[str(tmp_path)])) == 0
+
+    captured = capfd.readouterr()
+    clean_output = strip_ansi_codes(captured.out)
+
+    # Check that Path objects were properly converted to strings
+    # The output should contain file paths without causing TypeError
+    assert "device1.yaml" in clean_output
+    assert "device2.yaml" in clean_output
+    assert "SUCCESS" in clean_output
+    assert "SUMMARY" in clean_output
+
+    # Verify run_external_process was called for each file
+    assert mock_run_external_process.call_count == 2
+
+
+def test_command_update_all_with_failures(
+    tmp_path: Path,
+    mock_run_external_process: Mock,
+    capfd: CaptureFixture[str],
+) -> None:
+    """Test command_update_all handles mixed success/failure cases properly."""
+    yaml1 = tmp_path / "success_device.yaml"
+    yaml1.write_text("""
+esphome:
+  name: success_device
+
+esp32:
+  board: nodemcu-32s
+""")
+
+    yaml2 = tmp_path / "failed_device.yaml"
+    yaml2.write_text("""
+esphome:
+  name: failed_device
+
+esp8266:
+  board: nodemcuv2
+""")
+
+    setup_core(tmp_path=tmp_path)
+
+    # Mock mixed results - first succeeds, second fails
+    mock_run_external_process.side_effect = [0, 1]
+
+    # Should return 1 (failure) since one device failed
+    assert command_update_all(MockArgs(configuration=[str(tmp_path)])) == 1
+
+    captured = capfd.readouterr()
+    clean_output = strip_ansi_codes(captured.out)
+
+    # Check that both success and failure are properly displayed
+    assert "SUCCESS" in clean_output
+    assert "ERROR" in clean_output or "FAILED" in clean_output
+    assert "SUMMARY" in clean_output
+
+    # Files are processed in alphabetical order, so we need to check which one succeeded/failed
+    # The mock_run_external_process.side_effect = [0, 1] applies to files in alphabetical order
+    # So "failed_device.yaml" gets 0 (success) and "success_device.yaml" gets 1 (failure)
+    assert "failed_device.yaml: SUCCESS" in clean_output
+    assert "success_device.yaml: FAILED" in clean_output
+
+
+def test_command_update_all_empty_directory(
+    tmp_path: Path,
+    mock_run_external_process: Mock,
+    capfd: CaptureFixture[str],
+) -> None:
+    """Test command_update_all with an empty directory (no YAML files)."""
+    setup_core(tmp_path=tmp_path)
+
+    assert command_update_all(MockArgs(configuration=[str(tmp_path)])) == 0
+    mock_run_external_process.assert_not_called()
+
+    captured = capfd.readouterr()
+    clean_output = strip_ansi_codes(captured.out)
+
+    assert "SUMMARY" in clean_output
+
+
+def test_command_update_all_single_file(
+    tmp_path: Path,
+    mock_run_external_process: Mock,
+    capfd: CaptureFixture[str],
+) -> None:
+    """Test command_update_all with a single YAML file specified."""
+    yaml_file = tmp_path / "single_device.yaml"
+    yaml_file.write_text("""
+esphome:
+  name: single_device
+
+esp32:
+  board: nodemcu-32s
+""")
+
+    setup_core(tmp_path=tmp_path)
+    mock_run_external_process.return_value = 0
+
+    assert command_update_all(MockArgs(configuration=[str(yaml_file)])) == 0
+
+    captured = capfd.readouterr()
+    clean_output = strip_ansi_codes(captured.out)
+
+    assert "single_device.yaml" in clean_output
+    assert "SUCCESS" in clean_output
+    mock_run_external_process.assert_called_once()
+
+
+def test_command_update_all_path_formatting_in_color_calls(
+    tmp_path: Path,
+    mock_run_external_process: Mock,
+    capfd: CaptureFixture[str],
+) -> None:
+    """Test that Path objects are properly converted when passed to color() function."""
+    yaml_file = tmp_path / "test-device_123.yaml"
+    yaml_file.write_text("""
+esphome:
+  name: test-device_123
+
+esp32:
+  board: nodemcu-32s
+""")
+
+    setup_core(tmp_path=tmp_path)
+    mock_run_external_process.return_value = 0
+
+    assert command_update_all(MockArgs(configuration=[str(tmp_path)])) == 0
+
+    captured = capfd.readouterr()
+    clean_output = strip_ansi_codes(captured.out)
+
+    assert "test-device_123.yaml" in clean_output
+    assert "Updating" in clean_output
+    assert "SUCCESS" in clean_output
+    assert "SUMMARY" in clean_output
+
+    # Should not have any Python error messages
+    assert "TypeError" not in clean_output
+    assert "can only concatenate str" not in clean_output
+
+
+def test_command_clean_all_success(
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test command_clean_all when writer.clean_all() succeeds."""
+    args = MockArgs(configuration=["/path/to/config1", "/path/to/config2"])
+
+    # Set logger level to capture INFO messages
+    with (
+        caplog.at_level(logging.INFO),
+        patch("esphome.writer.clean_all") as mock_clean_all,
+    ):
+        result = command_clean_all(args)
+
+        assert result == 0
+        mock_clean_all.assert_called_once_with(["/path/to/config1", "/path/to/config2"])
+
+        # Check that success message was logged
+        assert "Done!" in caplog.text
+
+
+def test_command_clean_all_oserror(
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test command_clean_all when writer.clean_all() raises OSError."""
+    args = MockArgs(configuration=["/path/to/config1"])
+
+    # Create a mock OSError with a specific message
+    mock_error = OSError("Permission denied: cannot delete directory")
+
+    # Set logger level to capture ERROR and INFO messages
+    with (
+        caplog.at_level(logging.INFO),
+        patch("esphome.writer.clean_all", side_effect=mock_error) as mock_clean_all,
+    ):
+        result = command_clean_all(args)
+
+        assert result == 1
+        mock_clean_all.assert_called_once_with(["/path/to/config1"])
+
+        # Check that error message was logged
+        assert (
+            "Error cleaning all files: Permission denied: cannot delete directory"
+            in caplog.text
+        )
+        # Should not have success message
+        assert "Done!" not in caplog.text
+
+
+def test_command_clean_all_oserror_no_message(
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test command_clean_all when writer.clean_all() raises OSError without message."""
+    args = MockArgs(configuration=["/path/to/config1"])
+
+    # Create a mock OSError without a message
+    mock_error = OSError()
+
+    # Set logger level to capture ERROR and INFO messages
+    with (
+        caplog.at_level(logging.INFO),
+        patch("esphome.writer.clean_all", side_effect=mock_error) as mock_clean_all,
+    ):
+        result = command_clean_all(args)
+
+        assert result == 1
+        mock_clean_all.assert_called_once_with(["/path/to/config1"])
+
+        # Check that error message was logged (should show empty string for OSError without message)
+        assert "Error cleaning all files:" in caplog.text
+        # Should not have success message
+        assert "Done!" not in caplog.text
+
+
+def test_command_clean_all_args_used() -> None:
+    """Test that command_clean_all uses args.configuration parameter."""
+    # Test with different configuration paths
+    args1 = MockArgs(configuration=["/path/to/config1"])
+    args2 = MockArgs(configuration=["/path/to/config2", "/path/to/config3"])
+
+    with patch("esphome.writer.clean_all") as mock_clean_all:
+        result1 = command_clean_all(args1)
+        result2 = command_clean_all(args2)
+
+        assert result1 == 0
+        assert result2 == 0
+        assert mock_clean_all.call_count == 2
+
+        # Verify the correct configuration paths were passed
+        mock_clean_all.assert_any_call(["/path/to/config1"])
+        mock_clean_all.assert_any_call(["/path/to/config2", "/path/to/config3"])
+
+
+def test_upload_program_ota_static_ip_with_mqttip(
+    mock_mqtt_get_ip: Mock,
+    mock_run_ota: Mock,
+    tmp_path: Path,
+) -> None:
+    """Test upload_program with static IP and MQTTIP (issue #11260).
+
+    This tests the scenario where a device has manual_ip (static IP) configured
+    and MQTT is also configured. The devices list contains both the static IP
+    and "MQTTIP" magic string. This previously failed because only the first
+    device was checked for MQTT resolution.
+    """
+    setup_core(platform=PLATFORM_ESP32, tmp_path=tmp_path)
+
+    mock_mqtt_get_ip.return_value = ["192.168.2.50"]  # Different subnet
+    mock_run_ota.return_value = (0, "192.168.1.100")
+
+    config = {
+        CONF_OTA: [
+            {
+                CONF_PLATFORM: CONF_ESPHOME,
+                CONF_PORT: 3232,
+            }
+        ],
+        CONF_MQTT: {
+            CONF_BROKER: "mqtt.local",
+        },
+    }
+    args = MockArgs(username="user", password="pass", client_id="client")
+    # Simulates choose_upload_log_host returning static IP + MQTTIP
+    devices = ["192.168.1.100", "MQTTIP"]
+
+    exit_code, host = upload_program(config, args, devices)
+
+    assert exit_code == 0
+    assert host == "192.168.1.100"
+
+    # Verify MQTT was resolved
+    mock_mqtt_get_ip.assert_called_once_with(config, "user", "pass", "client")
+
+    # Verify espota2.run_ota was called with both IPs
+    expected_firmware = (
+        tmp_path / ".esphome" / "build" / "test" / ".pioenvs" / "test" / "firmware.bin"
+    )
+    mock_run_ota.assert_called_once_with(
+        ["192.168.1.100", "192.168.2.50"], 3232, None, expected_firmware
+    )
+
+
+def test_upload_program_ota_multiple_mqttip_resolves_once(
+    mock_mqtt_get_ip: Mock,
+    mock_run_ota: Mock,
+    tmp_path: Path,
+) -> None:
+    """Test that MQTT resolution only happens once even with multiple MQTT magic strings."""
+    setup_core(platform=PLATFORM_ESP32, tmp_path=tmp_path)
+
+    mock_mqtt_get_ip.return_value = ["192.168.2.50", "192.168.2.51"]
+    mock_run_ota.return_value = (0, "192.168.2.50")
+
+    config = {
+        CONF_OTA: [
+            {
+                CONF_PLATFORM: CONF_ESPHOME,
+                CONF_PORT: 3232,
+            }
+        ],
+        CONF_MQTT: {
+            CONF_BROKER: "mqtt.local",
+        },
+    }
+    args = MockArgs(username="user", password="pass", client_id="client")
+    # Multiple MQTT magic strings in the list
+    devices = ["MQTTIP", "MQTT", "192.168.1.100"]
+
+    exit_code, host = upload_program(config, args, devices)
+
+    assert exit_code == 0
+    assert host == "192.168.2.50"
+
+    # Verify MQTT was only resolved once despite multiple MQTT magic strings
+    mock_mqtt_get_ip.assert_called_once_with(config, "user", "pass", "client")
+
+    # Verify espota2.run_ota was called with all unique IPs
+    expected_firmware = (
+        tmp_path / ".esphome" / "build" / "test" / ".pioenvs" / "test" / "firmware.bin"
+    )
+    mock_run_ota.assert_called_once_with(
+        ["192.168.2.50", "192.168.2.51", "192.168.1.100"], 3232, None, expected_firmware
+    )
+
+
+def test_upload_program_ota_mqttip_deduplication(
+    mock_mqtt_get_ip: Mock,
+    mock_run_ota: Mock,
+    tmp_path: Path,
+) -> None:
+    """Test that duplicate IPs are filtered when MQTT returns same IP as static IP."""
+    setup_core(platform=PLATFORM_ESP32, tmp_path=tmp_path)
+
+    # MQTT returns the same IP as the static IP
+    mock_mqtt_get_ip.return_value = ["192.168.1.100"]
+    mock_run_ota.return_value = (0, "192.168.1.100")
+
+    config = {
+        CONF_OTA: [
+            {
+                CONF_PLATFORM: CONF_ESPHOME,
+                CONF_PORT: 3232,
+            }
+        ],
+        CONF_MQTT: {
+            CONF_BROKER: "mqtt.local",
+        },
+    }
+    args = MockArgs(username="user", password="pass", client_id="client")
+    devices = ["192.168.1.100", "MQTTIP"]
+
+    exit_code, host = upload_program(config, args, devices)
+
+    assert exit_code == 0
+    assert host == "192.168.1.100"
+
+    # Verify MQTT was resolved
+    mock_mqtt_get_ip.assert_called_once_with(config, "user", "pass", "client")
+
+    # Verify espota2.run_ota was called with deduplicated IPs (only one instance of 192.168.1.100)
+    # Note: Current implementation doesn't dedupe, so we'll get the IP twice
+    # This test documents current behavior - deduplication could be future enhancement
+    mock_run_ota.assert_called_once()
+    call_args = mock_run_ota.call_args[0]
+    # Should contain both the original IP and MQTT-resolved IP (even if duplicate)
+    assert "192.168.1.100" in call_args[0]
+
+
+@patch("esphome.components.api.client.run_logs")
+def test_show_logs_api_static_ip_with_mqttip(
+    mock_run_logs: Mock,
+    mock_mqtt_get_ip: Mock,
+) -> None:
+    """Test show_logs with static IP and MQTTIP (issue #11260).
+
+    This tests the scenario where a device has manual_ip (static IP) configured
+    and MQTT is also configured. The devices list contains both the static IP
+    and "MQTTIP" magic string.
+    """
+    setup_core(
+        config={
+            "logger": {},
+            CONF_API: {},
+            CONF_MQTT: {CONF_BROKER: "mqtt.local"},
+        },
+        platform=PLATFORM_ESP32,
+    )
+    mock_run_logs.return_value = 0
+    mock_mqtt_get_ip.return_value = ["192.168.2.50"]
+
+    args = MockArgs(username="user", password="pass", client_id="client")
+    # Simulates choose_upload_log_host returning static IP + MQTTIP
+    devices = ["192.168.1.100", "MQTTIP"]
+
+    result = show_logs(CORE.config, args, devices)
+
+    assert result == 0
+
+    # Verify MQTT was resolved
+    mock_mqtt_get_ip.assert_called_once_with(CORE.config, "user", "pass", "client")
+
+    # Verify run_logs was called with both IPs
+    mock_run_logs.assert_called_once_with(
+        CORE.config, ["192.168.1.100", "192.168.2.50"]
+    )
+
+
+@patch("esphome.components.api.client.run_logs")
+def test_show_logs_api_multiple_mqttip_resolves_once(
+    mock_run_logs: Mock,
+    mock_mqtt_get_ip: Mock,
+) -> None:
+    """Test that MQTT resolution only happens once for show_logs with multiple MQTT magic strings."""
+    setup_core(
+        config={
+            "logger": {},
+            CONF_API: {},
+            CONF_MQTT: {CONF_BROKER: "mqtt.local"},
+        },
+        platform=PLATFORM_ESP32,
+    )
+    mock_run_logs.return_value = 0
+    mock_mqtt_get_ip.return_value = ["192.168.2.50", "192.168.2.51"]
+
+    args = MockArgs(username="user", password="pass", client_id="client")
+    # Multiple MQTT magic strings in the list
+    devices = ["MQTTIP", "192.168.1.100", "MQTT"]
+
+    result = show_logs(CORE.config, args, devices)
+
+    assert result == 0
+
+    # Verify MQTT was only resolved once despite multiple MQTT magic strings
+    mock_mqtt_get_ip.assert_called_once_with(CORE.config, "user", "pass", "client")
+
+    # Verify run_logs was called with all unique IPs (MQTT strings replaced with IPs)
+    # Note: "MQTT" is a different magic string from "MQTTIP", but both trigger MQTT resolution
+    # The _resolve_network_devices helper filters out both after first resolution
+    mock_run_logs.assert_called_once_with(
+        CORE.config, ["192.168.2.50", "192.168.2.51", "192.168.1.100"]
+    )
+
+
+def test_upload_program_ota_mqtt_timeout_fallback(
+    mock_mqtt_get_ip: Mock,
+    mock_run_ota: Mock,
+    tmp_path: Path,
+) -> None:
+    """Test upload_program falls back to other devices when MQTT times out."""
+    setup_core(platform=PLATFORM_ESP32, tmp_path=tmp_path)
+
+    # MQTT times out
+    mock_mqtt_get_ip.side_effect = EsphomeError("Failed to find IP via MQTT")
+    mock_run_ota.return_value = (0, "192.168.1.100")
+
+    config = {
+        CONF_OTA: [
+            {
+                CONF_PLATFORM: CONF_ESPHOME,
+                CONF_PORT: 3232,
+            }
+        ],
+        CONF_MQTT: {
+            CONF_BROKER: "mqtt.local",
+        },
+    }
+    args = MockArgs(username="user", password="pass", client_id="client")
+    # Static IP first, MQTTIP second
+    devices = ["192.168.1.100", "MQTTIP"]
+
+    exit_code, host = upload_program(config, args, devices)
+
+    # Should succeed using the static IP even though MQTT failed
+    assert exit_code == 0
+    assert host == "192.168.1.100"
+
+    # Verify MQTT was attempted
+    mock_mqtt_get_ip.assert_called_once_with(config, "user", "pass", "client")
+
+    # Verify espota2.run_ota was called with only the static IP (MQTT failed)
+    expected_firmware = (
+        tmp_path / ".esphome" / "build" / "test" / ".pioenvs" / "test" / "firmware.bin"
+    )
+    mock_run_ota.assert_called_once_with(
+        ["192.168.1.100"], 3232, None, expected_firmware
+    )
+
+
+@patch("esphome.components.api.client.run_logs")
+def test_show_logs_api_mqtt_timeout_fallback(
+    mock_run_logs: Mock,
+    mock_mqtt_get_ip: Mock,
+) -> None:
+    """Test show_logs falls back to other devices when MQTT times out."""
+    setup_core(
+        config={
+            "logger": {},
+            CONF_API: {},
+            CONF_MQTT: {CONF_BROKER: "mqtt.local"},
+        },
+        platform=PLATFORM_ESP32,
+    )
+    mock_run_logs.return_value = 0
+    # MQTT times out
+    mock_mqtt_get_ip.side_effect = EsphomeError("Failed to find IP via MQTT")
+
+    args = MockArgs(username="user", password="pass", client_id="client")
+    # Static IP first, MQTTIP second
+    devices = ["192.168.1.100", "MQTTIP"]
+
+    result = show_logs(CORE.config, args, devices)
+
+    # Should succeed using the static IP even though MQTT failed
+    assert result == 0
+
+    # Verify MQTT was attempted
+    mock_mqtt_get_ip.assert_called_once_with(CORE.config, "user", "pass", "client")
+
+    # Verify run_logs was called with only the static IP (MQTT failed)
+    mock_run_logs.assert_called_once_with(CORE.config, ["192.168.1.100"])
diff --git a/tests/unit_tests/test_platformio_api.py b/tests/unit_tests/test_platformio_api.py
new file mode 100644
index 0000000000..07948cc6ad
--- /dev/null
+++ b/tests/unit_tests/test_platformio_api.py
@@ -0,0 +1,636 @@
+"""Tests for platformio_api.py path functions."""
+
+import json
+import os
+from pathlib import Path
+import shutil
+from types import SimpleNamespace
+from unittest.mock import MagicMock, Mock, patch
+
+import pytest
+
+from esphome import platformio_api
+from esphome.core import CORE, EsphomeError
+
+
+def test_idedata_firmware_elf_path(setup_core: Path) -> None:
+    """Test IDEData.firmware_elf_path returns correct path."""
+    CORE.build_path = setup_core / "build" / "test"
+    CORE.name = "test"
+    raw_data = {"prog_path": "/path/to/firmware.elf"}
+    idedata = platformio_api.IDEData(raw_data)
+
+    assert idedata.firmware_elf_path == Path("/path/to/firmware.elf")
+
+
+def test_idedata_firmware_bin_path(setup_core: Path) -> None:
+    """Test IDEData.firmware_bin_path returns Path with .bin extension."""
+    CORE.build_path = setup_core / "build" / "test"
+    CORE.name = "test"
+    prog_path = str(Path("/path/to/firmware.elf"))
+    raw_data = {"prog_path": prog_path}
+    idedata = platformio_api.IDEData(raw_data)
+
+    result = idedata.firmware_bin_path
+    assert isinstance(result, Path)
+    expected = Path("/path/to/firmware.bin")
+    assert result == expected
+    assert str(result).endswith(".bin")
+
+
+def test_idedata_firmware_bin_path_preserves_directory(setup_core: Path) -> None:
+    """Test firmware_bin_path preserves the directory structure."""
+    CORE.build_path = setup_core / "build" / "test"
+    CORE.name = "test"
+    prog_path = str(Path("/complex/path/to/build/firmware.elf"))
+    raw_data = {"prog_path": prog_path}
+    idedata = platformio_api.IDEData(raw_data)
+
+    result = idedata.firmware_bin_path
+    expected = Path("/complex/path/to/build/firmware.bin")
+    assert result == expected
+
+
+def test_idedata_extra_flash_images(setup_core: Path) -> None:
+    """Test IDEData.extra_flash_images returns list of FlashImage objects."""
+    CORE.build_path = setup_core / "build" / "test"
+    CORE.name = "test"
+    raw_data = {
+        "prog_path": "/path/to/firmware.elf",
+        "extra": {
+            "flash_images": [
+                {"path": "/path/to/bootloader.bin", "offset": "0x1000"},
+                {"path": "/path/to/partition.bin", "offset": "0x8000"},
+            ]
+        },
+    }
+    idedata = platformio_api.IDEData(raw_data)
+
+    images = idedata.extra_flash_images
+    assert len(images) == 2
+    assert all(isinstance(img, platformio_api.FlashImage) for img in images)
+    assert images[0].path == Path("/path/to/bootloader.bin")
+    assert images[0].offset == "0x1000"
+    assert images[1].path == Path("/path/to/partition.bin")
+    assert images[1].offset == "0x8000"
+
+
+def test_idedata_extra_flash_images_empty(setup_core: Path) -> None:
+    """Test extra_flash_images returns empty list when no extra images."""
+    CORE.build_path = setup_core / "build" / "test"
+    CORE.name = "test"
+    raw_data = {"prog_path": "/path/to/firmware.elf", "extra": {"flash_images": []}}
+    idedata = platformio_api.IDEData(raw_data)
+
+    images = idedata.extra_flash_images
+    assert images == []
+
+
+def test_idedata_cc_path(setup_core: Path) -> None:
+    """Test IDEData.cc_path returns compiler path."""
+    CORE.build_path = setup_core / "build" / "test"
+    CORE.name = "test"
+    raw_data = {
+        "prog_path": "/path/to/firmware.elf",
+        "cc_path": "/Users/test/.platformio/packages/toolchain-xtensa32/bin/xtensa-esp32-elf-gcc",
+    }
+    idedata = platformio_api.IDEData(raw_data)
+
+    assert (
+        idedata.cc_path
+        == "/Users/test/.platformio/packages/toolchain-xtensa32/bin/xtensa-esp32-elf-gcc"
+    )
+
+
+def test_flash_image_dataclass() -> None:
+    """Test FlashImage dataclass stores path and offset correctly."""
+    image = platformio_api.FlashImage(path=Path("/path/to/image.bin"), offset="0x10000")
+
+    assert image.path == Path("/path/to/image.bin")
+    assert image.offset == "0x10000"
+
+
+def test_load_idedata_returns_dict(
+    setup_core: Path, mock_run_platformio_cli_run
+) -> None:
+    """Test _load_idedata returns parsed idedata dict when successful."""
+    CORE.build_path = setup_core / "build" / "test"
+    CORE.name = "test"
+
+    # Create required files
+    platformio_ini = setup_core / "build" / "test" / "platformio.ini"
+    platformio_ini.parent.mkdir(parents=True, exist_ok=True)
+    platformio_ini.touch()
+
+    idedata_path = setup_core / ".esphome" / "idedata" / "test.json"
+    idedata_path.parent.mkdir(parents=True, exist_ok=True)
+    idedata_path.write_text('{"prog_path": "/test/firmware.elf"}')
+
+    mock_run_platformio_cli_run.return_value = '{"prog_path": "/test/firmware.elf"}'
+
+    config = {"name": "test"}
+    result = platformio_api._load_idedata(config)
+
+    assert result is not None
+    assert isinstance(result, dict)
+    assert result["prog_path"] == "/test/firmware.elf"
+
+
+def test_load_idedata_uses_cache_when_valid(
+    setup_core: Path, mock_run_platformio_cli_run: Mock
+) -> None:
+    """Test _load_idedata uses cached data when unchanged."""
+    CORE.build_path = str(setup_core / "build" / "test")
+    CORE.name = "test"
+
+    # Create platformio.ini
+    platformio_ini = setup_core / "build" / "test" / "platformio.ini"
+    platformio_ini.parent.mkdir(parents=True, exist_ok=True)
+    platformio_ini.write_text("content")
+
+    # Create idedata cache file that's newer
+    idedata_path = setup_core / ".esphome" / "idedata" / "test.json"
+    idedata_path.parent.mkdir(parents=True, exist_ok=True)
+    idedata_path.write_text('{"prog_path": "/cached/firmware.elf"}')
+
+    # Make idedata newer than platformio.ini
+    platformio_ini_mtime = platformio_ini.stat().st_mtime
+    os.utime(idedata_path, (platformio_ini_mtime + 1, platformio_ini_mtime + 1))
+
+    config = {"name": "test"}
+    result = platformio_api._load_idedata(config)
+
+    # Should not call _run_idedata since cache is valid
+    mock_run_platformio_cli_run.assert_not_called()
+
+    assert result["prog_path"] == "/cached/firmware.elf"
+
+
+def test_load_idedata_regenerates_when_platformio_ini_newer(
+    setup_core: Path, mock_run_platformio_cli_run: Mock
+) -> None:
+    """Test _load_idedata regenerates when platformio.ini is newer."""
+    CORE.build_path = str(setup_core / "build" / "test")
+    CORE.name = "test"
+
+    # Create idedata cache file first
+    idedata_path = setup_core / ".esphome" / "idedata" / "test.json"
+    idedata_path.parent.mkdir(parents=True, exist_ok=True)
+    idedata_path.write_text('{"prog_path": "/old/firmware.elf"}')
+
+    # Create platformio.ini that's newer
+    idedata_mtime = idedata_path.stat().st_mtime
+    platformio_ini = setup_core / "build" / "test" / "platformio.ini"
+    platformio_ini.parent.mkdir(parents=True, exist_ok=True)
+    platformio_ini.write_text("content")
+    # Make platformio.ini newer than idedata
+    os.utime(platformio_ini, (idedata_mtime + 1, idedata_mtime + 1))
+
+    # Mock platformio to return new data
+    new_data = {"prog_path": "/new/firmware.elf"}
+    mock_run_platformio_cli_run.return_value = json.dumps(new_data)
+
+    config = {"name": "test"}
+    result = platformio_api._load_idedata(config)
+
+    # Should call _run_idedata since platformio.ini is newer
+    mock_run_platformio_cli_run.assert_called_once()
+
+    assert result["prog_path"] == "/new/firmware.elf"
+
+
+def test_load_idedata_regenerates_on_corrupted_cache(
+    setup_core: Path, mock_run_platformio_cli_run: Mock
+) -> None:
+    """Test _load_idedata regenerates when cache file is corrupted."""
+    CORE.build_path = str(setup_core / "build" / "test")
+    CORE.name = "test"
+
+    # Create platformio.ini
+    platformio_ini = setup_core / "build" / "test" / "platformio.ini"
+    platformio_ini.parent.mkdir(parents=True, exist_ok=True)
+    platformio_ini.write_text("content")
+
+    # Create corrupted idedata cache file
+    idedata_path = setup_core / ".esphome" / "idedata" / "test.json"
+    idedata_path.parent.mkdir(parents=True, exist_ok=True)
+    idedata_path.write_text('{"prog_path": invalid json')
+
+    # Make idedata newer so it would be used if valid
+    platformio_ini_mtime = platformio_ini.stat().st_mtime
+    os.utime(idedata_path, (platformio_ini_mtime + 1, platformio_ini_mtime + 1))
+
+    # Mock platformio to return new data
+    new_data = {"prog_path": "/new/firmware.elf"}
+    mock_run_platformio_cli_run.return_value = json.dumps(new_data)
+
+    config = {"name": "test"}
+    result = platformio_api._load_idedata(config)
+
+    # Should call _run_idedata since cache is corrupted
+    mock_run_platformio_cli_run.assert_called_once()
+
+    assert result["prog_path"] == "/new/firmware.elf"
+
+
+def test_run_idedata_parses_json_from_output(
+    setup_core: Path, mock_run_platformio_cli_run: Mock
+) -> None:
+    """Test _run_idedata extracts JSON from platformio output."""
+    config = {"name": "test"}
+
+    expected_data = {
+        "prog_path": "/path/to/firmware.elf",
+        "cc_path": "/path/to/gcc",
+        "extra": {"flash_images": []},
+    }
+
+    # Simulate platformio output with JSON embedded
+    mock_run_platformio_cli_run.return_value = (
+        f"Some preamble\n{json.dumps(expected_data)}\nSome postamble"
+    )
+
+    result = platformio_api._run_idedata(config)
+
+    assert result == expected_data
+
+
+def test_run_idedata_raises_on_no_json(
+    setup_core: Path, mock_run_platformio_cli_run: Mock
+) -> None:
+    """Test _run_idedata raises EsphomeError when no JSON found."""
+    config = {"name": "test"}
+
+    mock_run_platformio_cli_run.return_value = "No JSON in this output"
+
+    with pytest.raises(EsphomeError):
+        platformio_api._run_idedata(config)
+
+
+def test_run_idedata_raises_on_invalid_json(
+    setup_core: Path, mock_run_platformio_cli_run: Mock
+) -> None:
+    """Test _run_idedata raises on malformed JSON."""
+    config = {"name": "test"}
+    mock_run_platformio_cli_run.return_value = '{"invalid": json"}'
+
+    # The ValueError from json.loads is re-raised
+    with pytest.raises(ValueError):
+        platformio_api._run_idedata(config)
+
+
+def test_run_platformio_cli_sets_environment_variables(
+    setup_core: Path, mock_run_external_command: Mock
+) -> None:
+    """Test run_platformio_cli sets correct environment variables."""
+    CORE.build_path = str(setup_core / "build" / "test")
+
+    with patch.dict(os.environ, {}, clear=False):
+        mock_run_external_command.return_value = 0
+        platformio_api.run_platformio_cli("test", "arg")
+
+        # Check environment variables were set
+        assert os.environ["PLATFORMIO_FORCE_COLOR"] == "true"
+        assert (
+            setup_core / "build" / "test"
+            in Path(os.environ["PLATFORMIO_BUILD_DIR"]).parents
+            or Path(os.environ["PLATFORMIO_BUILD_DIR"]) == setup_core / "build" / "test"
+        )
+        assert "PLATFORMIO_LIBDEPS_DIR" in os.environ
+        assert "PYTHONWARNINGS" in os.environ
+
+        # Check command was called correctly
+        mock_run_external_command.assert_called_once()
+        args = mock_run_external_command.call_args[0]
+        assert "platformio" in args
+        assert "test" in args
+        assert "arg" in args
+
+
+def test_run_platformio_cli_run_builds_command(
+    setup_core: Path, mock_run_platformio_cli: Mock
+) -> None:
+    """Test run_platformio_cli_run builds correct command."""
+    CORE.build_path = str(setup_core / "build" / "test")
+    mock_run_platformio_cli.return_value = 0
+
+    config = {"name": "test"}
+    platformio_api.run_platformio_cli_run(config, True, "extra", "args")
+
+    mock_run_platformio_cli.assert_called_once_with(
+        "run", "-d", CORE.build_path, "-v", "extra", "args"
+    )
+
+
+def test_run_compile(setup_core: Path, mock_run_platformio_cli_run: Mock) -> None:
+    """Test run_compile with process limit."""
+    from esphome.const import CONF_COMPILE_PROCESS_LIMIT, CONF_ESPHOME
+
+    CORE.build_path = str(setup_core / "build" / "test")
+    config = {CONF_ESPHOME: {CONF_COMPILE_PROCESS_LIMIT: 4}}
+    mock_run_platformio_cli_run.return_value = 0
+
+    platformio_api.run_compile(config, verbose=True)
+
+    mock_run_platformio_cli_run.assert_called_once_with(config, True, "-j4")
+
+
+def test_get_idedata_caches_result(
+    setup_core: Path, mock_run_platformio_cli_run: Mock
+) -> None:
+    """Test get_idedata caches result in CORE.data."""
+    from esphome.const import KEY_CORE
+
+    CORE.build_path = str(setup_core / "build" / "test")
+    CORE.name = "test"
+    CORE.data[KEY_CORE] = {}
+
+    # Create platformio.ini to avoid regeneration
+    platformio_ini = setup_core / "build" / "test" / "platformio.ini"
+    platformio_ini.parent.mkdir(parents=True, exist_ok=True)
+    platformio_ini.write_text("content")
+
+    # Mock platformio to return data
+    idedata = {"prog_path": "/test/firmware.elf"}
+    mock_run_platformio_cli_run.return_value = json.dumps(idedata)
+
+    config = {"name": "test"}
+
+    # First call should load and cache
+    result1 = platformio_api.get_idedata(config)
+    mock_run_platformio_cli_run.assert_called_once()
+
+    # Second call should use cache from CORE.data
+    result2 = platformio_api.get_idedata(config)
+    mock_run_platformio_cli_run.assert_called_once()  # Still only called once
+
+    assert result1 is result2
+    assert isinstance(result1, platformio_api.IDEData)
+    assert result1.firmware_elf_path == Path("/test/firmware.elf")
+
+
+def test_idedata_addr2line_path_windows(setup_core: Path) -> None:
+    """Test IDEData.addr2line_path on Windows."""
+    raw_data = {"prog_path": "/path/to/firmware.elf", "cc_path": "C:\\tools\\gcc.exe"}
+    idedata = platformio_api.IDEData(raw_data)
+
+    result = idedata.addr2line_path
+    assert result == "C:\\tools\\addr2line.exe"
+
+
+def test_idedata_addr2line_path_unix(setup_core: Path) -> None:
+    """Test IDEData.addr2line_path on Unix."""
+    raw_data = {"prog_path": "/path/to/firmware.elf", "cc_path": "/usr/bin/gcc"}
+    idedata = platformio_api.IDEData(raw_data)
+
+    result = idedata.addr2line_path
+    assert result == "/usr/bin/addr2line"
+
+
+def test_patch_structhash(setup_core: Path) -> None:
+    """Test patch_structhash monkey patches platformio functions."""
+    # Create simple namespace objects to act as modules
+    mock_cli = SimpleNamespace()
+    mock_helpers = SimpleNamespace()
+    mock_run = SimpleNamespace(cli=mock_cli, helpers=mock_helpers)
+
+    # Mock platformio modules
+    with patch.dict(
+        "sys.modules",
+        {
+            "platformio.run.cli": mock_cli,
+            "platformio.run.helpers": mock_helpers,
+            "platformio.run": mock_run,
+            "platformio.project.helpers": MagicMock(),
+            "platformio.fs": MagicMock(),
+            "platformio": MagicMock(),
+        },
+    ):
+        # Call patch_structhash
+        platformio_api.patch_structhash()
+
+        # Verify both modules had clean_build_dir patched
+        # Check that clean_build_dir was set on both modules
+        assert hasattr(mock_cli, "clean_build_dir")
+        assert hasattr(mock_helpers, "clean_build_dir")
+
+        # Verify they got the same function assigned
+        assert mock_cli.clean_build_dir is mock_helpers.clean_build_dir
+
+        # Verify it's a real function (not a Mock)
+        assert callable(mock_cli.clean_build_dir)
+        assert mock_cli.clean_build_dir.__name__ == "patched_clean_build_dir"
+
+
+def test_patched_clean_build_dir_removes_outdated(setup_core: Path) -> None:
+    """Test patched_clean_build_dir removes build dir when platformio.ini is newer."""
+    build_dir = setup_core / "build"
+    build_dir.mkdir()
+    platformio_ini = setup_core / "platformio.ini"
+    platformio_ini.write_text("config")
+
+    # Make platformio.ini newer than build_dir
+    build_mtime = build_dir.stat().st_mtime
+    os.utime(platformio_ini, (build_mtime + 1, build_mtime + 1))
+
+    # Track if directory was removed
+    removed_paths: list[Path] = []
+
+    def track_rmtree(path: Path) -> None:
+        removed_paths.append(path)
+        shutil.rmtree(path)
+
+    # Create mock modules that patch_structhash expects
+    mock_cli = SimpleNamespace()
+    mock_helpers = SimpleNamespace()
+    mock_project_helpers = MagicMock()
+    mock_project_helpers.get_project_dir.return_value = str(setup_core)
+    mock_fs = SimpleNamespace(rmtree=track_rmtree)
+
+    with patch.dict(
+        "sys.modules",
+        {
+            "platformio": SimpleNamespace(fs=mock_fs),
+            "platformio.fs": mock_fs,
+            "platformio.project.helpers": mock_project_helpers,
+            "platformio.run": SimpleNamespace(cli=mock_cli, helpers=mock_helpers),
+            "platformio.run.cli": mock_cli,
+            "platformio.run.helpers": mock_helpers,
+        },
+    ):
+        # Call patch_structhash to install the patched function
+        platformio_api.patch_structhash()
+
+        # Call the patched function
+        mock_helpers.clean_build_dir(str(build_dir), [])
+
+        # Verify directory was removed and recreated
+        assert len(removed_paths) == 1
+        assert removed_paths[0] == build_dir
+        assert build_dir.exists()  # makedirs recreated it
+
+
+def test_patched_clean_build_dir_keeps_updated(setup_core: Path) -> None:
+    """Test patched_clean_build_dir keeps build dir when it's up to date."""
+    build_dir = setup_core / "build"
+    build_dir.mkdir()
+    test_file = build_dir / "test.txt"
+    test_file.write_text("test content")
+
+    platformio_ini = setup_core / "platformio.ini"
+    platformio_ini.write_text("config")
+
+    # Make build_dir newer than platformio.ini
+    ini_mtime = platformio_ini.stat().st_mtime
+    os.utime(build_dir, (ini_mtime + 1, ini_mtime + 1))
+
+    # Track if rmtree is called
+    removed_paths: list[str] = []
+
+    def track_rmtree(path: str) -> None:
+        removed_paths.append(path)
+
+    # Create mock modules
+    mock_cli = SimpleNamespace()
+    mock_helpers = SimpleNamespace()
+    mock_project_helpers = MagicMock()
+    mock_project_helpers.get_project_dir.return_value = str(setup_core)
+    mock_fs = SimpleNamespace(rmtree=track_rmtree)
+
+    with patch.dict(
+        "sys.modules",
+        {
+            "platformio": SimpleNamespace(fs=mock_fs),
+            "platformio.fs": mock_fs,
+            "platformio.project.helpers": mock_project_helpers,
+            "platformio.run": SimpleNamespace(cli=mock_cli, helpers=mock_helpers),
+            "platformio.run.cli": mock_cli,
+            "platformio.run.helpers": mock_helpers,
+        },
+    ):
+        # Call patch_structhash to install the patched function
+        platformio_api.patch_structhash()
+
+        # Call the patched function
+        mock_helpers.clean_build_dir(str(build_dir), [])
+
+        # Verify rmtree was NOT called
+        assert len(removed_paths) == 0
+
+        # Verify directory and file still exist
+        assert build_dir.exists()
+        assert test_file.exists()
+        assert test_file.read_text() == "test content"
+
+
+def test_patched_clean_build_dir_creates_missing(setup_core: Path) -> None:
+    """Test patched_clean_build_dir creates build dir when it doesn't exist."""
+    build_dir = setup_core / "build"
+    platformio_ini = setup_core / "platformio.ini"
+    platformio_ini.write_text("config")
+
+    # Ensure build_dir doesn't exist
+    assert not build_dir.exists()
+
+    # Track if rmtree is called
+    removed_paths: list[str] = []
+
+    def track_rmtree(path: str) -> None:
+        removed_paths.append(path)
+
+    # Create mock modules
+    mock_cli = SimpleNamespace()
+    mock_helpers = SimpleNamespace()
+    mock_project_helpers = MagicMock()
+    mock_project_helpers.get_project_dir.return_value = str(setup_core)
+    mock_fs = SimpleNamespace(rmtree=track_rmtree)
+
+    with patch.dict(
+        "sys.modules",
+        {
+            "platformio": SimpleNamespace(fs=mock_fs),
+            "platformio.fs": mock_fs,
+            "platformio.project.helpers": mock_project_helpers,
+            "platformio.run": SimpleNamespace(cli=mock_cli, helpers=mock_helpers),
+            "platformio.run.cli": mock_cli,
+            "platformio.run.helpers": mock_helpers,
+        },
+    ):
+        # Call patch_structhash to install the patched function
+        platformio_api.patch_structhash()
+
+        # Call the patched function
+        mock_helpers.clean_build_dir(str(build_dir), [])
+
+        # Verify rmtree was NOT called
+        assert len(removed_paths) == 0
+
+        # Verify directory was created
+        assert build_dir.exists()
+
+
+def test_process_stacktrace_esp8266_exception(setup_core: Path, caplog) -> None:
+    """Test process_stacktrace handles ESP8266 exceptions."""
+    config = {"name": "test"}
+
+    # Test exception type parsing
+    line = "Exception (28):"
+    backtrace_state = False
+
+    result = platformio_api.process_stacktrace(config, line, backtrace_state)
+
+    assert "Access to invalid address: LOAD (wild pointer?)" in caplog.text
+    assert result is False
+
+
+def test_process_stacktrace_esp8266_backtrace(
+    setup_core: Path, mock_decode_pc: Mock
+) -> None:
+    """Test process_stacktrace handles ESP8266 multi-line backtrace."""
+    config = {"name": "test"}
+
+    # Start of backtrace
+    line1 = ">>>stack>>>"
+    state = platformio_api.process_stacktrace(config, line1, False)
+    assert state is True
+
+    # Backtrace content with addresses
+    line2 = "40201234 40205678"
+    state = platformio_api.process_stacktrace(config, line2, state)
+    assert state is True
+    assert mock_decode_pc.call_count == 2
+
+    # End of backtrace
+    line3 = "<< None:
+    """Test process_stacktrace handles ESP32 single-line backtrace."""
+    config = {"name": "test"}
+
+    line = "Backtrace: 0x40081234:0x3ffb1234 0x40085678:0x3ffb5678"
+    state = platformio_api.process_stacktrace(config, line, False)
+
+    # Should decode both addresses
+    assert mock_decode_pc.call_count == 2
+    mock_decode_pc.assert_any_call(config, "40081234")
+    mock_decode_pc.assert_any_call(config, "40085678")
+    assert state is False
+
+
+def test_process_stacktrace_bad_alloc(
+    setup_core: Path, mock_decode_pc: Mock, caplog
+) -> None:
+    """Test process_stacktrace handles bad alloc messages."""
+    config = {"name": "test"}
+
+    line = "last failed alloc call: 40201234(512)"
+    state = platformio_api.process_stacktrace(config, line, False)
+
+    assert "Memory allocation of 512 bytes failed at 40201234" in caplog.text
+    mock_decode_pc.assert_called_once_with(config, "40201234")
+    assert state is False
diff --git a/tests/unit_tests/test_resolver.py b/tests/unit_tests/test_resolver.py
new file mode 100644
index 0000000000..b4cca05d9f
--- /dev/null
+++ b/tests/unit_tests/test_resolver.py
@@ -0,0 +1,169 @@
+"""Tests for the DNS resolver module."""
+
+from __future__ import annotations
+
+import re
+import socket
+from unittest.mock import patch
+
+from aioesphomeapi.core import ResolveAPIError, ResolveTimeoutAPIError
+from aioesphomeapi.host_resolver import AddrInfo, IPv4Sockaddr, IPv6Sockaddr
+import pytest
+
+from esphome.core import EsphomeError
+from esphome.resolver import RESOLVE_TIMEOUT, AsyncResolver
+
+
+@pytest.fixture
+def mock_addr_info_ipv4() -> AddrInfo:
+    """Create a mock IPv4 AddrInfo."""
+    return AddrInfo(
+        family=socket.AF_INET,
+        type=socket.SOCK_STREAM,
+        proto=socket.IPPROTO_TCP,
+        sockaddr=IPv4Sockaddr(address="192.168.1.100", port=6053),
+    )
+
+
+@pytest.fixture
+def mock_addr_info_ipv6() -> AddrInfo:
+    """Create a mock IPv6 AddrInfo."""
+    return AddrInfo(
+        family=socket.AF_INET6,
+        type=socket.SOCK_STREAM,
+        proto=socket.IPPROTO_TCP,
+        sockaddr=IPv6Sockaddr(address="2001:db8::1", port=6053, flowinfo=0, scope_id=0),
+    )
+
+
+def test_async_resolver_successful_resolution(mock_addr_info_ipv4: AddrInfo) -> None:
+    """Test successful DNS resolution."""
+    with patch(
+        "esphome.resolver.hr.async_resolve_host",
+        return_value=[mock_addr_info_ipv4],
+    ) as mock_resolve:
+        resolver = AsyncResolver(["test.local"], 6053)
+        result = resolver.resolve()
+
+        assert result == [mock_addr_info_ipv4]
+        mock_resolve.assert_called_once_with(
+            ["test.local"], 6053, timeout=RESOLVE_TIMEOUT
+        )
+
+
+def test_async_resolver_multiple_hosts(
+    mock_addr_info_ipv4: AddrInfo, mock_addr_info_ipv6: AddrInfo
+) -> None:
+    """Test resolving multiple hosts."""
+    mock_results = [mock_addr_info_ipv4, mock_addr_info_ipv6]
+
+    with patch(
+        "esphome.resolver.hr.async_resolve_host",
+        return_value=mock_results,
+    ) as mock_resolve:
+        resolver = AsyncResolver(["test1.local", "test2.local"], 6053)
+        result = resolver.resolve()
+
+        assert result == mock_results
+        mock_resolve.assert_called_once_with(
+            ["test1.local", "test2.local"], 6053, timeout=RESOLVE_TIMEOUT
+        )
+
+
+def test_async_resolver_resolve_api_error() -> None:
+    """Test handling of ResolveAPIError."""
+    error_msg = "Failed to resolve"
+    with patch(
+        "esphome.resolver.hr.async_resolve_host",
+        side_effect=ResolveAPIError(error_msg),
+    ):
+        resolver = AsyncResolver(["test.local"], 6053)
+        with pytest.raises(
+            EsphomeError, match=re.escape(f"Error resolving IP address: {error_msg}")
+        ):
+            resolver.resolve()
+
+
+def test_async_resolver_timeout_error() -> None:
+    """Test handling of ResolveTimeoutAPIError."""
+    error_msg = "Resolution timed out"
+
+    with patch(
+        "esphome.resolver.hr.async_resolve_host",
+        side_effect=ResolveTimeoutAPIError(error_msg),
+    ):
+        resolver = AsyncResolver(["test.local"], 6053)
+        # Match either "Timeout" or "Error" since ResolveTimeoutAPIError is a subclass of ResolveAPIError
+        # and depending on import order/test execution context, it might be caught as either
+        with pytest.raises(
+            EsphomeError,
+            match=f"(Timeout|Error) resolving IP address: {re.escape(error_msg)}",
+        ):
+            resolver.resolve()
+
+
+def test_async_resolver_generic_exception() -> None:
+    """Test handling of generic exceptions."""
+    error = RuntimeError("Unexpected error")
+    with patch(
+        "esphome.resolver.hr.async_resolve_host",
+        side_effect=error,
+    ):
+        resolver = AsyncResolver(["test.local"], 6053)
+        with pytest.raises(RuntimeError, match="Unexpected error"):
+            resolver.resolve()
+
+
+def test_async_resolver_thread_timeout() -> None:
+    """Test timeout when thread doesn't complete in time."""
+    # Mock the start method to prevent actual thread execution
+    with (
+        patch.object(AsyncResolver, "start"),
+        patch("esphome.resolver.hr.async_resolve_host"),
+    ):
+        resolver = AsyncResolver(["test.local"], 6053)
+        # Override event.wait to simulate timeout (return False = timeout occurred)
+        with (
+            patch.object(resolver.event, "wait", return_value=False),
+            pytest.raises(
+                EsphomeError, match=re.escape("Timeout resolving IP address")
+            ),
+        ):
+            resolver.resolve()
+
+        # Verify thread start was called
+        resolver.start.assert_called_once()
+
+
+def test_async_resolver_ip_addresses(mock_addr_info_ipv4: AddrInfo) -> None:
+    """Test resolving IP addresses."""
+    with patch(
+        "esphome.resolver.hr.async_resolve_host",
+        return_value=[mock_addr_info_ipv4],
+    ) as mock_resolve:
+        resolver = AsyncResolver(["192.168.1.100"], 6053)
+        result = resolver.resolve()
+
+        assert result == [mock_addr_info_ipv4]
+        mock_resolve.assert_called_once_with(
+            ["192.168.1.100"], 6053, timeout=RESOLVE_TIMEOUT
+        )
+
+
+def test_async_resolver_mixed_addresses(
+    mock_addr_info_ipv4: AddrInfo, mock_addr_info_ipv6: AddrInfo
+) -> None:
+    """Test resolving mix of hostnames and IP addresses."""
+    mock_results = [mock_addr_info_ipv4, mock_addr_info_ipv6]
+
+    with patch(
+        "esphome.resolver.hr.async_resolve_host",
+        return_value=mock_results,
+    ) as mock_resolve:
+        resolver = AsyncResolver(["test.local", "192.168.1.100", "::1"], 6053)
+        result = resolver.resolve()
+
+        assert result == mock_results
+        mock_resolve.assert_called_once_with(
+            ["test.local", "192.168.1.100", "::1"], 6053, timeout=RESOLVE_TIMEOUT
+        )
diff --git a/tests/unit_tests/test_storage_json.py b/tests/unit_tests/test_storage_json.py
new file mode 100644
index 0000000000..a3a38960e7
--- /dev/null
+++ b/tests/unit_tests/test_storage_json.py
@@ -0,0 +1,660 @@
+"""Tests for storage_json.py path functions."""
+
+from datetime import datetime
+import json
+from pathlib import Path
+import sys
+from unittest.mock import MagicMock, Mock, patch
+
+import pytest
+
+from esphome import storage_json
+from esphome.const import CONF_DISABLED, CONF_MDNS
+from esphome.core import CORE
+
+
+def test_storage_path(setup_core: Path) -> None:
+    """Test storage_path returns correct path for current config."""
+    CORE.config_path = setup_core / "my_device.yaml"
+
+    result = storage_json.storage_path()
+
+    data_dir = Path(CORE.data_dir)
+    expected = data_dir / "storage" / "my_device.yaml.json"
+    assert result == expected
+
+
+def test_ext_storage_path(setup_core: Path) -> None:
+    """Test ext_storage_path returns correct path for given filename."""
+    result = storage_json.ext_storage_path("other_device.yaml")
+
+    data_dir = Path(CORE.data_dir)
+    expected = data_dir / "storage" / "other_device.yaml.json"
+    assert result == expected
+
+
+def test_ext_storage_path_handles_various_extensions(setup_core: Path) -> None:
+    """Test ext_storage_path works with different file extensions."""
+    result_yml = storage_json.ext_storage_path("device.yml")
+    assert str(result_yml).endswith("device.yml.json")
+
+    result_no_ext = storage_json.ext_storage_path("device")
+    assert str(result_no_ext).endswith("device.json")
+
+    result_path = storage_json.ext_storage_path("my/device.yaml")
+    assert str(result_path).endswith("device.yaml.json")
+
+
+def test_esphome_storage_path(setup_core: Path) -> None:
+    """Test esphome_storage_path returns correct path."""
+    result = storage_json.esphome_storage_path()
+
+    data_dir = Path(CORE.data_dir)
+    expected = data_dir / "esphome.json"
+    assert result == expected
+
+
+def test_ignored_devices_storage_path(setup_core: Path) -> None:
+    """Test ignored_devices_storage_path returns correct path."""
+    result = storage_json.ignored_devices_storage_path()
+
+    data_dir = Path(CORE.data_dir)
+    expected = data_dir / "ignored-devices.json"
+    assert result == expected
+
+
+def test_trash_storage_path(setup_core: Path) -> None:
+    """Test trash_storage_path returns correct path."""
+    CORE.config_path = setup_core / "configs" / "device.yaml"
+
+    result = storage_json.trash_storage_path()
+
+    expected = setup_core / "configs" / "trash"
+    assert result == expected
+
+
+def test_archive_storage_path(setup_core: Path) -> None:
+    """Test archive_storage_path returns correct path."""
+    CORE.config_path = setup_core / "configs" / "device.yaml"
+
+    result = storage_json.archive_storage_path()
+
+    expected = setup_core / "configs" / "archive"
+    assert result == expected
+
+
+def test_storage_path_with_subdirectory(setup_core: Path) -> None:
+    """Test storage paths work correctly when config is in subdirectory."""
+    subdir = setup_core / "configs" / "basement"
+    subdir.mkdir(parents=True, exist_ok=True)
+    CORE.config_path = subdir / "sensor.yaml"
+
+    result = storage_json.storage_path()
+
+    data_dir = Path(CORE.data_dir)
+    expected = data_dir / "storage" / "sensor.yaml.json"
+    assert result == expected
+
+
+def test_storage_json_firmware_bin_path_property(setup_core: Path) -> None:
+    """Test StorageJSON firmware_bin_path property."""
+    storage = storage_json.StorageJSON(
+        storage_version=1,
+        name="test_device",
+        friendly_name="Test Device",
+        comment=None,
+        esphome_version="2024.1.0",
+        src_version=None,
+        address="192.168.1.100",
+        web_port=80,
+        target_platform="ESP32",
+        build_path="build/test_device",
+        firmware_bin_path="/path/to/firmware.bin",
+        loaded_integrations={"wifi", "api"},
+        loaded_platforms=set(),
+        no_mdns=False,
+    )
+
+    assert storage.firmware_bin_path == "/path/to/firmware.bin"
+
+
+def test_storage_json_save_creates_directory(
+    setup_core: Path, tmp_path: Path, mock_write_file_if_changed: Mock
+) -> None:
+    """Test StorageJSON.save creates storage directory if it doesn't exist."""
+    storage_dir = tmp_path / "new_data" / "storage"
+    storage_file = storage_dir / "test.json"
+
+    assert not storage_dir.exists()
+
+    storage = storage_json.StorageJSON(
+        storage_version=1,
+        name="test",
+        friendly_name="Test",
+        comment=None,
+        esphome_version="2024.1.0",
+        src_version=None,
+        address="test.local",
+        web_port=None,
+        target_platform="ESP8266",
+        build_path=None,
+        firmware_bin_path=None,
+        loaded_integrations=set(),
+        loaded_platforms=set(),
+        no_mdns=False,
+    )
+
+    storage.save(str(storage_file))
+    mock_write_file_if_changed.assert_called_once()
+    call_args = mock_write_file_if_changed.call_args[0]
+    assert call_args[0] == str(storage_file)
+
+
+def test_storage_json_from_wizard(setup_core: Path) -> None:
+    """Test StorageJSON.from_wizard creates correct storage object."""
+    storage = storage_json.StorageJSON.from_wizard(
+        name="my_device",
+        friendly_name="My Device",
+        address="my_device.local",
+        platform="ESP32",
+    )
+
+    assert storage.name == "my_device"
+    assert storage.friendly_name == "My Device"
+    assert storage.address == "my_device.local"
+    assert storage.target_platform == "ESP32"
+    assert storage.build_path is None
+    assert storage.firmware_bin_path is None
+
+
+@pytest.mark.skipif(sys.platform == "win32", reason="HA addons don't run on Windows")
+@patch("esphome.core.is_ha_addon")
+def test_storage_paths_with_ha_addon(mock_is_ha_addon: bool, tmp_path: Path) -> None:
+    """Test storage paths when running as Home Assistant addon."""
+    mock_is_ha_addon.return_value = True
+
+    CORE.config_path = tmp_path / "test.yaml"
+
+    result = storage_json.storage_path()
+    # When is_ha_addon is True, CORE.data_dir returns "/data"
+    # This is the standard mount point for HA addon containers
+    expected = Path("/data") / "storage" / "test.yaml.json"
+    assert result == expected
+
+    result = storage_json.esphome_storage_path()
+    expected = Path("/data") / "esphome.json"
+    assert result == expected
+
+
+def test_storage_json_as_dict() -> None:
+    """Test StorageJSON.as_dict returns correct dictionary."""
+    storage = storage_json.StorageJSON(
+        storage_version=1,
+        name="test_device",
+        friendly_name="Test Device",
+        comment="Test comment",
+        esphome_version="2024.1.0",
+        src_version=1,
+        address="192.168.1.100",
+        web_port=80,
+        target_platform="ESP32",
+        build_path="/path/to/build",
+        firmware_bin_path="/path/to/firmware.bin",
+        loaded_integrations={"wifi", "api", "ota"},
+        loaded_platforms={"sensor", "binary_sensor"},
+        no_mdns=True,
+        framework="arduino",
+        core_platform="esp32",
+    )
+
+    result = storage.as_dict()
+
+    assert result["storage_version"] == 1
+    assert result["name"] == "test_device"
+    assert result["friendly_name"] == "Test Device"
+    assert result["comment"] == "Test comment"
+    assert result["esphome_version"] == "2024.1.0"
+    assert result["src_version"] == 1
+    assert result["address"] == "192.168.1.100"
+    assert result["web_port"] == 80
+    assert result["esp_platform"] == "ESP32"
+    assert result["build_path"] == "/path/to/build"
+    assert result["firmware_bin_path"] == "/path/to/firmware.bin"
+    assert "api" in result["loaded_integrations"]
+    assert "wifi" in result["loaded_integrations"]
+    assert "ota" in result["loaded_integrations"]
+    assert result["loaded_integrations"] == sorted(
+        ["wifi", "api", "ota"]
+    )  # Should be sorted
+    assert "sensor" in result["loaded_platforms"]
+    assert result["loaded_platforms"] == sorted(
+        ["sensor", "binary_sensor"]
+    )  # Should be sorted
+    assert result["no_mdns"] is True
+    assert result["framework"] == "arduino"
+    assert result["core_platform"] == "esp32"
+
+
+def test_storage_json_to_json() -> None:
+    """Test StorageJSON.to_json returns valid JSON string."""
+    storage = storage_json.StorageJSON(
+        storage_version=1,
+        name="test",
+        friendly_name="Test",
+        comment=None,
+        esphome_version="2024.1.0",
+        src_version=None,
+        address="test.local",
+        web_port=None,
+        target_platform="ESP8266",
+        build_path=None,
+        firmware_bin_path=None,
+        loaded_integrations=set(),
+        loaded_platforms=set(),
+        no_mdns=False,
+    )
+
+    json_str = storage.to_json()
+
+    # Should be valid JSON
+    parsed = json.loads(json_str)
+    assert parsed["name"] == "test"
+    assert parsed["storage_version"] == 1
+
+    # Should end with newline
+    assert json_str.endswith("\n")
+
+
+def test_storage_json_save(tmp_path: Path) -> None:
+    """Test StorageJSON.save writes file correctly."""
+    storage = storage_json.StorageJSON(
+        storage_version=1,
+        name="test",
+        friendly_name="Test",
+        comment=None,
+        esphome_version="2024.1.0",
+        src_version=None,
+        address="test.local",
+        web_port=None,
+        target_platform="ESP32",
+        build_path=None,
+        firmware_bin_path=None,
+        loaded_integrations=set(),
+        loaded_platforms=set(),
+        no_mdns=False,
+    )
+
+    save_path = tmp_path / "test.json"
+
+    with patch("esphome.storage_json.write_file_if_changed") as mock_write:
+        storage.save(str(save_path))
+        mock_write.assert_called_once_with(str(save_path), storage.to_json())
+
+
+def test_storage_json_from_esphome_core(setup_core: Path) -> None:
+    """Test StorageJSON.from_esphome_core creates correct storage object."""
+    # Mock CORE object
+    mock_core = MagicMock()
+    mock_core.name = "my_device"
+    mock_core.friendly_name = "My Device"
+    mock_core.comment = "A test device"
+    mock_core.address = "192.168.1.50"
+    mock_core.web_port = 8080
+    mock_core.target_platform = "esp32"
+    mock_core.is_esp32 = True
+    mock_core.build_path = "/build/my_device"
+    mock_core.firmware_bin = "/build/my_device/firmware.bin"
+    mock_core.loaded_integrations = {"wifi", "api"}
+    mock_core.loaded_platforms = {"sensor"}
+    mock_core.config = {CONF_MDNS: {CONF_DISABLED: True}}
+    mock_core.target_framework = "esp-idf"
+
+    with patch("esphome.components.esp32.get_esp32_variant") as mock_variant:
+        mock_variant.return_value = "ESP32-C3"
+
+        result = storage_json.StorageJSON.from_esphome_core(mock_core, old=None)
+
+    assert result.name == "my_device"
+    assert result.friendly_name == "My Device"
+    assert result.comment == "A test device"
+    assert result.address == "192.168.1.50"
+    assert result.web_port == 8080
+    assert result.target_platform == "ESP32-C3"
+    assert result.build_path == "/build/my_device"
+    assert result.firmware_bin_path == "/build/my_device/firmware.bin"
+    assert result.loaded_integrations == {"wifi", "api"}
+    assert result.loaded_platforms == {"sensor"}
+    assert result.no_mdns is True
+    assert result.framework == "esp-idf"
+    assert result.core_platform == "esp32"
+
+
+def test_storage_json_from_esphome_core_mdns_enabled(setup_core: Path) -> None:
+    """Test from_esphome_core with mDNS enabled."""
+    mock_core = MagicMock()
+    mock_core.name = "test"
+    mock_core.friendly_name = "Test"
+    mock_core.comment = None
+    mock_core.address = "test.local"
+    mock_core.web_port = None
+    mock_core.target_platform = "esp8266"
+    mock_core.is_esp32 = False
+    mock_core.build_path = "/build"
+    mock_core.firmware_bin = "/build/firmware.bin"
+    mock_core.loaded_integrations = set()
+    mock_core.loaded_platforms = set()
+    mock_core.config = {}  # No MDNS config means enabled
+    mock_core.target_framework = "arduino"
+
+    result = storage_json.StorageJSON.from_esphome_core(mock_core, old=None)
+
+    assert result.no_mdns is False
+
+
+def test_storage_json_load_valid_file(tmp_path: Path) -> None:
+    """Test StorageJSON.load with valid JSON file."""
+    storage_data = {
+        "storage_version": 1,
+        "name": "loaded_device",
+        "friendly_name": "Loaded Device",
+        "comment": "Loaded from file",
+        "esphome_version": "2024.1.0",
+        "src_version": 2,
+        "address": "10.0.0.1",
+        "web_port": 8080,
+        "esp_platform": "ESP32",
+        "build_path": "/loaded/build",
+        "firmware_bin_path": "/loaded/firmware.bin",
+        "loaded_integrations": ["wifi", "api"],
+        "loaded_platforms": ["sensor"],
+        "no_mdns": True,
+        "framework": "arduino",
+        "core_platform": "esp32",
+    }
+
+    file_path = tmp_path / "storage.json"
+    file_path.write_text(json.dumps(storage_data))
+
+    result = storage_json.StorageJSON.load(file_path)
+
+    assert result is not None
+    assert result.name == "loaded_device"
+    assert result.friendly_name == "Loaded Device"
+    assert result.comment == "Loaded from file"
+    assert result.esphome_version == "2024.1.0"
+    assert result.src_version == 2
+    assert result.address == "10.0.0.1"
+    assert result.web_port == 8080
+    assert result.target_platform == "ESP32"
+    assert result.build_path == Path("/loaded/build")
+    assert result.firmware_bin_path == Path("/loaded/firmware.bin")
+    assert result.loaded_integrations == {"wifi", "api"}
+    assert result.loaded_platforms == {"sensor"}
+    assert result.no_mdns is True
+    assert result.framework == "arduino"
+    assert result.core_platform == "esp32"
+
+
+def test_storage_json_load_invalid_file(tmp_path: Path) -> None:
+    """Test StorageJSON.load with invalid JSON file."""
+    file_path = tmp_path / "invalid.json"
+    file_path.write_text("not valid json{")
+
+    result = storage_json.StorageJSON.load(file_path)
+
+    assert result is None
+
+
+def test_storage_json_load_nonexistent_file() -> None:
+    """Test StorageJSON.load with non-existent file."""
+    result = storage_json.StorageJSON.load("/nonexistent/file.json")
+
+    assert result is None
+
+
+def test_storage_json_equality() -> None:
+    """Test StorageJSON equality comparison."""
+    storage1 = storage_json.StorageJSON(
+        storage_version=1,
+        name="test",
+        friendly_name="Test",
+        comment=None,
+        esphome_version="2024.1.0",
+        src_version=1,
+        address="test.local",
+        web_port=80,
+        target_platform="ESP32",
+        build_path="/build",
+        firmware_bin_path="/firmware.bin",
+        loaded_integrations={"wifi"},
+        loaded_platforms=set(),
+        no_mdns=False,
+    )
+
+    storage2 = storage_json.StorageJSON(
+        storage_version=1,
+        name="test",
+        friendly_name="Test",
+        comment=None,
+        esphome_version="2024.1.0",
+        src_version=1,
+        address="test.local",
+        web_port=80,
+        target_platform="ESP32",
+        build_path="/build",
+        firmware_bin_path="/firmware.bin",
+        loaded_integrations={"wifi"},
+        loaded_platforms=set(),
+        no_mdns=False,
+    )
+
+    storage3 = storage_json.StorageJSON(
+        storage_version=1,
+        name="different",  # Different name
+        friendly_name="Test",
+        comment=None,
+        esphome_version="2024.1.0",
+        src_version=1,
+        address="test.local",
+        web_port=80,
+        target_platform="ESP32",
+        build_path="/build",
+        firmware_bin_path="/firmware.bin",
+        loaded_integrations={"wifi"},
+        loaded_platforms=set(),
+        no_mdns=False,
+    )
+
+    assert storage1 == storage2
+    assert storage1 != storage3
+    assert storage1 != "not a storage object"
+
+
+def test_esphome_storage_json_as_dict() -> None:
+    """Test EsphomeStorageJSON.as_dict returns correct dictionary."""
+    storage = storage_json.EsphomeStorageJSON(
+        storage_version=1,
+        cookie_secret="secret123",
+        last_update_check="2024-01-15T10:30:00",
+        remote_version="2024.1.1",
+    )
+
+    result = storage.as_dict()
+
+    assert result["storage_version"] == 1
+    assert result["cookie_secret"] == "secret123"
+    assert result["last_update_check"] == "2024-01-15T10:30:00"
+    assert result["remote_version"] == "2024.1.1"
+
+
+def test_esphome_storage_json_last_update_check_property() -> None:
+    """Test EsphomeStorageJSON.last_update_check property."""
+    storage = storage_json.EsphomeStorageJSON(
+        storage_version=1,
+        cookie_secret="secret",
+        last_update_check="2024-01-15T10:30:00",
+        remote_version=None,
+    )
+
+    # Test getter
+    result = storage.last_update_check
+    assert isinstance(result, datetime)
+    assert result.year == 2024
+    assert result.month == 1
+    assert result.day == 15
+    assert result.hour == 10
+    assert result.minute == 30
+
+    # Test setter
+    new_date = datetime(2024, 2, 20, 15, 45, 30)
+    storage.last_update_check = new_date
+    assert storage.last_update_check_str == "2024-02-20T15:45:30"
+
+
+def test_esphome_storage_json_last_update_check_invalid() -> None:
+    """Test EsphomeStorageJSON.last_update_check with invalid date."""
+    storage = storage_json.EsphomeStorageJSON(
+        storage_version=1,
+        cookie_secret="secret",
+        last_update_check="invalid date",
+        remote_version=None,
+    )
+
+    result = storage.last_update_check
+    assert result is None
+
+
+def test_esphome_storage_json_to_json() -> None:
+    """Test EsphomeStorageJSON.to_json returns valid JSON string."""
+    storage = storage_json.EsphomeStorageJSON(
+        storage_version=1,
+        cookie_secret="mysecret",
+        last_update_check="2024-01-15T10:30:00",
+        remote_version="2024.1.1",
+    )
+
+    json_str = storage.to_json()
+
+    # Should be valid JSON
+    parsed = json.loads(json_str)
+    assert parsed["cookie_secret"] == "mysecret"
+    assert parsed["storage_version"] == 1
+
+    # Should end with newline
+    assert json_str.endswith("\n")
+
+
+def test_esphome_storage_json_save(tmp_path: Path) -> None:
+    """Test EsphomeStorageJSON.save writes file correctly."""
+    storage = storage_json.EsphomeStorageJSON(
+        storage_version=1,
+        cookie_secret="secret",
+        last_update_check=None,
+        remote_version=None,
+    )
+
+    save_path = tmp_path / "esphome.json"
+
+    with patch("esphome.storage_json.write_file_if_changed") as mock_write:
+        storage.save(str(save_path))
+        mock_write.assert_called_once_with(str(save_path), storage.to_json())
+
+
+def test_esphome_storage_json_load_valid_file(tmp_path: Path) -> None:
+    """Test EsphomeStorageJSON.load with valid JSON file."""
+    storage_data = {
+        "storage_version": 1,
+        "cookie_secret": "loaded_secret",
+        "last_update_check": "2024-01-20T14:30:00",
+        "remote_version": "2024.1.2",
+    }
+
+    file_path = tmp_path / "esphome.json"
+    file_path.write_text(json.dumps(storage_data))
+
+    result = storage_json.EsphomeStorageJSON.load(str(file_path))
+
+    assert result is not None
+    assert result.storage_version == 1
+    assert result.cookie_secret == "loaded_secret"
+    assert result.last_update_check_str == "2024-01-20T14:30:00"
+    assert result.remote_version == "2024.1.2"
+
+
+def test_esphome_storage_json_load_invalid_file(tmp_path: Path) -> None:
+    """Test EsphomeStorageJSON.load with invalid JSON file."""
+    file_path = tmp_path / "invalid.json"
+    file_path.write_text("not valid json{")
+
+    result = storage_json.EsphomeStorageJSON.load(str(file_path))
+
+    assert result is None
+
+
+def test_esphome_storage_json_load_nonexistent_file() -> None:
+    """Test EsphomeStorageJSON.load with non-existent file."""
+    result = storage_json.EsphomeStorageJSON.load("/nonexistent/file.json")
+
+    assert result is None
+
+
+def test_esphome_storage_json_get_default() -> None:
+    """Test EsphomeStorageJSON.get_default creates default storage."""
+    with patch("esphome.storage_json.os.urandom") as mock_urandom:
+        # Mock urandom to return predictable bytes
+        mock_urandom.return_value = b"test" * 16  # 64 bytes
+
+        result = storage_json.EsphomeStorageJSON.get_default()
+
+    assert result.storage_version == 1
+    assert len(result.cookie_secret) == 128  # 64 bytes hex = 128 chars
+    assert result.last_update_check is None
+    assert result.remote_version is None
+
+
+def test_esphome_storage_json_equality() -> None:
+    """Test EsphomeStorageJSON equality comparison."""
+    storage1 = storage_json.EsphomeStorageJSON(
+        storage_version=1,
+        cookie_secret="secret",
+        last_update_check="2024-01-15T10:30:00",
+        remote_version="2024.1.1",
+    )
+
+    storage2 = storage_json.EsphomeStorageJSON(
+        storage_version=1,
+        cookie_secret="secret",
+        last_update_check="2024-01-15T10:30:00",
+        remote_version="2024.1.1",
+    )
+
+    storage3 = storage_json.EsphomeStorageJSON(
+        storage_version=1,
+        cookie_secret="different",  # Different secret
+        last_update_check="2024-01-15T10:30:00",
+        remote_version="2024.1.1",
+    )
+
+    assert storage1 == storage2
+    assert storage1 != storage3
+    assert storage1 != "not a storage object"
+
+
+def test_storage_json_load_legacy_esphomeyaml_version(tmp_path: Path) -> None:
+    """Test loading storage with legacy esphomeyaml_version field."""
+    storage_data = {
+        "storage_version": 1,
+        "name": "legacy_device",
+        "friendly_name": "Legacy Device",
+        "esphomeyaml_version": "1.14.0",  # Legacy field name
+        "address": "legacy.local",
+        "esp_platform": "ESP8266",
+    }
+
+    file_path = tmp_path / "legacy.json"
+    file_path.write_text(json.dumps(storage_data))
+
+    result = storage_json.StorageJSON.load(file_path)
+
+    assert result is not None
+    assert result.esphome_version == "1.14.0"  # Should map to esphome_version
diff --git a/tests/unit_tests/test_substitutions.py b/tests/unit_tests/test_substitutions.py
index 3208923116..59396a4a83 100644
--- a/tests/unit_tests/test_substitutions.py
+++ b/tests/unit_tests/test_substitutions.py
@@ -1,10 +1,13 @@
 import glob
 import logging
-import os
+from pathlib import Path
 
-from esphome import yaml_util
+from esphome import config as config_module, yaml_util
 from esphome.components import substitutions
-from esphome.const import CONF_PACKAGES
+from esphome.config_helpers import merge_config
+from esphome.const import CONF_PACKAGES, CONF_SUBSTITUTIONS
+from esphome.core import CORE
+from esphome.util import OrderedDict
 
 _LOGGER = logging.getLogger(__name__)
 
@@ -18,11 +21,10 @@ def sort_dicts(obj):
     """Recursively sort dictionaries for order-insensitive comparison."""
     if isinstance(obj, dict):
         return {k: sort_dicts(obj[k]) for k in sorted(obj)}
-    elif isinstance(obj, list):
+    if isinstance(obj, list):
         # Lists are not sorted; we preserve order
         return [sort_dicts(i) for i in obj]
-    else:
-        return obj
+    return obj
 
 
 def dict_diff(a, b, path=""):
@@ -31,10 +33,8 @@ def dict_diff(a, b, path=""):
     if isinstance(a, dict) and isinstance(b, dict):
         a_keys = set(a)
         b_keys = set(b)
-        for key in a_keys - b_keys:
-            diffs.append(f"{path}/{key} only in actual")
-        for key in b_keys - a_keys:
-            diffs.append(f"{path}/{key} only in expected")
+        diffs.extend(f"{path}/{key} only in actual" for key in a_keys - b_keys)
+        diffs.extend(f"{path}/{key} only in expected" for key in b_keys - a_keys)
         for key in a_keys & b_keys:
             diffs.extend(dict_diff(a[key], b[key], f"{path}/{key}"))
     elif isinstance(a, list) and isinstance(b, list):
@@ -42,19 +42,21 @@ def dict_diff(a, b, path=""):
         for i in range(min_len):
             diffs.extend(dict_diff(a[i], b[i], f"{path}[{i}]"))
         if len(a) > len(b):
-            for i in range(min_len, len(a)):
-                diffs.append(f"{path}[{i}] only in actual: {a[i]!r}")
+            diffs.extend(
+                f"{path}[{i}] only in actual: {a[i]!r}" for i in range(min_len, len(a))
+            )
         elif len(b) > len(a):
-            for i in range(min_len, len(b)):
-                diffs.append(f"{path}[{i}] only in expected: {b[i]!r}")
+            diffs.extend(
+                f"{path}[{i}] only in expected: {b[i]!r}"
+                for i in range(min_len, len(b))
+            )
     elif a != b:
         diffs.append(f"\t{path}: actual={a!r} expected={b!r}")
     return diffs
 
 
-def write_yaml(path, data):
-    with open(path, "w", encoding="utf-8") as f:
-        f.write(yaml_util.dump(data))
+def write_yaml(path: Path, data: dict) -> None:
+    path.write_text(yaml_util.dump(data), encoding="utf-8")
 
 
 def test_substitutions_fixtures(fixture_path):
@@ -64,11 +66,10 @@ def test_substitutions_fixtures(fixture_path):
 
     failures = []
     for source_path in sources:
+        source_path = Path(source_path)
         try:
-            expected_path = source_path.replace(".input.yaml", ".approved.yaml")
-            test_case = os.path.splitext(os.path.basename(source_path))[0].replace(
-                ".input", ""
-            )
+            expected_path = source_path.with_suffix("").with_suffix(".approved.yaml")
+            test_case = source_path.with_suffix("").stem
 
             # Load using ESPHome's YAML loader
             config = yaml_util.load_yaml(source_path)
@@ -81,12 +82,12 @@ def test_substitutions_fixtures(fixture_path):
             substitutions.do_substitution_pass(config, None)
 
             # Also load expected using ESPHome's loader, or use {} if missing and DEV_MODE
-            if os.path.isfile(expected_path):
+            if expected_path.is_file():
                 expected = yaml_util.load_yaml(expected_path)
             elif DEV_MODE:
                 expected = {}
             else:
-                assert os.path.isfile(expected_path), (
+                assert expected_path.is_file(), (
                     f"Expected file missing: {expected_path}"
                 )
 
@@ -97,16 +98,14 @@ def test_substitutions_fixtures(fixture_path):
             if got_sorted != expected_sorted:
                 diff = "\n".join(dict_diff(got_sorted, expected_sorted))
                 msg = (
-                    f"Substitution result mismatch for {os.path.basename(source_path)}\n"
+                    f"Substitution result mismatch for {source_path.name}\n"
                     f"Diff:\n{diff}\n\n"
                     f"Got:      {got_sorted}\n"
                     f"Expected: {expected_sorted}"
                 )
                 # Write out the received file when test fails
                 if DEV_MODE:
-                    received_path = os.path.join(
-                        os.path.dirname(source_path), f"{test_case}.received.yaml"
-                    )
+                    received_path = source_path.with_name(f"{test_case}.received.yaml")
                     write_yaml(received_path, config)
                     print(msg)
                     failures.append(msg)
@@ -122,3 +121,200 @@ def test_substitutions_fixtures(fixture_path):
     if DEV_MODE:
         _LOGGER.error("Tests passed, but Dev mode is enabled.")
     assert not DEV_MODE  # make sure DEV_MODE is disabled after you are finished.
+
+
+def test_substitutions_with_command_line_maintains_ordered_dict() -> None:
+    """Test that substitutions remain an OrderedDict when command line substitutions are provided,
+    and that move_to_end() can be called successfully.
+
+    This is a regression test for https://github.com/esphome/esphome/issues/11182
+    where the config would become a regular dict and fail when move_to_end() was called.
+    """
+    # Create an OrderedDict config with substitutions
+    config = OrderedDict()
+    config["esphome"] = {"name": "test"}
+    config[CONF_SUBSTITUTIONS] = {"var1": "value1", "var2": "value2"}
+    config["other_key"] = "other_value"
+
+    # Command line substitutions that should override
+    command_line_subs = {"var2": "override", "var3": "new_value"}
+
+    # Call do_substitution_pass with command line substitutions
+    substitutions.do_substitution_pass(config, command_line_subs)
+
+    # Verify that config is still an OrderedDict
+    assert isinstance(config, OrderedDict), "Config should remain an OrderedDict"
+
+    # Verify substitutions are at the beginning (move_to_end with last=False)
+    keys = list(config.keys())
+    assert keys[0] == CONF_SUBSTITUTIONS, "Substitutions should be first key"
+
+    # Verify substitutions were properly merged
+    assert config[CONF_SUBSTITUTIONS]["var1"] == "value1"
+    assert config[CONF_SUBSTITUTIONS]["var2"] == "override"
+    assert config[CONF_SUBSTITUTIONS]["var3"] == "new_value"
+
+    # Verify config[CONF_SUBSTITUTIONS] is also an OrderedDict
+    assert isinstance(config[CONF_SUBSTITUTIONS], OrderedDict), (
+        "Substitutions should be an OrderedDict"
+    )
+
+
+def test_substitutions_without_command_line_maintains_ordered_dict() -> None:
+    """Test that substitutions work correctly without command line substitutions."""
+    config = OrderedDict()
+    config["esphome"] = {"name": "test"}
+    config[CONF_SUBSTITUTIONS] = {"var1": "value1"}
+    config["other_key"] = "other_value"
+
+    # Call without command line substitutions
+    substitutions.do_substitution_pass(config, None)
+
+    # Verify that config is still an OrderedDict
+    assert isinstance(config, OrderedDict), "Config should remain an OrderedDict"
+
+    # Verify substitutions are at the beginning
+    keys = list(config.keys())
+    assert keys[0] == CONF_SUBSTITUTIONS, "Substitutions should be first key"
+
+
+def test_substitutions_after_merge_config_maintains_ordered_dict() -> None:
+    """Test that substitutions work after merge_config (packages scenario).
+
+    This is a regression test for https://github.com/esphome/esphome/issues/11182
+    where using packages would cause config to become a regular dict, breaking move_to_end().
+    """
+    # Simulate what happens with packages - merge two OrderedDict configs
+    base_config = OrderedDict()
+    base_config["esphome"] = {"name": "base"}
+    base_config[CONF_SUBSTITUTIONS] = {"var1": "value1"}
+
+    package_config = OrderedDict()
+    package_config["sensor"] = [{"platform": "template"}]
+    package_config[CONF_SUBSTITUTIONS] = {"var2": "value2"}
+
+    # Merge configs (simulating package merge)
+    merged_config = merge_config(base_config, package_config)
+
+    # Verify merged config is still an OrderedDict
+    assert isinstance(merged_config, OrderedDict), (
+        "Merged config should be an OrderedDict"
+    )
+
+    # Now try to run substitution pass on the merged config
+    substitutions.do_substitution_pass(merged_config, None)
+
+    # Should not raise AttributeError
+    assert isinstance(merged_config, OrderedDict), (
+        "Config should still be OrderedDict after substitution pass"
+    )
+    keys = list(merged_config.keys())
+    assert keys[0] == CONF_SUBSTITUTIONS, "Substitutions should be first key"
+
+
+def test_validate_config_with_command_line_substitutions_maintains_ordered_dict(
+    tmp_path,
+) -> None:
+    """Test that validate_config preserves OrderedDict when merging command-line substitutions.
+
+    This tests the code path in config.py where result[CONF_SUBSTITUTIONS] is set
+    using merge_dicts_ordered() with command-line substitutions provided.
+    """
+    # Create a minimal valid config
+    test_config = OrderedDict()
+    test_config["esphome"] = {"name": "test_device", "platform": "ESP32"}
+    test_config[CONF_SUBSTITUTIONS] = OrderedDict({"var1": "value1", "var2": "value2"})
+    test_config["esp32"] = {"board": "esp32dev"}
+
+    # Command line substitutions that should override
+    command_line_subs = {"var2": "override", "var3": "new_value"}
+
+    # Set up CORE for the test with a proper Path object
+    test_yaml = tmp_path / "test.yaml"
+    test_yaml.write_text("# test config")
+    CORE.config_path = test_yaml
+
+    # Call validate_config with command line substitutions
+    result = config_module.validate_config(test_config, command_line_subs)
+
+    # Verify that result[CONF_SUBSTITUTIONS] is an OrderedDict
+    assert isinstance(result.get(CONF_SUBSTITUTIONS), OrderedDict), (
+        "Result substitutions should be an OrderedDict"
+    )
+
+    # Verify substitutions were properly merged
+    assert result[CONF_SUBSTITUTIONS]["var1"] == "value1"
+    assert result[CONF_SUBSTITUTIONS]["var2"] == "override"
+    assert result[CONF_SUBSTITUTIONS]["var3"] == "new_value"
+
+
+def test_validate_config_without_command_line_substitutions_maintains_ordered_dict(
+    tmp_path,
+) -> None:
+    """Test that validate_config preserves OrderedDict without command-line substitutions.
+
+    This tests the code path in config.py where result[CONF_SUBSTITUTIONS] is set
+    using merge_dicts_ordered() when command_line_substitutions is None.
+    """
+    # Create a minimal valid config
+    test_config = OrderedDict()
+    test_config["esphome"] = {"name": "test_device", "platform": "ESP32"}
+    test_config[CONF_SUBSTITUTIONS] = OrderedDict({"var1": "value1", "var2": "value2"})
+    test_config["esp32"] = {"board": "esp32dev"}
+
+    # Set up CORE for the test with a proper Path object
+    test_yaml = tmp_path / "test.yaml"
+    test_yaml.write_text("# test config")
+    CORE.config_path = test_yaml
+
+    # Call validate_config without command line substitutions
+    result = config_module.validate_config(test_config, None)
+
+    # Verify that result[CONF_SUBSTITUTIONS] is an OrderedDict
+    assert isinstance(result.get(CONF_SUBSTITUTIONS), OrderedDict), (
+        "Result substitutions should be an OrderedDict"
+    )
+
+    # Verify substitutions are unchanged
+    assert result[CONF_SUBSTITUTIONS]["var1"] == "value1"
+    assert result[CONF_SUBSTITUTIONS]["var2"] == "value2"
+
+
+def test_merge_config_preserves_ordered_dict() -> None:
+    """Test that merge_config preserves OrderedDict type.
+
+    This is a regression test to ensure merge_config doesn't lose OrderedDict type
+    when merging configs, which causes AttributeError on move_to_end().
+    """
+    # Test OrderedDict + dict = OrderedDict
+    od = OrderedDict([("a", 1), ("b", 2)])
+    d = {"b": 20, "c": 3}
+    result = merge_config(od, d)
+    assert isinstance(result, OrderedDict), (
+        "OrderedDict + dict should return OrderedDict"
+    )
+
+    # Test dict + OrderedDict = OrderedDict
+    d = {"a": 1, "b": 2}
+    od = OrderedDict([("b", 20), ("c", 3)])
+    result = merge_config(d, od)
+    assert isinstance(result, OrderedDict), (
+        "dict + OrderedDict should return OrderedDict"
+    )
+
+    # Test OrderedDict + OrderedDict = OrderedDict
+    od1 = OrderedDict([("a", 1), ("b", 2)])
+    od2 = OrderedDict([("b", 20), ("c", 3)])
+    result = merge_config(od1, od2)
+    assert isinstance(result, OrderedDict), (
+        "OrderedDict + OrderedDict should return OrderedDict"
+    )
+
+    # Test that dict + dict still returns regular dict (no unnecessary conversion)
+    d1 = {"a": 1, "b": 2}
+    d2 = {"b": 20, "c": 3}
+    result = merge_config(d1, d2)
+    assert isinstance(result, dict), "dict + dict should return dict"
+    assert not isinstance(result, OrderedDict), (
+        "dict + dict should not return OrderedDict"
+    )
diff --git a/tests/unit_tests/test_util.py b/tests/unit_tests/test_util.py
new file mode 100644
index 0000000000..85873caea8
--- /dev/null
+++ b/tests/unit_tests/test_util.py
@@ -0,0 +1,404 @@
+"""Tests for esphome.util module."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import pytest
+
+from esphome import util
+
+
+def test_list_yaml_files_with_files_and_directories(tmp_path: Path) -> None:
+    """Test that list_yaml_files handles both files and directories."""
+    # Create directory structure
+    dir1 = tmp_path / "configs"
+    dir1.mkdir()
+    dir2 = tmp_path / "more_configs"
+    dir2.mkdir()
+
+    # Create YAML files in directories
+    (dir1 / "config1.yaml").write_text("test: 1")
+    (dir1 / "config2.yml").write_text("test: 2")
+    (dir1 / "not_yaml.txt").write_text("not yaml")
+
+    (dir2 / "config3.yaml").write_text("test: 3")
+
+    # Create standalone YAML files
+    standalone1 = tmp_path / "standalone.yaml"
+    standalone1.write_text("test: 4")
+    standalone2 = tmp_path / "another.yml"
+    standalone2.write_text("test: 5")
+
+    # Test with mixed input (directories and files)
+    configs = [
+        dir1,
+        standalone1,
+        dir2,
+        standalone2,
+    ]
+
+    result = util.list_yaml_files(configs)
+
+    # Should include all YAML files but not the .txt file
+    assert set(result) == {
+        dir1 / "config1.yaml",
+        dir1 / "config2.yml",
+        dir2 / "config3.yaml",
+        standalone1,
+        standalone2,
+    }
+    # Check that results are sorted
+    assert result == sorted(result)
+
+
+def test_list_yaml_files_only_directories(tmp_path: Path) -> None:
+    """Test list_yaml_files with only directories."""
+    dir1 = tmp_path / "dir1"
+    dir1.mkdir()
+    dir2 = tmp_path / "dir2"
+    dir2.mkdir()
+
+    (dir1 / "a.yaml").write_text("test: a")
+    (dir1 / "b.yml").write_text("test: b")
+    (dir2 / "c.yaml").write_text("test: c")
+
+    result = util.list_yaml_files([dir1, dir2])
+
+    assert set(result) == {
+        dir1 / "a.yaml",
+        dir1 / "b.yml",
+        dir2 / "c.yaml",
+    }
+    assert result == sorted(result)
+
+
+def test_list_yaml_files_only_files(tmp_path: Path) -> None:
+    """Test list_yaml_files with only files."""
+    file1 = tmp_path / "file1.yaml"
+    file2 = tmp_path / "file2.yml"
+    file3 = tmp_path / "file3.yaml"
+    non_yaml = tmp_path / "not_yaml.json"
+
+    file1.write_text("test: 1")
+    file2.write_text("test: 2")
+    file3.write_text("test: 3")
+    non_yaml.write_text("{}")
+
+    # Include a non-YAML file to test filtering
+    result = util.list_yaml_files(
+        [
+            file1,
+            file2,
+            file3,
+            non_yaml,
+        ]
+    )
+
+    assert set(result) == {
+        file1,
+        file2,
+        file3,
+    }
+    assert result == sorted(result)
+
+
+def test_list_yaml_files_empty_directory(tmp_path: Path) -> None:
+    """Test list_yaml_files with an empty directory."""
+    empty_dir = tmp_path / "empty"
+    empty_dir.mkdir()
+
+    result = util.list_yaml_files([empty_dir])
+
+    assert result == []
+
+
+def test_list_yaml_files_nonexistent_path(tmp_path: Path) -> None:
+    """Test list_yaml_files with a nonexistent path raises an error."""
+    nonexistent = tmp_path / "nonexistent"
+    existing = tmp_path / "existing.yaml"
+    existing.write_text("test: 1")
+
+    # Should raise an error for non-existent directory
+    with pytest.raises(FileNotFoundError):
+        util.list_yaml_files([nonexistent, existing])
+
+
+def test_list_yaml_files_mixed_extensions(tmp_path: Path) -> None:
+    """Test that both .yaml and .yml extensions are recognized."""
+    dir1 = tmp_path / "configs"
+    dir1.mkdir()
+
+    yaml_file = dir1 / "config.yaml"
+    yml_file = dir1 / "config.yml"
+    other_file = dir1 / "config.txt"
+
+    yaml_file.write_text("test: yaml")
+    yml_file.write_text("test: yml")
+    other_file.write_text("test: txt")
+
+    result = util.list_yaml_files([dir1])
+
+    assert set(result) == {
+        yaml_file,
+        yml_file,
+    }
+
+
+def test_list_yaml_files_does_not_recurse_into_subdirectories(tmp_path: Path) -> None:
+    """Test that list_yaml_files only finds files in specified directory, not subdirectories."""
+    # Create directory structure with YAML files at different depths
+    root = tmp_path / "configs"
+    root.mkdir()
+
+    # Create YAML files in the root directory
+    (root / "config1.yaml").write_text("test: 1")
+    (root / "config2.yml").write_text("test: 2")
+    (root / "device.yaml").write_text("test: device")
+
+    # Create subdirectory with YAML files (should NOT be found)
+    subdir = root / "subdir"
+    subdir.mkdir()
+    (subdir / "nested1.yaml").write_text("test: nested1")
+    (subdir / "nested2.yml").write_text("test: nested2")
+
+    # Create deeper subdirectory (should NOT be found)
+    deep_subdir = subdir / "deeper"
+    deep_subdir.mkdir()
+    (deep_subdir / "very_nested.yaml").write_text("test: very_nested")
+
+    # Test listing files from the root directory
+    result = util.list_yaml_files([str(root)])
+
+    # Should only find the 3 files in root, not the 3 in subdirectories
+    assert len(result) == 3
+
+    # Check that only root-level files are found
+    assert root / "config1.yaml" in result
+    assert root / "config2.yml" in result
+    assert root / "device.yaml" in result
+
+    # Ensure nested files are NOT found
+    for r in result:
+        r_str = str(r)
+        assert "subdir" not in r_str
+        assert "deeper" not in r_str
+        assert "nested1.yaml" not in r_str
+        assert "nested2.yml" not in r_str
+        assert "very_nested.yaml" not in r_str
+
+
+def test_list_yaml_files_excludes_secrets(tmp_path: Path) -> None:
+    """Test that secrets.yaml and secrets.yml are excluded."""
+    root = tmp_path / "configs"
+    root.mkdir()
+
+    # Create various YAML files including secrets
+    (root / "config.yaml").write_text("test: config")
+    (root / "secrets.yaml").write_text("wifi_password: secret123")
+    (root / "secrets.yml").write_text("api_key: secret456")
+    (root / "device.yaml").write_text("test: device")
+
+    result = util.list_yaml_files([str(root)])
+
+    # Should find 2 files (config.yaml and device.yaml), not secrets
+    assert len(result) == 2
+    assert root / "config.yaml" in result
+    assert root / "device.yaml" in result
+    assert root / "secrets.yaml" not in result
+    assert root / "secrets.yml" not in result
+
+
+def test_list_yaml_files_excludes_hidden_files(tmp_path: Path) -> None:
+    """Test that hidden files (starting with .) are excluded."""
+    root = tmp_path / "configs"
+    root.mkdir()
+
+    # Create regular and hidden YAML files
+    (root / "config.yaml").write_text("test: config")
+    (root / ".hidden.yaml").write_text("test: hidden")
+    (root / ".backup.yml").write_text("test: backup")
+    (root / "device.yaml").write_text("test: device")
+
+    result = util.list_yaml_files([str(root)])
+
+    # Should find only non-hidden files
+    assert len(result) == 2
+    assert root / "config.yaml" in result
+    assert root / "device.yaml" in result
+    assert root / ".hidden.yaml" not in result
+    assert root / ".backup.yml" not in result
+
+
+def test_filter_yaml_files_basic() -> None:
+    """Test filter_yaml_files function."""
+    files = [
+        Path("/path/to/config.yaml"),
+        Path("/path/to/device.yml"),
+        Path("/path/to/readme.txt"),
+        Path("/path/to/script.py"),
+        Path("/path/to/data.json"),
+        Path("/path/to/another.yaml"),
+    ]
+
+    result = util.filter_yaml_files(files)
+
+    assert len(result) == 3
+    assert Path("/path/to/config.yaml") in result
+    assert Path("/path/to/device.yml") in result
+    assert Path("/path/to/another.yaml") in result
+    assert Path("/path/to/readme.txt") not in result
+    assert Path("/path/to/script.py") not in result
+    assert Path("/path/to/data.json") not in result
+
+
+def test_filter_yaml_files_excludes_secrets() -> None:
+    """Test that filter_yaml_files excludes secrets files."""
+    files = [
+        Path("/path/to/config.yaml"),
+        Path("/path/to/secrets.yaml"),
+        Path("/path/to/secrets.yml"),
+        Path("/path/to/device.yaml"),
+        Path("/some/dir/secrets.yaml"),
+    ]
+
+    result = util.filter_yaml_files(files)
+
+    assert len(result) == 2
+    assert Path("/path/to/config.yaml") in result
+    assert Path("/path/to/device.yaml") in result
+    assert Path("/path/to/secrets.yaml") not in result
+    assert Path("/path/to/secrets.yml") not in result
+    assert Path("/some/dir/secrets.yaml") not in result
+
+
+def test_filter_yaml_files_excludes_hidden() -> None:
+    """Test that filter_yaml_files excludes hidden files."""
+    files = [
+        Path("/path/to/config.yaml"),
+        Path("/path/to/.hidden.yaml"),
+        Path("/path/to/.backup.yml"),
+        Path("/path/to/device.yaml"),
+        Path("/some/dir/.config.yaml"),
+    ]
+
+    result = util.filter_yaml_files(files)
+
+    assert len(result) == 2
+    assert Path("/path/to/config.yaml") in result
+    assert Path("/path/to/device.yaml") in result
+    assert Path("/path/to/.hidden.yaml") not in result
+    assert Path("/path/to/.backup.yml") not in result
+    assert Path("/some/dir/.config.yaml") not in result
+
+
+def test_filter_yaml_files_case_sensitive() -> None:
+    """Test that filter_yaml_files is case-sensitive for extensions."""
+    files = [
+        Path("/path/to/config.yaml"),
+        Path("/path/to/config.YAML"),
+        Path("/path/to/config.YML"),
+        Path("/path/to/config.Yaml"),
+        Path("/path/to/config.yml"),
+    ]
+
+    result = util.filter_yaml_files(files)
+
+    # Should only match lowercase .yaml and .yml
+    assert len(result) == 2
+
+    # Check the actual suffixes to ensure case-sensitive filtering
+    result_suffixes = [p.suffix for p in result]
+    assert ".yaml" in result_suffixes
+    assert ".yml" in result_suffixes
+
+    # Verify the filtered files have the expected names
+    result_names = [p.name for p in result]
+    assert "config.yaml" in result_names
+    assert "config.yml" in result_names
+    # Ensure uppercase extensions are NOT included
+    assert "config.YAML" not in result_names
+    assert "config.YML" not in result_names
+    assert "config.Yaml" not in result_names
+
+
+@pytest.mark.parametrize(
+    ("input_str", "expected"),
+    [
+        # Empty string
+        ("", "''"),
+        # Simple strings that don't need quoting
+        ("hello", "hello"),
+        ("test123", "test123"),
+        ("file.txt", "file.txt"),
+        ("/path/to/file", "/path/to/file"),
+        ("user@host", "user@host"),
+        ("value:123", "value:123"),
+        ("item,list", "item,list"),
+        ("path-with-dash", "path-with-dash"),
+        # Strings that need quoting
+        ("hello world", "'hello world'"),
+        ("test\ttab", "'test\ttab'"),
+        ("line\nbreak", "'line\nbreak'"),
+        ("semicolon;here", "'semicolon;here'"),
+        ("pipe|symbol", "'pipe|symbol'"),
+        ("redirect>file", "'redirect>file'"),
+        ("redirect None:
+    """Test shlex_quote properly escapes shell arguments."""
+    assert util.shlex_quote(input_str) == expected
+
+
+def test_shlex_quote_safe_characters() -> None:
+    """Test that safe characters are not quoted."""
+    # These characters are considered safe and shouldn't be quoted
+    safe_chars = (
+        "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789@%+=:,./-_"
+    )
+    for char in safe_chars:
+        assert util.shlex_quote(char) == char
+        assert util.shlex_quote(f"test{char}test") == f"test{char}test"
+
+
+def test_shlex_quote_unsafe_characters() -> None:
+    """Test that unsafe characters trigger quoting."""
+    # These characters should trigger quoting
+    unsafe_chars = ' \t\n;|>&<$`"\\?*[](){}!#~^'
+    for char in unsafe_chars:
+        result = util.shlex_quote(f"test{char}test")
+        assert result.startswith("'")
+        assert result.endswith("'")
+
+
+def test_shlex_quote_edge_cases() -> None:
+    """Test edge cases for shlex_quote."""
+    # Multiple single quotes
+    assert util.shlex_quote("'''") == "''\"'\"''\"'\"''\"'\"''"
+
+    # Mixed quotes
+    assert util.shlex_quote('"\'"') == "'\"'\"'\"'\"'"
+
+    # Only whitespace
+    assert util.shlex_quote(" ") == "' '"
+    assert util.shlex_quote("\t") == "'\t'"
+    assert util.shlex_quote("\n") == "'\n'"
+    assert util.shlex_quote("   ") == "'   '"
diff --git a/tests/unit_tests/test_vscode.py b/tests/unit_tests/test_vscode.py
index 6e0bde23b2..63bdf3e255 100644
--- a/tests/unit_tests/test_vscode.py
+++ b/tests/unit_tests/test_vscode.py
@@ -1,5 +1,5 @@
 import json
-import os
+from pathlib import Path
 from unittest.mock import Mock, patch
 
 from esphome import vscode
@@ -22,8 +22,7 @@ def _run_repl_test(input_data):
             call[0][0] for call in mock_stdout.write.call_args_list
         ).strip()
         splitted_output = full_output.split("\n")
-        remove_version = splitted_output[1:]  # remove first entry with version info
-        return remove_version
+        return splitted_output[1:]  # remove first entry with version info
 
 
 def _validate(file_path: str):
@@ -46,7 +45,7 @@ RESULT_NO_ERROR = '{"type": "result", "yaml_errors": [], "validation_errors": []
 
 
 def test_multi_file():
-    source_path = os.path.join("dir_path", "x.yaml")
+    source_path = str(Path("dir_path", "x.yaml"))
     output_lines = _run_repl_test(
         [
             _validate(source_path),
@@ -63,7 +62,7 @@ esp8266:
 
     expected_lines = [
         _read_file(source_path),
-        _read_file(os.path.join("dir_path", "secrets.yaml")),
+        _read_file(str(Path("dir_path", "secrets.yaml"))),
         RESULT_NO_ERROR,
     ]
 
@@ -71,7 +70,7 @@ esp8266:
 
 
 def test_shows_correct_range_error():
-    source_path = os.path.join("dir_path", "x.yaml")
+    source_path = str(Path("dir_path", "x.yaml"))
     output_lines = _run_repl_test(
         [
             _validate(source_path),
@@ -99,7 +98,7 @@ esp8266:
 
 
 def test_shows_correct_loaded_file_error():
-    source_path = os.path.join("dir_path", "x.yaml")
+    source_path = str(Path("dir_path", "x.yaml"))
     output_lines = _run_repl_test(
         [
             _validate(source_path),
@@ -122,7 +121,7 @@ packages:
     validation_error = error["validation_errors"][0]
     assert validation_error["message"].startswith("[broad] is an invalid option for")
     range = validation_error["range"]
-    assert range["document"] == os.path.join("dir_path", ".pkg.esp8266.yaml")
+    assert range["document"] == str(Path("dir_path", ".pkg.esp8266.yaml"))
     assert range["start_line"] == 1
     assert range["start_col"] == 2
     assert range["end_line"] == 1
diff --git a/tests/unit_tests/test_wizard.py b/tests/unit_tests/test_wizard.py
index ab20b2abb5..fd53a0b0b7 100644
--- a/tests/unit_tests/test_wizard.py
+++ b/tests/unit_tests/test_wizard.py
@@ -1,9 +1,11 @@
 """Tests for the wizard.py file."""
 
-import os
+from pathlib import Path
+from typing import Any
 from unittest.mock import MagicMock
 
 import pytest
+from pytest import MonkeyPatch
 
 from esphome.components.bk72xx.boards import BK72XX_BOARD_PINS
 from esphome.components.esp32.boards import ESP32_BOARD_PINS
@@ -15,8 +17,9 @@ import esphome.wizard as wz
 
 
 @pytest.fixture
-def default_config():
+def default_config() -> dict[str, Any]:
     return {
+        "type": "basic",
         "name": "test-name",
         "platform": "ESP8266",
         "board": "esp01_1m",
@@ -27,7 +30,7 @@ def default_config():
 
 
 @pytest.fixture
-def wizard_answers():
+def wizard_answers() -> list[str]:
     return [
         "test-node",  # Name of the node
         "ESP8266",  # platform
@@ -52,7 +55,9 @@ def test_sanitize_quotes_replaces_with_escaped_char():
     assert output_str == '\\"key\\": \\"value\\"'
 
 
-def test_config_file_fallback_ap_includes_descriptive_name(default_config):
+def test_config_file_fallback_ap_includes_descriptive_name(
+    default_config: dict[str, Any],
+):
     """
     The fallback AP should include the node and a descriptive name
     """
@@ -66,7 +71,9 @@ def test_config_file_fallback_ap_includes_descriptive_name(default_config):
     assert 'ssid: "Test Node Fallback Hotspot"' in config
 
 
-def test_config_file_fallback_ap_name_less_than_32_chars(default_config):
+def test_config_file_fallback_ap_name_less_than_32_chars(
+    default_config: dict[str, Any],
+):
     """
     The fallback AP name must be less than 32 chars.
     Since it is composed of the node name and "Fallback Hotspot" this can be too long and needs truncating
@@ -81,7 +88,7 @@ def test_config_file_fallback_ap_name_less_than_32_chars(default_config):
     assert 'ssid: "A Very Long Name For This Node"' in config
 
 
-def test_config_file_should_include_ota(default_config):
+def test_config_file_should_include_ota(default_config: dict[str, Any]):
     """
     The Over-The-Air update should be enabled by default
     """
@@ -94,7 +101,9 @@ def test_config_file_should_include_ota(default_config):
     assert "ota:" in config
 
 
-def test_config_file_should_include_ota_when_password_set(default_config):
+def test_config_file_should_include_ota_when_password_set(
+    default_config: dict[str, Any],
+):
     """
     The Over-The-Air update should be enabled when a password is set
     """
@@ -108,14 +117,16 @@ def test_config_file_should_include_ota_when_password_set(default_config):
     assert "ota:" in config
 
 
-def test_wizard_write_sets_platform(default_config, tmp_path, monkeypatch):
+def test_wizard_write_sets_platform(
+    default_config: dict[str, Any], tmp_path: Path, monkeypatch: MonkeyPatch
+):
     """
     If the platform is not explicitly set, use "ESP8266" if the board is one of the ESP8266 boards
     """
     # Given
     del default_config["platform"]
     monkeypatch.setattr(wz, "write_file", MagicMock())
-    monkeypatch.setattr(CORE, "config_path", os.path.dirname(tmp_path))
+    monkeypatch.setattr(CORE, "config_path", tmp_path.parent)
 
     # When
     wz.wizard_write(tmp_path, **default_config)
@@ -125,8 +136,49 @@ def test_wizard_write_sets_platform(default_config, tmp_path, monkeypatch):
     assert "esp8266:" in generated_config
 
 
+def test_wizard_empty_config(tmp_path: Path, monkeypatch: MonkeyPatch):
+    """
+    The wizard should be able to create an empty configuration
+    """
+    # Given
+    empty_config = {
+        "type": "empty",
+        "name": "test-empty",
+    }
+    monkeypatch.setattr(wz, "write_file", MagicMock())
+    monkeypatch.setattr(CORE, "config_path", tmp_path.parent)
+
+    # When
+    wz.wizard_write(tmp_path, **empty_config)
+
+    # Then
+    generated_config = wz.write_file.call_args.args[1]
+    assert generated_config == ""
+
+
+def test_wizard_upload_config(tmp_path: Path, monkeypatch: MonkeyPatch):
+    """
+    The wizard should be able to import an base64 encoded configuration
+    """
+    # Given
+    empty_config = {
+        "type": "upload",
+        "name": "test-upload",
+        "file_text": "# imported file 📁\n\n",
+    }
+    monkeypatch.setattr(wz, "write_file", MagicMock())
+    monkeypatch.setattr(CORE, "config_path", tmp_path.parent)
+
+    # When
+    wz.wizard_write(tmp_path, **empty_config)
+
+    # Then
+    generated_config = wz.write_file.call_args.args[1]
+    assert generated_config == "# imported file 📁\n\n"
+
+
 def test_wizard_write_defaults_platform_from_board_esp8266(
-    default_config, tmp_path, monkeypatch
+    default_config: dict[str, Any], tmp_path: Path, monkeypatch: MonkeyPatch
 ):
     """
     If the platform is not explicitly set, use "ESP8266" if the board is one of the ESP8266 boards
@@ -136,7 +188,7 @@ def test_wizard_write_defaults_platform_from_board_esp8266(
     default_config["board"] = [*ESP8266_BOARD_PINS][0]
 
     monkeypatch.setattr(wz, "write_file", MagicMock())
-    monkeypatch.setattr(CORE, "config_path", os.path.dirname(tmp_path))
+    monkeypatch.setattr(CORE, "config_path", tmp_path.parent)
 
     # When
     wz.wizard_write(tmp_path, **default_config)
@@ -147,7 +199,7 @@ def test_wizard_write_defaults_platform_from_board_esp8266(
 
 
 def test_wizard_write_defaults_platform_from_board_esp32(
-    default_config, tmp_path, monkeypatch
+    default_config: dict[str, Any], tmp_path: Path, monkeypatch: MonkeyPatch
 ):
     """
     If the platform is not explicitly set, use "ESP32" if the board is one of the ESP32 boards
@@ -157,7 +209,7 @@ def test_wizard_write_defaults_platform_from_board_esp32(
     default_config["board"] = [*ESP32_BOARD_PINS][0]
 
     monkeypatch.setattr(wz, "write_file", MagicMock())
-    monkeypatch.setattr(CORE, "config_path", os.path.dirname(tmp_path))
+    monkeypatch.setattr(CORE, "config_path", tmp_path.parent)
 
     # When
     wz.wizard_write(tmp_path, **default_config)
@@ -168,7 +220,7 @@ def test_wizard_write_defaults_platform_from_board_esp32(
 
 
 def test_wizard_write_defaults_platform_from_board_bk72xx(
-    default_config, tmp_path, monkeypatch
+    default_config: dict[str, Any], tmp_path: Path, monkeypatch: MonkeyPatch
 ):
     """
     If the platform is not explicitly set, use "BK72XX" if the board is one of BK72XX boards
@@ -178,7 +230,7 @@ def test_wizard_write_defaults_platform_from_board_bk72xx(
     default_config["board"] = [*BK72XX_BOARD_PINS][0]
 
     monkeypatch.setattr(wz, "write_file", MagicMock())
-    monkeypatch.setattr(CORE, "config_path", os.path.dirname(tmp_path))
+    monkeypatch.setattr(CORE, "config_path", tmp_path.parent)
 
     # When
     wz.wizard_write(tmp_path, **default_config)
@@ -189,7 +241,7 @@ def test_wizard_write_defaults_platform_from_board_bk72xx(
 
 
 def test_wizard_write_defaults_platform_from_board_ln882x(
-    default_config, tmp_path, monkeypatch
+    default_config: dict[str, Any], tmp_path: Path, monkeypatch: MonkeyPatch
 ):
     """
     If the platform is not explicitly set, use "LN882X" if the board is one of LN882X boards
@@ -199,7 +251,7 @@ def test_wizard_write_defaults_platform_from_board_ln882x(
     default_config["board"] = [*LN882X_BOARD_PINS][0]
 
     monkeypatch.setattr(wz, "write_file", MagicMock())
-    monkeypatch.setattr(CORE, "config_path", os.path.dirname(tmp_path))
+    monkeypatch.setattr(CORE, "config_path", tmp_path.parent)
 
     # When
     wz.wizard_write(tmp_path, **default_config)
@@ -210,7 +262,7 @@ def test_wizard_write_defaults_platform_from_board_ln882x(
 
 
 def test_wizard_write_defaults_platform_from_board_rtl87xx(
-    default_config, tmp_path, monkeypatch
+    default_config: dict[str, Any], tmp_path: Path, monkeypatch: MonkeyPatch
 ):
     """
     If the platform is not explicitly set, use "RTL87XX" if the board is one of RTL87XX boards
@@ -220,7 +272,7 @@ def test_wizard_write_defaults_platform_from_board_rtl87xx(
     default_config["board"] = [*RTL87XX_BOARD_PINS][0]
 
     monkeypatch.setattr(wz, "write_file", MagicMock())
-    monkeypatch.setattr(CORE, "config_path", os.path.dirname(tmp_path))
+    monkeypatch.setattr(CORE, "config_path", tmp_path.parent)
 
     # When
     wz.wizard_write(tmp_path, **default_config)
@@ -230,7 +282,7 @@ def test_wizard_write_defaults_platform_from_board_rtl87xx(
     assert "rtl87xx:" in generated_config
 
 
-def test_safe_print_step_prints_step_number_and_description(monkeypatch):
+def test_safe_print_step_prints_step_number_and_description(monkeypatch: MonkeyPatch):
     """
     The safe_print_step function prints the step number and the passed description
     """
@@ -254,7 +306,7 @@ def test_safe_print_step_prints_step_number_and_description(monkeypatch):
     assert any(f"STEP {step_num}" in arg for arg in all_args)
 
 
-def test_default_input_uses_default_if_no_input_supplied(monkeypatch):
+def test_default_input_uses_default_if_no_input_supplied(monkeypatch: MonkeyPatch):
     """
     The default_input() function should return the supplied default value if the user doesn't enter anything
     """
@@ -270,7 +322,7 @@ def test_default_input_uses_default_if_no_input_supplied(monkeypatch):
     assert retval == default_string
 
 
-def test_default_input_uses_user_supplied_value(monkeypatch):
+def test_default_input_uses_user_supplied_value(monkeypatch: MonkeyPatch):
     """
     The default_input() function should return the value that the user enters
     """
@@ -309,7 +361,7 @@ def test_wizard_rejects_path_with_invalid_extension():
     """
 
     # Given
-    config_file = "test.json"
+    config_file = Path("test.json")
 
     # When
     retval = wz.wizard(config_file)
@@ -318,29 +370,31 @@ def test_wizard_rejects_path_with_invalid_extension():
     assert retval == 1
 
 
-def test_wizard_rejects_existing_files(tmpdir):
+def test_wizard_rejects_existing_files(tmp_path):
     """
     The wizard should reject any configuration file that already exists
     """
 
     # Given
-    config_file = tmpdir.join("test.yaml")
-    config_file.write("")
+    config_file = tmp_path / "test.yaml"
+    config_file.write_text("")
 
     # When
-    retval = wz.wizard(str(config_file))
+    retval = wz.wizard(config_file)
 
     # Then
     assert retval == 2
 
 
-def test_wizard_accepts_default_answers_esp8266(tmpdir, monkeypatch, wizard_answers):
+def test_wizard_accepts_default_answers_esp8266(
+    tmp_path: Path, monkeypatch: MonkeyPatch, wizard_answers: list[str]
+):
     """
     The wizard should accept the given default answers for esp8266
     """
 
     # Given
-    config_file = tmpdir.join("test.yaml")
+    config_file = tmp_path / "test.yaml"
     input_mock = MagicMock(side_effect=wizard_answers)
     monkeypatch.setattr("builtins.input", input_mock)
     monkeypatch.setattr(wz, "safe_print", lambda t=None, end=None: 0)
@@ -348,13 +402,15 @@ def test_wizard_accepts_default_answers_esp8266(tmpdir, monkeypatch, wizard_answ
     monkeypatch.setattr(wz, "wizard_write", MagicMock())
 
     # When
-    retval = wz.wizard(str(config_file))
+    retval = wz.wizard(config_file)
 
     # Then
     assert retval == 0
 
 
-def test_wizard_accepts_default_answers_esp32(tmpdir, monkeypatch, wizard_answers):
+def test_wizard_accepts_default_answers_esp32(
+    tmp_path: Path, monkeypatch: MonkeyPatch, wizard_answers: list[str]
+):
     """
     The wizard should accept the given default answers for esp32
     """
@@ -362,7 +418,7 @@ def test_wizard_accepts_default_answers_esp32(tmpdir, monkeypatch, wizard_answer
     # Given
     wizard_answers[1] = "ESP32"
     wizard_answers[2] = "nodemcu-32s"
-    config_file = tmpdir.join("test.yaml")
+    config_file = tmp_path / "test.yaml"
     input_mock = MagicMock(side_effect=wizard_answers)
     monkeypatch.setattr("builtins.input", input_mock)
     monkeypatch.setattr(wz, "safe_print", lambda t=None, end=None: 0)
@@ -370,13 +426,15 @@ def test_wizard_accepts_default_answers_esp32(tmpdir, monkeypatch, wizard_answer
     monkeypatch.setattr(wz, "wizard_write", MagicMock())
 
     # When
-    retval = wz.wizard(str(config_file))
+    retval = wz.wizard(config_file)
 
     # Then
     assert retval == 0
 
 
-def test_wizard_offers_better_node_name(tmpdir, monkeypatch, wizard_answers):
+def test_wizard_offers_better_node_name(
+    tmp_path: Path, monkeypatch: MonkeyPatch, wizard_answers: list[str]
+):
     """
     When the node name does not conform, a better alternative is offered
     * Removes special chars
@@ -392,7 +450,7 @@ def test_wizard_offers_better_node_name(tmpdir, monkeypatch, wizard_answers):
         wz, "default_input", MagicMock(side_effect=lambda _, default: default)
     )
 
-    config_file = tmpdir.join("test.yaml")
+    config_file = tmp_path / "test.yaml"
     input_mock = MagicMock(side_effect=wizard_answers)
     monkeypatch.setattr("builtins.input", input_mock)
     monkeypatch.setattr(wz, "safe_print", lambda t=None, end=None: 0)
@@ -400,14 +458,16 @@ def test_wizard_offers_better_node_name(tmpdir, monkeypatch, wizard_answers):
     monkeypatch.setattr(wz, "wizard_write", MagicMock())
 
     # When
-    retval = wz.wizard(str(config_file))
+    retval = wz.wizard(config_file)
 
     # Then
     assert retval == 0
     assert wz.default_input.call_args.args[1] == expected_name
 
 
-def test_wizard_requires_correct_platform(tmpdir, monkeypatch, wizard_answers):
+def test_wizard_requires_correct_platform(
+    tmp_path: Path, monkeypatch: MonkeyPatch, wizard_answers: list[str]
+):
     """
     When the platform is not either esp32 or esp8266, the wizard should reject it
     """
@@ -415,7 +475,7 @@ def test_wizard_requires_correct_platform(tmpdir, monkeypatch, wizard_answers):
     # Given
     wizard_answers.insert(1, "foobar")  # add invalid entry for platform
 
-    config_file = tmpdir.join("test.yaml")
+    config_file = tmp_path / "test.yaml"
     input_mock = MagicMock(side_effect=wizard_answers)
     monkeypatch.setattr("builtins.input", input_mock)
     monkeypatch.setattr(wz, "safe_print", lambda t=None, end=None: 0)
@@ -423,13 +483,15 @@ def test_wizard_requires_correct_platform(tmpdir, monkeypatch, wizard_answers):
     monkeypatch.setattr(wz, "wizard_write", MagicMock())
 
     # When
-    retval = wz.wizard(str(config_file))
+    retval = wz.wizard(config_file)
 
     # Then
     assert retval == 0
 
 
-def test_wizard_requires_correct_board(tmpdir, monkeypatch, wizard_answers):
+def test_wizard_requires_correct_board(
+    tmp_path: Path, monkeypatch: MonkeyPatch, wizard_answers: list[str]
+):
     """
     When the board is not a valid esp8266 board, the wizard should reject it
     """
@@ -437,7 +499,7 @@ def test_wizard_requires_correct_board(tmpdir, monkeypatch, wizard_answers):
     # Given
     wizard_answers.insert(2, "foobar")  # add an invalid entry for board
 
-    config_file = tmpdir.join("test.yaml")
+    config_file = tmp_path / "test.yaml"
     input_mock = MagicMock(side_effect=wizard_answers)
     monkeypatch.setattr("builtins.input", input_mock)
     monkeypatch.setattr(wz, "safe_print", lambda t=None, end=None: 0)
@@ -445,13 +507,15 @@ def test_wizard_requires_correct_board(tmpdir, monkeypatch, wizard_answers):
     monkeypatch.setattr(wz, "wizard_write", MagicMock())
 
     # When
-    retval = wz.wizard(str(config_file))
+    retval = wz.wizard(config_file)
 
     # Then
     assert retval == 0
 
 
-def test_wizard_requires_valid_ssid(tmpdir, monkeypatch, wizard_answers):
+def test_wizard_requires_valid_ssid(
+    tmp_path: Path, monkeypatch: MonkeyPatch, wizard_answers: list[str]
+):
     """
     When the board is not a valid esp8266 board, the wizard should reject it
     """
@@ -459,7 +523,7 @@ def test_wizard_requires_valid_ssid(tmpdir, monkeypatch, wizard_answers):
     # Given
     wizard_answers.insert(3, "")  # add an invalid entry for ssid
 
-    config_file = tmpdir.join("test.yaml")
+    config_file = tmp_path / "test.yaml"
     input_mock = MagicMock(side_effect=wizard_answers)
     monkeypatch.setattr("builtins.input", input_mock)
     monkeypatch.setattr(wz, "safe_print", lambda t=None, end=None: 0)
@@ -467,7 +531,28 @@ def test_wizard_requires_valid_ssid(tmpdir, monkeypatch, wizard_answers):
     monkeypatch.setattr(wz, "wizard_write", MagicMock())
 
     # When
-    retval = wz.wizard(str(config_file))
+    retval = wz.wizard(config_file)
 
     # Then
     assert retval == 0
+
+
+def test_wizard_write_protects_existing_config(
+    tmp_path: Path, default_config: dict[str, Any], monkeypatch: MonkeyPatch
+):
+    """
+    The wizard_write function should not overwrite existing config files and return False
+    """
+    # Given
+    config_file = tmp_path / "test.yaml"
+    original_content = "# Original config content\n"
+    config_file.write_text(original_content)
+
+    monkeypatch.setattr(CORE, "config_path", tmp_path.parent)
+
+    # When
+    result = wz.wizard_write(config_file, **default_config)
+
+    # Then
+    assert result is False  # Should return False when file exists
+    assert config_file.read_text() == original_content
diff --git a/tests/unit_tests/test_writer.py b/tests/unit_tests/test_writer.py
new file mode 100644
index 0000000000..a4490fbbc0
--- /dev/null
+++ b/tests/unit_tests/test_writer.py
@@ -0,0 +1,1033 @@
+"""Test writer module functionality."""
+
+from collections.abc import Callable
+from pathlib import Path
+from typing import Any
+from unittest.mock import MagicMock, patch
+
+import pytest
+
+from esphome.core import EsphomeError
+from esphome.storage_json import StorageJSON
+from esphome.writer import (
+    CPP_AUTO_GENERATE_BEGIN,
+    CPP_AUTO_GENERATE_END,
+    CPP_INCLUDE_BEGIN,
+    CPP_INCLUDE_END,
+    GITIGNORE_CONTENT,
+    clean_build,
+    clean_cmake_cache,
+    storage_should_clean,
+    update_storage_json,
+    write_cpp,
+    write_gitignore,
+)
+
+
+@pytest.fixture
+def mock_copy_src_tree():
+    """Mock copy_src_tree to avoid side effects during tests."""
+    with patch("esphome.writer.copy_src_tree"):
+        yield
+
+
+@pytest.fixture
+def create_storage() -> Callable[..., StorageJSON]:
+    """Factory fixture to create StorageJSON instances."""
+
+    def _create(
+        loaded_integrations: list[str] | None = None, **kwargs: Any
+    ) -> StorageJSON:
+        return StorageJSON(
+            storage_version=kwargs.get("storage_version", 1),
+            name=kwargs.get("name", "test"),
+            friendly_name=kwargs.get("friendly_name", "Test Device"),
+            comment=kwargs.get("comment"),
+            esphome_version=kwargs.get("esphome_version", "2025.1.0"),
+            src_version=kwargs.get("src_version", 1),
+            address=kwargs.get("address", "test.local"),
+            web_port=kwargs.get("web_port", 80),
+            target_platform=kwargs.get("target_platform", "ESP32"),
+            build_path=kwargs.get("build_path", "/build"),
+            firmware_bin_path=kwargs.get("firmware_bin_path", "/firmware.bin"),
+            loaded_integrations=set(loaded_integrations or []),
+            loaded_platforms=kwargs.get("loaded_platforms", set()),
+            no_mdns=kwargs.get("no_mdns", False),
+            framework=kwargs.get("framework", "arduino"),
+            core_platform=kwargs.get("core_platform", "esp32"),
+        )
+
+    return _create
+
+
+def test_storage_should_clean_when_old_is_none(
+    create_storage: Callable[..., StorageJSON],
+) -> None:
+    """Test that clean is triggered when old storage is None."""
+    new = create_storage(loaded_integrations=["api", "wifi"])
+    assert storage_should_clean(None, new) is True
+
+
+def test_storage_should_clean_when_src_version_changes(
+    create_storage: Callable[..., StorageJSON],
+) -> None:
+    """Test that clean is triggered when src_version changes."""
+    old = create_storage(loaded_integrations=["api", "wifi"], src_version=1)
+    new = create_storage(loaded_integrations=["api", "wifi"], src_version=2)
+    assert storage_should_clean(old, new) is True
+
+
+def test_storage_should_clean_when_build_path_changes(
+    create_storage: Callable[..., StorageJSON],
+) -> None:
+    """Test that clean is triggered when build_path changes."""
+    old = create_storage(loaded_integrations=["api", "wifi"], build_path="/build1")
+    new = create_storage(loaded_integrations=["api", "wifi"], build_path="/build2")
+    assert storage_should_clean(old, new) is True
+
+
+def test_storage_should_clean_when_component_removed(
+    create_storage: Callable[..., StorageJSON],
+) -> None:
+    """Test that clean is triggered when a component is removed."""
+    old = create_storage(
+        loaded_integrations=["api", "wifi", "bluetooth_proxy", "esp32_ble_tracker"]
+    )
+    new = create_storage(loaded_integrations=["api", "wifi", "esp32_ble_tracker"])
+    assert storage_should_clean(old, new) is True
+
+
+def test_storage_should_clean_when_multiple_components_removed(
+    create_storage: Callable[..., StorageJSON],
+) -> None:
+    """Test that clean is triggered when multiple components are removed."""
+    old = create_storage(
+        loaded_integrations=["api", "wifi", "ota", "web_server", "logger"]
+    )
+    new = create_storage(loaded_integrations=["api", "wifi", "logger"])
+    assert storage_should_clean(old, new) is True
+
+
+def test_storage_should_not_clean_when_nothing_changes(
+    create_storage: Callable[..., StorageJSON],
+) -> None:
+    """Test that clean is not triggered when nothing changes."""
+    old = create_storage(loaded_integrations=["api", "wifi", "logger"])
+    new = create_storage(loaded_integrations=["api", "wifi", "logger"])
+    assert storage_should_clean(old, new) is False
+
+
+def test_storage_should_not_clean_when_component_added(
+    create_storage: Callable[..., StorageJSON],
+) -> None:
+    """Test that clean is not triggered when a component is only added."""
+    old = create_storage(loaded_integrations=["api", "wifi"])
+    new = create_storage(loaded_integrations=["api", "wifi", "ota"])
+    assert storage_should_clean(old, new) is False
+
+
+def test_storage_should_not_clean_when_other_fields_change(
+    create_storage: Callable[..., StorageJSON],
+) -> None:
+    """Test that clean is not triggered when non-relevant fields change."""
+    old = create_storage(
+        loaded_integrations=["api", "wifi"],
+        friendly_name="Old Name",
+        esphome_version="2024.12.0",
+    )
+    new = create_storage(
+        loaded_integrations=["api", "wifi"],
+        friendly_name="New Name",
+        esphome_version="2025.1.0",
+    )
+    assert storage_should_clean(old, new) is False
+
+
+def test_storage_edge_case_empty_integrations(
+    create_storage: Callable[..., StorageJSON],
+) -> None:
+    """Test edge case when old has integrations but new has none."""
+    old = create_storage(loaded_integrations=["api", "wifi"])
+    new = create_storage(loaded_integrations=[])
+    assert storage_should_clean(old, new) is True
+
+
+def test_storage_edge_case_from_empty_integrations(
+    create_storage: Callable[..., StorageJSON],
+) -> None:
+    """Test edge case when old has no integrations but new has some."""
+    old = create_storage(loaded_integrations=[])
+    new = create_storage(loaded_integrations=["api", "wifi"])
+    assert storage_should_clean(old, new) is False
+
+
+@patch("esphome.writer.clean_build")
+@patch("esphome.writer.StorageJSON")
+@patch("esphome.writer.storage_path")
+@patch("esphome.writer.CORE")
+def test_update_storage_json_logging_when_old_is_none(
+    mock_core: MagicMock,
+    mock_storage_path: MagicMock,
+    mock_storage_json_class: MagicMock,
+    mock_clean_build: MagicMock,
+    create_storage: Callable[..., StorageJSON],
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test that update_storage_json doesn't crash when old storage is None.
+
+    This is a regression test for the AttributeError that occurred when
+    old was None and we tried to access old.loaded_integrations.
+    """
+    # Setup mocks
+    mock_storage_path.return_value = "/test/path"
+    mock_storage_json_class.load.return_value = None  # Old storage is None
+
+    new_storage = create_storage(loaded_integrations=["api", "wifi"])
+    new_storage.save = MagicMock()  # Mock the save method
+    mock_storage_json_class.from_esphome_core.return_value = new_storage
+
+    # Call the function - should not raise AttributeError
+    with caplog.at_level("INFO"):
+        update_storage_json()
+
+    # Verify clean_build was called
+    mock_clean_build.assert_called_once()
+
+    # Verify the correct log message was used (not the component removal message)
+    assert "Core config or version changed, cleaning build files..." in caplog.text
+    assert "Components removed" not in caplog.text
+
+    # Verify save was called
+    new_storage.save.assert_called_once_with("/test/path")
+
+
+@patch("esphome.writer.clean_build")
+@patch("esphome.writer.StorageJSON")
+@patch("esphome.writer.storage_path")
+@patch("esphome.writer.CORE")
+def test_update_storage_json_logging_components_removed(
+    mock_core: MagicMock,
+    mock_storage_path: MagicMock,
+    mock_storage_json_class: MagicMock,
+    mock_clean_build: MagicMock,
+    create_storage: Callable[..., StorageJSON],
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test that update_storage_json logs removed components correctly."""
+    # Setup mocks
+    mock_storage_path.return_value = "/test/path"
+
+    old_storage = create_storage(loaded_integrations=["api", "wifi", "bluetooth_proxy"])
+    new_storage = create_storage(loaded_integrations=["api", "wifi"])
+    new_storage.save = MagicMock()  # Mock the save method
+
+    mock_storage_json_class.load.return_value = old_storage
+    mock_storage_json_class.from_esphome_core.return_value = new_storage
+
+    # Call the function
+    with caplog.at_level("INFO"):
+        update_storage_json()
+
+    # Verify clean_build was called
+    mock_clean_build.assert_called_once()
+
+    # Verify the correct log message was used with component names
+    assert (
+        "Components removed (bluetooth_proxy), cleaning build files..." in caplog.text
+    )
+    assert "Core config or version changed" not in caplog.text
+
+    # Verify save was called
+    new_storage.save.assert_called_once_with("/test/path")
+
+
+@patch("esphome.writer.CORE")
+def test_clean_cmake_cache(
+    mock_core: MagicMock,
+    tmp_path: Path,
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test clean_cmake_cache removes CMakeCache.txt file."""
+    # Create directory structure
+    pioenvs_dir = tmp_path / ".pioenvs"
+    pioenvs_dir.mkdir()
+    device_dir = pioenvs_dir / "test_device"
+    device_dir.mkdir()
+    cmake_cache_file = device_dir / "CMakeCache.txt"
+    cmake_cache_file.write_text("# CMake cache file")
+
+    # Setup mocks
+    mock_core.relative_pioenvs_path.return_value = pioenvs_dir
+    mock_core.name = "test_device"
+
+    # Verify file exists before
+    assert cmake_cache_file.exists()
+
+    # Call the function
+    with caplog.at_level("INFO"):
+        clean_cmake_cache()
+
+    # Verify file was removed
+    assert not cmake_cache_file.exists()
+
+    # Verify logging
+    assert "Deleting" in caplog.text
+    assert "CMakeCache.txt" in caplog.text
+
+
+@patch("esphome.writer.CORE")
+def test_clean_cmake_cache_no_pioenvs_dir(
+    mock_core: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test clean_cmake_cache when pioenvs directory doesn't exist."""
+    # Setup non-existent directory path
+    pioenvs_dir = tmp_path / ".pioenvs"
+
+    # Setup mocks
+    mock_core.relative_pioenvs_path.return_value = pioenvs_dir
+
+    # Verify directory doesn't exist
+    assert not pioenvs_dir.exists()
+
+    # Call the function - should not crash
+    clean_cmake_cache()
+
+    # Verify directory still doesn't exist
+    assert not pioenvs_dir.exists()
+
+
+@patch("esphome.writer.CORE")
+def test_clean_cmake_cache_no_cmake_file(
+    mock_core: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test clean_cmake_cache when CMakeCache.txt doesn't exist."""
+    # Create directory structure without CMakeCache.txt
+    pioenvs_dir = tmp_path / ".pioenvs"
+    pioenvs_dir.mkdir()
+    device_dir = pioenvs_dir / "test_device"
+    device_dir.mkdir()
+    cmake_cache_file = device_dir / "CMakeCache.txt"
+
+    # Setup mocks
+    mock_core.relative_pioenvs_path.return_value = pioenvs_dir
+    mock_core.name = "test_device"
+
+    # Verify file doesn't exist
+    assert not cmake_cache_file.exists()
+
+    # Call the function - should not crash
+    clean_cmake_cache()
+
+    # Verify file still doesn't exist
+    assert not cmake_cache_file.exists()
+
+
+@patch("esphome.writer.CORE")
+def test_clean_build(
+    mock_core: MagicMock,
+    tmp_path: Path,
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test clean_build removes all build artifacts."""
+    # Create directory structure and files
+    pioenvs_dir = tmp_path / ".pioenvs"
+    pioenvs_dir.mkdir()
+    (pioenvs_dir / "test_file.o").write_text("object file")
+
+    piolibdeps_dir = tmp_path / ".piolibdeps"
+    piolibdeps_dir.mkdir()
+    (piolibdeps_dir / "library").mkdir()
+
+    dependencies_lock = tmp_path / "dependencies.lock"
+    dependencies_lock.write_text("lock file")
+
+    # Create PlatformIO cache directory
+    platformio_cache_dir = tmp_path / ".platformio" / ".cache"
+    platformio_cache_dir.mkdir(parents=True)
+    (platformio_cache_dir / "downloads").mkdir()
+    (platformio_cache_dir / "http").mkdir()
+    (platformio_cache_dir / "tmp").mkdir()
+    (platformio_cache_dir / "downloads" / "package.tar.gz").write_text("package")
+
+    # Setup mocks
+    mock_core.relative_pioenvs_path.return_value = pioenvs_dir
+    mock_core.relative_piolibdeps_path.return_value = piolibdeps_dir
+    mock_core.relative_build_path.return_value = dependencies_lock
+
+    # Verify all exist before
+    assert pioenvs_dir.exists()
+    assert piolibdeps_dir.exists()
+    assert dependencies_lock.exists()
+    assert platformio_cache_dir.exists()
+
+    # Mock PlatformIO's ProjectConfig cache_dir
+    with patch(
+        "platformio.project.config.ProjectConfig.get_instance"
+    ) as mock_get_instance:
+        mock_config = MagicMock()
+        mock_get_instance.return_value = mock_config
+        mock_config.get.side_effect = (
+            lambda section, option: str(platformio_cache_dir)
+            if (section, option) == ("platformio", "cache_dir")
+            else ""
+        )
+
+        # Call the function
+        with caplog.at_level("INFO"):
+            clean_build()
+
+    # Verify all were removed
+    assert not pioenvs_dir.exists()
+    assert not piolibdeps_dir.exists()
+    assert not dependencies_lock.exists()
+    assert not platformio_cache_dir.exists()
+
+    # Verify logging
+    assert "Deleting" in caplog.text
+    assert ".pioenvs" in caplog.text
+    assert ".piolibdeps" in caplog.text
+    assert "dependencies.lock" in caplog.text
+    assert "PlatformIO cache" in caplog.text
+
+
+@patch("esphome.writer.CORE")
+def test_clean_build_partial_exists(
+    mock_core: MagicMock,
+    tmp_path: Path,
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test clean_build when only some paths exist."""
+    # Create only pioenvs directory
+    pioenvs_dir = tmp_path / ".pioenvs"
+    pioenvs_dir.mkdir()
+    (pioenvs_dir / "test_file.o").write_text("object file")
+
+    piolibdeps_dir = tmp_path / ".piolibdeps"
+    dependencies_lock = tmp_path / "dependencies.lock"
+
+    # Setup mocks
+    mock_core.relative_pioenvs_path.return_value = pioenvs_dir
+    mock_core.relative_piolibdeps_path.return_value = piolibdeps_dir
+    mock_core.relative_build_path.return_value = dependencies_lock
+
+    # Verify only pioenvs exists
+    assert pioenvs_dir.exists()
+    assert not piolibdeps_dir.exists()
+    assert not dependencies_lock.exists()
+
+    # Call the function
+    with caplog.at_level("INFO"):
+        clean_build()
+
+    # Verify only existing path was removed
+    assert not pioenvs_dir.exists()
+    assert not piolibdeps_dir.exists()
+    assert not dependencies_lock.exists()
+
+    # Verify logging - only pioenvs should be logged
+    assert "Deleting" in caplog.text
+    assert ".pioenvs" in caplog.text
+    assert ".piolibdeps" not in caplog.text
+    assert "dependencies.lock" not in caplog.text
+
+
+@patch("esphome.writer.CORE")
+def test_clean_build_nothing_exists(
+    mock_core: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test clean_build when no build artifacts exist."""
+    # Setup paths that don't exist
+    pioenvs_dir = tmp_path / ".pioenvs"
+    piolibdeps_dir = tmp_path / ".piolibdeps"
+    dependencies_lock = tmp_path / "dependencies.lock"
+
+    # Setup mocks
+    mock_core.relative_pioenvs_path.return_value = pioenvs_dir
+    mock_core.relative_piolibdeps_path.return_value = piolibdeps_dir
+    mock_core.relative_build_path.return_value = dependencies_lock
+
+    # Verify nothing exists
+    assert not pioenvs_dir.exists()
+    assert not piolibdeps_dir.exists()
+    assert not dependencies_lock.exists()
+
+    # Call the function - should not crash
+    clean_build()
+
+    # Verify nothing was created
+    assert not pioenvs_dir.exists()
+    assert not piolibdeps_dir.exists()
+    assert not dependencies_lock.exists()
+
+
+@patch("esphome.writer.CORE")
+def test_clean_build_platformio_not_available(
+    mock_core: MagicMock,
+    tmp_path: Path,
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test clean_build when PlatformIO is not available."""
+    # Create directory structure and files
+    pioenvs_dir = tmp_path / ".pioenvs"
+    pioenvs_dir.mkdir()
+
+    piolibdeps_dir = tmp_path / ".piolibdeps"
+    piolibdeps_dir.mkdir()
+
+    dependencies_lock = tmp_path / "dependencies.lock"
+    dependencies_lock.write_text("lock file")
+
+    # Setup mocks
+    mock_core.relative_pioenvs_path.return_value = pioenvs_dir
+    mock_core.relative_piolibdeps_path.return_value = piolibdeps_dir
+    mock_core.relative_build_path.return_value = dependencies_lock
+
+    # Verify all exist before
+    assert pioenvs_dir.exists()
+    assert piolibdeps_dir.exists()
+    assert dependencies_lock.exists()
+
+    # Mock import error for platformio
+    with (
+        patch.dict("sys.modules", {"platformio.project.config": None}),
+        caplog.at_level("INFO"),
+    ):
+        # Call the function
+        clean_build()
+
+    # Verify standard paths were removed but no cache cleaning attempted
+    assert not pioenvs_dir.exists()
+    assert not piolibdeps_dir.exists()
+    assert not dependencies_lock.exists()
+
+    # Verify no cache logging
+    assert "PlatformIO cache" not in caplog.text
+
+
+@patch("esphome.writer.CORE")
+def test_clean_build_empty_cache_dir(
+    mock_core: MagicMock,
+    tmp_path: Path,
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test clean_build when get_project_cache_dir returns empty/whitespace."""
+    # Create directory structure and files
+    pioenvs_dir = tmp_path / ".pioenvs"
+    pioenvs_dir.mkdir()
+
+    # Setup mocks
+    mock_core.relative_pioenvs_path.return_value = pioenvs_dir
+    mock_core.relative_piolibdeps_path.return_value = tmp_path / ".piolibdeps"
+    mock_core.relative_build_path.return_value = tmp_path / "dependencies.lock"
+
+    # Verify pioenvs exists before
+    assert pioenvs_dir.exists()
+
+    # Mock PlatformIO's ProjectConfig cache_dir to return whitespace
+    with patch(
+        "platformio.project.config.ProjectConfig.get_instance"
+    ) as mock_get_instance:
+        mock_config = MagicMock()
+        mock_get_instance.return_value = mock_config
+        mock_config.get.side_effect = (
+            lambda section, option: "   "  # Whitespace only
+            if (section, option) == ("platformio", "cache_dir")
+            else ""
+        )
+
+        # Call the function
+        with caplog.at_level("INFO"):
+            clean_build()
+
+    # Verify pioenvs was removed
+    assert not pioenvs_dir.exists()
+
+    # Verify no cache cleaning was attempted due to empty string
+    assert "PlatformIO cache" not in caplog.text
+
+
+@patch("esphome.writer.CORE")
+def test_write_gitignore_creates_new_file(
+    mock_core: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test write_gitignore creates a new .gitignore file when it doesn't exist."""
+    gitignore_path = tmp_path / ".gitignore"
+
+    # Setup mocks
+    mock_core.relative_config_path.return_value = gitignore_path
+
+    # Verify file doesn't exist
+    assert not gitignore_path.exists()
+
+    # Call the function
+    write_gitignore()
+
+    # Verify file was created with correct content
+    assert gitignore_path.exists()
+    assert gitignore_path.read_text() == GITIGNORE_CONTENT
+
+
+@patch("esphome.writer.CORE")
+def test_write_gitignore_skips_existing_file(
+    mock_core: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test write_gitignore doesn't overwrite existing .gitignore file."""
+    gitignore_path = tmp_path / ".gitignore"
+    existing_content = "# Custom gitignore\n/custom_dir/\n"
+    gitignore_path.write_text(existing_content)
+
+    # Setup mocks
+    mock_core.relative_config_path.return_value = gitignore_path
+
+    # Verify file exists with custom content
+    assert gitignore_path.exists()
+    assert gitignore_path.read_text() == existing_content
+
+    # Call the function
+    write_gitignore()
+
+    # Verify file was not modified
+    assert gitignore_path.exists()
+    assert gitignore_path.read_text() == existing_content
+
+
+@patch("esphome.writer.write_file_if_changed")  # Mock to capture output
+@patch("esphome.writer.copy_src_tree")  # Keep this mock as it's complex
+@patch("esphome.writer.CORE")
+def test_write_cpp_with_existing_file(
+    mock_core: MagicMock,
+    mock_copy_src_tree: MagicMock,
+    mock_write_file: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test write_cpp when main.cpp already exists."""
+    # Create a real file with markers
+    main_cpp = tmp_path / "main.cpp"
+    existing_content = f"""#include "esphome.h"
+{CPP_INCLUDE_BEGIN}
+// Old includes
+{CPP_INCLUDE_END}
+void setup() {{
+{CPP_AUTO_GENERATE_BEGIN}
+// Old code
+{CPP_AUTO_GENERATE_END}
+}}
+void loop() {{}}"""
+    main_cpp.write_text(existing_content)
+
+    # Setup mocks
+    mock_core.relative_src_path.return_value = main_cpp
+    mock_core.cpp_global_section = "// Global section"
+
+    # Call the function
+    test_code = "  // New generated code"
+    write_cpp(test_code)
+
+    # Verify copy_src_tree was called
+    mock_copy_src_tree.assert_called_once()
+
+    # Get the content that would be written
+    mock_write_file.assert_called_once()
+    written_path, written_content = mock_write_file.call_args[0]
+
+    # Check that markers are preserved and content is updated
+    assert CPP_INCLUDE_BEGIN in written_content
+    assert CPP_INCLUDE_END in written_content
+    assert CPP_AUTO_GENERATE_BEGIN in written_content
+    assert CPP_AUTO_GENERATE_END in written_content
+    assert test_code in written_content
+    assert "// Global section" in written_content
+
+
+@patch("esphome.writer.write_file_if_changed")  # Mock to capture output
+@patch("esphome.writer.copy_src_tree")  # Keep this mock as it's complex
+@patch("esphome.writer.CORE")
+def test_write_cpp_creates_new_file(
+    mock_core: MagicMock,
+    mock_copy_src_tree: MagicMock,
+    mock_write_file: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test write_cpp when main.cpp doesn't exist."""
+    # Setup path for new file
+    main_cpp = tmp_path / "main.cpp"
+
+    # Setup mocks
+    mock_core.relative_src_path.return_value = main_cpp
+    mock_core.cpp_global_section = "// Global section"
+
+    # Verify file doesn't exist
+    assert not main_cpp.exists()
+
+    # Call the function
+    test_code = "  // Generated code"
+    write_cpp(test_code)
+
+    # Verify copy_src_tree was called
+    mock_copy_src_tree.assert_called_once()
+
+    # Get the content that would be written
+    mock_write_file.assert_called_once()
+    written_path, written_content = mock_write_file.call_args[0]
+    assert written_path == main_cpp
+
+    # Check that all necessary parts are in the new file
+    assert '#include "esphome.h"' in written_content
+    assert CPP_INCLUDE_BEGIN in written_content
+    assert CPP_INCLUDE_END in written_content
+    assert CPP_AUTO_GENERATE_BEGIN in written_content
+    assert CPP_AUTO_GENERATE_END in written_content
+    assert test_code in written_content
+    assert "void setup()" in written_content
+    assert "void loop()" in written_content
+    assert "App.setup();" in written_content
+    assert "App.loop();" in written_content
+
+
+@pytest.mark.usefixtures("mock_copy_src_tree")
+@patch("esphome.writer.CORE")
+def test_write_cpp_with_missing_end_marker(
+    mock_core: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test write_cpp raises error when end marker is missing."""
+    # Create a file with begin marker but no end marker
+    main_cpp = tmp_path / "main.cpp"
+    existing_content = f"""#include "esphome.h"
+{CPP_AUTO_GENERATE_BEGIN}
+// Code without end marker"""
+    main_cpp.write_text(existing_content)
+
+    # Setup mocks
+    mock_core.relative_src_path.return_value = main_cpp
+
+    # Call should raise an error
+    with pytest.raises(EsphomeError, match="Could not find auto generated code end"):
+        write_cpp("// New code")
+
+
+@pytest.mark.usefixtures("mock_copy_src_tree")
+@patch("esphome.writer.CORE")
+def test_write_cpp_with_duplicate_markers(
+    mock_core: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test write_cpp raises error when duplicate markers exist."""
+    # Create a file with duplicate begin markers
+    main_cpp = tmp_path / "main.cpp"
+    existing_content = f"""#include "esphome.h"
+{CPP_AUTO_GENERATE_BEGIN}
+// First section
+{CPP_AUTO_GENERATE_END}
+{CPP_AUTO_GENERATE_BEGIN}
+// Duplicate section
+{CPP_AUTO_GENERATE_END}"""
+    main_cpp.write_text(existing_content)
+
+    # Setup mocks
+    mock_core.relative_src_path.return_value = main_cpp
+
+    # Call should raise an error
+    with pytest.raises(EsphomeError, match="Found multiple auto generate code begins"):
+        write_cpp("// New code")
+
+
+@patch("esphome.writer.CORE")
+def test_clean_all(
+    mock_core: MagicMock,
+    tmp_path: Path,
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test clean_all removes build and PlatformIO dirs."""
+    # Create build directories for multiple configurations
+    config1_dir = tmp_path / "config1"
+    config2_dir = tmp_path / "config2"
+    config1_dir.mkdir()
+    config2_dir.mkdir()
+
+    build_dir1 = config1_dir / ".esphome"
+    build_dir2 = config2_dir / ".esphome"
+    build_dir1.mkdir()
+    build_dir2.mkdir()
+    (build_dir1 / "dummy.txt").write_text("x")
+    (build_dir2 / "dummy.txt").write_text("x")
+
+    # Create PlatformIO directories
+    pio_cache = tmp_path / "pio_cache"
+    pio_packages = tmp_path / "pio_packages"
+    pio_platforms = tmp_path / "pio_platforms"
+    pio_core = tmp_path / "pio_core"
+    for d in (pio_cache, pio_packages, pio_platforms, pio_core):
+        d.mkdir()
+        (d / "keep").write_text("x")
+
+    # Mock ProjectConfig
+    with patch(
+        "platformio.project.config.ProjectConfig.get_instance"
+    ) as mock_get_instance:
+        mock_config = MagicMock()
+        mock_get_instance.return_value = mock_config
+
+        def cfg_get(section: str, option: str) -> str:
+            mapping = {
+                ("platformio", "cache_dir"): str(pio_cache),
+                ("platformio", "packages_dir"): str(pio_packages),
+                ("platformio", "platforms_dir"): str(pio_platforms),
+                ("platformio", "core_dir"): str(pio_core),
+            }
+            return mapping.get((section, option), "")
+
+        mock_config.get.side_effect = cfg_get
+
+        # Call
+        from esphome.writer import clean_all
+
+        with caplog.at_level("INFO"):
+            clean_all([str(config1_dir), str(config2_dir)])
+
+    # Verify deletions - .esphome directories remain but contents are cleaned
+    # The .esphome directory itself is not removed because it may contain storage
+    assert build_dir1.exists()
+    assert build_dir2.exists()
+
+    # Verify that files in .esphome were removed
+    assert not (build_dir1 / "dummy.txt").exists()
+    assert not (build_dir2 / "dummy.txt").exists()
+    assert not pio_cache.exists()
+    assert not pio_packages.exists()
+    assert not pio_platforms.exists()
+    assert not pio_core.exists()
+
+    # Verify logging mentions each
+    assert "Cleaning" in caplog.text
+    assert str(build_dir1) in caplog.text
+    assert str(build_dir2) in caplog.text
+    assert "PlatformIO cache" in caplog.text
+    assert "PlatformIO packages" in caplog.text
+    assert "PlatformIO platforms" in caplog.text
+    assert "PlatformIO core" in caplog.text
+
+
+@patch("esphome.writer.CORE")
+def test_clean_all_preserves_storage(
+    mock_core: MagicMock,
+    tmp_path: Path,
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test clean_all preserves storage directory."""
+    # Create build directory with storage subdirectory
+    config_dir = tmp_path / "config"
+    config_dir.mkdir()
+
+    build_dir = config_dir / ".esphome"
+    build_dir.mkdir()
+    (build_dir / "dummy.txt").write_text("x")
+    (build_dir / "other_file.txt").write_text("y")
+
+    # Create storage directory with content
+    storage_dir = build_dir / "storage"
+    storage_dir.mkdir()
+    (storage_dir / "storage.json").write_text('{"test": "data"}')
+    (storage_dir / "other_storage.txt").write_text("storage content")
+
+    # Call clean_all
+    from esphome.writer import clean_all
+
+    with caplog.at_level("INFO"):
+        clean_all([str(config_dir)])
+
+    # Verify .esphome directory still exists
+    assert build_dir.exists()
+
+    # Verify storage directory still exists with its contents
+    assert storage_dir.exists()
+    assert (storage_dir / "storage.json").exists()
+    assert (storage_dir / "other_storage.txt").exists()
+
+    # Verify storage contents are intact
+    assert (storage_dir / "storage.json").read_text() == '{"test": "data"}'
+    assert (storage_dir / "other_storage.txt").read_text() == "storage content"
+
+    # Verify other files were removed
+    assert not (build_dir / "dummy.txt").exists()
+    assert not (build_dir / "other_file.txt").exists()
+
+    # Verify logging mentions deletion
+    assert "Cleaning" in caplog.text
+    assert str(build_dir) in caplog.text
+
+
+@patch("esphome.writer.CORE")
+def test_clean_all_platformio_not_available(
+    mock_core: MagicMock,
+    tmp_path: Path,
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test clean_all when PlatformIO is not available."""
+    # Build dirs
+    config_dir = tmp_path / "config"
+    config_dir.mkdir()
+    build_dir = config_dir / ".esphome"
+    build_dir.mkdir()
+
+    # PlatformIO dirs that should remain untouched
+    pio_cache = tmp_path / "pio_cache"
+    pio_cache.mkdir()
+
+    from esphome.writer import clean_all
+
+    with (
+        patch.dict("sys.modules", {"platformio.project.config": None}),
+        caplog.at_level("INFO"),
+    ):
+        clean_all([str(config_dir)])
+
+        # Build dir contents cleaned, PlatformIO dirs remain
+        assert build_dir.exists()
+    assert pio_cache.exists()
+
+    # No PlatformIO-specific logs
+    assert "PlatformIO" not in caplog.text
+
+
+@patch("esphome.writer.CORE")
+def test_clean_all_partial_exists(
+    mock_core: MagicMock,
+    tmp_path: Path,
+) -> None:
+    """Test clean_all when only some build dirs exist."""
+    config_dir = tmp_path / "config"
+    config_dir.mkdir()
+    build_dir = config_dir / ".esphome"
+    build_dir.mkdir()
+
+    with patch(
+        "platformio.project.config.ProjectConfig.get_instance"
+    ) as mock_get_instance:
+        mock_config = MagicMock()
+        mock_get_instance.return_value = mock_config
+        # Return non-existent dirs
+        mock_config.get.side_effect = lambda *_args, **_kw: str(
+            tmp_path / "does_not_exist"
+        )
+
+        from esphome.writer import clean_all
+
+        clean_all([str(config_dir)])
+
+        assert build_dir.exists()
+
+
+@patch("esphome.writer.CORE")
+def test_clean_all_removes_non_storage_directories(
+    mock_core: MagicMock,
+    tmp_path: Path,
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test clean_all removes directories other than storage."""
+    # Create build directory with various subdirectories
+    config_dir = tmp_path / "config"
+    config_dir.mkdir()
+
+    build_dir = config_dir / ".esphome"
+    build_dir.mkdir()
+
+    # Create files
+    (build_dir / "file1.txt").write_text("content1")
+    (build_dir / "file2.txt").write_text("content2")
+
+    # Create storage directory (should be preserved)
+    storage_dir = build_dir / "storage"
+    storage_dir.mkdir()
+    (storage_dir / "storage.json").write_text('{"test": "data"}')
+
+    # Create other directories (should be removed)
+    cache_dir = build_dir / "cache"
+    cache_dir.mkdir()
+    (cache_dir / "cache_file.txt").write_text("cache content")
+
+    logs_dir = build_dir / "logs"
+    logs_dir.mkdir()
+    (logs_dir / "log1.txt").write_text("log content")
+
+    temp_dir = build_dir / "temp"
+    temp_dir.mkdir()
+    (temp_dir / "temp_file.txt").write_text("temp content")
+
+    # Call clean_all
+    from esphome.writer import clean_all
+
+    with caplog.at_level("INFO"):
+        clean_all([str(config_dir)])
+
+    # Verify .esphome directory still exists
+    assert build_dir.exists()
+
+    # Verify storage directory and its contents are preserved
+    assert storage_dir.exists()
+    assert (storage_dir / "storage.json").exists()
+    assert (storage_dir / "storage.json").read_text() == '{"test": "data"}'
+
+    # Verify files were removed
+    assert not (build_dir / "file1.txt").exists()
+    assert not (build_dir / "file2.txt").exists()
+
+    # Verify non-storage directories were removed
+    assert not cache_dir.exists()
+    assert not logs_dir.exists()
+    assert not temp_dir.exists()
+
+    # Verify logging mentions cleaning
+    assert "Cleaning" in caplog.text
+    assert str(build_dir) in caplog.text
+
+
+@patch("esphome.writer.CORE")
+def test_clean_all_preserves_json_files(
+    mock_core: MagicMock,
+    tmp_path: Path,
+    caplog: pytest.LogCaptureFixture,
+) -> None:
+    """Test clean_all preserves .json files."""
+    # Create build directory with various files
+    config_dir = tmp_path / "config"
+    config_dir.mkdir()
+
+    build_dir = config_dir / ".esphome"
+    build_dir.mkdir()
+
+    # Create .json files (should be preserved)
+    (build_dir / "config.json").write_text('{"config": "data"}')
+    (build_dir / "metadata.json").write_text('{"metadata": "info"}')
+
+    # Create non-.json files (should be removed)
+    (build_dir / "dummy.txt").write_text("x")
+    (build_dir / "other.log").write_text("log content")
+
+    # Call clean_all
+    from esphome.writer import clean_all
+
+    with caplog.at_level("INFO"):
+        clean_all([str(config_dir)])
+
+    # Verify .esphome directory still exists
+    assert build_dir.exists()
+
+    # Verify .json files are preserved
+    assert (build_dir / "config.json").exists()
+    assert (build_dir / "config.json").read_text() == '{"config": "data"}'
+    assert (build_dir / "metadata.json").exists()
+    assert (build_dir / "metadata.json").read_text() == '{"metadata": "info"}'
+
+    # Verify non-.json files were removed
+    assert not (build_dir / "dummy.txt").exists()
+    assert not (build_dir / "other.log").exists()
+
+    # Verify logging mentions cleaning
+    assert "Cleaning" in caplog.text
+    assert str(build_dir) in caplog.text
diff --git a/tests/unit_tests/test_yaml_util.py b/tests/unit_tests/test_yaml_util.py
index f31e9554dc..eac0ceabb8 100644
--- a/tests/unit_tests/test_yaml_util.py
+++ b/tests/unit_tests/test_yaml_util.py
@@ -1,9 +1,26 @@
-from esphome import yaml_util
+from pathlib import Path
+import shutil
+from unittest.mock import patch
+
+import pytest
+
+from esphome import core, yaml_util
 from esphome.components import substitutions
 from esphome.core import EsphomeError
+from esphome.util import OrderedDict
 
 
-def test_include_with_vars(fixture_path):
+@pytest.fixture(autouse=True)
+def clear_secrets_cache() -> None:
+    """Clear the secrets cache before each test."""
+    yaml_util._SECRET_VALUES.clear()
+    yaml_util._SECRET_CACHE.clear()
+    yield
+    yaml_util._SECRET_VALUES.clear()
+    yaml_util._SECRET_CACHE.clear()
+
+
+def test_include_with_vars(fixture_path: Path) -> None:
     yaml_file = fixture_path / "yaml_util" / "includetest.yaml"
 
     actual = yaml_util.load_yaml(yaml_file)
@@ -50,15 +67,214 @@ def test_parsing_with_custom_loader(fixture_path):
     """
     yaml_file = fixture_path / "yaml_util" / "includetest.yaml"
 
-    loader_calls = []
+    loader_calls: list[Path] = []
 
-    def custom_loader(fname):
+    def custom_loader(fname: Path):
         loader_calls.append(fname)
 
-    with open(yaml_file, encoding="utf-8") as f_handle:
+    with yaml_file.open(encoding="utf-8") as f_handle:
         yaml_util.parse_yaml(yaml_file, f_handle, custom_loader)
 
     assert len(loader_calls) == 3
-    assert loader_calls[0].endswith("includes/included.yaml")
-    assert loader_calls[1].endswith("includes/list.yaml")
-    assert loader_calls[2].endswith("includes/scalar.yaml")
+    assert loader_calls[0].parts[-2:] == ("includes", "included.yaml")
+    assert loader_calls[1].parts[-2:] == ("includes", "list.yaml")
+    assert loader_calls[2].parts[-2:] == ("includes", "scalar.yaml")
+
+
+def test_construct_secret_simple(fixture_path: Path) -> None:
+    """Test loading a YAML file with !secret tags."""
+    yaml_file = fixture_path / "yaml_util" / "test_secret.yaml"
+
+    actual = yaml_util.load_yaml(yaml_file)
+
+    # Check that secrets were properly loaded
+    assert actual["wifi"]["password"] == "super_secret_wifi"
+    assert actual["api"]["encryption"]["key"] == "0123456789abcdef"
+    assert actual["sensor"][0]["id"] == "my_secret_value"
+
+
+def test_construct_secret_missing(fixture_path: Path, tmp_path: Path) -> None:
+    """Test that missing secrets raise proper errors."""
+    # Create a YAML file with a secret that doesn't exist
+    test_yaml = tmp_path / "test.yaml"
+    test_yaml.write_text("""
+esphome:
+  name: test
+
+wifi:
+  password: !secret nonexistent_secret
+""")
+
+    # Create an empty secrets file
+    secrets_yaml = tmp_path / "secrets.yaml"
+    secrets_yaml.write_text("some_other_secret: value")
+
+    with pytest.raises(EsphomeError, match="Secret 'nonexistent_secret' not defined"):
+        yaml_util.load_yaml(test_yaml)
+
+
+def test_construct_secret_no_secrets_file(tmp_path: Path) -> None:
+    """Test that missing secrets.yaml file raises proper error."""
+    # Create a YAML file with a secret but no secrets.yaml
+    test_yaml = tmp_path / "test.yaml"
+    test_yaml.write_text("""
+wifi:
+  password: !secret some_secret
+""")
+
+    # Mock CORE.config_path to avoid NoneType error
+    with (
+        patch.object(core.CORE, "config_path", tmp_path / "main.yaml"),
+        pytest.raises(EsphomeError, match="secrets.yaml"),
+    ):
+        yaml_util.load_yaml(test_yaml)
+
+
+def test_construct_secret_fallback_to_main_config_dir(
+    fixture_path: Path, tmp_path: Path
+) -> None:
+    """Test fallback to main config directory for secrets."""
+    # Create a subdirectory with a YAML file that uses secrets
+    subdir = tmp_path / "subdir"
+    subdir.mkdir()
+
+    test_yaml = subdir / "test.yaml"
+    test_yaml.write_text("""
+wifi:
+  password: !secret test_secret
+""")
+
+    # Create secrets.yaml in the main directory
+    main_secrets = tmp_path / "secrets.yaml"
+    main_secrets.write_text("test_secret: main_secret_value")
+
+    # Mock CORE.config_path to point to main directory
+    with patch.object(core.CORE, "config_path", tmp_path / "main.yaml"):
+        actual = yaml_util.load_yaml(test_yaml)
+        assert actual["wifi"]["password"] == "main_secret_value"
+
+
+def test_construct_include_dir_named(fixture_path: Path, tmp_path: Path) -> None:
+    """Test !include_dir_named directive."""
+    # Copy fixture directory to temporary location
+    src_dir = fixture_path / "yaml_util"
+    dst_dir = tmp_path / "yaml_util"
+    shutil.copytree(src_dir, dst_dir)
+
+    # Create test YAML that uses include_dir_named
+    test_yaml = dst_dir / "test_include_named.yaml"
+    test_yaml.write_text("""
+sensor: !include_dir_named named_dir
+""")
+
+    actual = yaml_util.load_yaml(test_yaml)
+    actual_sensor = actual["sensor"]
+
+    # Check that files were loaded with their names as keys
+    assert isinstance(actual_sensor, OrderedDict)
+    assert "sensor1" in actual_sensor
+    assert "sensor2" in actual_sensor
+    assert "sensor3" in actual_sensor  # Files from subdirs are included with basename
+
+    # Check content of loaded files
+    assert actual_sensor["sensor1"]["platform"] == "template"
+    assert actual_sensor["sensor1"]["name"] == "Sensor 1"
+    assert actual_sensor["sensor2"]["platform"] == "template"
+    assert actual_sensor["sensor2"]["name"] == "Sensor 2"
+
+    # Check that subdirectory files are included with their basename
+    assert actual_sensor["sensor3"]["platform"] == "template"
+    assert actual_sensor["sensor3"]["name"] == "Sensor 3 in subdir"
+
+    # Check that hidden files and non-YAML files are not included
+    assert ".hidden" not in actual_sensor
+    assert "not_yaml" not in actual_sensor
+
+
+def test_construct_include_dir_named_empty_dir(tmp_path: Path) -> None:
+    """Test !include_dir_named with empty directory."""
+    # Create empty directory
+    empty_dir = tmp_path / "empty_dir"
+    empty_dir.mkdir()
+
+    test_yaml = tmp_path / "test.yaml"
+    test_yaml.write_text("""
+sensor: !include_dir_named empty_dir
+""")
+
+    actual = yaml_util.load_yaml(test_yaml)
+
+    # Should return empty OrderedDict
+    assert isinstance(actual["sensor"], OrderedDict)
+    assert len(actual["sensor"]) == 0
+
+
+def test_construct_include_dir_named_with_dots(tmp_path: Path) -> None:
+    """Test that include_dir_named ignores files starting with dots."""
+    # Create directory with various files
+    test_dir = tmp_path / "test_dir"
+    test_dir.mkdir()
+
+    # Create visible file
+    visible_file = test_dir / "visible.yaml"
+    visible_file.write_text("key: visible_value")
+
+    # Create hidden file
+    hidden_file = test_dir / ".hidden.yaml"
+    hidden_file.write_text("key: hidden_value")
+
+    # Create hidden directory with files
+    hidden_dir = test_dir / ".hidden_dir"
+    hidden_dir.mkdir()
+    hidden_subfile = hidden_dir / "subfile.yaml"
+    hidden_subfile.write_text("key: hidden_subfile_value")
+
+    test_yaml = tmp_path / "test.yaml"
+    test_yaml.write_text("""
+test: !include_dir_named test_dir
+""")
+
+    actual = yaml_util.load_yaml(test_yaml)
+
+    # Should only include visible file
+    assert "visible" in actual["test"]
+    assert actual["test"]["visible"]["key"] == "visible_value"
+
+    # Should not include hidden files or directories
+    assert ".hidden" not in actual["test"]
+    assert ".hidden_dir" not in actual["test"]
+
+
+def test_find_files_recursive(fixture_path: Path, tmp_path: Path) -> None:
+    """Test that _find_files works recursively through include_dir_named."""
+    # Copy fixture directory to temporary location
+    src_dir = fixture_path / "yaml_util"
+    dst_dir = tmp_path / "yaml_util"
+    shutil.copytree(src_dir, dst_dir)
+
+    # This indirectly tests _find_files by using include_dir_named
+    test_yaml = dst_dir / "test_include_recursive.yaml"
+    test_yaml.write_text("""
+all_sensors: !include_dir_named named_dir
+""")
+
+    actual = yaml_util.load_yaml(test_yaml)
+
+    # Should find sensor1.yaml, sensor2.yaml, and subdir/sensor3.yaml (all flattened)
+    assert len(actual["all_sensors"]) == 3
+    assert "sensor1" in actual["all_sensors"]
+    assert "sensor2" in actual["all_sensors"]
+    assert "sensor3" in actual["all_sensors"]
+
+
+def test_secret_values_tracking(fixture_path: Path) -> None:
+    """Test that secret values are properly tracked for dumping."""
+    yaml_file = fixture_path / "yaml_util" / "test_secret.yaml"
+
+    yaml_util.load_yaml(yaml_file)
+
+    # Check that secret values are tracked
+    assert "super_secret_wifi" in yaml_util._SECRET_VALUES
+    assert yaml_util._SECRET_VALUES["super_secret_wifi"] == "wifi_password"
+    assert "0123456789abcdef" in yaml_util._SECRET_VALUES
+    assert yaml_util._SECRET_VALUES["0123456789abcdef"] == "api_key"